benchmarkingblog

Benchmarking and Systems Performance

The Top Nine Performance Fallacies in Oracle Press Releases

with 2 comments

Another week, another Oracle press release on performance. Whether you call it Groundhog Day, deja vu, or same old same old, I guess I’m just getting tired of writing about these. So I decided to do something about it. Here are the top nine things I’ve encountered that you should ask yourself:

  • Is Oracle comparing themselves with themselves?
  • Is Oracle comparing themselves with another vendor’s result — from 2 years ago?
  • Is Oracle using an industry standard benchmark or just their own internal performance test?
  • If an industry standard benchmark is used, is it one that anyone cares about?
  • Is Oracle using an “independent” benchmark that they OWN?
  • Are the products used in the performance test even available?
  • Do the comparisons take into account different hardware and software configurations?
  • Does the performance claim have a footnote with all the details?
  • Do the metrics cited make any sense?
  • I totally believe in reuse. Please refer to this blog entry whenever an Oracle press release on performance is encountered.

    Oh, and everyone always says it’s got to be a Top Ten list. There has to be a tenth fallacy. Let us know yours here.

    ************************************************

    The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

    technorati tags: , , , , ,,,,,,,,,

    About these ads

    Written by benchmarkingblog

    April 21, 2011 at 2:19 pm

    Posted in Oracle

    Tagged with ,

    2 Responses

    Subscribe to comments with RSS.

    1. Interesting

      Clarisse Taaffe-Hedglin

      April 21, 2011 at 4:34 pm

    2. In their latest TPC-C benchmark result, Oracle satisfied the letter of the law but not the spirit of the law regarding their software license and maintenance costs (which are included in the $/TPC-C calculations). I believe they chose to include unusual low-cost software licenses and a bare minimum support and maintenance plan that users would be very unlikely to choose for any database application. Oralce had to do that to achieve a good price/performance results given the very large number of SPARC servers and cores required to get their leading result.

      While that’s not against TPC rules, it’s clearly misleading.

      Joe Barkan

      April 27, 2011 at 9:20 am


    Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s

    %d bloggers like this: