benchmarkingblog

Benchmarking and Systems Performance

The Benchmark Games: Oracle’s New TPC-C

with 3 comments

How does a place like Manchester, New Hampshire become the epicenter of the world these days? The big P, politics.

I think that many of us are getting tired of the circus. Lately, I don’t really care about what Mitt did to Ron, Rick’s strategy, or all the attacks on Newt. I only truly care about what the candidates stand for and what kind of leaders they would be.

Sometimes the political arena even reminds me of The Hunger Games. A deadly reality show driven by appearances, theatrics, and insane rules. Whether we’re talking about post-apocalyptic Panem or the world today, when we focus more on the game rather than what it means, that’s truly sad.

Oracle just published a new x86 TPC-C OLTP benchmark result, claiming a “world record.” And the mechanics of that claim are just plain sad.

Oracle compares their brand new benchmark result with an IBM Power result from 2007. Yes, 2007. And this very old Power result is still 1.68x the performance per core of the brand new Oracle result. Oracle also compares their new result with an IBM x3850 X5 result that is half the size of the configuration of the Oracle result — if you do the math the IBM result with DB2 is actually 1.25x greater performance per core than the Oracle result. Oracle forgets to mention anything about price performance here — probably because the Oracle result is over 1.6x more expensive than the IBM x86 result. And, if you can believe it, Oracle then proceeds to pick on a poor little HP system.(1)

How sad.

************************************************

(1) Oracle Sun Fire X4800 M2 server (8 chips/80 cores/160 threads) – 4,803,718 tpmC, US$.98/tpmC, available 06/26/12. IBM Power 570 server (8 chips/16 cores/32 threads) -1,616,162 tpmC, US$3.54 /tpmC, available 11/21/2007. IBM x3850 X5 (4 chips/40 cores/80 threads) – 3,014,684 tpmC, US$.59/tpmC, available 09/22/11. HP ProLiant DL580 G7 (4 chips/32 cores/64 threads), 1,807,347 tpmC, US$.49/tpmC, available 10/15/10. Results as of 1/17/12. Source: http://www.tpc.org .
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , , , ,,,,,,,,,,,,,

About these ads

Written by benchmarkingblog

January 17, 2012 at 10:50 pm

Posted in Oracle, TPC-C

Tagged with , ,

3 Responses

Subscribe to comments with RSS.

  1. Well, if nothing else, Oracle is consistent in their approach. A crappy benchmark result can always be given enough spin to make it look great.

  2. Oracle had no choice on its comparisons with the 5 year old Power 570 as there are no more recent results that are unclustered like for example, there are no Power7 results above 2-CPUs on a system that can support 8-CPUs (Power 780), So the bigger question is why hasn’t IBM published any non-clustered Power7 benchmarks above 2-socket? Scaling perhaps? I/O issues? IOPS issues? Where are the IBM 8-socket (x86 or Power7) results to prove your claims on faster per core performance? Its clearly bench marketing to compare per core performance of systems with different # of CPUs & cores. Can DB2 scale linearly to 8-sockets? Theres no proof, no benchmarks to show this.

    Have you looked at the TPC-C Executive Summary disclosures of the x4800 M2 and IBM System x3850 X5? The Oracle Hardware costs are almost half that of the IBM System x3850 X5! Sure Oracle used Flash Arrays which are quite expensive and clearly boosted the $/perf values but IBM could as easily used Flash as well if it would make a difference. Clearly it can’t as DB2 doesn’t take advantage of Flash like Oracle DB does-otherwise IBM would have!

    Phil

    January 19, 2012 at 4:39 am

    • Phil,

      re: “Oracle had no choice” to compare vs. 2007 hardware…as there are no more recent results that are unclustered” What??? You’re kidding, right? You’ve got me laughing.

      Or maybe you just missed the “more recent results that are unclustered”. For your convenience, here they are:

      http://tpc.org/tpcc/results/tpcc_perf_results.asp?resulttype=noncluster

      Or for the database-literate types out there, here’s a flat file:

      http://tpc.org/downloaded_result_files/tpcc_results.xls

      If Oracle’s benchmarketeers had chosen to compare against IBM’s 2008 nonclustered result, they’d have had to explain why, in 2012, Oracle are still not able to achieve even 80% of what IBM did on Power four years ago.

      As far as x86 goes, I think the “bigger question” is why Oracle has never once published a single TPC-E result? In case you werent paying attention Phil, TPC-E is the newer benchmark that reflects more accurately modern e-commerce driven transactional workloads.

      So…everyone is on-board with TPC-E…except Oracle. Hmmm…

      http://tpc.org/tpce/results/tpce_results.asp

      Oracle’s comparisons of it’s newest stuff to old systems running old benchmarks really is sad…especially given the once proud heritage of Sun.

      Sigh…

      Erica

      February 2, 2012 at 11:39 am


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: