Oracle’s SPARC T5 and M5 Benchmarks: Lather, Rinse, Repeat

I think I’ve said this before but one of my most absolute favorite movies is Groundhog Day. (Attention: spoiler is coming but since the fricking movie is from 1993 and most of us were old even way back then, I don’t think I will be ruining it for anyone.) Groundhog Day is an American comedy film directed by Harold Ramis and starring Bill Murray and Andie MacDowell (who by the way I’ve been told that I sort of look like which is really cool since she does L’Oréal ads). In the film an arrogant and egocentric TV weatherman, covering the annual Groundhog Day event, finds himself repeating the same day again and again.

The phrase “Groundhog Day” now has entered common lexicon as a reference to an unpleasant situation that continually repeats, or seems to.

And I would say that is exactly what we have with Oracle’s new SPARC T5 and M5 benchmarks.

Just as with every Oracle processor announcement, the benchmark results do the same thing. Many of the claims are Oracle’s own benchmarks that are not published and audited. There are a small number of industry standard benchmarks — and of course these are ones where it is extremely difficult, if not impossible, to compare to other relevant results. For price claims, Oracle, as they’ve done in the past, only factors in the price of the pizza box – make sure you add in the all-important software and storage.

Let’s take a look at the T5 and M5 benchmark results:

SAP: The IBM POWER7+ with DB2 10 SAP SD 2-tier result from back in September was 1.3x greater per core than the M5 and 1.9x greater than the T5 result.(1) The IBM average database request time was also much better and the CPU utilization of the IBM system was also more effective.

TPC-C: An IBM POWER6 result from 2008, 2 generations ago, is 42% higher per core than the new T5 result on this OLTP benchmark. An IBM POWER7 result from 2010, 1 generation ago, is 2.2x better performance per core than the Oracle result. (2) The price for all Oracle database software support used in computing the price/performance for this benchmark is $2300/year – I can only guess what you get for that. Also note that this benchmark used Oracle Partitioning which may not be realistic for your real world workloads. The Oracle database software is not even available until September.

SPECjEnterprise2010: Oracle’s T5 result needed four times the number of database cores, four times the amount of memory and significantly more storage than the IBM POWER7 result. (3)

SPECjbb2013: For Java business, let’s run a benchmark that can only be compared with a couple of ProLiants, one of our old T4s, and a Supermicro. (4)

SPECcpu: IBM Power Systems is #1 – don’t forget to look at number of cores for integer and floating point claims.

TPC-H: Ha, got you. There is no TPC-H. Funny, was expecting one based on what we saw for the T4. I wonder why . . .

The other benchmark claims? These are once again ones that either are Oracle’s own benchmarks or ones nobody cares about because they don’t look like anything we actually run. Chance of departure from useful benchmark results: 100%.

Don’t let these claims distract from asking about the business value delivered by these systems.

I wake up every day, right here, right in Cleveland, and it’s always snowing, and there’s nothing I can do about it. “Winter, slumbering in the open air, wears on its smiling face a dream… of spring.”

************************************************
(1)IBM Power 780 (3.72 GHz) two-tier SAP SD Standard Application Benchmark result (SAP enhancement package 5 for the SAP ERP 6.0 application: 12 processors / 96 cores / 384 threads, POWER7+, 1536 GB memory, 57,024 SD benchmark users, running AIX® 7.1 and DB2® 10, dialog resp.: 0.98s, line items/hour: 6,234,330, Dialog steps/hour: 18,703,000, SAPS: 311,720, DB time (dialog/ update): 0.009s / 0.014s, CPU utilization: 99%, Certification #2012033

Oracle SPARC Server M5-32 SAP SD 2-tier result of 85,050 users, Average dialog response time: 0.80 seconds, Fully processed order line items per hour: 9,452,000,Dialog steps per hour: 28,356,000,SAPS: 472,600,Average database request time (dialog/update): 0.018 sec / 0.044 sec,CPU utilization of central server: 82%,Operating system, central server: Solaris 11,RDBMS: Oracle 11g,SAP Business Suite software: SAP enhancement package 5 for SAP ERP 6.0,32 processors / 192 cores / 1536 threads,SPARC M5, 3.60 GHz, 16 KB (D) and 16 KB (I) L1 cache and128 KB L2 cache per core, 48 MB L3 cache per processor,4096 GB main memory,Certification #2013009

Oracle SPARC Server T5-8 SAP SD 2-tier result of 40,000 users,Average dialog response time: 0.86 seconds,Fully processed order line items per hour: 4,419,000,Dialog steps per hour: 13,257,000,SAPS: 220,950,Average database request time (dialog/update): 0.049 sec / 0.131 sec,CPU utilization of central server: 88%, Operating system, central server: Solaris 11,RDBMS: Oracle 11g,SAP Business Suite software: SAP enhancement package 5 for SAP ERP 6.0, 8 processors / 128 cores / 1024 threads,SPARC T5, 3.60 GHz, 16 KB (D) and 16 KB (I) L1 cache and 128 KB L2 cache per core, 8 MB L3 cache per processor,2048 GB main memory,Certification #2013008.

(2) IBM Power 780 (2 chips, 8 cores, 32 threads) with IBM DB2 9.5 (1,200,011 tpmC, $.69/tpmC, configuration available 10/13/10); IBM Power 595 (5 GHz, 32 chips, 64 cores, 128 threads) with IBM DB2 9.5 (6,085,166 tpmC, $2.81/tpmC, configuration available 12/10/08); vs. Oracle SPARC T5-8 (8 chips, 128 cores, 1024 threads – 8,552,523 tpmC, $.55/tpmC, configuration available 9/25/13).

(3) WebSphere Application Server V7 on IBM Power 780 and DB2 on IBM Power 750 Express, (64 core app server, 32 core db server), 16,646.34 SPECjEnterprise2010 EjOPS vs. SPARC T5-8 server (SPARC T5-8 server base package, 8x SPARC T5 16-core processors, 128x16GB-1066 DIMMS, 2x600GB 10K RPM 2.5” SAS-2 HDD result of SPARC T5-8, 57,422.17 SPECjEnterprise2010 EjOPS.

(4) http://www.oracle.com/us/solutions/performance-scalability/sparc-t5-2-specjbb2013-1925099.html

Sources: http://www.spec.org, http://www.tpc.org, http://www.sap.com. Results current as of 3/26/13.

TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
over the world.

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance,TPC-C,SAP,SPECjEnterprise,SPECjbb,Siebel,Oracle,SPARC,M5,T5,POWER6,POWER7,storage,DB2,benchmark,benchmarking

Oracle’s New T5 TPC-C: Where’s the SPARC?, Part II

With Oracle’s new SPARC server announcement today, we are all still waiting in anticipation (take your pick of Rocky Horror or Carole King) for something exciting. The just released TPC-C benchmark result surely is not.

Here are some reasons why:

The performance of the Oracle T5-8 (even with the use of Oracle database partitioning) is downright lackluster. An IBM POWER6 result from 2008, 2 generations ago, is 42% higher per core. An IBM POWER7 result from 2010, 1 generation ago, is 2.2x better performance per core than the Oracle result. (1)

The price for all Oracle software support used in computing the price/performance for this benchmark is $2300/year. I can only guess what you get for that.

The Oracle database software is not even available until September. Yes, September.

It’s keeping me wa a a a aiting . . .

************************************************

(1) IBM Power 780 (2 chips, 8 cores, 32 threads) with IBM DB2 9.5 (1,200,011 tpmC, $.69/tpmC, configuration available 10/13/10); IBM Power 595 (5 GHz, 32 chips, 64 cores, 128 threads) with IBM DB2 9.5 (6,085,166 tpmC, $2.81/tpmC, configuration available 12/10/08); vs. Oracle SPARC T5-8 (8 chips, 128 cores, 1024 threads – 8,552,523 tpmC, $.55/tpmC, configuration available 9/25/13).
Source: http://www.tpc.org. Results current as of 3/26/13.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance,TPC-C,Oracle,SPARC,M5,T5,POWER6,POWER7,storage,DB2,benchmark,benchmarking

Lackluster Larry

As we await that big wave called OpenWorld to reach our shores, let’s take a look at what we have actually seen from Oracle in the last few days.

• Questions on Oracle’s latest earnings and disappointing performance in hardware
• A new ad on the front page of the Wall Street Journal that attempts to make a groundbreaking statement about the very old news that there are public clouds and there are private clouds.
• A press release on a new TPC-C OLTP benchmark result.The problem is that it’s actually a Cisco benchmark result that just happens to use some Oracle software. And the problem is that the Cisco system in the comparison is 2 years newer, needs 2x the number of cores, and uses over 60% more storage than the IBM result cited. Even so, the older IBM system is 1.49x times the performance per core of the Cisco system. (1)

Based on this news plus what I’ve heard may be coming that huge OpenWorld wave may turn out to be merely a minor ripple in our IT landscape.

************************************************

(1)Cisco UCS C240 M3, 1,609,186.39 tpmC, $0.47/tpmC, (2 processors/16 cores/32 threads) available 9/27/12 vs. IBM Power 780 Server Model 9179-MHB with IBM DB2 9.5, 1,200,011.00 tpmC, $0.69/tpmC, available 10/13/10 (2 processors/8 cores/32 threads). Results current as of 9/28/12. Source: http://www.tpc.org.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).
The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance,TPC,Cisco,Oracle,UCS,C240,benchmark,11g,Linux,TPC-C,availability,date,benchmarking

Oracle, The Dates They are A-Changin´

One of the great things about using industry standard benchmarks for performance evaluation is that you can trust the data. It is run according to benchmark kits developed by a committee. It is audited. It produces data and the results don’t change.

Except when they do.

One of the very useful pieces of data that is part of a TPC benchmark result, in addition to performance and price performance, is the availability date. This is the date for which all components, hardware and software are available for purchase. There are TPC rules around this: “The Committed delivery date for general availability (availability date) of products used in the price calculations must be reported. The Availability Date must be reported on the first page of the Executive Summary and with a precision of one day. When the priced system includes products with different availability dates, the reported availability date for the priced system must be the date at which all Components are committed to be Generally Available. Each Component used in the Priced Configuration is considered to be Available on the Availability Date unless an earlier date is specified.”

As a consumer of industry standard benchmarks, we trust that the vendor has made extensive and detailed plans to meet this date and that we can use this date in our own systems planning.

It is not very common to change this date after the result is published. It is not very common to change this date on the exact date that the system is supposed to become available. Oracle just did both — yesterday they delayed the system availability date on a TPC-C Sun Fire X4800 result.(1) If you take a look at the Full Disclosure Report you can see that the components that aren’t ready are no small extraneous part of the configuration — they’re huge DIMMs, memory modules.

I’m going to take a wild guess that these components that are not yet available are very much needed to produce this benchmark result.

************************************************

(1)Oracle Sun Fire X4800 M2 server (8 chips/80 cores/160 threads) – 5,055,888 tpmC, US$.89/tpmC, available 7/10/12.
Source: http://www.tpc.org. Result current as of 6/27/12.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).
The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance,TPC,Sun,Oracle,X4800,M2,benchmark,11g,Linux,TPC-C,availability,date,benchmarking

Expert Integrated Systems: A Revolutionary Repast

This past weekend I needed to make dinner for 15 people. Twice.

I organized my shopping into two trips, not ten. I cooked one side dish but bought another. One night I grilled, one I roasted. I set the table ahead of time. I admit it — I even bought bagged salad.

The point here is simplification. Do what we need to with limited resources. Save time, reduce costs, get the performance we require. And that’s one of the main points of Expert Integrated Systems.

With new IBM PureSystems, you get the best of all worlds. A combination of the flexibility of a general purpose system with the simplicity of an appliance. It’s revolutionary and evolutionary. And all of this with outstanding performance, of course:

• View the excellent results on the SAP SD (Sales and Distribution) benchmark on the new IBM Flex System p460 and x240 with DB2. (1)
• Analyze the #1 x86 16-core TPC-C result on the IBM Flex System x240. (2)
• Read the press release, watch the video.
• See more great benchmark results.
• Peruse the rPerf data for the IBM Flex System p260 and p460.
• Follow IBM PureSystems.

Performance plus powerful systems management capabilities. Game-changing technology. Like my automatic dishwasher, which was also quite popular this weekend, and luckily performed quite well.

************************************************

(1) IBM Flex System p460 – Source: sap.com/benchmark. Configuration and results of the IBM Flex System p460 on the two-tier SAP SD standard application benchmark running SAP enhancement package 4 for SAP ERP 6.0 (Unicode): 4 processors /32 cores / 128 threads, POWER7, 3.55 GHz, 256 GB memory, 17,000 SAP SD benchmark users, dialog response: 0.96 seconds, line items/hour: 1,861,670, dialog steps/hour: 5,585,000, SAPS: 93,080, database response time (dialog/update): 0.011 sec / 0.021 sec, CPU utilization: 99%, OS: AIX 7.1, DB2 9.7; Certification #2012015.
IBM Flex System x240 – Source: sap.com/benchmark. Configuration and results of the IBM Flex System x240 on the two-tier SAP SD standard application benchmark running SAP enhancement package 4 for SAP ERP 6.0 (Unicode): 2 processors /16 cores / 32 threads, Intel Xeon Processor E5-2690, 2.90 GHz, 64 KB L1 cache and
256 KB L2 cache per core, 20 MB L3 cache per processor,
128 GB main memory, 7,960 SAP SD benchmark users, dialog response: 0.98 seconds, line items/hour: 870,330, dialog steps/hour: 2,611,000, SAPS: 43,520, database response time (dialog/update): 0.012 sec / 0.010 sec, CPU utilization: 99%, OS: Windows Server 2008 R2 Enterprise Edition, DB2 9.7; Certification #2012016.
Results valid as of 4/11/2012.

(2)TPC-C result on IBM Flex System x240, Performance (tpmC): 1,503,544, Price/Performance: .53 USD, Pro! cessors / MHz of Server: Intel Xeon Processor E5-2690 2.90GHz / 2.90GHz
Total # of Processors / Cores / Threads: 2 / 16 / 32, Availability Date: 8/16/2012. Result current as of 4/11/12. Source: http://www.tpc.org

SAP and all SAP logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries.

TPC-H, TPC-C, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance, SPEC, PureAnalytics,PureDatabase,Flex,NGP,POWER7,expert,

benchmark,Intel,integrated,SAP,rPerf,TPC-C,SD,PureSystems,Intel,PureFlex,

p460,p260,DB2,appliance,Troy,Power,PureApplication,benchmarking

Is Oracle Fleecing You?

A couple of days ago, I bought a new fleece hiking jacket. In small, in black, on sale. With a hood. Made by a high end manufacturer whose name evokes a part of a mountain. The plush is luxuriously thick, the zipper hardware is incredible, and the tailoring just feels so right. I had tried on other jackets that just did not stack up. There was no comparison, apples and oranges, from different planets. And that reminded me once again of how some performance comparisons are made.

This week, Oracle claimed x86 “world-record” performance with the Sun Fire X4800 M2 on industry standard Java middleware and transactional database benchmarks. They compared their results to results from IBM. Here’s what you need to know:

For the Java SPECjEnterprise2010 benchmark, Oracle needed more cores in both the application server and the database server. Oracle used over 4x the storage disks and over 7x the cache that was used in the IBM result. Oracle conveniently cites price/performance (which isn’t even a metric in this benchmark) for the application tier only. The picture would probably look very different if they included the important database tier (with all those costly Oracle licenses). Oracle compares their brand new result with an IBM result from over a year ago.(1) Maybe it’s time for a new . . .

For the transactional TPC-C benchmark, when you analyze the comparison correctly, the IBM result is actually 19% better performance per core than the Oracle result. The IBM configuration has been available for months, the Oracle configuration is not even available. And the Oracle result is 1.5x more expensive.(2)

When I got to the checkout line with my perfect fleece jacket, they told me that the store was having a special one day sale, another 20% off. I got myself superior performance and price/performance — and you can’t get much better than that.

************************************************

(1) WebSphere Application Server V7 on IBM Power 780 and DB2 on IBM Power 750 Express, (64 core app server, 32 core db server), 16,646.34 SPECjEnterprise2010 EjOPS vs. Oracle WebLogic Server 12c and Oracle Database 11g Release 2 with Oracle Linux running on a Sun Fire X4800 M2 server(5U) with eight Intel Xeon E7-8870 2.4 GHz processors, (80 cores, 8 chips, 10 cores/chip, 2 threads/core) 27,150.05 SPECjEnterprise2010 EjOPS.
(2) IBM System x3850 X5 (4 chips/40 cores/80 threads) – 3,014,684 tpmC, US$.59/tpmC, available 09/22/11 vs. Oracle Sun Fire X4800 M2 server (8 chips/80 cores/160 threads) – 5,055,888 tpmC, US$.89/tpmC, available 06/26/12.
Sources: http://www.tpc.org, http://www.spec.org. Results current as of 3/30/12.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).
SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).
The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance, SPEC, SPECjEnterprise,TPC-C,780,750,POWER7,WebSphere,benchmark,Xeon,11g,WebLogic,M4,E7-8870,

Intel,x3850,Sun,Fire,Java,X4800,Power,Oracle,benchmarking

The Benchmark Games: Oracle’s New TPC-C

How does a place like Manchester, New Hampshire become the epicenter of the world these days? The big P, politics.

I think that many of us are getting tired of the circus. Lately, I don’t really care about what Mitt did to Ron, Rick’s strategy, or all the attacks on Newt. I only truly care about what the candidates stand for and what kind of leaders they would be.

Sometimes the political arena even reminds me of The Hunger Games. A deadly reality show driven by appearances, theatrics, and insane rules. Whether we’re talking about post-apocalyptic Panem or the world today, when we focus more on the game rather than what it means, that’s truly sad.

Oracle just published a new x86 TPC-C OLTP benchmark result, claiming a “world record.” And the mechanics of that claim are just plain sad.

Oracle compares their brand new benchmark result with an IBM Power result from 2007. Yes, 2007. And this very old Power result is still 1.68x the performance per core of the brand new Oracle result. Oracle also compares their new result with an IBM x3850 X5 result that is half the size of the configuration of the Oracle result — if you do the math the IBM result with DB2 is actually 1.25x greater performance per core than the Oracle result. Oracle forgets to mention anything about price performance here — probably because the Oracle result is over 1.6x more expensive than the IBM x86 result. And, if you can believe it, Oracle then proceeds to pick on a poor little HP system.(1)

How sad.

************************************************

(1) Oracle Sun Fire X4800 M2 server (8 chips/80 cores/160 threads) – 4,803,718 tpmC, US$.98/tpmC, available 06/26/12. IBM Power 570 server (8 chips/16 cores/32 threads) -1,616,162 tpmC, US$3.54 /tpmC, available 11/21/2007. IBM x3850 X5 (4 chips/40 cores/80 threads) – 3,014,684 tpmC, US$.59/tpmC, available 09/22/11. HP ProLiant DL580 G7 (4 chips/32 cores/64 threads), 1,807,347 tpmC, US$.49/tpmC, available 10/15/10. Results as of 1/17/12. Source: http://www.tpc.org .
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance, Sun, X4800,TPC-C,Oracle,ProLiant,X5,M2,benchmark,x86,x3850,HP,DL580,E7-8870,G7,benchmarking

Cisco/Oracle Super Saturday TPC-C

So now they’re trying to put another one over on us. “Super Saturday” for retailers, where you shop till you drop the Saturday before Christmas, has been officially moved to tomorrow, one week early. To give us more sales and more time to shop until we drop. I’m just getting tired of being told that I should shop on Black Friday, Cyber Monday, Super Saturday, or any other time. It’s sort of like the greeting card companies saying let’s make a new holiday to celebrate your dog.

And that’s how I’m feeling lately being told about the latest Oracle and Cisco benchmark claims on the newest TPC-C OLTP benchmark result. Here are the 3 things you need to know:

• The Benchmark: Oracle didn’t even run this benchmark. And they didn’t even run it on their own hardware.
• The Performance: Note the special pricey Violin memory arrays that Cisco/Oracle used. And even an IBM result from over a year and a half ago on this benchmark is 70% better performance per core than this result.(1)
• The Pricing: Cisco and Oracle both take advantage of Super Saturday pricing methods in these results. Cisco shows a 57% “large purchase discount” for hardware. Pricing for Oracle 11g is for the limited standard edition. And very minimal support is included.

I know that both American Greetings and my black lab would really like that idea for the new holiday to celebrate your dog. I already bought my lab a large braided rawhide with red and green bows. So you know they have me.

Happy Holidays!

************************************************

(1) An 8-core IBM Power 780 (2 chips, 32 threads) with IBM DB2 9.5 is the best 8-core system (1,200,011 tpmC, $.69/tpmC, configuration available 10/13/10) vs. Oracle Database 11g Release 2 Standard Edition One and Oracle Linux on Cisco UCS c250 M2 Extended-Memory Server, 1,053,100 tpmC, $0.58/tpmC, available 12/7/2011.
Source: http://www.tpc.org. Results current as of 12/16/11.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance, Sun, Cisco,TPC-C,Oracle,11g,UCS,POWER7,benchmark,C250,standard,Linux,extended,memory,OLTP,benchmarking