benchmarkingblog

Elisabeth Stahl on Benchmarking and IT Optimization

Archive for the ‘SPECjbb’ Category

POWER8 (and No. 23) is King

with one comment

So it was just announced that LeBron James will be back wearing No. 23 for his return to the Cleveland Cavaliers next season. Cleveland, the old “mistake by the lake” and my current hometown, has had a pretty good last few weeks. With two major coups — a US political national convention and (against all odds) LeBron coming home, there couldn’t be a better place to be.

POWER8 systems have had an awesome month as well. As we know, on top of enterprise RAS and security, performance of POWER8 is king. Let’s take a look at the two latest leadership benchmark results that have been published in two key workloads — Java and virtualization.

  • SPECjbb2013 is THE Java server benchmark. It is based on a world-wide supermarket company with an IT infrastructure that handles a mix of point-of-sale requests, online purchases and data-mining operations. Two metrics are important – a pure throughput metric and a metric that measures critical throughput under service-level agreements (SLAs) specifying response times. So very real world. And the IBM Power S824 (POWER8) system achieved over 95% better performance per core than  Oracle SPARC and over 2x better than x86 Cisco UCS. (1)

 

  • The SPECvirt_sc2013 virtualization benchmark reflects the heavy volume and sudden peaks of traffic faced by datacenter servers used for virtualized server consolidation. It provides four workloads based on real-world traffic faced by a web server, a Java application server, an email server and a batch server. Again, here are the workloads we use every day.And the IBM Power S824 (POWER8) system achieved the #1 result per core and is over 2x the performance per core of the HP DL560 x86 system. (2)

 

So we are talking a winning technology with POWER8 — and a winning city (LeBron, please take us there!)

************************************************

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

(1)SPECjbb2013: IBM Power S824 (POWER8, 3.52 GHz, 4 chips/24 cores/192 threads), 167958 SPECjbb2013-MultiJVM max-jOPS, 27041 SPECjbb2013-MultiJVM critical-jOPS vs. Oracle SPARC T5-2 (2 chips/32 cores/256 threads) 114492 SPECjbb2013-MultiJVM max-jOPS, 43963 SPECjbb2013-MultiJVM critical-jOPS vs. Cisco UCS C460 (4 chips/60 cores/120 threads) 201117 SPECjbb2013-MultiJVM max-jOPS, 52784 SPECjbb2013-MultiJVM critical-jOPS . Source: http://www.spec.org
(2)SPECvirt_sc2013: IBM Power S824 (POWER8, 3.52 GHz, 24-core), PowerVM Enterprise Edition 2.2.3, 1370@79 SPECvirt_sc2013@VMs vs. HP DL560 (32-core, Intel Xeon E5-4650), 908@50 SPECvirt_sc2013VMs. Source: http://www.spec.org

All results current as of July 28, 2014.

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

technorati tags: , , ,,,,,,,,,,,

,,,,

Written by benchmarkingblog

July 28, 2014 at 3:41 pm

Posted in POWER8, SPECjbb, SPECvirt

Tagged with , ,

Oracle’s SPARC T5 and M5 Benchmarks: Lather, Rinse, Repeat

with 21 comments

I think I’ve said this before but one of my most absolute favorite movies is Groundhog Day. (Attention: spoiler is coming but since the fricking movie is from 1993 and most of us were old even way back then, I don’t think I will be ruining it for anyone.) Groundhog Day is an American comedy film directed by Harold Ramis and starring Bill Murray and Andie MacDowell (who by the way I’ve been told that I sort of look like which is really cool since she does L’Oréal ads). In the film an arrogant and egocentric TV weatherman, covering the annual Groundhog Day event, finds himself repeating the same day again and again.

The phrase “Groundhog Day” now has entered common lexicon as a reference to an unpleasant situation that continually repeats, or seems to.

And I would say that is exactly what we have with Oracle’s new SPARC T5 and M5 benchmarks.

Just as with every Oracle processor announcement, the benchmark results do the same thing. Many of the claims are Oracle’s own benchmarks that are not published and audited. There are a small number of industry standard benchmarks — and of course these are ones where it is extremely difficult, if not impossible, to compare to other relevant results. For price claims, Oracle, as they’ve done in the past, only factors in the price of the pizza box – make sure you add in the all-important software and storage.

Let’s take a look at the T5 and M5 benchmark results:

  • SAP: The IBM POWER7+ with DB2 10 SAP SD 2-tier result from back in September was 1.3x greater per core than the M5 and 1.9x greater than the T5 result.(1) The IBM average database request time was also much better and the CPU utilization of the IBM system was also more effective.
  • TPC-C: An IBM POWER6 result from 2008, 2 generations ago, is 42% higher per core than the new T5 result on this OLTP benchmark. An IBM POWER7 result from 2010, 1 generation ago, is 2.2x better performance per core than the Oracle result. (2) The price for all Oracle database software support used in computing the price/performance for this benchmark is $2300/year – I can only guess what you get for that. Also note that this benchmark used Oracle Partitioning which may not be realistic for your real world workloads. The Oracle database software is not even available until September.
  • SPECjEnterprise2010: Oracle’s T5 result needed four times the number of database cores, four times the amount of memory and significantly more storage than the IBM POWER7 result. (3)
  • SPECjbb2013: For Java business, let’s run a benchmark that can only be compared with a couple of ProLiants, one of our old T4s, and a Supermicro. (4)
  • SPECcpu: IBM Power Systems is #1 – don’t forget to look at number of cores for integer and floating point claims.
  • TPC-H: Ha, got you. There is no TPC-H. Funny, was expecting one based on what we saw for the T4. I wonder why . . .
  • The other benchmark claims? These are once again ones that either are Oracle’s own benchmarks or ones nobody cares about because they don’t look like anything we actually run. Chance of departure from useful benchmark results: 100%.
  • Don’t let these claims distract from asking about the business value delivered by these systems.

    I wake up every day, right here, right in Cleveland, and it’s always snowing, and there’s nothing I can do about it. “Winter, slumbering in the open air, wears on its smiling face a dream… of spring.”

    ************************************************
    (1)IBM Power 780 (3.72 GHz) two-tier SAP SD Standard Application Benchmark result (SAP enhancement package 5 for the SAP ERP 6.0 application: 12 processors / 96 cores / 384 threads, POWER7+, 1536 GB memory, 57,024 SD benchmark users, running AIX® 7.1 and DB2® 10, dialog resp.: 0.98s, line items/hour: 6,234,330, Dialog steps/hour: 18,703,000, SAPS: 311,720, DB time (dialog/ update): 0.009s / 0.014s, CPU utilization: 99%, Certification #2012033

    Oracle SPARC Server M5-32 SAP SD 2-tier result of 85,050 users, Average dialog response time: 0.80 seconds, Fully processed order line items per hour: 9,452,000,Dialog steps per hour: 28,356,000,SAPS: 472,600,Average database request time (dialog/update): 0.018 sec / 0.044 sec,CPU utilization of central server: 82%,Operating system, central server: Solaris 11,RDBMS: Oracle 11g,SAP Business Suite software: SAP enhancement package 5 for SAP ERP 6.0,32 processors / 192 cores / 1536 threads,SPARC M5, 3.60 GHz, 16 KB (D) and 16 KB (I) L1 cache and128 KB L2 cache per core, 48 MB L3 cache per processor,4096 GB main memory,Certification #2013009

    Oracle SPARC Server T5-8 SAP SD 2-tier result of 40,000 users,Average dialog response time: 0.86 seconds,Fully processed order line items per hour: 4,419,000,Dialog steps per hour: 13,257,000,SAPS: 220,950,Average database request time (dialog/update): 0.049 sec / 0.131 sec,CPU utilization of central server: 88%, Operating system, central server: Solaris 11,RDBMS: Oracle 11g,SAP Business Suite software: SAP enhancement package 5 for SAP ERP 6.0, 8 processors / 128 cores / 1024 threads,SPARC T5, 3.60 GHz, 16 KB (D) and 16 KB (I) L1 cache and 128 KB L2 cache per core, 8 MB L3 cache per processor,2048 GB main memory,Certification #2013008.

    (2) IBM Power 780 (2 chips, 8 cores, 32 threads) with IBM DB2 9.5 (1,200,011 tpmC, $.69/tpmC, configuration available 10/13/10); IBM Power 595 (5 GHz, 32 chips, 64 cores, 128 threads) with IBM DB2 9.5 (6,085,166 tpmC, $2.81/tpmC, configuration available 12/10/08); vs. Oracle SPARC T5-8 (8 chips, 128 cores, 1024 threads – 8,552,523 tpmC, $.55/tpmC, configuration available 9/25/13).

    (3) WebSphere Application Server V7 on IBM Power 780 and DB2 on IBM Power 750 Express, (64 core app server, 32 core db server), 16,646.34 SPECjEnterprise2010 EjOPS vs. SPARC T5-8 server (SPARC T5-8 server base package, 8x SPARC T5 16-core processors, 128x16GB-1066 DIMMS, 2x600GB 10K RPM 2.5” SAS-2 HDD result of SPARC T5-8, 57,422.17 SPECjEnterprise2010 EjOPS.

    (4) http://www.oracle.com/us/solutions/performance-scalability/sparc-t5-2-specjbb2013-1925099.html

    Sources: http://www.spec.org, http://www.tpc.org, http://www.sap.com. Results current as of 3/26/13.

    TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

    SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
    logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
    over the world.

    SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

    The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

    technorati tags: , , ,,,,,,,,,,,,,,,

    Written by benchmarkingblog

    March 26, 2013 at 5:53 pm

    Oracle, I Knew You Were Trouble When You Walked In (to SPECjbb2013)

    with one comment

    I’m always on the lookout for anything that helps me eat well. My newest interest these days is a type of rye crisp cracker that looks like cardboard, tastes a bit better, but has absolutely no fat. (They are really good if you put a huge slice of brie on them.)

    I also love reading about nutrition. I love Bloomberg’s ban on big soda. I love the concept of using color to focus on a well-rounded and healthy diet. Does lime jello count in the green group?

    Pomegranate juice, which has many claims on being good for you, was one of those things. Deep red, and also great for an amazing sauce reduction for chicken. Well, I recently read that the Federal Trade Commission just barred claims of pomegranate juice helping heart disease and other ailments until truly proven — based on deceptive advertising claims.

    Which reminded me of something else. Oracle recently claimed a “world record” in the SPECjbb2013 Java benchmark. What you need to know is that this benchmark does not have 100 other results to compare to. It doesn’t have 20. It doesn’t even have 10. This “world record result” is based on just two other published results. Both by Intel. With older JDK. And fewer and older processors.

    Like when your black lab starts to sit on your friend’s chihuahua. Or a claim that your juice cures cancer.

    ************************************************

    SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

    The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

    technorati tags: , , ,,,,,,,,,,

    ,

    Written by benchmarkingblog

    February 25, 2013 at 10:41 am

    Posted in Oracle, SPECjbb

    Tagged with ,

    %d bloggers like this: