benchmarkingblog

Elisabeth Stahl on Benchmarking and IT Optimization

Archive for the ‘Oracle’ Category

Guns and Butter at OpenWorld

with one comment

I guess when you are really really rich you can do things like miss your own keynote to go to a sporting event. Or get prices wrong by millions of dollars.

Yes, I took Econ 1A in college (though I may remember more about the cute boy in the row in front of me than supply and demand). I clearly remember grasping the intricate graphs and complex formulas in the thick colorful book by Samuelson.

But that preparation did not seem to help this week in trying to understand the new Oracle “Economics” at OpenWorld. A quick search did not lead to any scholarly articles on “near linear pricing.” If there is any sort of “re-engineering” of economics going on, it has not been picked up by the MBA programs just yet.

So when you see any pricing comparisons from Oracle these days, here is what you need to know:

  • Sometimes the systems compared have different numbers of processor cores. Sometimes the systems are the same “size” but size does not equal the performance of what can be run on the system.
  • Sometimes the systems compared have different amounts of memory. Sometimes the systems have the same amount of memory but amount of memory does not equal the performance of what can be run on the system.
  • Sometimes Oracle includes no software on their system and includes software on the other vendor’s system.
  • Sometimes Oracle does not include the expensive Oracle database license costs, which by the way are calculated by core.
  • Sometimes the systems compared have very very different types of support and maintenance.
  • Sometimes the systems compared have very different types and amounts of storage included. Or no storage at all. As we know, storage can be a large part of a system’s configuration and price.

There has been absolutely NO substantiation to justify equivalent price configurations for equivalent throughput systems in these comparisons.

What is ultimately important is what non-functional requirements the system gives you at a certain price. Compare, and do the TCO. And tell Oracle: I don’t buy sockets, I buy performance.

************************************************

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,

,,,,,,,

Written by benchmarkingblog

September 25, 2013 at 10:36 am

Posted in Oracle

Tagged with ,

The Wizard of OpenWorld

with one comment

Sometimes it’s great to see something for the hundredth time.

On Saturday night I went to see one of the all time greats, The Wizard of Oz — in 3D. The huge IMAX screen and 3D effects pulled you into the movie. I was dancing with the Munchkins and really skipping down that yellow brick road.

And sometimes you just want to cackle and destroy like the Wicked Witch of the West because you are being forced to see something for the hundredth time.

At Oracle OpenWorld’s keynote last night, the industry benchmarks that were highlighted made me want to do just that.

  • Oracle with Fujitsu claimed “14 World #1’s.” Then of course, doing what they do time and again, they only actually discussed a few of them.
  • In the SAP SD 2-tier comparison, Fujitsu/Oracle’s result was from 2013. IBM’s from 2010. Fujitsu/Oracle’s result used 640 cores, IBM only 256. IBM’s result was actually over 2x the users per core of the Oracle/Fujitsu result. We have surely seen this before, ain’t it the truth?(1)
  • The SPECjbb2013 comparison highlighted the M10 against some undesignated x86 system. Like the cowardly lion picking on little Toto.
  • The third benchmark was Stream, relevant for the very few in the commercial world.
  • Larry compared the M6-32 “Big Memory Machine” against a Power System. With absolutely no details and data to back the claim. We’ve seen this over and over as well.
  • Make no doubt about it. Absolutely none of these performance benchmarks have any pricing component whatsoever as a metric. And any pricing that is shown should be analyzed – what storage is included, what maintenance and support costs, is software added in? We’ve seen creative accounting here so many times before.

What was so special about seeing The Wizard of Oz on the big screen in 3D was that you noticed all of these incredible details (like the colorful birds, the beautiful expanse of red poppies, and the stage hand behind the apple trees) that you had never seen before. What was so NOT special about the OpenWorld keynote was that you were seeing the same old story — but with almost no details behind it. Once again.

************************************************

(1) IBM Power 795 (4.00 GHz) two-tier SAP SD Standard Application Benchmark result (SAP enhancement package 4 for SAP ERP 6.0 (Unicode): 32 processors / 256 cores / 1024 threads, POWER7, 4096 GB memory, 126,063 SAP SD benchmark users, OS: AIX 7.1, DB2 9.7. Certification #: 2010046 vs. Fujitsu M10-48 (40 processors / 640 cores / 1280 threads,153,000 SAP SD benchmark users, Oracle. Certification #: 2013014. Source: http://www.sap.com/benchmark. Results as of 9/23/13.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
over the world.

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,,,,,

,,

,,,

Written by benchmarkingblog

September 23, 2013 at 8:59 am

Posted in Oracle

Tagged with ,

Taking the Wind Out of Oracle’s Sails

with one comment

I don’t always read the sports pages. But lately, with the US Open, the Olympics win for Japan, and college football, how could I not?

And lo and behold — instead of a splashy ad on the front page of the paper, there was an article this week deep into the sports section — about Oracle.

It appears that the Oracle team in the America’s Cup competition was in the news — not for doing well — but for receiving penalties. The penalties, the harshest in America’s Cup history, were imposed for illegally modifying 45-foot catamarans.

One place where we would like to think that “illegal modifications” are also not tolerated is in benchmarking.

Oracle this week claimed performance and price performance leadership based on the Storage Performance Council SPC-2 benchmark. I’m sure that with this being an industry standard benchmark there were no modifications – but that doesn’t mean that there were not some difficulties with comparisons claimed. Here’s what you need to know:

  • The Oracle ZFS Storage ZS3-4 result was just released. The IBM and HP results they compare to are from 2012, a lifetime ago in the benchmarking world.
  • The Oracle storage result used a 2-node cluster and 1.6x the physical capacity of the IBM DS8700 result.(1)
  • A fit for purpose methodology is needed for these storage comparisons – are you running analytics or critical batch processing? Different workloads require different levels of nonfunctional requirements which translate into different types of storage.
  • With storage, it’s essential to compare all the options, including many of the new flash offerings.
  • What is the reliability and support for these storage devices? Instead of just price/performance, make sure you study the real TCO.

 

It matters whether you win or lose. But it also matters how you play the game.

************************************************

(1) Results as of September 10, 2013, for more information go to http://www.storageperformance.org/results SPC-2. Results for Oracle ZFS Storage ZS3-4 are 17,244.22 SPC-2 MBPS™, $22.53 SPC-2 Price-Performance. Full results are available at http://www.storageperformance.org/results/benchmark_results_spc2#b00067. Results for IBM DS8870 are 15,423.66 SPC-2 MBPS, $131.21 SPC-2 Price-Performance. Full results are available at http://www.storageperformance.org/results/benchmark_results_spc2#b00062. Results for HP P9500 XP Disk Array are 13,147.87 SPC-2 MBPS, $88.34 SPC-2 Price-Performance. Full results are available at http://www.storageperformance.org/results/benchmark_results_spc2#b00056

SPC Benchmark-1 and SPC Benchmark-2 are trademarks of the Storage Performance Council.

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , , , ,,,,,,,,,

Written by benchmarkingblog

September 11, 2013 at 3:05 pm

Posted in Oracle, storage

Tagged with , ,

Oracle’s SPARC T5 and M5 Benchmarks: Lather, Rinse, Repeat

with 21 comments

I think I’ve said this before but one of my most absolute favorite movies is Groundhog Day. (Attention: spoiler is coming but since the fricking movie is from 1993 and most of us were old even way back then, I don’t think I will be ruining it for anyone.) Groundhog Day is an American comedy film directed by Harold Ramis and starring Bill Murray and Andie MacDowell (who by the way I’ve been told that I sort of look like which is really cool since she does L’Oréal ads). In the film an arrogant and egocentric TV weatherman, covering the annual Groundhog Day event, finds himself repeating the same day again and again.

The phrase “Groundhog Day” now has entered common lexicon as a reference to an unpleasant situation that continually repeats, or seems to.

And I would say that is exactly what we have with Oracle’s new SPARC T5 and M5 benchmarks.

Just as with every Oracle processor announcement, the benchmark results do the same thing. Many of the claims are Oracle’s own benchmarks that are not published and audited. There are a small number of industry standard benchmarks — and of course these are ones where it is extremely difficult, if not impossible, to compare to other relevant results. For price claims, Oracle, as they’ve done in the past, only factors in the price of the pizza box – make sure you add in the all-important software and storage.

Let’s take a look at the T5 and M5 benchmark results:

  • SAP: The IBM POWER7+ with DB2 10 SAP SD 2-tier result from back in September was 1.3x greater per core than the M5 and 1.9x greater than the T5 result.(1) The IBM average database request time was also much better and the CPU utilization of the IBM system was also more effective.
  • TPC-C: An IBM POWER6 result from 2008, 2 generations ago, is 42% higher per core than the new T5 result on this OLTP benchmark. An IBM POWER7 result from 2010, 1 generation ago, is 2.2x better performance per core than the Oracle result. (2) The price for all Oracle database software support used in computing the price/performance for this benchmark is $2300/year – I can only guess what you get for that. Also note that this benchmark used Oracle Partitioning which may not be realistic for your real world workloads. The Oracle database software is not even available until September.
  • SPECjEnterprise2010: Oracle’s T5 result needed four times the number of database cores, four times the amount of memory and significantly more storage than the IBM POWER7 result. (3)
  • SPECjbb2013: For Java business, let’s run a benchmark that can only be compared with a couple of ProLiants, one of our old T4s, and a Supermicro. (4)
  • SPECcpu: IBM Power Systems is #1 – don’t forget to look at number of cores for integer and floating point claims.
  • TPC-H: Ha, got you. There is no TPC-H. Funny, was expecting one based on what we saw for the T4. I wonder why . . .
  • The other benchmark claims? These are once again ones that either are Oracle’s own benchmarks or ones nobody cares about because they don’t look like anything we actually run. Chance of departure from useful benchmark results: 100%.
  • Don’t let these claims distract from asking about the business value delivered by these systems.

    I wake up every day, right here, right in Cleveland, and it’s always snowing, and there’s nothing I can do about it. “Winter, slumbering in the open air, wears on its smiling face a dream… of spring.”

    ************************************************
    (1)IBM Power 780 (3.72 GHz) two-tier SAP SD Standard Application Benchmark result (SAP enhancement package 5 for the SAP ERP 6.0 application: 12 processors / 96 cores / 384 threads, POWER7+, 1536 GB memory, 57,024 SD benchmark users, running AIX® 7.1 and DB2® 10, dialog resp.: 0.98s, line items/hour: 6,234,330, Dialog steps/hour: 18,703,000, SAPS: 311,720, DB time (dialog/ update): 0.009s / 0.014s, CPU utilization: 99%, Certification #2012033

    Oracle SPARC Server M5-32 SAP SD 2-tier result of 85,050 users, Average dialog response time: 0.80 seconds, Fully processed order line items per hour: 9,452,000,Dialog steps per hour: 28,356,000,SAPS: 472,600,Average database request time (dialog/update): 0.018 sec / 0.044 sec,CPU utilization of central server: 82%,Operating system, central server: Solaris 11,RDBMS: Oracle 11g,SAP Business Suite software: SAP enhancement package 5 for SAP ERP 6.0,32 processors / 192 cores / 1536 threads,SPARC M5, 3.60 GHz, 16 KB (D) and 16 KB (I) L1 cache and128 KB L2 cache per core, 48 MB L3 cache per processor,4096 GB main memory,Certification #2013009

    Oracle SPARC Server T5-8 SAP SD 2-tier result of 40,000 users,Average dialog response time: 0.86 seconds,Fully processed order line items per hour: 4,419,000,Dialog steps per hour: 13,257,000,SAPS: 220,950,Average database request time (dialog/update): 0.049 sec / 0.131 sec,CPU utilization of central server: 88%, Operating system, central server: Solaris 11,RDBMS: Oracle 11g,SAP Business Suite software: SAP enhancement package 5 for SAP ERP 6.0, 8 processors / 128 cores / 1024 threads,SPARC T5, 3.60 GHz, 16 KB (D) and 16 KB (I) L1 cache and 128 KB L2 cache per core, 8 MB L3 cache per processor,2048 GB main memory,Certification #2013008.

    (2) IBM Power 780 (2 chips, 8 cores, 32 threads) with IBM DB2 9.5 (1,200,011 tpmC, $.69/tpmC, configuration available 10/13/10); IBM Power 595 (5 GHz, 32 chips, 64 cores, 128 threads) with IBM DB2 9.5 (6,085,166 tpmC, $2.81/tpmC, configuration available 12/10/08); vs. Oracle SPARC T5-8 (8 chips, 128 cores, 1024 threads – 8,552,523 tpmC, $.55/tpmC, configuration available 9/25/13).

    (3) WebSphere Application Server V7 on IBM Power 780 and DB2 on IBM Power 750 Express, (64 core app server, 32 core db server), 16,646.34 SPECjEnterprise2010 EjOPS vs. SPARC T5-8 server (SPARC T5-8 server base package, 8x SPARC T5 16-core processors, 128x16GB-1066 DIMMS, 2x600GB 10K RPM 2.5” SAS-2 HDD result of SPARC T5-8, 57,422.17 SPECjEnterprise2010 EjOPS.

    (4) http://www.oracle.com/us/solutions/performance-scalability/sparc-t5-2-specjbb2013-1925099.html

    Sources: http://www.spec.org, http://www.tpc.org, http://www.sap.com. Results current as of 3/26/13.

    TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

    SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
    logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
    over the world.

    SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

    The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

    technorati tags: , , ,,,,,,,,,,,,,,,

    Written by benchmarkingblog

    March 26, 2013 at 5:53 pm

    Oracle’s New T5 TPC-C: Where’s the SPARC?, Part II

    with 5 comments

    With Oracle’s new SPARC server announcement today, we are all still waiting in anticipation (take your pick of Rocky Horror or Carole King) for something exciting. The just released TPC-C benchmark result surely is not.

    Here are some reasons why:

  • The performance of the Oracle T5-8 (even with the use of Oracle database partitioning) is downright lackluster. An IBM POWER6 result from 2008, 2 generations ago, is 42% higher per core. An IBM POWER7 result from 2010, 1 generation ago, is 2.2x better performance per core than the Oracle result. (1)
  • The price for all Oracle software support used in computing the price/performance for this benchmark is $2300/year. I can only guess what you get for that.
  • The Oracle database software is not even available until September. Yes, September.
  • It’s keeping me wa a a a aiting . . .

    ************************************************

    (1) IBM Power 780 (2 chips, 8 cores, 32 threads) with IBM DB2 9.5 (1,200,011 tpmC, $.69/tpmC, configuration available 10/13/10); IBM Power 595 (5 GHz, 32 chips, 64 cores, 128 threads) with IBM DB2 9.5 (6,085,166 tpmC, $2.81/tpmC, configuration available 12/10/08); vs. Oracle SPARC T5-8 (8 chips, 128 cores, 1024 threads – 8,552,523 tpmC, $.55/tpmC, configuration available 9/25/13).
    Source: http://www.tpc.org. Results current as of 3/26/13.
    TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

    The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

    technorati tags: , , ,,,,,,,,,,,

    Written by benchmarkingblog

    March 26, 2013 at 2:23 pm

    Posted in Oracle, SPARC T5, TPC-C

    Tagged with , , ,

    New Oracle M5 and T5 SAP Benchmark Results: No SPARC at all

    with 14 comments

    If you were hoping for some Last Friday Night excitement from Oracle’s new SPARC servers announcement this week, we haven’t seen it yet. Oracle just this morning published two SAP SD 2-tier benchmark results, one on the M5-32 and one on the T5-8.

    The IBM POWER7+ with DB2 10 result from back in September was 1.3x greater per core than the M5 and 1.9x greater than the T5 result.(1) The IBM average database request time was also much better and the CPU utilization of the IBM system was also more effective.

    Will the sun come out tomorrow for Oracle?

    ************************************************

    (1)IBM Power 780 (3.72 GHz) two-tier SAP SD Standard Application Benchmark result (SAP enhancement package 5 for the SAP ERP 6.0 application: 12 processors / 96 cores / 384 threads, POWER7+, 1536 GB memory, 57,024 SD benchmark users, running AIX® 7.1 and DB2® 10, dialog resp.: 0.98s, line items/hour: 6,234,330, Dialog steps/hour: 18,703,000, SAPS: 311,720, DB time (dialog/ update): 0.009s / 0.014s, CPU utilization: 99%, Certification #2012033

    Oracle SPARC Server M5-32 SAP SD 2-tier result of 85,050 users, Average dialog response time: 0.80 seconds, Fully processed order line items per hour: 9,452,000,Dialog steps per hour: 28,356,000,SAPS: 472,600,Average database request time (dialog/update): 0.018 sec / 0.044 sec,CPU utilization of central server: 82%,Operating system, central server: Solaris 11,RDBMS: Oracle 11g,SAP Business Suite software: SAP enhancement package 5 for SAP ERP 6.0,32 processors / 192 cores / 1536 threads,SPARC M5, 3.60 GHz, 16 KB (D) and 16 KB (I) L1 cache and128 KB L2 cache per core, 48 MB L3 cache per processor,4096 GB main memory,Certification #2013009

    Oracle SPARC Server T5-8 SAP SD 2-tier result of 40,000 users,Average dialog response time: 0.86 seconds,Fully processed order line items per hour: 4,419,000,Dialog steps per hour: 13,257,000,SAPS: 220,950,Average database request time (dialog/update): 0.049 sec / 0.131 sec,CPU utilization of central server: 88%, Operating system, central server: Solaris 11,RDBMS: Oracle 11g,SAP Business Suite software: SAP enhancement package 5 for SAP ERP 6.0, 8 processors / 128 cores / 1024 threads,SPARC T5, 3.60 GHz, 16 KB (D) and 16 KB (I) L1 cache and 128 KB L2 cache per core, 8 MB L3 cache per processor,2048 GB main memory,Certification #2013008.

    Source: http://www.sap.com; Results current as of 03/25/12.

    SAP and all SAP logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries. Other names may be trademarks of their respective owners.

    The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

    technorati tags: , , ,,,,,,,,,,,

    Written by benchmarkingblog

    March 25, 2013 at 12:28 pm

    Posted in Oracle, SAP, SPARC T5

    Tagged with , , , , ,

    Oracle, Blowin’ in the Wind

    leave a comment »

    Oracle announced this morning the general availability of the Oracle Database Appliance X3-2. It now supports virtualization.

    Hmmm, not exactly groundbreaking dare I say.

    Along with this Oracle announcement were many claims of bigger and better — than a previous version.
    With no real data and no real comparisons. And all I could think of was a Saturday morning a couple of months ago.

    You see, I was in an industrial supply store this one Saturday morning (the things we do for love). Looking at, of all things, leaf blowers.

    • Some models of these new leaf blowers claimed to blow harder that others. Surely harder than the old broken one at home. Which never really blew all that well. In fact, I probably ended up using my beloved rake more than that big hunk of plastic.
    • These new leaf blowers claimed to be bigger and better. That meant that they actually weighed a ton more and had to be converted from a hand held model to a huge backpack. Ghostbuster time. And you needed earplugs.
    • They let you try it out in the store. But instead of heavy wet leaves, they had you blow strips of paper.
    • The advertised pricing did not include the expensive oil that you needed for it — without that specific oil, the machine would die.

    So it comes down to the fact that claims are similar all over. But does the appliance finish the job fast, reliably, and at the right total price — so that you can get to what you really want to be doing.

    ************************************************

    The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

    technorati tags: , , ,,,,,,,,,

    ,,

    Written by benchmarkingblog

    March 5, 2013 at 11:06 am

    Posted in Appliance, Oracle

    Tagged with , , ,

    %d bloggers like this: