Elisabeth Stahl on Benchmarking and IT Optimization

Taking the Wind Out of Oracle’s Sails

with one comment

I don’t always read the sports pages. But lately, with the US Open, the Olympics win for Japan, and college football, how could I not?

And lo and behold — instead of a splashy ad on the front page of the paper, there was an article this week deep into the sports section — about Oracle.

It appears that the Oracle team in the America’s Cup competition was in the news — not for doing well — but for receiving penalties. The penalties, the harshest in America’s Cup history, were imposed for illegally modifying 45-foot catamarans.

One place where we would like to think that “illegal modifications” are also not tolerated is in benchmarking.

Oracle this week claimed performance and price performance leadership based on the Storage Performance Council SPC-2 benchmark. I’m sure that with this being an industry standard benchmark there were no modifications – but that doesn’t mean that there were not some difficulties with comparisons claimed. Here’s what you need to know:

  • The Oracle ZFS Storage ZS3-4 result was just released. The IBM and HP results they compare to are from 2012, a lifetime ago in the benchmarking world.
  • The Oracle storage result used a 2-node cluster and 1.6x the physical capacity of the IBM DS8700 result.(1)
  • A fit for purpose methodology is needed for these storage comparisons – are you running analytics or critical batch processing? Different workloads require different levels of nonfunctional requirements which translate into different types of storage.
  • With storage, it’s essential to compare all the options, including many of the new flash offerings.
  • What is the reliability and support for these storage devices? Instead of just price/performance, make sure you study the real TCO.


It matters whether you win or lose. But it also matters how you play the game.


(1) Results as of September 10, 2013, for more information go to SPC-2. Results for Oracle ZFS Storage ZS3-4 are 17,244.22 SPC-2 MBPS™, $22.53 SPC-2 Price-Performance. Full results are available at Results for IBM DS8870 are 15,423.66 SPC-2 MBPS, $131.21 SPC-2 Price-Performance. Full results are available at Results for HP P9500 XP Disk Array are 13,147.87 SPC-2 MBPS, $88.34 SPC-2 Price-Performance. Full results are available at

SPC Benchmark-1 and SPC Benchmark-2 are trademarks of the Storage Performance Council.

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , , , ,,,,,,,,,

Written by benchmarkingblog

September 11, 2013 at 3:05 pm

Posted in Oracle, storage

Tagged with , ,

One Response

Subscribe to comments with RSS.

  1. Really interesting blog. Let me count the ways you’re offbase: Isn’t SPC-2 a fit for purpose streaming I/O benchmark? If you didn’t think it mattered, why did you publish a benchmark result? If 2012 is too old for your taste, you can always publish a new benchmark using DS8800 or better yet XIV 😉 They did publish a SPECsfs benchmark, right? Your fit-for-purpose storage offerings still don’t beat price/performance (TCO as you call it) of the ZS3-4; one unified storage beats your fit-for-purpose offerings. Now that’s sad, don’t you think?

    No benchmark is perfect, however you all agreed on these benchmark specifications. When you’re beat, you wish to complain/whine. On the other hand when you’ve a winning result, you can’t stop talking about it. Remember 2009 when IBM was beating their chest with their TPC-C benchmark using 11,000 disks? Where was your fit-for-purpose thinking then?

    Humility is our friend. Difference between champ/chump is one letter and vice-versa.


    September 16, 2013 at 11:39 pm

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: