benchmarkingblog

Benchmarking and Systems Performance

Why IT Infrastructure Really Really Matters

leave a comment »

I went apple picking with the dog last weekend. The orchard was sodden with rain but the trees were heavy with beautiful fruit. I picked one, took a bite. My lab took many bites. The Melrose apple had beautiful red skin and lovely white fruit and was incredibly crisp.

But the taste was not so sweet. And certainly not as sweet as previous years. I later heard that this year, because of various aspects of the infrastructure (like temperature and rainfall in this case), none of the types of apples have been as sweet.

Infrastructure matters.

When we talk about things that really matter to us in our business – like availability of our systems, security of our business, performance of our applications – ultimately we are talking about satisfaction of our most important entity, our customers.

Analytics

What drives these nonfunctional requirements of our business ends up being our underlying infrastructure. So in the end, our IT infrastructure plays a critical role in our success.

Just like the proliferation of pumpkins lately, IBM has a slew of awesome announcements today that address this critical IT infrastructure. Power Systems and Smarter Storage, as well as PureSystems and other IBM technologies, bring together industry leading capabilities for the best enterprise-class infrastructure with virtualization and cloud technology including:

  • Enterprise-class systems: Leadership performance, resilience and resource sharing
  • Enterprise-class Virtualization and Cloud Management
  • Flexible, efficient workload deployment with Elastic Capacity on Demand (COD) and Power Integrated Facility for Linux (IFLs)
  • Power Enterprise Pool with Mobile COD delivers unprecedented availability, security, flexibility
  • Big Data and analytics focus: IBM BLU Acceleration Solution – Power Systems Edition.
  • Cloud storage and Storwize offerings for efficiency and value.

Analytics

Infrastructure was never so sweet.

************************************************

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,

,,,,,,,

Written by benchmarkingblog

October 8, 2013 at 12:05 pm

Posted in announcement

Tagged with

Guns and Butter at OpenWorld

with one comment

I guess when you are really really rich you can do things like miss your own keynote to go to a sporting event. Or get prices wrong by millions of dollars.

Yes, I took Econ 1A in college (though I may remember more about the cute boy in the row in front of me than supply and demand). I clearly remember grasping the intricate graphs and complex formulas in the thick colorful book by Samuelson.

But that preparation did not seem to help this week in trying to understand the new Oracle “Economics” at OpenWorld. A quick search did not lead to any scholarly articles on “near linear pricing.” If there is any sort of “re-engineering” of economics going on, it has not been picked up by the MBA programs just yet.

So when you see any pricing comparisons from Oracle these days, here is what you need to know:

  • Sometimes the systems compared have different numbers of processor cores. Sometimes the systems are the same “size” but size does not equal the performance of what can be run on the system.
  • Sometimes the systems compared have different amounts of memory. Sometimes the systems have the same amount of memory but amount of memory does not equal the performance of what can be run on the system.
  • Sometimes Oracle includes no software on their system and includes software on the other vendor’s system.
  • Sometimes Oracle does not include the expensive Oracle database license costs, which by the way are calculated by core.
  • Sometimes the systems compared have very very different types of support and maintenance.
  • Sometimes the systems compared have very different types and amounts of storage included. Or no storage at all. As we know, storage can be a large part of a system’s configuration and price.

There has been absolutely NO substantiation to justify equivalent price configurations for equivalent throughput systems in these comparisons.

What is ultimately important is what non-functional requirements the system gives you at a certain price. Compare, and do the TCO. And tell Oracle: I don’t buy sockets, I buy performance.

************************************************

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,

,,,,,,,

Written by benchmarkingblog

September 25, 2013 at 10:36 am

Posted in Oracle

Tagged with ,

Born to Run Benchmarks

with 9 comments

With apologies to Bruce, you can’t start a fire with a SPARC. A fire of proof points, that is.

In two different instances Oracle’s recent announcements on SPARC benchmark data have been lacking — and certainly couldn’t start any flame of passion at OpenWorld.

The first involved the announcement of the SPARC M6-32 server and engineered system. The press release only had a footnote for “estimated” performance of some mysterious sort. Oracle’s benchmark website actually discussed some benchmarks for this new system — but 1) there was no competitive information and 2) they were on Oracle’s very own benchmarks.

In the second case, the SPARC T5-8 was highlighted on the Java end-to-end SPECjEnterprise2010 benchmark. A record was claimed — in actuality, the IBM Power 780 had 19% greater overall performance and 49% greater application server performance per core than the Oracle system.(1)

Additionally, keep in mind that whenever costs are presented in Oracle’s comparisons, they need to be scrutinized to the highest degree. What storage is included, what software is included, what support and maintenance is included? Is an apple being compared to a pineapple?

(P.S. After I wrote this I discovered that today is actually Bruce Springsteen’s birthday. How weird is that?)

************************************************

(1)SPECjEnterprise2010 result of 36,571.36 on 1 x SPARC T5-8 (8 chips, 128 cores, 3.6 GHz SPARC T5);Oracle WebLogic 12c (12.1.2);Oracle Database 12c (12.1.0.1) vs. IBM result of 10,902.30 on 1 x IBM Power 780(8 chips, 32 cores, 4.42 GHz POWER7+);WebSphere Application Server V8.5;IBM DB2 Universal Database 10.1; Source: http://www.spec.org. Results as of 9/23/13.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,

,,,,,,

Written by benchmarkingblog

September 23, 2013 at 9:22 pm

The Wizard of OpenWorld

with one comment

Sometimes it’s great to see something for the hundredth time.

On Saturday night I went to see one of the all time greats, The Wizard of Oz — in 3D. The huge IMAX screen and 3D effects pulled you into the movie. I was dancing with the Munchkins and really skipping down that yellow brick road.

And sometimes you just want to cackle and destroy like the Wicked Witch of the West because you are being forced to see something for the hundredth time.

At Oracle OpenWorld’s keynote last night, the industry benchmarks that were highlighted made me want to do just that.

  • Oracle with Fujitsu claimed “14 World #1′s.” Then of course, doing what they do time and again, they only actually discussed a few of them.
  • In the SAP SD 2-tier comparison, Fujitsu/Oracle’s result was from 2013. IBM’s from 2010. Fujitsu/Oracle’s result used 640 cores, IBM only 256. IBM’s result was actually over 2x the users per core of the Oracle/Fujitsu result. We have surely seen this before, ain’t it the truth?(1)
  • The SPECjbb2013 comparison highlighted the M10 against some undesignated x86 system. Like the cowardly lion picking on little Toto.
  • The third benchmark was Stream, relevant for the very few in the commercial world.
  • Larry compared the M6-32 “Big Memory Machine” against a Power System. With absolutely no details and data to back the claim. We’ve seen this over and over as well.
  • Make no doubt about it. Absolutely none of these performance benchmarks have any pricing component whatsoever as a metric. And any pricing that is shown should be analyzed – what storage is included, what maintenance and support costs, is software added in? We’ve seen creative accounting here so many times before.

What was so special about seeing The Wizard of Oz on the big screen in 3D was that you noticed all of these incredible details (like the colorful birds, the beautiful expanse of red poppies, and the stage hand behind the apple trees) that you had never seen before. What was so NOT special about the OpenWorld keynote was that you were seeing the same old story — but with almost no details behind it. Once again.

************************************************

(1) IBM Power 795 (4.00 GHz) two-tier SAP SD Standard Application Benchmark result (SAP enhancement package 4 for SAP ERP 6.0 (Unicode): 32 processors / 256 cores / 1024 threads, POWER7, 4096 GB memory, 126,063 SAP SD benchmark users, OS: AIX 7.1, DB2 9.7. Certification #: 2010046 vs. Fujitsu M10-48 (40 processors / 640 cores / 1280 threads,153,000 SAP SD benchmark users, Oracle. Certification #: 2013014. Source: http://www.sap.com/benchmark. Results as of 9/23/13.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
over the world.

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,,,,,

,,

,,,

Written by benchmarkingblog

September 23, 2013 at 8:59 am

Posted in Oracle

Tagged with ,

On Big Data: Count Me In, But Do It Right

with one comment

Our local high school is now offering a new class in introductory statistics. And from what I’ve been seeing lately, we need this like my dog needs rawhide. (You see otherwise he will chew on sticks, rocks, and cement.)

I was recently reviewing some availability statistics. A regulatory group (which shall remain unnamed) was comparing number of outages between different types of equipment. Which is all very fine. The problem was that they were counting numbers of times, not percentage of times. Which means very little when you may have hundreds of instances of one type of equipment — and a total of ONE of another.

Another fallacy was that they were analyzing the 95% of the outages that had to do with one maintenance issue that had recently been solved – so what they really needed to focus on was the other 5% — and especially the outliers.

Another technique that drives me crazy is when someone rounds up when they should round down.

I’m not saying that everyone needs to have a deep understanding of multivariate ANOVA or the like. But with the plethora of Big Data applications and the way data is now woven into our society and in everything we do, it becomes exceedingly important to analyze and understand it in the right way.

We love to say “Do the Math.” But we need to make sure that when we do the math, we use the data in the correct and very best way to solve the problem.

************************************************

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , , , ,,,,,,

Written by benchmarkingblog

September 18, 2013 at 11:42 am

Posted in Big Data

Tagged with

Case of the Missing Benchmark and Other Cisco Tales

leave a comment »

Whether it’s Sherlock Holmes or Nancy Drew, it’s hard not to love mystery stories. It’s so great at the end when you realize, oh my gosh, I should have seen that coming. Or, I’m amazing, of course I saw that coming.

This week there was a large amount of hoopla around the announcement of the Intel Xeon Processor E5-2600 v2 product family. Which is all wonderful. But what is really interesting is Cisco’s new claim of 6 world record benchmarks surrounding the announcement.

Now as we know, Cisco has a history of claiming #1 benchmarks by counting not just current #1 records, but records since the beginning of time. Let’s see a few other tricks that Cisco is using in claiming performance “records” :

  • Oracle E-Business Suite Applications R12 Benchmark — It’s not hard to beat a previous generation of yourself.
  • SPECjbb2013 Benchmark (Java server performance) — Again, claim is essentially over themselves.
  • VMware View Planner Benchmark (desktop virtualization performance) — This is great but how hard is it really to be #1 when you are the only one.

 

But what is really interesting about Cisco’s list of benchmarks is what is missing. You see, Cisco also published an SAP SD 2-tier result but it is noticeably missing from the “world record” list.

Maybe just maybe because it happens to be behind three others — HP, Fujitsu, and IBM.

************************************************

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

VMware, the VMware “boxes” logo and design, Virtual SMP and VMotion are registered trademarks or trademarks (the “Marks”) of VMware, Inc. in the United States and/or other jurisdictions.

SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
over the world.

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,,,,,,,,,,

Written by benchmarkingblog

September 12, 2013 at 12:41 pm

Posted in Cisco, Intel

Tagged with , ,

Taking the Wind Out of Oracle’s Sails

with one comment

I don’t always read the sports pages. But lately, with the US Open, the Olympics win for Japan, and college football, how could I not?

And lo and behold — instead of a splashy ad on the front page of the paper, there was an article this week deep into the sports section — about Oracle.

It appears that the Oracle team in the America’s Cup competition was in the news — not for doing well — but for receiving penalties. The penalties, the harshest in America’s Cup history, were imposed for illegally modifying 45-foot catamarans.

One place where we would like to think that “illegal modifications” are also not tolerated is in benchmarking.

Oracle this week claimed performance and price performance leadership based on the Storage Performance Council SPC-2 benchmark. I’m sure that with this being an industry standard benchmark there were no modifications – but that doesn’t mean that there were not some difficulties with comparisons claimed. Here’s what you need to know:

  • The Oracle ZFS Storage ZS3-4 result was just released. The IBM and HP results they compare to are from 2012, a lifetime ago in the benchmarking world.
  • The Oracle storage result used a 2-node cluster and 1.6x the physical capacity of the IBM DS8700 result.(1)
  • A fit for purpose methodology is needed for these storage comparisons – are you running analytics or critical batch processing? Different workloads require different levels of nonfunctional requirements which translate into different types of storage.
  • With storage, it’s essential to compare all the options, including many of the new flash offerings.
  • What is the reliability and support for these storage devices? Instead of just price/performance, make sure you study the real TCO.

 

It matters whether you win or lose. But it also matters how you play the game.

************************************************

(1) Results as of September 10, 2013, for more information go to http://www.storageperformance.org/results SPC-2. Results for Oracle ZFS Storage ZS3-4 are 17,244.22 SPC-2 MBPS™, $22.53 SPC-2 Price-Performance. Full results are available at http://www.storageperformance.org/results/benchmark_results_spc2#b00067. Results for IBM DS8870 are 15,423.66 SPC-2 MBPS, $131.21 SPC-2 Price-Performance. Full results are available at http://www.storageperformance.org/results/benchmark_results_spc2#b00062. Results for HP P9500 XP Disk Array are 13,147.87 SPC-2 MBPS, $88.34 SPC-2 Price-Performance. Full results are available at http://www.storageperformance.org/results/benchmark_results_spc2#b00056

SPC Benchmark-1 and SPC Benchmark-2 are trademarks of the Storage Performance Council.

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , , , ,,,,,,,,,

Written by benchmarkingblog

September 11, 2013 at 3:05 pm

Posted in Oracle, storage

Tagged with , ,

%d bloggers like this: