benchmarkingblog

Elisabeth Stahl on Benchmarking and IT Optimization

Archive for the ‘Cisco’ Category

War and Peace with HP’s New Gen9

leave a comment »

I love my book club. We vote on what books to read and we always choose interesting and enlightening fiction and non-fiction. We always serve great food and drink. And, best of all, you don’t even have to read the book.

So I was really excited last month to hear about the book we would be reading. My good friend said she liked the book and the cover showed a woman in an olive grove. What could be better? Until I actually read the book and realized that there was nothing really that thrilling behind the cover.

And that’s exactly how I felt this week reading about HP’s newest announcement on their x86 HP ProLiant Gen9 servers. I mean, for goodness sake, we are at Gen9 — I’m sure we’ve come so far and there must be some thrilling new features and performance behind them.

But alas, once again, my excitement was thwarted by what I actually read.

There were 3 claims for these new systems:

  • Better TCO, optimization, and automation — but only comparing this new Gen9 to G6. I should hope there would be improvement.
  • Faster provisioning — The footnote says “Based on anonymous customer results” — hmmm. Plus the software isn’t even available yet for Gen9.
  • Improved performance — all it says is “equivalent controllers” in a “controlled environment.” — really hard to say what that means.

When I see claims like these with no real data all I can think is what real data looks like and how IBM Power Systems continue to surpass in performance.

And it reminds me that the olive grove book is no Pride and Prejudice.

At least HP tried to address performance — unlike Cisco, where it was totally ignored.

For next month we are reading Gabriel Garcia Marquez – so at least I will know to expect magic realism.

************************************************

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,

Written by benchmarkingblog

August 29, 2014 at 2:27 pm

Posted in announcement, Cisco, HP

Tagged with

Case of the Missing Benchmark and Other Cisco Tales

leave a comment »

Whether it’s Sherlock Holmes or Nancy Drew, it’s hard not to love mystery stories. It’s so great at the end when you realize, oh my gosh, I should have seen that coming. Or, I’m amazing, of course I saw that coming.

This week there was a large amount of hoopla around the announcement of the Intel Xeon Processor E5-2600 v2 product family. Which is all wonderful. But what is really interesting is Cisco’s new claim of 6 world record benchmarks surrounding the announcement.

Now as we know, Cisco has a history of claiming #1 benchmarks by counting not just current #1 records, but records since the beginning of time. Let’s see a few other tricks that Cisco is using in claiming performance “records” :

  • Oracle E-Business Suite Applications R12 Benchmark — It’s not hard to beat a previous generation of yourself.
  • SPECjbb2013 Benchmark (Java server performance) — Again, claim is essentially over themselves.
  • VMware View Planner Benchmark (desktop virtualization performance) — This is great but how hard is it really to be #1 when you are the only one.

 

But what is really interesting about Cisco’s list of benchmarks is what is missing. You see, Cisco also published an SAP SD 2-tier result but it is noticeably missing from the “world record” list.

Maybe just maybe because it happens to be behind three others — HP, Fujitsu, and IBM.

************************************************

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

VMware, the VMware “boxes” logo and design, Virtual SMP and VMotion are registered trademarks or trademarks (the “Marks”) of VMware, Inc. in the United States and/or other jurisdictions.

SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
over the world.

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,,,,,,,,,,

Written by benchmarkingblog

September 12, 2013 at 12:41 pm

Posted in Cisco, Intel

Tagged with , ,

Cisco Virtualization, the Price You Pay

with one comment

We were lucky enough recently to have two comparable Cisco SAP SD benchmark results published. The results used pretty much the same hardware, the same software, the same benchmark kit. The big difference was that one was virtualized and one was not.

The performance metric for this benchmark is the number of SAP benchmark users. For the regular configuration the number of users was 6530. The virtualized version was 1000 less.(1)

That’s a considerable difference when it comes to running your business.

Compare that to the legacy results of PowerVM performance on this very same benchmark. The results of both the virtualized and non-virtualized versions are essentially the same.(2)

Power Systems servers implement a virtualization architecture with components embedded in the hardware, firmware and operating system software. The capabilities of this integrated virtualization architecture are thus significantly different and in many areas more advanced.

Without paying the performance price.

************************************************

(1) SAP SD 2-tier Cisco UCS B200 M3, 2 processors / 16 cores / 32 threads, Intel Xeon Processor E5-2690, 2.90 GHz, 64 KB L1 cache and 256 KB L2 cache per core, 20 MB L3 cache per processor, 256 GB main memory; SAP SD benchmark users: 6,530 , Average dialog response time: 0.98 seconds, Throughput: Fully processed order line items/hour: 713,670, Dialog steps/hour: 2,141,000, SAPS: 35,680. Average database request time (dialog/update): 0.015 sec / 0.036 sec, CPU utilization: 99%, Red Hat Enterprise Linux 6.3, Sybase ASE 15.7, SAP enhancement package 5 for SAP ERP 6.0, Certification #2013001.
SAP SD 2-tier Cisco UCS B200 M3, 2 processors / 16 cores / 32 threads, Intel Xeon Processor E5-2690, 2.90 GHz, 64 KB L1 cache and 256 KB L2 cache per core, 20 MB L3 cache per processor, 256 GB main memory; 1 virtual machine (VM) using 32 virtual CPUs. CPU utilization of VM1 (DB/Dia/Upd/Msg/Enq): 97%, Number of SAP SD benchmark users: 5,530; Average dialog response time:0.96 seconds;Throughput: Fully processed order line items/hour: 605,330; Dialog steps/hour:1,816,000;SAPS:30,270; Average database request time (dialog/update):0.021 sec / 0.045 sec; CPU utilization of central server:97%; Operating system, central server:Red Hat Enterprise Linux 6.4 on KVM; RDBMS: Sybase ASE 15.7; SAP enhancement package 5 for SAP ERP 6.0; Certification #2013007.

(2) SAP SD 2-tier IBM Power 570, 2 processors /4 cores / 8 threads, POWER6, 4.7 GHz,Number of SAP SD benchmark users: 2035, users/core = 508.75; Average dialog response time: 1.99 seconds;Throughput:Fully Processed Order Line items/hour: 203,670;Dialog steps/hour: 611,000;SAPS: 10,180;Average DB request time (dia/upd): 0.011 sec / 0.015 sec;CPU utilization of central server: 99%;Operating System central server: AIX 5L Version 5.3;RDBMS: Oracle 10g;SAP ECC Release: 6.0;Certification #2007037.
SAP SD 2-tier IBM Power 570, 2 processors /4 cores / 8 threads using 2 virtual cpus, POWER6, 4.7GHz; Number of SAP SD benchmark users: 1020, users/core=510;Average dialog response time: 1.99 seconds;Throughput:Fully Processed Order Line items/hour: 102,000;Dialog steps/hour: 306,000;SAPS: 5,100;Average DB request time (dia/upd): 0.005 sec / 0.009 sec;CPU utilization of central server: 50%;CPU utilization inside virtual machine: 99%;Operating System central server: AIX 6.1 on IBM Power VM (using 2 virtual CPUs);RDBMS: DB2 9.5;SAP ECC Release: 6.0; Certification #2008080

http://www.sap.com. Results current as of 3/20/13.

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

SAP and all SAP logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries. Other names may be trademarks of their respective owners.

technorati tags: , , ,,,,,,,,,,,

Written by benchmarkingblog

March 20, 2013 at 5:01 pm

Posted in Cisco, SAP

Tagged with , ,

Cisco: TV Everywhere, Benchmarks Nowhere

leave a comment »

No more having to remember when Glee is on. Or what channel. Or whether I should record it.

I read this morning that Cisco has just announced a new video platform — one that will in the future enable the TV shows and movies you want to watch to find you via software that uses past viewing patterns. And the ability to watch on any device via a cloud-based digital video recorder.

The “Above and beyond TV Everywhere” concept.

If only Cisco’s benchmarks could be so exciting.

Cisco’s new SAP SD 2-tier benchmark result today is so lackluster, you certainly wouldn’t want it to find you. Or record the result.

The IBM Flex System x240 had 21% better performance and the Flex System p260 (POWER7+) had 53% better performance.(1)

Thursday on Glee is the Michael Jackson episode. Expect a “Bad” performance as well.

************************************************

(1)IBM Flex System p260 Compute Node, 2 processors / 16 cores / 64 threads, IBM POWER7+, 4.10 GHz, 32 KB (I) and 32 KB (D) L1 cache and 256 KB L2 cache per core, 10 MB L3 cache per core, 256 GB main memory. Number of SAP SD benchmark users: 10,000, Average dialog response time: 0.97 seconds, Fully processed order line items/hour: 1,094,000, Dialog steps/hour: 3,282,000, SAPS:54,700, Average database request time (dialog/update):0.010 sec / 0.017 sec, CPU utilization of central server:99%, Operating system, central server:AIX 7.1, RDBMS:DB2 10, SAP Business Suite software:SAP enhancement package 5 for SAP ERP 6.0, Certification #2012035.

IBM Flex System x240 – IBM Flex System x240 on the two-tier SAP SD running SAP enhancement package 4 for SAP ERP 6.0 (Unicode): 2 processors /16 cores / 32 threads, Intel Xeon Processor E5-2690, 2.90 GHz, 64 KB L1 cache and 256 KB L2 cache per core, 20 MB L3 cache per processor,128 GB main memory, 7,960 SAP SD benchmark users, dialog response: 0.98 seconds, line items/hour: 870,330, dialog steps/hour: 2,611,000, SAPS: 43,520, database response time (dialog/update): 0.012 sec / 0.010 sec, CPU utilization: 99%, OS: Windows Server 2008 R2 Enterprise Edition, DB2 9.7; Certification #2012016.

Cisco UCS B200 M3, 2 processors / 16 cores / 32 threads, Intel Xeon Processor E5-2690, 2.90 GHz, 64 KB L1 cache and 256 KB L2 cache per core, 20 MB L3 cache per processor, 256 GB main memory; SAP SD benchmark users: 6,530 , Average dialog response time: 0.98 seconds, Throughput: Fully processed order line items/hour: 713,670, Dialog steps/hour: 2,141,000, SAPS: 35,680
Average database request time (dialog/update): 0.015 sec / 0.036 sec, CPU utilization: 99%, Red Hat Enterprise Linux 6.3, Sybase ASE 15.7, SAP enhancement package 5 for SAP ERP 6.0, Certification #2013001.

http://www.sap.com. Results current as of 1/8/13.

SAP and all SAP logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries. Other names may be trademarks of their respective owners.

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,

,,,,,

Written by benchmarkingblog

January 8, 2013 at 4:30 pm

Posted in Cisco, PureSystems, SAP

Tagged with , , ,

Lackluster Larry

with 2 comments

As we await that big wave called OpenWorld to reach our shores, let’s take a look at what we have actually seen from Oracle in the last few days.

  • Questions on Oracle’s latest earnings and disappointing performance in hardware
  • A new ad on the front page of the Wall Street Journal that attempts to make a groundbreaking statement about the very old news that there are public clouds and there are private clouds.
  • A press release on a new TPC-C OLTP benchmark result.The problem is that it’s actually a Cisco benchmark result that just happens to use some Oracle software. And the problem is that the Cisco system in the comparison is 2 years newer, needs 2x the number of cores, and uses over 60% more storage than the IBM result cited. Even so, the older IBM system is 1.49x times the performance per core of the Cisco system. (1)

Based on this news plus what I’ve heard may be coming that huge OpenWorld wave may turn out to be merely a minor ripple in our IT landscape.

************************************************

(1)Cisco UCS C240 M3, 1,609,186.39 tpmC, $0.47/tpmC, (2 processors/16 cores/32 threads) available 9/27/12 vs. IBM Power 780 Server Model 9179-MHB with IBM DB2 9.5, 1,200,011.00 tpmC, $0.69/tpmC, available 10/13/10 (2 processors/8 cores/32 threads). Results current as of 9/28/12. Source: http://www.tpc.org.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).
The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,,

Written by benchmarkingblog

September 28, 2012 at 11:42 am

Posted in Cisco, Cloud, Oracle, TPC-C

Tagged with , ,

Cisco’s March Madness

with 2 comments

I am told that my 5 foot 5 grandmother was a basketball star in high school. The story goes that she racked up many points per game and may have even made some records. I am pretty sure that if she played Jeremy Lin today, she would not be able to keep those records.

The point here is simple: You need to differentiate between adding up records from the beginning of time vs. counting the number of current records.

Last week, Cisco announced new UCS systems, claiming 63 industry benchmark world records overall. Here’s what you need to know:

  • Cisco claims 63 UCS records and provides a link. The link goes to a document which lists 50 proof points. If you’re claiming 63 records, it would be good to see 63 records.
  • Almost a quarter of these “records” are one benchmark and one benchmark only — a virtualization benchmark which only highlights very limited performance ability using a very specific set of products. Not quite the “breadth” that Cisco claims.
  • Some of the results cited are records purely because they were “first to market.” So Cisco was merely first to publish a result on a certain benchmark in a certain category — and may have only been the best for a short while because they were first.
  • Finally, and most importantly, here’s the clincher, so pay really close attention. Cisco’s claimed “records” are “Record as of Publication Date.” This means that Cisco counts records and puts notches in their belt if they have a result that has ever been on top, even for a day over the last three years. The real question is how many of those 63 benchmark results are currently #1 ?   . . .  For many, many years IBM has published web pages with #1 Power benchmarks. With over 100 #1 benchmarks. These over 100 benchmarks are all benchmarks where IBM is currently #1. At the moment in time. Not #1’s culled from all time. I know this. I know this because I personally and painstakingly have to update these very pages.

I would love to see a count of how many records IBM would have if they included #1’s from the beginning of time. I would love to see my grandmother on the court with Jeremy Lin. But I just can’t fathom the awesomeness of either.

************************************************

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , , , ,,,,,,,,

,,,,,,,,

,,,,

Written by benchmarkingblog

March 12, 2012 at 3:38 pm

Posted in Cisco

Tagged with , ,

Cisco/Oracle Super Saturday TPC-C

with 2 comments

So now they’re trying to put another one over on us. “Super Saturday” for retailers, where you shop till you drop the Saturday before Christmas, has been officially moved to tomorrow, one week early. To give us more sales and more time to shop until we drop. I’m just getting tired of being told that I should shop on Black Friday, Cyber Monday, Super Saturday, or any other time. It’s sort of like the greeting card companies saying let’s make a new holiday to celebrate your dog.

And that’s how I’m feeling lately being told about the latest Oracle and Cisco benchmark claims on the newest TPC-C OLTP benchmark result. Here are the 3 things you need to know:

  • The Benchmark: Oracle didn’t even run this benchmark. And they didn’t even run it on their own hardware.
  • The Performance: Note the special pricey Violin memory arrays that Cisco/Oracle used. And even an IBM result from over a year and a half ago on this benchmark is 70% better performance per core than this result.(1)
  • The Pricing: Cisco and Oracle both take advantage of Super Saturday pricing methods in these results. Cisco shows a 57% “large purchase discount” for hardware. Pricing for Oracle 11g is for the limited standard edition. And very minimal support is included.

I know that both American Greetings and my black lab would really like that idea for the new holiday to celebrate your dog. I already bought my lab a large braided rawhide with red and green bows. So you know they have me.

Happy Holidays!

************************************************

(1) An 8-core IBM Power 780 (2 chips, 32 threads) with IBM DB2 9.5 is the best 8-core system (1,200,011 tpmC, $.69/tpmC, configuration available 10/13/10) vs. Oracle Database 11g Release 2 Standard Edition One and Oracle Linux on Cisco UCS c250 M2 Extended-Memory Server, 1,053,100 tpmC, $0.58/tpmC, available 12/7/2011.
Source: http://www.tpc.org. Results current as of 12/16/11.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , , , ,,,,,,,,,,,,,

Written by benchmarkingblog

December 16, 2011 at 10:37 am

Posted in Cisco, Oracle, POWER7, TPC-C

Tagged with , , , ,

%d bloggers like this: