benchmarkingblog

Elisabeth Stahl on Benchmarking and IT Optimization

Archive for the ‘SPEC’ Category

T4 MIA

with 2 comments

I was recently looking to do a comparison of a couple of systems and figured the easiest data to look at would be SPEC CPU data. I mean it’s almost a given that even though these integer and floating point benchmarks can be, should I say, pretty boring (yawn), at least you’ll find what you are looking for because everyone publishes pretty much everything on these.

To my surprise I realized that I did not see the one system I really needed to look at – a SPARC T4 result. And then I started to laugh to myself (it was pretty late).

Of course. T4 is the one where you can find results for some really strange benchmarks but none of the regular ones where you can actually compare anything.

And then I remembered something else. I went out to my blog just to confirm it and there it was staring at me in the face. The #1 all time most popular search term that gets people to read my blog is just that — “sparc t4 benchmark.” Followed closely by “sparc t4.” “sparc t4 specint.” “sparc t4 benchmarks.” “specint t4.” “t4 specint.” And “sparc t4 performance.”

Pretty sad that for this type of basic data you can not find it on the SPEC site. You can not find it on the Gartner site. You can not even find it on the Oracle site. For this type of basic performance data you need to go to an IBM site to find out that this data just doesn’t even exist.

************************************************

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,

Written by benchmarkingblog

February 13, 2013 at 9:51 am

Posted in SPARC T4, SPEC

Tagged with , ,

Life in the Fast Lane

leave a comment »

I recently wrote about a college visit — Big Data, Performance, and Coconut Smoothies might as well be titled Big Data at the Big Ten. And the ability to collect, measure, and analyze this massive flood of data for meaningful insights requires important non-functional IT requirements like reliability, availability, security, and of course performance.Smarter Computing

The IBM announcement of new products and offerings today leverages cloud to improve efficiency, focuses on data to deliver more actionable insight, and secures this critical data to protect and reduce risk. It’s Cloud Ready, Data Ready, Security Ready. And, of course, Performance Ready with:

  • New #1 Power Systems benchmarks for the new POWER7+ systems across a wide portfolio of applications including SAP, Java, and Technical Computing. Read about them here.
  • All the benchmarks and rPerf data in the performance report here.
  • Proof points like this one: The new 48-core IBM Power 760 with DB2 10 achieved the best 48-core two-tier SAP SD standard application benchmark result, surpassing the 80-core HP DL980. (1)
  • Performance details on the new POWER7+ systems including Power 730, Power 740, Power 750, Power 760, PowerLinux 7R2.
  • Leadership Storage performance including performance enhancements to IBM XIV. New caching algorithms increase performance compared to previous models up to 4.5 times for random and 5 times for sequential database workloads.

One thing that struck me on this college visit (besides how young everyone looks) is how college truly is an exciting new beginning. Everything is shining, brand spanking new. But this doesn’t only have to happen in college. There are frequently new beginnings throughout life.

Years ago, when social media was a relatively new concept for many of us, my manager at the time (who was always a source of exciting, new, and crazy ideas) “suggested” that I consider writing something called a blog. A downright scary idea, I implemented it anyway.

This week, this esteemed manager, colleague, mentor, associate is retiring after a very successful career of exciting ideas. To start a new beginning in the fast lane. And I will miss his inspiration. And his crazy ideas. Like a blog.

************************************************

(1)The 48-core IBM Power 760 (3.41 GHz) achieved the best 48-core two-tier SAP SD standard application benchmark result running SAP enhancement package 5 for the SAP ERP 6.0 application; 8 processors / 48 cores / 192 threads, POWER7+, 1024 GB memory, 25,488 SD benchmark users, running AIX® 7.1 and DB2® 10, dialog resp.: 0.99s, line items/hour: 2,784,330, Dialog steps/hour: 8,353,000 SAPS: 139,220, DB time (dialog/ update): .009s/.015s, CPU utilization: 99%, Certification #2013004. vs. HP DL980 G7, Xeon, SQL Server 2008, 25,160 users, 8 processors/80 cores/160 threads, SAP enhancement package 4 for SAP ERP 6.0, Certification # 2011021. http://www.sap.com. Results current as of 2/5/13.

SAP and all SAP logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries. Other names may be trademarks of their respective owners.

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,

,,,,,,,,,

Written by benchmarkingblog

February 5, 2013 at 12:24 am

A Tale of IBM Great Expectations

leave a comment »

“It was the best of times, it was the worst of times . . .”

Let’s look at the worst first.

Last week HP announced new Itanium servers. But don’t get too excited. Were there industry standard benchmarks announced in the press release? — No, just “internal lab testing.”

Last week I was meeting with some IT Analysts. They told me how disappointed they were with a certain vendor. “We’ll probably never see an Exadata benchmark.”

Now for the best.

Yesterday IBM once again was a leader in the TOP500 list of supercomputers with:

  • Most systems in TOP500 with 193. HP had 148. Oracle had 6.
  • Most installed aggregate throughput with over 66.2 out of 162 Petaflops. IBM has had this lead for an amazing 27 lists in a row.
  • Most in TOP 10 with 6.
  • Fastest Intel based system.
  • 26 of 30 most energy-efficient systems.

And today, IBM announced new Flex technologies for IBM PureSystems, including POWER7+ compute nodes. And there were benchmarks. Amazing benchmarks, including the IBM PureSystems #1 result on the SAP SD 2-tier 16-core benchmark. (1)

“Take nothing on its looks; take everything on evidence. There’s no better rule.”

“It was the age of wisdom . . . we had everything before us . . .”

************************************************

(1) IBM Flex System p260 Compute Node, 2 processors / 16 cores / 64 threads, IBM POWER7+, 4.10 GHz, 32 KB (I) and 32 KB (D) L1 cache and 256 KB L2 cache per core, 10 MB L3 cache per core, 256 GB main memory. Number of SAP SD benchmark users: 10,000, Average dialog response time: 0.97 seconds, Fully processed order line items/hour: 1,094,000, Dialog steps/hour: 3,282,000, SAPS:54,700, Average database request time (dialog/update):0.010 sec / 0.017 sec, CPU utilization of central server:99%, Operating system, central server:AIX 7.1, RDBMS:DB2 10, SAP Business Suite software:SAP enhancement package 5 for SAP ERP 6.0, Certification number: 2012035; http://www.sap.com.

Results current as of 11/13/12.

SAP and all SAP logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries. Other names may be trademarks of their respective owners.

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,

,,,,,,

Written by benchmarkingblog

November 13, 2012 at 11:21 am

Cloud and Data and Security, Oh My

leave a comment »

Cloud Ready. Data Ready. Security Ready. With outstanding performance.

Important themes for the Smarter Computing announcements today.

The new offerings include:
1) the most powerful enterprise Power Systems to date, POWER7+ systems,
2) a new high-end disk storage system, the DS8870, and
3) key software for the IBM zEnterprise EC12.

These new technologies are designed to help organizations improve security, take advantage of cloud computing, and manage and analyze the vast amounts of big data in our world today.

************************************************

(1) ) Storage Performance Council, “SPC Benchmark 2™ Full Disclosure Report IBM Corporation, IBM System Storage DS8870,” October 2012. Source: http://www.storageperformance.org/results/benchmark_results_spc2. SPC Benchmark-1 and SPC Benchmark-2 are trademarks of the Storage Performance Council.

SAP and all SAP logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries. Other names may be trademarks of their respective owners.

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,,,,,,,

Written by benchmarkingblog

October 3, 2012 at 12:16 am

Posted in announcement, Cloud, POWER7, SAP, Smarter, SPC, SPEC, zEnterprise

Tagged with

A Better Future — and Much Faster with IBM

leave a comment »

Is the future always better? Sometimes it’s not so clear.

Take reading. Now that everyone is reading on electronic devices, we complain about getting distracted by email and tweets. No wonder we’re not finishing Anna Karenina.

Take brushing teeth. I read this morning that kids these days are requiring oral surgery to fill numerous cavities — because they are now imbibing more juice drinks and parents are not making them brush their teeth because the kids don’t like brushing their teeth. Cry now or really cry later.

Take voting. Since moving to Ohio, I get hundreds of calls the day before Super Tuesday. Now they’re all automated so I can’t even hang up on anybody.

And sometimes our frustrations with the future should just be authored as a First World Problems meme.

Today there is something really exciting for our future. The new IBM System x and BladeCenter systems with Intel Xeon Processor E5-2600 series processors were just announced.

And with them, there were many wonderful benchmarks, outstanding leadership proof points from IBM including OLTP (TPC-E), SAP SD Two-Tier Standard Application Benchmark, virtualization (SPECvirt_sc2010), Java (SPECjbb2005), energy efficiency (SPECpower_ssj2008) and high performance computing (SPEC CPU2006). A whole portfolio of diverse workloads.

Making Smarter Computing real.

And a much faster future.

************************************************

Sources: http://www.tpc.org, http://www.spec.org, http://www.sap.com/benchmark.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).
SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).
SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
over the world.
The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , , , ,,,,,,,,,,

,,,,,,,,,

,

Written by benchmarkingblog

March 6, 2012 at 10:42 pm

Posted in announcement, SAP, SPEC, TPC

Tagged with

Is There a “Right” and “Wrong” with Benchmarks?

with 4 comments

Like many of our politicians lately, Oracle has been proselytizing — in this case that there are certain benchmarks which are right and certain benchmarks which are wrong.

In their evaluation Oracle claims to focus on all the right benchmarks while claiming that everyone else is focusing on the wrong ones. Let’s take a look at their reviews:

  • Thumbs up for SPECjEnterprise – Yes, I wholeheartedly agree, a good benchmark to use for Java applications. And that’s why IBM has outstanding results on this benchmark. Oracle’s T4 result needed four times the number of app nodes, twice the number of cores, almost four times the amount of memory and significantly more storage than the IBM POWER7 result.(1)
  • Thumbs up for Oracle ebusiness, JDEdwards, PeopleSoft, Siebel – Good when you need to model these exact applications but keep in mind that Oracle owns these “independent” benchmarks. Even with a stacked deck, IBM has #1’s here.
  • Thumbs up for TPC-H – Yes, totally agree that this benchmark is valuable for business intelligence applications. Which is why IBM just published a leadership 3TB TPC-H result.(2) And Oracle’s T4 result needed 2.7x the amount of storage and 14x the number of streams than the IBM POWER7 1TB result.(3)
  • Thumbs up for SPECweb2005 – Funny that Oracle likes this one since there is a definite lack of publishes here anymore and this benchmark is being retired in January.
  • Thumbs up for TPC-C – Even funnier that Oracle likes this one since Sun avoided publishing on this for many, many years claiming this benchmark was too old. Now, as a fair-weather friend, it’s a favorite. IBM has the top nonclustered result (even with an older Power system).(4)
  • Thumbs down for SAP SD – Interesting that this valuable benchmark for SAP users is on Oracle’s hit list. This benchmark is a wonderful measure for SAP users and has many proof points from many vendors. Is that why Oracle tries to avoid this one and instead publishes on other more obscure SAP benchmarks like SD-Parallel and ATO?
  • Thumbs down for SPECjAppServer2004 – Looking to fill up space, since Oracle knows as well as we do that this benchmark was retired almost a year ago. Also ironic because I do seem to remember that Oracle/Sun used to really like this benchmark. Alot.
  • Thumbs down for SPECcpu and SPECjbb – These are nice and easy for integer, floating point, and Java business apps. Surely valuable in their own way.
  • Thumbs down for Stream – Ignoring HPC users are we? We’ve seen this before with the TOP500. IBM has the most systems with 213. Oracle had 12.
  • Keep in mind that Oracle has only published their latest SPARC T4 results on 2 industry standard benchmarks — that’s right, only 2. Which is fine, if you don’t mind steering with your eyes closed.

    My view is that you may want and need to look at different aspects of systems across a wide portfolio of many different types of benchmarks. Which is why IBM is #1 — in over 100.

    ************************************************

    (1)Oracle WebLogic Server 11g and Oracle Database 11g Release 2 with Oracle Real Application Clusters and Oracle Solaris running on a four-node SPARC T4-4 cluster, each system with four SPARC T4 3GHz processors, (128 core app server, 64 core db server), 40,104.86 SPECjEnterprise2010 EjOPS vs. WebSphere Application Server V7 on IBM Power 780 and DB2 on IBM Power 750 Express, (64 core app server, 32 core db server), 16,646.34 SPECjEnterprise2010 EjOPS.
    (2)IBM POWER 780 (8 sockets/32 cores/128 threads) 192,001 QphH@3000GB, $6.37/QphH@3000GB, available 11/30/11.
    (3)SPARC T4-4 server (4 sockets/32 cores/256 threads) 201,487 QphH@1000GB, $4.60/QphH@1000GB, available 10/30/11. IBM POWER 780 Model 9179-MHB server (8 sockets/32 cores/128 threads) 164,747.2 QphH@1000GB, $6.85/QphH@1000GB, available 3/31/11.
    (4)A 64-core IBM Power 595 (5 GHz, 32 chips, 128 threads) with IBM DB2 9.5 is the best overall single system (6,085,166 tpmC, $2.81/tpmC, configuration available 12/10/08).
    Sources: http://www.spec.org, http://www.tpc.org, http://www.top500.org. Results current as of 10/26/11.

    TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

    SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
    logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
    over the world.

    SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

    The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

    technorati tags: , , , , ,,,,,,,,,,,,,

    Written by benchmarkingblog

    October 26, 2011 at 12:25 pm

    How can I make use of benchmarks in my purchasing decision?

    leave a comment »

    How can I make use of benchmarks in my purchasing decision?

  • Where is the best beach to snorkel off of in March?
  • If I choose the Rocky Mountains for my one week vacation, will I get altitude sickness?
  • What about Mongolia?
  • Sometimes choosing a vacation spot can be even more fun than the vacation itself. What do you use for input when you’re planning your time away? Weather statistics, websites, travel books, a recommendation from your brother-in-law — all can be great tools.

    An IT purchasing decision can be just as exciting.

    And one important tool to use in that decision should be benchmarks.

    Three types of benchmarks should be considered: industry standard, vendor, and client benchmarks. Industry standard benchmarks such as TPC and SPEC are especially valuable when considering a hardware or software purchasing decision involving such workloads as online transaction processing, business intelligence, Java, web, and high performance computing. Vendor benchmarks such as those from SAP are extremely useful because they could include the exact type of application that matches your solution. And client benchmarks, where you actually run your home grown applications on a vendor’s configuration, can give you a clarity that you just may not be able to get any other way.

    By analyzing the hardware and software used in these benchmarks and comparing them to your own, you can model anticipated workload on a system you are considering. You can compare one result with another. You can study the impact of an upgrade in software, addition of more processor cores, one system vs. a cluster, a change in an operating system, a different database.

    Benchmark data is not going to give you the absolute answer on what to purchase. Performance, of course, is only one factor out of many to consider in a decision. But oh what a factor it is.

    And if you have an opinion on the White Mountains vs. the Rocky Mountains in early June, please let me know.

    ************************************************
    Required Stuff

    SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

    TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

    SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
    logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
    over the world.

    The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

    technorati tags: , , , , ,,,,,

    Written by benchmarkingblog

    March 11, 2011 at 4:08 pm

    Posted in SAP, SPEC, TPC

    Tagged with , , ,

    %d bloggers like this: