benchmarkingblog

Elisabeth Stahl on Benchmarking and IT Optimization

Archive for the ‘TPC’ Category

A Better Future — and Much Faster with IBM

leave a comment »

Is the future always better? Sometimes it’s not so clear.

Take reading. Now that everyone is reading on electronic devices, we complain about getting distracted by email and tweets. No wonder we’re not finishing Anna Karenina.

Take brushing teeth. I read this morning that kids these days are requiring oral surgery to fill numerous cavities — because they are now imbibing more juice drinks and parents are not making them brush their teeth because the kids don’t like brushing their teeth. Cry now or really cry later.

Take voting. Since moving to Ohio, I get hundreds of calls the day before Super Tuesday. Now they’re all automated so I can’t even hang up on anybody.

And sometimes our frustrations with the future should just be authored as a First World Problems meme.

Today there is something really exciting for our future. The new IBM System x and BladeCenter systems with Intel Xeon Processor E5-2600 series processors were just announced.

And with them, there were many wonderful benchmarks, outstanding leadership proof points from IBM including OLTP (TPC-E), SAP SD Two-Tier Standard Application Benchmark, virtualization (SPECvirt_sc2010), Java (SPECjbb2005), energy efficiency (SPECpower_ssj2008) and high performance computing (SPEC CPU2006). A whole portfolio of diverse workloads.

Making Smarter Computing real.

And a much faster future.

************************************************

Sources: http://www.tpc.org, http://www.spec.org, http://www.sap.com/benchmark.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).
SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).
SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
over the world.
The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , , , ,,,,,,,,,,

,,,,,,,,,

,

Written by benchmarkingblog

March 6, 2012 at 10:42 pm

Posted in announcement, SAP, SPEC, TPC

Tagged with

Is There a “Right” and “Wrong” with Benchmarks?

with 4 comments

Like many of our politicians lately, Oracle has been proselytizing — in this case that there are certain benchmarks which are right and certain benchmarks which are wrong.

In their evaluation Oracle claims to focus on all the right benchmarks while claiming that everyone else is focusing on the wrong ones. Let’s take a look at their reviews:

  • Thumbs up for SPECjEnterprise – Yes, I wholeheartedly agree, a good benchmark to use for Java applications. And that’s why IBM has outstanding results on this benchmark. Oracle’s T4 result needed four times the number of app nodes, twice the number of cores, almost four times the amount of memory and significantly more storage than the IBM POWER7 result.(1)
  • Thumbs up for Oracle ebusiness, JDEdwards, PeopleSoft, Siebel – Good when you need to model these exact applications but keep in mind that Oracle owns these “independent” benchmarks. Even with a stacked deck, IBM has #1’s here.
  • Thumbs up for TPC-H – Yes, totally agree that this benchmark is valuable for business intelligence applications. Which is why IBM just published a leadership 3TB TPC-H result.(2) And Oracle’s T4 result needed 2.7x the amount of storage and 14x the number of streams than the IBM POWER7 1TB result.(3)
  • Thumbs up for SPECweb2005 – Funny that Oracle likes this one since there is a definite lack of publishes here anymore and this benchmark is being retired in January.
  • Thumbs up for TPC-C – Even funnier that Oracle likes this one since Sun avoided publishing on this for many, many years claiming this benchmark was too old. Now, as a fair-weather friend, it’s a favorite. IBM has the top nonclustered result (even with an older Power system).(4)
  • Thumbs down for SAP SD – Interesting that this valuable benchmark for SAP users is on Oracle’s hit list. This benchmark is a wonderful measure for SAP users and has many proof points from many vendors. Is that why Oracle tries to avoid this one and instead publishes on other more obscure SAP benchmarks like SD-Parallel and ATO?
  • Thumbs down for SPECjAppServer2004 – Looking to fill up space, since Oracle knows as well as we do that this benchmark was retired almost a year ago. Also ironic because I do seem to remember that Oracle/Sun used to really like this benchmark. Alot.
  • Thumbs down for SPECcpu and SPECjbb – These are nice and easy for integer, floating point, and Java business apps. Surely valuable in their own way.
  • Thumbs down for Stream – Ignoring HPC users are we? We’ve seen this before with the TOP500. IBM has the most systems with 213. Oracle had 12.
  • Keep in mind that Oracle has only published their latest SPARC T4 results on 2 industry standard benchmarks — that’s right, only 2. Which is fine, if you don’t mind steering with your eyes closed.

    My view is that you may want and need to look at different aspects of systems across a wide portfolio of many different types of benchmarks. Which is why IBM is #1 — in over 100.

    ************************************************

    (1)Oracle WebLogic Server 11g and Oracle Database 11g Release 2 with Oracle Real Application Clusters and Oracle Solaris running on a four-node SPARC T4-4 cluster, each system with four SPARC T4 3GHz processors, (128 core app server, 64 core db server), 40,104.86 SPECjEnterprise2010 EjOPS vs. WebSphere Application Server V7 on IBM Power 780 and DB2 on IBM Power 750 Express, (64 core app server, 32 core db server), 16,646.34 SPECjEnterprise2010 EjOPS.
    (2)IBM POWER 780 (8 sockets/32 cores/128 threads) 192,001 QphH@3000GB, $6.37/QphH@3000GB, available 11/30/11.
    (3)SPARC T4-4 server (4 sockets/32 cores/256 threads) 201,487 QphH@1000GB, $4.60/QphH@1000GB, available 10/30/11. IBM POWER 780 Model 9179-MHB server (8 sockets/32 cores/128 threads) 164,747.2 QphH@1000GB, $6.85/QphH@1000GB, available 3/31/11.
    (4)A 64-core IBM Power 595 (5 GHz, 32 chips, 128 threads) with IBM DB2 9.5 is the best overall single system (6,085,166 tpmC, $2.81/tpmC, configuration available 12/10/08).
    Sources: http://www.spec.org, http://www.tpc.org, http://www.top500.org. Results current as of 10/26/11.

    TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

    SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
    logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
    over the world.

    SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

    The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

    technorati tags: , , , , ,,,,,,,,,,,,,

    Written by benchmarkingblog

    October 26, 2011 at 12:25 pm

    eX5 Does Mile High TPC-C

    leave a comment »

    I often get tired during a long hike — but nothing a sit down and a few peanut M&Ms can’t take care of. But a few years ago, at about 9000 feet on the edge of a glacier in Montana, that certainly didn’t do the trick. I would walk about three steps, have to sit on a rock, and then I felt as if I wanted to sleep forever. Instead of the usual pondering of how many more miles did we have and how much higher did we have to go, I started to ask deeper questions like why in the world I was doing this at all. The view of the glacier was tremendous but was it worth the feeling that my body had been taken over by aliens?

    When I think about “highest ever” I now wonder:

    • Is it possible to enjoy hiking in the Rocky Mountains just for the weekend without acclimation?
    • When the Junior Olympics are in Denver, do the athletes who make their homes in higher altitudes have an advantage?
    • When planning a trip, do I now add altitude to the list of things I need to worry about besides weather, direct flights, crowds, and the language I have to communicate in?

     

    But for our “highest ever” this week there are no questions. Just pure celebration. You see, the IBM System x3850 X5 and DB2 just delivered the highest x86-64 performance score ever achieved on the TPC-C benchmark.(1)

    The TPC-C benchmark simulates an order-entry environment of a wholesale supplier — entering and delivering orders, recording payments, checking the status of orders, and monitoring the level of stock at the warehouses. TPC-C represents any industry that must manage, sell, or distribute a product or service. Current results show that clients who deploy IBM technology could see more orders entered, faster monitoring, distribution, and delivery.

    This configuration used several cool newer technologies — IBM eX5 with E7 processors, MAX5 memory expansion, the amazing DB2 9.7, and SSD storage.

    So tell your “highest ever” story here. Keeping it clean, of course.

    ************************************************

    (1) IBM System x®3850 X5 (Intel Xeon E7-8870 processors 2.40GHz, 4 processors/40 cores/80 threads) result of 3,014,684 tpmC, $.59 USD/tpmC, available 9/22/11, DB2 9.7, SUSE Linux® Enterprise Server 11 (SP1),
    Source: http://www.tpc.org. Results current as of 8/3/11.

    TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

    The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

    technorati tags: , , , , ,,,,,,,,,,,,,

    Written by benchmarkingblog

    August 3, 2011 at 3:29 pm

    Posted in DB2, MAX5, SSD, TPC, x3850, X5

    Tagged with , , ,

    How can I make use of benchmarks in my purchasing decision?

    leave a comment »

    How can I make use of benchmarks in my purchasing decision?

  • Where is the best beach to snorkel off of in March?
  • If I choose the Rocky Mountains for my one week vacation, will I get altitude sickness?
  • What about Mongolia?
  • Sometimes choosing a vacation spot can be even more fun than the vacation itself. What do you use for input when you’re planning your time away? Weather statistics, websites, travel books, a recommendation from your brother-in-law — all can be great tools.

    An IT purchasing decision can be just as exciting.

    And one important tool to use in that decision should be benchmarks.

    Three types of benchmarks should be considered: industry standard, vendor, and client benchmarks. Industry standard benchmarks such as TPC and SPEC are especially valuable when considering a hardware or software purchasing decision involving such workloads as online transaction processing, business intelligence, Java, web, and high performance computing. Vendor benchmarks such as those from SAP are extremely useful because they could include the exact type of application that matches your solution. And client benchmarks, where you actually run your home grown applications on a vendor’s configuration, can give you a clarity that you just may not be able to get any other way.

    By analyzing the hardware and software used in these benchmarks and comparing them to your own, you can model anticipated workload on a system you are considering. You can compare one result with another. You can study the impact of an upgrade in software, addition of more processor cores, one system vs. a cluster, a change in an operating system, a different database.

    Benchmark data is not going to give you the absolute answer on what to purchase. Performance, of course, is only one factor out of many to consider in a decision. But oh what a factor it is.

    And if you have an opinion on the White Mountains vs. the Rocky Mountains in early June, please let me know.

    ************************************************
    Required Stuff

    SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

    TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

    SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
    logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
    over the world.

    The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

    technorati tags: , , , , ,,,,,

    Written by benchmarkingblog

    March 11, 2011 at 4:08 pm

    Posted in SAP, SPEC, TPC

    Tagged with , , ,