T4 MIA

I was recently looking to do a comparison of a couple of systems and figured the easiest data to look at would be SPEC CPU data. I mean it’s almost a given that even though these integer and floating point benchmarks can be, should I say, pretty boring (yawn), at least you’ll find what you are looking for because everyone publishes pretty much everything on these.

To my surprise I realized that I did not see the one system I really needed to look at – a SPARC T4 result. And then I started to laugh to myself (it was pretty late).

Of course. T4 is the one where you can find results for some really strange benchmarks but none of the regular ones where you can actually compare anything.

And then I remembered something else. I went out to my blog just to confirm it and there it was staring at me in the face. The #1 all time most popular search term that gets people to read my blog is just that — “sparc t4 benchmark.” Followed closely by “sparc t4.” “sparc t4 specint.” “sparc t4 benchmarks.” “specint t4.” “t4 specint.” And “sparc t4 performance.”

Pretty sad that for this type of basic data you can not find it on the SPEC site. You can not find it on the Gartner site. You can not even find it on the Oracle site. For this type of basic performance data you need to go to an IBM site to find out that this data just doesn’t even exist.

************************************************

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance,SPARC,T4,SPEC,SPECcpu,CPU2006,SPECfp,POWER7+,SPECint,Power,benchmark,benchmarking

Notes from Oracle, Chapter 3 in the Series

Once again, I received a marketing email from Oracle this morning. And once again, hoping that it contained exciting new data on the T5, I was sadly disappointed.

• I can not fathom why they continue to address the email to a bizarre truncation of my first name. I continue to question my confidence in a company that specializes in data.
• The note stated “According to our records, you have older SPARC systems.” I should really check in my garage. Again, ironic for a company that wants clients to use them for data and record keeping.
• The email links to a new video. From April.
• The video claims a humongous performance improvement with SPARC T4. When you watch the video, it’s actually merely a comparison from Oracle’s very own baby M3000s consolidated to T4’s. So I would definitely expect even more than humongous.

I’ve gotten some notes and tweets about how amusing these “Notes from Oracle” are. And we all could certainly use a laugh these days. So please, Mr. Oracle Business Development Manager, keep those cards and letters coming.

************************************************

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance,SPARC,Sun,Oracle,T4,T5,benchmark,11g,SPARC64,M3000,hardware,communication,

benchmarking

Oracle’s Historical Fiction

I love reading historical fiction. It’s somewhat like the real thing, but not quite. And definitely more exciting.

This morning I received another email from Oracle that reminded me of just that.

• The note was once again addressed to “Elis”. I can not fathom why they continue to truncate my first name to only four letters. Doesn’t give me much confidence in a software company that specializes in data.
• The note contained a quote from an unnamed “Infrastructure Architect” from an anonymous “Major Credit Card” company on Oracle/Sun benefits. I’m thinking that just maybe that architect didn’t really want to be known throughout history for that one.
• The note stated “Our records show that you have older SPARC systems.” How funny, maybe they’re in my basement hidden behind the washing machine. Again, a data company that surely got that historical data wrong.
• The note links to an independent report on customer perspectives. The report is from May. May 2011. A lifetime ago in the IT world.
• The independent report is full of various quotes, all from anonymous people at anonymous companies.
• This independent report is written by a company called ORC. Hmmm, have I seen those letters somewhere before?

************************************************

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance,SPARC,Sun,Oracle,T4,Supercluster,benchmark,11g,ORC,software,hardware,

communication,benchmarking

You’ve Got Mail, Weird Mail from Oracle

This morning I received an email from Oracle. Not surprising since I signed up to get some of their communications a few years ago. What surprised me is what the email contained.

• First of all, the email was addressed to “Elis”. Now I have gone by many nicknames in my past, but never “Elis”. I can only guess that for some reason Oracle’s programs can not handle the 9 characters in my first name and truncate to 4. Doesn’t give me much confidence in a software company that specializes in data.
• The second part of the email to catch my eye was the picture displayed in the center. Not a cool logo, not a detailed shot of the hardware, but a picture of Larry talking.
• Finally, and here is the clincher, the email focused on an announcement of a new system. At first I got all excited, what could this one possibly be. And then I read the details. The announcement cited in the letter dated today was actually from back in September 2011, almost 1 full year ago. And I have to ask, is that the best you can do Oracle, is there not anything else more recent than almost a year ago worth having me look at?

 

After I read the note, I decided not to delete it because I enjoyed it so much. And I had to ask myself — Was this even a true communication from Oracle or someone spamming a little fun here?

************************************************

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance,SPARC,Sun,Oracle,T4,Supercluster,benchmark,11g,Larry,

software,hardware,communication,benchmarking

I Want to Buy a Zoo, Not an Oracle System

Most people don’t think about going to the zoo in the middle of winter. But it’s actually the very best time to go.

No crowds, the zebra fried oreo shacks are closed, and the animals are at their very best. And, after all, isn’t that what it’s really all about?

Last week, I had a spectacular visit. The polar bears were playing with dead Christmas trees, the seals and sea lions were playing with their rubber balls, and the grizzly cubs were playing with themselves.

And then there’s the Rainforest. Imagine stepping from a cold snowy Cleveland day into a zoological tropical paradise. Sort of like the feeling you get when the plane doors open, you get your carry-on from the overhead, and you step off in Miami. My rainforest favorites are the tropical monkeys, the river otters, and, of course, the anteaters. But alas, last week, the anteaters were still, lying on their sides, hiding behind a crop of rocks.

And that’s how I’ve been feeling lately about Oracle.

See what’s hiding behind the Oracle SPARC SuperCluster and Exadata systems. “Must buy” storage server software.

See what’s hiding behind that pricing in the Oracle benchmark. Artificially low support costs.

See what’s hiding behind Oracle’s “Itanium roadmap.” An investigation into Oracle’s “potentially abusive” practices.

What else do YOU think Oracle is hiding? All thoughts welcome.

************************************************

Oracle SPARC T4-4 server (4 sockets/32 cores/256 threads) 205,792 QphH@3000GB, $4.10/QphH@3000GB, available 5/31/12.
Source: http://www.tpc.org. Results current as of 1/11/12.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance, Sun, SPARC,TPC-H,Oracle,11g,SuperCluster,T4-4,benchmark,Itanium,Autorite,HP,pricing,memory,Exadata,benchmarking

All I Want for Christmas is a Good T4 Benchmark

I read an article the other day about a Santa school where aspiring Santas go to learn about how to be Santa Claus. Important concepts like how to comb your beard, how to make that amazing Ho Ho Ho, and where to properly keep your hands when a kid is on your lap.

In this economy one of the new lessons that the Santa school offers is on how to reduce kids’ expectations. Santa’s workshop has too many toys to produce Johnny, Santa can’t fit everything into his sack, a bunch of elves got sick.

Oracle announced a new TPC-H BI 3TB benchmark result today on the Oracle SPARC T4-4 server. Expectations surely need to be reduced on this one because here’s what you need to know:

• Oracle once again used an embarrassing amount of storage to run this benchmark — almost three times the amount of storage that IBM used.(1)
• Oracle once again used 128 query streams for this benchmark compared with IBM’s 8. TPC-H has a limited number of query variations; so when you run a lot of streams, you have a high probability that the same queries will be requested more than once. Oracle is greatly increasing the probability that they will have the results of the queries stored in their cache — which may not be representative of how their product would perform in a truly ad hoc query environment.
• Oracle once again published a benchmark where the configuration isn’t even available — until May 31, 2012.
• Oracle once again included extremely minimal support in their pricing. Does $2300 a year sound like what you are paying for software “incident server support” . . . ?

 

 

The most important lesson for Santas — Never promise anything.

************************************************

(1)IBM POWER 780 (8 sockets/32 cores/128 threads) 192,001 QphH@3000GB, $6.37/QphH@3000GB, available 11/30/11, 15,610 GB storage. Oracle SPARC T4-4 server (4 sockets/32 cores/256 threads) 205,792 QphH@3000GB, $4.10/QphH@3000GB, available 5/31/12, 45,600 GB storage.
Source: http://www.tpc.org. Results current as of 11/30/11.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance, Sun, 3TB,TPC-H,SPARC,Oracle,T4,POWER7,benchmark,Solaris,11g,Java,Supercluster,T4-4,BI,benchmarking

Is There a “Right” and “Wrong” with Benchmarks?

Like many of our politicians lately, Oracle has been proselytizing — in this case that there are certain benchmarks which are right and certain benchmarks which are wrong.

In their evaluation Oracle claims to focus on all the right benchmarks while claiming that everyone else is focusing on the wrong ones. Let’s take a look at their reviews:

Thumbs up for SPECjEnterprise – Yes, I wholeheartedly agree, a good benchmark to use for Java applications. And that’s why IBM has outstanding results on this benchmark. Oracle’s T4 result needed four times the number of app nodes, twice the number of cores, almost four times the amount of memory and significantly more storage than the IBM POWER7 result.(1)

Thumbs up for Oracle ebusiness, JDEdwards, PeopleSoft, Siebel – Good when you need to model these exact applications but keep in mind that Oracle owns these “independent” benchmarks. Even with a stacked deck, IBM has #1’s here.

Thumbs up for TPC-H – Yes, totally agree that this benchmark is valuable for business intelligence applications. Which is why IBM just published a leadership 3TB TPC-H result.(2) And Oracle’s T4 result needed 2.7x the amount of storage and 14x the number of streams than the IBM POWER7 1TB result.(3)

Thumbs up for SPECweb2005 – Funny that Oracle likes this one since there is a definite lack of publishes here anymore and this benchmark is being retired in January.

Thumbs up for TPC-C – Even funnier that Oracle likes this one since Sun avoided publishing on this for many, many years claiming this benchmark was too old. Now, as a fair-weather friend, it’s a favorite. IBM has the top nonclustered result (even with an older Power system).(4)

Thumbs down for SAP SD – Interesting that this valuable benchmark for SAP users is on Oracle’s hit list. This benchmark is a wonderful measure for SAP users and has many proof points from many vendors. Is that why Oracle tries to avoid this one and instead publishes on other more obscure SAP benchmarks like SD-Parallel and ATO?

Thumbs down for SPECjAppServer2004 – Looking to fill up space, since Oracle knows as well as we do that this benchmark was retired almost a year ago. Also ironic because I do seem to remember that Oracle/Sun used to really like this benchmark. Alot.

Thumbs down for SPECcpu and SPECjbb – These are nice and easy for integer, floating point, and Java business apps. Surely valuable in their own way.

Thumbs down for Stream – Ignoring HPC users are we? We’ve seen this before with the TOP500. IBM has the most systems with 213. Oracle had 12.

Keep in mind that Oracle has only published their latest SPARC T4 results on 2 industry standard benchmarks — that’s right, only 2. Which is fine, if you don’t mind steering with your eyes closed.

My view is that you may want and need to look at different aspects of systems across a wide portfolio of many different types of benchmarks. Which is why IBM is #1 — in over 100.

************************************************

(1)Oracle WebLogic Server 11g and Oracle Database 11g Release 2 with Oracle Real Application Clusters and Oracle Solaris running on a four-node SPARC T4-4 cluster, each system with four SPARC T4 3GHz processors, (128 core app server, 64 core db server), 40,104.86 SPECjEnterprise2010 EjOPS vs. WebSphere Application Server V7 on IBM Power 780 and DB2 on IBM Power 750 Express, (64 core app server, 32 core db server), 16,646.34 SPECjEnterprise2010 EjOPS.
(2)IBM POWER 780 (8 sockets/32 cores/128 threads) 192,001 QphH@3000GB, $6.37/QphH@3000GB, available 11/30/11.
(3)SPARC T4-4 server (4 sockets/32 cores/256 threads) 201,487 QphH@1000GB, $4.60/QphH@1000GB, available 10/30/11. IBM POWER 780 Model 9179-MHB server (8 sockets/32 cores/128 threads) 164,747.2 QphH@1000GB, $6.85/QphH@1000GB, available 3/31/11.
(4)A 64-core IBM Power 595 (5 GHz, 32 chips, 128 threads) with IBM DB2 9.5 is the best overall single system (6,085,166 tpmC, $2.81/tpmC, configuration available 12/10/08).
Sources: http://www.spec.org, http://www.tpc.org, http://www.top500.org. Results current as of 10/26/11.

TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

SAP, mySAP and other SAP product and service names mentioned herein as well as their respective
logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all
over the world.

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance, BI, TPC,TPC-H,SPARC,Oracle,T4,POWER7,benchmark,TPC-C,SPEC,Stream,SAP,T4-4,Java,benchmarking

Hey Oracle, TPC-H Shouldn’t be a Cakewalk

Here in the northern hemisphere, tis the season of pumpkins and Fall Festivals. One of my favorite festival events just has to be the Cakewalk. Where can you play a game, win an amazing cake, and ensure a great dessert without even having to turn the oven on ?

One of the ways that I’ve made sure to win one of the cakes is to buy a lot of tickets. If I get my friends to play on my behalf, I’ve really ensured that the probability that I’ll win is high. Disclosure: I do feel guilty when I do this and a little kid doesn’t win.

And that’s what we are seeing with Oracle TPC-H results. Oracle’s latest BI benchmark result was announced last week on the new SPARC T4-4. Besides the high load time, unrealistically low maintenance coverage, and huge amount of storage needed, take a look at the queries run. Two facts stand out.

The query execution times are incredibly variable — for the same exact query. As an example, Query 6 had a maximum of 4140.2 — and a minimum of only 5.1 !

The number of streams of queries was 128. Oracle ran 128 times, IBM only 9. (1)

What does this mean? My colleague Juan asked me this just yesterday.

TPC-H has a limited number of query variations; so when you run a lot of streams, you have a high probability that the same queries will be requested more than once. Oracle is greatly increasing the probability that they will have the results of the queries stored in their cache — which may not be representative of how their product would perform in a truly ad hoc query environment.

Getting results this way, it’s a piece of cake.

************************************************

(1)SPARC T4-4 server (4 sockets/32 cores/256 threads) 201,487 QphH@1000GB, $4.60/QphH@1000GB, available 10/30/11. IBM POWER 780 Model 9179-MHB server (8 sockets/32 cores/128 threads) 164,747.2 QphH@1000GB, $6.85/QphH@1000GB, available 3/31/11.
Sources: http://www.spec.org, http://www.tpc.org. Results current as of 10/5/11.

TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance, BI, TPC,TPC-H,SPARC,Oracle,T4,POWER7,benchmark,Solaris,11g,11g,Supercluster,T4-4,cache,benchmarking

Gaga over POWER7

When you do something provocative, like wear a dress made of meat or advertise on the front page of the Wall Street Journal, you better know what you are doing.

Oracle today ran an ad that claims that the new SPARC Supercluster runs Oracle and Java twice as fast as the IBM Power 795. Let’s take a look at this claim.

None, that’s right none, of the benchmarks announced with SPARC T4 were run with the Supercluster configuration. So there really are no real data points for this Oracle system.

The footnote for the ad states that the claim is based on Oracle internal testing plus some sort of available information about the Power 795. Pricing is certainly not substantiated. How can you even do the math?

The IBM Power 795 is #1 in real world industry standard published benchmarks. As an example, real data shows that the IBM Power 795 is 3.9 times the performance of the Oracle/Sun M9000 for the SAP SD 2-tier benchmark, which we do have data for.(1)

If you’re going to be on the front page of a newspaper or have three heads on the cover of a magazine, make sure you know what you are doing.

************************************************

(1) IBM Power 795 on two-tier SAP SD standard application benchmark running SAP enhancement package 4 for the SAP ERP 6.0 application (Unicode): 32 processors / 256 cores / 1,024 threads, POWER7, 4.0 GHz, 126,063 SAP SD benchmark users, 0.98 seconds dialog response time, 96 percent CPU utilization running AIX® 7.1 and DB2® 9.7. Certification #2010046 vs. SUN M9000 on the two-tier SAP SD standard application benchmark running SAP enhancement package 4 for SAP ERP 6.0 (Unicode): 64 processors / 256 cores / 512 threads, 32,000 SAP SD benchmark users, SPARC64 VII, 2.88 GHz, Solaris 10, Oracle 10g , cert# 2009046
Sources: http://www.sap.com/benchmark. Results current as of 9/29/11.

SAP, mySAP and other SAP product and service names mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world.

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance, 795, SPECjEnterprise,TPC-H,SPARC,Oracle,T4,POWER7,benchmark,Solaris,11g,Java,Supercluster,T4-4,BI,benchmarking

SPARC T4 to the core

Yesterday I went apple picking in rural Ohio. That makes sense.

It’s not something that most people associate with California even though California is actually one of the top apple-producing states. But it works rather well for this SPARC analysis.

I usually love apple picking – with the doomed sun of autumn, the crunchy sweetness of the fruit, the dog wolfing down the cores. But there were certain aspects of my trip yesterday that were plainly unimpressive.

Sort of like the latest SPARC T4 benchmark results announced by Oracle today:

Oracle claimed nine T4 world records. 7 of the 9 are not industry standard benchmarks but Oracle’s own benchmarks, most based on internal testing. Sort of like when we called the orchard and they said that many varieties were available for picking. When we got there, only a few could really be picked. Where was that renowned low hanging fruit?

Some Oracle claims compared the new T4 results with previous benchmark versions, never a good idea. Like encouraging your kids to climb on the fruit-bearing trees. Some results compared Oracle to Oracle. If you read carefully, some didn’t compare to anything.

Oracle claimed a “generational increase in performance” over previous versions. Note that this claim (which has no published benchmarks behind it) focuses on single threaded applications – how many of those do you have? And you can easily get a 5x improvement when you start from a very very small seed.

Oracle’s SPECjEnterprise2010 Java T4 benchmark result, which was highlighted, needed four times the number of app nodes, twice the number of cores, almost four times the amount of memory and significantly more storage than the IBM POWER7 result.(1) Oracle’s price performance and space metric claims (which are not even official benchmark metrics) were calculated only for the application tier of this benchmark, basically ignoring the all important database server, software and storage. Sort of like eating only the pulp of the apple and ignoring all the vitamins in the skin.

Oracle’s T4 TPC-H 1TB BI benchmark result, another benchmark which was highlighted, actually had a longer load time than the IBM result from last year. Oracle’s storage use was ludicrous, like the number of apples my Labrador ended up eating; Oracle’s total storage needed to the database size ratio was 10.80 compared to the IBM value of 3.97. Oracle needed 128 streams of queries, IBM only 9. And make sure to note the extremely low and unrealistic Oracle maintenance costs used to get to the price performance number.(2)

The range and results of these benchmarks are ultimately disappointing. Instead of making a wonderful pie and apple rings last night, we swept up chips of dried orchard mud in the dark.

************************************************

(1)Oracle WebLogic Server 11g and Oracle Database 11g Release 2 with Oracle Real Application Clusters and Oracle Solaris running on a four-node SPARC T4-4 cluster, each system with four SPARC T4 3GHz processors, (128 core app server, 64 core db server), 40,104.86 SPECjEnterprise2010 EjOPS vs. WebSphere Application Server V7 on IBM Power 780 and DB2 on IBM Power 750 Express, (64 core app server, 32 core db server), 16,646.34 SPECjEnterprise2010 EjOPS.
(2)SPARC T4-4 server (4 sockets/32 cores/256 threads) 201,487 QphH@1000GB, $4.60/QphH@1000GB, available 10/30/11. IBM POWER 780 Model 9179-MHB server (8 sockets/32 cores/128 threads) 164,747.2 QphH@1000GB, $6.85/QphH@1000GB, available 3/31/11.
Sources: http://www.spec.org, http://www.tpc.org. Results current as of 9/26/11.

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: IBM, systems, performance, SPEC, SPECjEnterprise,TPC-H,SPARC,Oracle,T4,POWER7,benchmark,Solaris,11g,Java,Supercluster,T4-4,BI,benchmarking