benchmarkingblog

Elisabeth Stahl on Benchmarking and IT Optimization

Posts Tagged ‘SPARC

Oracle Meets That ’70s Show

leave a comment »

Last week I made the annual spring break pilgrimage to my childhood home in the shadows of the cherry blossoms.

What always strikes me when I visit — and you’ve probably had the same experience — is how nothing, almost nothing, has changed since I lived there four decades ago. Yes, there’s a huge TV with cable now. And a cell phone, though not so smart yet. And an iPad that always needs something done to it. But other than these few new features, the general layout and beauty of the interior is essentially the same.

Which I love. Why get new kitchen cabinets when you can take the beautiful solid wood ones and have them refinished? Why buy new cheap chairs when 50’s Danish Modern is built so well and gorgeous to boot?

But one of the best examples of this retro environment, hands down, has to be the downstairs bathroom. When entering you are transported to the time of Nixon and Sonny and Cher. The colors are tremendous – bright bright yellows and oranges. Big plaid wallpaper. And wicker accessories. A 70’s dream of a bathroom. And you know what — it still looks great. The glamour of everything from the 70’s has returned in full force in this one tiny room.

But some things are not meant to come back. And that includes the way some vendors compare systems and benchmarks.

I recently saw a comparison from Oracle comparing the SPARC T7-1 vs. the IBM Power System S824. It brought me right back to when I started blogging almost ten years ago, when we were all inundated with benchmark flaws. Let’s take a look at some of the details :

  • The tool Oracle used to compare the systems is NOT an industry standard benchmark audited by a third party. It is a tool that can be used by anyone. Oracle ran all tests themselves.
  • The tool used is adapted from the TPC-C benchmark, which Oracle themselves has stated in the past that they feel is dated.
  • The disks used in the systems compared are not the same – HDD vs. SAS.
  • The logs and database files for the IBM test were not run on the IBM system – they were run on a different Oracle system.
  • Solaris 11.3 was used for the logs and database file systems on the Oracle side; Solaris 11.2 was used for the IBM configuration.

 

A photo of my childhood downstairs bathroom was Instagrammed recently. It received 35 likes, over half of them from students at the best design school in the country. That makes sense. Oracle’s benchmark comparisons don’t.

 

************************************************

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,,,

Advertisements

Written by benchmarkingblog

March 23, 2016 at 10:07 am

Posted in Oracle, POWER8, SPARC

Tagged with , ,

Back in Time with Oracle

with one comment

Some of you may know that this week was a very big one for “Back to the Future” movie fans. On Wednesday, Oct. 21, 2015, at 4:29 p.m., our today caught up to the tomorrow depicted in “Back to the Future, Part II.” In that 1989 film, a DeLorean time machine appears from 30 years in the past.

To those who love time travel, this is a really big deal. Some towns even went so far as to rename themselves to the featured city in the film. Ceremonies worldwide were performed at exactly 4:29PM.

And this reminded me of a benchmark result that was just published today by Oracle on the SAP SD benchmark.

As we move into newer digital workloads, some of the older industry benchmarks have gone by the wayside. Many of us have spent a lot of time analyzing these newer workloads and developing new metrics for them. But one classic benchmark is still extremely appropriate for many of today’s applications – and that is the suite of SAP benchmarks.

But this new Oracle result just published is clearly dated — even though it is a brand new result on a brand new Oracle SPARC system. The IBM Power Systems result with DB2 from over 1 year ago is over 2X better performance per core than this new Oracle SPARC result. (1)

What’s really exciting, unlike this new benchmark result, is that many of the predictions of the future in the “Back to the Future” movie were right on. But I am still waiting for the dog-walking drone.

************************************************

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

(1)IBM Power Enterprise System E870 on the two-tier SAP SD standard application benchmark running SAP enhancement package 5 for the SAP ERP 6.0 application; 8 processors / 80 cores / 640 threads, POWER8; 4.19GHz, 2048 GB memory, 79,750 SD benchmark users, running AIX® 7.1 and DB2® 10.5, dialog response: 0.97 seconds, order line items/hour: 8,722,000, dialog steps/hour: 26,166,000, SAPS: 436,100, Database response time (dialog/update): 0.013 sec / 0.026 sec, CPU utilization: 99%, Cert #2014034 vs. Oracle SPARC T7-2 result of 30,800 users, Average dialog response time: 0.96 seconds, Fully processed order line items/hour: 3,372,000, Dialog steps/hour: 10,116,000, SAPS: 168,600, Average database request time (dialog/update):0.022 sec / 0.047 sec, CPU utilization of central server:98%, Operating system, central server: Solaris 11, RDBMS: Oracle 12c, SAP Business Suite software:SAP enhancement package 5 for SAP ERP 6.0, Certification number: #2015050, SPARC T7-2, 2 processors / 64 cores / 512 threads,SPARC M7 4.133 GHz, 16 KB (D) and 16 KB (I) L1 cache per core, 256 KB (D) L2 cache per 2 cores and 256KB (I) per 4 cores, 64 MB L3 cache per processor, 1024 GB main memory

SAP and all SAP logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries. All other product and service names mentioned are the trademarks of their respective companies.

technorati tags: , , ,,,,,,,,,,,

Written by benchmarkingblog

October 23, 2015 at 12:59 pm

Posted in Oracle, SAP

Tagged with , , , , , ,

Oracle’s SPARC Enhancements: Construction or Wind ?

leave a comment »

Two nights ago I spent a lovely 6 hours in the airport. Flight cancelled, next plane delayed for incoming aircraft, no runways to be had in one of the largest airports in the country. Announcement 1: There was only one runway because the others were under construction. Announcement 2: There was only one runway that could be used because the wind patterns were strange.

All you want is to get home to your couch and your dog. At the same time it would be great to get the real story on what is happening. Just because you want to know, you want it to make sense.

And that’s exactly how I was feeling again as I read one of Oracle’s recent press releases on the Fujitsu SPARC M10 “enhancements.” The claim was for “15 world records.” I decided to take a look at each one just to know — was it the construction or the wind ?

1. Oracle needed 2.5x more cores/memory than IBM. The IBM result was from 4 years ago.
2. Oracle needed 2x more cores/memory than IBM. The IBM result was from 4 years ago.
3. Oracle compared themselves with themselves.
4. Oracle compared themselves with themselves.
5. Oracle needed 2x more cores than SGI.
6. Oracle compared themselves with themselves.
7. Oracle needed 2x more cores than IBM.
8. Oracle compared themselves with themselves.
9. Oracle needed 4x more cores than IBM.
10. Oracle compared themselves with themselves.
11. Oracle picked on little x86.
12. Oracle compared themselves with themselves.
13. Oracle needed 16x more cores than IBM. The IBM result was from 6 years ago.
14. Oracle needed 8x more cores than IBM. The IBM result was from 6 years ago.
15. Oracle needed 8x more cores than IBM. The IBM result was from 6 years ago.

Also note that there are really only 4 different benchmarks here. And notably all but 2 of these 15 are in the Technical Computing space, using simple component type benchmarks.

So that’s the real story. The other real story is that if I had driven the 500 miles I would have been home much faster.

************************************************

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,

,

Written by benchmarkingblog

April 11, 2014 at 2:49 pm

Posted in SPARC

Tagged with , , ,

Come on Oracle, Get “With It” Benchmarking

leave a comment »

I admit that many weekends this time of year you will find me (when I’m not enslaved by the leaf blower) curled up with a good old book on the old couch with my thankfully not so old dog.

But this weekend I truly was “with it.”

On Saturday night I attended one of the most sought-after sold out concert events ever to hit this town. I got to see a Pink concert that included not only 17 of her best songs but Pink flying through the air doing acrobatics that you simply would not believe. A rock concert rolled right into the circus, truly amazing.

And then to top it all off, on Sunday night I attended one of the most sought-after sold out movie events of the year. I got to see the latest Hunger Games, Catching Fire, second in this awesome trilogy which could be even more popular than the first.

So on Monday when I saw the latest Oracle SPARC T5-4 benchmark result on the TPC-H decision support benchmark (1), all I could think was how so “not with it.”

Like Gangnam Style this year. Or What Does the Fox Say this month.

Hey, I’m the first one to like legacy. My closet is filled with vintage looks. I love retro — just not when it comes to benchmarks.

Here is what you need to know.

  • First of all, this is TPC-H. Yawn. We’re ready for something new here.
  • Most of the TPC-H results are grayed out in this category, considered “historic.” This result is right next to a result from IBM — from 2007 (yes you heard that right).
  • Total Storage to Database Size ratio is a massive 60.80. Talk about overkill on storage to achieve performance. This number is many many times the ratio we’ve seen from other results.
  • Load time is a whopping 9.63 hours.
  • 128 query streams are needed. Most results use many, many fewer. That’s because TPC-H has a limited number of query variations; so when you run a lot of streams, you have a high probability that the same queries will be requested more than once. Oracle is greatly increasing the probability that they will have the results of the queries stored in their cache — which may not be representative of how their product would perform in a truly ad hoc query environment.
  • Oracle once again included extremely minimal support in their pricing. Does $2300 a year sound like what you are paying for software support?

************************************************
(1) Oracle TPC-H of 377,594 QphH@10000GB,$4.65 per QphH,Availability 11/25/13,Oracle Database 11g R2 Enterprise Edition w/Partitioning,SPARC T5 3.6 GHz; Total # of Processors: 4,Total # of Cores: 64,Total # of Threads: 512.
Source: http://www.tpc.org. Results current as of 11/25/13.

TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,,,

,,

Written by benchmarkingblog

November 26, 2013 at 11:28 am

Posted in SPARC T5, TPC-H

Tagged with , ,

Born to Run Benchmarks

with 9 comments

With apologies to Bruce, you can’t start a fire with a SPARC. A fire of proof points, that is.

In two different instances Oracle’s recent announcements on SPARC benchmark data have been lacking — and certainly couldn’t start any flame of passion at OpenWorld.

The first involved the announcement of the SPARC M6-32 server and engineered system. The press release only had a footnote for “estimated” performance of some mysterious sort. Oracle’s benchmark website actually discussed some benchmarks for this new system — but 1) there was no competitive information and 2) they were on Oracle’s very own benchmarks.

In the second case, the SPARC T5-8 was highlighted on the Java end-to-end SPECjEnterprise2010 benchmark. A record was claimed — in actuality, the IBM Power 780 had 19% greater overall performance and 49% greater application server performance per core than the Oracle system.(1)

Additionally, keep in mind that whenever costs are presented in Oracle’s comparisons, they need to be scrutinized to the highest degree. What storage is included, what software is included, what support and maintenance is included? Is an apple being compared to a pineapple?

(P.S. After I wrote this I discovered that today is actually Bruce Springsteen’s birthday. How weird is that?)

************************************************

(1)SPECjEnterprise2010 result of 36,571.36 on 1 x SPARC T5-8 (8 chips, 128 cores, 3.6 GHz SPARC T5);Oracle WebLogic 12c (12.1.2);Oracle Database 12c (12.1.0.1) vs. IBM result of 10,902.30 on 1 x IBM Power 780(8 chips, 32 cores, 4.42 GHz POWER7+);WebSphere Application Server V8.5;IBM DB2 Universal Database 10.1; Source: http://www.spec.org. Results as of 9/23/13.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,

,,,,,,

Written by benchmarkingblog

September 23, 2013 at 9:22 pm

The National Security on the T5-4 and Big Data

with 5 comments

There’s been a lot of talk the last few days on Big Data and when it’s “right” to capture and use it. Some say it’s a real invasion of privacy. Others realistically point out that it is the best way to counter terrorism.

Whichever you believe, the important thing is that Big Data is being discussed not just in geeky meetings with IT managers but by everybody. When your neighbor across the street stops trimming his tree branches just to talk to you about it, you know it’s hot stuff.

So I was particularly interested to see that Oracle just published a new TPC-H data benchmark result on the SPARC T5-4.

And here is what hits you like a train.

  • Why is this publish at only the 3TB size when all the talk these days is on much larger amounts of data?
  • Why is the Total Storage to Database Size ratio a whopping 29? Talk about overkill on storage to achieve performance. This number is many times the ratio we’ve seen from other results.
  • Why is the memory to database size % a whopping 66.6? Again, much more than you should need and what we normally see.
  • Why are there 192 query streams needed? Most results use many, many fewer. That’s because TPC-H has a limited number of query variations; so when you run a lot of streams, you have a high probability that the same queries will be requested more than once. Oracle is greatly increasing the probability that they will have the results of the queries stored in their cache — which may not be representative of how their product would perform in a truly ad hoc query environment.
  • Why isn’t the configuration available now? Because key elements of the storage are not ready.
  • Why did Oracle once again include extremely minimal support in their pricing? Does $2300 a year sound like what you are paying for software “incident server support” . . . ? You don’t even need to answer this one.

Comments are welcome at your own risk.

************************************************
(1) Oracle TPC-H of 409,721 QphH@3000GB,$3.94 per QphH,Availability 09/24/13,Oracle Database 11g R2 Enterprise Edition w/Partitioning,SPARC T5 3.6 GHz; Total # of Processors: 4,Total # of Cores: 64,Total # of Threads: 512.
Source: http://www.tpc.org. Results current as of 6/12/13.
TPC-C ,TPC-H, and TPC-E are trademarks of the Transaction Performance Processing Council (TPPC).

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,,,,

,

Written by benchmarkingblog

June 12, 2013 at 3:36 pm

Posted in SPARC T5, TPC-H

Tagged with , , ,

Shoe Fetish or Benchmark Comparison ?

with 5 comments

Last month I visited the Fashion Institute of Technology’s new exhibit “Shoe Obsession.” And for anyone who relishes shoes, this was the place to be. You enter the dark rooms and the glass cases are absolutely glowing in light, highlighting the SHOES. There’s Manolo Blahnik, Christian Louboutin, Prada and many more, as far as the eye can see. Each shoe is made out of a huge array of materials — plastics, metals, beads, ribbons, velvet, even mirrors. Many have 6 inch heels. Or even higher. Gorgeous.

But of course most of these shoes you could never even wear — and not because there’s only one of them. These shoes don’t even make sense as shoes. What ultimately matters is that you can’t do what you need to do with shoes which is walk in them.

Many times I see benchmark comparisons that don’t really focus on the right things as well. Here’s why in comparisons of systems, cores ultimately matter:

  • Cores are the processing units for computation.
  • Cores are used to charge for software licensing.
  • Cores represent a more apples-to-apples method of comparing systems of varying technologies.
  • The right Cores enable efficient virtualization and consolidation which ultimately leads to better total cost of ownership.

Interesting that when these facts are so clear that Oracle’s newest ad on the front page of the Wall Street Journal totally ignores processor cores and many other important components in the comparisons. As you look at the SPECjEnterprise2010 comparisons, here is what you need to know:

  • The IBM benchmark result is from 2012, the Oracle result is brand new. As we know, this is a lifetime of difference for benchmarking.
  • Oracle needed 4x the number of processing cores and 3x the amount of memory than IBM for this benchmark. See all the details here and here.
  • The IBM POWER7+ Power 780 actually has over 1.5x more performance per core than the Oracle SPARC T5 system.(1)
  • Cost is not even a metric of this benchmark. And note that server cost does not include storage and the all expensive software licensing costs, which by the way, are calculated per core.

 

I like shoes and benchmark comparisons which make sense. Give me my New Balance any day. I can walk for miles in them, they look good, and their TCO screams.

Bottom line: Oracle’s latest comparative advertisement targeting IBM Power Systems, like so many before them, strains credulity. Caveat emptor.

************************************************

(1)SPARC T5-8 (8-chip, 128 cores), 27,843.57 SPECjEnterprise2010 EjOPS; IBM Power 780 (8-chips, 32 cores), 10,902.30 SPECjEnterprise2010 EjOPS. Sources: http://www.spec.org. Results current as of 5/23/13.
SPEC and the benchmark name SPECjEnterprise are registered trademarks of the Standard Performance Evaluation Corporation.

The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,,,

,,

Written by benchmarkingblog

May 23, 2013 at 11:45 am

%d bloggers like this: