benchmarkingblog

Benchmarking and Systems Performance

On Fencing Claims and Real World Benchmarks

leave a comment »

I am so intimately familiar with fencing it’s not funny.

I’ve had épées, foils, and even sabers at my house. I’ve been to Junior Olympics (as a spectator, of course). My washer has seen fencing jackets, knickers, and a various assortment of brightly colored and very sweaty high socks.

I grew up with a wooden hitching post fence in my yard. My current neighbor has a white picket fence that I look at every day. There is a chain link fence in the back so my black lab can’t get into trouble.

But the fencing I really wanted to talk about is the fencing of claims.

OK, so maybe if a metric is not an overall #1 it does make sense to look at it in a slightly different way. I get per core. I get single system. I get #1 for a particular subset of a benchmark suite. But I would say it has just about gone too far.

A claim I saw the other day was fenced not by a piece of hardware, not a particular type of system, not a benchmark category. This claim was a “world record” for a very specific enhancement package of a very specific version of a very specific type of application software. It’s like saying I am the #1 grape picker in the world with purple eyes wearing yellow pants with green stripes on them. Oh, and pink stilettos. Oh and by the way, those grapes are actually raisins.

Enough already with playing these segmentation games.

Many times what really matters most is how your specific workload performs on a specific system.

spedometer

For that I would recommend

 

 

************************************************
The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , ,,,,,,,,,,,,,,,

About these ads

Written by benchmarkingblog

April 15, 2013 at 4:20 pm

Posted in SAP

Tagged with ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: