Elisabeth Stahl on Benchmarking and IT Optimization

Avoid “Jump the Gun” Benchmark Tests

leave a comment »

I was talking with someone the other day and noticed something funny. They were chomping at the bit to do some deep down Java tuning. Let’s make ten changes at once and blow this thing out of the water tuning. What they didn’t yet have was a clue on where they were going and how they would even know if they got there.

Before starting any systems performance testing or benchmarking, here are some of my best practices:

  • First things first, define your benchmark objectives. You need success metrics so you know that you have succeeded. They can be response times, they can be transaction rates, they can be users, they can be anything — as long as they are something.
  • Document your hardware/software architecture. Include device names and specifications for systems, network, storage, applications.
  • Implement just one change variable at a time. (OK, sometimes we can get away with a couple.)
  • Keep a change log — what tests were run, what changes were made, what the results were, what your conclusions were for that specific test.
  • Map your tests to what performance reports you based your conclusions on. Sometimes using codes or special syntax when you name your reports helps.
  • Keep going, don’t give up, you will get there.

Some of this we learned in science class. Some of this is common sense. But you’d be surprised sometimes by how much sense these days is uncommon.


The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

technorati tags: , , , , ,,,,,,,



Written by benchmarkingblog

April 13, 2012 at 11:53 am

Posted in Performance, Uncategorized

Tagged with ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: