Guido Krüger's Web Service

SpeedTimer

Problem Description

"Java programs generally suffer from bad performance", many people complain.

Although this is often true, it's not usually the fault of the language. Instead, performance depends a lot on the programming style and kind of language elements the author choosed. Little is known about the performance characteristics of simple language statements. For example, most programmers know that StringBuffers are generally faster than Strings. But do they also know why this is the case and how much they are faster?

For example, consider the following code statements:

Version 1 Version 2

String s = ""; for (int i = 0; i < 100; ++i) { s += " "; }

StringBuffer sb = new StringBuffer(110); for (int i = 0; i < 100; ++i) { sb.append(" "); } String s = sb.toString();

Version 1	Version 2
String s = ""; for (int i = 0; i < 100; ++i) { s += " "; }	StringBuffer sb = new StringBuffer(110); for (int i = 0; i < 100; ++i) { sb.append(" "); } String s = sb.toString();

The following questions immediately appear:

How long might a certain system need to carry out each of these statement sequences?
Is the second one really so much faster?
If yes, what's the actual speed difference between both of them?

The `SpeedTimer` and `SpeedTimerGenerator` classes

Fortunately, there are ways to measure these kinds of speed timings using standard language features:

Measure the hardware timer resolution.
Measure the overhead of empty loop iteration.
Measure the speed of the code by successively running it in a loop until the elapsed time is long enough to give meaningful results.

The SpeedTimer class of the gk.app.speed package automatically carries out steps #1 and #2 and provides a template for step #3. It's not easy to fully automate step #3 since we have to keep out the influence of method call and loop iteration overhead and we have to make sure that there are enough iterations (compared to the timer resolution of the machine the test in running on) to give meaningful results.

To ease step #3, the gk.app.speed package also includes a second class SpeedTimerGenerator which could be used to create a fully operational test program to time given code sequences. The SpeedTimerGenerator program is called with a text file name as argument. It then creates a new SpeedTimerClient.java file in the current directory which is derived from the SpeedTimer class. The contents of the class is based on the text file given as argument. It must have the following format:

codeline
codeline
...
---
codeline
codeline
...
---
...

For each consecutive codeline sequence, the generator creates a method with the given code running inside a timing loop. In the main method, it also creates code to call the generated methods and to write the timing results to standard output. The resulting SpeedTimerClient class can be started as a Java application and now performs all the measurements automatically.

Example

We will now show an example which actually measures the runtime performance of the statements given above.

Install and compile the neccessary files as mentioned in the Usage notes section.
Create an empty directory and move to that directory.

Create a text file stsource.txt with the following contents:

String s = "";
for (int i = 0; i < 100; ++i) {
  s += " ";
}
---
StringBuffer sb = new StringBuffer(110);
for (int i = 0; i < 100; ++i) {
  sb.append(" ");
}
String s = sb.toString();

Use the SpeedTimerGenerator application to create a SpeedTimerClient application from the stsource.txt file:
```
java gk.app.speed.SpeedTimerGenerator stsource.txt
```
Compile the newly created .java-File:
```
javac SpeedTimerClient.java
```
Run the SpeedTimerClient application:
```
java SpeedTimerClient
```
It now performs the calibrations and measures the running times of both statement sequences. The output of the program (on my computer) is:
```
resolution is 55.0 ms.
time per test is 2750.0 ms.
calibrated loop overhead is 16.2 ns.
-----------
String s = "";
for (int i = 0; i < 100; ++i) {
  s += " ";
}
(1.4 ms.)
-----------
StringBuffer sb = new StringBuffer(110);
for (int i = 0; i < 100; ++i) {
  sb.append(" ");
}
String s = sb.toString();
(93.7 us.)
```
This means the timer resolution is 1/18.2 s. (Windows 95) and the minimum time used to carry out each test is 2750 ms. (currently 50 times the timer resolution). A single loop iteration needs 16.2 ns. (Pentium II with 266 MHz, JDK 1.2Beta4, JIT enabled). The timing results clearly state that the StringBuffer sequence is more than 10 times faster than the String sequence. We also know the absolute running times of both code sequences.

Another way to time statements

If it's not possible to simply write the statements into a consecutive number of code lines, the timing section could be written by hand. The SpeedTimer class provides an example:

/**
 * Measures the time which is needed to run a particular statement
 * sequence. The return value is in milliseconds and is retrieved by
 * successively calling the statements in a growing loop until the
 * elapsed time (retrieved by System.currentTimeMillis()) is long
 * enough to give meaningful results.
 */
public double timeStatements()
{
  long cnt, loops = 0, t1, t2, runtime;
  System.gc();
  do {
    //double the number of iterations
    loops = 2 * loops + 1;
    //System.out.print("loops = " + loops);
    t1 = System.currentTimeMillis();
    for (cnt = 0; cnt < loops; ++cnt) {
      //--->>>statements to be timed should start here<<<---
      Object o = new Object();
      //--->>>end of statements to be timed<<<---
    }
    t2 = System.currentTimeMillis();
    runtime = t2 - t1;
    //System.out.println(": " + runtime);
  } while (runtime < TIMEPERTEST);
  return ((double)runtime / cnt) - looptime;
}