java PerformanceTest 50 100
Hence, the general command is:
java PerformanceTest [# of repetitions] [# of iterations]
Once the repetition value is supplied, the test suite repeats each test for the specified amount of time and then automatically generates the average values. Unsurprisingly, the average times for each test are more consistent than the individual values (which could have a 10-20 ms margin of error).
On my aging core i3 machine, testing each operation fifty times for one million iterations yielded the following times:
.getClass.getName().equals("A"): 48ms
.getClass().getName() == "A": 65ms
a.equals(Type.A): 48ms
a == Type.A: 58ms
a instanceof A: 58ms
Testing each operation 50 times for ten million iterations yielded the following times:
.getClass.getName().equals("A"): 441ms
.getClass().getName() == "A": 624ms
a.equals(Type.A): 451ms
a == Type.A: 536ms
a instanceof A: 539ms
(Note how increasing the # of iterations by ten increased the running times by ten as well)
With reliable data in hand, it's now reasonable to conclude that:
1. Trying to elucidate a class' identity using .getClass().getName() == will far and away take the longest amount of time.
2. Conversely, using .getClass.getName().equals("A") for the same purpose will take the least amount of time.
3. Using the equals method associated with the enum (i.e. a.equals(Type.A)), will result in a relatively short running time.
4. instanceof and a == Type.A are neither the slowest nor the fastest ways to determine class identity.
Representing the relative efficiency of all five options visually you get (from least efficient to most efficient):
.getClass().getName() == "A" > a instanceof A > a == Type.A > a.equals(Type.A) > .getClass.getName().equals("A")
Thus, the data favours a class hierarchy design in which every entity (i.e. operation, integer, function) is represented by its own class (Category A in the previous entry). From an efficiency standpoint, the current system is not the worst. However, letting a full fledged math library handle all of the ASTNode operations will result in a marginal increase in efficiency, which is the end-goal of this project.
Here are the results I am getting on my desktop with 50 repetitions and 10 000 000 iterations:
ReplyDelete.getClass.getName().equals("A"): 281ms
.getClass().getName() == "A": 281ms
a.equals(Type.A): 280ms
a == Type.A: 278ms
a instanceof A: 278ms
Using java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
on Centos 6.5.
I think the differences are too little, even with your results Victor, to influence too much
the design decisions
Hello Nico,
ReplyDeleteThanks for taking a look at the code.
I played with the test a few times after posting this, and I agree with you that the differences are not significant enough to influence design decisions. Instead of trying to tease out inefficiencies from the 'bottom-up' like I've been trying to do thus far, I have decided to start from the absolute top aka the Simulation Core Library. I was able to do the profiling just fine. However, I wasn't sure what JSBML's exact role was viz a viz the Simulation Core Library.
This is something I am currently working on, and I will have an entry related to the subject shortly.