Ennhancing Unit Tests for Performance Monitoring #2

Welcome back,

as I wrote in my last posting on unit tests, they can be used to get a runtime profile for a certain region that you are interested in. One general shortcoming of unit tests when testing the inner structure of a method or region is, that they that focus on behavior that can externally be seen. We propose an alternative usage of the assertion mechanism in order to measure the performance profile of a method in its inner body. Without our approach you would have to restructure your code to get this kind of information. At the same time this would also changes the timing behavior. When dealing with concurrent programming this is exactly, what you don't want. When monitoring the performance what you would do today is let a profiler run you application which captures everything you can get. This ends up in a huge pile of runtime information that might not even be relevant for the region you are interested in. As I've mentioned before: A parallel application is a queue of sequential and parallel code. The pearl neckless. Remember?!

Throughout an application there are many different parameters that affect the runtime behavior, reaching from general aspects such as the design pattern that is used to describe the architecture, to more specific ones such as the number of threads executing an application or the size of global data structures. The concrete value of each parameter can be manifold and is orthogonal to all other parameters. Thus, each parameter defines one dimension in a hypercube depicting the complexity space of an application. It can easily be seen that already a very small application with -let's say- 5 parameters results in a 5-dimensional cube.

Now, what's got all this to do with performance optimization and the pearl necklace you might ask? Calm down and relax. I'll get to it: When capturing the whole application and deducing parameters along the way, you easily end up inside exploding space due to the complexity. Instead, our approach enables you to focus on certain regions of interest. You can at first focus on those areas of sequential code that use most of the runtime and focus on monitoring those parallel regions alongside the whole development phase. Also, we keep a history table to be able to capture and analyze the effects changes had on your "pearls".

K!