There is no lack of benchmark in any field of computer science. However, traditional benchmarks fail to address the issue of relevance to real applications. Often times the behavior of the benchmark's workloads does not match that of the application of interest, in which case the benchmark might give misleading information regarding which platform is the best suited for the intended application.
We propose an innovative approach to benchmarking called application-specific benchmarking, based on the principle that system performance be measured in the context of applications in which users are interested. This is achieved by incorporating characteristics of the application of interest into the benchmarking process, yielding performance metrics that reflect the expected behavior of a particular application across a range of different platforms.
This research explores methodologies for application-specific benchmarking and the HBench framework that applies these techniques to performance evaluation of different types of systems. A list of platforms that we have experimented with includes:
This page is currently maintained by Alexandra Fedorova (fedorova at eecs.harvard.edu).