Skip to content
This repository has been archived by the owner on Jun 19, 2024. It is now read-only.

Commit

Permalink
Small document describing the benchmark result pages and graphs.
Browse files Browse the repository at this point in the history
  • Loading branch information
schani committed Aug 16, 2012
1 parent 1349e27 commit b70fd8e
Showing 1 changed file with 65 additions and 0 deletions.
65 changes: 65 additions & 0 deletions RESULT-DOC.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Index

The
[index page](http://storage.bos.xamarin.com/mono-gcbench/index.html)
shows all configurations on which the benchmark suite was run. For
the last revision, it shows the average benchmark runtime relative to
the best performing revision so far, as well as the relative runtime
of the worst-performing benchmark of the current revision.

The configuration overview graphs are explained below.

# Configuration page

[Configuration pages](http://storage.bos.xamarin.com/mono-gcbench/default-sgen/index.html)
summarize individual configuration results. The table has a row for
each benchmark, giving times and revisions for the best and worst
results, as well as absolute and relative times for the latest three
revisions.

It also shows small versions of the benchmark graphs for each
benchmark. Those graphs are explained below.

In the rightmost column it gives graphs of native code size, as
reported by `mono --stats`.

## Configuration overview graph

In the
[configuration overview graph](http://storage.bos.xamarin.com/mono-gcbench/default-sgen/combined_large.png)
revision time moves from left to right, i.e. earlier revisions are
left, later revisions are right. The vertical axis is relative
runtime, so lower is better.

The main (black) line shows the average relative benchmark runtime,
i.e. it gives a good measure of how each revision performs on
average. The green and red lines show the best and worst performing
individual benchmark runtimes per revision, i.e. they indicate whether
a revision has any outliers in terms of performance.

The green and red markers indicate the best and worst performing
revisions, respectively, based on average relative benchmark runtime.

# Individual benchmark page

Each individual
[benchmark page](http://storage.bos.xamarin.com/mono-gcbench/default-sgen/graph4.html)
lists results for all revisions the particular benchmark was run on
the particular configuration. It presents average, minimum and
maximum run times (in seconds), native code size, and gives a link to
the raw run time files.

## Benchmark graph

In the
[benchmark graph](http://storage.bos.xamarin.com/mono-gcbench/default-sgen/graph4_large.png),
just as in the configuration overview graph, revision time moves from
left to right and the vertical axis indicates runtime, in this case
absolute.

The main (black) line shows the average runtime, over three runs,
whereas the gray area shows the variation of the runs, i.e. slowest to
fastest.

The green and red markers indicate the best and worst performing
revisions for the benchmark, respectively.

0 comments on commit b70fd8e

Please sign in to comment.