Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
According to the Joel test, we should be able to make the build in one step. Ideally, we would want the documentation to be updated automatically after running the benchmark scripts (and not have to copy/paste values into a .py file and run that). This PR is a first step in that direction.
These changes support easy generation of timing graphs. It works by using a common interface to print the timing result and scores, which also stores these values and finally saves them to file (using pickle). Once the benchmark file (for a task, e.g. "svm") has been run on all the datasets, the user (and in the future the buildbot) can run
$ python plot.py task
to generate the plot in one step.To use, run
$ python bench_svm.py arcene
$ python bench_svm.py madelon
$ python plot.py svm
The next step is to integrate the scores and timings directly into the documentation.