Skip to content

Commit

Permalink
Documentation fixes. (google#2705)
Browse files Browse the repository at this point in the history
  • Loading branch information
inferno-chromium authored Aug 15, 2019
1 parent 5c1e1aa commit 055f983
Show file tree
Hide file tree
Showing 14 changed files with 122 additions and 136 deletions.
14 changes: 5 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,16 @@ Currently, OSS-Fuzz supports C and C++ code, though other languages supported by
Read our [detailed documentation](https://google.github.io/oss-fuzz) to learn how to use OSS-Fuzz.

## Trophies
As of August 2019, OSS-Fuzz has found [~14,000] bugs in over [200] open source
As of August 2019, OSS-Fuzz has found over [14,000] bugs in over [200] open source
projects.

[~14,000]: https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=-status%3AWontFix%2CDuplicate+-Infra
[14,000]: https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=-status%3AWontFix%2CDuplicate+-Infra
[200]: https://github.com/google/oss-fuzz/tree/master/projects

## Blog posts

* 2016-12-01 ([Open Source](https://opensource.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html),
[Testing](https://testing.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html),
[Security](https://security.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html))
* 2017-05-08 ([Open Source](https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html),
[Testing](https://testing.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html),
[Security](https://security.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html))
* 2018-11-06 ([Security](https://security.googleblog.com/2018/11/a-new-chapter-for-oss-fuzz.html))
* 2016-12-01 - [Announcing OSS-Fuzz: Continuous fuzzing for open source software](https://opensource.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html).
* 2017-05-08 - [OSS-Fuzz: Five months later, and rewarding projects](https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html)
* 2018-11-06 - [A New Chapter for OSS-Fuzz](https://security.googleblog.com/2018/11/a-new-chapter-for-oss-fuzz.html)


16 changes: 10 additions & 6 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,18 @@
# Prerequisites
# Readme

Use the following instructions to make documentation changes locally.

## Prerequisites
```bash
sudo apt install ruby bundler
bundle install --path vendor/bundle
$ sudo apt install ruby bundler
$ bundle install --path vendor/bundle
```

# Serving locally
## Serving locally
```bash
bundle exec jekyll serve
$ bundle exec jekyll serve
```

# Theme documentation
## Theme documentation
We are using the [just the docs](https://pmarsceill.github.io/just-the-docs/)
theme.
42 changes: 22 additions & 20 deletions docs/advanced-topics/code_coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,64 +20,63 @@ Code Coverage].

Docker images get regularly updated with a newer version of build tools, build
configurations, scripts, and other changes. It is recommended to use the most
recent images for a better user experience.
recent images.


```bash
python infra/helper.py pull_images
$ python infra/helper.py pull_images
```


## Build fuzz targets

Code Coverage report generation requires a special build configuration to be
used. In order to produce such build for your project, run:
used. To create a code coverage build for your project, run:

```bash
python infra/helper.py build_image $project_name
python infra/helper.py build_fuzzers --sanitizer=coverage $project_name
$ python infra/helper.py build_image $PROJECT_NAME
$ python infra/helper.py build_fuzzers --sanitizer=coverage $PROJECT_NAME
```


## Establish access to GCS

To get a good understanding of quality of the fuzz testing established for your
project, code coverage should be generated by running fuzz targets against the
corpus aggregated by OSS-Fuzz. The helper script will download the corpus
automatically using `gsutil` tool. To make it work, you need:
To get a good understanding of quality of fuzz testing established for your
project, code coverage reports should be generated by running fuzz targets
against the corpus aggregated by OSS-Fuzz. Set up `gsutil` and ensure that
you have access to the corpora using:

* Install [gsutil tool]
* Check whether you have access to the corpus for your project:

```bash
gsutil ls gs://${project_name}-corpus.clusterfuzz-external.appspot.com/
$ gsutil ls gs://${PROJECT_NAME}-corpus.clusterfuzz-external.appspot.com/
```

If you see an authorization error from the command above, run:

```bash
gcloud auth login
$ gcloud auth login
```

and try again. Once `gsutil` works, you can run the report generation.


## Generating code coverage reports
## Generate code coverage reports

### Full project report

To generate code coverage report using the corpus aggregated on OSS-Fuzz, run:

```bash
python infra/helper.py coverage $project_name
$ python infra/helper.py coverage $PROJECT_NAME
```

If you want to generate code coverage report using the corpus you have locally,
copy the corpus into `build/corpus/$project_name/$fuzz_target/` directories for
copy the corpus into `build/corpus/$PROJECT_NAME/<fuzz_target_name>/` directories for
each fuzz target, then run:

```bash
python infra/helper.py coverage --no-corpus-download $project_name
$ python infra/helper.py coverage --no-corpus-download $PROJECT_NAME
```

### Single fuzz target
Expand All @@ -86,14 +85,15 @@ You can generate a code coverage report for a particular fuzz target with
`--fuzz-target` argument:

```bash
python infra/helper.py coverage --fuzz-target=<fuzz_target_name> $project_name
$ python infra/helper.py coverage --fuzz-target=<fuzz_target_name> $PROJECT_NAME
```

In this mode, you can specify an arbitrary corpus location for the fuzz target
via `--corpus-dir` to be used instead of the corpus downloaded from OSS-Fuzz:

```bash
python infra/helper.py coverage --fuzz-target=<fuzz_target_name> --corpus-dir=<my_local_corpus_dir> $project_name
$ python infra/helper.py coverage --fuzz-target=<fuzz_target_name> \
--corpus-dir=<my_local_corpus_dir> $PROJECT_NAME
```

### Additional arguments for `llvm-cov`
Expand All @@ -102,14 +102,16 @@ You may want to use some of the options of [llvm-cov tool], for example,
`-ignore-filename-regex=`. You can pass those to the helper script after `--`:

```bash
python infra/helper.py coverage $project_name -- -ignore-filename-regex=.*code/to/be/ignored/.* <other_extra_args>
$ python infra/helper.py coverage $PROJECT_NAME -- \
-ignore-filename-regex=.*code/to/be/ignored/.* <other_extra_args>
```

To specify particular source files or directories to show in the report, list
their paths at the end of the extra arguments sequence, for example:

```bash
python infra/helper.py coverage zlib -- <other_extra_args> /src/zlib/inftrees.c /src/zlib_uncompress_fuzzer.cc /src/zlib/zutil.c
$ python infra/helper.py coverage zlib -- \
<other_extra_args> /src/zlib/inftrees.c /src/zlib_uncompress_fuzzer.cc /src/zlib/zutil.c
```

If you want OSS-Fuzz to use some extra arguments when generating code coverage
Expand Down
6 changes: 4 additions & 2 deletions docs/advanced-topics/corpora.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,12 +52,14 @@ And then run the following command to copy the corpus to a directory on your
machine.

```bash
gsutil -m rsync gs://<bucket_path> <local_directory>
$ gsutil -m rsync gs://<bucket_path> <local_directory>
```
Following the expat example above, this would be:

```bash
gsutil -m rsync gs://expat-corpus.clusterfuzz-external.appspot.com/libFuzzer/expat_parse_fuzzer <local_directory>
$ gsutil -m rsync \
gs://expat-corpus.clusterfuzz-external.appspot.com/libFuzzer/expat_parse_fuzzer \
<local_directory>
```

## Corpus backups
Expand Down
33 changes: 17 additions & 16 deletions docs/advanced-topics/ideal_integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,10 @@ Every [fuzz target](http://libfuzzer.info/#fuzz-target):
* Is [maintained by code owners](#fuzz-target) in their RCS (Git, SVN, etc).
* Is [built with the rest of the tests](#build-support) - no bit rot!
* Has a [seed corpus](#seed-corpus) with good [code coverage](#coverage).
* Has a [dictionary](#dictionary), if applicable.
* Is [continuously tested on the seed corpus](#regression-testing) with
[ASan/UBSan/MSan](https://github.com/google/sanitizers).
* Is [fast and has no OOMs](#performance).
* Has a [fuzzing dictionary](#fuzzing-dictionary), if applicable

## Fuzz Target
The code of the [fuzz target(s)](http://libfuzzer.info/#fuzz-target) should be
Expand All @@ -41,7 +41,8 @@ arise as source code changes over time.

Make sure to fuzz the target locally for a small period of time to ensure that
it does not crash, hang, or run out of memory instantly.
See details at http://libfuzzer.info and http://tutorial.libfuzzer.info.
You can read more about what makes a good fuzz target [here]
(https://github.com/google/fuzzing/blob/master/docs/good-fuzz-target.md)

The interface between the [fuzz target]((http://libfuzzer.info/#fuzz-target))
and the fuzzing engines is C, so you may use C or C++ to implement the fuzz target.
Expand Down Expand Up @@ -102,19 +103,7 @@ Examples:
[openssl](https://github.com/openssl/openssl/tree/master/fuzz),
[nss](https://github.com/mozilla/nss-fuzzing-corpus) (corpus in a separate repo).

## Regression Testing
The fuzz targets should be regularly tested (not necessarily fuzzed!) as a part
of the project's regression testing process.
One way to do so is to link the fuzz target with a simple standalone driver
(e.g. [this one](https://github.com/llvm-mirror/compiler-rt/tree/master/lib/fuzzer/standalone))
that runs the provided inputs and use this driver with the seed corpus created
in previous step. It is recommended to use
[sanitizers](https://github.com/google/sanitizers) during regression testing.

Examples: [SQLite](https://www.sqlite.org/src/artifact/d9f1a6f43e7bab45),
[openssl](https://github.com/openssl/openssl/blob/master/fuzz/test-corpus.c)

## Fuzzing dictionary
## Dictionary
For some input types, a simple dictionary of tokens used by the input language
can have a dramatic positive effect on fuzzing efficiency.
For example, when fuzzing an XML parser, a dictionary of XML tokens will help.
Expand All @@ -137,11 +126,23 @@ documentation page.
Coverage can often be improved by adding dictionaries, more inputs for seed
corpora, and fixing timeouts/out-of-memory bugs in your targets.

## Regression Testing
The fuzz targets should be regularly tested (not necessarily fuzzed!) as a part
of the project's regression testing process.
One way to do so is to link the fuzz target with a simple standalone driver
(e.g. [this one](https://github.com/llvm-mirror/compiler-rt/tree/master/lib/fuzzer/standalone))
that runs the provided inputs and use this driver with the seed corpus created
in previous step. It is recommended to use
[sanitizers](https://github.com/google/sanitizers) during regression testing.

Examples: [SQLite](https://www.sqlite.org/src/artifact/d9f1a6f43e7bab45),
[openssl](https://github.com/openssl/openssl/blob/master/fuzz/test-corpus.c)

## Performance
Fuzz targets should also be performant, as high memory usage and/or slow
execution speed can slow the down the growth of coverage and finding of new
bugs. ClusterFuzz provides a
[performance analyzer]({{ site.baseurl }}/furthur-reading/clusterfuzz)
[performance analyzer]({{ site.baseurl }}/furthur-reading/clusterfuzz/#performance-analyzer)
for each fuzz target that shows problems that are impacting the performance of
the fuzz target.

Expand Down
11 changes: 7 additions & 4 deletions docs/advanced-topics/reproducing.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Download it. If the issue is not public, you will need to login using your
[Google account](https://support.google.com/accounts/answer/176347?hl=en)
([why?]({{ site.baseurl }}/faq/#why-do-you-require-a-google-account-for-authentication))
that the bug report CCs.
This file contains the bytes that were fed to the [Fuzz Target](http://libfuzzer.info/#fuzz-target).
This file contains the bytes that were fed to the [fuzz target](http://libfuzzer.info/#fuzz-target).

If you have already
[integrated]({{ site.baseurl }}/advanced-topics/ideal-integration/)
Expand Down Expand Up @@ -94,7 +94,8 @@ $ python infra/helper.py reproduce libxml2 libxml2_xml_read_memory_fuzzer ~/Down
## Reproduce using local source checkout

```bash
$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> $PROJECT_NAME <source_path>
$ python infra/helper.py build_fuzzers \
--sanitizer <address/memory/undefined> $PROJECT_NAME <source_path>
$ python infra/helper.py reproduce $PROJECT_NAME <fuzz_target_name> <testcase_path>
```

Expand All @@ -118,8 +119,10 @@ correctly configured, even if it succeeded. To reproduce these locally, run:

```bash
$ python infra/helper.py build_image $PROJECT_NAME
$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> --engine <libfuzzer/afl/honggfuzz> $PROJECT_NAME
$ python infra/helper.py check_build --sanitizer <address/memory/undefined> --engine <libfuzzer/afl/honggfuzz> $PROJECT_NAME <fuzz_target_name>
$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> \
--engine <libfuzzer/afl/honggfuzz> $PROJECT_NAME
$ python infra/helper.py check_build --sanitizer <address/memory/undefined> \
--engine <libfuzzer/afl/honggfuzz> $PROJECT_NAME <fuzz_target_name>
```

For reproducing a `coverage` build failure, follow
Expand Down
6 changes: 3 additions & 3 deletions docs/furthur-reading/fuzzer_environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,12 @@ are not available on the bot runtime environment (where the fuzz targets run).

If you need these dependencies in the runtime environment, you can either
- Install the packages via Dockerfile
([example](https://github.com/google/oss-fuzz/blob/master/projects/tor/Dockerfile#L19))
([example](https://github.com/google/oss-fuzz/blob/2d5e2ef84f281e6ab789055aa735606d3122fda9/projects/tor/Dockerfile#L19))
and then link statically against them
([example](https://github.com/google/oss-fuzz/blob/master/projects/tor/build.sh#L40))
([example](https://github.com/google/oss-fuzz/blob/2d5e2ef84f281e6ab789055aa735606d3122fda9/projects/tor/build.sh#L40))
- Or build the dependencies statically in
[build.sh]({{ site.baseurl }}/getting-started/new-project-guide/#buildsh)
([example](https://github.com/google/oss-fuzz/blob/master/projects/ffmpeg/build.sh#L26)).
([example](https://github.com/google/oss-fuzz/blob/64f8b6593da141b97c98c7bc6f07df92c42ee010/projects/ffmpeg/build.sh#L26)).

All build artifacts needed during fuzz target execution should be inside the
`$OUT` directory. Only those artifacts are archived and used on the bots.
Expand Down
Loading

0 comments on commit 055f983

Please sign in to comment.