[tor-commits] [tor/master] Try to tweak fuzzing.md to correspond to my changes

[nickm at torproject.org]

commit 44fa14c0e28ac26a551169d5621648db9bc08da9
Author: Nick Mathewson <nickm at torproject.org>
Date:   Wed Dec 14 08:12:41 2016 -0500

    Try to tweak fuzzing.md to correspond to my changes
---
 doc/HACKING/Fuzzing.md      | 36 +++++++++++++++++++-----------------
 src/test/fuzz/fuzz_multi.sh | 34 ++++++++++++++++++++++++++++++++++
 src/test/fuzz_multi.sh      | 34 ----------------------------------
 3 files changed, 53 insertions(+), 51 deletions(-)

diff --git a/doc/HACKING/Fuzzing.md b/doc/HACKING/Fuzzing.md
index 703f028..36f0fc4 100644
--- a/doc/HACKING/Fuzzing.md
+++ b/doc/HACKING/Fuzzing.md
@@ -3,6 +3,11 @@
 To run the fuzzing test cases in a deterministic fashion, use:
   make fuzz
 
+  [I've turned this off for now. - NM]
+
+To build the fuzzing harness binaries, use:
+  make fuzzers
+
 == Guided Fuzzing with AFL
 
 There is no HTTPS, hash, or signature for American Fuzzy Lop's source code, so
@@ -22,7 +27,7 @@ To Find The ASAN Memory Limit: (64-bit only)
 On 64-bit platforms, afl needs to know how much memory ASAN uses.
 Or, you can configure tor without --enable-expensive-hardening, then use
   make fuzz
-to run the generated test cases through an ASAN-enabled fuzz_dir.
+to run the generated test cases through an ASAN-enabled fuzz-http.
 Read afl/docs/notes_for_asan.txt for more details.
 
   Download recidivm from http://jwilk.net/software/recidivm
@@ -36,10 +41,14 @@ Read afl/docs/notes_for_asan.txt for more details.
   (Normally, recidivm would output a figure automatically, but in some cases,
   the fuzzing harness will hang when the memory limit is too small.)
 
+You could also just say "none" instead of the memory limit below, if you
+don't care about memory limits.
+
+
 To Run:
-  mkdir -p src/test/fuzz/fuzz_dir_testcase src/test/fuzz/fuzz_dir_findings
-  echo "dummy" > src/test/fuzz/fuzz_dir_testcase/minimal.case
-  ../afl/afl-fuzz -i src/test/fuzz/fuzz_dir_testcase -o src/test/fuzz/fuzz_dir_findings -m <asan-memory-limit> -- src/test/fuzz_dir
+  mkdir -p src/test/fuzz/fuzz_http_findings
+  ../afl/afl-fuzz -i src/test/fuzz/data/http -x src/test/fuzz/dict/http -o src/test/fuzz/fuzz_http_findings -m <asan-memory-limit> -- src/test/fuzz_dir
+
 
 AFL has a multi-core mode, check the documentation for details.
 You might find the included fuzz-multi.sh script useful for this.
@@ -48,19 +57,12 @@ macOS (OS X) requires slightly more preparation, including:
 * using afl-clang (or afl-clang-fast from the llvm directory)
 * disabling external crash reporting (AFL will guide you through this step)
 
-AFL may also benefit from using dictionary files for text-based inputs: these
-can be placed in src/test/fuzz/fuzz_dir_dictionary/.
-
-Multiple dictionaries can be used with AFL, you should choose a combination of
-dictionaries that targets the code you are fuzzing.
-
 == Writing Tor fuzzers
 
-A tor fuzzing harness should:
-* read input from standard input (many fuzzing frameworks also accept file
-  names)
-* parse that input
-* produce results on standard output (this assists in diagnosing errors)
+A tor fuzzing harness should have:
+* a fuzz_init() function to set up any necessary global state.
+* a fuzz_main() function to receive input and pass it to a parser.
+* a fuzz_cleanup() function to clear global state.
 
 Most fuzzing frameworks will produce many invalid inputs - a tor fuzzing
 harness should rejecting invalid inputs without crashing or behaving badly.
@@ -78,8 +80,8 @@ Check if a hang is reproducible before reporting it. Sometimes, processing
 valid inputs may take a second or so, particularly with the fuzzer and
 sanitizers enabled.
 
-To see what fuzz_dir is doing with a test case, call it like this:
-  src/test/fuzz_dir --debug < /path/to/test.case
+To see what fuzz-http is doing with a test case, call it like this:
+  src/test/fuzz/fuzz-http --debug < /path/to/test.case
 
 (Logging is disabled while fuzzing to increase fuzzing speed.)
 
diff --git a/src/test/fuzz/fuzz_multi.sh b/src/test/fuzz/fuzz_multi.sh
new file mode 100755
index 0000000..b4a17ed
--- /dev/null
+++ b/src/test/fuzz/fuzz_multi.sh
@@ -0,0 +1,34 @@
+MEMLIMIT_BYTES=21990500990976
+
+N_CPUS=1
+if [ $# -ge 1 ]; then
+    N_CPUS="$1"
+    shift
+fi
+
+FILTER=echo
+
+for i in `seq -w "$N_CPUS"`; do
+    if [ "$i" -eq 1 ]; then
+        if [ "$N_CPUS" -eq 1 ]; then
+            INSTANCE=""
+            NUMBER=""
+        else
+            INSTANCE="-M"
+            NUMBER="$i"
+        fi
+    else
+        INSTANCE="-S"
+        NUMBER="$i"
+    fi
+    # use whatever remains on the command-line to prefix the fuzzer command
+    # you have to copy and paste and run these commands yourself
+    "$FILTER" "$@" \
+        ../afl/afl-fuzz \
+        -i src/test/fuzz/fuzz_dir_testcase \
+        -o src/test/fuzz/fuzz_dir_findings \
+        -x src/test/fuzz/fuzz_dir_dictionary/fuzz_dir_http_header.dct \
+        -m "$MEMLIMIT_BYTES" \
+        "$INSTANCE" "$NUMBER" \
+        -- src/test/fuzz_dir
+done
diff --git a/src/test/fuzz_multi.sh b/src/test/fuzz_multi.sh
deleted file mode 100755
index b4a17ed..0000000
--- a/src/test/fuzz_multi.sh
+++ /dev/null
@@ -1,34 +0,0 @@
-MEMLIMIT_BYTES=21990500990976
-
-N_CPUS=1
-if [ $# -ge 1 ]; then
-    N_CPUS="$1"
-    shift
-fi
-
-FILTER=echo
-
-for i in `seq -w "$N_CPUS"`; do
-    if [ "$i" -eq 1 ]; then
-        if [ "$N_CPUS" -eq 1 ]; then
-            INSTANCE=""
-            NUMBER=""
-        else
-            INSTANCE="-M"
-            NUMBER="$i"
-        fi
-    else
-        INSTANCE="-S"
-        NUMBER="$i"
-    fi
-    # use whatever remains on the command-line to prefix the fuzzer command
-    # you have to copy and paste and run these commands yourself
-    "$FILTER" "$@" \
-        ../afl/afl-fuzz \
-        -i src/test/fuzz/fuzz_dir_testcase \
-        -o src/test/fuzz/fuzz_dir_findings \
-        -x src/test/fuzz/fuzz_dir_dictionary/fuzz_dir_http_header.dct \
-        -m "$MEMLIMIT_BYTES" \
-        "$INSTANCE" "$NUMBER" \
-        -- src/test/fuzz_dir
-done