commit b210aefb7fd22e49077f7be6ef481ca0bf20d2bb Author: Bartosz Duszel bartosz.duszel@protonmail.com Date: Thu Mar 26 16:19:42 2020 +0100

doc: Replace "underline" with "## Section name".

This approach doesn't require variable-length underlines. --- doc/HACKING/CircuitPaddingQuickStart.md | 3 ++ doc/HACKING/CodeStructure.md | 2 ++ doc/HACKING/CodingStandards.md | 51 +++++++++------------------ doc/HACKING/CodingStandardsRust.md | 33 ++++++------------ doc/HACKING/EndOfLifeTor.md | 7 ++-- doc/HACKING/Fuzzing.md | 2 -- doc/HACKING/GettingStarted.md | 9 ++--- doc/HACKING/GettingStartedRust.md | 27 +++++---------- doc/HACKING/HelpfulTools.md | 61 ++++++++++++--------------------- doc/HACKING/HowToReview.md | 18 ++++------ doc/HACKING/Module.md | 8 ++--- doc/HACKING/README.1st.md | 13 ++++--- doc/HACKING/ReleasingTor.md | 15 ++++---- doc/HACKING/Tracing.md | 12 +++---- doc/HACKING/WritingTests.md | 39 +++++++++------------ doc/HACKING/android/Simpleperf.md | 1 - 16 files changed, 114 insertions(+), 187 deletions(-)

diff --git a/doc/HACKING/CircuitPaddingQuickStart.md b/doc/HACKING/CircuitPaddingQuickStart.md index 167ff9f29..2780b5c6e 100644 --- a/doc/HACKING/CircuitPaddingQuickStart.md +++ b/doc/HACKING/CircuitPaddingQuickStart.md @@ -47,6 +47,7 @@ If you want to install on your localsystem, run `make install`. For our case we just want the tor binary at `src/app/tor`.

## Use tor in TB and at a relay + Download and install a fresh Tor Browser (TB) from torproject.org. Make sure it works. From the command line, relative to the folder created when you extracted TB, run `./Browser/start-tor-browser --verbose` to get some basic log output. @@ -75,6 +76,7 @@ relay. Start TB, visit a website, and manually confirm that the middle is used by looking at the circuit display.

## Add a bare-bones APE padding machine + Now the fun part. We have several resources at our disposal (mind that links might be broken in the future, just search for the headings): - The official [Circuit Padding Developer @@ -192,6 +194,7 @@ register our machines: We run `make` to get a new `tor` binary and copy it to our local TB.

## Run the machine + To be able to view circuit info events in the console as we launch TB, we add `Log [circ]info notice stdout` to `torrc` of TB. diff --git a/doc/HACKING/CodeStructure.md b/doc/HACKING/CodeStructure.md index 814ee6300..d387018f9 100644 --- a/doc/HACKING/CodeStructure.md +++ b/doc/HACKING/CodeStructure.md @@ -1,3 +1,5 @@ +# Code Structure + TODO: revise this to talk about how things are, rather than how things have changed.

diff --git a/doc/HACKING/CodingStandards.md b/doc/HACKING/CodingStandards.md index 73f8f95a2..a181759d6 100644 --- a/doc/HACKING/CodingStandards.md +++ b/doc/HACKING/CodingStandards.md @@ -1,5 +1,4 @@ -Coding conventions for Tor -========================== +# Coding conventions for Tor

tl;dr:

@@ -10,8 +9,7 @@ tl;dr: - Run `make distcheck` if you have made changes to build system components - Add a file in `changes` for your branch.

-Patch checklist ---------------- +## Patch checklist

If possible, send your patch as one of these (in descending order of preference) @@ -42,8 +40,7 @@ If you have changed build system components: - For example, if you have changed Makefiles, autoconf files, or anything else that affects the build system.

-License issues -============== +## License issues

Tor is distributed under the license terms in the LICENSE -- in brief, the "3-clause BSD license". If you send us code to @@ -57,8 +54,7 @@ Some compatible licenses include: - 2-clause BSD - CC0 Public Domain Dedication

-How we use Git branches -======================= +## How we use Git branches

Each main development series (like 0.2.1, 0.2.2, etc) has its main work applied to a single branch. At most one series can be the development series @@ -89,8 +85,7 @@ conflicts in the ChangeLog when it comes time to merge your branch into Tor. Best advice: don't try to keep an independent branch forked for more than 6 months and expect it to merge cleanly. Try to merge pieces early and often.

-How we log changes -================== +## How we log changes

When you do a commit that needs a ChangeLog entry, add a new file to the `changes` toplevel subdirectory. It should have the format of a @@ -196,8 +191,7 @@ Why use changes files instead of entries in the ChangeLog? * Having every single commit touch the ChangeLog file tended to create zillions of merge conflicts.

-Whitespace and C conformance ----------------------------- +## Whitespace and C conformance

Invoke `make check-spaces` from time to time, so it can tell you about deviations from our C whitespace style. Generally, we use: @@ -228,8 +222,7 @@ you're using gcc, you should invoke the configure script with the option `--enable-fatal-warnings`. This will tell the compiler to make all warnings into errors.

-Functions to use; functions not to use --------------------------------------- +## Functions to use; functions not to use

We have some wrapper functions like `tor_malloc`, `tor_free`, `tor_strdup`, and `tor_gettimeofday;` use them instead of their generic equivalents. (They @@ -277,8 +270,7 @@ version prefixed with `tor_` instead: strtok_r, memmem, memstr, asprintf, localtime_r, gmtime_r, inet_aton, inet_ntop, inet_pton, getpass, ntohll, htonll. (This list is incomplete.)

-What code can use what other code? ----------------------------------- +## What code can use what other code?

We're trying to simplify Tor's structure over time. In the long run, we want Tor to be structured as a set of modules with *no circular dependencies*. @@ -295,8 +287,7 @@ included except those specifically permitted by the `.may_include` file. When editing one of these files, please make sure that you are not introducing any cycles into Tor's dependency graph.

-Floating point math is hard ---------------------------- +## Floating point math is hard

Floating point arithmetic as typically implemented by computers is very counterintuitive. Failure to adequately analyze floating point @@ -355,8 +346,7 @@ For more detailed (and math-intensive) background, see [What Every Computer Scientist Should Know About Floating-Point Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html).

-Other C conventions -------------------- +## Other C conventions

The `a ? b : c` trinary operator only goes inside other expressions; don't use it as a replacement for if. (You can ignore this inside macro @@ -365,8 +355,7 @@ definitions when necessary.) Assignment operators shouldn't nest inside other expressions. (You can ignore this inside macro definitions when necessary.)

-Binary data and wire formats ----------------------------- +## Binary data and wire formats

Use pointer to `char` when representing NUL-terminated string. To represent arbitrary binary data, use pointer to `uint8_t`. (Many older Tor APIs ignore @@ -385,8 +374,7 @@ for more information about trunnel.

For information on adding new trunnel code to Tor, see src/trunnel/README

-Calling and naming conventions ------------------------------- +## Calling and naming conventions

Whenever possible, functions should return -1 on error and 0 on success.

@@ -409,15 +397,13 @@ probably time to create an enum. If you find that you are passing three or more flags to a function, it's probably time to create a flags argument that takes a bitfield.

-What To Optimize ----------------- +## What To Optimize

Don't optimize anything if it's not in the critical path. Right now, the critical path seems to be AES, logging, and the network itself. Feel free to do your own profiling to determine otherwise.

-Log conventions ---------------- +## Log conventions

[FAQ - Log Levels](https://www.torproject.org/docs/faq#LogLevel)

@@ -433,8 +419,7 @@ end-users that they aren't expected to understand the message (perhaps with a string like "internal error"). Option (A) is to be preferred to option (B).

-Assertions In Tor ------------------ +## Assertions In Tor

Assertions should be used for bug-detection only. Don't use assertions to detect bad user inputs, network errors, resource exhaustion, or similar @@ -453,8 +438,7 @@ macro, as in: if (BUG(ptr == NULL)) return -1;

-Allocator conventions ---------------------- +## Allocator conventions

By convention, any tor type with a name like `abc_t` should be allocated by a function named `abc_new()`. This function should never return @@ -495,8 +479,7 @@ to use it as a function callback), define it with a name like When deallocating, don't say e.g. `if (x) tor_free(x)`. The convention is to have deallocators do nothing when NULL pointer is passed.

-Doxygen comment conventions ---------------------------- +## Doxygen comment conventions

Say what functions do as a series of one or more imperative sentences, as though you were telling somebody how to be the function. In other words, DO diff --git a/doc/HACKING/CodingStandardsRust.md b/doc/HACKING/CodingStandardsRust.md index 22b6f9343..97026c9b7 100644 --- a/doc/HACKING/CodingStandardsRust.md +++ b/doc/HACKING/CodingStandardsRust.md @@ -1,11 +1,9 @@ -Rust Coding Standards -======================= +# Rust Coding Standards

You MUST follow the standards laid out in `doc/HACKING/CodingStandards.md`, where applicable.

-Module/Crate Declarations ---------------------------- +## Module/Crate Declarations

Each Tor C module which is being rewritten MUST be in its own crate. See the structure of `src/rust` for examples. @@ -53,8 +51,7 @@ If you have any external modules as dependencies (e.g. `extern crate libc;`), you MUST declare them in your crate's `lib.rs` and NOT in any other module.

-Dependencies and versions ---------------------------- +## Dependencies and versions

In general, we use modules from only the Rust standard library whenever possible. We will review including external crates on a @@ -80,8 +77,7 @@ Currently, Tor requires that you use the latest stable Rust version. At some point in the future, we will freeze on a given stable Rust version, to ensure backward compatibility with stable distributions that ship it.

-Updating/Adding Dependencies ------------------------------- +## Updating/Adding Dependencies

To add/remove/update dependencies, first add your dependencies, exactly specifying their versions, into the appropriate *crate-level* @@ -100,8 +96,7 @@ Next, run `/scripts/maint/updateRustDependencies.sh`. Then, go into `src/ext/rust` and commit the changes to the `tor-rust-dependencies` repo.

-Documentation ---------------- +## Documentation

You MUST include `#![deny(missing_docs)]` in your crate.

@@ -117,14 +112,12 @@ types/constants/objects/functions/methods, you SHOULD also include an You MUST document your module with _module docstring_ comments, i.e. `//!` at the beginning of each line.

-Style -------- +## Style

You SHOULD consider breaking up large literal numbers with `_` when it makes it more human readable to do so, e.g. `let x: u64 = 100_000_000_000`.

-Testing ---------- +## Testing

All code MUST be unittested and integration tested.

@@ -147,8 +140,7 @@ should put: } }

-Benchmarking --------------- +## Benchmarking

The external `test` crate can be used for most benchmarking. However, using this crate requires nightly Rust. Since we may want to switch to a more @@ -185,23 +177,20 @@ Finally, to write your benchmark code, in } }

-Fuzzing ---------- +## Fuzzing

If you wish to fuzz parts of your code, please see the [cargo fuzz](https://github.com/rust-fuzz/cargo-fuzz) crate, which uses [libfuzzer-sys](https://github.com/rust-fuzz/libfuzzer-sys).

-Whitespace & Formatting -------------------------- +## Whitespace & Formatting

You MUST run `rustfmt` (https://github.com/rust-lang-nursery/rustfmt) on your code before your code will be merged. You can install rustfmt by doing `cargo install rustfmt-nightly` and then run it with `cargo fmt`.

-Safety --------- +## Safety

You SHOULD read [the nomicon](https://doc.rust-lang.org/nomicon/) before writing Rust FFI code. It is *highly advised* that you read and write normal Rust code diff --git a/doc/HACKING/EndOfLifeTor.md b/doc/HACKING/EndOfLifeTor.md index cada2da7f..0735dc311 100644 --- a/doc/HACKING/EndOfLifeTor.md +++ b/doc/HACKING/EndOfLifeTor.md @@ -1,12 +1,11 @@ -End of Life on an old release series ------------------------------------- +# End of Life on an old release series

Here are the steps that the maintainer should take when an old Tor release series reaches End of Life. Note that they are _only_ for entire series that have reached their planned EOL: they do not apply to security-related deprecations of individual versions.

-### 0. Preliminaries +## 0. Preliminaries

0. A few months before End of Life: Write a deprecation announcement. @@ -16,7 +15,7 @@ deprecations of individual versions. Send the announcement to tor-announce, tor-talk, tor-relays, and the packagers.

-### 1. On the day +## 1. On the day

1. Open tickets to remove the release from: - the jenkins builds diff --git a/doc/HACKING/Fuzzing.md b/doc/HACKING/Fuzzing.md index c2db7e985..41853a8a2 100644 --- a/doc/HACKING/Fuzzing.md +++ b/doc/HACKING/Fuzzing.md @@ -11,7 +11,6 @@ To run the fuzzing test cases in a deterministic fashion, use: This won't actually fuzz Tor! It will just run all the fuzz binaries on our existing set of testcases for the fuzzer.

- ## Different kinds of fuzzing

Right now we support three different kinds of fuzzer. @@ -51,7 +50,6 @@ But the fuzzing harness should crash if tor fails an assertion, triggers a bug, or accesses memory it shouldn't. This helps fuzzing frameworks detect "interesting" cases.

- ## Guided Fuzzing with AFL

There is no HTTPS, hash, or signature for American Fuzzy Lop's source code, so diff --git a/doc/HACKING/GettingStarted.md b/doc/HACKING/GettingStarted.md index 3be9917d3..c2ca74d96 100644 --- a/doc/HACKING/GettingStarted.md +++ b/doc/HACKING/GettingStarted.md @@ -1,5 +1,4 @@ -Getting started in Tor development -================================== +# Getting started in Tor development

Congratulations! You've found this file, and you're reading it! This means that you might be interested in getting started in developing Tor. @@ -14,8 +13,7 @@ dump of important information, you might like reading the You probably should skim some of the topic headings there before you write your first patch.

-Required background -------------------- +## Required background

First, I'm going to assume that you can build Tor from source, and that you know enough of the C language to read and write it. (See the README @@ -30,8 +28,7 @@ Most Tor developers develop using some Unix-based system, such as GNU/Linux, BSD, or macOS. It's okay to develop on Windows if you want, but you're going to have a more difficult time.

-Getting your first patch into Tor ---------------------------------- +## Getting your first patch into Tor

Once you've reached this point, here's what you need to know.

diff --git a/doc/HACKING/GettingStartedRust.md b/doc/HACKING/GettingStartedRust.md index 9c3101e9a..247ea5c69 100644 --- a/doc/HACKING/GettingStartedRust.md +++ b/doc/HACKING/GettingStartedRust.md @@ -1,8 +1,6 @@ -Hacking on Rust in Tor -======================== +# Hacking on Rust in Tor

-Getting Started ------------------ +## Getting Started

Please read or review our documentation on Rust coding standards (`doc/HACKING/CodingStandardsRust.md`) before doing anything. @@ -22,8 +20,7 @@ Please be patient with the other people who are working on getting more Rust code into Tor, because they are graciously donating their free time to contribute to this effort.

-Resources for learning Rust ------------------------------ +## Resources for learning Rust

**Beginning resources**

@@ -48,8 +45,7 @@ is For learning more about FFI and Rust, see Jake Goulding's [Rust FFI Omnibus](http://jakegoulding.com/rust-ffi-omnibus/).

-Compiling Tor with Rust enabled ---------------------------------- +## Compiling Tor with Rust enabled

You will need to run the `configure` script with the `--enable-rust` flag to explicitly build with Rust. Additionally, you will need to @@ -86,8 +82,7 @@ topmost directory of the repository, this would be:

TOR_RUST_DEPENDENCIES=`pwd`/src/ext/rust/crates ./configure --enable-rust

-Identifying which modules to rewrite -====================================== +## Identifying which modules to rewrite

The places in the Tor codebase that are good candidates for porting to Rust are: @@ -115,8 +110,7 @@ interconnected your target module is. The output will tell you each module name, along with a set of every module that the module calls. Modules which call fewer other modules are better targets.

-Writing your Rust module -========================== +## Writing your Rust module

Strive to change the C API as little as possible.

@@ -138,8 +132,7 @@ Where `<name_of_c_object>` can be an enum, struct, constant, etc. Then, do the same in the C code, to note that rust will need to be changed when the C does.

-Adding your Rust module to Tor's build system ------------------------------------------------ +## Adding your Rust module to Tor's build system

0. Your translation of the C module should live in its own crate(s) in the `src/rust/` directory. @@ -153,8 +146,7 @@ dependency of other Rust modules): `src/rust/tor_util/Cargo.toml` and include it in `src/rust/tor_rust/lib.rs`

-How to test your Rust code ----------------------------- +## How to test your Rust code

Everything should be tested full stop. Even non-public functionality.

@@ -174,7 +166,6 @@ Tor's integration tests should also pass:

make test-stem

-Submitting a patch -===================== +## Submitting a patch

Please follow the instructions in `doc/HACKING/GettingStarted.md`. diff --git a/doc/HACKING/HelpfulTools.md b/doc/HACKING/HelpfulTools.md index ae892c34a..15bd15331 100644 --- a/doc/HACKING/HelpfulTools.md +++ b/doc/HACKING/HelpfulTools.md @@ -1,11 +1,10 @@ -Useful tools -============ +# Useful tools

These aren't strictly necessary for hacking on Tor, but they can help track down bugs.

-Travis/Appveyor CI ------------------- +## Travis/Appveyor CI + It's CI.

Looks like this: @@ -29,8 +28,7 @@ for your fork to build commits outside of PRs too: Builds should show up on the web at travis-ci.com and on IRC at #tor-ci on OFTC. If they don't, ask #tor-dev (also on OFTC).

-Jenkins -------- +## Jenkins

It's CI/builders. Looks like this: https://jenkins.torproject.org

@@ -43,8 +41,7 @@ Builds Linux and Windows cross-compilation. Runs Linux tests. Builds should show up on the web at jenkins.torproject.org and on IRC at #tor-bots on OFTC. If they don't, ask #tor-dev (also on OFTC).

-Valgrind --------- +## Valgrind

valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/app/tor

@@ -52,16 +49,14 @@ Valgrind pass `--undef-value-errors=no` to valgrind, or rebuild your openssl with `-DPURIFY`.)

-Coverity --------- +## Coverity

Nick regularly runs the coverity static analyzer on the Tor codebase.

The preprocessor define `__COVERITY__` is used to work around instances where coverity picks up behavior that we wish to permit.

-clang Static Analyzer ---------------------- +## clang Static Analyzer

The clang static analyzer can be run on the Tor codebase using Xcode (WIP) or a command-line build. @@ -69,8 +64,7 @@ or a command-line build. The preprocessor define `__clang_analyzer__` is used to work around instances where clang picks up behavior that we wish to permit.

-clang Runtime Sanitizers ------------------------- +## clang Runtime Sanitizers

To build the Tor codebase with the clang Address and Undefined Behavior sanitizers, see the file `contrib/clang/sanitize_blacklist.txt`. @@ -78,8 +72,7 @@ sanitizers, see the file `contrib/clang/sanitize_blacklist.txt`. Preprocessor workarounds for instances where clang picks up behavior that we wish to permit are also documented in the blacklist file.

-Running lcov for unit test coverage ------------------------------------ +## Running lcov for unit test coverage

Lcov is a utility that generates pretty HTML reports of test code coverage. To generate such a report: @@ -96,8 +89,7 @@ output directory, use `make coverage-html HTML_COVER_DIR=./funky_new_cov_dir`. Coverage diffs using lcov are not currently implemented, but are being investigated (as of July 2014).

-Running the unit tests ----------------------- +## Running the unit tests

To quickly run all the tests distributed with Tor:

@@ -123,8 +115,7 @@ working connection to the internet:

make test-full-online

-Running gcov for unit test coverage ------------------------------------ +## Running gcov for unit test coverage

./configure --enable-coverage make @@ -164,8 +155,7 @@ lines? If you run ./scripts/test/cov-exclude, it marks excluded unreached lines with 'x', and excluded reached lines with '!!!'.

-Running integration tests -------------------------- +## Running integration tests

We have the beginnings of a set of scripts to run integration tests using Chutney. To try them, set CHUTNEY_PATH to your chutney source directory, and @@ -174,14 +164,12 @@ run `make test-network`. We also have scripts to run integration tests using Stem. To try them, set `STEM_SOURCE_DIR` to your Stem source directory, and run `test-stem`.

-Profiling Tor -------------- +## Profiling Tor

Ongoing notes about Tor profiling can be found at https://pad.riseup.net/p/profiling-tor

-Profiling Tor with oprofile ---------------------------- +## Profiling Tor with oprofile

The oprofile tool runs (on Linux only!) to tell you what functions Tor is spending its CPU time in, so we can identify performance bottlenecks. @@ -206,8 +194,7 @@ Here are some basic instructions * `opreport -l that_dir/*` - Profit

-Profiling Tor with perf ------------------------ +## Profiling Tor with perf

This works with a running Tor, and requires root.

@@ -236,8 +223,7 @@ This works with a running Tor, and requires root. report -g > <FILENAME>.out`. Then you can compress that and put it somewhere public.

-Profiling Tor with gperftools aka Google-performance-tools ----------------------------------------------------------- +## Profiling Tor with gperftools aka Google-performance-tools

This should work on nearly any unixy system. It doesn't seem to be compatible with RunAsDaemon though. @@ -255,8 +241,7 @@ performance! See the gperftools manual for more info, but basically:

3. Run `pprof src/app/tor /tmp/profile` to start the REPL.

-Generating and analyzing a callgraph ------------------------------------- +## Generating and analyzing a callgraph

0. Build Tor on linux or mac, ideally with -O0 or -fno-inline.

@@ -266,8 +251,7 @@ Generating and analyzing a callgraph Note that currently the callgraph generator can't detect calls that pass through function pointers.

-Getting emacs to edit Tor source properly ------------------------------------------ +## Getting emacs to edit Tor source properly

Nick likes to put the following snippet in his .emacs file:

@@ -315,8 +299,7 @@ If you use emacs for editing Tor and nothing else, you could always just say: There is probably a better way to do this. No, we are probably not going to clutter the files with emacs stuff.

-Building a tag file (code index) --------------------------------- +## Building a tag file (code index)

Many functions in tor use `MOCK_IMPL` wrappers for unit tests. Your tag-building program must be told how to handle this syntax. @@ -340,8 +323,7 @@ instead: A vim-compatible tag file will be generated by default. If you use emacs, add the `-e` flag to generate an emacs-compatible tag file.

-Doxygen -------- +## Doxygen

We use the 'doxygen' utility to generate documentation from our source code. Here's how to use it: @@ -396,8 +378,7 @@ source code. Here's how to use it: 6. See the Doxygen manual for more information; this summary just scratches the surface.

-Style and best-practices checking --------------------------------- +## Style and best-practices checking

We use scripts to check for various problems in the formatting and style of our source code. The "check-spaces" test detects a bunch of violations diff --git a/doc/HACKING/HowToReview.md b/doc/HACKING/HowToReview.md index 2325e7017..7815e7663 100644 --- a/doc/HACKING/HowToReview.md +++ b/doc/HACKING/HowToReview.md @@ -1,5 +1,4 @@ -How to review a patch -===================== +# How to review a patch

Some folks have said that they'd like to review patches more often, but they don't know how. @@ -9,9 +8,7 @@ So, here are a bunch of things to check for when reviewing a patch! Note that if you can't do every one of these, that doesn't mean you can't do a good review! Just make it clear what you checked for and what you didn't.

- -Top-level smell-checks ----------------------- +## Top-level smell-checks

(Difficulty: easy)

@@ -37,10 +34,9 @@ Top-level smell-checks - If this changes anything in the code, is there a "changes" file?

-Let's look at the code! ------------------------ +## Let's look at the code!

-- Does the code conform to CodingStandards.txt? +- Does the code conform to `CodingStandards.md`?

- Does the code leak memory?

@@ -60,8 +56,7 @@ Let's look at the code! - Is there duplicated code that could be turned into a function?

-Let's look at the documentation! --------------------------------- +## Let's look at the documentation!

- Does the documentation conform to CodingStandards.txt?

@@ -70,8 +65,7 @@ Let's look at the documentation! - Can you predict what the function will do from its documentation?

-Let's think about security! ---------------------------- +## Let's think about security!

- If there are any arrays, buffers, are you 100% sure that they cannot overflow? diff --git a/doc/HACKING/Module.md b/doc/HACKING/Module.md index 781bb978f..f8a9773d4 100644 --- a/doc/HACKING/Module.md +++ b/doc/HACKING/Module.md @@ -1,9 +1,9 @@ -# Modules in Tor # +# Modules in Tor

This document describes the build system and coding standards when writing a module in Tor.

-## What is a module? ## +## What is a module?

In the context of the tor code base, a module is a subsystem that we can selectively enable or disable, at `configure` time. @@ -27,7 +27,7 @@ because you would actually want to run one without the other.) To disable a module, pass `--disable-module-{dirauth,relay}` at configure time. All modules are currently enabled by default.

-## Build System ## +## Build System

The changes to the build system are pretty straightforward.

@@ -55,7 +55,7 @@ Finally, your module will automatically be included in the `TOR_MODULES_ALL_ENABLED` variable which is used to build the unit tests. They always build everything in order to test everything.

-## Coding ## +## Coding

As mentioned above, a module should be isolated in its own directories, suffixed with the name of the module, in `src/*/`. diff --git a/doc/HACKING/README.1st.md b/doc/HACKING/README.1st.md index 376bbfb6a..34f525af0 100644 --- a/doc/HACKING/README.1st.md +++ b/doc/HACKING/README.1st.md @@ -1,5 +1,6 @@ -In this directory ------------------ +# README.1st + +## In this directory

This directory has helpful information about what you need to know to hack on Tor! @@ -21,8 +22,7 @@ with Tor. If it's your job to put out Tor releases, see `ReleasingTor.md` so that you don't miss any steps!

-Additional Information ------------------------ +## Additional Information

For full information on how Tor is supposed to work, look at the files in [Tor specification](https://gitweb.torproject.org/torspec.git/tree). @@ -34,8 +34,7 @@ For the latest version of the code, get a copy of git, and

git clone https://git.torproject.org/git/tor

-Stay in touch ------------------------ +## Stay in touch

We talk about Tor on the `tor-talk` mailing list. Design proposals and discussion belong on the `tor-dev` mailing list. We hang around on @@ -47,7 +46,7 @@ get started working with Tor.

Happy hacking!

-TODO +## TODO -----------------------

XXXXX also describe diff --git a/doc/HACKING/ReleasingTor.md b/doc/HACKING/ReleasingTor.md index fbc5de54e..2464d8afb 100644 --- a/doc/HACKING/ReleasingTor.md +++ b/doc/HACKING/ReleasingTor.md @@ -1,10 +1,9 @@ -Putting out a new release -------------------------- +# Putting out a new release

Here are the steps that the maintainer should take when putting out a new Tor release:

-### 0. Preliminaries +## 0. Preliminaries

1. Get at least two of weasel/arma/Sebastian to put the new version number in their approved versions list. Give them a few @@ -17,7 +16,7 @@ new Tor release: date of a TB that contains it. See note below in "commit, upload, announce".

-### I. Make sure it works +## I. Make sure it works

1. Make sure that CI passes: have a look at Travis (https://travis-ci.org/torproject/tor/branches), Appveyor @@ -50,7 +49,7 @@ new Tor release: libevent and openssl, so using valgrind will sometimes find extra memory leaks.)

-### II. Write a changelog +## II. Write a changelog

1a. (Alpha release variant)

@@ -135,7 +134,7 @@ new Tor release: to start sorting and condensing entries. (Generally, we don't edit the text of existing entries, though.)

-### III. Making the source release. +## III. Making the source release.

1. In `maint-0.?.x`, bump the version number in `configure.ac` and run `make update-versions` to update version numbers in other @@ -161,7 +160,7 @@ new Tor release: If it is not, you'll need to poke Roger, Weasel, and Sebastian again: see item 0.1 at the start of this document.

-### IV. Commit, upload, announce +## IV. Commit, upload, announce

1. Sign the tarball, then sign and push the git tag:

@@ -236,7 +235,7 @@ new Tor release: For templates to use when announcing, see: https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/Announce...

-### V. Aftermath and cleanup +## V. Aftermath and cleanup

1. If it's a stable release, bump the version number in the `maint-x.y.z` branch to "newversion-dev", and do a `merge -s ours` diff --git a/doc/HACKING/Tracing.md b/doc/HACKING/Tracing.md index 24fa76131..e1e97abe6 100644 --- a/doc/HACKING/Tracing.md +++ b/doc/HACKING/Tracing.md @@ -1,16 +1,16 @@ -# Tracing # +# Tracing

This document describes how the event tracing subsystem works in tor so developers can add events to the code base but also hook them to an event tracing framework.

-## Basics ### +## Basics

Event tracing is separated in two concepts, trace events and a tracer. The tracing subsystem can be found in `src/trace`. The `events.h` header file is the main file that maps the different tracers to trace events.

-### Events ### +### Events

A trace event is basically a function from which we can pass any data that we want to collect. In addition, we specify a context for the event such as @@ -39,7 +39,7 @@ How `argc` is collected or used has nothing to do with the instrumentation the trace events and collection framework (tracer) are decoupled. You _can_ have trace events without a tracer.

-### Tracer ### +### Tracer

In `src/trace/events.h`, we map the `tor_trace()` function to the right tracer. A tracer support is only enabled at compile time. For instance, the @@ -47,7 +47,7 @@ file `src/trace/debug.h` contains the mapping of the generic tracing function `tor_trace()` to the `log_debug()` function. More specialized function can be mapped depending on the tracepoint.

-## Build System ## +## Build System

This section describes how it is integrated into the build system of tor.

@@ -66,7 +66,7 @@ configure option:

--enable-tracing-debug

-## Instrument Tor ## +## Instrument Tor

This is pretty easy. Let's say you want to add a trace event in `src/feature/rend/rendcache.c`, you only have to add this include statement: diff --git a/doc/HACKING/WritingTests.md b/doc/HACKING/WritingTests.md index c946402b4..d21202052 100644 --- a/doc/HACKING/WritingTests.md +++ b/doc/HACKING/WritingTests.md @@ -1,5 +1,4 @@ -Writing tests for Tor: an incomplete guide -========================================== +# Writing tests for Tor: an incomplete guide

Tor uses a variety of testing frameworks and methodologies to try to keep from introducing bugs. The major ones are: @@ -18,8 +17,7 @@ keep from introducing bugs. The major ones are:

5. The Shadow network simulator.

-How to run these tests ----------------------- +## How to run these tests

### The easy version

@@ -63,7 +61,7 @@ The former are those that should finish in a few seconds; the latter tend to take more time, and may include CPU-intensive operations, deliberate delays, and stuff like that.

-### Finding test coverage +## Finding test coverage

Test coverage is a measurement of which lines your tests actually visit.

@@ -111,7 +109,7 @@ To count new or modified uncovered lines in D2, you can run:

./scripts/test/cov-diff ${D1} ${D2}" | grep '^+ *#' | wc -l

-### Marking lines as unreachable by tests +## Marking lines as unreachable by tests

You can mark a specific line as unreachable by using the special string LCOV_EXCL_LINE. You can mark a range of lines as unreachable @@ -125,8 +123,7 @@ unreached lines with 'x', and excluded reached lines with '!!!'. Note: you should never do this unless the line is meant to 100% unreachable by actual code.

-What kinds of test should I write? ----------------------------------- +## What kinds of test should I write?

Integration testing and unit testing are complementary: it's probably a good idea to make sure that your code is hit by both if you can. @@ -141,8 +138,7 @@ If your code adds new externally visible functionality to Tor, it would be great to have a test for that functionality. That's where integration tests more usually come in.

-Unit and regression tests: Does this function do what it's supposed to? ------------------------------------------------------------------------ +## Unit and regression tests: Does this function do what it's supposed to?

Most of Tor's unit tests are made using the "tinytest" testing framework. You can see a guide to using it in the tinytest manual at @@ -163,7 +159,7 @@ If you have created a new test file, you will need to:

I use the term "unit test" and "regression tests" very sloppily here.

-### A simple example +## A simple example

Here's an example of a test function for a simple function in util.c:

@@ -205,7 +201,7 @@ Finally, remember that by convention, all `*_free()` functions that Tor defines are defined to accept NULL harmlessly. Thus, you don't need to say `if (contents)` in the cleanup block.

-### Exposing static functions for testing +## Exposing static functions for testing

Sometimes you need to test a function, but you don't want to expose it outside its usual module. @@ -226,7 +222,7 @@ For example, `crypto_curve25519.h` contains: The `crypto_curve25519.c` file and the `test_crypto.c` file both define `CRYPTO_CURVE25519_PRIVATE`, so they can see this declaration.

-### STOP! Does this test really test? +## STOP! Does this test really test?

When writing tests, it's not enough to just generate coverage on all the lines of the code that you're testing: It's important to make sure that @@ -267,7 +263,7 @@ it's supposed to do, and fail otherwise. Try to design your tests so that they check for the code's intended and documented functionality as much as possible.

-### Mock functions for testing in isolation +## Mock functions for testing in isolation

Often we want to test that a function works right, but the function to be tested depends on other functions whose behavior is hard to observe, @@ -308,7 +304,7 @@ And later, you can restore the original function with: For more information, see the definitions of this mocking logic in `testsupport.h`.

-### Okay but what should my tests actually do? +## Okay but what should my tests actually do?

We talk above about "test coverage" -- making sure that your tests visit every line of code, or every branch of code. But visiting the code isn't @@ -379,7 +375,7 @@ Based on the implementation, we now see three more edge cases to test: * Removing an element from the end of the list * Removing an element from a position other than the end of the list.

-### What should my tests NOT do? +## What should my tests NOT do?

Tests shouldn't require a network connection.

@@ -397,7 +393,7 @@ When possible, tests should not be over-fit to the implementation. That is, the test should verify that the documented behavior is implemented, but should not break if other permissible behavior is later implemented.

-### Advanced techniques: Namespaces +## Advanced techniques: Namespaces

Sometimes, when you're doing a lot of mocking at once, it's convenient to isolate your identifiers within a single namespace. If this were C++, we'd @@ -409,8 +405,7 @@ them, you define `NS_MODULE` to a prefix to be used for your identifiers, and then use other macros in place of identifier names. See `src/test/test.h` for more documentation.

-Integration tests: Calling Tor from the outside ------------------------------------------------ +## Integration tests: Calling Tor from the outside

Some tests need to invoke Tor from the outside, and shouldn't run from the same process as the Tor test program. Reasons for doing this might include: @@ -430,8 +425,7 @@ wrapped, add a new shell script to `TESTS`, and the new program to makefile (eg `${PYTHON}` for a python interpreter), then make sure that the makefile exports them.

-Writing integration tests with Stem ------------------------------------ +## Writing integration tests with Stem

The 'stem' library includes extensive tests for the Tor controller protocol. You can run stem tests from tor with `make test-stem`, or see @@ -477,8 +471,7 @@ you notice any strange behaviour that seems totally unreasonable. Check out the `test_exit_policy()` function in abovementioned file to see the final implementation for this test.

-System testing with Chutney ---------------------------- +## System testing with Chutney

The 'chutney' program configures and launches a set of Tor relays, authorities, and clients on your local host. It has a `test network` diff --git a/doc/HACKING/android/Simpleperf.md b/doc/HACKING/android/Simpleperf.md index 25f39a3d2..c7e63a7c8 100644 --- a/doc/HACKING/android/Simpleperf.md +++ b/doc/HACKING/android/Simpleperf.md @@ -95,4 +95,3 @@ was spend on the call. Start Tor the normal way via Orbot and collect the logs from your computer using

$ adb logcat -