[tor-commits] [tor/master] Split the old doc/HACKING into several new files
nickm at torproject.org
nickm at torproject.org
Fri Oct 9 14:40:57 UTC 2015
commit a11cb74d2910178284898df478b5058352f24279
Author: Nick Mathewson <nickm at torproject.org>
Date: Fri Oct 9 09:38:59 2015 -0400
Split the old doc/HACKING into several new files
---
doc/HACKING/CodingStandards.txt | 220 +++++++++++++
doc/HACKING/GettingStarted.txt | 29 +-
doc/HACKING/HACKING_old | 662 ---------------------------------------
doc/HACKING/HelpfulTools.txt | 303 ++++++++++++++++++
doc/HACKING/ReleasingTor.txt | 124 ++++++++
doc/include.am | 6 +-
6 files changed, 678 insertions(+), 666 deletions(-)
diff --git a/doc/HACKING/CodingStandards.txt b/doc/HACKING/CodingStandards.txt
new file mode 100644
index 0000000..fa4990d
--- /dev/null
+++ b/doc/HACKING/CodingStandards.txt
@@ -0,0 +1,220 @@
+Coding conventions for Tor
+--------------------------
+
+tl;dr:
+
+ * Run configure with '--enable-gcc-warnings'
+ * Run 'make check-spaces' to catch whitespace errors
+ * Document your functions
+ * Write unit tests
+ * Add a file in 'changes' for your branch.
+
+Patch checklist
+~~~~~~~~~~~~~~~
+
+If possible, send your patch as one of these (in descending order of
+preference)
+
+ - A git branch we can pull from
+ - Patches generated by git format-patch
+ - A unified diff
+
+Did you remember...
+
+ - To build your code while configured with --enable-gcc-warnings?
+ - To run "make check-spaces" on your code?
+ - To run "make check-docs" to see whether all new options are on
+ the manpage?
+ - To write unit tests, as possible?
+ - To base your code on the appropriate branch?
+ - To include a file in the "changes" directory as appropriate?
+
+How we use Git branches
+-----------------------
+
+Each main development series (like 0.2.1, 0.2.2, etc) has its main work
+applied to a single branch. At most one series can be the development series
+at a time; all other series are maintenance series that get bug-fixes only.
+The development series is built in a git branch called "master"; the
+maintenance series are built in branches called "maint-0.2.0", "maint-0.2.1",
+and so on. We regularly merge the active maint branches forward.
+
+For all series except the development series, we also have a "release" branch
+(as in "release-0.2.1"). The release series is based on the corresponding
+maintenance series, except that it deliberately lags the maint series for
+most of its patches, so that bugfix patches are not typically included in a
+maintenance release until they've been tested for a while in a development
+release. Occasionally, we'll merge an urgent bugfix into the release branch
+before it gets merged into maint, but that's rare.
+
+If you're working on a bugfix for a bug that occurs in a particular version,
+base your bugfix branch on the "maint" branch for the first supported series
+that has that bug. (As of June 2013, we're supporting 0.2.3 and later.) If
+you're working on a new feature, base it on the master branch.
+
+
+How we log changes
+------------------
+
+When you do a commit that needs a ChangeLog entry, add a new file to
+the "changes" toplevel subdirectory. It should have the format of a
+one-entry changelog section from the current ChangeLog file, as in
+
+ o Major bugfixes:
+ - Fix a potential buffer overflow. Fixes bug 99999; bugfix on
+ 0.3.1.4-beta.
+
+To write a changes file, first categorize the change. Some common categories
+are: Minor bugfixes, Major bugfixes, Minor features, Major features, Code
+simplifications and refactoring. Then say what the change does. If
+it's a bugfix, mention what bug it fixes and when the bug was
+introduced. To find out which Git tag the change was introduced in,
+you can use "git describe --contains <sha1 of commit>".
+
+If at all possible, try to create this file in the same commit where you are
+making the change. Please give it a distinctive name that no other branch will
+use for the lifetime of your change. To verify the format of the changes file,
+you can use "make check-changes".
+
+When we go to make a release, we will concatenate all the entries
+in changes to make a draft changelog, and clear the directory. We'll
+then edit the draft changelog into a nice readable format.
+
+What needs a changes file?::
+ A not-exhaustive list: Anything that might change user-visible
+ behavior. Anything that changes internals, documentation, or the build
+ system enough that somebody could notice. Big or interesting code
+ rewrites. Anything about which somebody might plausibly wonder "when
+ did that happen, and/or why did we do that" 6 months down the line.
+
+Why use changes files instead of Git commit messages?::
+ Git commit messages are written for developers, not users, and they
+ are nigh-impossible to revise after the fact.
+
+Why use changes files instead of entries in the ChangeLog?::
+ Having every single commit touch the ChangeLog file tended to create
+ zillions of merge conflicts.
+
+Whitespace and C conformance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Invoke "make check-spaces" from time to time, so it can tell you about
+deviations from our C whitespace style. Generally, we use:
+
+ - Unix-style line endings
+ - K&R-style indentation
+ - No space before newlines
+ - A blank line at the end of each file
+ - Never more than one blank line in a row
+ - Always spaces, never tabs
+ - No more than 79-columns per line.
+ - Two spaces per indent.
+ - A space between control keywords and their corresponding paren
+ "if (x)", "while (x)", and "switch (x)", never "if(x)", "while(x)", or
+ "switch(x)".
+ - A space between anything and an open brace.
+ - No space between a function name and an opening paren. "puts(x)", not
+ "puts (x)".
+ - Function declarations at the start of the line.
+
+We try hard to build without warnings everywhere. In particular, if you're
+using gcc, you should invoke the configure script with the option
+"--enable-gcc-warnings". This will give a bunch of extra warning flags to
+the compiler, and help us find divergences from our preferred C style.
+
+Functions to use
+~~~~~~~~~~~~~~~~
+
+We have some wrapper functions like tor_malloc, tor_free, tor_strdup, and
+tor_gettimeofday; use them instead of their generic equivalents. (They
+always succeed or exit.)
+
+You can get a full list of the compatibility functions that Tor provides by
+looking through src/common/util.h and src/common/compat.h. You can see the
+available containers in src/common/containers.h. You should probably
+familiarize yourself with these modules before you write too much code, or
+else you'll wind up reinventing the wheel.
+
+Use 'INLINE' instead of 'inline', so that we work properly on Windows.
+
+Calling and naming conventions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Whenever possible, functions should return -1 on error and 0 on success.
+
+For multi-word identifiers, use lowercase words combined with
+underscores. (e.g., "multi_word_identifier"). Use ALL_CAPS for macros and
+constants.
+
+Typenames should end with "_t".
+
+Function names should be prefixed with a module name or object name. (In
+general, code to manipulate an object should be a module with the same name
+as the object, so it's hard to tell which convention is used.)
+
+Functions that do things should have imperative-verb names
+(e.g. buffer_clear, buffer_resize); functions that return booleans should
+have predicate names (e.g. buffer_is_empty, buffer_needs_resizing).
+
+If you find that you have four or more possible return code values, it's
+probably time to create an enum. If you find that you are passing three or
+more flags to a function, it's probably time to create a flags argument that
+takes a bitfield.
+
+What To Optimize
+~~~~~~~~~~~~~~~~
+
+Don't optimize anything if it's not in the critical path. Right now, the
+critical path seems to be AES, logging, and the network itself. Feel free to
+do your own profiling to determine otherwise.
+
+Log conventions
+~~~~~~~~~~~~~~~
+
+https://www.torproject.org/docs/faq#LogLevel
+
+No error or warning messages should be expected during normal OR or OP
+operation.
+
+If a library function is currently called such that failure always means ERR,
+then the library function should log WARN and let the caller log ERR.
+
+Every message of severity INFO or higher should either (A) be intelligible
+to end-users who don't know the Tor source; or (B) somehow inform the
+end-users that they aren't expected to understand the message (perhaps
+with a string like "internal error"). Option (A) is to be preferred to
+option (B).
+
+
+
+Doxygen comment conventions
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Say what functions do as a series of one or more imperative sentences, as
+though you were telling somebody how to be the function. In other words, DO
+NOT say:
+
+ /** The strtol function parses a number.
+ *
+ * nptr -- the string to parse. It can include whitespace.
+ * endptr -- a string pointer to hold the first thing that is not part
+ * of the number, if present.
+ * base -- the numeric base.
+ * returns: the resulting number.
+ */
+ long strtol(const char *nptr, char **nptr, int base);
+
+Instead, please DO say:
+
+ /** Parse a number in radix <b>base</b> from the string <b>nptr</b>,
+ * and return the result. Skip all leading whitespace. If
+ * <b>endptr</b> is not NULL, set *<b>endptr</b> to the first character
+ * after the number parsed.
+ **/
+ long strtol(const char *nptr, char **nptr, int base);
+
+Doxygen comments are the contract in our abstraction-by-contract world: if
+the functions that call your function rely on it doing something, then your
+function should mention that it does that something in the documentation. If
+you rely on a function doing something beyond what is in its documentation,
+then you should watch out, or it might do something else later.
diff --git a/doc/HACKING/GettingStarted.txt b/doc/HACKING/GettingStarted.txt
index 1f4b434..a5fcfb7 100644
--- a/doc/HACKING/GettingStarted.txt
+++ b/doc/HACKING/GettingStarted.txt
@@ -192,9 +192,34 @@ Once you've reached this point, here's what you need to know.
Where do I go from here?
------------------------
-doc/HACKING
-doc/WritingTests.txt
+More helpful resources
+----------------------
+
+
+For full information on how Tor is supposed to work, look at the files in
+https://gitweb.torproject.org/torspec.git/tree
+
+For an explanation of how to change Tor's design to work differently, look at
+https://gitweb.torproject.org/torspec.git/blob_plain/HEAD:/proposals/001-process.txt
+
+For the latest version of the code, get a copy of git, and
+
+ git clone https://git.torproject.org/git/tor
+
+We talk about Tor on the tor-talk mailing list. Design proposals and
+discussion belong on the tor-dev mailing list. We hang around on
+irc.oftc.net, with general discussion happening on #tor and development
+happening on #tor-dev.
+
+The other files in this "HACKING" directory may also be useful as you
+get started working with Tor.
+
+Happy hacking!
+
+XXXXX also describe
+
+doc/HACKING/WritingTests.txt
torguts.git
diff --git a/doc/HACKING/HACKING_old b/doc/HACKING/HACKING_old
deleted file mode 100644
index 5a1454e..0000000
--- a/doc/HACKING/HACKING_old
+++ /dev/null
@@ -1,662 +0,0 @@
-Hacking Tor: An Incomplete Guide
-================================
-
-Important links
----------------
-
-For full information on how Tor is supposed to work, look at the files in
-https://gitweb.torproject.org/torspec.git/tree
-
-For an explanation of how to change Tor's design to work differently, look at
-https://gitweb.torproject.org/torspec.git/blob_plain/HEAD:/proposals/001-process.txt
-
-For the latest version of the code, get a copy of git, and
-
- git clone https://git.torproject.org/git/tor
-
-We talk about Tor on the tor-talk mailing list. Design proposals and
-discussion belong on the tor-dev mailing list. We hang around on
-irc.oftc.net, with general discussion happening on #tor and development
-happening on #tor-dev.
-
-For a nice quick-start guide to hacking on Tor, have a look at
-doc/GettingStarted.txt, included with the Tor distribution.
-
-How we use Git branches
------------------------
-
-Each main development series (like 0.2.1, 0.2.2, etc) has its main work
-applied to a single branch. At most one series can be the development series
-at a time; all other series are maintenance series that get bug-fixes only.
-The development series is built in a git branch called "master"; the
-maintenance series are built in branches called "maint-0.2.0", "maint-0.2.1",
-and so on. We regularly merge the active maint branches forward.
-
-For all series except the development series, we also have a "release" branch
-(as in "release-0.2.1"). The release series is based on the corresponding
-maintenance series, except that it deliberately lags the maint series for
-most of its patches, so that bugfix patches are not typically included in a
-maintenance release until they've been tested for a while in a development
-release. Occasionally, we'll merge an urgent bugfix into the release branch
-before it gets merged into maint, but that's rare.
-
-If you're working on a bugfix for a bug that occurs in a particular version,
-base your bugfix branch on the "maint" branch for the first supported series
-that has that bug. (As of June 2013, we're supporting 0.2.3 and later.) If
-you're working on a new feature, base it on the master branch.
-
-
-How we log changes
-------------------
-
-When you do a commit that needs a ChangeLog entry, add a new file to
-the "changes" toplevel subdirectory. It should have the format of a
-one-entry changelog section from the current ChangeLog file, as in
-
- o Major bugfixes:
- - Fix a potential buffer overflow. Fixes bug 99999; bugfix on
- 0.3.1.4-beta.
-
-To write a changes file, first categorize the change. Some common categories
-are: Minor bugfixes, Major bugfixes, Minor features, Major features, Code
-simplifications and refactoring. Then say what the change does. If
-it's a bugfix, mention what bug it fixes and when the bug was
-introduced. To find out which Git tag the change was introduced in,
-you can use "git describe --contains <sha1 of commit>".
-
-If at all possible, try to create this file in the same commit where you are
-making the change. Please give it a distinctive name that no other branch will
-use for the lifetime of your change. To verify the format of the changes file,
-you can use "make check-changes".
-
-When we go to make a release, we will concatenate all the entries
-in changes to make a draft changelog, and clear the directory. We'll
-then edit the draft changelog into a nice readable format.
-
-What needs a changes file?::
- A not-exhaustive list: Anything that might change user-visible
- behavior. Anything that changes internals, documentation, or the build
- system enough that somebody could notice. Big or interesting code
- rewrites. Anything about which somebody might plausibly wonder "when
- did that happen, and/or why did we do that" 6 months down the line.
-
-Why use changes files instead of Git commit messages?::
- Git commit messages are written for developers, not users, and they
- are nigh-impossible to revise after the fact.
-
-Why use changes files instead of entries in the ChangeLog?::
- Having every single commit touch the ChangeLog file tended to create
- zillions of merge conflicts.
-
-Useful tools
-------------
-
-These aren't strictly necessary for hacking on Tor, but they can help track
-down bugs.
-
-Jenkins
-~~~~~~~
-
-https://jenkins.torproject.org
-
-Dmalloc
-~~~~~~~
-
-The dmalloc library will keep track of memory allocation, so you can find out
-if we're leaking memory, doing any double-frees, or so on.
-
- dmalloc -l ~/dmalloc.log
- (run the commands it tells you)
- ./configure --with-dmalloc
-
-Valgrind
-~~~~~~~~
-
-valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor
-
-(Note that if you get a zillion openssl warnings, you will also need to
-pass --undef-value-errors=no to valgrind, or rebuild your openssl
-with -DPURIFY.)
-
-Coverity
-~~~~~~~~
-
-Nick regularly runs the coverity static analyzer on the Tor codebase.
-
-The preprocessor define __COVERITY__ is used to work around instances
-where coverity picks up behavior that we wish to permit.
-
-clang Static Analyzer
-~~~~~~~~~~~~~~~~~~~~~
-
-The clang static analyzer can be run on the Tor codebase using Xcode (WIP)
-or a command-line build.
-
-The preprocessor define __clang_analyzer__ is used to work around instances
-where clang picks up behavior that we wish to permit.
-
-clang Runtime Sanitizers
-~~~~~~~~~~~~~~~~
-
-To build the Tor codebase with the clang Address and Undefined Behavior
-sanitizers, see the file contrib/clang/sanitize_blacklist.txt.
-
-Preprocessor workarounds for instances where clang picks up behavior that
-we wish to permit are also documented in the blacklist file.
-
-Running lcov for unit test coverage
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Lcov is a utility that generates pretty HTML reports of test code coverage.
-To generate such a report:
-
------
- ./configure --enable-coverage
- make
- make coverage-html
- $BROWSER ./coverage_html/index.html
------
-
-This will run the tor unit test suite `./src/test/test` and generate the HTML
-coverage code report under the directory ./coverage_html/. To change the
-output directory, use `make coverage-html HTML_COVER_DIR=./funky_new_cov_dir`.
-
-Coverage diffs using lcov are not currently implemented, but are being
-investigated (as of July 2014).
-
-Running the unit tests
-~~~~~~~~~~~~~~~~~~~~~~
-
-To quickly run all the tests distributed with Tor:
------
- make check
------
-
-To run the fast unit tests only:
------
- make test
------
-
-To selectively run just some tests (the following can be combined
-arbitrarily):
------
- ./src/test/test <name_of_test> [<name of test 2>] ...
- ./src/test/test <prefix_of_name_of_test>.. [<prefix_of_name_of_test2>..] ...
- ./src/test/test :<name_of_excluded_test> [:<name_of_excluded_test2]...
------
-
-To run all tests, including those based on Stem or Chutney:
------
- make test-full
------
-
-To run all tests, including those basedd on Stem or Chutney that require a
-working connection to the internet:
------
- make test-full-online
------
-
-Running gcov for unit test coverage
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
------
- ./configure --enable-coverage
- make
- make check
- # or--- make test-full ? make test-full-online?
- mkdir coverage-output
- ./scripts/test/coverage coverage-output
------
-
-(On OSX, you'll need to start with "--enable-coverage CC=clang".)
-
-Then, look at the .gcov files in coverage-output. '-' before a line means
-that the compiler generated no code for that line. '######' means that the
-line was never reached. Lines with numbers were called that number of times.
-
-If that doesn't work:
- * Try configuring Tor with --disable-gcc-hardening
- * You might need to run 'make clean' after you run './configure'.
-
-If you make changes to Tor and want to get another set of coverage results,
-you can run "make reset-gcov" to clear the intermediary gcov output.
-
-If you have two different "coverage-output" directories, and you want to see
-a meaningful diff between them, you can run:
-
------
- ./scripts/test/cov-diff coverage-output1 coverage-output2 | less
------
-
-In this diff, any lines that were visited at least once will have coverage
-"1". This lets you inspect what you (probably) really want to know: which
-untested lines were changed? Are there any new untested lines?
-
-Running integration tests
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-We have the beginnings of a set of scripts to run integration tests using
-Chutney. To try them, set CHUTNEY_PATH to your chutney source directory, and
-run "make test-network".
-
-We also have scripts to run integration tests using Stem. To try them, set
-STEM_SOURCE_DIR to your Stem source directory, and run "test-stem".
-
-Profiling Tor with oprofile
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The oprofile tool runs (on Linux only!) to tell you what functions Tor is
-spending its CPU time in, so we can identify berformance pottlenecks.
-
-Here are some basic instructions
-
- - Build tor with debugging symbols (you probably already have, unless
- you messed with CFLAGS during the build process).
- - Build all the libraries you care about with debugging symbols
- (probably you only care about libssl, maybe zlib and Libevent).
- - Copy this tor to a new directory
- - Copy all the libraries it uses to that dir too (ldd ./tor will
- tell you)
- - Set LD_LIBRARY_PATH to include that dir. ldd ./tor should now
- show you it's using the libs in that dir
- - Run that tor
- - Reset oprofiles counters/start it
- * "opcontrol --reset; opcontrol --start", if Nick remembers right.
- - After a while, have it dump the stats on tor and all the libs
- in that dir you created.
- * "opcontrol --dump;"
- * "opreport -l that_dir/*"
- - Profit
-
-Generating and analyzing a callgraph
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-1. Run ./scripts/maint/generate_callgraph.sh . This will generate a
- bunch of files in a new ./callgraph directory.
-
-2. Run ./scripts/maint/analyze_callgraph.py callgraph/src/*/* . This
- will do a lot of graph operations and then dump out a new
- "callgraph.pkl" file, containing data in Python's "pickle" format.
-
-3. Run ./scripts/maint/display_callgraph.py . It will display:
- - the number of functions reachable from each function.
- - all strongly-connnected components in the Tor callgraph
- - the largest bottlenecks in the largest SCC in the Tor callgraph.
-
-Note that currently the callgraph generator can't detect calls that pass
-through function pointers.
-
-Coding conventions
-------------------
-
-Patch checklist
-~~~~~~~~~~~~~~~
-
-If possible, send your patch as one of these (in descending order of
-preference)
-
- - A git branch we can pull from
- - Patches generated by git format-patch
- - A unified diff
-
-Did you remember...
-
- - To build your code while configured with --enable-gcc-warnings?
- - To run "make check-spaces" on your code?
- - To run "make check-docs" to see whether all new options are on
- the manpage?
- - To write unit tests, as possible?
- - To base your code on the appropriate branch?
- - To include a file in the "changes" directory as appropriate?
-
-Whitespace and C conformance
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Invoke "make check-spaces" from time to time, so it can tell you about
-deviations from our C whitespace style. Generally, we use:
-
- - Unix-style line endings
- - K&R-style indentation
- - No space before newlines
- - A blank line at the end of each file
- - Never more than one blank line in a row
- - Always spaces, never tabs
- - No more than 79-columns per line.
- - Two spaces per indent.
- - A space between control keywords and their corresponding paren
- "if (x)", "while (x)", and "switch (x)", never "if(x)", "while(x)", or
- "switch(x)".
- - A space between anything and an open brace.
- - No space between a function name and an opening paren. "puts(x)", not
- "puts (x)".
- - Function declarations at the start of the line.
-
-We try hard to build without warnings everywhere. In particular, if you're
-using gcc, you should invoke the configure script with the option
-"--enable-gcc-warnings". This will give a bunch of extra warning flags to
-the compiler, and help us find divergences from our preferred C style.
-
-Getting emacs to edit Tor source properly
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Nick likes to put the following snippet in his .emacs file:
-
------
- (add-hook 'c-mode-hook
- (lambda ()
- (font-lock-mode 1)
- (set-variable 'show-trailing-whitespace t)
-
- (let ((fname (expand-file-name (buffer-file-name))))
- (cond
- ((string-match "^/home/nickm/src/libevent" fname)
- (set-variable 'indent-tabs-mode t)
- (set-variable 'c-basic-offset 4)
- (set-variable 'tab-width 4))
- ((string-match "^/home/nickm/src/tor" fname)
- (set-variable 'indent-tabs-mode nil)
- (set-variable 'c-basic-offset 2))
- ((string-match "^/home/nickm/src/openssl" fname)
- (set-variable 'indent-tabs-mode t)
- (set-variable 'c-basic-offset 8)
- (set-variable 'tab-width 8))
- ))))
------
-
-You'll note that it defaults to showing all trailing whitespace. The "cond"
-test detects whether the file is one of a few C free software projects that I
-often edit, and sets up the indentation level and tab preferences to match
-what they want.
-
-If you want to try this out, you'll need to change the filename regex
-patterns to match where you keep your Tor files.
-
-If you use emacs for editing Tor and nothing else, you could always just say:
-
------
- (add-hook 'c-mode-hook
- (lambda ()
- (font-lock-mode 1)
- (set-variable 'show-trailing-whitespace t)
- (set-variable 'indent-tabs-mode nil)
- (set-variable 'c-basic-offset 2)))
------
-
-There is probably a better way to do this. No, we are probably not going
-to clutter the files with emacs stuff.
-
-
-Functions to use
-~~~~~~~~~~~~~~~~
-
-We have some wrapper functions like tor_malloc, tor_free, tor_strdup, and
-tor_gettimeofday; use them instead of their generic equivalents. (They
-always succeed or exit.)
-
-You can get a full list of the compatibility functions that Tor provides by
-looking through src/common/util.h and src/common/compat.h. You can see the
-available containers in src/common/containers.h. You should probably
-familiarize yourself with these modules before you write too much code, or
-else you'll wind up reinventing the wheel.
-
-Use 'INLINE' instead of 'inline', so that we work properly on Windows.
-
-Calling and naming conventions
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Whenever possible, functions should return -1 on error and 0 on success.
-
-For multi-word identifiers, use lowercase words combined with
-underscores. (e.g., "multi_word_identifier"). Use ALL_CAPS for macros and
-constants.
-
-Typenames should end with "_t".
-
-Function names should be prefixed with a module name or object name. (In
-general, code to manipulate an object should be a module with the same name
-as the object, so it's hard to tell which convention is used.)
-
-Functions that do things should have imperative-verb names
-(e.g. buffer_clear, buffer_resize); functions that return booleans should
-have predicate names (e.g. buffer_is_empty, buffer_needs_resizing).
-
-If you find that you have four or more possible return code values, it's
-probably time to create an enum. If you find that you are passing three or
-more flags to a function, it's probably time to create a flags argument that
-takes a bitfield.
-
-What To Optimize
-~~~~~~~~~~~~~~~~
-
-Don't optimize anything if it's not in the critical path. Right now, the
-critical path seems to be AES, logging, and the network itself. Feel free to
-do your own profiling to determine otherwise.
-
-Log conventions
-~~~~~~~~~~~~~~~
-
-https://www.torproject.org/docs/faq#LogLevel
-
-No error or warning messages should be expected during normal OR or OP
-operation.
-
-If a library function is currently called such that failure always means ERR,
-then the library function should log WARN and let the caller log ERR.
-
-Every message of severity INFO or higher should either (A) be intelligible
-to end-users who don't know the Tor source; or (B) somehow inform the
-end-users that they aren't expected to understand the message (perhaps
-with a string like "internal error"). Option (A) is to be preferred to
-option (B).
-
-Doxygen
-~~~~~~~~
-
-We use the 'doxygen' utility to generate documentation from our
-source code. Here's how to use it:
-
- 1. Begin every file that should be documented with
- /**
- * \file filename.c
- * \brief Short description of the file.
- **/
-
- (Doxygen will recognize any comment beginning with /** as special.)
-
- 2. Before any function, structure, #define, or variable you want to
- document, add a comment of the form:
-
- /** Describe the function's actions in imperative sentences.
- *
- * Use blank lines for paragraph breaks
- * - and
- * - hyphens
- * - for
- * - lists.
- *
- * Write <b>argument_names</b> in boldface.
- *
- * \code
- * place_example_code();
- * between_code_and_endcode_commands();
- * \endcode
- */
-
- 3. Make sure to escape the characters "<", ">", "\", "%" and "#" as "\<",
- "\>", "\\", "\%", and "\#".
-
- 4. To document structure members, you can use two forms:
-
- struct foo {
- /** You can put the comment before an element; */
- int a;
- int b; /**< Or use the less-than symbol to put the comment
- * after the element. */
- };
-
- 5. To generate documentation from the Tor source code, type:
-
- $ doxygen -g
-
- To generate a file called 'Doxyfile'. Edit that file and run
- 'doxygen' to generate the API documentation.
-
- 6. See the Doxygen manual for more information; this summary just
- scratches the surface.
-
-Doxygen comment conventions
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Say what functions do as a series of one or more imperative sentences, as
-though you were telling somebody how to be the function. In other words, DO
-NOT say:
-
- /** The strtol function parses a number.
- *
- * nptr -- the string to parse. It can include whitespace.
- * endptr -- a string pointer to hold the first thing that is not part
- * of the number, if present.
- * base -- the numeric base.
- * returns: the resulting number.
- */
- long strtol(const char *nptr, char **nptr, int base);
-
-Instead, please DO say:
-
- /** Parse a number in radix <b>base</b> from the string <b>nptr</b>,
- * and return the result. Skip all leading whitespace. If
- * <b>endptr</b> is not NULL, set *<b>endptr</b> to the first character
- * after the number parsed.
- **/
- long strtol(const char *nptr, char **nptr, int base);
-
-Doxygen comments are the contract in our abstraction-by-contract world: if
-the functions that call your function rely on it doing something, then your
-function should mention that it does that something in the documentation. If
-you rely on a function doing something beyond what is in its documentation,
-then you should watch out, or it might do something else later.
-
-Putting out a new release
--------------------------
-
-Here are the steps Roger takes when putting out a new Tor release:
-
-1) Use it for a while, as a client, as a relay, as a hidden service,
-and as a directory authority. See if it has any obvious bugs, and
-resolve those.
-
-1.5) As applicable, merge the maint-X branch into the release-X branch.
-
-2) Gather the changes/* files into a changelog entry, rewriting many
-of them and reordering to focus on what users and funders would find
-interesting and understandable.
-
- 2.1) Make sure that everything that wants a bug number has one.
- Make sure that everything which is a bugfix says what version
- it was a bugfix on.
- 2.2) Concatenate them.
- 2.3) Sort them by section. Within each section, sort by "version it's
- a bugfix on", else by numerical ticket order.
-
- 2.4) Clean them up:
-
- Standard idioms:
- "Fixes bug 9999; bugfix on 0.3.3.3-alpha."
-
- One space after a period.
-
- Make stuff very terse
-
- Make sure each section name ends with a colon
-
- Describe the user-visible problem right away
-
- Mention relevant config options by name. If they're rare or unusual,
- remind people what they're for
-
- Avoid starting lines with open-paren
-
- Present and imperative tense: not past.
-
- 'Relays', not 'servers' or 'nodes' or 'Tor relays'.
-
- "Stop FOOing", not "Fix a bug where we would FOO".
-
- Try not to let any given section be longer than about a page. Break up
- long sections into subsections by some sort of common subtopic. This
- guideline is especially important when organizing Release Notes for
- new stable releases.
-
- If a given changes stanza showed up in a different release (e.g.
- maint-0.2.1), be sure to make the stanzas identical (so people can
- distinguish if these are the same change).
-
- 2.5) Merge them in.
-
- 2.6) Clean everything one last time.
-
- 2.7) Run ./scripts/maint/format_changelog.py to make it prettier.
-
-3) Compose a short release blurb to highlight the user-facing
-changes. Insert said release blurb into the ChangeLog stanza. If it's
-a stable release, add it to the ReleaseNotes file too. If we're adding
-to a release-0.2.x branch, manually commit the changelogs to the later
-git branches too.
-
-4) In maint-0.2.x, bump the version number in configure.ac and run
- scripts/maint/updateVersions.pl to update version numbers in other
- places, and commit. Then merge maint-0.2.x into release-0.2.x.
-
- (NOTE: TO bump the version number, edit configure.ac, and then run
- either make, or 'perl scripts/maint/updateVersions.pl', depending on
- your version.)
-
-5) Make dist, put the tarball up somewhere, and tell #tor about it. Wait
-a while to see if anybody has problems building it. Try to get Sebastian
-or somebody to try building it on Windows.
-
-6) Get at least two of weasel/arma/sebastian to put the new version number
-in their approved versions list.
-
-7) Sign the tarball, then sign and push the git tag:
- gpg -ba <the_tarball>
- git tag -u <keyid> tor-0.2.x.y-status
- git push origin tag tor-0.2.x.y-status
-
-8a) scp the tarball and its sig to the dist website, i.e.
-/srv/dist-master.torproject.org/htdocs/ on dist-master. When you want
-it to go live, you run "static-update-component dist.torproject.org"
-on dist-master.
-
-8b) Edit "include/versions.wmi" and "Makefile" to note the new version.
-
-9) Email the packagers (cc'ing tor-assistants) that a new tarball is up.
- The current list of packagers is:
- {weasel,gk,mikeperry} at torproject dot org
- {blueness} at gentoo dot org
- {paul} at invizbox dot io
- {ondrej.mikle} at gmail dot com
- {lfleischer} at archlinux dot org
-
-10) Add the version number to Trac. To do this, go to Trac, log in,
-select "Admin" near the top of the screen, then select "Versions" from
-the menu on the left. At the right, there will be an "Add version"
-box. By convention, we enter the version in the form "Tor:
-0.2.2.23-alpha" (or whatever the version is), and we select the date as
-the date in the ChangeLog.
-
-11) Forward-port the ChangeLog.
-
-12) Wait up to a day or two (for a development release), or until most
-packages are up (for a stable release), and mail the release blurb and
-changelog to tor-talk or tor-announce.
-
- (We might be moving to faster announcements, but don't announce until
- the website is at least updated.)
-
-13) If it's a stable release, bump the version number in the maint-x.y.z
- branch to "newversion-dev", and do a "merge -s ours" merge to avoid
- taking that change into master. Do a similar 'merge -s theirs'
- merge to get the change (and only that change) into release. (Some
- of the build scripts require that maint merge cleanly into release.)
-
diff --git a/doc/HACKING/HelpfulTools.txt b/doc/HACKING/HelpfulTools.txt
new file mode 100644
index 0000000..f5a0c97
--- /dev/null
+++ b/doc/HACKING/HelpfulTools.txt
@@ -0,0 +1,303 @@
+Useful tools
+------------
+
+These aren't strictly necessary for hacking on Tor, but they can help track
+down bugs.
+
+Jenkins
+~~~~~~~
+
+https://jenkins.torproject.org
+
+Dmalloc
+~~~~~~~
+
+The dmalloc library will keep track of memory allocation, so you can find out
+if we're leaking memory, doing any double-frees, or so on.
+
+ dmalloc -l ~/dmalloc.log
+ (run the commands it tells you)
+ ./configure --with-dmalloc
+
+Valgrind
+~~~~~~~~
+
+valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor
+
+(Note that if you get a zillion openssl warnings, you will also need to
+pass --undef-value-errors=no to valgrind, or rebuild your openssl
+with -DPURIFY.)
+
+Coverity
+~~~~~~~~
+
+Nick regularly runs the coverity static analyzer on the Tor codebase.
+
+The preprocessor define __COVERITY__ is used to work around instances
+where coverity picks up behavior that we wish to permit.
+
+clang Static Analyzer
+~~~~~~~~~~~~~~~~~~~~~
+
+The clang static analyzer can be run on the Tor codebase using Xcode (WIP)
+or a command-line build.
+
+The preprocessor define __clang_analyzer__ is used to work around instances
+where clang picks up behavior that we wish to permit.
+
+clang Runtime Sanitizers
+~~~~~~~~~~~~~~~~
+
+To build the Tor codebase with the clang Address and Undefined Behavior
+sanitizers, see the file contrib/clang/sanitize_blacklist.txt.
+
+Preprocessor workarounds for instances where clang picks up behavior that
+we wish to permit are also documented in the blacklist file.
+
+Running lcov for unit test coverage
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Lcov is a utility that generates pretty HTML reports of test code coverage.
+To generate such a report:
+
+-----
+ ./configure --enable-coverage
+ make
+ make coverage-html
+ $BROWSER ./coverage_html/index.html
+-----
+
+This will run the tor unit test suite `./src/test/test` and generate the HTML
+coverage code report under the directory ./coverage_html/. To change the
+output directory, use `make coverage-html HTML_COVER_DIR=./funky_new_cov_dir`.
+
+Coverage diffs using lcov are not currently implemented, but are being
+investigated (as of July 2014).
+
+Running the unit tests
+~~~~~~~~~~~~~~~~~~~~~~
+
+To quickly run all the tests distributed with Tor:
+-----
+ make check
+-----
+
+To run the fast unit tests only:
+-----
+ make test
+-----
+
+To selectively run just some tests (the following can be combined
+arbitrarily):
+-----
+ ./src/test/test <name_of_test> [<name of test 2>] ...
+ ./src/test/test <prefix_of_name_of_test>.. [<prefix_of_name_of_test2>..] ...
+ ./src/test/test :<name_of_excluded_test> [:<name_of_excluded_test2]...
+-----
+
+To run all tests, including those based on Stem or Chutney:
+-----
+ make test-full
+-----
+
+To run all tests, including those basedd on Stem or Chutney that require a
+working connection to the internet:
+-----
+ make test-full-online
+-----
+
+Running gcov for unit test coverage
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+-----
+ ./configure --enable-coverage
+ make
+ make check
+ # or--- make test-full ? make test-full-online?
+ mkdir coverage-output
+ ./scripts/test/coverage coverage-output
+-----
+
+(On OSX, you'll need to start with "--enable-coverage CC=clang".)
+
+Then, look at the .gcov files in coverage-output. '-' before a line means
+that the compiler generated no code for that line. '######' means that the
+line was never reached. Lines with numbers were called that number of times.
+
+If that doesn't work:
+ * Try configuring Tor with --disable-gcc-hardening
+ * You might need to run 'make clean' after you run './configure'.
+
+If you make changes to Tor and want to get another set of coverage results,
+you can run "make reset-gcov" to clear the intermediary gcov output.
+
+If you have two different "coverage-output" directories, and you want to see
+a meaningful diff between them, you can run:
+
+-----
+ ./scripts/test/cov-diff coverage-output1 coverage-output2 | less
+-----
+
+In this diff, any lines that were visited at least once will have coverage
+"1". This lets you inspect what you (probably) really want to know: which
+untested lines were changed? Are there any new untested lines?
+
+Running integration tests
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We have the beginnings of a set of scripts to run integration tests using
+Chutney. To try them, set CHUTNEY_PATH to your chutney source directory, and
+run "make test-network".
+
+We also have scripts to run integration tests using Stem. To try them, set
+STEM_SOURCE_DIR to your Stem source directory, and run "test-stem".
+
+Profiling Tor with oprofile
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The oprofile tool runs (on Linux only!) to tell you what functions Tor is
+spending its CPU time in, so we can identify berformance pottlenecks.
+
+Here are some basic instructions
+
+ - Build tor with debugging symbols (you probably already have, unless
+ you messed with CFLAGS during the build process).
+ - Build all the libraries you care about with debugging symbols
+ (probably you only care about libssl, maybe zlib and Libevent).
+ - Copy this tor to a new directory
+ - Copy all the libraries it uses to that dir too (ldd ./tor will
+ tell you)
+ - Set LD_LIBRARY_PATH to include that dir. ldd ./tor should now
+ show you it's using the libs in that dir
+ - Run that tor
+ - Reset oprofiles counters/start it
+ * "opcontrol --reset; opcontrol --start", if Nick remembers right.
+ - After a while, have it dump the stats on tor and all the libs
+ in that dir you created.
+ * "opcontrol --dump;"
+ * "opreport -l that_dir/*"
+ - Profit
+
+Generating and analyzing a callgraph
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. Run ./scripts/maint/generate_callgraph.sh . This will generate a
+ bunch of files in a new ./callgraph directory.
+
+2. Run ./scripts/maint/analyze_callgraph.py callgraph/src/*/* . This
+ will do a lot of graph operations and then dump out a new
+ "callgraph.pkl" file, containing data in Python's "pickle" format.
+
+3. Run ./scripts/maint/display_callgraph.py . It will display:
+ - the number of functions reachable from each function.
+ - all strongly-connnected components in the Tor callgraph
+ - the largest bottlenecks in the largest SCC in the Tor callgraph.
+
+Note that currently the callgraph generator can't detect calls that pass
+through function pointers.
+
+Getting emacs to edit Tor source properly
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Nick likes to put the following snippet in his .emacs file:
+
+-----
+ (add-hook 'c-mode-hook
+ (lambda ()
+ (font-lock-mode 1)
+ (set-variable 'show-trailing-whitespace t)
+
+ (let ((fname (expand-file-name (buffer-file-name))))
+ (cond
+ ((string-match "^/home/nickm/src/libevent" fname)
+ (set-variable 'indent-tabs-mode t)
+ (set-variable 'c-basic-offset 4)
+ (set-variable 'tab-width 4))
+ ((string-match "^/home/nickm/src/tor" fname)
+ (set-variable 'indent-tabs-mode nil)
+ (set-variable 'c-basic-offset 2))
+ ((string-match "^/home/nickm/src/openssl" fname)
+ (set-variable 'indent-tabs-mode t)
+ (set-variable 'c-basic-offset 8)
+ (set-variable 'tab-width 8))
+ ))))
+-----
+
+You'll note that it defaults to showing all trailing whitespace. The "cond"
+test detects whether the file is one of a few C free software projects that I
+often edit, and sets up the indentation level and tab preferences to match
+what they want.
+
+If you want to try this out, you'll need to change the filename regex
+patterns to match where you keep your Tor files.
+
+If you use emacs for editing Tor and nothing else, you could always just say:
+
+-----
+ (add-hook 'c-mode-hook
+ (lambda ()
+ (font-lock-mode 1)
+ (set-variable 'show-trailing-whitespace t)
+ (set-variable 'indent-tabs-mode nil)
+ (set-variable 'c-basic-offset 2)))
+-----
+
+There is probably a better way to do this. No, we are probably not going
+to clutter the files with emacs stuff.
+
+
+Doxygen
+~~~~~~~~
+
+We use the 'doxygen' utility to generate documentation from our
+source code. Here's how to use it:
+
+ 1. Begin every file that should be documented with
+ /**
+ * \file filename.c
+ * \brief Short description of the file.
+ **/
+
+ (Doxygen will recognize any comment beginning with /** as special.)
+
+ 2. Before any function, structure, #define, or variable you want to
+ document, add a comment of the form:
+
+ /** Describe the function's actions in imperative sentences.
+ *
+ * Use blank lines for paragraph breaks
+ * - and
+ * - hyphens
+ * - for
+ * - lists.
+ *
+ * Write <b>argument_names</b> in boldface.
+ *
+ * \code
+ * place_example_code();
+ * between_code_and_endcode_commands();
+ * \endcode
+ */
+
+ 3. Make sure to escape the characters "<", ">", "\", "%" and "#" as "\<",
+ "\>", "\\", "\%", and "\#".
+
+ 4. To document structure members, you can use two forms:
+
+ struct foo {
+ /** You can put the comment before an element; */
+ int a;
+ int b; /**< Or use the less-than symbol to put the comment
+ * after the element. */
+ };
+
+ 5. To generate documentation from the Tor source code, type:
+
+ $ doxygen -g
+
+ To generate a file called 'Doxyfile'. Edit that file and run
+ 'doxygen' to generate the API documentation.
+
+ 6. See the Doxygen manual for more information; this summary just
+ scratches the surface.
+
diff --git a/doc/HACKING/ReleasingTor.txt b/doc/HACKING/ReleasingTor.txt
new file mode 100644
index 0000000..8321ac5
--- /dev/null
+++ b/doc/HACKING/ReleasingTor.txt
@@ -0,0 +1,124 @@
+
+Putting out a new release
+-------------------------
+
+Here are the steps Roger takes when putting out a new Tor release:
+
+1) Use it for a while, as a client, as a relay, as a hidden service,
+and as a directory authority. See if it has any obvious bugs, and
+resolve those.
+
+1.5) As applicable, merge the maint-X branch into the release-X branch.
+
+2) Gather the changes/* files into a changelog entry, rewriting many
+of them and reordering to focus on what users and funders would find
+interesting and understandable.
+
+ 2.1) Make sure that everything that wants a bug number has one.
+ Make sure that everything which is a bugfix says what version
+ it was a bugfix on.
+ 2.2) Concatenate them.
+ 2.3) Sort them by section. Within each section, sort by "version it's
+ a bugfix on", else by numerical ticket order.
+
+ 2.4) Clean them up:
+
+ Standard idioms:
+ "Fixes bug 9999; bugfix on 0.3.3.3-alpha."
+
+ One space after a period.
+
+ Make stuff very terse
+
+ Make sure each section name ends with a colon
+
+ Describe the user-visible problem right away
+
+ Mention relevant config options by name. If they're rare or unusual,
+ remind people what they're for
+
+ Avoid starting lines with open-paren
+
+ Present and imperative tense: not past.
+
+ 'Relays', not 'servers' or 'nodes' or 'Tor relays'.
+
+ "Stop FOOing", not "Fix a bug where we would FOO".
+
+ Try not to let any given section be longer than about a page. Break up
+ long sections into subsections by some sort of common subtopic. This
+ guideline is especially important when organizing Release Notes for
+ new stable releases.
+
+ If a given changes stanza showed up in a different release (e.g.
+ maint-0.2.1), be sure to make the stanzas identical (so people can
+ distinguish if these are the same change).
+
+ 2.5) Merge them in.
+
+ 2.6) Clean everything one last time.
+
+ 2.7) Run ./scripts/maint/format_changelog.py to make it prettier.
+
+3) Compose a short release blurb to highlight the user-facing
+changes. Insert said release blurb into the ChangeLog stanza. If it's
+a stable release, add it to the ReleaseNotes file too. If we're adding
+to a release-0.2.x branch, manually commit the changelogs to the later
+git branches too.
+
+4) In maint-0.2.x, bump the version number in configure.ac and run
+ scripts/maint/updateVersions.pl to update version numbers in other
+ places, and commit. Then merge maint-0.2.x into release-0.2.x.
+
+ (NOTE: TO bump the version number, edit configure.ac, and then run
+ either make, or 'perl scripts/maint/updateVersions.pl', depending on
+ your version.)
+
+5) Make dist, put the tarball up somewhere, and tell #tor about it. Wait
+a while to see if anybody has problems building it. Try to get Sebastian
+or somebody to try building it on Windows.
+
+6) Get at least two of weasel/arma/sebastian to put the new version number
+in their approved versions list.
+
+7) Sign the tarball, then sign and push the git tag:
+ gpg -ba <the_tarball>
+ git tag -u <keyid> tor-0.2.x.y-status
+ git push origin tag tor-0.2.x.y-status
+
+8a) scp the tarball and its sig to the dist website, i.e.
+/srv/dist-master.torproject.org/htdocs/ on dist-master. When you want
+it to go live, you run "static-update-component dist.torproject.org"
+on dist-master.
+
+8b) Edit "include/versions.wmi" and "Makefile" to note the new version.
+
+9) Email the packagers (cc'ing tor-assistants) that a new tarball is up.
+ The current list of packagers is:
+ {weasel,gk,mikeperry} at torproject dot org
+ {blueness} at gentoo dot org
+ {paul} at invizbox dot io
+ {ondrej.mikle} at gmail dot com
+ {lfleischer} at archlinux dot org
+
+10) Add the version number to Trac. To do this, go to Trac, log in,
+select "Admin" near the top of the screen, then select "Versions" from
+the menu on the left. At the right, there will be an "Add version"
+box. By convention, we enter the version in the form "Tor:
+0.2.2.23-alpha" (or whatever the version is), and we select the date as
+the date in the ChangeLog.
+
+11) Forward-port the ChangeLog.
+
+12) Wait up to a day or two (for a development release), or until most
+packages are up (for a stable release), and mail the release blurb and
+changelog to tor-talk or tor-announce.
+
+ (We might be moving to faster announcements, but don't announce until
+ the website is at least updated.)
+
+13) If it's a stable release, bump the version number in the maint-x.y.z
+ branch to "newversion-dev", and do a "merge -s ours" merge to avoid
+ taking that change into master. Do a similar 'merge -s theirs'
+ merge to get the change (and only that change) into release. (Some
+ of the build scripts require that maint merge cleanly into release.)
diff --git a/doc/include.am b/doc/include.am
index 5decfe5..819d0c1 100644
--- a/doc/include.am
+++ b/doc/include.am
@@ -39,9 +39,11 @@ EXTRA_DIST+= doc/asciidoc-helper.sh \
doc/state-contents.txt \
doc/torrc_format.txt \
doc/TUNING \
- doc/HACKING/WritingTests.txt \
+ doc/HACKING/CodingStandards.txt \
doc/HACKING/GettingStarted.txt \
- doc/HACKING/HACKING_old
+ doc/HACKING/HelpfulTools.txt \
+ doc/HACKING/ReleasingTor.txt \
+ doc/HACKING/WritingTests.txt
docdir = @docdir@
More information about the tor-commits
mailing list