- tor-commits - lists.torproject.org

r24746: {website} Asking org preference on the GSoC app template to help with (website/trunk/about/en)
by Damian Johnson 13 May '11

13 May '11

Author: atagar Date: 2011-05-13 16:10:10 +0000 (Fri, 13 May 2011) New Revision: 24746 Modified: website/trunk/about/en/gsoc.wml Log: Asking org preference on the GSoC app template to help with deduplication next year Modified: website/trunk/about/en/gsoc.wml =================================================================== --- website/trunk/about/en/gsoc.wml 2011-05-13 15:51:39 UTC (rev 24745) +++ website/trunk/about/en/gsoc.wml 2011-05-13 16:10:10 UTC (rev 24746) @@ -166,6 +166,11 @@ Interacting with us on IRC will help us get to know you, and help you get to know our community.</li> + <li>Are you applying to other projects for GSoC and, if so, what would be + your preference if you're accepted to both? Having a stated preference + helps with the deduplication process and will not impact if we accept your + application or not.</li> + <li>Is there anything else we should know that will make us like your project more?</li>

1 0

r24745: {website} Adding gsoc proposals for this year's students (in website/trunk/about: en gsocProposal)
by Damian Johnson 13 May '11

13 May '11

Author: atagar Date: 2011-05-13 15:51:39 +0000 (Fri, 13 May 2011) New Revision: 24745 Added: website/trunk/about/gsocProposal/gsoc11-proposal-metadataToolkit.txt Modified: website/trunk/about/en/gsoc.wml Log: Adding gsoc proposals for this year's students Modified: website/trunk/about/en/gsoc.wml =================================================================== --- website/trunk/about/en/gsoc.wml 2011-05-12 14:09:44 UTC (rev 24744) +++ website/trunk/about/en/gsoc.wml 2011-05-13 15:51:39 UTC (rev 24745) @@ -212,6 +212,10 @@ <li><h4><a href="http://tor.spanning-tree.org/proposal.html">DNSEL Rewrite</a> by Harry Bock</h4></li> <li><h4><a href="http://kjb.homeunix.com/wp-content/uploads/2010/05/KevinBerry-GSoC2010-TorP…">Extending Tor Network Metrics</a> by Kevin Berry</h4></li> <li><h4><a href="../about/gsocProposal/gsoc10-proposal-soat.txt">SOAT Expansion</a> by John Schanck</h4></li> + <li><h4><a href="http://inspirated.com/uploads/tor-gsoc-11.pdf">GTK+ Frontend and Client Mode Improvements for arm</a> by Kamran Khan</h4></li> + <li><h4><a href="http://www.gsathya.in/gsoc11.html">Orbot + ORLib</a> by Sathya Gunasekaran</h4></li> + <li><h4><a href="http://blanu.net/TorSummerOfCodeProposal.pdf">Blocking-resistant Transport Evaluation Framework</a> by Brandon Wiley</h4></li> + <li><h4><a href="../about/gsocProposal/gsoc11-proposal-metadataToolkit.txt">Metadata Anonymisation Toolkit</a> by Julien Voisin</h4></li> <li><h4><a href="http://www.atagar.com/misc/gsocBlog09/">Website Pootle Translation</a> by Damian Johnson</h4></li> </ul> Added: website/trunk/about/gsocProposal/gsoc11-proposal-metadataToolkit.txt =================================================================== --- website/trunk/about/gsocProposal/gsoc11-proposal-metadataToolkit.txt (rev 0) +++ website/trunk/about/gsocProposal/gsoc11-proposal-metadataToolkit.txt 2011-05-13 15:51:39 UTC (rev 24745) @@ -0,0 +1,231 @@ +Hello, +I am Julien Voisin, undergraduate computer science student from France. + +I am interested to work on the “Meta-data anonymizing toolkit for file publication” project. +I know there is already a student interested in by this project, but I really want to do it : +I needed it for my own and have already thought about a potential design some time ago. + +I would like to work for the EFF, because I am very concerned about privacy issues on the Internet. +I think privacy is an essential right, and not just an option. +Especially, I would really enjoy to work for the Tor project (or Tails, since it's heavily based on him). +I am using it for quite some time and would like to get more involved and contribute back! + +I use F/OSS on a daily basis (Ubuntu, Debian, Archlinux and Gentoo). +So far my major contributions were the writing of documentations for archLinux, openmw, xda-forum and Ubuntu. +Recently I have released a little matrix manipulation library written in C, +originally for an academic project (http://dustri.org/lib/) +I am interested to do the debian package, but heard that it can be quite tricky, so a little +help would be much appreciated + +I do not have any major plan for this summer (but my holidays only begins the june 4th), so I can fully focus on the project and reasonably think that I could commit 5-6 hours per day on it. + +Requirement/Deliverables: + + * A command line and a GUI tool having both the following capabilities (in order of importance): + o Listing the metadatas embedded in a given file + o A batch mode to handle a whole directory (or set of directories) + o The ability to scan files packed in the most common archive formats + o A nice binding for srm (Secure ReMoval) or shred (GNU utils) to properly remove the original file containing the evil metas + o Let the user delete/modify a specific meta + + * Should run on the most common OS/architectures (And especially on Debian Squeeze, since Tails is based on it.) + * The whole thing should be easily extensible (especially it should be easy to add support for new file formats) + * The proper functioning of the software should be easily testable + +I'd like to do this project in Python, because I already have done some personal projects whith it (for which I also used subversion) : an IRC bot tailored for logging (dustri.org/tor/depluie.tar.bz2 still under heavy WIP), a battery monitor, a simple search engine indexing FTP servers, ... + +Why is Python a good choice for implementing this project ? + + 1. I am experienced with the language + 2. There are plenty of libraries to read/write metadatas, among them is Hachoir (https://bitbucket.org/haypo/hachoir/)that looks very promising since it supports quite a few file formats + 3. It is easy to wrap other libraries for our needs (even if they are not written in Python !) + 4. Runs on almost every OS/architecture, what is a great benefit for portability + 5. It is easy to make unit tests (thanks to the built-in Unittest module) + +Proposed design: + +The proposed design has three main components : one lib, a command line tool and a GUI tool. + +The aim of the library (described with more details in the next part) is to make the development of tools easy. A special attention will be made on the API that it exposes. The ultimate goal being to be able to add the support of new file format in the library without changing the whole code of the tools. + +Meta reading/writing library : + +A library to read and write metas for various file formats. The main goal is to provide an abstraction interface (for the file format and for the underlying libraries used). +At first it would only wrap Hachoir. +Why hachoir : + + * Autofix: Hachoir is able to open invalid / truncated files + * Lazy: Open a file is very fast since no information is read from file, data are read and/or computed when the user ask for it + + * Types: Hachoir has many predefined field types (integer, bit, string, etc.) and supports string with charset (ISO-8859-1, UTF-8, UTF-16, ...) + * Addresses and sizes are stored in bit, so flags are stored as classic fields + * Editor: Using Hachoir representation of data, you can edit, insert, remove data and then save in a new file. + * Meta : Support a very large scale of file format + +But we could also wrap other libraries to support a particular file format. Or write ourself the support for a format, although this should be avoided if possible (it looks simple at first, but supporting different versions of the format and maintaining the thing over time is extremely time consuming) +The must would be to make the children libraries optional dependencies. + +One typical use case of the lib is to ask for metadatas for a file, if the format is supported a list (or maybe a tree) of metas is returned. + +Both the GUI and the cmdline tool will use this lib. + +The cmdline/GUI tool features: + + * List all the meta + * Removing all the meta + * Anonymising all the meta + * Let the user chose wich meta he wants to modify + * Support archives anonymisation + * Secure removal + * Cleaning wholes folder recursively + +GUI: +Essentially the GUI tool would do the same features as for the cmd line too. +I do not have a significant GUI development experience, but I'm planing to fix that point during community bonding period. + +Proposed development methods: +One way to develop this would be to do it layer by layer : first implementing the meta reading/writing lib for ale the formats in one shot, then making the command line application, … + +However for this project, developing feature by feature seems more appropriate : +Starting by a skeleton implementing a thin slice of functionality that traverses most of the layers. +For example, I could start by focussing only on EXIF metas : make sure that the meta reading/writing library supports EXIF, then make the command line tool using the previous library. +And only then, when the skeleton is actually working, add supports for other features/formats. + +This allows a more incremental development flow and after only a few weeks I would be able to deliver a working system. The list of features supported in the first iterations would be ridiculously short but at least that would enable me to get feedbacks and notice quickly if I am on the wrong tracks. +Also, since I would add one feature at a time, the structure of the system will tend to easily accommodate that. Thanks to this, adding the support of a new file format will be made easy, even after the end of the GSOC. + + +Testing +For such a tool, since the smallest crack could compromise the user, testing is critically important. I plan to implement two main kind of testings : + +Unit tests + +To test the proper working of the meta accessing library. Maintaining a collection of files with metas for every format we should support. Using Python's unittest I can setup expectations to make sure that the library will finds the good fields. + +End-to-End testing: + +A script (or set of script) to test the proper working of the command line tool. One way is to run the tool in batch mode on the input test files set. If in the output we are still able to find metas, the system is not doing his job right. + +It is possible to write this script in Python, possibly using again the unittest lib (even if here the goal is to execute an external executable and see how it's behaving) to get output of the results that is consistent with the unit tests. + +The goal is to be able to automatically test the whole system in a few commands (one for the unit tests one for the end-to-end tests). It will be easy to add a new document in the test set, so if someone from the community provides a file that was not cleaned properly, we can easily reproduce the problem and then decide on the proper action to take. The ability to tests everything easily might help a user to make sure that everything works fine on his OS/architecture. Also, if for one reason after some months it is decided to change from Hachoir to another underlying library (yes a pretty radical decision, just given as example) we still have the tests, so valuable to check than everything is still working with the new library. + +Timeline: + + * Community Bonding Period (in order of importance) + o Playing around with pygobject + o Playing with Hachoir + o Learning git + + * First two weeks : + o create the structure in the repository (directories, README, ..) + o Create a skeleton + + o Objectives : to have a deployable working system as soon as possible(even if the list of features is ridiculous). So that I can show you my work in an incremental way thereafter and get feedbacks early. + o The lib will handle reading/writing EXIF fields (using Hachoir) + o A set of tests files (and automated unit tests) to demonstrate that the lib does the job + o The beginning of the command line tool, at this point must list and delete EXIF meta + o An automated end-to-end test to show that the command line tool does properly remove the EXIF + +After this first step (making the skeleton) I should be able to deliver a working system right after adding each of the following features. I Hope to get feedbacks so can fix problems quickly + + * 3 weeks + o adding support for (in order of importance) pdf, zip/tar/bzip (just the meta, not the content yet), jpeg/png/bmp, ogg/mpeg1-2-3, exe... + o For every type of meta, that involves : + + Creating some input test files with meta data + + Implementing the feature in the library + + Asserting that the lib does the job with unit tests + + Modifying the cmd line tool to support the feature (if necessary) + + Checking that the cmd line tool can properly delete this type of meta with automated end-to-end test + + * about one day + o Enable the command line tool to set a specific meta to a chosen value + + * about 1 day + o Implementation of the “batch mode” in the cmdline tool, to clean a whole folder + o Implementation of secure removal + + * about 2 days : + o Add support for deep archive cleanup + + Clean the content of the archives + + Make a list of non supported format, for which we warn the user that only the container can be cleaned from meta, not the content (at first that will include rar, 7zip, ..) + + The supported formats will be those supported natively by Python ( bzip2, gzip, tar ) + + Create some test archives for each supported format containing various files with metas + + Implement the deep cleanup for the format + + Assert that the command line passes the end-to-end tests (that is, it can correctly clean the content of the test archives) + + * about 2 days + o Add support for complete deletion of the original files + o Make a binding nice for shred (should not be to hard using Python) + o Implement the feature in the command line tool + + * 3 weeks + o Implementation of the GUI tool + o At this stage, I can use the experience from implementing the cmdline tool to implement the GUI tool, having the same features. + + * 1 week + o Add support for more format (might be based on requests from the community) + + * Remaining weeks + o I want to keep those remaining week in case of problems, and for + + Remaining/polishing cleanup + + Bugfixing + + Integration work + + Missing features + + Packaging + + Final documentation + * Every Week-end : + o Documentation time : both end-user, and design. I do not like to document my code while I'm coding it : it slows a lot the development process, but it's not a good thing to delay it too much : week-ends seems fine for this. + o A blog-post, and a mail on the mailing list about what I have done in the week. + +About the anonymisation process : + +Since I'll relay on Hachoir in a first time, I don't know how much it's +effective on every case. +I am planing to do some tests during the Community Bonding Period. + +The plan is to first rely on the capabilities of Hachoir. I don't know yet +how effective it is for each case. I am planing to do some tests during the +Community Bonding Period. Following the test strategy described before, it +will be easy to add a new document in the test set. If I could make a test +file with metas not supported by Hachoir (or someone from the community +provide such a file), we could then decide on the proper action : propose a +patch to Hachoir or use another library for this specific format. + +Doing R&D about supporting exotics fields, or improve existing support will +depend of my progress. Speaking of the fingerprints, the subject of the +project is “metadata anonymisation”, and not “fingerprint detection” : they +are too many softwares, too many subtle ways to alter/mark a file (and I +don't even speak of steganography). + +So, nop, I'm not planing to implement fingerprinting detection. Not that it +is not interesting, it's just that it's not realistic to support it +correctly (actually it's not realistic to support it at all, given that I +must first support the metas, in such a short time frame). Would you agree +that it is better to first focus on making a working tool that does the job +for the known metadatas ? + +That done, if the design is good we could easily add support for more exotic +fields (or some kind of fingerprinting). I think we should never loose track +of the sad truth : no matter how big the effort we spend on this, making a +comprehensive tool able of detecting every kind of meta and every pattern of +fingerprinting is just not feasible. + +About archives anonymisation + +Task order for anonymisation of an archive : + + 1. Extract the archive + 2. Anonymise his content + 3. Recompress the archive + 4. Securely deleting the extracted content + 5. Anonymise the new created archive + +And I'm planing to support the extraction only of archive formats supported by python (without this limitation, the tool will not be portable). If the user try to anonymize a .rar for example, the tool will popup “WARNING - only the container will be anonymize, and not the content - WARNING” + +As for what I expect from my mentor, I think he should try to be available (not immediately but in the 48 hours) when I need him specifically (e.g. technical questions no one else on IRC can answer) but he doesn't need to check on me all the time. I'm fine with mails to (since intrigeri is not always “online”, it will be I think, the best solution to communicate with him). I'd like to do a weekly reunion, on irc or jabber, to discuss things more smoothly. + +I'm using irc quite a lot, and I'm hanging on #tor, and #tor-dev (nick : jvoisin). +I'm planing to do a blogpost every week-end, about the advancement of the project. +You can mail me at : julien.voisin(a)dustri.org for more information. +

1 0

[tor/master] Teach retry_listener about "auto" ports.
by nickm＠torproject.org 13 May '11

13 May '11

commit 61c06cbc66ecfaece20501417ef81b3f6f202391 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon May 2 15:26:39 2011 -0400 Teach retry_listener about "auto" ports. Otherwise, it will just immediately close any port declared with "auto" on the grounds that it wasn't configured. Now, it will allow "auto" to match any port. This means FWIW if you configure a socks port with SocksPort 9999 and then transition to SocksPort auto, the original socksport will not get closed and reopened. I'm considering this a feature. --- src/or/connection.c | 19 ++++++++++++++++--- 1 files changed, 16 insertions(+), 3 deletions(-) diff --git a/src/or/connection.c b/src/or/connection.c index 47e5423..bc18dab 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -1772,10 +1772,23 @@ retry_listeners(int type, config_line_t *cfg, if (!parse_addr_port(LOG_WARN, wanted->value, &address, NULL, &port)) { int addr_matches = !strcasecmp(address, conn->address); + int port_matches; tor_free(address); - if (! port) - port = port_option; - if (port == conn->port && addr_matches) { + if (port) { + /* The Listener line has a port */ + port_matches = (port == conn->port); + } else if (port_option == CFG_AUTO_PORT) { + /* The Listener line has no port, and the Port line is "auto". + * "auto" matches anything; transitions from any port to + * "auto" succeed. */ + port_matches = 1; + } else { + /* The Listener line has no port, and the Port line is "auto". + * "auto" matches anything; transitions from any port to + * "auto" succeed. */ + port_matches = (port_option == conn->port); + } + if (port_matches && addr_matches) { line = wanted; break; }

1 0

[tor/master] Correct the signature for is_listening_on_low_port for "auto" ports
by nickm＠torproject.org 13 May '11

13 May '11

commit 6f5998fd73849b98c9c1a017845c0669c32fda5d Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon May 2 15:31:49 2011 -0400 Correct the signature for is_listening_on_low_port for "auto" ports --- src/or/config.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/src/or/config.c b/src/or/config.c index 055a4b8..914c4cb 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -2623,7 +2623,7 @@ options_init(or_options_t *options) * it is, or 0 if it isn't or the concept of a low port isn't applicable for * the platform we're on. */ static int -is_listening_on_low_port(uint16_t port_option, +is_listening_on_low_port(int port_option, const config_line_t *listen_options) { #ifdef MS_WINDOWS

1 0

[tor/master] Add a new "tor_sockaddr_to_str()" function
by nickm＠torproject.org 13 May '11

13 May '11

commit 28cc7b01800e3e2a46a995b2ca18b0d2d5cdc804 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon May 2 16:43:11 2011 -0400 Add a new "tor_sockaddr_to_str()" function It does what it says on the tin. It turns out I'll want this in a couple of places. --- src/common/address.c | 30 ++++++++++++++++++++++++++++++ src/common/address.h | 1 + 2 files changed, 31 insertions(+), 0 deletions(-) diff --git a/src/common/address.c b/src/common/address.c index 34e109f..1c72539 100644 --- a/src/common/address.c +++ b/src/common/address.c @@ -43,6 +43,9 @@ #ifdef HAVE_SYS_PARAM_H #include <sys/param.h> /* FreeBSD needs this to know what version it is */ #endif +#ifdef HAVE_SYS_UN_H +#include <sys/un.h> +#endif #include <stdarg.h> #include <stdio.h> #include <stdlib.h> @@ -120,6 +123,33 @@ tor_addr_from_sockaddr(tor_addr_t *a, const struct sockaddr *sa, return 0; } +/** Return a newly allocated string holding the address described in + * <b>sa</b>. AF_UNIX, AF_UNSPEC, AF_INET, and AF_INET6 are supported. */ +char * +tor_sockaddr_to_str(const struct sockaddr *sa) +{ + char address[TOR_ADDR_BUF_LEN]; + char *result; + tor_addr_t addr; + uint16_t port; +#ifdef HAVE_SYS_UN_H + if (sa->sa_family == AF_UNIX) { + struct sockaddr_un *s_un = (struct sockaddr_un *)sa; + tor_asprintf(&result, "unix:%s", s_un->sun_path); + return result; + } +#endif + if (sa->sa_family == AF_UNSPEC) + return tor_strdup("unspec"); + + if (tor_addr_from_sockaddr(&addr, sa, &port) < 0) + return NULL; + if (! tor_addr_to_str(address, &addr, sizeof(address), 1)) + return NULL; + tor_asprintf(&result, "%s:%d", address, (int)port); + return result; +} + /** Set address <b>a</b> to the unspecified address. This address belongs to * no family. */ void diff --git a/src/common/address.h b/src/common/address.h index d05a3de..9a7656f 100644 --- a/src/common/address.h +++ b/src/common/address.h @@ -44,6 +44,7 @@ socklen_t tor_addr_to_sockaddr(const tor_addr_t *a, uint16_t port, int tor_addr_from_sockaddr(tor_addr_t *a, const struct sockaddr *sa, uint16_t *port_out); void tor_addr_make_unspec(tor_addr_t *a); +char *tor_sockaddr_to_str(const struct sockaddr *sa); /** Return an in6_addr* equivalent to <b>a</b>, or NULL if <b>a</b> is not * an IPv6 address. */

1 0

[tor/master] Advertise correct DirPort/ORPort when configured with "auto"
by nickm＠torproject.org 13 May '11

13 May '11

commit 3da661b2422dbf5a098eb3b8ad9dd2033599016f Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon May 2 15:51:30 2011 -0400 Advertise correct DirPort/ORPort when configured with "auto" We'll eventually want to do more work here to make sure that the ports are stable over multiple invocations. Otherwise, turning your node on and off will get you a new DirPort/ORPort needlessly. --- src/or/config.c | 2 +- src/or/dirserv.c | 6 +++--- src/or/router.c | 38 ++++++++++++++++++++++++++++++++++---- src/or/router.h | 3 +++ 4 files changed, 41 insertions(+), 8 deletions(-) diff --git a/src/or/config.c b/src/or/config.c index 914c4cb..b06c8c9 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -563,7 +563,7 @@ static int or_state_validate(or_state_t *old_options, or_state_t *options, static int or_state_load(void); static int options_init_logs(or_options_t *options, int validate_only); -static int is_listening_on_low_port(uint16_t port_option, +static int is_listening_on_low_port(int port_option, const config_line_t *listen_options); static uint64_t config_parse_memunit(const char *s, int *ok); diff --git a/src/or/dirserv.c b/src/or/dirserv.c index eadc2d7..79b68cd 100644 --- a/src/or/dirserv.c +++ b/src/or/dirserv.c @@ -2699,8 +2699,8 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, voter->sigs = smartlist_create(); voter->address = hostname; voter->addr = addr; - voter->dir_port = options->DirPort; - voter->or_port = options->ORPort; + voter->dir_port = router_get_advertised_dir_port(options); + voter->or_port = router_get_advertised_or_port(options); voter->contact = tor_strdup(contact); if (options->V3AuthUseLegacyKey) { authority_cert_t *c = get_my_v3_legacy_cert(); @@ -2806,7 +2806,7 @@ generate_v2_networkstatus_opinion(void) "dir-options%s%s%s%s\n" "%s" /* client version line, server version line. */ "dir-signing-key\n%s", - hostname, ipaddr, (int)options->DirPort, + hostname, ipaddr, (int)router_get_advertised_dir_port(options), fingerprint, contact, published, diff --git a/src/or/router.c b/src/or/router.c index a7148ea..616a290 100644 --- a/src/or/router.c +++ b/src/or/router.c @@ -704,8 +704,8 @@ init_keys(void) ds = router_get_trusteddirserver_by_digest(digest); if (!ds) { ds = add_trusted_dir_server(options->Nickname, NULL, - (uint16_t)options->DirPort, - (uint16_t)options->ORPort, + router_get_advertised_dir_port(options), + router_get_advertised_or_port(options), digest, v3_digest, type); @@ -1165,6 +1165,36 @@ consider_publishable_server(int force) } } +/** Return the port that we should advertise as our ORPort; this is either + * the one configured in the ORPort option, or the one we actually bound to + * if ORPort is "auto". */ +uint16_t +router_get_advertised_or_port(or_options_t *options) +{ + if (options->ORPort == CFG_AUTO_PORT) { + connection_t *c = connection_get_by_type(CONN_TYPE_OR_LISTENER); + if (c) + return c->port; + return 0; + } + return options->ORPort; +} + +/** Return the port that we should advertise as our DirPort; this is either + * the one configured in the DirPort option, or the one we actually bound to + * if DirPort is "auto". */ +uint16_t +router_get_advertised_dir_port(or_options_t *options) +{ + if (options->DirPort == CFG_AUTO_PORT) { + connection_t *c = connection_get_by_type(CONN_TYPE_DIR_LISTENER); + if (c) + return c->port; + return 0; + } + return options->DirPort; +} + /* * OR descriptor generation. */ @@ -1398,8 +1428,8 @@ router_rebuild_descriptor(int force) ri->address = tor_dup_ip(addr); ri->nickname = tor_strdup(options->Nickname); ri->addr = addr; - ri->or_port = options->ORPort; - ri->dir_port = options->DirPort; + ri->or_port = router_get_advertised_or_port(options); + ri->dir_port = router_get_advertised_dir_port(options); ri->cache_info.published_on = time(NULL); ri->onion_pkey = crypto_pk_dup_key(get_onion_key()); /* must invoke from * main thread */ diff --git a/src/or/router.h b/src/or/router.h index 5e021f6..1bf1a51 100644 --- a/src/or/router.h +++ b/src/or/router.h @@ -50,6 +50,9 @@ int authdir_mode_publishes_statuses(or_options_t *options); int authdir_mode_tests_reachability(or_options_t *options); int authdir_mode_bridge(or_options_t *options); +uint16_t router_get_advertised_or_port(or_options_t *options); +uint16_t router_get_advertised_dir_port(or_options_t *options); + int server_mode(or_options_t *options); int public_server_mode(or_options_t *options); int advertised_server_mode(void);

1 0

[tor/master] Write automatically-chosen control ports to a file.
by nickm＠torproject.org 13 May '11

13 May '11

commit dad12188a6ca957f6fde1eb602fd98b2fa93b1a4 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon May 9 12:13:37 2011 -0400 Write automatically-chosen control ports to a file. --- changes/feature3076 | 3 +++ doc/tor.1.txt | 5 +++++ src/or/config.c | 1 + src/or/connection.c | 3 +++ src/or/control.c | 39 +++++++++++++++++++++++++++++++++++++++ src/or/control.h | 2 ++ src/or/main.c | 6 ++++-- src/or/or.h | 3 +++ 8 files changed, 60 insertions(+), 2 deletions(-) diff --git a/changes/feature3076 b/changes/feature3076 index aadd296..ed42e45 100644 --- a/changes/feature3076 +++ b/changes/feature3076 @@ -6,3 +6,6 @@ and ports that are bound for listeners for a given connection type. This is useful for if the user has selected SocksPort "auto", and you need to know which port got chosen. + - There is a ControlPortWriteToFile option that tells Tor to write + its actual control port or ports to a chosen file. + diff --git a/doc/tor.1.txt b/doc/tor.1.txt index e48342e..606580d 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -191,6 +191,11 @@ Other options can be specified either on the command-line (--option the default GID. [Making the file readable by other groups is not yet implemented; let us know if you need this for some reason.] (Default: 0). +**ControlPortWriteToFile** __Path__:: + If set, Tor writes the address and port of any control port it opens to + this address. Usable by controllers to learn the actual control port + when ControlPort is set to "auto". + **DataDirectory** __DIR__:: Store working data in DIR (Default: @LOCALSTATEDIR@/lib/tor) diff --git a/src/or/config.c b/src/or/config.c index b06c8c9..5eb6229 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -206,6 +206,7 @@ static config_var_t _option_vars[] = { V(ContactInfo, STRING, NULL), V(ControlListenAddress, LINELIST, NULL), V(ControlPort, PORT, "0"), + V(ControlPortWriteToFile, FILENAME, NULL), V(ControlSocket, LINELIST, NULL), V(CookieAuthentication, BOOL, "0"), V(CookieAuthFileGroupReadable, BOOL, "0"), diff --git a/src/or/connection.c b/src/or/connection.c index bc18dab..01b533d 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -998,6 +998,9 @@ connection_create_listener(const struct sockaddr *listensockaddr, "%s listening on port %u.", conn_type_to_string(type), gotPort); + if (type == CONN_TYPE_CONTROL_LISTENER) + control_ports_write_to_file(); + conn->state = LISTENER_STATE_READY; if (start_reading) { connection_start_reading(conn); diff --git a/src/or/control.c b/src/or/control.c index 19e8539..6346742 100644 --- a/src/or/control.c +++ b/src/or/control.c @@ -508,6 +508,45 @@ connection_printf_to_buf(control_connection_t *conn, const char *format, ...) connection_write_to_buf(buf, len, TO_CONN(conn)); } +/** Write all of the open control ports to ControlPortWriteToFile */ +void +control_ports_write_to_file(void) +{ + smartlist_t *lines; + char *joined = NULL; + or_options_t *options = get_options(); + + if (!options->ControlPortWriteToFile) + return; + + lines = smartlist_create(); + + SMARTLIST_FOREACH_BEGIN(get_connection_array(), const connection_t *, conn) { + char *port_str = NULL; + if (conn->type != CONN_TYPE_CONTROL_LISTENER || conn->marked_for_close) + continue; +#ifdef AF_UNIX + if (conn->socket_family == AF_UNIX) { + tor_asprintf(&port_str, "UNIX_PORT=%s\n", conn->address); + smartlist_add(lines, port_str); + continue; + } +#endif + tor_asprintf(&port_str, "PORT=%s:%d\n", conn->address, conn->port); + smartlist_add(lines, port_str); + } SMARTLIST_FOREACH_END(conn); + + joined = smartlist_join_strings(lines, "", 0, NULL); + + if (write_str_to_file(options->ControlPortWriteToFile, joined, 0) < 0) { + log_warn(LD_CONTROL, "Writing %s failed: %s", + options->ControlPortWriteToFile, strerror(errno)); + } + tor_free(joined); + SMARTLIST_FOREACH(lines, char *, cp, tor_free(cp)); + smartlist_free(lines); +} + /** Send a "DONE" message down the control connection <b>conn</b>. */ static void send_control_done(control_connection_t *conn) diff --git a/src/or/control.h b/src/or/control.h index 2ae96b4..a73ed5d 100644 --- a/src/or/control.h +++ b/src/or/control.h @@ -15,6 +15,8 @@ void control_update_global_event_mask(void); void control_adjust_event_log_severity(void); +void control_ports_write_to_file(void); + /** Log information about the connection <b>conn</b>, protecting it as with * CONN_LOG_PROTECT. Example: * diff --git a/src/or/main.c b/src/or/main.c index 7bae59c..15682d5 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -2046,12 +2046,14 @@ void tor_cleanup(void) { or_options_t *options = get_options(); - /* Remove our pid file. We don't care if there was an error when we - * unlink, nothing we could do about it anyways. */ if (options->command == CMD_RUN_TOR) { time_t now = time(NULL); + /* Remove our pid file. We don't care if there was an error when we + * unlink, nothing we could do about it anyways. */ if (options->PidFile) unlink(options->PidFile); + if (options->ControlPortWriteToFile) + unlink(options->ControlPortWriteToFile); if (accounting_is_enabled(options)) accounting_record_bandwidth_usage(now, get_or_state()); or_state_mark_dirty(get_or_state(), 0); /* force an immediate save. */ diff --git a/src/or/or.h b/src/or/or.h index 0a089f7..412aac9 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -2874,6 +2874,9 @@ typedef struct { * the defaults have changed. */ int _UsingTestNetworkDefaults; + /** File where we should write the ControlPort. */ + char *ControlPortWriteToFile; + } or_options_t; /** Persistent state for an onion router, as saved to disk. */

1 0

[tor/master] new GETINFO command to return list of listeners of a given type
by nickm＠torproject.org 13 May '11

13 May '11

commit c55c8f0d4926481ea2cf49e14915688de2eb4938 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon May 2 16:44:37 2011 -0400 new GETINFO command to return list of listeners of a given type --- changes/feature3076 | 7 ++++- src/or/control.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 2 deletions(-) diff --git a/changes/feature3076 b/changes/feature3076 index 5822850..aadd296 100644 --- a/changes/feature3076 +++ b/changes/feature3076 @@ -1,5 +1,8 @@ o Minor features - The options SocksPort, ControlPort, and so on now all accept an optional value "auto" that opens a socket on an OS-selected port. - - + o Minor features (controller) + - GETINFO net/listeners/(type) now returns a list of the addresses + and ports that are bound for listeners for a given connection + type. This is useful for if the user has selected SocksPort + "auto", and you need to know which port got chosen. diff --git a/src/or/control.c b/src/or/control.c index c536242..19e8539 100644 --- a/src/or/control.c +++ b/src/or/control.c @@ -1407,6 +1407,63 @@ munge_extrainfo_into_routerinfo(const char *ri_body, signed_descriptor_t *ri, return tor_strndup(ri_body, ri->signed_descriptor_len); } +/** Implementation helper for GETINFO: answers requests for information about + * which ports are bound. */ +static int +getinfo_helper_listeners(control_connection_t *control_conn, + const char *question, + char **answer, const char **errmsg) +{ + int type; + smartlist_t *res; + + (void)control_conn; + (void)errmsg; + + if (!strcmp(question, "net/listeners/or")) + type = CONN_TYPE_OR_LISTENER; + else if (!strcmp(question, "net/listeners/dir")) + type = CONN_TYPE_DIR_LISTENER; + else if (!strcmp(question, "net/listeners/socks")) + type = CONN_TYPE_AP_LISTENER; + else if (!strcmp(question, "net/listeners/trans")) + type = CONN_TYPE_AP_TRANS_LISTENER; + else if (!strcmp(question, "net/listeners/natd")) + type = CONN_TYPE_AP_NATD_LISTENER; + else if (!strcmp(question, "net/listeners/dns")) + type = CONN_TYPE_AP_DNS_LISTENER; + else if (!strcmp(question, "net/listeners/control")) + type = CONN_TYPE_CONTROL_LISTENER; + else + return 0; /* unknown key */ + + res = smartlist_create(); + SMARTLIST_FOREACH_BEGIN(get_connection_array(), connection_t *, conn) { + char *addr; + struct sockaddr_storage ss; + socklen_t ss_len = sizeof(ss); + + if (conn->type != type || conn->marked_for_close || conn->s < 0) + continue; + + if (getsockname(conn->s, (struct sockaddr *)&ss, &ss_len) < 0) { + tor_asprintf(&addr, "%s:%d", conn->address, (int)conn->port); + } else { + char *tmp = tor_sockaddr_to_str((struct sockaddr *)&ss); + addr = esc_for_log(tmp); + tor_free(tmp); + } + if (addr) + smartlist_add(res, addr); + } SMARTLIST_FOREACH_END(conn); + + *answer = smartlist_join_strings(res, " ", 0, NULL); + + SMARTLIST_FOREACH(res, char *, cp, tor_free(cp)); + smartlist_free(res); + return 0; +} + /** Implementation helper for GETINFO: knows the answers for questions about * directory information. */ static int @@ -1861,6 +1918,7 @@ static const getinfo_item_t getinfo_items[] = { "All non-expired, non-superseded router descriptors."), ITEM("desc/all-recent-extrainfo-hack", dir, NULL), /* Hack. */ PREFIX("extra-info/digest/", dir, "Extra-info documents by digest."), + PREFIX("net/listeners/", listeners, "Bound addresses by type"), ITEM("ns/all", networkstatus, "Brief summary of router status (v2 directory format)"), PREFIX("ns/id/", networkstatus,

1 0

[tor/master] Merge branch 'feature3076_squashed' into maint-0.2.2
by nickm＠torproject.org 13 May '11

13 May '11

commit 5f2a1a7b4f20be121cd30def95cf7789924ca70a Merge: e0d5a6e 7f654a6 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri May 13 10:43:41 2011 -0400 Merge branch 'feature3076_squashed' into maint-0.2.2 changes/feature3076 | 14 +++++++ doc/tor.1.txt | 47 ++++++++++++++++------- src/common/address.c | 30 ++++++++++++++ src/common/address.h | 1 + src/or/config.c | 61 ++++++++++++++--------------- src/or/connection.c | 67 ++++++++++++++++++++++++-------- src/or/control.c | 105 ++++++++++++++++++++++++++++++++++++++++++++++++++ src/or/control.h | 2 + src/or/dirserv.c | 6 +- src/or/main.c | 6 ++- src/or/or.h | 9 ++++ src/or/router.c | 38 ++++++++++++++++-- src/or/router.h | 3 + 13 files changed, 319 insertions(+), 70 deletions(-)

1 0

[tor/master] Add a ControlPortFileGroupWritable option
by nickm＠torproject.org 13 May '11

13 May '11

commit 7f654a6a6fb5e956b996eece36ff95e590a6ad63 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu May 12 19:17:48 2011 -0400 Add a ControlPortFileGroupWritable option --- changes/feature3076 | 5 ++++- doc/tor.1.txt | 5 +++++ src/or/config.c | 1 + src/or/control.c | 8 ++++++++ src/or/or.h | 2 ++ 5 files changed, 20 insertions(+), 1 deletions(-) diff --git a/changes/feature3076 b/changes/feature3076 index ed42e45..a3dcec8 100644 --- a/changes/feature3076 +++ b/changes/feature3076 @@ -7,5 +7,8 @@ type. This is useful for if the user has selected SocksPort "auto", and you need to know which port got chosen. - There is a ControlPortWriteToFile option that tells Tor to write - its actual control port or ports to a chosen file. + its actual control port or ports to a chosen file. If the option + ControlPortFileGroupReadable is set, the file is created as + group-readable. + diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 606580d..d95d764 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -196,6 +196,11 @@ Other options can be specified either on the command-line (--option this address. Usable by controllers to learn the actual control port when ControlPort is set to "auto". +**ControlPortFileGroupReadable** **0**|**1**:: + If this option is set to 0, don't allow the filesystem group to read the + control port file. If the option is set to 1, make the control port + file readable by the default GID. (Default: 0). + **DataDirectory** __DIR__:: Store working data in DIR (Default: @LOCALSTATEDIR@/lib/tor) diff --git a/src/or/config.c b/src/or/config.c index 5eb6229..a7ff28f 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -206,6 +206,7 @@ static config_var_t _option_vars[] = { V(ContactInfo, STRING, NULL), V(ControlListenAddress, LINELIST, NULL), V(ControlPort, PORT, "0"), + V(ControlPortFileGroupReadable,BOOL, "0"), V(ControlPortWriteToFile, FILENAME, NULL), V(ControlSocket, LINELIST, NULL), V(CookieAuthentication, BOOL, "0"), diff --git a/src/or/control.c b/src/or/control.c index 6346742..384e579 100644 --- a/src/or/control.c +++ b/src/or/control.c @@ -542,6 +542,14 @@ control_ports_write_to_file(void) log_warn(LD_CONTROL, "Writing %s failed: %s", options->ControlPortWriteToFile, strerror(errno)); } +#ifndef MS_WINDOWS + if (options->ControlPortFileGroupReadable) { + if (chmod(options->ControlPortWriteToFile, 0640)) { + log_warn(LD_FS,"Unable to make %s group-readable.", + options->ControlPortWriteToFile); + } + } +#endif tor_free(joined); SMARTLIST_FOREACH(lines, char *, cp, tor_free(cp)); smartlist_free(lines); diff --git a/src/or/or.h b/src/or/or.h index 412aac9..a73d98a 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -2876,6 +2876,8 @@ typedef struct { /** File where we should write the ControlPort. */ char *ControlPortWriteToFile; + /** Should that file be group-readable? */ + int ControlPortFileGroupReadable; } or_options_t;

1 0