- tor-dev - lists.torproject.org

RFC: remove TorUtil.Enum
by Sean Robinson 18 Jun '11

18 Jun '11

I have been reviewing the TorCtl package and I will be submitting tickets (usually with patches) for clean-ups, fixes, etc. Before I submit a ticket for removing TorUtil.Enum, I wanted to hear from other devs. There are no users of this class within TorCtl, so Enum is now cruft. Does anyone have a problem with removing Enum? If I do not get negative feedback about this patch, I will submit it later in the week. The following patch is also attached to this message as a gzipped patch file. diff --git a/TorUtil.py b/TorUtil.py index 3163d09..1d27144 100644 --- a/TorUtil.py +++ b/TorUtil.py @@ -22,7 +22,7 @@ if sys.version_info < (2, 5): else: from hashlib import sha1 -__all__ = ["Enum", "Enum2", "Callable", "sort_list", "quote", "escape_dots", "unescape_dots", +__all__ = ["Enum2", "Callable", "sort_list", "quote", "escape_dots", "unescape_dots", "BufSock", "secret_to_key", "urandom_rng", "s2k_gen", "s2k_check", "plog", "ListenSocket", "zprob", "logfile", "loglevel", "loglevels"] @@ -127,16 +127,6 @@ def read_config(filename): loglevel = config.get('TorCtl', 'loglevel') -class Enum(object): - """ Defines an ordered dense name-to-number 1-1 mapping """ - def __init__(self, start, names): - self.nameOf = {} - idx = start - for name in names: - setattr(self,name,idx) - self.nameOf[idx] = name - idx += 1 - class Enum2(object): """ Defines an ordered sparse name-to-number 1-1 mapping """ def __init__(self, **args): -- 1.7.5.1 -- Sean Robinson WiFi Radar - http://wifi-radar.berlios.de Python WiFi - http://pythonwifi.wikispot.org pymnl - http://pymnl.wikispot.org

2 2

Looking at Bridge users oddities
by Arturo 17 Jun '11

17 Jun '11

Hi, Jake made me notice that there has been a huge spike in tor bridge users in Italy in the past week (https://metrics.torproject.org/users.html?graph=bridge-users&start=2011-02-…) This prompted me to hack up a quick little script to analyse the Tor bridge usage data and see if it is possible to draw some conclusions from some patterns that emerge in it. I don't feel like expressing a personal an opinion at this point on what this data might mean, but I think spikes in Bridge usage could be used do draw conclusions on socio-political happenings. What the script does is it basically looks for spikes in Bridge traffic by giving it a time frame and a factor for triggering the alert. As already said it's just a quick hack so don't expect it to be performing or even well written :P. A couple of neat things that came out: factor | date1 date2 1: users at date1 2: users at date2 (country code) 23.0512820513 | 2011-06-10 2011-05-29 1: 1798 2: 78 (es) 22.0707070707 | 2011-06-10 2011-06-05 1: 4370 2: 198 (it) 4.14 | 2011-06-11 2011-06-04 1: 207 2: 50 (tw) 5.13333333333 | 2011-06-11 2011-06-05 1: 231 2: 45 (ar) 5.42857142857 | 2011-06-11 2011-06-05 1: 836 2: 154 (br) 4.38181818182 | 2011-06-11 2011-06-01 1: 241 2: 55 (il) 16.5208333333 | 2011-02-15 2011-01-18 1: 48 2: 793 (de) 15.4285714286 | 2010-08-23 2010-07-26 1: 648 2: 42 (au) 13.7916666667 | 2011-03-11 2011-02-15 1: 1655 2: 120 (sa) A one liner to get some interesting stats: python bridge-user-alert.py -f 4 | sort -g Hope somebody finds this little research useful, Cheers, Art.

1 0

Re: [tor-dev] license question
by Ted Smith 16 Jun '11

16 Jun '11

On Thu, 2011-06-16 at 16:51 +0200, Frederic Bor wrote: > Hello, > > > Im currently implementing transparent proxying for tor with Layered Service > Provider feature on windows. Microsoft provides a sample to demonstrate how > to implement an empty LSP (this is a lot of work, even to do nothing). This > sample is distributed under the license attached in this email. I have been > advised on #tor-dev to ask here about its compatibility with 3-clause BSD. > > It seems compatible to me but the following restriction may be a problem in > an open source context: > > "You may not [...] modify or distribute the source code of any Distributable > Code so that any part of it becomes subject to an Excluded License. An > Excluded License is one that requires, as a condition of use, modification > or distribution, that: > - the code be disclosed or distributed in source code form, or > - others have the right to modify it." > > The wording "requires, as a condition of use, modification or distribution" > seems to allow to use the sample in tor, since there is no such requirement > in 3-clause BSD. > > Does this seem right to you? You might want to ask licensing(a)fsf.org -- that looks like a clause designed to exclude copyleft licenses, so they'd probably know about it and what effects it would have on non-copyleft licenses.

1 0

license question
by Frederic Bor 16 Jun '11

16 Jun '11

Hello, Im currently implementing transparent proxying for tor with Layered Service Provider feature on windows. Microsoft provides a sample to demonstrate how to implement an empty LSP (this is a lot of work, even to do nothing). This sample is distributed under the license attached in this email. I have been advised on #tor-dev to ask here about its compatibility with 3-clause BSD. It seems compatible to me but the following restriction may be a problem in an open source context: "You may not [...] modify or distribute the source code of any Distributable Code so that any part of it becomes subject to an Excluded License. An Excluded License is one that requires, as a condition of use, modification or distribution, that: - the code be disclosed or distributed in source code form, or - others have the right to modify it." The wording "requires, as a condition of use, modification or distribution" seems to allow to use the sample in tor, since there is no such requirement in 3-clause BSD. Does this seem right to you? Cheers, Frederic

1 0

Thoughts on simplified packaging
by Andrew Lewman 15 Jun '11

15 Jun '11

As I talk to more and more people, they are confused by all of the different software packages we offer. From the simple, "I just want to be a relay, why is this so hard?" to "I just want to safely get to the BBC, I don't care about configuration and such. Can't you just make it a simple download and have it work by default?" I've written down my thoughts at https://svn.torproject.org/svn/projects/roadmaps/proposed-package-roadmap.t…. I'm looking for your thoughts and feedback. The goal is to simplify what we produce and make it easy for people to get the software and configuration they want. -- Andrew pgp 0x74ED336B

4 3

Python TorCtl development questions
by Sean Robinson 12 Jun '11

12 Jun '11

I have begun using the Python TorCtl package to develop a gui TOR controller. So, I am looking at the TorCtl source to understand how to use it properly. In this examination, I have begun adding comments, renaming vars to be more explicit, etc. How open are the developers to these changes? Will the devs accept patches to remove unused code. I'm looking at you, TorCtl.TorUtil.Enum. Is there interest in making TorCtl into a more mainstream form of Python package? See https://trac.torproject.org/projects/tor/ticket/2585 How far back in Python versions are we trying to maintain compatibility? As an example: TorCtl.TorUtil.Callable seems to be used to create static methods, but the staticmethod function to do this was added in Py2.2 (Dec 2001) and the @staticmethod decorator form was added in Py2.4 (Nov 2004). And thank you for TorCtl, it certainly saved me time not to have to implement a protocol engine from scratch. -- Sean Robinson WiFi Radar - http://wifi-radar.berlios.de Python WiFi - http://pythonwifi.wikispot.org pymnl - http://pymnl.wikispot.org

2 3

Will people running a relay be blocked from accessing CN destinations?
by tagnaq 11 Jun '11

11 Jun '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Ian, I made a new thread to avoid this discussion in the 'The Torouter and the DreamPlug' thread. > On Thu, Jun 09, 2011 at 11:47:10PM +0200, tagnaq wrote: >>> > > Doesn't "make random people into public (middle-only) relays" have the >>> > > (well maybe not "problem", but "issue"?) that when GFW blocks them, they >>> > > (the random people who bought an Excito/etc.) won't be able to connect >>> > > to anything in .cn any more? Although I don't _often_ connect to .cn >>> > > domains, it seems unfortunate to effectively auto-ban these people from >>> > > Chinese websites. >> > >> > I did not experience any problems connecting to .cn while using a relay >> > IP address. I think they are just blocking an IP:port combination and >> > not the entire IP address. >> > ...but things might change > Hmm. I wonder what happens if the packets are fragmented so that the > TCP port information isn't in the first fragment... possibilities: a) a fragmented IP packet doesn't get blocked b) they don't allow IP fragmentation (Don't Framgent Bit set) c) their firewall is able to find out whether the fragment is part of a blocked destination (IP:port) -----BEGIN PGP SIGNATURE----- iF4EAREKAAYFAk3znv4ACgkQyM26BSNOM7ZB5QD+J7p1OSqD7uopViiigmop84sp nlzbSV6dqK2ZvT+PrbUA/i2hLQKHkYZfVUQqTp4hu2o9bp5GbHsNylNP6l1lKByB =aKCK -----END PGP SIGNATURE-----

4 6

The Torouter and the DreamPlug
by Runa A. Sandvik 11 Jun '11

11 Jun '11

Hi everyone, DreamPlug is a new plug computer from GlobalScale Technologies: http://www.globalscaletechnologies.com/c-5-dreamplugs.aspx. The spec looks good, it runs Ubuntu by default and it doesn't cost too much. I thought that the DreamPlug was going to be very user friendly and a potential candidate for the Torouter project. (Maybe) I was wrong. When the SheevaPlug came out a couple of years ago, it shipped with a web interface that enabled users to change various network options. I thought the DreamPlug would ship with some kind of interface as well (and my plan was to just plug in a Tor page). This is not the case; it ships with lighthttpd by default and displays a static and very simple placeholder page on 192.168.1.1. If we want to use the DreamPlug for the Torouter, we will have to write a web interface for easy configuration of Tor. The interface should probably also provide options to better secure the DreamPlug. Downloading and installing Tor isn't a problem, but the configuration side of things can be tricky for users who aren't used to the command line. Thoughts? Comments? Thanks, -- Runa A. Sandvik

14 44

Too many cooks spoil the broth---or: how about we clean up the wiki?
by Karsten Loesing 11 Jun '11

11 Jun '11

Hello everyone, <rant>Today I looked at our wiki to add roadmaps for some products and realized what a mess the wiki is. This wasn't obvious to me, because I stopped using the wiki long ago, well, because it was such a mess. But boy, it has become worse!</rant> I think the main problem is that there's no obvious structure in the wiki, so everyone just adds their stuff anywhere, maybe thinking that someone else will move it to the right place. The other problem is that there's no such person cleaning up. Here's my plan: I'd like to rename (possibly a lot of) wiki pages so that the naming scheme implies a kind of wiki structure. I'd also add a page saying where stuff should go, and we'll beat up everyone not adhering to the structure in a joint effort. By the way, what happens if we rename wiki pages? I guess Trac will update all internal links, but external links (e.g., from the website) will be broken? We could fix our own website, but other websites would still be broken. How bad would that be? Here's a suggested naming scheme. In this scheme, there are three top-level directories (or rather, page prefixes): - doc/ contains all documentation for Tor users, relay operators, and the community in general. This is the stuff that people would usually expect in a wiki. - org/ contains deliverables promised to our sponsors, meeting plans, product roadmaps, project progress, etc. This stuff is in the wiki to make Tor development as transparent as possible. In theory, non-developers wouldn't require write permissions to these pages. - trac/ contains Trac's own documentation that it generously dumped into our Tor wiki. In theory, nobody would require write permissions to these pages. I'll go through the existing wiki pages and say where they should go. The following pages would move to doc/ (e.g., "DNS Hijacking" would become "doc/DNS Hijacking"): DNS Hijacking HTTPSEverywhere/SSLObservatorySubmission ImportantGoogleChromeBugs OnionCat TorALaymansGuide TorFAQUnanswered TorInChroot TorToggle UsingTorWithVservers badRelays badRelays/trotskyIps talks talks/MoscowState2010 TheOnionRouter TheOnionRouter/AnonymousPublicSpeech TheOnionRouter/BIND TheOnionRouter/BandwidthLimitChangeController TheOnionRouter/BlockNonTorTrafficDebian TheOnionRouter/BlockingDiagnostics TheOnionRouter/BlockingIrc TheOnionRouter/CentralizedTorServer TheOnionRouter/ConnectOverSSH TheOnionRouter/ContestEntries TheOnionRouter/ContestEntries/DesignFeedback TheOnionRouter/CronBandwidthLimit TheOnionRouter/DebianLive TheOnionRouter/EmbeddedTips TheOnionRouter/ExitEnclave TheOnionRouter/FAQUnanswered TheOnionRouter/FireFoxTorPerf TheOnionRouter/FirefoxRemoteDNS TheOnionRouter/GoodBadISPs TheOnionRouter/Gsoc2009 TheOnionRouter/Gsoc2009/ThandyTorrents TheOnionRouter/HiddenServiceNames TheOnionRouter/HowTo_post_on_or-talk_mailinglist TheOnionRouter/LegalStuff TheOnionRouter/LiveCDBestPractices TheOnionRouter/MacOSXUpgradeToTrunk TheOnionRouter/OpenbsdChrootedTor TheOnionRouter/OpenbsdChrootedTorControlScript TheOnionRouter/OpenbsdChrootedTorScript TheOnionRouter/OperationalSecurity TheOnionRouter/Portable_Tor TheOnionRouter/PreventingDnsLeaksInTor TheOnionRouter/Preventing_Tor_DNS_Leaks TheOnionRouter/PrivoxyConfig TheOnionRouter/PrivoxyPatches TheOnionRouter/Publicfile TheOnionRouter/ReducedExitPolicy TheOnionRouter/RemailingAndTor TheOnionRouter/RunTwoPrivoxys TheOnionRouter/SshPortForwardedTor TheOnionRouter/SummerOfCode TheOnionRouter/SupportPrograms TheOnionRouter/SupportPrograms/Polipo TheOnionRouter/TSocksPatches TheOnionRouter/TermsObserved TheOnionRouter/TorAbuseTemplates TheOnionRouter/TorExitNodeHowto TheOnionRouter/TorGuideUniversities TheOnionRouter/TorInChroot TheOnionRouter/TorInChroot-new TheOnionRouter/TorInTheMedia TheOnionRouter/TorMirror TheOnionRouter/TorOnDebian/PackagesForArchitectures TheOnionRouter/TorShirt TheOnionRouter/TorWin32TorWS TheOnionRouter/TorALaymansGuide TheOnionRouter/TorDNSExitList TheOnionRouter/TorFAQ TheOnionRouter/TorIPCOP TheOnionRouter/TorifyHOWTO TheOnionRouter/TorifyHOWTO/EMail TheOnionRouter/TorifyHOWTO/FTP TheOnionRouter/TorifyHOWTO/InstantMessaging TheOnionRouter/TorifyHOWTO/IrcSilc TheOnionRouter/TorifyHOWTO/Misc TheOnionRouter/TorifyHOWTO/Misc/Putty TheOnionRouter/TorifyHOWTO/Putty TheOnionRouter/TorifyHOWTO/SystracePolicy TheOnionRouter/TorifyHOWTO/WebBrowsers TheOnionRouter/Torouter TheOnionRouter/Torouter_notes TheOnionRouter/Torwin32DNS TheOnionRouter/Transocks TheOnionRouter/TransocksifyingTor TheOnionRouter/TransparentProxy TheOnionRouter/VMWareThroughTor TheOnionRouter/VerifyingSignatures TheOnionRouter/WebProxyHowto TheOnionRouter/Wikipedia TheOnionRouter/WindowsBufferProblems CodingForTor CodingForTor/CodingInC CodingForTor/CodingInR CodingForTor/LoggingAndDocumentation CodingInC CoreTorTracUsers build build/AddingBuildSlaves build/BuildSignoff build/ListOfPackages dev/HowToReleaseTor dev/SupportPolicy dev/SupportPolicy_v2 The following pages would move to org/meetings/: 2010TorAnnualDevMeeting 2011MiniDevMeeting 2011MiniDevMeeting/notes 2011StockholmHackfest 2011TorAnnualDevMeeting The product roadmaps that are not there yet would go into org/roadmaps/. Everything related to project management would go into org/projects/: TorAgileProcess projects/HowWeDoProjectManagement projects/ProjMgt/SetUpProjectMgt Products projects/ProductsandAssignments projects projects/2009FinancialandComplianceAudit projects/BridgeDB projects/CalculateDirreqShares projects/EmailAutoResponder projects/ExperimentalBridgeBundles projects/Infrastructure projects/Metrics projects/Metrics/DesignAndDeployDescriptorDatabase projects/Metrics/ImproveMetricsPortal projects/Metrics/PublishMetricsDataAndTools projects/Metrics/SafelyCountClients projects/Thandy projects/Tor projects/TorBrowserBundle projects/TorBrowserBundle/Alpha projects/TorBrowserBundle/OSX projects/TorBrowserBundle/OSX/Security projects/TorBulkExitlist projects/TorButton projects/TorCheck projects/TorFlow projects/TorStatus projects/Tor/MultithreadedCrypto projects/Weather projects/arm projects/mirror projects/mirror/torrent projects/vidalia The deliverables promised to our sponsors would go into org/sponsors/: sponsors sponsors/SponsorA sponsors/SponsorB sponsors/SponsorD sponsors/SponsorD/December2010 sponsors/SponsorD/June2011 sponsors/SponsorD/March2011 sponsors/SponsorD/September2010 sponsors/SponsorE sponsors/SponsorE/PhaseOne sponsors/SponsorF sponsors/SponsorF/Year1 And finally, the following pages are about using Trac and would go into trac/ (it's possible that Trac wouldn't like us renaming some of its pages, but I guess we'll find out): InterTrac InterWiki RecentChanges TitleIndex TracAccessibility TracAdmin TracBackup TracBrowser TracCgi TracChangeset TracEnvironment TracFastCgi TracFineGrainedPermissions TracGuide TracImport TracIni TracInstall TracInterfaceCustomization TracLinks TracLogging TracModPython TracModWSGI TracNavigation TracNotification TracPermissions TracPlugins TracQuery TracReports TracRevisionLog TracRoadmap TracRss TracSearch TracStandalone TracSupport TracSyntaxColoring TracTickets TracTicketsCustomFields TracTimeline TracUnicode TracUpgrade TracWiki TracWorkflow WikiDeletePage WikiFormatting WikiHtml WikiMacros WikiNewPage WikiPageNames WikiProcessors WikiRestructuredText WikiRestructuredTextLinks WikiStart That's it. I realize there are lots of cooks on this list, so I expect there will be feedback. Thanks, Karsten

4 8

Re: [tor-dev] Tor and BGP integration
by Arturo 09 Jun '11

09 Jun '11

Hello, This seems to me like a really neat idea! Reading from real time BGP feeds is not a simple task and I think it might be a bit of an overhead for the average Tor user. On the other hand it could be a good idea to have some nodes run tools to generate exit policies or at least provide BGP routing information in a easy to consume format for the tor clients to use. I have been doing some experimenting on getting and analysing BGP information and these are the resources that I have found useful: - http://archive.routeviews.org/ - For getting raw MRT format BGP feeds and also the output of `bgp show ip` - http://www-01.pch.net/resources/data/routing-tables/archive/2011/ - Routing information from other "probes" - https://github.com/hellais/PyRT - A quick patch I made to make PyRT work on python > 2.3 (useful for parsing MRT format files), original: https://research.sprintlabs.com/pyrt/ IXPs: - http://www.pch.net/resources/data/routing-tables/looking-glass/ - https://www.peeringdb.com/private/index.php - Very complete database on peers and ASN information. They also provide a very nice API for interacting with their database. I have been working on some python code for adding BGP support to blockfinder. It basically just parses the results of the `bgp show ip` command outputs obtainable from route views and stores the routing paths inside a database. It needs to be cleaned up a bit, but if you are interested in using it I can hack on it some more. Let me know of the progress you make :) - Arturo `hellais` > Hello from Iceland, > > Linus invited me to Reykjavik to talk about Tor at the NORDUnet conference > and this idea is the result of a bit of feedback from some network operators > here. > > Tor needs a way to be friendly to large network operators who wish to enable > exiting to anonymous communication for their networks. These network > operators are happy to allow anyone to pass traffic to their relays as entry > nodes, middle nodes and even limited exit nodes. > > Linus and I have been discussing methods of automating this process and of > course BGP integration makes a lot of sense. Generally, a network operator > has a set of AS numbers for their network blocks and as they want people to > connect to many of their services, it helps quite a bit to allow exiting to > those services from their own Tor relays. > > We came up with two main ideas for making this happen. > > One method would be to write a program where given an AS number and a BGP > feed, we parse all of the advertised network blocks and emit exit policy > lines that accepts all traffic for the AS. This would allow for a web > service similar to BulkExitList.py for network aware exit policy generation > and relay operators would simply need to add this to their Tor configs > manually. For mostly static networks, a cronjob would be fine and Tor > doesn't need to know about AS numbers internally. > > Another method would be to write a controller that watches for BGP network > updates and Tor would add relevant exit policy lines for any configured AS. > This would allow any Tor relay to dynamically learn about network changes if > it has access to a BGP feed patched into a controller. This could be > implemented by adding some configuration options to Tor that let Tor know > which AS numbers matter to which router. It may also allow for the router to > auto learn it's own likely family network but it lacks any kind of > bi-directional confirmation, still it seems useful information to have... > > It would be fantastic if someone offered a hidden service NORDUNet BGPMon > feed. This would help enable the first method of generating network aware > exit policies; this would also help with the development of AS awareness in > Tor itself. In the future, I imagine that it makes a lot of sense for > circuit building to be BGP aware as mere netblocks will not be very useful > in an ipv6 world, they're already mostly irrelevant. > > Anyway, food for thought. Linus and I will probably hack on some of these > ideas in the near future. > > All the best, > Jake

1 0