September 2015 - tor-commits - lists.torproject.org

[gettor/master] Info about current, pending, and new ideas for providers and distribution methods
by ilv＠torproject.org 23 Sep '15

23 Sep '15

commit 7a4fe26f81bb1309c88ea58c27a727a084475c5e Author: ilv <ilv(a)users.noreply.github.com> Date: Wed Oct 29 22:37:34 2014 -0300 Info about current, pending, and new ideas for providers and distribution methods --- distribution_methods.txt | 21 +++++++++++++++++++++ providers.txt | 12 ++++++++++++ 2 files changed, 33 insertions(+) diff --git a/distribution_methods.txt b/distribution_methods.txt new file mode 100644 index 0000000..5cc25c8 --- /dev/null +++ b/distribution_methods.txt @@ -0,0 +1,21 @@ +================================== CURRENT ==================================== + +* SMTP: this one is ready and tested. + +* XMPP: this one is ready and tested. We'll probably deploy SMTP, tough. + + +================================== IN DEVELOPMENT ============================= + +* Twitter: This one is half done. I forked a twitter bot from wfn's work on +BridgeDB, but I got confused with some Twitter policies. I thought that the +bot *had* to follow someone back to send him/her DMs, but it turns out that +you just need to be followed, not follow back (thanks to mrphs for pointing +out this). + + + +================================== (CRAZY) IDEAS ============================== + +* Dropbox: This is just a thought, I haven't done proper research. What if +someone could ask for gettor links like this: "!gettor"? diff --git a/providers.txt b/providers.txt new file mode 100644 index 0000000..5fc8ac7 --- /dev/null +++ b/providers.txt @@ -0,0 +1,12 @@ +================================== CURRENT ==================================== + +* Dropbox: Implemented. + +================================== TO BE IMPLEMENTED ========================= + +* Google Drive: Not implemented yet. + +================================== IDEAS ===================================== + +* Dropbox: it is known that one can download tarballs from Github. Maybe we +can have an official gettor account and a folder synchronized with dist.tp.o.

1 0

[gettor/master] Fixed Github name on ideas for providers
by ilv＠torproject.org 23 Sep '15

23 Sep '15

commit 64512bc1dd26bfe1d8c5a7d97edaaa09fc04889b Author: ilv <ilv(a)users.noreply.github.com> Date: Wed Oct 29 22:39:56 2014 -0300 Fixed Github name on ideas for providers --- providers.txt | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/providers.txt b/providers.txt index 5fc8ac7..0f57719 100644 --- a/providers.txt +++ b/providers.txt @@ -8,5 +8,6 @@ ================================== IDEAS ===================================== -* Dropbox: it is known that one can download tarballs from Github. Maybe we -can have an official gettor account and a folder synchronized with dist.tp.o. +* Github: it is known that one can download tarballs from Github. Maybe we can +have an official gettor account and a folder synchronized with dist.tp.o. +

1 0

[gettor/master] Added specs done at the beginning of gsoc. Not updated to the current status of development, but still useful.
by ilv＠torproject.org 23 Sep '15

23 Sep '15

commit 2003220ef45fad665d7c688049266118e9ab9a4f Author: ilv <ilv(a)users.noreply.github.com> Date: Wed Nov 5 20:00:51 2014 -0300 Added specs done at the beginning of gsoc. Not updated to the current status of development, but still useful. --- spec/design/blacklist.txt | 74 ++++++++++++++++++++++ spec/design/core.txt | 155 +++++++++++++++++++++++++++++++++++++++++++++ spec/design/smtp.txt | 118 ++++++++++++++++++++++++++++++++++ spec/design/twitter.txt | 99 +++++++++++++++++++++++++++++ spec/overview.txt | 94 +++++++++++++++++++++++++++ 5 files changed, 540 insertions(+) diff --git a/spec/design/blacklist.txt b/spec/design/blacklist.txt new file mode 100644 index 0000000..34ce40e --- /dev/null +++ b/spec/design/blacklist.txt @@ -0,0 +1,74 @@ + Google Summer of Code 2014 GetTor Revamp - Blacklist module + Author: Israel Leiva - <israel.leiva(a)usach.cl> + Last update: 2014-05-16 + Version: 0.01 + Changes: First version + +1. Preface + + Since GetTor was created it has been a collection of functions and + classes separated in various modules. As its main purpose was + to serve files over SMTP, almost all current files have SMTP-related + procedures, including black and white lists. The proposed design for + the blacklist module intends to separate GetTor services from the + blacklist procedures. + +2. Blacklist module + + The main functionalities the blacklist module should provide are: + + * Check if a given entry is blacklisted for a given service + * Add/update/remove entries. + * Provide a standard critera to prevent flood. + +3. Design + + The new design should consist of the following files, directories and + functions: + + * lib/gettor/blacklist/: Directory for storing blacklisted users of + different services. + + ----- service1.blacklist: users blocked for service1 + ----- service2.blacklist: users blocked for service2 + + * lib/gettor/blacklist.py: Blacklist module of GetTor. + + isBlacklisted(address, provider) + Check if the given address is in the blacklist of services and + if it acomplish certain restrictions. For example, it could + check when was the last time it was updated on the blacklist. + + +4. Roadmap + + A possible example of how the blacklist module could work: + + a. A service receives a request and call the blacklist module. + b. The blacklist module check the blacklist for that service. + c. If the user (hashed) is present in the blacklist, it checks when + was the last time it was updated. If this date is more than X days + ago, updates the entry with the current date and returns false. If + not, returns true. If the user is not present in the file, it adds + the user with the current date and returns false. + + +5. Discussion + +5.1 Mistakes + + Mistakes concerning the package requested should be consider. If I + send a message asking for a Linux 'es' request but I write 'en' instead + of 'es', should I be able to ask again? or wait until I'm no longer + blacklisted? + +5.2 Users + + The method presented above (Roadmap) should consider weekly or monthly + clean-up of the list. + +5.3 SorryMessage + + May be when a user tries to send too many requests we could send him + a message saying that he won't be able to ask for packages in the next + X days. diff --git a/spec/design/core.txt b/spec/design/core.txt new file mode 100644 index 0000000..ce07238 --- /dev/null +++ b/spec/design/core.txt @@ -0,0 +1,155 @@ + Google Summer of Code 2014 GetTor Revamp - Core module + Author: Israel Leiva - <israel.leiva(a)usach.cl, ilv(a)riseup.net> + Last update: 2014-07-15 + Version: 0.05 + Changes: + [0.05] + Added fingerprint support + [0.04] + Changed log format in 'Design' + Added section 'Features' + Deleted stuff from 'Discussion' and added some others + [0.03] + Changed proposed format for link files to RFC 882 (ConfigParser). + Read configuration from file with ConfigParser. + [0.02] + Combine official mirrors with providers links (as another provider). + Eliminated on demand link generation. Now it reads from files. + Modified description according to PEP-8. + [0.01] + First version. + + +1. Preface + + Since GetTor was created it has been a collection of functions and + classes separated in various modules. As its main purpose was + to serve files over SMTP, almost all current files have SMTP-related + procedures, including address normalization, message composition, etc. + The proposed design for the core module intends to separate GetTor + main functionalities which are independent of the service that + transports the bundles. + +2. Core module + + The main functionalities the core module should provide are: + + * Receive a request with OS version, bundle language, and respond + with the respective links. + * Read links from providers files. + * Log anonymous transactions. + +3. Design + + The new design should consist of the following files, directories and + methods: + + * gettor.cfg: Configuration values, e.g. base directory. + + * providers/: Directory for generated links. Should be specified on + gettor.cfg. + + ----- provider1.links: links from provider1. + ----- provider2.links: links from provider2. + ----- mirrors.links: links of official mirrors. + + All this data is generated automatically. + + * logs/: Directory for logs. Should be specified on gettor.cfg + + ----- all.log + ----- info.log + ----- debug.log + ----- warn.log + ----- error.log + + * Core module of GetTor. + + __init(config_file)__ + Creates a new Core object. It reads its configuration from + the config_file using ConfigParser. + + get_links(operating_system, locale) + Public method to obtain the links. Returns links for + operating_system in locale language. It checks if the operating + system and locale are supported and then calls the private + method _get_links() + + + Example: get_links('linux', 'en') + + _get_links(operating_system, locale) + Gets the links for a specific operating system and locale + according to the options received. It reads all the .links + files inside the providers directory. Each one of these files + should follow the ConfigParser's format. There should be a + section [provider] with the option 'name' for the provider's + name (e.g. Dropbox), and a section [key] with the option + 'fingerprint' for the key's fingerprint that signed the + uploaded packages. + + Following sections should specify the operating system and + its options should be the locale. When more than one link is + available per operating system and locale (always) the links + should be specified as a multiline value. Each link has the + format: + + link link_signature + + Example: + + [provider] + name: Dropbox + + [key] + fingerprint: 123A 456B 789C 012D 345E 678F 901G 234H 567I 890J + + [linux] + en: https://foo.bar https://foo.bar.asc, + https://foo.bar https://foo.bar.asc + + es: https://bar.baz https://bar.baz.asc + + PROVIDER NAME + operating_system locale link package_signature + + NOTE: For now, the official mirrors are considered just as + another provider. + + + _log_request(operating_system, locale) + Log information about the request for future stats (e.g. which + OS from which service is the most required). All logging + should be done with the logging module. + + * Providers: There should be one module/script per provider in charge + of generating the .links file with the proper format, including + the official mirrors. + + +4. Roadmap + + An example of how the core module work: + + a. The SMTP service receives a request. + b. The SMTP calls core.get_links() with the options sent by the user. + c. get_links() calls to _log_request(). + d. get_links() calls to _get_links(). + e. get_links() constructs a message with the information obtained. + f. get_links() returns the message previously constructed. + g. The SMTP service creates a message with the links obtained and + send it to the user. + +5. Discussion + +5.1 Daily logs + + Currently, logs are separated by level of information (debug, info, error). + Is it necessary to do this by days/weeks too? + +6. Features + + Possible features to be added in the future (open to discussion) + + a. Send HTTP links (currently some official mirrors are HTTP only). + diff --git a/spec/design/smtp.txt b/spec/design/smtp.txt new file mode 100644 index 0000000..e86e864 --- /dev/null +++ b/spec/design/smtp.txt @@ -0,0 +1,118 @@ + Google Summer of Code 2014 GetTor Revamp - SMTP module + Author: Israel Leiva - <israel.leiva(a)usach.cl> + Last update: 2014-05-16 + Version: 0.01 + Changes: First version + +1. Preface + + Since GetTor was created it has been a collection of functions and + classes separated in various modules. As its main purpose was + to serve files over SMTP, almost all current files have SMTP-related + procedures, including address normalization, message composition, etc. + The proposed design for the SMTP module intends to separate GetTor + main functionalities from the services, in this case, SMTP. + +2. SMTP module + + The main functionalities the SMTP module should provide are: + + * Receive requests via mail. + * Identify user instructions, such as ask for help or for a specific + bundle (OS version, language). + * Get the required links from the core module. + * Send different types of answers to the user. + * Manage black lists to avoid flood. + * Log requests for stats (anonymous). + +3. Design + + The new design should consist of the following files, directories and + functions: + + * lib/gettor/services/smtp.py: SMTP module of GetTor. + + __init__(configuration path) + Creates an object according to the configuration values. + + processEmail(email object) + Process emails received (by forwarding). + + __parseEmail(email object) + Parse the raw email sent by processMail(). Check for multi-part + emails and then parse the text part. It also tries to get the + locale information from the user's address. + + __parseText(email object) + Parse the text part of an email looking for the package requested. + + __getFrom(email object) + Returns the from address of an email object. + + __getLocale(address) + Tries to get the locale information from an email address. + + __checkBlacklist(address) + Check if the given address is blacklisted by comparing the + hashed address. If address is not present, it's added. If present, + check for the date when it was added. Yet to define how many + days will be considered for blacklisting or if another method + will be used. For this it uses the blacklist module. + + __sendReply(address) + Sends a reply to the user with the links required. It asks for + the links to the core module. + + __sendDelayAlert(address) + If enabled (on configuration), sends a delay message to the user + letting him know that the links are on their way. + + __sendHelp(address) + Sends a message to the user with help instructions. + + __createEmail(from, to, subject, body) + Creates an email object. + + __logRequest(options) + Log information about the request for future stats (e.g. which + OS and language are the most required). If this is called + after a failure, a copy of the email should be stored. + + * BASE_DIR/logs/: Directory for logs. The BASE_DIR should be in the + configuration file. + + ----- yyyy-mm-dd.log: daily log of requests. + + +4. Roadmap + + An example of how the SMTP module should work: + + a. The SMTP service receives a request (via forwarding). + b. The email sender is checked for blacklisting (comparing hashes). + c. The email is parsed, obtaining the package requested and the + locale information. + d. If the email was asking for help, a help reply is sent. + e. If the email was invalid, the process break. This fail is logged + and the email that triggered it, too. + f. If the email was valid and the delay alert is set, then a reply + informing the links are on their way is sent. + g. If the email was valid, the SMTP service asks for the links to the + code module. + h. After that, a reply is sent to the user. + i. In all cases the request is logged (with no user information). + + +5. Discussion + +5.1 Email forwarding + + Are we going to support forwarding emails as ForwardPackage() did in + the old GetTor? + +5.2 Blacklist sublists + + Now with less types of request (two if no forwarding is added), creating + sublists for different types of requests necessary to blacklist and + email address? Or should we blacklist it if it floods anything? + diff --git a/spec/design/twitter.txt b/spec/design/twitter.txt new file mode 100644 index 0000000..46a178a --- /dev/null +++ b/spec/design/twitter.txt @@ -0,0 +1,99 @@ + Google Summer of Code 2014 GetTor Revamp - Twitter module + Author: Israel Leiva - <israel.leiva(a)usach.cl> + Last update: 2014-05-16 + Version: 0.01 + Changes: First version + +1. Preface + + Since GetTor was created it has been a collection of functions and + classes separated in various modules. As its main purpose was + to serve files over SMTP, almost all current files have SMTP-related + procedures, including address normalization, message composition, etc. + The proposed design for the Twitter module intends to separate GetTor + main functionalities from the services, in this case, Twitter. + +2. Twitter module + + The main functionalities the Twitter module should provide are: + + * Receive requests via direct messages. + * Identify user instructions, such as ask for help or for a specific + bundle (OS version, language). + * Get the required links from the core module. + * Send different types of answers to the user. + * Split answers to fit Twitter's format. + * Manage black lists to avoid flood. + * Log requests for stats (anonymous). + +3. Design + + The new design should consist of the following files, directories and + functions: + + * lib/gettor/services/Twitter.py: Twitter module of GetTor. + + __init__(configuration path) + Creates an object according to the configuration values. + + processDM(message) + Process direct messages received. + + __parseDM(message) + Parse the direct message received. Check for the package requested + and the locale information. + + __getUser(message) + Gets the user from the message sent. + + __checkBlacklist(user) + Check if the given user is blacklisted by comparing the + hashed user. Yet to define how many days will be considered for + blacklisting or if another method will be used. For this it uses + the blacklist module. + + __sendReply(user) + Sends a reply to the user with the links required. It asks for + the links to the core module and then split them. + + __sendHelp(user) + Sends a message to the user with help instructions. + + __splitMessage(message) + Receives the links message and split it according to Twitter's + format. + + __CheckNewFollowers() + In order to ask for links the user has to follow the GetTor + account. The Twitter module will be constantly checking for + new followers and follow them back. + + __FollowUser(user) + Follow the given user. + + __logRequest(options) + Log information about the request for future stats (e.g. which + OS and language are the most required). If this is called + after a failure, a copy of the DM should be stored. + + * BASE_DIR/logs/: Directory for logs. BASE_DIR should be in the + configuration file. + + ----- yyyy-mm-dd.log: daily log of requests. + + +4. Roadmap + + An example of how the Twitter module should work: + + a. The Twitter account receives a DM. + b. The Twitter service check if is a valid message and if the user is + in the blacklist, and then tries to obtain the package requested and + the locale information. + c. The Twitter service asks for the links to the core module, then it + splits the message received to adopt Twitter's format. + d. One or more DMs are sent back to the user. + e. For all this, the user must follow the GetTor account. The Twitter + service will be constantly checking for new followers and following + them back. + diff --git a/spec/overview.txt b/spec/overview.txt new file mode 100644 index 0000000..435d256 --- /dev/null +++ b/spec/overview.txt @@ -0,0 +1,94 @@ + Google Summer of Code 2014 GetTor Revamp - Overview + Author: Israel Leiva - <israel.leiva(a)usach.cl> + Last update: 2014-05-16 + Version: 0.01 + Changes: First version + +1. Background + + GetTor was created as a program for serving Tor and related files over + SMTP, thus avoiding direct and indirect censorship of Tor's software, + in particular, the Tor Browser Bundle (TBB). Users interact with GetTor + by sending emails to a specific email address. In the past, after the + user specified his OS and language, GetTor would send him an attachment + with the required package. This worked until the bundles were too large + to be sent as attachments in most email providers. In order to fix this + GetTor started to send links instead. + +2. Current status + + The GetTor status can be summarized in the following points: + + * Emails are sent to gettor(a)torproject.org + * The GetTor reply contains: TBB links, signatures (with text guides + for verification), mirrors, support instructions in six languages. + * Dropbox links are sent to download the TBB and signatures. + * Users can not ask for packages in their language. + * English-only replies are sent. + * Any email directed to GetTor is replied with the same information, + there is no recognition of instructions. + * Links generation is not fully automated. + * All code is written in Python. Various parts are not currently used. + * Current repositories are [0] and [1]. + +3. Proposal + + The accepted proposal [2] for Google Summer of Code (GSoC) 2014 proposes + rewriting the current GetTor using a modular design, with a core module + that handles the main GetTor functionalities, and several other modules, + one for each service (e.g. SMTP), which can interact with the core and + send replies to the users. Three modules will be developed for the + purposes of GSoC: SMTP, Twitter, Skype|XMPP. + +3.1. Goals + + The main goals of this proposal are the following: + + * Provide old GetTor functionalites, such as replies in several + languages and recognize user instructions. + * Send fewer information in each reply. + * Support more providers for uploading the TBB packages. + * Automate links generation. + * Clearer, modular and well-documented code. + * Possibilty to create new modules for other common services. + +3.2. Design + + Preliminar designs for the core module and the services can be found + in the design/ folder. All services consider creating a python script + to add the logic for using them. For example, there should be a script + that receives the emails and uses the SMTP module. For simplicity, + I've tried to specify mostly the main functions of every module; there + are some functions, like opening and writing files, that were not + considered in this preliminar phase. + +4. Discussion + +4.1. Skype + + My co-mentor for GSoC, Nima, has publicly rejected the idea of creating + a module for Skype and proposed to implement one for XMPP instead. + I've chosen Skype for its popularity, but I have no other main reason + to maintain this option, so it's probable that XMPP transport will be + implemented. + +4.2. Storing links + + My original proposal considered the fact that links could be stored + somewhere with restricted access, ideally a git repository. Nima + mentioned that ideally the links shouldn't be stored. May be this idea + could be used only to the mirrors and providers configuration (see + core module design). + +4.3. Generating unique URLs + + Nima mentioned that unique URLs could be generated for each request, + and in case the user don't have access to SSL, these links could be + served and later deleted or recycled. I like this idea. + +4. References + + [0] https://gitweb.torproject.org/gettor.git + [1] https://gitweb.torproject.org/user/sukhbir/gettor.git + [2] https://ileiva.github.io/gettor_proposal.html +

1 0

[gettor/master] Added basic instructions to get the SMTP component working
by ilv＠torproject.org 23 Sep '15

23 Sep '15

commit c0e30f84c1252aaafe8bb54451a1e3636db3db3a Author: ilv <ilv(a)users.noreply.github.com> Date: Wed Oct 29 12:15:58 2014 -0300 Added basic instructions to get the SMTP component working --- INSTALL-SMTP | 16 ++++++++++++++++ UPLOAD-DROPBOX | 20 ++++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/INSTALL-SMTP b/INSTALL-SMTP new file mode 100644 index 0000000..ae7099f --- /dev/null +++ b/INSTALL-SMTP @@ -0,0 +1,16 @@ +1) Get the current repo: + +$ git clone https://github.com/ileiva/gettor.git + +2) Create the SQLite database: + +$ python scripts/create_db.py -c /path/to/db + +3) Modify the core.cfg, smtp.cfg, and blacklist.cfg accordingly. + +4) Check if supported locales ared in /etc/aliases. If not, add it. + +5) Add the following line to ~/.forward (in gettor home's directory) + +|"python2.7 /path/to/gettor/smtp_demo.py" + diff --git a/UPLOAD-DROPBOX b/UPLOAD-DROPBOX new file mode 100644 index 0000000..dc0728a --- /dev/null +++ b/UPLOAD-DROPBOX @@ -0,0 +1,20 @@ +1) Get the current repo: + +$ git clone https://github.com/ileiva/gettor.git + +2) Check if you have Dropbox and GnuPG Python modules. If not, install them: + +$ pip install dropbox gnupg + +4) Get the PGP key that signs the Tor Browser Bundles. + +3) Change general configuration and account info in dropbox.cfg. + +3) Run the script: + +$ python dropbox.py + +If everything works fine, you should see a dropbox.links file inside the +'providers' directory. The script will take the files on upload_dir +(dropbox.cfg) that end up on .xz and .xz.asc respectively. A script for +getting the latest bundles from dist.tp.o is pending. \ No newline at end of file

1 0

[gettor/master] Improved README
by ilv＠torproject.org 23 Sep '15

23 Sep '15

commit dee89596a3df5faf0d2042352f577a2248966b35 Author: ilv <ilv(a)users.noreply.github.com> Date: Wed Nov 5 21:25:07 2014 -0300 Improved README --- README.md | 130 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 126 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index e194589..cf1c5f7 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,128 @@ -GetTor -====== +GetTor Revamp +============= -GetTor development. Project started at the Google Summer of Code 2014, under the -Tor Project organization. +GetTor Revamp done during the Google Summer of Code 2014 for the Tor Project. +This repository continues to being used for improvements and further +development. + +What is GetTor? +=============== + +GetTor was created as a program for serving Tor and related files over SMTP, +thus avoiding direct and indirect censorship of Tor's software, in particular, +the Tor Browser Bundle (TBB). Users interacted with GetTor by sending emails +to a specific email address. After the user specified his OS and language, +GetTor would send him an email with an attachment containing the requested +package. This worked well for a while, but the bundles started to get too +large for being sent as attachments in most email providers. In order to fix +this, GetTor started to send (Dropbox) links instead of attachments. + +What are the goals of the new GetTor? +===================================== + +Here is a list of the main goals the new GetTor should accomplish: + + * Safe. Remember we are serving people under heavy censorship. + * Easy to use. The fewer user interactions, the better. + * Clean code. It should be clear to other developers/contributors how GetTor + works and how can it be improved. + * Automated. We should try to automate things as much as possible. + * Language and provider friendly. It should be easy to support new languages + and to add new providers for storing packages and generate links. + + How does the new GetTor works? + ============================== + +Links files: Currently links are saved in files with the '.links' extension, +using the ConfigParser format (RFC 882). A sample link file should look like +this: + + --- BEGIN FILE --- + [provider] + name = CoolCloudProvider + + [key] + fingerprint = AAAA BBBB CCCC DDDD EEEE FFFF GGGG HHHH IIII JJJJ + + [linux] + en = Package (64-bit): https://cool.cloud.link64 + ASC signature (64-bit): https://cool.cloud.link64.asc + Package SHA256 checksum (64-bit): superhash64, + Package (32-bit): https://cool.cloud.link32 + ASC signature (32-bit): https://cool.cloud.link32.asc + Package SHA256 checksum (32-bit): superhash32 + + [windows] + ... + + [osx] + ... + --- END FILE --- + +You can also check providers/dropbox.links for a better example. + +Core: the heart of GetTor. Receives requests for links for a certain OS and +language and respond accordingly. It also presents an easy way for scripts +to create links file. + +SMTP: Receives requests via email, process them, contact the core module if +necessary and respond to the user in the specified language. People can send +blank or dummy emails to it to receive a help message describing how to ask +for links. Email forwarding is used to redirect the emails to GetTor. + +XMPP: Same as above, but via XMPP (account needed). It has been tested with +dukgo.com, jabber.ccc.de, riseup.net. It doesn't seem to be able to interact +with gtalk users. + +Twitter: Receive requests via Twitter direct messages, contact the core module +if necessary and respond to the user in the specified language. Unfinished. + +DB: Store anonymous info about the people that interact with GetTor in order +to keep count of the number of requests per person and avoid malicious users +that try to collapse the service. It also keeps count of how many requests +GetTor has received during its lifetime. A lot of other data was being saved +in the original gsoc project, but it was changed to save the minimum. + +Blacklist: Provide a mechanism to avoid flood and deny interaction to +malicious users. + +Providers scripts: every supported provider should have a script to +automatically upload packages to 'the cloud' and create the corresponding +links files. The script should consider the following steps: + + 1) Upload the packages. + 2) Get the sha256 checksum of the files uploaded. + 3) Get the PGP key fingerprint that signed the files. + 4) Check for .asc file for every package uploaded. + 5) Put all together in a '.link' file (using the core module). + + +What is the current status of the new GetTor? +============================================= + +Tested internally, and currently waiting to be deployed. + + +How can I help? +================ + +If you have ideas to improve GetTor and/or add new providers, please tell us! +I'm currently the lead developer on this, so if you comments/doubts on the code +you can send me an e-mail to ilv _at_ riseup _dot_ net. For openning tickets +you should use the trac[0] and select the GetTor component. Some neat ideas we +could use are the following: + +0) Report bugs! +1) Create script for new providers, namely: Google Drive, Github. Check +providers.txt +2) Create a new module for distributing links. Check distribution_methods.txt +3) Finish the Twitter module. +5) Propose code/behaviour improvements. +4) Update the specs. + + +References +=========== + +[0] https://trac.torproject.org/projects/tor/query?status=accepted&status=assig…

1 0

[gettor/master] Forgot the [u'url'] to modify to the actual url strings, not the objects
by ilv＠torproject.org 23 Sep '15

23 Sep '15

commit f27f95a610a8a44e8133d6ef81fc8e29a0779370 Author: ilv <ilv(a)users.noreply.github.com> Date: Tue Oct 28 22:03:04 2014 -0300 Forgot the [u'url'] to modify to the actual url strings, not the objects --- dropbox.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/dropbox.py b/dropbox.py index 59f398c..7bf60cb 100644 --- a/dropbox.py +++ b/dropbox.py @@ -176,9 +176,9 @@ if __name__ == '__main__': # build links link_file = client.share(file, short_url=False) # if someone finds how to do this with the API, please tell me! - link_file = link_file.replace('?dl=0', '?dl=1') + link_file[u'url'] = link_file[u'url'].replace('?dl=0', '?dl=1') link_asc = client.share(asc, short_url=False) - link_asc = link_asc.replace('?dl=0', '?dl=1') + link_asc[u'url'] = link_asc[u'url'].replace('?dl=0', '?dl=1') osys, arch, lc = get_bundle_info(file) link = "Package (%s-bit): %s\nASC signature (%s-bit): %s\n"\

1 0

[gettor/master] Added few missing words
by ilv＠torproject.org 23 Sep '15

23 Sep '15

commit a0d6874b76bd8f9cf71e80c47ae3615cbaf8dd07 Author: ilv <ilv(a)users.noreply.github.com> Date: Wed Oct 29 23:03:38 2014 -0300 Added few missing words --- distribution_methods.txt | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/distribution_methods.txt b/distribution_methods.txt index 58516b9..8bbdd4b 100644 --- a/distribution_methods.txt +++ b/distribution_methods.txt @@ -2,15 +2,15 @@ * SMTP: this one is ready and tested. -* XMPP: this one is ready and tested. We'll probably deploy SMTP, tough. +* XMPP: this one is ready and tested. We'll probably deploy SMTP first, tough. ================================== IN DEVELOPMENT ============================= -* Twitter: This one is half done. I forked a twitter bot from wfn's work on -BridgeDB, but I got confused with some Twitter policies. I thought that the -bot *had* to follow someone back to send him/her DMs, but it turns out that -you just need to be followed, not follow back (thanks to mrphs for pointing -out this). +* Twitter: This one is half done. I forked a twitter bot from wfn's gsoc work +on BridgeDB, but I got confused with some Twitter policies. I thought that the +bot *had* to follow someone back to send him/her DMs, but it turns out that you +just need to be followed, not follow back (thanks to mrphs for pointing out +this). ================================== (CRAZY) IDEAS ==============================

1 0

[gettor/master] Update core.py (remove unused module)
by ilv＠torproject.org 23 Sep '15

23 Sep '15

commit 0c06bc77f6d898b2dfee7c916084156b658116fe Author: Sukhbir Singh <s3singh(a)uwaterloo.ca> Date: Sun Nov 16 18:52:19 2014 +0530 Update core.py (remove unused module) --- gettor/core.py | 1 - 1 file changed, 1 deletion(-) diff --git a/gettor/core.py b/gettor/core.py index 792add0..88fb707 100644 --- a/gettor/core.py +++ b/gettor/core.py @@ -12,7 +12,6 @@ import os import re -import inspect import logging import tempfile import ConfigParser

1 0

[gettor/master] Enhanced README format
by ilv＠torproject.org 23 Sep '15

23 Sep '15

commit b0cbbedbab3ddac54c177ca03f9e73d4f0e93029 Author: ilv <ilv(a)users.noreply.github.com> Date: Wed Nov 5 21:30:44 2014 -0300 Enhanced README format --- README.md | 57 +++++++++++++++++++++++++++++++-------------------------- 1 file changed, 31 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index cf1c5f7..1884334 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ What is GetTor? =============== GetTor was created as a program for serving Tor and related files over SMTP, -thus avoiding direct and indirect censorship of Tor's software, in particular, +thus avoiding direct and indirect _censorship_ of Tor's software, in particular, the Tor Browser Bundle (TBB). Users interacted with GetTor by sending emails to a specific email address. After the user specified his OS and language, GetTor would send him an email with an attachment containing the requested @@ -22,22 +22,26 @@ What are the goals of the new GetTor? Here is a list of the main goals the new GetTor should accomplish: - * Safe. Remember we are serving people under heavy censorship. + * Safe. Remember we are serving people under _heavy censorship_. * Easy to use. The fewer user interactions, the better. * Clean code. It should be clear to other developers/contributors how GetTor - works and how can it be improved. + works and how it can be improved. * Automated. We should try to automate things as much as possible. * Language and provider friendly. It should be easy to support new languages and to add new providers for storing packages and generate links. - How does the new GetTor works? - ============================== -Links files: Currently links are saved in files with the '.links' extension, +How does the new GetTor works? +============================== + +Below are some specifications and core concepts on how the new GetTor works. + +*Links files*: Currently links are saved in files with the '.links' extension, using the ConfigParser format (RFC 882). A sample link file should look like this: - --- BEGIN FILE --- +--- BEGIN FILE --- + [provider] name = CoolCloudProvider @@ -57,44 +61,45 @@ this: [osx] ... - --- END FILE --- + +--- END FILE --- You can also check providers/dropbox.links for a better example. -Core: the heart of GetTor. Receives requests for links for a certain OS and +*Core*: the heart of GetTor. Receives requests for links for a certain OS and language and respond accordingly. It also presents an easy way for scripts to create links file. -SMTP: Receives requests via email, process them, contact the core module if +*SMTP*: Receives requests via email, process them, contact the core module if necessary and respond to the user in the specified language. People can send blank or dummy emails to it to receive a help message describing how to ask for links. Email forwarding is used to redirect the emails to GetTor. -XMPP: Same as above, but via XMPP (account needed). It has been tested with +*XMPP*: Same as above, but via XMPP (account needed). It has been tested with dukgo.com, jabber.ccc.de, riseup.net. It doesn't seem to be able to interact with gtalk users. -Twitter: Receive requests via Twitter direct messages, contact the core module +*Twitter*: Receive requests via Twitter direct messages, contact the core module if necessary and respond to the user in the specified language. Unfinished. -DB: Store anonymous info about the people that interact with GetTor in order +*DB*: Store anonymous info about the people that interact with GetTor in order to keep count of the number of requests per person and avoid malicious users that try to collapse the service. It also keeps count of how many requests GetTor has received during its lifetime. A lot of other data was being saved in the original gsoc project, but it was changed to save the minimum. -Blacklist: Provide a mechanism to avoid flood and deny interaction to +*Blacklist*: Provide a mechanism to avoid flood and deny interaction to malicious users. -Providers scripts: every supported provider should have a script to +*Providers scripts*: every supported provider should have a script to automatically upload packages to 'the cloud' and create the corresponding links files. The script should consider the following steps: - 1) Upload the packages. - 2) Get the sha256 checksum of the files uploaded. - 3) Get the PGP key fingerprint that signed the files. - 4) Check for .asc file for every package uploaded. - 5) Put all together in a '.link' file (using the core module). + * Upload the packages. + * Get the sha256 checksum of the files uploaded. + * Get the PGP key fingerprint that signed the files. + * Check for .asc file for every package uploaded. + * Put all together in a '.link' file (using the core module). What is the current status of the new GetTor? @@ -112,13 +117,13 @@ you can send me an e-mail to ilv _at_ riseup _dot_ net. For openning tickets you should use the trac[0] and select the GetTor component. Some neat ideas we could use are the following: -0) Report bugs! -1) Create script for new providers, namely: Google Drive, Github. Check + * Report bugs! + * Create script for new providers, namely: Google Drive, Github. Check providers.txt -2) Create a new module for distributing links. Check distribution_methods.txt -3) Finish the Twitter module. -5) Propose code/behaviour improvements. -4) Update the specs. + * Create a new module for distributing links. Check distribution_methods.txt + * Finish the Twitter module. + * Propose code/behaviour improvements. + * Update the specs. References

1 0

[gettor/master] Update dropbox.py (add UploadError class)
by ilv＠torproject.org 23 Sep '15

23 Sep '15

commit 6a19ed1192ff537a1c7f231f2fc3a27a391d6a73 Author: Sukhbir Singh <s3singh(a)uwaterloo.ca> Date: Sun Nov 16 18:59:17 2014 +0530 Update dropbox.py (add UploadError class) --- dropbox.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/dropbox.py b/dropbox.py index 3753b88..c9cf932 100644 --- a/dropbox.py +++ b/dropbox.py @@ -19,6 +19,8 @@ import ConfigParser import dropbox import gettor.core +class UploadError(Exception): + pass def valid_format(file): """Check for valid bundle format

1 0