tor-commits November 2012

tor-commits@lists.torproject.org

18 participants
1508 discussions

[tor-cloud/master] Build private obfsproxy bridge images
by runa＠torproject.org 28 Nov '12

28 Nov '12

commit d57e58ab37ce96b215a1b7dc1322277aad94f245 Author: Runa A. Sandvik <runa.sandvik(a)gmail.com> Date: Wed Nov 28 16:30:36 2012 +0000 Build private obfsproxy bridge images --- ec2-prep.sh | 14 ++++++++++++-- 1 files changed, 12 insertions(+), 2 deletions(-) diff --git a/ec2-prep.sh b/ec2-prep.sh index 4de2241..ef8ee7b 100644 --- a/ec2-prep.sh +++ b/ec2-prep.sh @@ -233,9 +233,10 @@ ORPort 443 # advertise 443 but bind to 9001). ORListenAddress 0.0.0.0:9001 -# Start Tor as a private bridge. +# Start Tor as a private obfsproxy bridge BridgeRelay 1 PublishServerDescriptor 0 +ServerTransportPlugin obfs2 exec /usr/bin/obfsproxy --managed # Never send or receive more than 10GB of data per week. The accounting # period runs from 10 AM on the 1st day of the week (Monday) to the same @@ -247,8 +248,17 @@ AccountingMax 10 GB # so it shouldn't expose the operator to abuse complaints. ExitPolicy reject *:* EOF + +# Edit /var/lib/tor/state and change the obfs port +/etc/init.d/tor reload +sleep 30s +/etc/init.d/tor stop +sleep 30s +sed -i 's/TransportProxy.*/TransportProxy obfs2 0.0.0.0:52176/' /var/lib/tor/state +/etc/init.d/tor start +sleep 30s echo "Done configuring the system, will reboot" -echo "Your system has been configured as a private Tor bridge, see https://cloud.torproject.org/ for more info" > /etc/ec2-prep.sh +echo "Your system has been configured as a private obfsproxy Tor bridge, see https://cloud.torproject.org/ for more info" > /etc/ec2-prep.sh reboot fi

1 0

[translation/torbirdy_completed] Update translations for torbirdy_completed
by translation＠torproject.org 28 Nov '12

28 Nov '12

commit 3bd148780f291a195b42ce8a6add976a6df9c68f Author: Translation commit bot <translation(a)torproject.org> Date: Wed Nov 28 16:16:06 2012 +0000 Update translations for torbirdy_completed --- eu/torbirdy.dtd | 2 ++ eu/torbirdy.properties | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/eu/torbirdy.dtd b/eu/torbirdy.dtd index 09014fe..0423daf 100644 --- a/eu/torbirdy.dtd +++ b/eu/torbirdy.dtd @@ -33,6 +33,8 @@ <!ENTITY torbirdy.prefs.imap.key "i"> <!ENTITY torbirdy.prefs.startup_folder.label "Abioan azkenengoz erabilitako karpeta aukeratu [lehenetsia: ezgaituta, karpeta lokalak aukeratuko dira]"> <!ENTITY torbirdy.prefs.startup_folder.key "l"> +<!ENTITY torbirdy.prefs.timezone.label "Thunderbirden denbora gunea UTCra ez ezarri [lehenetsia: UTCra ezarri]"> +<!ENTITY torbirdy.prefs.timezone.key "z"> <!ENTITY torbirdy.prefs.enigmail_throwkeyid.label "Ezgaitu GPG '--throw-keyids' [lehenetsia: gaituta]"> <!ENTITY torbirdy.prefs.enigmail_throwkeyid.key "g"> <!ENTITY torbirdy.prefs.account_specific "Kontuaren araberakoa"> diff --git a/eu/torbirdy.properties b/eu/torbirdy.properties index 8e5628a..15cc9d5 100644 --- a/eu/torbirdy.properties +++ b/eu/torbirdy.properties @@ -11,3 +11,5 @@ torbirdy.email.prompt=TorBirdyk Thunderbirden autokonfigurazio morroia ezgaitu d torbirdy.email.advanced=Mesedez jakin ezazu TorBirdyren ezarpen aurreratuak aldatzea gomendatuta EZ dagoela.\n\nZer egiten ari zaren argi badaukazu bakarrik jarraitu beharko zenuke. torbirdy.email.advanced.nextwarning=Abisu hau hurrengo aldian erakutsi torbirdy.email.advanced.title=TorBirdy ezarpen aurreratuak + +torbirdy.restart=Denbora gunearen hobespena efektua izan dezan, itxi ezazu mesedez Thunderbird eta ireki ezazu berriz.

1 0

[translation/torbirdy] Update translations for torbirdy
by translation＠torproject.org 28 Nov '12

28 Nov '12

commit b1292aabddc2841881091e57501c522150199bd7 Author: Translation commit bot <translation(a)torproject.org> Date: Wed Nov 28 16:16:03 2012 +0000 Update translations for torbirdy --- eu/torbirdy.dtd | 4 ++-- eu/torbirdy.properties | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/eu/torbirdy.dtd b/eu/torbirdy.dtd index 16c9d93..0423daf 100644 --- a/eu/torbirdy.dtd +++ b/eu/torbirdy.dtd @@ -33,8 +33,8 @@ <!ENTITY torbirdy.prefs.imap.key "i"> <!ENTITY torbirdy.prefs.startup_folder.label "Abioan azkenengoz erabilitako karpeta aukeratu [lehenetsia: ezgaituta, karpeta lokalak aukeratuko dira]"> <!ENTITY torbirdy.prefs.startup_folder.key "l"> -<!ENTITY torbirdy.prefs.timezone.label ""> -<!ENTITY torbirdy.prefs.timezone.key ""> +<!ENTITY torbirdy.prefs.timezone.label "Thunderbirden denbora gunea UTCra ez ezarri [lehenetsia: UTCra ezarri]"> +<!ENTITY torbirdy.prefs.timezone.key "z"> <!ENTITY torbirdy.prefs.enigmail_throwkeyid.label "Ezgaitu GPG '--throw-keyids' [lehenetsia: gaituta]"> <!ENTITY torbirdy.prefs.enigmail_throwkeyid.key "g"> <!ENTITY torbirdy.prefs.account_specific "Kontuaren araberakoa"> diff --git a/eu/torbirdy.properties b/eu/torbirdy.properties index 4c680f1..15cc9d5 100644 --- a/eu/torbirdy.properties +++ b/eu/torbirdy.properties @@ -12,4 +12,4 @@ torbirdy.email.advanced=Mesedez jakin ezazu TorBirdyren ezarpen aurreratuak alda torbirdy.email.advanced.nextwarning=Abisu hau hurrengo aldian erakutsi torbirdy.email.advanced.title=TorBirdy ezarpen aurreratuak -# torbirdy.restart=For the time zone preference to take effect, please close Thunderbird and start it again. +torbirdy.restart=Denbora gunearen hobespena efektua izan dezan, itxi ezazu mesedez Thunderbird eta ireki ezazu berriz.

1 0

[translation/tails-persistence-setup] Update translations for tails-persistence-setup
by translation＠torproject.org 28 Nov '12

28 Nov '12

commit f750ac797bdd6b922237d3a6fbb87dbda446182f Author: Translation commit bot <translation(a)torproject.org> Date: Wed Nov 28 16:15:58 2012 +0000 Update translations for tails-persistence-setup --- eu/eu.po | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/eu/eu.po b/eu/eu.po index 01dc7f2..a485582 100644 --- a/eu/eu.po +++ b/eu/eu.po @@ -10,8 +10,8 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: https://trac.torproject.org/projects/tor\n" "POT-Creation-Date: 2012-11-27 21:00+0100\n" -"PO-Revision-Date: 2012-11-28 09:09+0000\n" -"Last-Translator: totorika93 <totorika93(a)gmail.com>\n" +"PO-Revision-Date: 2012-11-28 16:05+0000\n" +"Last-Translator: Antxon Baldarra <baldarra(a)euskalerria.org>\n" "Language-Team: LANGUAGE <LL(a)li.org>\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" @@ -141,7 +141,7 @@ msgstr "%s gailuak ez dauka bolumen iraunkorrik." msgid "" "Cannot delete the persistent volume while in use. You should restart Tails " "without persistence." -msgstr "" +msgstr "Ezin da bolumen iraunkorra ezabatu erabilia izaten ari den bitartean. Tails iraunkortasunik gabe berrabiarazi beharko zenuke." #: ../lib/Tails/Persistence/Setup.pm:483 msgid "Persistence volume is not unlocked." @@ -153,11 +153,11 @@ msgstr "Bolumen iraunkorra ez dago muntatuta." #: ../lib/Tails/Persistence/Setup.pm:493 msgid "Persistence volume is not readable. Permissions or ownership problems?" -msgstr "" +msgstr "Bolumen iraunkorra ezin da irakurri. Baimen edo jabetza arazoak?" #: ../lib/Tails/Persistence/Setup.pm:498 msgid "Persistence volume is not writable. Maybe it was mounted read-only?" -msgstr "" +msgstr "Bolumen iraunkorrean ezin da idatzi. Irakurtzeko soilik muntatu al zen agian?" #: ../lib/Tails/Persistence/Setup.pm:507 #, perl-format @@ -215,7 +215,7 @@ msgid "" "understood. Tails can't help you if you use it wrong! See <a " "href='file:///usr/share/doc/tails/website/doc/first_steps/persistence.en.html'>Tails" " documentation about persistence</a> to learn more." -msgstr "" +msgstr "<b>Kontuz!</b> Iraunkortasuna erabiltzeak ondo ulertu behar diren ondoriak ditu. Tailsek ezin zaitu lagundu oker erabiltzen baduzu! Ikusi <a href='file:///usr/share/doc/tails/website/doc/first_steps/persistence.en.html'>iraunkortasunari buruzko Tails dokumentazioa</a> gehiago ikasteko." #: ../lib/Tails/Persistence/Step/Bootstrap.pm:144 msgid "Passphrase:"

1 0

r25921: {website} wrap in p tags. (website/trunk/about/en)
by Andrew Lewman 28 Nov '12

28 Nov '12

Author: phobos Date: 2012-11-28 15:45:11 +0000 (Wed, 28 Nov 2012) New Revision: 25921 Modified: website/trunk/about/en/contact.wml Log: wrap in p tags. Modified: website/trunk/about/en/contact.wml =================================================================== --- website/trunk/about/en/contact.wml 2012-11-28 15:33:56 UTC (rev 25920) +++ website/trunk/about/en/contact.wml 2012-11-28 15:45:11 UTC (rev 25921) @@ -25,9 +25,9 @@ <a id="support"></a> <h3><a class="anchor" href="#support">Support</a></h3> - <i>help(a)rt.torproject.org</i> is the address to use for Tor support + <p><i>help(a)rt.torproject.org</i> is the address to use for Tor support requests. This is staffed for helping people with questions about - all things Tor. This is best effort service with no guarantees. + all things Tor. This is best effort service with no guarantees.</p> <a id="email"></a> <h3><a class="anchor" href="#email">Email</a></h3> @@ -81,8 +81,7 @@ IP (VOIP), VOIP+ZRTP, SIP, Video, and desktop sharing through the <a href="http://jitsi.org/">Jitsi Open Source Communicator</a>. Contact us at <em>torproject(a)jit.si</em> for live help, questions, or - feedback. This address does not receive emails, only XMPP and -VOIP.</p> + feedback. This address does not receive emails, only XMPP and VOIP.</p> <p>The best time to reach someone is 10:00 - 12:00 GMT-5 on Tuesday and 16:00 - 18:00 GMT-5 on Thursday.</p> @@ -104,8 +103,7 @@ will reach the Tor office. Should there be no answer, please leave a message with a way to reach you. Please be aware that your phone company and ours (and everyone in between) could listen to the - call. Please email <i>help(a)rt.torproject.org</i> for technical - support.</p> + call. Please email <i>help(a)rt.torproject.org</i> for support.</p> </div>

1 0

r25920: {website} fix the main nav too. (website/trunk/include)
by Andrew Lewman 28 Nov '12

28 Nov '12

Author: phobos Date: 2012-11-28 15:33:56 +0000 (Wed, 28 Nov 2012) New Revision: 25920 Modified: website/trunk/include/navigation.wmi Log: fix the main nav too. Modified: website/trunk/include/navigation.wmi =================================================================== --- website/trunk/include/navigation.wmi 2012-11-28 15:30:20 UTC (rev 25919) +++ website/trunk/include/navigation.wmi 2012-11-28 15:33:56 UTC (rev 25920) @@ -10,13 +10,13 @@ 'index' , 'Home', 'about/overview' , 'About Tor', 'docs/documentation' , 'Documentation', - 'projects/projects' , 'Projects', 'press/press' , 'Press', '<blog>' , 'Blog', 'http://www.printfection.com/torprojectstore' , 'Store', + 'about/contact' , 'Contact', ); my @calltoaction = ( - 'download/download-easy' , 'Download', + 'download/download-easy' , 'Download', 'getinvolved/volunteer' , 'Volunteer', 'donate/donate' , 'Donate', );

1 0

r25919: {website} fix the h3 bracket. (website/trunk/about/en)
by Andrew Lewman 28 Nov '12

28 Nov '12

Author: phobos Date: 2012-11-28 15:30:20 +0000 (Wed, 28 Nov 2012) New Revision: 25919 Modified: website/trunk/about/en/contact.wml Log: fix the h3 bracket. Modified: website/trunk/about/en/contact.wml =================================================================== --- website/trunk/about/en/contact.wml 2012-11-28 15:23:43 UTC (rev 25918) +++ website/trunk/about/en/contact.wml 2012-11-28 15:30:20 UTC (rev 25919) @@ -23,7 +23,7 @@ </ul> <a id="support"></a> - h3><a class="anchor" href="#support">Support</a></h3> + <h3><a class="anchor" href="#support">Support</a></h3> <i>help(a)rt.torproject.org</i> is the address to use for Tor support requests. This is staffed for helping people with questions about

1 0

r25918: {website} replace projects with contact so people can find it. Update (in website/trunk: about/en en)
by Andrew Lewman 28 Nov '12

28 Nov '12

Author: phobos Date: 2012-11-28 15:23:43 +0000 (Wed, 28 Nov 2012) New Revision: 25918 Modified: website/trunk/about/en/contact.wml website/trunk/en/navigation.wmi Log: replace projects with contact so people can find it. Update the contact page with a section for Support, so people aren't so confused about how to get support. Modified: website/trunk/about/en/contact.wml =================================================================== --- website/trunk/about/en/contact.wml 2012-11-27 04:03:01 UTC (rev 25917) +++ website/trunk/about/en/contact.wml 2012-11-28 15:23:43 UTC (rev 25918) @@ -13,6 +13,7 @@ <div id="maincol"> <h2>Tor: Contact</h2> <ul> + <li><a href="#support">Support</a></li> <li><a href="#email">Email</a></li> <li><a href="#identica">Identi.ca</a></li> <li><a href="#irc">IRC</a></li> @@ -21,21 +22,25 @@ <li><a href="#phone">Telephone</a></li> </ul> + <a id="support"></a> + h3><a class="anchor" href="#support">Support</a></h3> + + <i>help(a)rt.torproject.org</i> is the address to use for Tor support + requests. This is staffed for helping people with questions about + all things Tor. This is best effort service with no guarantees. + <a id="email"></a> <h3><a class="anchor" href="#email">Email</a></h3> - <i>tor-assistants at torproject.org</i> is the catch-all address for everything. It can be - used for reaching people working on Tor. - + <i>tor-assistants(a)torproject.org</i> is the catch-all address for + everything. It can be used for reaching people working on Tor. + A few other addresses are more specific: - + <ul> - <li><i>help(a)rt.torproject.org</i> is the address to use for Tor - support requests. This is staffed by volunteers helping people - with questions about all things Tor. This is best effort service - with no guarantees.</li> - <li><i>tor-translation(a)lists.torproject.org</i> can put new <a href="<page - getinvolved/translation>">website translations</a> into place, - and help answer questions about existing and new translations.</li> + <li><i>tor-translation(a)lists.torproject.org</i> can put new <a + href="<page getinvolved/translation>">website translations</a> + into place, and help answer questions about existing and + new translations.</li> <li><i>tordnsel at torproject.org</i> is the alias for the people responsible for the tordns exitlist.</li> <li><i>donations(a)torproject.org</i> is for questions and comments about <a @@ -48,7 +53,7 @@ and coordination, major gifts, contract inquiries, licensing and certification, etc.</li> </ul> - + <a id="identica"></a> <h3><a class="anchor" href="#identica">Identi.ca</a></h3> <p>Contact us via our microblogging Identi.ca account <a @@ -58,7 +63,7 @@ <h3><a class="anchor" href="#irc">IRC</a></h3> <p>Tor users and developers can also be found in the following channels on <a href="http://www.oftc.net/oftc/">OFTC</a>:</p> - + <ul> <li><i>#tor</i> - User and relay operation discussions. Having trouble running Tor or a question for other Tor users? Then this is the spot for @@ -69,14 +74,15 @@ <li><i>#nottor</i> - This is where Tor people hang out to talk about stuff that is not related to Tor.</li> </ul> - + <a id="livechat"></a> <h3><a class="anchor" href="#livechat">Live Chat</a></h3> <p>Tor is experimenting with live chat (XMPP/Jabber/GChat), Voice over IP (VOIP), VOIP+ZRTP, SIP, Video, and desktop sharing through the <a href="http://jitsi.org/">Jitsi Open Source Communicator</a>. Contact us at <em>torproject(a)jit.si</em> for live help, questions, or - feedback. This address does not receive emails, only XMPP and VOIP.</p> + feedback. This address does not receive emails, only XMPP and +VOIP.</p> <p>The best time to reach someone is 10:00 - 12:00 GMT-5 on Tuesday and 16:00 - 18:00 GMT-5 on Thursday.</p> Modified: website/trunk/en/navigation.wmi =================================================================== --- website/trunk/en/navigation.wmi 2012-11-27 04:03:01 UTC (rev 25917) +++ website/trunk/en/navigation.wmi 2012-11-28 15:23:43 UTC (rev 25918) @@ -10,10 +10,10 @@ 'index' , 'Home', 'about/overview' , 'About Tor', 'docs/documentation' , 'Documentation', - 'projects/projects' , 'Projects', 'press/press' , 'Press', '<blog>' , 'Blog', 'http://www.printfection.com/torprojectstore' , 'Store', + 'about/contact' , 'Contact', ); my @calltoaction = ( 'download/download-easy' , 'Download',

1 0

[ooni-probe/master] Fix missing argument in runTestCasesWithInput
by art＠torproject.org 28 Nov '12

28 Nov '12

commit 67c9043634fb55af2a76b331350cd156ba8fb437 Author: Arturo Filastò <art(a)fuffa.org> Date: Wed Nov 28 15:54:26 2012 +0100 Fix missing argument in runTestCasesWithInput --- ooni/runner.py | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-) diff --git a/ooni/runner.py b/ooni/runner.py index 5fa80ae..a4b3472 100644 --- a/ooni/runner.py +++ b/ooni/runner.py @@ -239,7 +239,7 @@ def runTestCasesWithInput(test_cases, test_input, yaml_reporter, test_methods_d.addCallback(tests_done, test_cases[0][0]) return test_methods_d -def runTestCasesWithInputUnit(test_cases, input_unit, oreporter): +def runTestCasesWithInputUnit(test_cases, input_unit, yaml_reporter, oonib_reporter): """ Runs the Test Cases that are given as input parallely. A Test Case is a subclass of ooni.nettest.NetTestCase and a list of @@ -252,14 +252,13 @@ def runTestCasesWithInputUnit(test_cases, input_unit, oreporter): input_unit: A generator that yields an input per iteration - oreporter: An instance of a subclass of ooni.reporter.OReporter """ log.debug("Running test cases with input unit") dl = [] for test_input in input_unit: log.debug("Running test with this input %s" % test_input) d = runTestCasesWithInput(test_cases, - test_input, oreporter) + test_input, yaml_reporter, oonib_reporter) dl.append(d) return defer.DeferredList(dl) @@ -373,11 +372,14 @@ def updateProgressMeters(test_filename, input_unit_factory, config.state[test_filename].per_item_average = 2.0 input_unit_idx = float(config.stateDict[test_filename]) - input_unit_items = float(len(input_unit_factory)) + input_unit_items = float(len(input_unit_factory) + 1) test_case_number = float(test_case_number) total_iterations = input_unit_items * test_case_number current_iteration = input_unit_idx * test_case_number + log.debug("input_unit_items: %s" % input_unit_items) + log.debug("test_case_number: %s" % test_case_number) + log.debug("Test case number: %s" % test_case_number) log.debug("Total iterations: %s" % total_iterations) log.debug("Current iteration: %s" % current_iteration)

1 0

[ooni-probe/master] Improve oonib
by art＠torproject.org 28 Nov '12

28 Nov '12

commit 19d533e4d1a473365d9794834cf4dea0a5a931a1 Author: Arturo Filastò <art(a)fuffa.org> Date: Wed Nov 28 14:52:31 2012 +0100 Improve oonib * More robust error handling * Add docstrings on not implemented collector parts * Refactor ooni reporting system --- ooni/__init__.py | 6 +- oonib/__init__.py | 2 + oonib/report/__init__.py | 8 ++ oonib/report/api.py | 156 ++++------------------------------------ oonib/report/file_collector.py | 145 +++++++++++++++++++++++++++++++++++++ 5 files changed, 172 insertions(+), 145 deletions(-) diff --git a/ooni/__init__.py b/ooni/__init__.py index bf98e16..36afc9a 100644 --- a/ooni/__init__.py +++ b/ooni/__init__.py @@ -11,9 +11,9 @@ from . import runner from . import templates from . import utils +__author__ = "Arturo Filastò" +__version__ = "0.0.7.1-alpha" + __all__ = ['config', 'inputunit', 'kit', 'lib', 'nettest', 'oonicli', 'reporter', 'runner', 'templates', 'utils'] - -__author__ = "Arturo Filastò" -__version__ = "0.0.7.1-alpha" diff --git a/oonib/__init__.py b/oonib/__init__.py index 1a853dd..ab7419c 100644 --- a/oonib/__init__.py +++ b/oonib/__init__.py @@ -14,6 +14,8 @@ from storm.uri import URI from storm.twisted.transact import Transactor from storm.databases.sqlite import SQLite +__version__ = '0.0.1' + from oonib import config database = SQLite(URI(config.main.database_uri)) diff --git a/oonib/report/__init__.py b/oonib/report/__init__.py index e69de29..96667b4 100644 --- a/oonib/report/__init__.py +++ b/oonib/report/__init__.py @@ -0,0 +1,8 @@ +def generateReportID(): + return otime.timestamp() + '_' + randomStr(20) + +class MissingField(Exception): + pass + +class InvalidRequestField(Exception): + pass diff --git a/oonib/report/api.py b/oonib/report/api.py index 5668857..9c42a54 100644 --- a/oonib/report/api.py +++ b/oonib/report/api.py @@ -1,7 +1,7 @@ """ -/report/do +/new -/report/pcap +/pcap This is the async pcap reporting system. It requires the client to have created a report already, but can work independently from test progress. @@ -18,47 +18,9 @@ from cyclone import web from ooni.utils import randomStr, otime from oonib import models, config - -backend_version = '0.0.1' - -def generateReportID(): - return otime.timestamp() + '_' + randomStr(20) - -class MissingField(Exception): - pass - -class InvalidRequestField(Exception): - pass - -def parseNewReportRequest(request): - """ - Here we parse a new report request. - """ - version_string = re.compile("[0-9A-Za-z_\-\.]+$") - name = re.compile("[a-zA-Z0-9_\- ]+$") - expected_request = {'software_name': name, - 'software_version': version_string, - 'test_name': name, - 'test_version': version_string, - 'progress': re.compile("[0-9]+$") - } - parsed_request = json.loads(request) - for k, regexp in expected_request.items(): - try: - value_to_check = parsed_request[k] - except KeyError: - raise MissingField(k) - print "Matching %s with %s | %s" % (regexp, value_to_check, k) - if re.match(regexp, str(value_to_check)): - continue - else: - raise InvalidRequestField(k) - return parsed_request +from oonib.report import file_collector def parseUpdateReportRequest(request): - # XXX this and the function above can probably be refactored into something - # more compact. There is quite a bit of code duplication going on here. - #db_report_id_regexp = re.compile("[a-zA-Z0-9]+$") # this is the regexp for the reports that include the timestamp @@ -80,92 +42,6 @@ def parseUpdateReportRequest(request): return parsed_request -class NewReportHandlerFile(web.RequestHandler): - """ - Responsible for creating and updating reports by writing to flat file. - """ - def post(self): - """ - Creates a new report with the input - - * Request - - {'software_name': 'XXX', - 'software_version': 'XXX', - 'test_name': 'XXX', - 'test_version': 'XXX', - 'progress': 'XXX', - 'content': 'XXX' - } - - Optional: - 'test_helper': 'XXX' - 'client_ip': 'XXX' - - * Response - - {'backend_version': 'XXX', 'report_id': 'XXX'} - - """ - # XXX here we should validate and sanitize the request - try: - report_data = parseNewReportRequest(self.request.body) - except InvalidRequestField, e: - raise web.HTTPError(400, "Invalid Request Field %s" % e) - except MissingField, e: - raise web.HTTPError(400, "Missing Request Field %s" % e) - - print "Parsed this data %s" % report_data - software_name = report_data['software_name'] - software_version = report_data['software_version'] - test_name = report_data['test_name'] - test_version = report_data['test_version'] - content = report_data['content'] - - report_id = generateReportID() - - #report_filename = '_'.join((report_id, - # report_data['software_name'], - # report_data['software_version'], - # report_data['test_name'], - # report_data['test_version'])) - - # The report filename contains the timestamp of the report plus a - # random nonce - report_filename = os.path.join(config.main.report_dir, report_id) - report_filename += '.yamloo' - - response = {'backend_version': backend_version, - 'report_id': report_id - } - - fp = open(report_filename, 'w+') - fp.write(report_data['content']) - fp.close() - self.write(response) - - def put(self): - """ - Update an already existing report. - - {'report_id': 'XXX', - 'content': 'XXX' - } - """ - parsed_request = parseUpdateReportRequest(self.request.body) - report_id = parsed_request['report_id'] - print "Got this request %s" % parsed_request - - report_filename = os.path.join(config.main.report_dir, report_id) - report_filename += '.yamloo' - try: - with open(report_filename, 'a+') as f: - # XXX this could be quite big. We should probably use the - # twisted.internet.fdesc module - print parsed_request['content'] - f.write(parsed_request['content']) - except IOError as e: - web.HTTPError(404, "Report not found") class NewReportHandlerDB(web.RequestHandler): """ @@ -177,7 +53,8 @@ class NewReportHandlerDB(web.RequestHandler): @defer.inlineCallbacks def post(self): """ - Creates a new report with the input + Creates a new report with the input to the database. + XXX this is not yet implemented. * Request @@ -193,23 +70,23 @@ class NewReportHandlerDB(web.RequestHandler): 'test_helper': 'XXX' 'client_ip': 'XXX' - * Response + * Response {'backend_version': 'XXX', 'report_id': 'XXX'} """ - parsed_request = json.loads(self.request.body) - # XXX here we should validate and sanitize the request - report_data = parsed_request + report_data = json.loads(self.request.body) new_report = models.Report() - print "Got %s as request" % parsed_request + log.debug("Got this request %s" % report_data) result = yield new_report.new(report_data) self.write(result) self.finish() def put(self): """ - Update an already existing report. + Update an already existing report with the database. + + XXX this is not yet implemented. {'report_id': 'XXX', 'content': 'XXX' @@ -217,15 +94,10 @@ class NewReportHandlerDB(web.RequestHandler): """ pass -class PCAPReportHandler(web.RequestHandler): - def get(self): - pass - - def post(self): - pass -reportingBackendAPI = [(r"/report/new", NewReportHandlerFile), - (r"/report/pcap", PCAPReportHandler) +reportingBackendAPI = [ + (r"/report", file_collector.NewReportHandlerFile), + (r"/pcap", file_collector.PCAPReportHandler) ] reportingBackend = web.Application(reportingBackendAPI, debug=True) diff --git a/oonib/report/file_collector.py b/oonib/report/file_collector.py new file mode 100644 index 0000000..9896784 --- /dev/null +++ b/oonib/report/file_collector.py @@ -0,0 +1,145 @@ +import random +import string +import json +import re +import os + +from oonib.report import generateReportID +from oonib.report import MissingField, InvalidRequestField + +from cyclone import web + +def parseNewReportRequest(request): + """ + Here we parse a new report request. + """ + version_string = re.compile("[0-9A-Za-z_\-\.]+$") + name = re.compile("[a-zA-Z0-9_\- ]+$") + + expected_request = {'software_name': name, + 'software_version': version_string, + 'test_name': name, + 'test_version': version_string + } + + parsed_request = json.loads(request) + for k, regexp in expected_request.items(): + try: + value_to_check = parsed_request[k] + except KeyError: + raise MissingField(k) + print "Matching %s with %s | %s" % (regexp, value_to_check, k) + if re.match(regexp, str(value_to_check)): + continue + else: + raise InvalidRequestField(k) + return parsed_request + +class NewReportHandlerFile(web.RequestHandler): + """ + Responsible for creating and updating reports by writing to flat file. + """ + def post(self): + """ + Creates a new report with the input + + * Request + + {'software_name': 'XXX', + 'software_version': 'XXX', + 'test_name': 'XXX', + 'test_version': 'XXX', + 'progress': 'XXX', + 'content': 'XXX' + } + + Optional: + 'test_helper': 'XXX' + 'client_ip': 'XXX' + + (not implemented, nor in client, nor in backend) + The idea behind these two fields is that it would be interesting to + also collect how the request was observed from the collectors point + of view. + + We use as a unique key the client_ip address and a time window. We + then need to tell the test_helper that is selected the client_ip + address and tell it to expect a connection from a probe in that time + window. + + Once the test_helper sees a connection from that client_ip it will + store for the testing session the data that it receives. + When the probe completes the report (or the time window is over) the + final report will include also the data collected from the + collectors view point. + + * Response + + {'backend_version': 'XXX', 'report_id': 'XXX'} + + """ + # XXX here we should validate and sanitize the request + try: + report_data = parseNewReportRequest(self.request.body) + except InvalidRequestField, e: + raise web.HTTPError(400, "Invalid Request Field %s" % e) + except MissingField, e: + raise web.HTTPError(400, "Missing Request Field %s" % e) + + print "Parsed this data %s" % report_data + software_name = report_data['software_name'] + software_version = report_data['software_version'] + test_name = report_data['test_name'] + test_version = report_data['test_version'] + content = report_data['content'] + + report_id = generateReportID() + + #report_filename = '_'.join((report_id, + # report_data['software_name'], + # report_data['software_version'], + # report_data['test_name'], + # report_data['test_version'])) + + # The report filename contains the timestamp of the report plus a + # random nonce + report_filename = os.path.join(config.main.report_dir, report_id) + report_filename += '.yamloo' + + response = {'backend_version': backend_version, + 'report_id': report_id + } + + fp = open(report_filename, 'w+') + fp.write(report_data['content']) + fp.close() + self.write(response) + + def put(self): + """ + Update an already existing report. + + {'report_id': 'XXX', + 'content': 'XXX' + } + """ + parsed_request = parseUpdateReportRequest(self.request.body) + report_id = parsed_request['report_id'] + print "Got this request %s" % parsed_request + + report_filename = os.path.join(config.main.report_dir, report_id) + report_filename += '.yamloo' + try: + with open(report_filename, 'a+') as f: + # XXX-Twisted change this to use t.i.a.fdesc and perhaps make a + # nice little object for it. + f.write(parsed_request['content']) + except IOError as e: + web.HTTPError(404, "Report not found") + +class PCAPReportHandler(web.RequestHandler): + def get(self): + pass + + def post(self): + pass

1 0

← Newer
1
...
7
8
9
10
11
12
13
...
151
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-commits November 2012