August 2013 - tor-commits - lists.torproject.org

[translation/https_everywhere] Update translations for https_everywhere
by translation＠torproject.org 27 Aug '13

27 Aug '13

commit 6316ea4d77d9600ccfe1115fa38f67876e698d49 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Aug 27 12:46:19 2013 +0000 Update translations for https_everywhere --- lv/https-everywhere.dtd | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/lv/https-everywhere.dtd b/lv/https-everywhere.dtd index 326d174..713a309 100644 --- a/lv/https-everywhere.dtd +++ b/lv/https-everywhere.dtd @@ -1,6 +1,6 @@ -<!ENTITY https-everywhere.about.title "Par HTTPS visur"> -<!ENTITY https-everywhere.about.ext_name "HTTPS visur"> -<!ENTITY https-everywhere.about.ext_description "Šifrē tīmekli! Automātiski lieto HTTPS drošību daudzās vietnēs."> +<!ENTITY https-everywhere.about.title "Par HTTPS Everywhere"> +<!ENTITY https-everywhere.about.ext_name "HTTPS Everywhere"> +<!ENTITY https-everywhere.about.ext_description "Šifrēt tīmekli! Automātiski lietot HTTPS drošību daudzās vietnēs."> <!ENTITY https-everywhere.about.version "Versija"> <!ENTITY https-everywhere.about.created_by "Autori"> <!ENTITY https-everywhere.about.librarians "Ruleset Librarians"> @@ -34,9 +34,9 @@ <!ENTITY https-everywhere.source.filename "Datnes_nosaukums"> <!ENTITY https-everywhere.source.unable_to_download "Nevar lejupielādēt avotu."> -<!ENTITY https-everywhere.popup.title ""> -<!ENTITY https-everywhere.popup.paragraph1 ""> -<!ENTITY https-everywhere.popup.paragraph2 ""> -<!ENTITY https-everywhere.popup.paragraph3 ""> -<!ENTITY https-everywhere.popup.keep ""> -<!ENTITY https-everywhere.popup.revert ""> +<!ENTITY https-everywhere.popup.title "HTTPS Everywhere 4.0development.11 ziņojums"> +<!ENTITY https-everywhere.popup.paragraph1 "Opā! Jūs lietojāt stabilu HTTPS Everywhere versiju, bet, iespējams, mūsu jaunākajā laidienā mēs nejauši jauninājām Jūsu līdzšinējo versiju ar izstrādātāju versiju."> +<!ENTITY https-everywhere.popup.paragraph2 "Vai vēlaties atgriezties pie stabilās versijas?"> +<!ENTITY https-everywhere.popup.paragraph3 "Mēs ļoti priecātos, ja Jūs turpinātu lietot izstrādātāju versiju un palīdzētu padarīt HTTPS Everywhere labāku! Iespējams, ka laiku pa laikam pamanīsit kādas kļūdas, par kurām varat ziņot rakstot angļu valodā https-everywhere(a)eff.org . Atvainojamies par sagādātajām neērtībām un pateicamies par to, ka lietojat HTTPS Everywhere."> +<!ENTITY https-everywhere.popup.keep "Izvēlos strādāt ar izstrādātāja versiju"> +<!ENTITY https-everywhere.popup.revert "Lejuplādēt jaunāko stabilo versiju">

1 0

[translation/https_everywhere_completed] Update translations for https_everywhere_completed
by translation＠torproject.org 27 Aug '13

27 Aug '13

commit 88b46018003f176521b5a16ec6a928363707e943 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Aug 27 11:16:14 2013 +0000 Update translations for https_everywhere_completed --- es/https-everywhere.dtd | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/es/https-everywhere.dtd b/es/https-everywhere.dtd index b82b623..bdf453b 100644 --- a/es/https-everywhere.dtd +++ b/es/https-everywhere.dtd @@ -34,4 +34,9 @@ <!ENTITY https-everywhere.source.filename "Nombre de fichero"> <!ENTITY https-everywhere.source.unable_to_download "No fue posible descargar el código."> - +<!ENTITY https-everywhere.popup.title "Notificación de HTTPS Everywhere 4.0development.11"> +<!ENTITY https-everywhere.popup.paragraph1 "Uups. Está usted usando la versión estable de HTTPS Everywhere, pero podríamos haberle elevado accidentalmente a la versión de desarrollo al publicar la última versión."> +<!ENTITY https-everywhere.popup.paragraph2 "¿Le gustaría volver a la versión estable?"> +<!ENTITY https-everywhere.popup.paragraph3 "Nos encantaría que continuase usando nuestra versión de desarrollo ¡y nos ayudara a hacer HTTPS Everywhere mejor! Podría encontrar que hay unos pocos fallos más aquí y allá, de los que puede informar a 'https-everywhere(a)eff.org'. Disculpe la inconveniencia, y gracias por usar HTTPS Everywhere."> +<!ENTITY https-everywhere.popup.keep "Manténganme en la versión de desarrollo."> +<!ENTITY https-everywhere.popup.revert "Descargar la última versión estable.">

1 0

[translation/https_everywhere] Update translations for https_everywhere
by translation＠torproject.org 27 Aug '13

27 Aug '13

commit baf6429915ef58841fc3f35635a86930940f7e48 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Aug 27 11:16:09 2013 +0000 Update translations for https_everywhere --- es/https-everywhere.dtd | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/es/https-everywhere.dtd b/es/https-everywhere.dtd index 25f20e6..bdf453b 100644 --- a/es/https-everywhere.dtd +++ b/es/https-everywhere.dtd @@ -34,9 +34,9 @@ <!ENTITY https-everywhere.source.filename "Nombre de fichero"> <!ENTITY https-everywhere.source.unable_to_download "No fue posible descargar el código."> -<!ENTITY https-everywhere.popup.title ""> -<!ENTITY https-everywhere.popup.paragraph1 ""> -<!ENTITY https-everywhere.popup.paragraph2 ""> -<!ENTITY https-everywhere.popup.paragraph3 ""> -<!ENTITY https-everywhere.popup.keep ""> -<!ENTITY https-everywhere.popup.revert ""> +<!ENTITY https-everywhere.popup.title "Notificación de HTTPS Everywhere 4.0development.11"> +<!ENTITY https-everywhere.popup.paragraph1 "Uups. Está usted usando la versión estable de HTTPS Everywhere, pero podríamos haberle elevado accidentalmente a la versión de desarrollo al publicar la última versión."> +<!ENTITY https-everywhere.popup.paragraph2 "¿Le gustaría volver a la versión estable?"> +<!ENTITY https-everywhere.popup.paragraph3 "Nos encantaría que continuase usando nuestra versión de desarrollo ¡y nos ayudara a hacer HTTPS Everywhere mejor! Podría encontrar que hay unos pocos fallos más aquí y allá, de los que puede informar a 'https-everywhere(a)eff.org'. Disculpe la inconveniencia, y gracias por usar HTTPS Everywhere."> +<!ENTITY https-everywhere.popup.keep "Manténganme en la versión de desarrollo."> +<!ENTITY https-everywhere.popup.revert "Descargar la última versión estable.">

1 0

[translation/https_everywhere_completed] Update translations for https_everywhere_completed
by translation＠torproject.org 27 Aug '13

27 Aug '13

commit 9ad0abc52c2660f5d4fdc7a03c6a4551230c8806 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Aug 27 10:16:16 2013 +0000 Update translations for https_everywhere_completed --- ru/https-everywhere.dtd | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/ru/https-everywhere.dtd b/ru/https-everywhere.dtd index 260ea3b..23f9c7b 100644 --- a/ru/https-everywhere.dtd +++ b/ru/https-everywhere.dtd @@ -34,4 +34,9 @@ <!ENTITY https-everywhere.source.filename "Имя файла"> <!ENTITY https-everywhere.source.unable_to_download "Не могу загрузить исходник"> - +<!ENTITY https-everywhere.popup.title "HTTPS Everywhere 4.0development.11 уведомление"> +<!ENTITY https-everywhere.popup.paragraph1 "Ой. Вы использовали стабильную версию HTTPS Everywhere, но мы, возможно, случайно обновили Вам её до разрабатываемой версии."> +<!ENTITY https-everywhere.popup.paragraph2 "Хотели бы Вы вернуться к стабильной версии?"> +<!ENTITY https-everywhere.popup.paragraph3 "Мы были бы рады, если бы Вы продолжили использовать разрабатываемую версию, и помогли нам сделать HTTPS Everywhere лучше! Вы могли бы найти несколько ошибок, и прислать их нам на https-everywhere(a)eff.org. Извините за неудобства, и спасибо за использование HTTPS Everywhere."> +<!ENTITY https-everywhere.popup.keep "Остаться на разрабатываемой версии"> +<!ENTITY https-everywhere.popup.revert "Загрузить последнюю стабильную версию">

1 0

[translation/https_everywhere] Update translations for https_everywhere
by translation＠torproject.org 27 Aug '13

27 Aug '13

commit aae82a3d95699129e8d9bc61520bf13578d97054 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Aug 27 10:16:11 2013 +0000 Update translations for https_everywhere --- ru/https-everywhere.dtd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ru/https-everywhere.dtd b/ru/https-everywhere.dtd index 2f33090..23f9c7b 100644 --- a/ru/https-everywhere.dtd +++ b/ru/https-everywhere.dtd @@ -34,8 +34,8 @@ <!ENTITY https-everywhere.source.filename "Имя файла"> <!ENTITY https-everywhere.source.unable_to_download "Не могу загрузить исходник"> -<!ENTITY https-everywhere.popup.title ""> -<!ENTITY https-everywhere.popup.paragraph1 ""> +<!ENTITY https-everywhere.popup.title "HTTPS Everywhere 4.0development.11 уведомление"> +<!ENTITY https-everywhere.popup.paragraph1 "Ой. Вы использовали стабильную версию HTTPS Everywhere, но мы, возможно, случайно обновили Вам её до разрабатываемой версии."> <!ENTITY https-everywhere.popup.paragraph2 "Хотели бы Вы вернуться к стабильной версии?"> <!ENTITY https-everywhere.popup.paragraph3 "Мы были бы рады, если бы Вы продолжили использовать разрабатываемую версию, и помогли нам сделать HTTPS Everywhere лучше! Вы могли бы найти несколько ошибок, и прислать их нам на https-everywhere(a)eff.org. Извините за неудобства, и спасибо за использование HTTPS Everywhere."> <!ENTITY https-everywhere.popup.keep "Остаться на разрабатываемой версии">

1 0

[ooni-probe/master] Link TaskManagers by Least Available Slots
by art＠torproject.org 27 Aug '13

27 Aug '13

commit 0467a6ea8c33a35beb315e56de3b2f5bdae605d6 Author: aagbsn <aagbsn(a)extc.org> Date: Thu Aug 22 13:00:45 2013 +0200 Link TaskManagers by Least Available Slots A LinkedTaskManager only has availableSlots if its child TaskManager also has availableSlots. Children LinkedTaskManagers must notify the parent LinkedTaskManager when a task is complete because the task queue is event-driven. --- ooni/director.py | 4 ++++ ooni/managers.py | 28 ++++++++++++++++++++++++++-- 2 files changed, 30 insertions(+), 2 deletions(-) diff --git a/ooni/director.py b/ooni/director.py index 2c48bb0..1e3f062 100644 --- a/ooni/director.py +++ b/ooni/director.py @@ -71,6 +71,10 @@ class Director(object): self.reportEntryManager = ReportEntryManager() self.reportEntryManager.director = self + # Link the TaskManager's by least available slots. + self.measurementManager.child = self.reportEntryManager + # Notify the parent when tasks complete # XXX deadlock!? + self.reportEntryManager.parent = self.measurementManager self.successfulMeasurements = 0 self.failedMeasurements = 0 diff --git a/ooni/managers.py b/ooni/managers.py index ff7c2f2..cc2d067 100644 --- a/ooni/managers.py +++ b/ooni/managers.py @@ -129,7 +129,31 @@ class TaskManager(object): """ raise NotImplemented -class MeasurementManager(TaskManager): +class LinkedTaskManager(TaskManager): + def __init__(self): + super(LinkedTaskManager, self).__init__() + self.child = None + self.parent = None + + @property + def availableSlots(self): + mySlots = self.concurrency - len(self._active_tasks) + if self.child: + s = self.child.availableSlots + return min(s, mySlots) + return mySlots + + def _succeeded(self, result, task): + super(LinkedTaskManager, self)._succeeded(result, task) + if self.parent: + self.parent._fillSlots() + + def _failed(self, result, task): + super(LinkedTaskManager, self)._failed(result, task) + if self.parent: + self.parent._fillSlots() + +class MeasurementManager(LinkedTaskManager): """ This is the Measurement Tracker. In here we keep track of active measurements and issue new measurements once the active ones have been completed. @@ -155,7 +179,7 @@ class MeasurementManager(TaskManager): def failed(self, failure, measurement): pass -class ReportEntryManager(TaskManager): +class ReportEntryManager(LinkedTaskManager): def __init__(self): if config.advanced.reporting_retries: self.retries = config.advanced.reporting_retries

1 0

[ooni-probe/master] Handle more failure types
by art＠torproject.org 27 Aug '13

27 Aug '13

commit 783fc4673a674bec54b958d23f3356bcc0c27286 Author: Arturo Filastò <art(a)fuffa.org> Date: Thu Aug 22 15:20:16 2013 +0200 Handle more failure types * Make sure tests will still work properly even if Tor is not running. --- ooni/errors.py | 17 +++++++++++++---- ooni/templates/httpt.py | 7 +++++-- 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/ooni/errors.py b/ooni/errors.py index c9d5fbe..908ed55 100644 --- a/ooni/errors.py +++ b/ooni/errors.py @@ -2,7 +2,7 @@ from twisted.internet.defer import TimeoutError as DeferTimeoutError from twisted.web._newclient import ResponseNeverReceived from twisted.internet.error import ConnectionRefusedError, TCPTimedOutError -from twisted.internet.error import DNSLookupError +from twisted.internet.error import DNSLookupError, ConnectError, ConnectionLost from twisted.internet.error import TimeoutError as GenericTimeoutError from txsocksx.errors import SOCKSError @@ -26,7 +26,8 @@ def handleAllFailures(failure): SOCKSError, MethodsNotAcceptedError, AddressNotSupported, ConnectionError, NetworkUnreachable, ConnectionLostEarly, ConnectionNotAllowed, NoAcceptableMethods, ServerFailure, - HostUnreachable, ConnectionRefused, TTLExpired, CommandNotSupported) + HostUnreachable, ConnectionRefused, TTLExpired, CommandNotSupported, + ConnectError, ConnectionLost) return failureToString(failure) @@ -47,9 +48,17 @@ def failureToString(failure): string = None if isinstance(failure.value, ConnectionRefusedError): - log.err("Connection refused. The backend may be down") + log.err("Connection refused.") string = 'connection_refused_error' + elif isinstance(failure.value, ConnectionLost): + log.err("Connection lost.") + string = 'connection_lost_error' + + elif isinstance(failure.value, ConnectError): + log.err("Connect error.") + string = 'connect_error' + elif isinstance(failure.value, gaierror): log.err("Address family for hostname not supported") string = 'address_family_not_supported_error' @@ -110,7 +119,7 @@ def failureToString(failure): string = 'socks_error' else: - log.err("Unknown failure type: %s" % type(failure)) + log.err("Unknown failure type: %s" % type(failure.value)) string = 'unknown_failure %s' % str(failure.value) return string diff --git a/ooni/templates/httpt.py b/ooni/templates/httpt.py index 0bca5df..472cc77 100644 --- a/ooni/templates/httpt.py +++ b/ooni/templates/httpt.py @@ -286,6 +286,9 @@ class HTTPTest(NetTestCase): request['url'] = url request['headers'] = headers request['body'] = body + request['tor'] = False + if use_tor: + request['tor'] = True if self.randomizeUA: log.debug("Randomizing user agent") @@ -304,9 +307,9 @@ class HTTPTest(NetTestCase): headers = TrueHeaders(request['headers']) def errback(failure, request): - failure_string = handleAllFailures(failure) log.err("Error performing %s" % request) - HTTPTest.addToReport(self, request, failure_string=failure_string) + failure_string = handleAllFailures(failure) + self.addToReport(request, failure_string=failure_string) return failure d = agent.request(request['method'], request['url'], headers,

1 0

[ooni-probe/master] Write the number of currently running tasks to a text file
by art＠torproject.org 27 Aug '13

27 Aug '13

commit f8f2890502721a8bc47ba6997e5984fc40e303a2 Author: Arturo Filastò <art(a)fuffa.org> Date: Thu Aug 22 15:40:23 2013 +0200 Write the number of currently running tasks to a text file --- ooni/nettest.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/ooni/nettest.py b/ooni/nettest.py index 4c1c6fa..53dcb15 100644 --- a/ooni/nettest.py +++ b/ooni/nettest.py @@ -403,6 +403,10 @@ class NetTestState(object): def checkAllTasksDone(self): log.debug("Checking all tasks for completion %s == %s" % (self.doneTasks, self.tasks)) + with open('task_count_log.txt', 'a+') as f: + diff = int(self.tasks) - int(self.doneTasks) + f.write("%s, %s, %s, %s\n" % (otime.timestamp(), self.doneTasks, self.tasks, diff)) + if self.completedScheduling and \ self.doneTasks == self.tasks: self.allTasksDone.callback(self.doneTasks)

1 0

[ooni-probe/master] Reschedule failures immediately
by art＠torproject.org 27 Aug '13

27 Aug '13

commit c3cc01f9a35943d2c864632f3fb0d280d1c5e96a Author: aagbsn <aagbsn(a)extc.org> Date: Sun Aug 25 14:50:41 2013 +0200 Reschedule failures immediately Reschedule failures immediately so they do not accrue at the end of the task list and eat memory. --- ooni/managers.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ooni/managers.py b/ooni/managers.py index cc2d067..36b791e 100644 --- a/ooni/managers.py +++ b/ooni/managers.py @@ -37,8 +37,8 @@ class TaskManager(object): if task.failures <= self.retries: log.debug("Rescheduling...") - self._tasks = itertools.chain(self._tasks, - makeIterable(task)) + self._tasks = itertools.chain(makeIterable(task), self._tasks) + else: # This fires the errback when the task is done but has failed. log.err('Permanent failure for %s' % task)

1 0

[ooni-probe/master] Track the number of failures, not the instances.
by art＠torproject.org 27 Aug '13

27 Aug '13

commit 23251cffeb0d23290206ad85ebc8e66f76e02945 Author: aagbsn <aagbsn(a)extc.org> Date: Sun Aug 25 14:59:35 2013 +0200 Track the number of failures, not the instances. The number of failures on very large scans (1m hosts) results in a large list. This commit tracks the number of failures instead of the objects. --- ooni/managers.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/ooni/managers.py b/ooni/managers.py index 36b791e..a229044 100644 --- a/ooni/managers.py +++ b/ooni/managers.py @@ -22,7 +22,7 @@ class TaskManager(object): def __init__(self): self._tasks = iter(()) self._active_tasks = [] - self.failures = [] + self.failures = 0 def _failed(self, failure, task): """ @@ -33,7 +33,7 @@ class TaskManager(object): log.exception(failure) self._active_tasks.remove(task) - self.failures.append((failure, task)) + self.failures = self.failures + 1 if task.failures <= self.retries: log.debug("Rescheduling...") @@ -85,7 +85,7 @@ class TaskManager(object): @property def failedMeasurements(self): - return len(self.failures) + return self.failures @property def availableSlots(self): @@ -110,7 +110,7 @@ class TaskManager(object): """ This is called to start the task manager. """ - self.failures = [] + self.failures = 0 self._fillSlots()

1 0