- tor-commits - lists.torproject.org

[stem/master] Descriptor pydoc mistake
by atagar＠torproject.org 24 Jul '12

24 Jul '12

commit 051ed27b99a5b8796777b86360a41bec52527b12 Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Jul 24 08:42:56 2012 -0700 Descriptor pydoc mistake The parse_file() helper no longer runs over bridge descriptors, just relays. The reason for this is that only relay descriptors exist in the cached-descriptors file so there's little need for this to include hacks to differentiate between the two. --- stem/descriptor/server_descriptor.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/stem/descriptor/server_descriptor.py b/stem/descriptor/server_descriptor.py index f54d5b1..759d6a5 100644 --- a/stem/descriptor/server_descriptor.py +++ b/stem/descriptor/server_descriptor.py @@ -70,8 +70,8 @@ SINGLE_FIELDS = ( def parse_file(descriptor_file, validate = True): """ - Iterates over the server descriptors in a file. This can read either relay or - bridge server descriptors. + Iterates over the server descriptors in a file. This can only read relay or + descriptors. :param file descriptor_file: file with descriptor content :param bool validate: checks the validity of the descriptor's content if True, skips these checks otherwise

1 0

[tor/master] Fix small memleak introduced in recent patch; fixe 6455.
by nickm＠torproject.org 24 Jul '12

24 Jul '12

commit 08e65ce04fbc68389095c627ae752a9784f5739a Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jul 24 10:20:00 2012 -0400 Fix small memleak introduced in recent patch; fixe 6455. --- src/or/config.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/src/or/config.c b/src/or/config.c index 0271cfd..281c458 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -4025,7 +4025,7 @@ options_validate(or_options_t *old_options, or_options_t *options, log_notice(LD_GENERAL, "Tor is not configured as a relay but you specified" " a ServerTransportPlugin line (%s). The ServerTransportPlugin " "line will be ignored.", - esc_for_log(options->ServerTransportPlugin->value)); + escaped(options->ServerTransportPlugin->value)); } if (options->ConstrainedSockets) {

1 0

[translation/vidalia_alpha] Update translations for vidalia_alpha
by translation＠torproject.org 24 Jul '12

24 Jul '12

commit 3fd2c4c5d6fb5053f105e03eb32803d711c6c868 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Jul 24 14:15:11 2012 +0000 Update translations for vidalia_alpha --- tr/vidalia_tr.po | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tr/vidalia_tr.po b/tr/vidalia_tr.po index f7fa015..77335a9 100644 --- a/tr/vidalia_tr.po +++ b/tr/vidalia_tr.po @@ -7,7 +7,7 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: https://trac.torproject.org/projects/tor\n" "POT-Creation-Date: 2012-03-21 17:46+0000\n" -"PO-Revision-Date: 2012-07-24 12:34+0000\n" +"PO-Revision-Date: 2012-07-24 13:57+0000\n" "Last-Translator: erg26 <ergungorler(a)gmail.com>\n" "Language-Team: translations(a)vidalia-project.net\n" "MIME-Version: 1.0\n" @@ -3269,15 +3269,15 @@ msgstr "Panoya Kopyala" msgctxt "StatusEventWidget" msgid "The Tor Software is Running" -msgstr "" +msgstr "Tor Programı Çalışıyor" msgctxt "StatusEventWidget" msgid "You are currently running version \"%1\" of the Tor software." -msgstr "" +msgstr "Şu anda Tor programının \"%1\" versiyonu çalışıyor." msgctxt "StatusEventWidget" msgid "The Tor Software is not Running" -msgstr "" +msgstr "Tor Programı Çalışmıyor" msgctxt "StatusEventWidget" msgid ""

1 0

[metrics-lib/master] Make parse history in descriptor reader more accessible.
by karsten＠torproject.org 24 Jul '12

24 Jul '12

commit 20f9d5574f1a25fe586eff18b7142894d65fb4ec Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Sat Jul 21 12:11:47 2012 +0200 Make parse history in descriptor reader more accessible. So far, the only way to prevent files from being parsed repeatedly in distinct runs was to specify a history file that only metrics-lib was supposed to read and write. However, some applications may want to specify the list of files to exclude themselves, or they may want to learn which files have been excluded and which have been parsed. These applications shouldn't be forced to mess with the history file. Add three methods to the descriptor reader for these applications. They should also play nicely together with the history file approach. AFAIK, stem has methods with the same purpose but a slightly different semantic. --- .../torproject/descriptor/DescriptorReader.java | 19 +++++ .../descriptor/impl/DescriptorReaderImpl.java | 73 +++++++++++++++---- 2 files changed, 76 insertions(+), 16 deletions(-) diff --git a/src/org/torproject/descriptor/DescriptorReader.java b/src/org/torproject/descriptor/DescriptorReader.java index 167f8b6..2783bd4 100644 --- a/src/org/torproject/descriptor/DescriptorReader.java +++ b/src/org/torproject/descriptor/DescriptorReader.java @@ -4,6 +4,7 @@ package org.torproject.descriptor; import java.io.File; import java.util.Iterator; +import java.util.SortedMap; /* Read descriptors from one or more local directories. */ public interface DescriptorReader { @@ -23,6 +24,24 @@ public interface DescriptorReader { * last modified timestamp and the absolute path of a file. */ public void setExcludeFiles(File historyFile); + /* Exclude files if they haven't changed since the corresponding last + * modified timestamps. Can be used instead of (or in addition to) a + * history file. */ + public void setExcludedFiles(SortedMap<String, Long> excludedFiles); + + /* Return files and lost modified timestamps of files that exist in the + * input directory or directories, but that have been excluded from + * parsing, because they haven't changed since they were last read. + * Can be used instead of (or in addition to) a history file when + * combined with the set of parsed files. */ + public SortedMap<String, Long> getExcludedFiles(); + + /* Return files and last modified timestamps of files that exist in the + * input directory or directories and that have been parsed. Can be + * used instead of (or in addition to) a history file when combined with + * the set of excluded files. */ + public SortedMap<String, Long> getParsedFiles(); + /* Fail descriptor parsing when encountering an unrecognized line. This * is not set by default, because the Tor specifications allow for new * lines to be added that shall be ignored by older Tor versions. But diff --git a/src/org/torproject/descriptor/impl/DescriptorReaderImpl.java b/src/org/torproject/descriptor/impl/DescriptorReaderImpl.java index ddb6ea6..e3fb908 100644 --- a/src/org/torproject/descriptor/impl/DescriptorReaderImpl.java +++ b/src/org/torproject/descriptor/impl/DescriptorReaderImpl.java @@ -59,6 +59,31 @@ public class DescriptorReaderImpl implements DescriptorReader { this.historyFile = historyFile; } + private SortedMap<String, Long> excludedFiles; + public void setExcludedFiles(SortedMap<String, Long> excludedFiles) { + if (this.hasStartedReading) { + throw new IllegalStateException("Reconfiguration is not permitted " + + "after starting to read."); + } + this.excludedFiles = excludedFiles; + } + + public SortedMap<String, Long> getExcludedFiles() { + if (this.reader == null || !this.reader.hasFinishedReading) { + throw new IllegalStateException("Operation is not permitted before " + + "finishing to read."); + } + return new TreeMap<String, Long>(this.reader.excludedFilesAfter); + } + + public SortedMap<String, Long> getParsedFiles() { + if (this.reader == null || !this.reader.hasFinishedReading) { + throw new IllegalStateException("Operation is not permitted before " + + "finishing to read."); + } + return new TreeMap<String, Long>(this.reader.parsedFilesAfter); + } + private boolean failUnrecognizedDescriptorLines = false; public void setFailUnrecognizedDescriptorLines() { if (this.hasStartedReading) { @@ -68,6 +93,7 @@ public class DescriptorReaderImpl implements DescriptorReader { this.failUnrecognizedDescriptorLines = true; } + private DescriptorReaderRunnable reader; public Iterator<DescriptorFile> readDescriptors() { if (this.hasStartedReading) { throw new IllegalStateException("Initiating reading is only " @@ -76,10 +102,10 @@ public class DescriptorReaderImpl implements DescriptorReader { this.hasStartedReading = true; BlockingIteratorImpl<DescriptorFile> descriptorQueue = new BlockingIteratorImpl<DescriptorFile>(); - DescriptorReaderRunnable reader = new DescriptorReaderRunnable( - this.directories, this.tarballs, descriptorQueue, - this.historyFile, this.failUnrecognizedDescriptorLines); - new Thread(reader).start(); + this.reader = new DescriptorReaderRunnable(this.directories, + this.tarballs, descriptorQueue, this.historyFile, + this.excludedFiles, this.failUnrecognizedDescriptorLines); + new Thread(this.reader).start(); return descriptorQueue; } @@ -88,15 +114,24 @@ public class DescriptorReaderImpl implements DescriptorReader { private List<File> tarballs; private BlockingIteratorImpl<DescriptorFile> descriptorQueue; private File historyFile; + private SortedMap<String, Long> + excludedFilesBefore = new TreeMap<String, Long>(), + excludedFilesAfter = new TreeMap<String, Long>(), + parsedFilesAfter = new TreeMap<String, Long>(); private DescriptorParser descriptorParser; + private boolean hasFinishedReading = false; private DescriptorReaderRunnable(List<File> directories, List<File> tarballs, BlockingIteratorImpl<DescriptorFile> descriptorQueue, - File historyFile, boolean failUnrecognizedDescriptorLines) { + File historyFile, SortedMap<String, Long> excludedFiles, + boolean failUnrecognizedDescriptorLines) { this.directories = directories; this.tarballs = tarballs; this.descriptorQueue = descriptorQueue; this.historyFile = historyFile; + if (excludedFiles != null) { + this.excludedFilesBefore = excludedFiles; + } this.descriptorParser = new DescriptorParserImpl(); this.descriptorParser.setFailUnrecognizedDescriptorLines( failUnrecognizedDescriptorLines); @@ -105,12 +140,10 @@ public class DescriptorReaderImpl implements DescriptorReader { this.readOldHistory(); this.readDescriptors(); this.readTarballs(); + this.hasFinishedReading = true; this.descriptorQueue.setOutOfDescriptors(); this.writeNewHistory(); } - private SortedMap<String, Long> - oldHistory = new TreeMap<String, Long>(), - newHistory = new TreeMap<String, Long>(); private void readOldHistory() { if (this.historyFile == null) { return; @@ -127,7 +160,7 @@ public class DescriptorReaderImpl implements DescriptorReader { long lastModifiedMillis = Long.parseLong(line.substring(0, line.indexOf(" "))); String absolutePath = line.substring(line.indexOf(" ") + 1); - this.oldHistory.put(absolutePath, lastModifiedMillis); + this.excludedFilesBefore.put(absolutePath, lastModifiedMillis); } br.close(); } catch (IOException e) { @@ -146,7 +179,10 @@ public class DescriptorReaderImpl implements DescriptorReader { } BufferedWriter bw = new BufferedWriter(new FileWriter( this.historyFile)); - for (Map.Entry<String, Long> e : this.newHistory.entrySet()) { + SortedMap<String, Long> newHistory = new TreeMap<String, Long>(); + newHistory.putAll(this.excludedFilesAfter); + newHistory.putAll(this.parsedFilesAfter); + for (Map.Entry<String, Long> e : newHistory.entrySet()) { String absolutePath = e.getKey(); long lastModifiedMillis = e.getValue(); bw.write(String.valueOf(lastModifiedMillis) + " " + absolutePath @@ -172,11 +208,14 @@ public class DescriptorReaderImpl implements DescriptorReader { } else { String absolutePath = file.getAbsolutePath(); long lastModifiedMillis = file.lastModified(); - this.newHistory.put(absolutePath, lastModifiedMillis); - if (this.oldHistory.containsKey(absolutePath) && - this.oldHistory.get(absolutePath) == lastModifiedMillis) { + if (this.excludedFilesBefore.containsKey(absolutePath) && + this.excludedFilesBefore.get(absolutePath) == + lastModifiedMillis) { + this.excludedFilesAfter.put(absolutePath, + lastModifiedMillis); continue; } + this.parsedFilesAfter.put(absolutePath, lastModifiedMillis); DescriptorFileImpl descriptorFile = new DescriptorFileImpl(); try { descriptorFile.setDirectory(directory); @@ -206,11 +245,13 @@ public class DescriptorReaderImpl implements DescriptorReader { } String absolutePath = tarball.getAbsolutePath(); long lastModifiedMillis = tarball.lastModified(); - this.newHistory.put(absolutePath, lastModifiedMillis); - if (this.oldHistory.containsKey(absolutePath) && - this.oldHistory.get(absolutePath) == lastModifiedMillis) { + if (this.excludedFilesBefore.containsKey(absolutePath) && + this.excludedFilesBefore.get(absolutePath) == + lastModifiedMillis) { + this.excludedFilesAfter.put(absolutePath, lastModifiedMillis); continue; } + this.parsedFilesAfter.put(absolutePath, lastModifiedMillis); try { FileInputStream in = new FileInputStream(tarball); if (in.available() > 0) {

1 0

[ooni-probe/master] Implement SSL support for OONIB
by art＠torproject.org 24 Jul '12

24 Jul '12

commit 59daf0150e2661be93ddde2bee57feb3ca7ac5b6 Author: Arturo Filastò <arturo(a)filasto.net> Date: Tue Jul 24 15:10:37 2012 +0200 Implement SSL support for OONIB --- .gitignore | 1 + oonib/README.md | 10 ++++++++++ oonib/backends/ssl.py | 7 +++++++ oonib/oonibackend.conf | 8 -------- oonib/oonibackend.conf.sample | 10 ++++++++++ oonib/oonibackend.py | 11 ++++++++++- 6 files changed, 38 insertions(+), 9 deletions(-) diff --git a/.gitignore b/.gitignore index 553482d..7f270bb 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,4 @@ proxy-lists/italy-dns-ips.txt proxy-lists/italy-http-ips.txt private/* /ooni/plugins/dropin.cache +oonib/oonibackend.conf diff --git a/oonib/README.md b/oonib/README.md new file mode 100644 index 0000000..6823d06 --- /dev/null +++ b/oonib/README.md @@ -0,0 +1,10 @@ +# Generate self signed certs for OONIB + + openssl genrsa -des3 -out private.key 4096 + openssl req -new -key private.key -out server.csr + cp private.key private.key.org + # Remove passphrase from key + openssl rsa -in private.key.org -out private.key + openssl x509 -req -days 365 -in server.csr -signkey private.key -out certificate.crt + rm private.key.org + diff --git a/oonib/backends/ssl.py b/oonib/backends/ssl.py new file mode 100644 index 0000000..5f19686 --- /dev/null +++ b/oonib/backends/ssl.py @@ -0,0 +1,7 @@ +from twisted.internet import ssl + +class SSLContext(ssl.DefaultOpenSSLContextFactory): + def __init__(self, config): + ssl.DefaultOpenSSLContextFactory.__init__(self, config.main.ssl_private_key, + config.main.ssl_certificate) + diff --git a/oonib/oonibackend.conf b/oonib/oonibackend.conf deleted file mode 100644 index 5265045..0000000 --- a/oonib/oonibackend.conf +++ /dev/null @@ -1,8 +0,0 @@ -[main] -http_port = 8080 -dns_udp_port = 5354 -dns_tcp_port = 8002 -daphn3_port = 9666 -server_version = Apache -[daphn3] -pcap_file = /Users/y/Documents/workspace/ooni-probe.new/oonib/server.pcap diff --git a/oonib/oonibackend.conf.sample b/oonib/oonibackend.conf.sample new file mode 100644 index 0000000..a5cbbd3 --- /dev/null +++ b/oonib/oonibackend.conf.sample @@ -0,0 +1,10 @@ +[main] +http_port = 8080 +dns_udp_port = 5354 +dns_tcp_port = 8002 +daphn3_port = 9666 +server_version = Apache +ssl_private_key = /path/to/private.key +ssl_certificate = /path/to/certificate.crt +[daphn3] +pcap_file = /path/to/server.pcap diff --git a/oonib/oonibackend.py b/oonib/oonibackend.py index fe1a760..c5a866b 100755 --- a/oonib/oonibackend.py +++ b/oonib/oonibackend.py @@ -18,6 +18,7 @@ from twisted.names import dns from oonib.common import config from oonib.backends.http import HTTPBackend +from oonib.backends.ssl import SSLContext from oonib.backends.dns import ProxyDNSServer from oonib.backends.daphn3 import Daphn3Server @@ -26,7 +27,15 @@ server.version = config.main.server_version application = service.Application('oonibackend') serviceCollection = service.IServiceCollection(application) -internet.TCPServer(int(config.main.http_port), server.Site(HTTPBackend())).setServiceParent(serviceCollection) + +internet.TCPServer(int(config.main.http_port), + server.Site(HTTPBackend()) + ).setServiceParent(serviceCollection) + +internet.SSLServer(int(config.main.ssl_port), + server.Site(HTTPBackend()), + SSLContext(config), + ).setServiceParent(serviceCollection) # Start the DNS Server related services TCPDNSServer = ProxyDNSServer()

1 0

[translation/vidalia_alpha] Update translations for vidalia_alpha
by translation＠torproject.org 24 Jul '12

24 Jul '12

commit 29f2f052b2f663249274644fdb5fe46dde469151 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Jul 24 12:45:13 2012 +0000 Update translations for vidalia_alpha --- tr/vidalia_tr.po | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/tr/vidalia_tr.po b/tr/vidalia_tr.po index 8733f63..f7fa015 100644 --- a/tr/vidalia_tr.po +++ b/tr/vidalia_tr.po @@ -7,7 +7,7 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: https://trac.torproject.org/projects/tor\n" "POT-Creation-Date: 2012-03-21 17:46+0000\n" -"PO-Revision-Date: 2012-07-20 12:44+0000\n" +"PO-Revision-Date: 2012-07-24 12:34+0000\n" "Last-Translator: erg26 <ergungorler(a)gmail.com>\n" "Language-Team: translations(a)vidalia-project.net\n" "MIME-Version: 1.0\n" @@ -2667,11 +2667,11 @@ msgstr "SOCKS 4" msgctxt "NetworkPage" msgid "SOCKS 5" -msgstr "" +msgstr "SOCKS 5" msgctxt "NetworkPage" msgid "HTTP / HTTPS" -msgstr "" +msgstr "HTTP / HTTPS" msgctxt "NetworkPage" msgid "You must specify one or more bridges." @@ -3183,19 +3183,19 @@ msgstr "<a href=\"bridges.finding\">Nasıl bir köprü bulabilirim?</a>" msgctxt "ServerPage" msgid "<a href=\"#bridgeHelp\">What's this?</a>" -msgstr "" +msgstr "<a href=\"#bridgeHelp\">Bu nedir?</a>" msgctxt "ServerPage" msgid "Automatically distribute my bridge address" -msgstr "" +msgstr "Köprü adresimi otomatik olarak dağıt" msgctxt "ServerPage" msgid "Relay traffic inside the Tor network (non-exit relay)" -msgstr "" +msgstr "Tor ağında anahtarlama trafiği (çıkış anahtarlaması olmayan)" msgctxt "ServerPage" msgid "Relay traffic for the Tor network (exit relay)" -msgstr "" +msgstr "Tor ağı için anahtarlama trafiği (çıkış anahtarlaması)" msgctxt "ServerPage" msgid "Mirror the Relay Directory" @@ -3265,7 +3265,7 @@ msgstr "" msgctxt "StatusEventWidget" msgid "Copy to Clipboard" -msgstr "" +msgstr "Panoya Kopyala" msgctxt "StatusEventWidget" msgid "The Tor Software is Running"

1 0

[ooni-probe/master] Merge branch 'master' of git://github.com/hellais/ooni-probe
by art＠torproject.org 24 Jul '12

24 Jul '12

1 0

[ooni-probe/master] Merge branch 'master' of git://github.com/hellais/ooni-probe
by art＠torproject.org 24 Jul '12

24 Jul '12

1 0

[ooni-probe/master] Merge branch 'twisted' of git://github.com/hellais/ooni-probe
by art＠torproject.org 24 Jul '12

24 Jul '12

commit 1f7c456001b9237be4bb3c1e2e468fa55da489e0 Merge: caaf6d2 507d0d1 Author: Arturo Filastò <hellais(a)gmail.com> Date: Thu May 3 11:50:45 2012 +0200 Merge branch 'twisted' of git://github.com/hellais/ooni-probe lib/traceroute.py | 32 ++++ lib/txtraceroute.py | 389 ++++++++++++++++++++++++++++++++++++++++++++++++++ oonicli.py | 142 ++++++++++++++++++ ooniprobe.py | 3 +- plugins/__init__.py | 3 + plugins/dropin.cache | 48 ++++++ plugins/skel.py | 20 +++ plugoo/interface.py | 2 + plugoo/nodes.py | 12 +- plugoo/reports.py | 5 +- plugoo/tests.py | 99 +++++++++++++ plugoo/work.py | 162 +++++++++++++++++++++ plugoo/workers.py | 36 ----- skel.py | 10 ++ unittest/tests.py | 27 ++++ 15 files changed, 946 insertions(+), 44 deletions(-)

1 0

[ooni-probe/master] Update readme with some more useful information
by art＠torproject.org 24 Jul '12

24 Jul '12

commit bf2079f09b8257be0acd4bb2d76a0f362b6943bd Author: Arturo Filastò <arturo(a)filasto.net> Date: Tue Jul 24 13:34:52 2012 +0200 Update readme with some more useful information convert readme to markdown. --- README | 70 ---------------------------------------------- README.md | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 93 insertions(+), 70 deletions(-) diff --git a/README b/README deleted file mode 100644 index 75262bf..0000000 --- a/README +++ /dev/null @@ -1,70 +0,0 @@ -ooni-probe - Open Observatory of Network Interference - -"The Net interprets censorship as damage and routes around it." - - John Gilmore; TIME magazine (6 December 1993) - -OONI, the Open Observatory of Network Interference, is a global observation -network which aims is to collect high quality data using open methodologies, -using Free and Open Source Software (FL/OSS) to share observations and data -about the various types, methods, and amounts of network tampering in the world. - -With the belief that unfettered access to information is a intrinsic human right, -OONI seeks to observe levels of surveillance, censorship, and network discrimination -in order for people worldwide to have a clearer understanding of the ways in -which their access to information is filtered. - -The end goal of OONI is to collect data which can show an accurate -topology of network interference and censorship. Through this topology, it will be -possible to see what the internet looks like from nearly any location, including -what sites are censored, or have been tampered with, and by whom. We're calling -it filternet. - -OONI uses open methodologies and the data will be provided in raw -format to allow any researcher to indipendently draw their conclusions -from the results OONI tests. - -There are currently projects aimed at measuring censorship in one -way or another but they either use non open methodologies or their -tools are not open sources. OONI aims at filling up this gap by -creating the first open source framework for developing network -tests and collecting data on censorship. - -OONI revolves around three major concepts: Assets, Tests and -Reports. - -# Assets - -Assets are the inputs used inside Tests to detect censorship events. -These can be URL lists, keywords, ip addresses, packets or any kind -of set of data. -In the python specific implementation this is represented as a python -iterable object. This means that the Testing framework will be able -to iterate through every element in the Asset. - -# Tests - -This is the core of OONI. These are the actual tests that will be run -using as input (if an input is required) the Assets. -Tests can be summarized as an experiment and a control. The control -represents the expected result and the experiment is the network operation -being performed on the live network. If the experiment does not match up -with the control then a censorship event had occured. - -OONI probe provides some useful functionality to the application developer -that may be useful when developing censorship detection tests. For example -it is possible to make a request over the Tor network easily or use a fast -and flexible non-blocking HTTP client implementation. - -# Reports - -This is the data that is collected from the test. OONI probe provides a -flexible means of storing results and uploading this data to a remote -server or a flat file. - -The Test developer should include in the report as much data as possible -and can contain raw packet dumps as well as structured synthetic results. - -In future on top of ooni-probe Reports it will be possible to develop -flexible post-processing tools to allow data-visualization guru's to -properly visualize and contextualize the resulting data. - diff --git a/README.md b/README.md new file mode 100644 index 0000000..b39cb2b --- /dev/null +++ b/README.md @@ -0,0 +1,93 @@ +# ooni-probe - Open Observatory of Network Interference + +"The Net interprets censorship as damage and routes around it." + - John Gilmore; TIME magazine (6 December 1993) + +OONI, the Open Observatory of Network Interference, is a global observation +network which aims is to collect high quality data using open methodologies, +using Free and Open Source Software (FL/OSS) to share observations and data +about the various types, methods, and amounts of network tampering in the world. + +# Let's get started with this already! + +To run OONI-probe without having to install it you must tell python that it can +import modules from the root of ooni-probe. + +You must therefore run from the root of the repo: + + export PYTHONPATH=$PYTHONPATH:`pwd` + +Then to see what tests are available: + + cd ooni + python ooniprobe.py + +If you see some errors see INSTALL to install the missing dependencies. + +To list the help for a specific test: + + python ooniprobe.py httpt --help + + +# More details + +With the belief that unfettered access to information is a intrinsic human right, +OONI seeks to observe levels of surveillance, censorship, and network discrimination +in order for people worldwide to have a clearer understanding of the ways in +which their access to information is filtered. + +The end goal of OONI is to collect data which can show an accurate +topology of network interference and censorship. Through this topology, it will be +possible to see what the internet looks like from nearly any location, including +what sites are censored, or have been tampered with, and by whom. We're calling +it filternet. + +OONI uses open methodologies and the data will be provided in raw +format to allow any researcher to indipendently draw their conclusions +from the results OONI tests. + +There are currently projects aimed at measuring censorship in one +way or another but they either use non open methodologies or their +tools are not open sources. OONI aims at filling up this gap by +creating the first open source framework for developing network +tests and collecting data on censorship. + +OONI revolves around three major concepts: Assets, Tests and +Reports. + +## Assets + +Assets are the inputs used inside Tests to detect censorship events. +These can be URL lists, keywords, ip addresses, packets or any kind +of set of data. +In the python specific implementation this is represented as a python +iterable object. This means that the Testing framework will be able +to iterate through every element in the Asset. + +## Tests + +This is the core of OONI. These are the actual tests that will be run +using as input (if an input is required) the Assets. +Tests can be summarized as an experiment and a control. The control +represents the expected result and the experiment is the network operation +being performed on the live network. If the experiment does not match up +with the control then a censorship event had occured. + +OONI probe provides some useful functionality to the application developer +that may be useful when developing censorship detection tests. For example +it is possible to make a request over the Tor network easily or use a fast +and flexible non-blocking HTTP client implementation. + +## Reports + +This is the data that is collected from the test. OONI probe provides a +flexible means of storing results and uploading this data to a remote +server or a flat file. + +The Test developer should include in the report as much data as possible +and can contain raw packet dumps as well as structured synthetic results. + +In future on top of ooni-probe Reports it will be possible to develop +flexible post-processing tools to allow data-visualization guru's to +properly visualize and contextualize the resulting data. +

1 0