August 2014 - ooni-dev - lists.torproject.org

Least Authority's Ooni Work is Complete
by Taylor Hornby 29 Aug '14

29 Aug '14

Hi OTF, Ooni, and M-Lab, Our work on Milestone E resulted in detailed integration test instructions being added to the ooni-support README: https://github.com/LeastAuthority/ooni-support/tree/combined-leastauthority… We ran through the integration testing steps many times. We identified some issues and either fixed or documented each one. We re-defined Milestone F as a timebox to help M-Lab engineers finish the mlab-ns integration. They made great progress. The pipeline from the Ooni collector to the Ooni bouncer, through mlab-ns, is nearly complete. There's only one issue left, an encoding problem, which is blocked on M-Lab engineering effort. Hopefully that issue will be fixed by the end of today -- when our timebox runs out -- so we get to see the whole thing working. After that, it will be up to Ooni and M-Lab to agree about who will be hosting the bouncer, and then M-Lab can start collecting data. Most of our work on Milestone F is documented in these tickets: https://github.com/m-lab-tools/ooni-support/issues?q=is%3Aissue+milestone%3… https://github.com/m-lab-tools/ooni-support/issues?q=is%3Aissue+milestone%3… Or, more specifically, https://github.com/m-lab-tools/ooni-support/issues/11 https://github.com/m-lab-tools/ooni-support/issues/10 If you have any questions or concerns, please let us know. Thanks, -- Taylor Hornby

1 0

I just deleted /home/mlab_ooni on mlab2.nuq0t. :(
by Taylor Hornby 25 Aug '14

25 Aug '14

Hi Ooni and M-Lab, I just accidentally ran slicebuild.sh on mlab2.nuq0t, which deleted everything in /home/mlab_ooni. I intended to run it on mlab1, but I copied and pasted the wrong SSH command, and didn't realize I was on the wrong server until I tried to SCP the rpm off of it. I'm REALLY sorry about this. If I recall correctly, Arturo had mlab2 running an instance of ooni-backend for testing or development. Please let me know what was on there and I'll work on my own time to get it back to the way it was. I hope no important experiment data was deleted. :( Sorry, -- Taylor Hornby

2 1

Re: [ooni-dev] http_requests test single responses
by jony-port＠bitmessage.ch 24 Aug '14

24 Aug '14

Hi, On Thu, 31 Jul 2014 14:05:49 +0200 Arturo Filastò <art(a)torproject.org> wrote: > On 7/24/14, 2:52 PM, jony-port(a)bitmessage.ch wrote: > > > > It seems that the single response of the http_requests requests > > test results in 'body_length_match' and 'headers_match' fields set > > to true. > > > > Yes that is correct. Those values are initialized to none and if a > response is received for both control and experiment they are set to > either true or false. But when there is only one request this should never be as true, since there is only one request, thus nothing to compare with or I miss anything? > > From my understanding a single response request should raise an > > error set by the 'control_failure' or 'experiment_failure' field. > > > Yes that is correct, when a control or experiment request fails the > relative key will be set with the error message. > > For the list of error strings you should see: > > https://gitweb.torproject.org/ooni/spec.git/blob/HEAD:/data-formats/df-000-… I can't find the error string when there is only a single request instead of two, as the http_requests test case implies.

2 3

Reminder: OONI dev meetings Friday 15:00 UTC (17:00 CET)
by Arturo Filastò 21 Aug '14

21 Aug '14

Hello Oonitarians, This is a reminder that the weekly dev meeting will be happening on the #ooni channel on irc.oftc.net this Friday (tomorrow and every following Friday) at 15:00 UTC (17:00 CET). Everyone is invited. ~ Art.

1 0

An Update and The Remaining "M-Lab" Tickets
by Taylor Hornby 21 Aug '14

21 Aug '14

Hi M-Lab and Ooni, It looks like Least Authority's work on Ooni and M-Lab integration is nearing completion. In the past weeks, the M-Lab engineers have worked very quickly to get the necessary changes to mlab-ns and the Nagios pipeline done. I'm personally really excited to see the first test deployment against the real mlab-ns! As we were working through our milestones, we sought to avoid blocking, so we stepped over a few small tasks that would involve waiting on M-Lab. These are all tagged with the "M-Lab" label in the issue tracker: https://github.com/m-lab-tools/ooni-support/issues?q=is%3Aopen+is%3Aissue+l… Someone from M-Lab (and possibly Ooni) should take a look over them before any serious deployment begins. The most significant outstanding issue is deciding where the Ooni Bouncer is going to be hosted: On an M-Lab sliver, or by The Tor Project? We designed the mlab-ns integration to be compatible with either option. https://github.com/m-lab-tools/ooni-support/issues/7 https://github.com/m-lab-tools/ooni-support/issues/9 So that's that! I'll be around until the end of the month to answer any questions about our work, and to help with the Nagios integration if needed. -- Taylor Hornby

3 2

ooniprobe 1.1.0 is out!
by Arturo Filastò 19 Aug '14

19 Aug '14

Hello beautiful humans! We are very pleased to announce that a new version of ooniprobe is now out. In this new release of ooniprobe we have added a new command line tool for listing the reports that have not been published to a collector and that allows the probe operator to choose which ones they would like to upload. We have also made some privacy improvements to the reports (we will sanitize all things that may look like file paths) and added metadata associated with the maxmind database being used by the probe operator. Here is a more detailed list of what has been done: * Annotate on disk which reports we have submitted and which ones we have not: https://trac.torproject.org/projects/tor/ticket/11860 * Add tool called oonireport for publishing unpublished ooniprobe reports to a collector: https://trac.torproject.org/projects/tor/ticket/11862 * Probe Report does not leak filepaths anymore: https://trac.torproject.org/projects/tor/ticket/12706 * Reports now include version information about the maxmind database being used: https://trac.torproject.org/projects/tor/ticket/12771 * We now do integrity checks on the ooniprobe.conf file so that we don't start the tool if the config file is missing some settings or is not consistent: https://trac.torproject.org/projects/tor/ticket/11983 (thanks to Alejandro López (kudrom)) * Improvements have been made to the sniffer subsystem (thanks to Alejandro López (kudrom)) * Fix the multi protocol traceroute test. https://trac.torproject.org/projects/tor/ticket/12883 Minor bug fixes: * Fix dns_spoof test (by kudrom) https://trac.torproject.org/projects/tor/ticket/12486 * ooni might not look at requiresTor: https://trac.torproject.org/projects/tor/ticket/11858 * ooni spits out gobs of tracebacks if Tor is not running and the OONI config says it will be: https://trac.torproject.org/projects/tor/ticket/11859 * The README for ooni-probe should mention the bugtracker and repository https://trac.torproject.org/projects/tor/ticket/11980 For the moment you can download the new version from pypi here: https://pypi.python.org/pypi/ooniprobe. Soon a there will also be a new debian package for it (thank to Lunar^). As always feedback is greatly appreciated so please submit any bugs you may find to our bug tracker: https://trac.torproject.org/projects/tor/newticket?component=Ooni Have fun! ~ Art.

1 0

Feedback on new OONI test deck format
by Arturo Filastò 13 Aug '14

13 Aug '14

Hi all, We have been discussing lately some changes that we would like to make to the ooni-probe test deck format [1] and would like to have some feedback on what we have come up with. For those of you not familiar with ooni-probe, a test deck is basically a way of telling it "Run this list of OONI tests with these inputs and by the way be sure you also set these options properly when doing so". The previous deck format involved a lot of boilerplate and did not make it possible to bake inputs into the deck itself. This new format is supposed to overcome some of the limitations of the old design and we hope that a major redesign will not be needed in the near future. The specification can be found in git [3], but to facilitate commenting I am also going to paste it into this email. All feedback is greatly appreciated. -- BEGIN SPEC -- # Test deck specification * version: 0.1.0 * date: 2014-08-13 * author: Alejandro López (kudrom), Arturo Filastò # 0. Terminology Analyst: The person who writes a deck. Collector: A machine running the ooni-backend that collects the reports generated by the execution of ooni-probe. Nettest: A test whose execution by ooniprobe creates a report sent later to the collector if provided. Ooni: Open observatory of network interference. Ooni-probe: The client side of ooni used to execute a set of nettests. Tester: The person who executes ooni-probe. YAML: A human-readable data serialization format. # 1. Rationale To ease the execution of nettests, the ooni developers came up with the idea of a container that would allow a tester to easily execute a bunch of configured nettests.That container was called a deck. This way, an analyst interested in a particular behaviour would write a deck; then, she would distribute that deck to every tester interested in the ongoing analysis. Finally, the tester would execute ooni-probe with that deck to properly create and send the nettests' reports. Unfortunately, right now both the deck format and ooni-probe don't allow that level of automation. The way we want to solve this situation is by writing a new spec for the deck format that would allow us to write, with great confidence, the given functionality to ooni-probe. The document that you're reading is that spec. # 2. Goals Allow an analyst to reuse well written and tested nettests in an easy way. Allow a tester to execute easily a battery of nettests with a complex configuration. Grant privacy of the tester. # 3. The data format Every deck is a yaml file composed of two major sections: the header and the body. The header is a dictionary that provides all the shared and global configuration of every nettest included in the deck. Its main purpose is to reduce boilerplate by letting the analyst express common behaviour in one section instead of in every nettest execution. The ooni-probe options allowed in this section are: 1. collector: Address of the collector of test results. 2. bouncer: Address of the bouncer for test helpers. 3. annotations: Annotate the report with a key:value[, key:value] format. 4. no-collector: Disable the collector. (FLAG) 5. no-geoip: Disable the geoip support. (FLAG) Every option of the header section is a dictionary's key except that labeled with a FLAG, which are members of a list called flags. So for example a valid header section is the following: ``` header: collector: 'http://localhost' annotations: key1: value1 key2: value2 flags: - no-collector ``` The deck header can also contain metadata associated to the test deck. The possible fields are: 1. name: The name of the test deck. 2. description: A short description of the test deck. 3. author: The author of the test deck. 4. version: A version number for the test deck. 5. requires-root: A flag to indicate that the test deck requires root to run. (FLAG) 6. requires-tor: A flag to indicate that the test deck requires tor to run. (FLAG) The header may also be omitted. The body is a list composed of one element per nettest execution. Every nettest execution is a dictionary composed of the following three keys: 1. nettest: name of the nettest to execute (MANDATORY) 2. local_options: local_options of the test execution (OPTIONAL) 3. global_options: global options of the test execution (OPTIONAL) In the same way that with the header, every option can be a member of a flag list if that options doesn't have any arguments. A valid body can be: ``` body: - nettest: manipulation/http_request local_options: url: 'http://torproject.org' global_options: collector: 'http://localhost' flags: - no-geoip - nettest: manipulation/captiveportal ``` All file paths must be relative and they must start with "deck/" if they are referring to files contained inside of the test deck. They must start with "http(o|s)://" if they are referring to files to be downloaded via Tor. All other file paths should be ignored and raise an exception. This is because we do not want an analyst creating a deck to be pass as arguments to a test arbitrary files on the testers filesystem. # 3.1 Container format The container proposed is tar+gzip because it's well supported in python. The deck container will be composed of a directory named "deck" containing the deck file and the inputs. The directory layout will be: deck/test.deck deck/input-filename-1.txt deck/input-filename-2.txt This will then be compressed using tar+gzip. # 4. Implementation details ## 4.1 Introduction Each execution of a nettest in ooniprobe needs four main inputs. 1. The global config file 2. The global cmd options 3. The nettest/deck 4. The local options for each nettest The first three ones are mandatory to run a nettest, the last one depends on the nettest. When a single nettest is executed, all options except the first one are passed in the cmd line. The difference between the global config options and the global cmd options is that the second one has some shared options with the config file plus some additional subcommands to ooniprobe. When a deck is involved in the execution, the last three inputs are meant to be passed (at least partially) in the deck. In fact, the deck is no more than the three last inputs, with some subtleties that I expect to explain in the following sections. ## 4.2 Who overwrites who In a single-nettest execution, the global config is parsed and a global object is built. Then the cmd line is parsed and it overwrites the global configs. This is done to allow the setting of the most dynamic options in the console without the need of writing them each time to the config file. All the ooni-probe's code base is allowed to access that global object, included the nettests. So, the global options affect the way ooni-probe behaves but also can affect the way the executing nettest behaves. That's the reason why the deck must provide the specification of some global options, but not all because there are some powerful options that would put in danger the tester. The allowed options are listed in the section [3. The data format]. In a deck-nettest execution, the header section of the deck is parsed first and a global object is built. Then the config file is parsed and the object is overwritten when it applies. Finally the cmd line of the ooni-probe is parsed and the object is overwritten. We read the header section first to avoid the analyst overwrite some sensitive options of the config file, which only should be modified by the tester. ## 4.3 Copy-on-write To reduce the boilerplate in the deck, the file is splitted into a shared section for all nettests and a local one for each nettest. The idea is to allow the writer of the deck to express common and global behaviour of all the nettests in the header of the deck and to put specific and local options in each nettest element, as was already explained in [3. The data format]. What this means for the execution of the nettests is that when a nettest overwrites a global configuration of the header, this change is only visible to this nettest, not to every nettest who may ever execute in the same instance of ooni-probe. So what follows is that every change to the global options of ooni-probe in the local section of a deck should attend a copy-on-write policy regarding the global config object. ## 4.4 The input file Ooni-probe invokes every nettest method with the information saved in a file called the input file. This file is part of the local configuration of every test, and therefore must be provided with the deck. So the deck should be a compressed container which includes both the deck file and every input file necessary to every nettest included in the deck. Otherwise, the analyst would have to send to the tester the input files separately, which is unacceptable. The container proposed is tar+gzip because it's well supported in python. # 5. Example deck The complete.deck provided with each installation of ooni-probe would be: ``` header: name: Complete description: Runs all the existing ooniprobe tests author: 'OONI <ooni-dev(a)lists.torproject.org>' version: 0.1.0 flags: - requires-root - requires-tor body: - nettest: blocking/http_request local_options: input_file: 'httpo://ihiderha53f36lsd.onion/input/37e60e13536f6afe47a830bfb6b371b5cf65da66d7ad65137344679b24fdccd1' - nettest: blocking/dns_consistency local_options: input_file: 'httpo://ihiderha53f36lsd.onion/input/37e60e13536f6afe47a830bfb6b371b5cf65da66d7ad65137344679b24fdccd1' - nettest: manipulation/http_invalid_request_line - nettest: manipulation/http_header_field_manipulation - nettest: manipulation/traceroute - nettest: blocking/http_host local_options: input_file: 'httpo://ihiderha53f36lsd.onion/input/37e60e13536f6afe47a830bfb6b371b5cf65da66d7ad65137344679b24fdccd1' ``` -- END SPEC -- ~ Art. [1] https://trac.torproject.org/projects/tor/ticket/12823 [2] https://gitweb.torproject.org/ooni/spec.git/blob_plain/HEAD:/test-decks/td-…

1 0

Rescheduling of next OONI development meeting
by Arturo Filastò 13 Aug '14

13 Aug '14

The next weekly OONI development meeting is supposed to happen on the 15th of August. Since the this day is holiday in a lot of countries I am going to propose we skip next weeks meeting and move it to the week after: the 22nd of August. The time will be the same 5 PM CET (3 PM UTC). The upcoming weeks the meeting should happen every Friday as usual. Happy holidays, ~ Art.

1 0

Bridge reachability results
by Arturo Filastò 11 Aug '14

11 Aug '14

As promised I published the bridge reachability measurements on the public ooni report hosting. You can find them here: https://ooni.torproject.org/reports/0.1/CN/ https://ooni.torproject.org/reports/0.1/RU/ https://ooni.torproject.org/reports/0.1/US/ Keep in mind that, as I was telling you, during some of the runs there were some issues with the measurements due to incompatibilities of ooniprobe with the old fedora version running on planetlab, so not all the measurements may be 100% accurate. They should still, at least, give you an idea of how the data format looks like and if it contains enough information for doing your parsing work. I would suggest we keep this discussion public and maintain the ooni-dev list in cc. ~ Art.

3 4

Ooni / M-Lab end-to-end integration testing document.
by Nathan Wilcox 07 Aug '14

07 Aug '14

Hi all, We've completed another milestone in our Ooni deployment work and there's now a section in the README.md for end-to-end integration testing (see below). The remaining bits of our agreed upon vision for Ooni deployment on M-Lab is integration with mlab-ns. The current codebase has a fully functional (though not secure!) "simulator" based deployment which serves as a proof of concept and strawman deployment for testing. It looks like Will is pushing through a few key changes to mlab-ns itself, so we're quite close to a fully integrated deployment. Our final milestone will focus on ensuring these different pieces come together, so you'll here from us as soon as that's ready. Regards, Nathan Link / pull request overview The README test section (this is currently only in our fork / pull request): https://github.com/LeastAuthority/ooni-support/blob/combined-leastauthority… The pull request link is here (and combines our mlab-ns-simulator changes from our previous milestone): https://github.com/m-lab-tools/ooni-support/pull/59 The primary ticket for tracking this documentation is now closed: https://github.com/m-lab-tools/ooni-support/issues/60 We also have been doing a bit of testing and prep-work for integration with the mlab-ns changes Will is working on, such as mentioned here: https://github.com/m-lab-tools/ooni-support/issues/56 We ran across some new issues and closed some old issues while working through this procedure: https://github.com/m-lab-tools/ooni-support/issues/62 - This is a minor issue in our simulator start/stop scripts which won't matter once full mlab-ns deployment is in place. These are several closed tickets along the way: https://github.com/m-lab-tools/ooni-support/issues/63 https://github.com/m-lab-tools/ooni-support/issues/64 https://github.com/m-lab-tools/ooni-support/issues/61

1 0

Ooni / M-Lab integration.
by Nathan Wilcox 05 Aug '14

05 Aug '14

Dear OTF, Ooni, and M-Lab, Summary ======= We've hashed out a design to integrate Ooni with mlab-ns on the M-Lab deployment, and we've implemented a fully functional deployment that approximates this by simulating mlab-ns (this is attached). This completes Milestone D of our contract with OTF. Design Goals ============ Our top goals for this integration are: It does not rely on any changes to upstream Ooni. (For example, probes still use a bouncer .onion, and the backend has stock bouncers, collectors, and test helpers running.) It can be disabled easily without redeploying the M-Lab backend. Our branch's ooni-support README.md has instructions to disable the integration, merely by editing a cron job to unset an ENABLED flag. There's no need to redeploy different versions of ooni-support. When enabled, it allows M-Lab operations to monitor collectors and test_helpers status with the same infrastructure as all other M-Lab tools. Future Architectural Changes ---------------------------- In the future, it may be nice to augment ooni / mlab-ns integration. For example, mlab-ns is designed to support different policies which may be useful to tools, such as geo-location of test_helpers. The Simulator ============= This deployment architecture uses a simulator. While it is fully functional and useful for testing it lacks security or robustness, so we want to emphasize *not to deploy this* to non-test environments. Rationale --------- There are three rationales for this approach: First, Least Authority didn't want to push through modifications to mlab-ns without first creating and testing a proof-of-concept. Second, we didn't want to block our effort on M-Lab engineering effort, so this allows a clean division of labor. Third, by creating and testing a working proof of concept we can help define the necessary changes to mlab-ns in a tightly scoped and concrete manner. Security -------- This system is insecure because it does not use the M-Lab nagios system to gather data, and instead lets anyone paste any data they want into the simulator. Nagios integration is future work captured in this ticket: * https://github.com/m-lab-tools/ooni-support/issues/10 Next Steps ========== Our contract with OTF proposes our next two milestones will focus on improving integration testing and unit test coverage. Our focus at that time was on test automation and documentation for diagnosing integration problems. Test automation has already been improved since that time, and we've accomplished most of the work for documentation: https://github.com/m-lab-tools/ooni-support/issues/60 Therefore, we propose to focus on some outstanding issues which will improve mlab-ns integration while continuing not to block on, or interfere with, M-Lab operations as follows: The primary change to mlab-ns will be to allow any tool to include arbitrary data per slivver to be gathered and distributed by mlab-ns. Ooni will use this to distribute data such as collector `.onion` addresses. The need for this change is discussed here: * https://github.com/m-lab-tools/ooni-support/issues/4 This proposed change is documented in this ticket: * https://github.com/m-lab-tools/ooni-support/issues/47 A secondary change is to implement `match=all` described in #47 above. It may not be necessary, so there is further investigation and testing necessary: https://github.com/m-lab-tools/ooni-support/issues/56 Along with these changes to `mlab-ns`, we need trivial updates to our integration scripts to work with mlab-ns rather than the simulator: * https://github.com/m-lab-tools/ooni-support/issues/10 * https://github.com/m-lab-tools/ooni-support/issues/11 Details & Links =============== Attached is a shortish overview of possible approaches to implement this integration. We've implemented a deployment with a mock mlab-ns (called mlab-ns-simulator) and the "arbitrary data" approach from the attached design document. The pull request is here: * https://github.com/m-lab-tools/ooni-support/pull/59 Specific details about this pull request: * A script for gathering necessary information from collectors and testhelpers, then updating the mlab-ns-simulator. * A script for updating a bouncer's state based on the mlan-ns-simulator. * A cron script to update the bouncer on an hourly schedule. * The mlab-ns-simulator itself, which approximates the production mlab-ns. * `.init/` script changes to automatically launch the simulator and bouncer on `mlab1.nuq0t.measurement-lab.org`. * Design documentation for mlab-ns integration (including this stepping stone architecture). * Each instructions to disable mlab-ns integration without any redeployment. We also created a subset pull request that has bug fixes but no mlab-ns integration features: * https://github.com/m-lab-tools/ooni-support/pull/58 Github Milestones ----------------- We split the mlab-ns-simulator deployment tasks out from the larger mlab-ns integration deployment. The mlab-ns-simulator milestone is at: * https://github.com/m-lab-tools/ooni-support/issues?q=is%3Aissue+milestone%3… The full mlab-ns integration milestone: * https://github.com/m-lab-tools/ooni-support/issues?q=is%3Aissue+milestone%3… As always, let us know if you have any feedback! -- Nathan Wilcox Least Authoritarian email: nathan(a)leastauthority.com twitter: @least_nathan PGP: 11169993 / AAAC 5675 E3F7 514C 67ED E9C9 3BFE 5263 1116 9993

3 8

OONI status report July 2014
by Arturo Filastò 05 Aug '14

05 Aug '14

# July This is what the OONI team was up to during the month of July. * Attended Tor dev meeting in Paris where we discussed about the bridge reachability project and the notion of an OONI consortium. * Define roadmap for the bridge reachability measurement project: https://trac.torproject.org/projects/tor/ticket/12544 * Fixed bug when a deck was missing some argument of a test https://gitweb.torproject.org/ooni-probe.git/commit/a9a24464b4b61800fe57874… * Improved error handling of oonibclient https://gitweb.torproject.org/ooni-probe.git/commit/d4cfecb34c146faf07c8726… * Published the bridge reachability results collected from CN, RU and US https://lists.torproject.org/pipermail/ooni-dev/2014-July/000118.html * Discussed with Choke point project some possible areas of collaboration: https://lists.torproject.org/pipermail/ooni-talk/2014-July/000003.html * Agreed to have weekly meetings on IRC #ooni every Friday at 5 PM CET. # August This month we plan to work on the following activities: * Specify the bridge reachability test: https://trac.torproject.org/projects/tor/ticket/12757 * Make ansible deployment script for bridge reachability probes on debian stable: https://trac.torproject.org/projects/tor/ticket/12769 * `daemonize` API breakage when upgrading from Twisted <= 13.1 to >= 13.2: https://trac.torproject.org/projects/tor/ticket/12644 * Probe Report should not leak the full PATH. https://trac.torproject.org/projects/tor/ticket/12706 * Have a list of all the the published reports so far https://trac.torproject.org/projects/tor/ticket/12770 * Add versioning information about the maxmind database being used https://trac.torproject.org/projects/tor/ticket/12772 * Design a test deck for Iran https://trac.torproject.org/projects/tor/ticket/12578 * "Failed to open reporter" might mean "the collector refused my report": https://trac.torproject.org/projects/tor/ticket/11855 ~ Art.

1 0

Detecting which Test Helpers are Running
by Taylor Hornby 02 Aug '14

02 Aug '14

Hello, We are working on scripts to create the Ooni bouncer configuration from multiple instances of Ooni running on M-Lab slices. So far, we have two scripts. One, which collects the portion of the bouncer.yaml out of an Ooni slice, and another script, which combines all of the parts into a single bouncer.yaml: https://github.com/LeastAuthority/ooni-support/blob/mlab-ns-simulator/bounc… https://github.com/LeastAuthority/ooni-support/blob/mlab-ns-simulator/bounc… One thing we would like the "getconfig.py" script to do is to dynamically determine which Test Helpers are enabled on the slice. I'm not sure how to do this, because the oonib.conf always contains entries for all the test helpers. So my question is this: How can we determine which Test Helpers are enabled by examining the oonib.conf or other files? Or, is this question misguided? Are *all* test helpers supposed to be in the bouncer.yaml file, even ones which are not running, and we don't want to have running? (For the first M-Lab deployment, there will be only one Test Helper, the http-return-headers one, so we could hard-code that one into the script, but we would like to make this script keep working when new tests are enabled). Thanks, -- Taylor Hornby

2 1

mlab-ns integration testing and easy-disable-feature.
by Nathan Wilcox 02 Aug '14

02 Aug '14

Dear Oonitarians, We're about to start testing some mlab-ns integration features on the M-Lab test slice. This will be potentially disruptive because we'll be reinitializing the slice, installing new bits and bobs, and we'll be using our own fork of ooni-support [1] rather than upstream ooni-support [2]. Please let us know if we should *not* do this soon. Today I planned to only poke mlab1.nuq0t.measurement-lab.org [3], but eventually we want to test at least two test slivvers. Meanwhile, what happens if we deploy some integration glue and it doesn't work, or has unanticipated problems in the future, or M-Lab policies change, etc? To anticipate those kinds of issues, we want to make sure that it's easy to toggle off the integration stuff. One of our key design goals has been to avoid modifying upstream ooni. We've just realized a new and related design goal: make sure it's easy to turn off the integration glue without requiring slice re-initialization, packages changes, repository changes, etc... So we just added a ticket for that [4]. Note that *while* we are doing this interactive testing, the two slivvers we're using will be pretty unstable, but *after* we reach a stable level of development, we should be able to flip the integration switch to off and the M-Lab test slice should operate as before (except with some extra stuff installed). If you have any concerns / feedback / comments, etc... feel free to email us, or respond to specific tickets. All of the tickets related to the mlab-ns integration in particular should live in the `ooni-support` issue tracker with the `LeastAuthority D` label [5]. Regards, Nathan links: [1] https://github.com/LeastAuthority/ooni-support [2] https://github.com/m-lab-tools/ooni-support [3] https://github.com/m-lab-tools/ooni-support/issues/50#issuecomment-50060748 [4] https://github.com/m-lab-tools/ooni-support/issues/54 [5] https://github.com/m-lab-tools/ooni-support/issues?direction=desc&labels=Le… -- Nathan Wilcox Least Authoritarian email: nathan(a)leastauthority.com twitter: @least_nathan PGP: 11169993 / AAAC 5675 E3F7 514C 67ED E9C9 3BFE 5263 1116 9993

2 1