[tor-bugs] #6414 [Ooni]: Automating Bridge Reachability Testing

[Tor Bug Tracker & Wiki]

#6414: Automating Bridge Reachability Testing
------------------------------------------------------------------------+---
 Reporter:  isis                                                        |          Owner:  isis          
     Type:  project                                                     |         Status:  needs_revision
 Priority:  normal                                                      |      Milestone:                
Component:  Ooni                                                        |        Version:                
 Keywords:  bridge-reachability metrics-db automation testing SponsorZ  |         Parent:                
   Points:                                                              |   Actualpoints:                
------------------------------------------------------------------------+---
Changes (by isis):

  * status:  accepted => needs_revision


Comment:

 At the end of September, it was decided that OONI's framework, which is
 what the bridge tests were based on, should be changed. The changed was
 supposed to be backwards compatible, though it was not. At first I thought
 that I should just write the tests using whatever I felt like using, and
 have them be their own separate project, but I really wanted them to be
 part of OONI.

 I had already been fighting and hacking around OONI's earlier framework
 some fair amount to get the tests to work, and I figured it shouldn't be
 difficult to get them to run with the new framework. I
 [https://gitweb.torproject.org/ooni-
 probe.git/commitdiff/270db771f360111efd46329796ec4567fedae0f8 completely
 rewrote the class converter] which was supposed to provide backwards
 compatibility. Then I rewrote more things. Then I fixed more things. I
 spent the entire month of October trying to get everything to work, and it
 was incredibly frustrating, to say the least. I really dislike failing,
 and even more admitting when I have (which is why it's so difficult for me
 to update these tickets), but that's bullshit and I need to swallow my
 pride.

 The biggest problem I am facing is twisted's reactor. In the old
 framework, tests would get called by a parent/wrapper script which
 controlled starting and stopping the reactor. I need to have a persistent
 Tor process connected to the reactor, which means fighting the wrapper. In
 the new framework, twisted trial (twisted's unittesting framework/module)
 is used as the basis of the design for OONI's framework. Trial also
 controls the reactor, and in a somewhat more severe manner. Per test (i.e.
 per single unittest method), the reactor is started, the class is set up,
 initialized, the parameters to the test method are set up in the
 instantiation, and then passed to the test method, and then the reactor is
 cleared and stopped. This is crucial to trial, because events left sitting
 on the reactor can fire the next time it is started, but not necessarily.
 If the reactor isn't cleared, it leads to very strange behaviour that is
 difficult to debug, so it makes sense for trial to do such a thing. I am
 still not entirely sure that the bridge tests will work with the new OONI
 framework, given that I already had an unexpected amount of difficultly
 with the old.

 Also, I am sorry that sometimes I fail at communicating.

 So, tl;dr: OONI's alpha release got pushed back by one month. The bridge
 tests are part of OONI, and should be done at that time.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/6414#comment:15>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online