[tor-bugs] #9874 [BridgeDB]: Research/design a way to automate testing of BridgeDB's HTTPS and email distributors

[Tor Bug Tracker & Wiki]

#9874: Research/design a way to automate testing of BridgeDB's HTTPS and email
distributors
-------------------------+-------------------------------------------------
     Reporter:  isis     |      Owner:  isis
         Type:           |     Status:  needs_revision
  enhancement            |  Milestone:
     Priority:  normal   |    Version:
    Component:           |   Keywords:  bridgedb-unittests, automation, ci,
  BridgeDB               |  bridgedb-gsoc-application
   Resolution:           |  Parent ID:
Actual Points:           |
       Points:           |
-------------------------+-------------------------------------------------
Changes (by isis):

 * cc: sysrqb, trygve (added)
 * status:  needs_review => needs_revision


Comment:

 Replying to [comment:12 trygve]:
 > Attached 'test_https.py'. This is a test suite used to test the BridgeDB
 HTTPS web page. All tests are written to use twisted.trial. The suite
 currently has three tests:
 >
 >     '''test_get_obfs2_ipv4''': uses mechanize and Beautiful soup to
 connect to the BridgeDB web page, follows some links, chooses 'obfs2' on
 the options page, enters the captcha and checks that at least one bridge
 line is returned of type 'obfs2'
 >     '''test_get_obfs3_ipv4''': as above, except gets 'obfs3' bridges
 >     '''test_get_obfs3_ipv6''': as above, except gets 'obfs3' ipv6
 bridges. Note: currently fails on my system due to lack of ipv6 bridge
 descriptors available.
 >
 > To run the tests:
 >
 >     generate some bridge descriptors (should contain at least one obfs2
 and obfs3 bridges)
 >     run bridgedb
 >     run 'trial test_https.py' from within the bridgedb virtualenv

 I had to do a bit of refactoring (not of your code, but of our CI scripts)
 to get this to work. All the work is in my `fix/9874-https`
 [https://gitweb.torproject.org/user/isis/bridgedb.git/shortlog/refs/heads/fix/9874-https
 branch], including
 [https://gitweb.torproject.org/user/isis/bridgedb.git/commitdiff/d5556300a9f53c80760e12b13c94a93d041ca9ff
 your script].

 Previously, we had tests in `lib/bridgedb/test/test_bridgedb.py` which
 started a BridgeDB instance (including copying a bunch of configs and
 other files around). Now
 [https://gitweb.torproject.org/user/isis/bridgedb.git/commitdiff/d4d753f29cdda65861a60a15285c13186f6ac1c6
 the BridgeDB instance is started before the CI infrastructure runs the
 tests] and
 [https://gitweb.torproject.org/user/isis/bridgedb.git/commitdiff/9873fad412081d35e508de5c8dd095a17fd921cb
 the old test_bridgedb.py tests were refactored] to use the already running
 instance.

 [https://travis-ci.org/isislovecruft/bridgedb/jobs/31886557#L1450 Your
 tests pass], all [https://travis-
 ci.org/isislovecruft/bridgedb/jobs/31886557#L1745 except the one]
 `test_get_obfs3_ipv6` test. This is because BridgeDB learns about a
 bridge's pluggable transports from the `transport` lines in
 [https://para.noid.cat/bridgedb/descriptors.html#a-bridge-extra-info-
 descriptor-with-pluggable-transports @type bridge-extrainfo descriptors],
 and `obfs3` (as with most PTs) [https://lists.torproject.org/pipermail
 /tor-talk/2014-June/033296.html cannot currently listen on an IPv6
 address] (#7961, #11211, #12138).

 You could change the `test_get_obfs3_ipv6` test to check for
 `scramblesuit` bridges on IPv4 if you like, but you'd need to patch
 [https://gitweb.torproject.org/user/isis/leekspin.git leekspin]
 [https://gitweb.torproject.org/user/isis/leekspin.git/blob/HEAD:/leekspin/extrainfo.py#l50
 to add "transport scramblesuit" lines]. Or simply remove the failing test,
 I'll accept either.

 Thanks, trygve, for all this work!

 --------

 By the way, I have attributed you in git as `trygve <tor-
 dev at lists.torproject.org>`, please let me know if I should change the
 attribution.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9874#comment:14>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online