[tor-bugs] #29761 [Core Tor/Chutney]: Track chutney CI failures, and tweak the allow failures settings

[Tor Bug Tracker & Wiki]

#29761: Track chutney CI failures, and tweak the allow failures settings
------------------------------+--------------------------------
 Reporter:  teor              |          Owner:  teor
     Type:  defect            |         Status:  assigned
 Priority:  Medium            |      Milestone:
Component:  Core Tor/Chutney  |        Version:
 Severity:  Normal            |     Resolution:
 Keywords:  chutney-ci        |  Actual Points:
Parent ID:                    |         Points:  1
 Reviewer:                    |        Sponsor:  Sponsor19-must
------------------------------+--------------------------------

Comment (by teor):

 Same 0.2.9 failures:
 https://travis-ci.org/torproject/chutney/jobs/505567525
 https://travis-ci.org/teor2345/chutney/jobs/505046093

 0.3.4 failure, before test-network-forgiving:
 https://travis-ci.org/teor2345/chutney/jobs/505554142

 Travis network failures:
 https://travis-ci.org/teor2345/chutney/jobs/505554152
 https://travis-ci.org/teor2345/chutney/jobs/505562574

 There have been about 13 builds, times 14 jobs per build.

 0.2.9 failed on 4/13 jobs, failing 4 builds.
 0.3.4 failed on 1/13 jobs, failing 1 build.
 Travis network failed on 2/182 jobs, failing 2 builds.

 Let's get the chutney failure rate below the Travis failure rate?

 Here's what I'm thinking of doing:
 * add a sleep between the allow failure rounds, to hopefully decorrelate
 the 0.2.9 failures
 * continue to monitor the Travis and 0.3.4 failure rate

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29761#comment:5>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online