[tor-bugs] #25331 [Applications/Tor Browser]: Test from #18912 failing

[Tor Bug Tracker & Wiki]

#25331: Test from #18912 failing
--------------------------------------+-----------------------------------
 Reporter:  arthuredelstein           |          Owner:  tbb-team
     Type:  defect                    |         Status:  needs_information
 Priority:  Medium                    |      Milestone:
Component:  Applications/Tor Browser  |        Version:
 Severity:  Normal                    |     Resolution:
 Keywords:                            |  Actual Points:
Parent ID:                            |         Points:
 Reviewer:                            |        Sponsor:
--------------------------------------+-----------------------------------

Comment (by arthuredelstein):

 Thanks for looking into this!

 I ran some tests on top of your patch rebased on top of mozilla-central
 (https://github.com/arthuredelstein/tor-browser/1436226+2). First I ran
 `./mach run` and in the browser console I entered
 `Services.prefs.prefHasUserValue("security.nocertdb")` and got `false`.

 Then I ran the test as before:
 {{{
 ./mach mochitest
 toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul
 }}}
 and got
 {{{
   TEST-UNEXPECTED-FAIL |
 toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul
 | Checking update.errorCode == MOZILLA_PKIX_ERROR_KEY_PINNING_FAILURE. If
 removing or changing this test, please notify tbb-dev at lists.torproject.org
 - got 2153394164, expected 2153398272
 }}}

 Next, I tried running the test with the --setpref flag as you suggested:
 {{{
 ./mach mochitest --setpref=security.nocertdb=false
 toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul
 }}}
 and the result I got was:
 {{{
   TEST-UNEXPECTED-FAIL |
 toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul
 | Checking update.errorCode == MOZILLA_PKIX_ERROR_KEY_PINNING_FAILURE. If
 removing or changing this test, please notify tbb-dev at lists.torproject.org
 - got 2153394164, expected 2153398272
 SimpleTest.is at chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:312:5
 checkErrorCode at chrome://mochitests/content/chrome/toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul:61:3
 }}}

 I then tried setting the flag to true:

 {{{
 ./mach mochitest --setpref=security.nocertdb=true
 toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul
 }}}

 and I got:

 {{{
   TEST-UNEXPECTED-FAIL |
 toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul
 | Checking update.errorCode == MOZILLA_PKIX_ERROR_KEY_PINNING_FAILURE. If
 removing or changing this test, please notify tbb-dev at lists.torproject.org
 - got 2153390067, expected 2153398272
 SimpleTest.is at chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:312:5
 checkErrorCode at chrome://mochitests/content/chrome/toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul:61:3
 }}}

 In sum, I think my build does not have Tor Browser pref settings, by
 default it should be operating equivalent to the pref being set to false.
 So when security.nocertdb is default or false, I get the unexpected error
 code of `2153394164` and when security.nocertdb is set to true, I get the
 unexpected error code of `2153390067`. Both error values are different
 from the expected error code `2153398272`.

 Next I'm going to try these same set of experiments on tor-browser.git.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25331#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online