[tor-bugs] #16577 [Tor Browser]: Verify that Tor Browser patch reverts cause test failures
Tor Bug Tracker & Wiki
blackhole at torproject.org
Mon Jul 20 14:32:58 UTC 2015
#16577: Verify that Tor Browser patch reverts cause test failures
---------------------------------+----------------------
Reporter: arthuredelstein | Owner: tbb-team
Type: task | Status: new
Priority: normal | Milestone:
Component: Tor Browser | Version:
Resolution: | Keywords: ff38-esr
Actual Points: | Parent ID:
Points: |
---------------------------------+----------------------
Comment (by boklm):
In order to check if our regression tests are failing without their
corresponding patches, I used my repo with split branches and split them
more to put the tests in a separate branch, to be able to push them
separately to Mozilla Try.
I did that first using the version of the patches based on 38.0.1esr and
here are the results:
{{{
38.0.1esr/2874-Block-Components.interfaces-from-content:
* 75e3f67964df Regression tests for #2874: Block Components.interfaces
from content
* 6d4165108321 Bug #2874: Block Components.interfaces from content
* d919ec4dedbe Enable tbb-tests/mochitest.ini
* 52459e0c7dd3 Bug #12620: TorBrowser regression tests folder
https://treeherder.mozilla.org/#/jobs?repo=try&revision=18cfee0eae19
38.0.1esr/2874-Block-Components.interfaces-from-content.test:
* 0801b13aea17 Regression tests for #2874: Block Components.interfaces
from content
* d919ec4dedbe Enable tbb-tests/mochitest.ini
* 52459e0c7dd3 Bug #12620: TorBrowser regression tests folder
https://treeherder.mozilla.org/#/jobs?repo=try&revision=3b8e8a64ccdd
busted: X
testfailed: M(5), M(bc1)
38.0.1esr/2875-Limit-device-and-system-specific-CSS-Media-Queries:
* 0721c0ce16d6 Regression tests for #2875: Limit device and system
specific CSS Media Queries.
* 09149b1417db Bug #2875: Limit device and system specific CSS Media
Queries.
* d919ec4dedbe Enable tbb-tests/mochitest.ini
* 52459e0c7dd3 Bug #12620: TorBrowser regression tests folder
https://treeherder.mozilla.org/#/jobs?repo=try&revision=fac4b042d797
testfailed: M(4), M(5), M(bc2)
38.0.1esr/2875-Limit-device-and-system-specific-CSS-Media-Queries.test:
* e7df7918a148 Regression tests for #2875: Limit device and system
specific CSS Media Queries.
* d919ec4dedbe Enable tbb-tests/mochitest.ini
* 52459e0c7dd3 Bug #12620: TorBrowser regression tests folder
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ec37fdbad233
busted: X
testfailed: M(5), M(bc1)
38.0.1esr/2950-Make-Permissions-Manager-memory-only:
* fd15a9ebb804 Regression tests for Bug #2950: Make Permissions Manager
memory-only
* 79ef2a1f2bde Bug #2950: Make Permissions Manager memory-only
* 6d16271a374e Enable tbb-tests/browser.ini
* 52459e0c7dd3 Bug #12620: TorBrowser regression tests folder
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0fcd667c0570
38.0.1esr/4755-Return-client-window-coordinates-for-mouse-event-screenX-Y:
* bebd5ea65b12 Regression tests for #4755: Return client window
coordinates for mouse event screenX/Y (for dragend, 0,0 is returned).
* 931ba1ed3956 Bug #4755: Return client window coordinates for mouse
event screenX/Y (for dragend, 0,0 is returned).
* d919ec4dedbe Enable tbb-tests/mochitest.ini
* 52459e0c7dd3 Bug #12620: TorBrowser regression tests folder
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b6663db2aa8e
testfailed: M(2), M(5)
38.0.1esr/4755-Return-client-window-coordinates-for-mouse-event-
screenX-Y.test:
* 8b70cce28921 Regression tests for #4755: Return client window
coordinates for mouse event screenX/Y (for dragend, 0,0 is returned).
* d919ec4dedbe Enable tbb-tests/mochitest.ini
* 52459e0c7dd3 Bug #12620: TorBrowser regression tests folder
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c97b19954a9b
busted: X
testfailed: M(5), M(bc1)
38.0.1esr/5856-Do-not-expose-physical-screen-info-via-window-
window.screen:
* a555a43c9424 Bug 13025: Lie about screen orientation.
* 3048f766632b Regression tests for #5856: Do not expose physical
screen info via window & window.screen.
* 5fbc32940b32 fixup! Bug #5856: Do not expose physical screen info via
window & window.screen.
* 292b0cd6d6e5 Bug #5856: Do not expose physical screen info via window
& window.screen.
* d919ec4dedbe Enable tbb-tests/mochitest.ini
* 52459e0c7dd3 Bug #12620: TorBrowser regression tests folder
https://treeherder.mozilla.org/#/jobs?repo=try&revision=61782741cc52
busted: M(3), M(bc1)
testfailed: M(1), M(2), M(4), M(5), Mn
38.0.1esr/5856-Do-not-expose-physical-screen-info-via-window-
window.screen.test:
* 71ecb463b046 Regression tests for #5856: Do not expose physical
screen info via window & window.screen.
* d919ec4dedbe Enable tbb-tests/mochitest.ini
* 52459e0c7dd3 Bug #12620: TorBrowser regression tests folder
https://treeherder.mozilla.org/#/jobs?repo=try&revision=91ea5927a49b
busted: X
testfailed: M(5), M(bc1)
38.0.1esr/isolation:
* c1e2dfc12244 fixup! Bug 13670.2: Isolate OCSP requests by first party
domain
* 3ce0db6951ca Bug 16300: Isolate Broadcast Channels to first party.
* c054d08392e0 fixup! Bug #13749.2: Regression tests for first-party
isolation of cache
* afa4c0776abe Bug 13439: No canvas prompt for content-callers.
* 935edc4208d6 Bug 13021: Prompt before allowing Canvas isPointIn*()
calls.
* 5c38cf9e2658 Bug #6253: Add canvas image extraction prompt.
* e424bc280f64 Bug 13900: Remove 3rd party HTTP auth tokens.
* 4f1f4b8aed26 Bug #15502, Part 2: Regression tests for blob URL
isolation
* d383dc1d54f1 Bug #15502. Isolate blob URLs to first party; no
blobURLs in Web Workers
* 25ea17272c4a fixup! Bug 13670.2: Isolate OCSP requests by first party
domain
* 1f36ee3d68ce Bug 13670.2: Isolate OCSP requests by first party domain
* 935c0e104682 Bug #13749.1: regression tests for first party isolation
of localStorage
* f4b3b48f1901 Bug #13749.2: Regression tests for first-party isolation
of cache
* 6a99e486c6ac Bug #6564: Isolate DOM storage to first party URI.
* fed97227fac5 Bug #6539: Isolate the Image Cache per url bar domain.
* f730e6fb0dbf Bug 13742: Isolate cache to URL bar domain.
* 553369720c84 Bug #10819: Add a pref, "privacy.thirdparty.isolate", to
allow the activation or deactivation of isolating DOM storage and image
caching by first party URI.
* ff065c6cb9f6 Bug #5742: API allows you to get the url bar URI for a
channel or nsIDocument.
* 5745c7c53605 Bug #3455.2. Allow RFC1929 authentication
(username/password) to SOCKS servers.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f04617e1e11b
busted: M(1), M(3), M(4), M(5), X
testfailed: M(2), M(bc1), M(bc2), M(bc3), Mn, R(C), R(R1), R(R2), R(R3),
R(R4)
38.0.1esr/isolation.test1:
* 73e313df2b5e Bug #13749.2: Regression tests for first-party isolation
of cache
https://treeherder.mozilla.org/#/jobs?repo=try&revision=515168b9ab50
busted: X
testfailed: M(bc1), M(bc2)
38.0.1esr/isolation.test2:
* 1bd90b7aad33 Bug #13749.1: regression tests for first party isolation
of localStorage
https://treeherder.mozilla.org/#/jobs?repo=try&revision=19d8c28c177c
busted: X
testfailed: M(4), M(bc1)
38.0.1esr/isolation.test3:
* 8fa4c6df059f Bug #15502, Part 2: Regression tests for blob URL
isolation
https://treeherder.mozilla.org/#/jobs?repo=try&revision=9b0d05c0bda9
busted: S, X
testfailed: M(1), M(bc1)
}}}
== OK: Bug #5856: Do not expose physical screen info via window &
window.screen. ==
As expected, tbb-tests/test_tor_bug5856.html is failing without the
corresponding patches. With the patches applied, we have other failures
but not tbb-tests/test_tor_bug5856.html.
== OK: Bug #2875: Limit device and system specific CSS Media Queries. ==
As expected, tbb-tests/test_tor_bug2875.html is failing without the
patches. With the patches applied, we have other failures but not tbb-
tests/test_tor_bug2875.html.
== Unknown: Bug #2950: Make Permissions Manager memory-only ==
With the patches applied, no failure. I forgot this one and did not try
yet the test alone.
== OK: Bug #4755: Return client window coordinates for mouse event
screenX/Y (for dragend, 0,0 is returned). ==
As expected, tbb-tests/test_tor_bug4755.html is failing without the
patches. With the patches applied, we have other failures, but not tbb-
tests/test_tor_bug4755.html.
== OK: Bug #2874: Block Components.interfaces from content ==
As expected, tbb-tests/test_tor_bug2874.html is failing without the
patches. With the patches applied, nothing failed.
== OK: Bug #13749.1: regression tests for first party isolation of
localStorage ==
As expected,
dom/tests/mochitest/localstorage/test_localStorageByFirstParty.html is
failing without the patches. With the patches, we have many other tests
failing, but not
dom/tests/mochitest/localstorage/test_localStorageByFirstParty.html.
== NOT OK: Bug #13749.2: Regression tests for first-party isolation of
cache ==
There seems to be something wrong with this one. As expected,
netwerk/test/browser/browser_cacheFirstParty.js is failing without the
patches. With the patches, we have many other tests failing, but
netwerk/test/browser/browser_cacheFirstParty.js is still failing too,
although the error is different in both cases (in bc2):
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f04617e1e11b
{{{
400 INFO TEST-UNEXPECTED-FAIL |
netwerk/test/browser/browser_cacheFirstParty.js | Cache entries expected
for iframe.html: 2 -
407 INFO TEST-UNEXPECTED-FAIL |
netwerk/test/browser/browser_cacheFirstParty.js | Cache entries expected
for audio.ogg: 2 -
408 INFO TEST-UNEXPECTED-FAIL |
netwerk/test/browser/browser_cacheFirstParty.js | Cache entries expected
for video.ogv: 2 -
431 INFO TEST-UNEXPECTED-FAIL |
netwerk/test/browser/browser_cacheFirstParty.js | Cache entries expected
for iframe.html: 1 -
}}}
== NOT OK: Bug #15502, Part 2: Regression tests for blob URL isolation ==
There seems to be something wrong to with this one too. As expected,
dom/base/test/test_tor_bug15502.html is failing without the patches. But
it is also failing with the patches applied, with the same error in both
cases:
{{{
2816 INFO TEST-UNEXPECTED-FAIL | dom/base/test/test_tor_bug15502.html |
Test timed out. - expected PASS
}}}
Or maybe I forgot a patch in this branch (38.0.1esr/isolation) to make
this test pass ?
I also updated my repo to use the latest version of the patches based on
38.1.0esr and push them to Mozilla Try. However I now have a strange error
with anything that I push to Try, even with the 38.1.0esr commit without
any of our patches:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=648171333847
I will need to push the new patches once this problem with Try is fixed.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/16577#comment:5>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
More information about the tor-bugs
mailing list