[tbb-dev] reconsider shipping uBO or enabling ETP?
gunes.acar at esat.kuleuven.be
Fri Jul 17 16:32:45 UTC 2020
On 7/15/20 8:31 PM, Dennis Jackson wrote:
> Hi Gunes, Matt,
> I did some measurements along these lines late last year using a tool I
> developed on an internship at Mozilla. In short it uses a fork of
> WebPageTest and some orchestration with the Google Compute Engine to
> spin up Tor Browser instances in Google data centers all over the world
> and measure how fast they load pages. In particular, the page load time
> is measured by recording a video of the page loading and establishing
> when elements have stopped popping in - i.e. when it is complete from
> the user's perspective. It also supports tweaking various preferences
> and loading extensions and so on.
> Unfortunately, although I inteded to spend several months of time
> working on it earlier this year, COVID caused a big shift in my plans
> and progress has been nowhere near as fast as I wanted. Its still
> prototype quality code and the documentation is pretty much non-existent
> at the moment.
> I've attached three graphs. The first two [1,2] show how Tor Browser
> performance varies in different geographic locations in its default
> settings and with UBlock enabled. The results show a similar improvement
> to yours Gunes and in general enabling Ublock really improves things for
> users located outside of Europe. The third graph  shows measurements from
> the USA and compares the default setting against Tracking Protection
> Enabled and UBlock modes. The results are surprisingly close, although
> UBlock does win out. I'll post a slightly more detailed write up on the
> Gitlab issue shortly.
> I'd be more than happy to collaborate if it is of interest. I think
> WebPageTest offers a great harness in terms of UX metrics, but as it
> officially targets Firefox and Chrome, I have to add any Tor-specific
> features that are needed, which is quite a lot of work. Conversely,
> tor-browser-selenium has much better interactions with Tor, but I would
> imagine it would be time consuming to replicate the visual performance
> testing hooks from WPT, so both tools would have a lot to offer.
> I think the performance argument for enabling Ublock Origin or EPT is
> pretty clear, so I imagine the decision should now hinge on the security
> consequences and the probability of an increase in website blocking, but
> happy to help out if there are further performance questions.
> [1,2,3] -
> On 15/07/2020 15:37, gunes acar wrote:
>> On 7/9/20 4:01 PM, Matthew Finkel wrote:
>>> On Wed, Jul 01, 2020 at 06:16:19PM +0200, gunes acar wrote:
>>>> Hi all,
>>> Hi Gunes,
>>>> I was wondering if Tor Browser team is open to reconsider shipping an
>>>> adblocker or turning on Firefox's Enhanced Tracking Protection (ETP)
>>>> mode, and whether you need some research input that may inform that
>>>> I am aware of issue #17569 and I agree with most of the arguments laid
>>>> out in the "No filters" part of the design doc. Especially that the
>>>> list-based approaches would not improve TB's existing privacy
>>>> protections with respect to curtailing online tracking.
>>>> However, tracking protection/adlocking also have significant performance
>>>> benefits [1, 2], which is the main reason why I'm bringing this up.
>>> Yes, indeed. We began investigating this last year, as well .
>>>> Just to find out whether there could be potential performance benefits
>>>> of shipping uBO with TB, I did a quick crawl of sites from the Trexa
>>>> list  using TB with and without uBlock Origin (uBO) installed. Having
>>>> uBO *reduced the median page load time by 27.6%* (from 7.6s to 5.5s) on
>>>> top 1K sites. You can imagine there would be a similar reduction in the
>>>> network footprint due to blocked requests.
>>>> Another reason why it may be timely to reconsider this issue is that
>>>> today all major browsers except Chrome ship a built-in adblocker or
>>>> tracking protection mode (Safari, Firefox, Edge, Opera, Brave...). So,
>>>> Tor Project may not really stand out for "damag[ing] the acceptance of
>>>> Tor users by sites" by blocking ads.
>>> While the acceptance of ad blockers is significantly higher now than it
>>> was a decade ago, a request from a Tor Browser instance (or from the Tor
>>> network, in general) is often still perceived as "potentially abusive"
>>> due to the existing hacker/darkweb association. Tor Browser remains at a
>>> disadvantage compared to the other browsers for this reason, and we are
>>> already in a difficult position with sites denying requests coming from
>>> the Tor network. Adding an ad blocker could harm Tor Browser's usability
>>> even more.
>>>> I and some colleagues could be interested in dedicating some research
>>>> time into studying the potential privacy and performance impact of
>>>> adding adblockers or enabling ETP mode -- esp. if the JIT issue (#23719)
>>>> makes uBO and other addons infeasible.
>>>> Can you let us know if there's willingness to reconsider shipping an
>>>> adblocker/ETP, and if so what research may help you with your decision?
>>> We are very much interested in this research area. In particular, we
>>> need an evaluation of which option (uBO, tracking protection, etc.)
>>> provides performance benefits and safety (what are the tradeoffs in
>>> their settings and what should we consider?). Tracking Protection has
>>> the advantage of being integrated into Tor Browser, however we have
>>> concerns about the way the deny list is updated/retrieved from Mozilla's
>>> servers and whether we should host our own list (and the cost/overhead
>>> of that). Similarly, uBO provides extensive filter lists but there is
>>> minimal oversight, so ensuring Tor Browser users continue receiving
>>> sufficient privacy and security protection is a critical part of this.
>>> For research topics, I would suggest investigating the performance
>>> benefits of additional configurations in uBO and Tracking Protection, as
>>> well as the privacy and security implications of the filter lists and
>>> how they are updated. When we have answers for more of these questions,
>>> then we can begin assessing what should be deployed in Tor Browser.
>>> Web sites blocking Tor is another important area, and if you are
>>> interested in exploring that, then we can discuss that separately, as
>>>>  https://dl.acm.org/doi/pdf/10.1145/3366423.3380292
>>>>  https://github.com/mozilla/trexa
>>>>  https://2019.www.torproject.org/projects/torbrowser/design/#philosophy
>>>  https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/30939
>>> tbb-dev mailing list
>>> tbb-dev at lists.torproject.org
>> Thanks, Matt,
>> That's very useful.
>> I apparently missed #30939. Glad to hear that you've already run some
>> preliminary measurements with Firefox's ETP (#33046) and results seemed
>> I'll keep working on the performance measurements and privacy/security
>> related questions that you mention, and report my findings on the Gitlab
>> issue (#30939).
>> Web sites blocking Tor is definitely an interesting subject. If you have
>> particular questions that prior research have not addressed (esp.
>> Khattak et al. (NDSS'16) and Singh et al. (USENIX'17)), please let me know.
>> PS: I was also happy to see that both you and Georg used my
>> tor-browser-selenium library  for some measurements. Please feel free
>> to file an issue if you have suggestions for improvement, or need any
>> modifications for your future needs.
>> tbb-dev mailing list
>> tbb-dev at lists.torproject.org
> tbb-dev mailing list
> tbb-dev at lists.torproject.org
That's pretty cool. I really like the effect of location in your plot,
and how users with (probably) slower network access would benefit
significantly more from this potential change.
I agree that using WPT is a good idea since its metrics are more user
oriented. I see that WPT uses FirefoxWebDriver under the hood, which
tells me it should be possible to swap in TorBrowserDriver (subclass of
FirefoxWebDriver) to make it work with Tor Browser.
Happy to discuss a possible collaboration. I'll reach out over personal
More information about the tbb-dev