[tor-bugs] #11630 [Tor bundles/installation]: Creating HTTPS-Everywhere's rules.sqlite is non-deterministic

Tor Bug Tracker & Wiki blackhole at torproject.org
Wed Sep 3 21:08:37 UTC 2014


#11630: Creating HTTPS-Everywhere's rules.sqlite is non-deterministic
------------------------------------------+----------------------
     Reporter:  gk                        |      Owner:  erinn
         Type:  defect                    |     Status:  reopened
     Priority:  normal                    |  Milestone:
    Component:  Tor bundles/installation  |    Version:
   Resolution:                            |   Keywords:
Actual Points:                            |  Parent ID:
       Points:                            |
------------------------------------------+----------------------

Comment (by jsha):

 Sorry it took me so long to look at this. The first diff that jumps out at
 me is offset 28-31 decimal (0x16-0x19), whose value is 1424 in the second
 file and 1428 (dec) in the first file. Per the docs at
 https://www.sqlite.org/fileformat.html, that's the in-header database size
 in pages. You probably don't still have the two file around, but if you
 do: were they different sizes?

 The other thing I noticed in this diff is that there is a large chunk of
 the rulesets that show up as a diff, where they are in a different order.
 Looking at the code, that makes sense. At the time of filing this bug, we
 used os.walk, which does not have an order guarantee
 (http://stackoverflow.com/a/5667552). Now we use glob.glob, which also
 does not have an order guarantee (http://stackoverflow.com/a/6774404). I
 will write a change to sort the files before loading them, which should
 hopefully make at least that part deterministic. I suspect it may have
 mysterious add-on effects that could fix the other diffs. Let's try it and
 see.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/11630#comment:20>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online


More information about the tor-bugs mailing list