[tor-bugs] #5232 [BridgeDB]: Import bridges into BridgeDB in a separate thread and database transaction

[Tor Bug Tracker & Wiki]

#5232: Import bridges into BridgeDB in a separate thread and database transaction
----------------------+-----------------------------------------------------
 Reporter:  karsten   |          Owner:  aagbsn
     Type:  defect    |         Status:  new   
 Priority:  major     |      Milestone:        
Component:  BridgeDB  |        Version:        
 Keywords:            |         Parent:        
   Points:            |   Actualpoints:        
----------------------+-----------------------------------------------------

Comment(by sysrqb):

 I thought I was close to having a patch for this, but it's slightly more
 complicated than I anticipated.

 for example:
 {{{
 ProgrammingError: SQLite objects created in a thread can only be used in
 that same thread.The object was created in thread id 3929742227200 and
 this is thread id 3929574946560
 }}}
 It appears that the majority of the time is spent during sqlite calls
 within the bridge weighting procedure. Attached will be a profiling of
 reload() during startup followed by a profiling after sending SIGHUP. I
 used a set of 1000 randomly generated descriptors and I updated the
 publication time of 187 descriptors prior to sending the SIGHUP (just so
 the stability code actually updated some bridge stats).

 I'm also attaching a profile of reload() with the call to the Stability
 code commented out. On my computer this reduces startup time from 124.377
 seconds to 4.889 seconds. I'm curious if we should comment out the call to
 addOrUpdateBridgeHistory() in load() for the time being until we are able
 to move the processing to a background thread and we have a mechanism for
 reachability testing implemented.

 We're currently not using the BridgeHistory information, but it has
 occurred to me that long-term bridge history will be useful. I'm unsure
 which is more important.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/5232#comment:5>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online