[tor-bugs] #28061 [Core Tor/sbws]: Check prioritisation, it should make 2 measurements that are 24 appart

[Tor Bug Tracker & Wiki]

#28061: Check prioritisation, it should make 2 measurements that are 24 appart
---------------------------+-------------------------------------
 Reporter:  juga           |          Owner:  (none)
     Type:  defect         |         Status:  new
 Priority:  Medium         |      Milestone:  sbws 1.0 (MVP nice)
Component:  Core Tor/sbws  |        Version:
 Severity:  Normal         |     Resolution:
 Keywords:                 |  Actual Points:
Parent ID:  #28042         |         Points:
 Reviewer:                 |        Sponsor:
---------------------------+-------------------------------------

Comment (by juga):

 Replying to [comment:14 pastly]:
 > Replying to [comment:13 juga]:
 > > pastly, i suspect that the reason why the same relay gets measured
 twice within minutes, is because the threads are not sharing the same
 vision on the `best_priority` iterator. Would you mind to have a look to
 it?
 >
 > I don't think the measurement threads have different copies or ideas
 about the RelayPrioritizer because it is created in the main thread and
 never given to worker threads.
 [https://github.com/torproject/sbws/blob/ed1d61c98044e9ed081a7462d94ae31630b75705/sbws/core/scanner.py#L338
 See it defined here and never given to the workers].

 I agree, i saw that, with "vision" i didn't mean copy

 > My untested theory is:
 >
 > A thread will grab one of the last relays in the RelayPrioritizer. While
 it is measuring that relay, the RelayPrioritizer is recalculating
 priority, finishes, and gives the same relay to another thread to also
 measure.

 by "vision" i was thinking something like that.

 >
 > This might be a good time to switch to using
 [https://docs.python.org/3.5/library/queue.html#queue.PriorityQueue
 Python's PriorityQueue] while figuring out the problem. I'll give it some
 thought this evening.

 multiprocessing also have a
 https://docs.python.org/2/library/multiprocessing.html#multiprocessing.Queue
 class. However i thought that if threads have to wait to 1 measurement to
 finish, then will slow down the process.
 What about just getting a list of relays to measure (instead of an
 iterator), and do not call `best_priority` again until the list is almost
 finished?.
 If you're a busy, i can try that.
 Thanks for looking at it.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/28061#comment:15>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online