[tor-bugs] #12859 [Stem]: Improve parsing speed with profile-directed code tuning

[Tor Bug Tracker & Wiki]

#12859: Improve parsing speed with profile-directed code tuning
-----------------------------+--------------------
     Reporter:  nickm        |      Owner:  atagar
         Type:  enhancement  |     Status:  closed
     Priority:  normal       |  Milestone:
    Component:  Stem         |    Version:
   Resolution:  fixed        |   Keywords:
Actual Points:               |  Parent ID:
       Points:               |
-----------------------------+--------------------
Changes (by atagar):

 * status:  needs_revision => closed
 * resolution:   => fixed


Old description:

> Hi!  I took a quick look at how Stem parses descriptors, and tried
> running cProfile over a networkstatus parse.  I don't think I have all of
> the issues solved, but I managed to get a few of the hot spots calmed
> down for a speed reduction of about 30%.
>
> I don't promise that these are all correct, though I am pretty sure that
> the first one is a big win that you should take.  It accounts for most of
> the speedup that I'm seeing.

New description:

 Hi!  I took a quick look at how Stem parses descriptors, and tried running
 cProfile over a networkstatus parse.  I don't think I have all of the
 issues solved, but I managed to get a few of the hot spots calmed down for
 a speed reduction of about 30%.

 I don't promise that these are all correct, though I am pretty sure that
 the first one is a big win that you should take.  It accounts for most of
 the speedup that I'm seeing.

 Thanks again for the patches!

--

Comment:

 Hi Nick, the testing failures were due to the third patch. I can certainly
 see your reason for it - when I saw list had a pop first thought was
 constant time for both ends but on reflection that would only work if
 there was an offset counter for the array head (which on reflection was
 foolish to expect, and [https://wiki.python.org/moin/TimeComplexity looks
 to not be the case]).

 I wasn't clear from the description how much it bought us in practice so I
 went ahead and pushed the other two. Patches welcome for correcting the
 pop() if it passes the tests.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/12859#comment:4>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online