[tor-bugs] #21312 [Obfuscation/Snowflake]: snowflake-client is pegged at 100% cpu

[Tor Bug Tracker & Wiki]

#21312: snowflake-client is pegged at 100% cpu
-----------------------------------+------------------------------
 Reporter:  arlolra                |          Owner:  arlolra
     Type:  defect                 |         Status:  needs_review
 Priority:  High                   |      Milestone:
Component:  Obfuscation/Snowflake  |        Version:
 Severity:  Major                  |     Resolution:
 Keywords:                         |  Actual Points:
Parent ID:                         |         Points:
 Reviewer:                         |        Sponsor:
-----------------------------------+------------------------------

Comment (by dcf):

 Replying to [comment:34 dcf]:
 > It does appear to be the case that proxy-go is crashing and being
 restarted. One of the instances was automatically restarted 6 times since
 I started it manually 30 minutes ago.
 > I'll let it run overnight, and tomorrow I plan to try an analogous patch
 to the one in comment:32. I've also re-started the memory and file
 descriptor monitors, in case the memory leak was only being masked by
 frequent restarts.

 Well, I tried the `nil` check patch for proxy-go, and it seems that it is
 still restarting frequently :( So either there's a crash that remains
 unfixed even after the `nil` check, or there's something else going on
 that we don't understand.

  * The first blue line is [https://gitweb.torproject.org/pluggable-
 transports/snowflake.git/commit/?id=44ab82bc612dac3f9605afd44f057ef199b2cc12
 44ab82bc61], which just adds a "starting" marker to the log.
  * The second blue line is [https://gitweb.torproject.org/pluggable-
 transports/snowflake.git/commit/?id=f2abf5b60c289103d8661b8e54b3d3aee43a4aaf
 f2abf5b60c], which tries to protect against the crash with a `nil` check.

  * The "appengine" section represents the proxies connected to
 snowflake.appspot.com broker; i.e. the one that users are actually using.
  * The "standalone" section represents the proxies connected to the
 [ticket:22874 standalone broker] snowflake-broker.bamsoftware.com; i.e.
 the one nobody is using unless they went out of their way to configure it.
 You'll notice that these ones are mostly being restarted externally every
 1h, 2h, 10h, which is how I configured them and is expected. I don't know
 what accounts for the restarts between about 2018-03-14 07:30:00 and
 2018-03-14 10:00:00.

 [[Image(proxy-go-starting.20180314.png)]]

 I'll try connecting a debugger to one of the frequently restarting
 instances and see if there's anything obvious.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21312#comment:39>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online