[tor-bugs] #21312 [Obfuscation/Snowflake]: snowflake-client is pegged at 100% cpu
Tor Bug Tracker & Wiki
blackhole at torproject.org
Thu Mar 15 04:07:36 UTC 2018
#21312: snowflake-client is pegged at 100% cpu
-----------------------------------+------------------------------
Reporter: arlolra | Owner: arlolra
Type: defect | Status: needs_review
Priority: High | Milestone:
Component: Obfuscation/Snowflake | Version:
Severity: Major | Resolution:
Keywords: | Actual Points:
Parent ID: | Points:
Reviewer: | Sponsor:
-----------------------------------+------------------------------
Comment (by dcf):
Replying to [comment:41 arlolra]:
> > Perhaps "Pure virtual method called"? I'm not sure.
>
> Well, `CGO_DestroyPeer` doesn't appear to be thread-safe; so let's fix
that. The client is currently defaulting to 1 concurrent peer, which
would explain why it doesn't hit this issue, if that's the cause. The
proxies, which are handling multiple peers, each of which are voluntarily
exiting every 30s of inactivity, seem like they could be hitting some
contention there, but possibly not at the rate things are failing?
I just reproduced the crash on my desktop. I ran `gdb --args ./proxy-go
-log snowflake.log`, then `r`, and just waited for someone to use it (it
took 20 minutes). It is indeed "pure virtual method called". It looks to
me like there was only one client connected at the time; so it seems that
contention among multiple clients isn't necessary.
The full log is in attachment:proxy-go-abort.20180314.log​. It has not
only gdb output, but also stdout/stderr and the contents of the log file,
interleaved. Potentially of note: the abort happens immediately after the
first `OnClose channel`. The client had been connected for 78 seconds.
Here is the last part of attachment:proxy-go-abort.20180314.log:
{{{
2018/03/15 03:53:11 Write 3113 bytes --> WebRTC
2018/03/15 03:53:11 Write 1571 bytes --> WebRTC
2018/03/15 03:53:11 Write 3627 bytes --> WebRTC
2018/03/15 03:53:11 OnMessage <--- 543 bytes
2018/03/15 03:53:14 broker returns: 504
2018/03/15 03:53:24 broker returns: 504
2018/03/15 03:53:39 broker returns: 504
2018/03/15 03:53:42 OnClose channel
INFO: peerconnection.go:503: fired OnIceConnectionStateChange: 1
INFO: peerconnection.go:435: fired OnSignalingStateChange: 1 Closed
[Thread 0x7fffd77fe700 (LWP 10782) exited]
[Thread 0x7fffd7fff700 (LWP 10781) exited]
[Thread 0x7ffff4587700 (LWP 10777) exited]
pure virtual method called
terminate called without an active exception
[Thread 0x7ffff4688700 (LWP 10776) exited]
[Thread 0x7fffedffb700 (LWP 10774) exited]
[Thread 0x7fffed7fa700 (LWP 10775) exited]
Thread 6 "proxy-go" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffef7fe700 (LWP 10491)]
__GI_raise (sig=sig at entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=sig at entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff686842a in __GI_abort () at abort.c:89
#2 0x00007ffff717f0ad in __gnu_cxx::__verbose_terminate_handler() () from
/usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00007ffff717d066 in ?? () from /usr/lib/x86_64-linux-
gnu/libstdc++.so.6
#4 0x00007ffff717d0b1 in std::terminate() () from /usr/lib/x86_64-linux-
gnu/libstdc++.so.6
#5 0x00007ffff717db8f in __cxa_pure_virtual () from /usr/lib/x86_64
-linux-gnu/libstdc++.so.6
#6 0x00005555558dd957 in rtc::scoped_refptr<Peer>::scoped_refptr (r=...,
this=<optimized out>)
at ./include/webrtc/base/scoped_ref_ptr.h:82
#7
__gnu_cxx::__ops::_Iter_pred<CGO_DestroyPeer(CGO_Peer)::<lambda(rtc::scoped_refptr<Peer>)>
>::operator()<__gnu_cxx::__normal_iterator<rtc::scoped_refptr<Peer>*,
std::vector<rtc::scoped_refptr<Peer> > > > (__it=...,
this=<synthetic pointer>) at
/usr/include/c++/6/bits/predefined_ops.h:241
#8 std::__find_if<__gnu_cxx::__normal_iterator<rtc::scoped_refptr<Peer>*,
std::vector<rtc::scoped_refptr<Peer> > >,
__gnu_cxx::__ops::_Iter_pred<CGO_DestroyPeer(CGO_Peer)::<lambda(rtc::scoped_refptr<Peer>)>
> > (__pred=..., __last=
{ptr_ = 0x7fffe0000298}, __first={ptr_ = 0x7fffe0000a20}) at
/usr/include/c++/6/bits/stl_algo.h:148
#9 std::__find_if<__gnu_cxx::__normal_iterator<rtc::scoped_refptr<Peer>*,
std::vector<rtc::scoped_refptr<Peer> > >,
__gnu_cxx::__ops::_Iter_pred<CGO_DestroyPeer(CGO_Peer)::<lambda(rtc::scoped_refptr<Peer>)>
> > (__pred=..., __last=...,
__first=...) at /usr/include/c++/6/bits/stl_algo.h:162
#10
std::__remove_if<__gnu_cxx::__normal_iterator<rtc::scoped_refptr<Peer>*,
std::vector<rtc::scoped_refptr<Peer> > >,
__gnu_cxx::__ops::_Iter_pred<CGO_DestroyPeer(CGO_Peer)::<lambda(rtc::scoped_refptr<Peer>)>
> > (__pred=..., __last=
{ptr_ = 0x7fffe0000298}, __first=...) at
/usr/include/c++/6/bits/stl_algo.h:860
#11 std::remove_if<__gnu_cxx::__normal_iterator<rtc::scoped_refptr<Peer>*,
std::vector<rtc::scoped_refptr<Peer> > >,
CGO_DestroyPeer(CGO_Peer)::<lambda(rtc::scoped_refptr<Peer>)> >
(__pred=..., __last=..., __first=...)
at /usr/include/c++/6/bits/stl_algo.h:937
#12 CGO_DestroyPeer (cgoPeer=0x7fffe0000a20)
at /home/david/go/src/github.com/keroserene/go-
webrtc/peerconnection.cc:249
#13 0x0000555555726350 in runtime.asmcgocall () at
/usr/lib/go-1.7/src/runtime/asm_amd64.s:590
#14 0x000000c420035de0 in ?? ()
#15 0x00005555556d0dcd in runtime.cgocall (fn=0x0, arg=0xc42006c1a0,
~r2=-276832584)
at /usr/lib/go-1.7/src/runtime/cgocall.go:115
#16 0x000000c42001c000 in ?? ()
#17 0x0000000000000000 in ?? ()
}}}
P.S. I think I'll be able to look at attachment:0001-Add-synchronization-
around-destroying-dataChannels.patch tomorrow.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21312#comment:43>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
More information about the tor-bugs
mailing list