Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively? Sorry if I already missed the discussion about this. https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0... Thanks!
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
On 9 Mar 2017, at 12:07, David Fifield <david@bamsoftware.com> wrote:
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
We are trying to work out if it's a side effect of how we count connections, and blocking from the UAE that terminates connections part way through. https://trac.torproject.org/projects/tor/ticket/21345 T -- Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------
On Thu, Mar 09, 2017 at 12:09:37PM +1100, teor wrote:
On 9 Mar 2017, at 12:07, David Fifield <david@bamsoftware.com> wrote:
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
We are trying to work out if it's a side effect of how we count connections, and blocking from the UAE that terminates connections part way through.
Excellent, thanks!
On Wed, Mar 08, 2017 at 05:07:36PM -0800, David Fifield wrote:
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
Ah, very neat, thanks for the link!
On 08 Mar (17:07:36), David Fifield wrote:
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
Seems obfs4 is now what they are "testing".... http://rougmnvswfsmd4dq.onion/userstats-bridge-transport.html?start=2016-12-... David
_______________________________________________ tor-project mailing list tor-project@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project
-- s+c4q/k6vRy8Ybfo7A+/tCkGtyqp87N/Ky1JGpS/QZA=
On Fri, Mar 10, 2017 at 03:01:56PM -0500, David Goulet wrote:
On 08 Mar (17:07:36), David Fifield wrote:
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
Seems obfs4 is now what they are "testing"....
http://rougmnvswfsmd4dq.onion/userstats-bridge-transport.html?start=2016-12-...
The UAE graph doesn't show an increase in obfs4: https://metrics.torproject.org/userstats-bridge-combined.html?start=2016-12-... The spike at the end of the overall obfs4 graph might not be a real sustained change, because in clients.csv it goes back to normal the next day. (The second-to-last column is the one to look at.) date,node,country,transport,version,lower,upper,clients,frac 2017-03-01,bridge,,obfs4,,,,34392,65 2017-03-02,bridge,,obfs4,,,,33200,66 2017-03-03,bridge,,obfs4,,,,33568,65 2017-03-04,bridge,,obfs4,,,,31734,64 2017-03-05,bridge,,obfs4,,,,31621,63 2017-03-06,bridge,,obfs4,,,,33240,65 2017-03-07,bridge,,obfs4,,,,34563,65 2017-03-08,bridge,,obfs4,,,,63618,34 2017-03-09,bridge,,obfs4,,,,35922,50 2017-03-10,bridge,,obfs4,,,,2045,25
On Fri, Mar 10, 2017 at 03:01:27PM -0800, David Fifield wrote:
On Fri, Mar 10, 2017 at 03:01:56PM -0500, David Goulet wrote:
On 08 Mar (17:07:36), David Fifield wrote:
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
Seems obfs4 is now what they are "testing"....
http://rougmnvswfsmd4dq.onion/userstats-bridge-transport.html?start=2016-12-...
The UAE graph doesn't show an increase in obfs4: https://metrics.torproject.org/userstats-bridge-combined.html?start=2016-12-...
The spike at the end of the overall obfs4 graph might not be a real sustained change, because in clients.csv it goes back to normal the next day. (The second-to-last column is the one to look at.)
date,node,country,transport,version,lower,upper,clients,frac 2017-03-01,bridge,,obfs4,,,,34392,65 2017-03-02,bridge,,obfs4,,,,33200,66 2017-03-03,bridge,,obfs4,,,,33568,65 2017-03-04,bridge,,obfs4,,,,31734,64 2017-03-05,bridge,,obfs4,,,,31621,63 2017-03-06,bridge,,obfs4,,,,33240,65 2017-03-07,bridge,,obfs4,,,,34563,65 2017-03-08,bridge,,obfs4,,,,63618,34 2017-03-09,bridge,,obfs4,,,,35922,50 2017-03-10,bridge,,obfs4,,,,2045,25
Yeah, it is interesting that obfs3/obfs4 possibly crossed: http://rougmnvswfsmd4dq.onion/userstats-bridge-transport.html?start=2017-03-... But I spoke with someone at IFF from the region last week and their current thought is that this is caused by some group running a bot (of some kind) and inflating our metrics for the country. They weren't sure about the goal of this, so our guess is probably as good as their's. Overall the usage pattern doesn't look extraordinarily artifical, except the jump of +200k relay users within a week. The rapid decay beginning on 03 Feb seems plausible. It's interesting, looking at the raw data, it seems this began on 12 or 13 Jan: date,node,country,transport,version,lower,upper,clients,frac 2017-01-06,relay,ae,,,5738,8495,7195,81 2017-01-07,relay,ae,,,5850,8968,7268,81 2017-01-08,relay,ae,,,6023,9340,7316,82 2017-01-09,relay,ae,,,5800,9458,7293,82 2017-01-10,relay,ae,,,5985,8905,7251,82 2017-01-11,relay,ae,,,5909,8751,7351,81 2017-01-12,relay,ae,,,5854,8869,7854,81 2017-01-13,relay,ae,,,5595,8914,9145,82 2017-01-14,relay,ae,,,5971,8564,10570,81 2017-01-15,relay,ae,,,6240,8442,11499,82 2017-01-16,relay,ae,,,6079,8711,30377,82 2017-01-17,relay,ae,,,6159,8552,119908,82 2017-01-18,relay,ae,,,6082,8886,208090,81 2017-01-19,relay,ae,,,6459,9547,258835,81 2017-01-20,relay,ae,,,7623,11028,317643,82 2017-01-21,relay,ae,,,8652,12783,318948,82 There is a jump of ~500 users on 12 Jan, but that's semi-plausible. The jump of ~1300 users on the 13th seems less likely. Between the 12th and 18th, there were (approx.) deltas of: 06 to 07: +50 07 to 08: +50 08 to 09: -20 09 to 10: -40 10 to 11: -0 11 to 12: +500 12 to 13: +1300 13 to 14: +1400 14 to 15: +900 15 to 16: +19000 16 to 17: +80000 17 to 18: +90000 18 to 19: +50000 19 to 20: +60000 20 to 21: +1000 And for bridges: date,node,country,transport,version,lower,upper,clients,frac 2017-01-25,bridge,ae,,,,,377,66 2017-01-26,bridge,ae,,,,,366,66 2017-01-27,bridge,ae,,,,,367,66 2017-01-28,bridge,ae,,,,,363,66 2017-01-29,bridge,ae,,,,,387,67 2017-02-01,bridge,ae,,,,,423,67 2017-02-02,bridge,ae,,,,,411,68 2017-02-03,bridge,ae,,,,,363,66 2017-02-04,bridge,ae,,,,,413,64 2017-02-05,bridge,ae,,,,,796,58 2017-02-06,bridge,ae,,,,,5961,65 2017-02-07,bridge,ae,,,,,8762,55 2017-02-08,bridge,ae,,,,,8057,51 2017-02-09,bridge,ae,,,,,27016,63 2017-02-10,bridge,ae,,,,,66323,65 2017-02-11,bridge,ae,,,,,82979,65 2017-02-12,bridge,ae,,,,,64968,64 2017-02-13,bridge,ae,,,,,77667,62 2017-02-14,bridge,ae,,,,,87850,53 2017-02-15,bridge,ae,,,,,47517,58 2017-02-16,bridge,ae,,,,,45346,54 2017-02-17,bridge,ae,,,,,82640,60 2017-02-18,bridge,ae,,,,,107386,60 2017-02-19,bridge,ae,,,,,105322,62 It seems, on average, there were ~380 bridge users throughout 2016 and 2017 until 2017-02-05. For consistency, the approximate deltas between 01 Feb and 19 Feb: 02 to 03: -50 03 to 04: +50 04 to 05: +370 05 to 06: +5200 06 to 07: +2800 07 to 08: -700 08 to 09: +19000 09 to 10: +39000 10 to 11: +16600 11 to 12: -18000 12 to 13: +13000 13 to 14: +10000 14 to 15: -40000 15 to 16: -2200 16 to 17: +37000 17 to 18: +25000 18 to 19: -2000 It's interesting that the bridge users count began increasing a few days after relay users began decreasing. Actually, I found which bridge is supporting these new users. I confirmed it isn't one of the default bridges. {"version":"4.0", "relays_published":"2017-03-13 22:00:00", "relays":[ ], "bridges_published":"2017-03-13 20:57:29", "bridges":[ {"nickname":"Unnamed","hashed_fingerprint":"220B66EBF7625B31D3313491C0B888E488F2E66B","or_addresses":["10.64.118.173:56651"],"last_seen":"2017-03-13 20:57:29","first_seen":"2016-01-18 11:55:20","running":true,"flags":["Fast","HSDir","Running","Stable","V2Dir","Valid"],"last_restarted":"2017-03-09 06:48:03","advertised_bandwidth":2503701,"platform":"Tor 0.2.9.5-alpha on Linux","transports":["scramblesuit","obfs3","obfs4"]} ]} https://onionoo.torproject.org/details?fingerprint=220B66EBF7625B31D3313491C... https://atlas.torproject.org/#details/3E0908F131AC417C48DDD835D78FB6887F4CD1... I'll follow up with additional analysis tomorrow, but here's the data from 2017-03-12 00:09:00 amnesia@amnesia:~$ grep -A 23 220B66EBF7625B31D3313491C0B888E488F2E66B 2017-03-12-00-09-00-extra-infos | grep -e "^extra-info" -e history -e dirreq-v3-reqs -e bridge-ips -e "ae=" extra-info Unnamed 220B66EBF7625B31D3313491C0B888E488F2E66B write-history 2017-03-11 19:14:11 (14400 s) 40817088512,48679548928,39163826176,34126496768,60959848448,85227308032 read-history 2017-03-11 19:14:11 (14400 s) 3655943168,4583458816,5928579072,6270611456,7911438336,10202891264 dirreq-write-history 2017-03-11 18:33:19 (14400 s) 56407040000,32424969216,44282493952,30598066176,49384162304,72785624064 dirreq-read-history 2017-03-11 18:33:19 (14400 s) 684358656,690675712,1961063424,1814886400,1891488768,2772764672 dirreq-v3-ips ae=115824,in=2504,nl=1256,us=888,jo=728,gb=720,de=496,sa=280,fr=240,om=200,ca=96,jp=80,bh=72,??=64,be=64,kw=56,qa=48,sg=32,it=24,pk=24,iq=16,ir=16,at=8,au=8,bd=8,bg=8,bn=8,br=8,by=8,ch=8,cl=8,cn=8,dj=8,dz=8,eg=8,hk=8,ie=8,il=8,kr=8,lb=8,lv=8,ly=8,md=8,mu=8,mx=8,ng=8,no=8,pr=8,ro=8,ru=8,sc=8,sd=8,se=8,si=8,so=8,tm=8,tn=8,tr=8,ua=8,uz=8,za=8 dirreq-v3-reqs ae=495328,nl=14928,us=7696,in=5136,gb=4168,fr=4128,de=3344,be=2984,jo=2240,it=928,sa=784,ca=544,om=440,qa=208,bh=184,ie=184,kw=176,jp=136,??=112,ch=104,sg=88,iq=56,at=48,bg=48,pk=48,ru=48,hk=32,ir=32,tr=32,bn=24,dz=16,il=16,lb=16,pr=16,se=16,so=16,au=8,bd=8,br=8,by=8,cl=8,cn=8,dj=8,eg=8,kr=8,lv=8,ly=8,md=8,mu=8,mx=8,ng=8,no=8,ro=8,sc=8,sd=8,si=8,tm=8,tn=8,ua=8,uz=8,za=8 bridge-ips ae=144992,in=4248,nl=1344,us=1104,jo=952,gb=800,de=560,sa=360,fr=304,om=280,ca=112,bh=104,jp=104,??=96,kw=80,be=64,qa=64,pk=32,sg=32,iq=24,it=24,so=24,bn=16,hk=16,ir=16,pr=16,ru=16,se=16,at=8,au=8,bd=8,bg=8,br=8,by=8,ch=8,cl=8,cn=8,dj=8,dz=8,eg=8,ie=8,il=8,is=8,kr=8,kz=8,lb=8,lv=8,ly=8,md=8,mu=8,mx=8,ng=8,no=8,ro=8,sc=8,sd=8,si=8,sk=8,tm=8,tn=8,tr=8,ua=8,uz=8,vn=8,ye=8,za=8
On Tue, Mar 14, 2017 at 12:02:46AM +0000, Matthew Finkel wrote:
It's interesting that the bridge users count began increasing a few days after relay users began decreasing. Actually, I found which bridge is supporting these new users. I confirmed it isn't one of the default bridges.
{"version":"4.0", "relays_published":"2017-03-13 22:00:00", "relays":[ ], "bridges_published":"2017-03-13 20:57:29", "bridges":[ {"nickname":"Unnamed","hashed_fingerprint":"220B66EBF7625B31D3313491C0B888E488F2E66B","or_addresses":["10.64.118.173:56651"],"last_seen":"2017-03-13 20:57:29","first_seen":"2016-01-18 11:55:20","running":true,"flags":["Fast","HSDir","Running","Stable","V2Dir","Valid"],"last_restarted":"2017-03-09 06:48:03","advertised_bandwidth":2503701,"platform":"Tor 0.2.9.5-alpha on Linux","transports":["scramblesuit","obfs3","obfs4"]} ]}
https://onionoo.torproject.org/details?fingerprint=220B66EBF7625B31D3313491C... https://atlas.torproject.org/#details/3E0908F131AC417C48DDD835D78FB6887F4CD1...
Nice find. I think your second link (to atlas.torproject.org) is wrong, because it's pointing to LeifEricson, which is one of the default bridges. The right link should be: https://atlas.torproject.org/#details/220B66EBF7625B31D3313491C0B888E488F2E6... The disparity in bytes read/written is interesting. The bridge has about 6× more written bytes than read bytes. That could lend support to the idea that the inflated statistics are caused by connections that are prematurely terminated.
On 03/13/2017 06:02 PM, Matthew Finkel wrote:
I'll follow up with additional analysis tomorrow, but here's the data from 2017-03-12 00:09:00
amnesia@amnesia:~$ grep -A 23 220B66EBF7625B31D3313491C0B888E488F2E66B 2017-03-12-00-09-00-extra-infos | grep -e "^extra-info" -e history -e dirreq-v3-reqs -e bridge-ips -e "ae="
Thank you for helping to do this analysis! I have been really curious what is going on but I hit a bit of a block on exactly what sort of query I should be running on information I've pulled down from collector. This grep seems perfect. -- Joshua Gay Communications Director Tor Project https://torproject.org GPG: 59F4 F183 7CC2 7193 3850 21A9 5211 5F6F E922 09E1
On Tue, Mar 14, 2017 at 12:02:46AM +0000, Matthew Finkel wrote:
But I spoke with someone at IFF from the region last week and their current thought is that this is caused by some group running a bot (of some kind)
Typically botnets have victims in many countries, though, right? How did they manage to contain their bots to just UAE hosts? (Geofenced malicious ads? A vulnerability in an app that only UAE people install? Malware on a government website that many people need to visit? Or maybe the bots are more widespread, but for some reason the bot operator chose to only transition the UAE hosts to using Tor?)
dirreq-v3-ips ae=115824,in=2504,nl=1256,us=888,jo=728,[...] dirreq-v3-reqs ae=495328,nl=14928,us=7696,in=5136,gb=4168,[...] bridge-ips ae=144992,in=4248,nl=1344,us=1104,jo=952,[...]
Those are huge numbers, and they convince me that the phenomenon is real -- there really are many many Tor clients connecting from many many different IP addresses. That said, when they shifted from vanilla Tor connections to bridge connections... they all shifted to one bridge? That lends a lot of credibility to the "a bunch of Tor clients, all using the same configuration, so it's all really coordinated" point. --Roger
On Tue, Mar 14, 2017 at 11:40:15AM -0400, Roger Dingledine wrote:
On Tue, Mar 14, 2017 at 12:02:46AM +0000, Matthew Finkel wrote:
But I spoke with someone at IFF from the region last week and their current thought is that this is caused by some group running a bot (of some kind)
Typically botnets have victims in many countries, though, right? How did they manage to contain their bots to just UAE hosts?
Back in 2008, a variant of the Conficker worm wouldn't infect Ukrainian hosts. It used to look at the victim's IP address and keyboard layout to figure out where you are from. I suppose you can do the reverse to target only UAE users, despite some false positives and negatives.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Dear team,
Typically botnets have victims in many countries, though, right? How did they manage to contain their bots to just UAE hosts?
Back in 2008, a variant of the Conficker worm wouldn't infect Ukrainian hosts. It used to look at the victim's IP address and keyboard layout to figure out where you are from. I suppose you can do the reverse to target only UAE users, despite some false positives and negatives.
Such behavior is not uncommon. The botherders will look for user agent strings, language packs, IP-to-CC (country code) mappings, and the like. Some of their customers are discriminating, e.g. "I only want bots in South Korea." The reasons range from spam sourcing to DDoS to gaming mayhem. There are multiple bots that report their CC as part of their nickname. Be well, Rabbi Rob. - -- Rabbi Rob Thomas Team Cymru "It is easy to believe in freedom of speech for those with whom we agree." - Leo McKern -----BEGIN PGP SIGNATURE----- iQIcBAEBCAAGBQJYyVvtAAoJEEPoYWL6hfKNYR8P/0Z53X3BS05ygXb2Ff3SbAxa kNXL0nVGZ9bA3Fdk2evGNhLkXT0fUuEvuFreiVmglCWbIE/3LMVsLAj3EC+qNMIY UfghUG0vyGQAATkzzkvHuC2NEVv0OcmzIYLbCr+rbpCKhYfvQ54OvpX+K4mW4X1a yrVPTzXXjoeR94wXGVbn6GmMXbXhrwy5jVtegzHKhYNHN9eTOJCMFpmdsPgYASVl OuFIjBpqNLG8sjo5T8znRTwn1uy4uKGyaLVsRhyCyXRK3fXPnNgrDBl2u5qijY0t MBL2A0SYRFEuwaiNGMAqnOiAJNQIe8u02xaN3QgAHIRaRsBH+dUMY8WQC9FOvEb5 t9dExm8KNfDgi2PlwcFqSJxdOyt2T264PNs/yAHIBBNvkL3/sq8FB18aIFBab3iI wY2sve9nOPXl4noKM1KszHHZ3zl3njYxhfoBUMgX9JW8U/Juedqn1X3ddwQcJGN7 nvTz4GwmcydoMe+g5XSYc+4GK7Cwzog16fFypnrjfC8nwY6OtCVP2Kj9fLb8zfyK QavF3MM0cojzjPJvmxQB5vEwpZCPUHYDuCSflhUKDh+p3LvXTq0vFJaNtI0uyy/Z ZTiQ04YXMrX5Jbnxc2KxzIwvtX8dgo3aDtRg6Z8IEkDTm7FpiPEQA8V6yM0nU4rs RCoWBpkaZnJn4eAei1ZL =C3Ys -----END PGP SIGNATURE-----
participants (8)
-
David Fifield -
David Goulet -
Joshua Gay -
Matthew Finkel -
Philipp Winter -
Rabbi Rob Thomas -
Roger Dingledine -
teor