Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
Thanks!
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
On 9 Mar 2017, at 12:07, David Fifield david@bamsoftware.com wrote:
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
We are trying to work out if it's a side effect of how we count connections, and blocking from the UAE that terminates connections part way through.
https://trac.torproject.org/projects/tor/ticket/21345
T
-- Tim Wilson-Brown (teor)
teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------
On Thu, Mar 09, 2017 at 12:09:37PM +1100, teor wrote:
On 9 Mar 2017, at 12:07, David Fifield david@bamsoftware.com wrote:
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
We are trying to work out if it's a side effect of how we count connections, and blocking from the UAE that terminates connections part way through.
Excellent, thanks!
On Wed, Mar 08, 2017 at 05:07:36PM -0800, David Fifield wrote:
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
Ah, very neat, thanks for the link!
On 08 Mar (17:07:36), David Fifield wrote:
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
Seems obfs4 is now what they are "testing"....
http://rougmnvswfsmd4dq.onion/userstats-bridge-transport.html?start=2016-12-...
David
tor-project mailing list tor-project@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project
On Fri, Mar 10, 2017 at 03:01:56PM -0500, David Goulet wrote:
On 08 Mar (17:07:36), David Fifield wrote:
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
Seems obfs4 is now what they are "testing"....
http://rougmnvswfsmd4dq.onion/userstats-bridge-transport.html?start=2016-12-...
The UAE graph doesn't show an increase in obfs4: https://metrics.torproject.org/userstats-bridge-combined.html?start=2016-12-...
The spike at the end of the overall obfs4 graph might not be a real sustained change, because in clients.csv it goes back to normal the next day. (The second-to-last column is the one to look at.)
date,node,country,transport,version,lower,upper,clients,frac 2017-03-01,bridge,,obfs4,,,,34392,65 2017-03-02,bridge,,obfs4,,,,33200,66 2017-03-03,bridge,,obfs4,,,,33568,65 2017-03-04,bridge,,obfs4,,,,31734,64 2017-03-05,bridge,,obfs4,,,,31621,63 2017-03-06,bridge,,obfs4,,,,33240,65 2017-03-07,bridge,,obfs4,,,,34563,65 2017-03-08,bridge,,obfs4,,,,63618,34 2017-03-09,bridge,,obfs4,,,,35922,50 2017-03-10,bridge,,obfs4,,,,2045,25
On Fri, Mar 10, 2017 at 03:01:27PM -0800, David Fifield wrote:
On Fri, Mar 10, 2017 at 03:01:56PM -0500, David Goulet wrote:
On 08 Mar (17:07:36), David Fifield wrote:
On Thu, Mar 09, 2017 at 01:05:15AM +0000, Matthew Finkel wrote:
Anyone know what caused the remarkable jump in direct and obfs3 users from the UAE that began on 16 Jan and 06 Feb, respectively?
Sorry if I already missed the discussion about this.
https://metrics.torproject.org/userstats-relay-country.html?start=2017-01-14... https://metrics.torproject.org/userstats-bridge-country.html?start=2017-02-0...
We don't know the cause AFAIK, but we have some entries for it with links, near the bottom of https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline#Unknown
Seems obfs4 is now what they are "testing"....
http://rougmnvswfsmd4dq.onion/userstats-bridge-transport.html?start=2016-12-...
The UAE graph doesn't show an increase in obfs4: https://metrics.torproject.org/userstats-bridge-combined.html?start=2016-12-...
The spike at the end of the overall obfs4 graph might not be a real sustained change, because in clients.csv it goes back to normal the next day. (The second-to-last column is the one to look at.)
date,node,country,transport,version,lower,upper,clients,frac 2017-03-01,bridge,,obfs4,,,,34392,65 2017-03-02,bridge,,obfs4,,,,33200,66 2017-03-03,bridge,,obfs4,,,,33568,65 2017-03-04,bridge,,obfs4,,,,31734,64 2017-03-05,bridge,,obfs4,,,,31621,63 2017-03-06,bridge,,obfs4,,,,33240,65 2017-03-07,bridge,,obfs4,,,,34563,65 2017-03-08,bridge,,obfs4,,,,63618,34 2017-03-09,bridge,,obfs4,,,,35922,50 2017-03-10,bridge,,obfs4,,,,2045,25
Yeah, it is interesting that obfs3/obfs4 possibly crossed:
http://rougmnvswfsmd4dq.onion/userstats-bridge-transport.html?start=2017-03-...
But I spoke with someone at IFF from the region last week and their current thought is that this is caused by some group running a bot (of some kind) and inflating our metrics for the country. They weren't sure about the goal of this, so our guess is probably as good as their's. Overall the usage pattern doesn't look extraordinarily artifical, except the jump of +200k relay users within a week. The rapid decay beginning on 03 Feb seems plausible.
It's interesting, looking at the raw data, it seems this began on 12 or 13 Jan:
date,node,country,transport,version,lower,upper,clients,frac 2017-01-06,relay,ae,,,5738,8495,7195,81 2017-01-07,relay,ae,,,5850,8968,7268,81 2017-01-08,relay,ae,,,6023,9340,7316,82 2017-01-09,relay,ae,,,5800,9458,7293,82 2017-01-10,relay,ae,,,5985,8905,7251,82 2017-01-11,relay,ae,,,5909,8751,7351,81 2017-01-12,relay,ae,,,5854,8869,7854,81 2017-01-13,relay,ae,,,5595,8914,9145,82 2017-01-14,relay,ae,,,5971,8564,10570,81 2017-01-15,relay,ae,,,6240,8442,11499,82 2017-01-16,relay,ae,,,6079,8711,30377,82 2017-01-17,relay,ae,,,6159,8552,119908,82 2017-01-18,relay,ae,,,6082,8886,208090,81 2017-01-19,relay,ae,,,6459,9547,258835,81 2017-01-20,relay,ae,,,7623,11028,317643,82 2017-01-21,relay,ae,,,8652,12783,318948,82
There is a jump of ~500 users on 12 Jan, but that's semi-plausible. The jump of ~1300 users on the 13th seems less likely. Between the 12th and 18th, there were (approx.) deltas of:
06 to 07: +50 07 to 08: +50 08 to 09: -20 09 to 10: -40 10 to 11: -0 11 to 12: +500 12 to 13: +1300 13 to 14: +1400 14 to 15: +900 15 to 16: +19000 16 to 17: +80000 17 to 18: +90000 18 to 19: +50000 19 to 20: +60000 20 to 21: +1000
And for bridges:
date,node,country,transport,version,lower,upper,clients,frac 2017-01-25,bridge,ae,,,,,377,66 2017-01-26,bridge,ae,,,,,366,66 2017-01-27,bridge,ae,,,,,367,66 2017-01-28,bridge,ae,,,,,363,66 2017-01-29,bridge,ae,,,,,387,67 2017-02-01,bridge,ae,,,,,423,67 2017-02-02,bridge,ae,,,,,411,68 2017-02-03,bridge,ae,,,,,363,66 2017-02-04,bridge,ae,,,,,413,64 2017-02-05,bridge,ae,,,,,796,58 2017-02-06,bridge,ae,,,,,5961,65 2017-02-07,bridge,ae,,,,,8762,55 2017-02-08,bridge,ae,,,,,8057,51 2017-02-09,bridge,ae,,,,,27016,63 2017-02-10,bridge,ae,,,,,66323,65 2017-02-11,bridge,ae,,,,,82979,65 2017-02-12,bridge,ae,,,,,64968,64 2017-02-13,bridge,ae,,,,,77667,62 2017-02-14,bridge,ae,,,,,87850,53 2017-02-15,bridge,ae,,,,,47517,58 2017-02-16,bridge,ae,,,,,45346,54 2017-02-17,bridge,ae,,,,,82640,60 2017-02-18,bridge,ae,,,,,107386,60 2017-02-19,bridge,ae,,,,,105322,62
It seems, on average, there were ~380 bridge users throughout 2016 and 2017 until 2017-02-05. For consistency, the approximate deltas between 01 Feb and 19 Feb:
02 to 03: -50 03 to 04: +50 04 to 05: +370 05 to 06: +5200 06 to 07: +2800 07 to 08: -700 08 to 09: +19000 09 to 10: +39000 10 to 11: +16600 11 to 12: -18000 12 to 13: +13000 13 to 14: +10000 14 to 15: -40000 15 to 16: -2200 16 to 17: +37000 17 to 18: +25000 18 to 19: -2000
It's interesting that the bridge users count began increasing a few days after relay users began decreasing. Actually, I found which bridge is supporting these new users. I confirmed it isn't one of the default bridges.
{"version":"4.0", "relays_published":"2017-03-13 22:00:00", "relays":[ ], "bridges_published":"2017-03-13 20:57:29", "bridges":[ {"nickname":"Unnamed","hashed_fingerprint":"220B66EBF7625B31D3313491C0B888E488F2E66B","or_addresses":["10.64.118.173:56651"],"last_seen":"2017-03-13 20:57:29","first_seen":"2016-01-18 11:55:20","running":true,"flags":["Fast","HSDir","Running","Stable","V2Dir","Valid"],"last_restarted":"2017-03-09 06:48:03","advertised_bandwidth":2503701,"platform":"Tor 0.2.9.5-alpha on Linux","transports":["scramblesuit","obfs3","obfs4"]} ]}
https://onionoo.torproject.org/details?fingerprint=220B66EBF7625B31D3313491C... https://atlas.torproject.org/#details/3E0908F131AC417C48DDD835D78FB6887F4CD1...
I'll follow up with additional analysis tomorrow, but here's the data from 2017-03-12 00:09:00
amnesia@amnesia:~$ grep -A 23 220B66EBF7625B31D3313491C0B888E488F2E66B 2017-03-12-00-09-00-extra-infos | grep -e "^extra-info" -e history -e dirreq-v3-reqs -e bridge-ips -e "ae=" extra-info Unnamed 220B66EBF7625B31D3313491C0B888E488F2E66B write-history 2017-03-11 19:14:11 (14400 s) 40817088512,48679548928,39163826176,34126496768,60959848448,85227308032 read-history 2017-03-11 19:14:11 (14400 s) 3655943168,4583458816,5928579072,6270611456,7911438336,10202891264 dirreq-write-history 2017-03-11 18:33:19 (14400 s) 56407040000,32424969216,44282493952,30598066176,49384162304,72785624064 dirreq-read-history 2017-03-11 18:33:19 (14400 s) 684358656,690675712,1961063424,1814886400,1891488768,2772764672 dirreq-v3-ips ae=115824,in=2504,nl=1256,us=888,jo=728,gb=720,de=496,sa=280,fr=240,om=200,ca=96,jp=80,bh=72,??=64,be=64,kw=56,qa=48,sg=32,it=24,pk=24,iq=16,ir=16,at=8,au=8,bd=8,bg=8,bn=8,br=8,by=8,ch=8,cl=8,cn=8,dj=8,dz=8,eg=8,hk=8,ie=8,il=8,kr=8,lb=8,lv=8,ly=8,md=8,mu=8,mx=8,ng=8,no=8,pr=8,ro=8,ru=8,sc=8,sd=8,se=8,si=8,so=8,tm=8,tn=8,tr=8,ua=8,uz=8,za=8 dirreq-v3-reqs ae=495328,nl=14928,us=7696,in=5136,gb=4168,fr=4128,de=3344,be=2984,jo=2240,it=928,sa=784,ca=544,om=440,qa=208,bh=184,ie=184,kw=176,jp=136,??=112,ch=104,sg=88,iq=56,at=48,bg=48,pk=48,ru=48,hk=32,ir=32,tr=32,bn=24,dz=16,il=16,lb=16,pr=16,se=16,so=16,au=8,bd=8,br=8,by=8,cl=8,cn=8,dj=8,eg=8,kr=8,lv=8,ly=8,md=8,mu=8,mx=8,ng=8,no=8,ro=8,sc=8,sd=8,si=8,tm=8,tn=8,ua=8,uz=8,za=8 bridge-ips ae=144992,in=4248,nl=1344,us=1104,jo=952,gb=800,de=560,sa=360,fr=304,om=280,ca=112,bh=104,jp=104,??=96,kw=80,be=64,qa=64,pk=32,sg=32,iq=24,it=24,so=24,bn=16,hk=16,ir=16,pr=16,ru=16,se=16,at=8,au=8,bd=8,bg=8,br=8,by=8,ch=8,cl=8,cn=8,dj=8,dz=8,eg=8,ie=8,il=8,is=8,kr=8,kz=8,lb=8,lv=8,ly=8,md=8,mu=8,mx=8,ng=8,no=8,ro=8,sc=8,sd=8,si=8,sk=8,tm=8,tn=8,tr=8,ua=8,uz=8,vn=8,ye=8,za=8
On Tue, Mar 14, 2017 at 12:02:46AM +0000, Matthew Finkel wrote:
It's interesting that the bridge users count began increasing a few days after relay users began decreasing. Actually, I found which bridge is supporting these new users. I confirmed it isn't one of the default bridges.
{"version":"4.0", "relays_published":"2017-03-13 22:00:00", "relays":[ ], "bridges_published":"2017-03-13 20:57:29", "bridges":[ {"nickname":"Unnamed","hashed_fingerprint":"220B66EBF7625B31D3313491C0B888E488F2E66B","or_addresses":["10.64.118.173:56651"],"last_seen":"2017-03-13 20:57:29","first_seen":"2016-01-18 11:55:20","running":true,"flags":["Fast","HSDir","Running","Stable","V2Dir","Valid"],"last_restarted":"2017-03-09 06:48:03","advertised_bandwidth":2503701,"platform":"Tor 0.2.9.5-alpha on Linux","transports":["scramblesuit","obfs3","obfs4"]} ]}
https://onionoo.torproject.org/details?fingerprint=220B66EBF7625B31D3313491C... https://atlas.torproject.org/#details/3E0908F131AC417C48DDD835D78FB6887F4CD1...
Nice find. I think your second link (to atlas.torproject.org) is wrong, because it's pointing to LeifEricson, which is one of the default bridges. The right link should be: https://atlas.torproject.org/#details/220B66EBF7625B31D3313491C0B888E488F2E6...
The disparity in bytes read/written is interesting. The bridge has about 6× more written bytes than read bytes. That could lend support to the idea that the inflated statistics are caused by connections that are prematurely terminated.
On 03/13/2017 06:02 PM, Matthew Finkel wrote:
I'll follow up with additional analysis tomorrow, but here's the data from 2017-03-12 00:09:00
amnesia@amnesia:~$ grep -A 23 220B66EBF7625B31D3313491C0B888E488F2E66B 2017-03-12-00-09-00-extra-infos | grep -e "^extra-info" -e history -e dirreq-v3-reqs -e bridge-ips -e "ae="
Thank you for helping to do this analysis!
I have been really curious what is going on but I hit a bit of a block on exactly what sort of query I should be running on information I've pulled down from collector. This grep seems perfect.
On Tue, Mar 14, 2017 at 12:02:46AM +0000, Matthew Finkel wrote:
But I spoke with someone at IFF from the region last week and their current thought is that this is caused by some group running a bot (of some kind)
Typically botnets have victims in many countries, though, right? How did they manage to contain their bots to just UAE hosts?
(Geofenced malicious ads? A vulnerability in an app that only UAE people install? Malware on a government website that many people need to visit? Or maybe the bots are more widespread, but for some reason the bot operator chose to only transition the UAE hosts to using Tor?)
dirreq-v3-ips ae=115824,in=2504,nl=1256,us=888,jo=728,[...] dirreq-v3-reqs ae=495328,nl=14928,us=7696,in=5136,gb=4168,[...] bridge-ips ae=144992,in=4248,nl=1344,us=1104,jo=952,[...]
Those are huge numbers, and they convince me that the phenomenon is real -- there really are many many Tor clients connecting from many many different IP addresses.
That said, when they shifted from vanilla Tor connections to bridge connections... they all shifted to one bridge? That lends a lot of credibility to the "a bunch of Tor clients, all using the same configuration, so it's all really coordinated" point.
--Roger
On Tue, Mar 14, 2017 at 11:40:15AM -0400, Roger Dingledine wrote:
On Tue, Mar 14, 2017 at 12:02:46AM +0000, Matthew Finkel wrote:
But I spoke with someone at IFF from the region last week and their current thought is that this is caused by some group running a bot (of some kind)
Typically botnets have victims in many countries, though, right? How did they manage to contain their bots to just UAE hosts?
Back in 2008, a variant of the Conficker worm wouldn't infect Ukrainian hosts. It used to look at the victim's IP address and keyboard layout to figure out where you are from. I suppose you can do the reverse to target only UAE users, despite some false positives and negatives.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Dear team,
Typically botnets have victims in many countries, though, right? How did they manage to contain their bots to just UAE hosts?
Back in 2008, a variant of the Conficker worm wouldn't infect Ukrainian hosts. It used to look at the victim's IP address and keyboard layout to figure out where you are from. I suppose you can do the reverse to target only UAE users, despite some false positives and negatives.
Such behavior is not uncommon. The botherders will look for user agent strings, language packs, IP-to-CC (country code) mappings, and the like. Some of their customers are discriminating, e.g. "I only want bots in South Korea." The reasons range from spam sourcing to DDoS to gaming mayhem.
There are multiple bots that report their CC as part of their nickname.
Be well, Rabbi Rob. - -- Rabbi Rob Thomas Team Cymru "It is easy to believe in freedom of speech for those with whom we agree." - Leo McKern
tor-project@lists.torproject.org
-
David Fifield -
David Goulet -
Joshua Gay -
Matthew Finkel -
Philipp Winter -
Rabbi Rob Thomas -
Roger Dingledine -
teor