tor-commits

tor-commits@lists.torproject.org

1 participants
215229 discussions

[translation/support-portal] https://gitweb.torproject.org/translation.git/commit/?h=support-portal
by translation＠torproject.org 23 Apr '20

23 Apr '20

commit 719b1651e84039fb83ed97a296bc9e7db4c61a48 Author: Translation commit bot <translation(a)torproject.org> Date: Thu Apr 23 06:24:09 2020 +0000 https://gitweb.torproject.org/translation.git/commit/?h=support-portal --- contents+fa.po | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/contents+fa.po b/contents+fa.po index e10a756d0f..3d8fafe93c 100644 --- a/contents+fa.po +++ b/contents+fa.po @@ -8809,7 +8809,7 @@ msgstr "" #: https//support.torproject.org/metrics/directories-reports/ #: (content/metrics/directories-reports/contents+en.lrquestion.title) msgid "Do all directories report these directory request numbers?" -msgstr "" +msgstr "آیا همه دایرکتوری‌ها این تعداد درخواست دایرکتوری را گزارش می‌کنند؟" #: https//support.torproject.org/metrics/directories-reports/ #: (content/metrics/directories-reports/contents+en.lrquestion.description) @@ -8817,6 +8817,8 @@ msgid "" "No, but we can see what fraction of directories reported them, and then we " "can extrapolate the total number in the network." msgstr "" +"نه، ولی ما می توانیم ببینیم چه کسری از دایرکتوری‌ها آن‌ها را گزارش کرده اند،" +" سپس مقدار کل را از شبکه استخراج می کنیم." #: https//support.torproject.org/metrics/graph-data-point/ #: (content/metrics/graph-data-point/contents+en.lrquestion.title) @@ -8824,6 +8826,8 @@ msgid "" "But I noticed that the last data point went up/down a bit since I last " "looked a few hours ago. Why is that?" msgstr "" +"اما من متوجه شدم که در آخرین مشاهدات میزان آن از چند ساعت پیش کمی تغییر کرده" +" است. دلیل این چیست؟" #: https//support.torproject.org/metrics/graph-data-point/ #: (content/metrics/graph-data-point/contents+en.lrquestion.description) @@ -8840,6 +8844,8 @@ msgid "" "But it's always possible that a directory reports data a few hours after we " "were confident enough, but which then slightly changed the graph." msgstr "" +"اما همیشه این احتمال وجود دارد که یک دایرکتوری داده را چند ساعت پس از اینکه " +"ما به اندازه کافی اطمنیان داشتیم گزارش کند، که این باعث تغییر گراف شده است." #: https//support.torproject.org/metrics/graphs/ #: (content/metrics/graphs/contents+en.lrquestion.title) @@ -8852,6 +8858,8 @@ msgid "" "Relays and bridges report some of the data in 24-hour intervals which may " "end at any time of the day." msgstr "" +"رله‌ها و پل‌ها برخی داده‌ها را در بازه‌های 24-ساعت گذشته گزارش کرده اند که " +"ممکن است در پایان روز به اتمام برسد." #: https//support.torproject.org/metrics/graphs/ #: (content/metrics/graphs/contents+en.lrquestion.description) @@ -8859,6 +8867,8 @@ msgid "" "And after such an interval is over relays and bridges might take another 18 " "hours to report the data." msgstr "" +"و پس از اینکه چنین بازه ای پایان یافت رله‌ها و پل‌ها ممکن است 18 است زمان " +"ببرند تا داده‌ها را گزارش کنند." #: https//support.torproject.org/metrics/graphs/ #: (content/metrics/graphs/contents+en.lrquestion.description) @@ -8867,13 +8877,16 @@ msgid "" "the last data point in a graph indicates a recent trend change which is in " "fact just an artifact of the algorithm." msgstr "" +"ما دو روز آخر را از گراف‌ها حذف کردیم، چون می خواهیم جلوی آخرین مشاهده در " +"گراف که نشان‌دهنده یک روند تغییر می باشد که در واقع دست‌ساختی از الگوریتم " +"است را بگیریم. " #: https//support.torproject.org/metrics/methodology/ #: (content/metrics/methodology/contents+en.lrquestion.title) msgid "" "Why do you believe the current approach to estimate user numbers is more " "accurate?" -msgstr "" +msgstr "چرا فکر می‌کنید که رویکرد فعلی برای تخمین تعداد کاربران دقیق‌تر است؟" #: https//support.torproject.org/metrics/methodology/ #: (content/metrics/methodology/contents+en.lrquestion.description) @@ -8881,6 +8894,8 @@ msgid "" "For direct users, we include all directories which we didn't do in the old " "approach." msgstr "" +"برای کاربران مستقیم، ما تمام دایرکتوری‌ها را که در رویکرد قدیمی در بر گرفته " +"نمی شد را در بر می‌گیریم." #: https//support.torproject.org/metrics/methodology/ #: (content/metrics/methodology/contents+en.lrquestion.description) @@ -8998,6 +9013,8 @@ msgid "" "No, because that user updates their list of relays as often as a user that " "doesn't change IP address over the day." msgstr "" +"نه، چون آن کاربر فهرست رله‌های خود را به اندازه کاربری که نشانی IP خود را در" +" طول روز تغییر نمی دهد بروز‌رسانی می‌کند." #: https//support.torproject.org/onionservices/onionservices-1/ #: (content/onionservices/onionservices-1/contents+en.lrquestion.title)

1 0

[translation/support-portal] https://gitweb.torproject.org/translation.git/commit/?h=support-portal
by translation＠torproject.org 22 Apr '20

22 Apr '20

commit b6b0570fc3298c2df2884f687b100f182521fbec Author: Translation commit bot <translation(a)torproject.org> Date: Wed Apr 22 22:24:28 2020 +0000 https://gitweb.torproject.org/translation.git/commit/?h=support-portal --- contents+fa.po | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/contents+fa.po b/contents+fa.po index 72cc0c23df..e10a756d0f 100644 --- a/contents+fa.po +++ b/contents+fa.po @@ -8756,6 +8756,9 @@ msgid "" "user numbers over a series of days and predicts the user number in the next " "days." msgstr "" +"ما یک سامانه شناسایی سانسور مبتنی بر بی هنجاری که به تعداد کاربران تخمین زده" +" شده در یک سری روز نگاه می کند و تعداد کاربران در روز‌های آتی را پیش‌بینی " +"می‌کند را می گردانیم." #: https//support.torproject.org/metrics/censorship-events/ #: (content/metrics/censorship-events/contents+en.lrquestion.description) @@ -8763,6 +8766,8 @@ msgid "" "If the actual number is higher or lower, this might indicate a possible " "censorship event or release of censorship." msgstr "" +"اگر عدد واقعی بیشتر یا کمتر باشد، این ممکن است نشان‌دهنده یک اتفاق سانسور " +"احتمالی یا آزاد سازی سانسور باشد." #: https//support.torproject.org/metrics/censorship-events/ #: (content/metrics/censorship-events/contents+en.lrquestion.description)

1 0

[translation/support-portal] https://gitweb.torproject.org/translation.git/commit/?h=support-portal
by translation＠torproject.org 22 Apr '20

22 Apr '20

commit b84c58dfef4d8ff6ff2bdaa9b00b7cb27b51540d Author: Translation commit bot <translation(a)torproject.org> Date: Wed Apr 22 21:54:34 2020 +0000 https://gitweb.torproject.org/translation.git/commit/?h=support-portal --- contents+fa.po | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/contents+fa.po b/contents+fa.po index 120fe30be0..72cc0c23df 100644 --- a/contents+fa.po +++ b/contents+fa.po @@ -2494,6 +2494,12 @@ msgid "" " address bar, include **https://** in the URL, and display the proper " "expected name for the website." msgstr "" +"هرچند، شما باید همچنا مراقب نوار نشانی اینترنتی باشید تا مطمئن شوید که " +"سایت‌هایی که در آن اطلاعات حساس وارد می‌کنید یک " +"[قفل](https://support.mozilla.org/en-US/kb/how-do-i-tell-if-my-connection-" +"is-secure) یا [نگارک پیاز](/onionservices/onionservices-5) در نوار آدرس " +"نمایش می دهد، دارای **https://** در نشانی اینترنتی می باشد، و نام مناسب را " +"برای سایت نمایش می دهد." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) @@ -2567,7 +2573,7 @@ msgstr "" #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) msgid "### Use bridges and/or find company" -msgstr "" +msgstr "### استفاده از پل‌ها و/یا پیدا کردن شرکت" #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) @@ -2584,6 +2590,8 @@ msgid "" "However, by default, it does not prevent somebody watching your Internet " "traffic from learning that you're using Tor." msgstr "" +"هرچند، به صورت پیش‌فرض، از اینکه شخصی پس از تماشای ترافیک شما متوجه شود از " +"تور استفاده می کنید جلوگیری نمی کند." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) @@ -3390,6 +3398,8 @@ msgid "" "These are example file names and will not exactly match the file names that " "you download." msgstr "" +"این اسامی فایل ها نمونه می باشند و دقیقاً با اسامی فایلی که شما دریافت کرده " +"اید تطابق نخواهند داشت." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -3621,6 +3631,8 @@ msgid "" "gone wrong and you cannot continue until you've figured out why this didn't " "work." msgstr "" +"اگر `./tor.keyring` پس از اجرای این دستور وجود نداشت، اشتباهی رخ داده است و " +"شما نمی توانید ادامه دهید تا زمانی که متوجه شوید چرا این کار نمی کند." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -6197,6 +6209,9 @@ msgid "" "menu), then click on \"Preferences\", and finally on \"Tor\" in the side " "bar." msgstr "" +"از آنسو، اگر این گزینه را نمی بینید و مرورگر تور باز می باشد، می توانید به " +"[منوی همبرگری (\"≡\")](../../glossary/hamburger-menu)، رفته، سپس روی " +"\"ترجیحات\" کلیک کرده، در پایان روی \"تور\" در نوار کناری کلیک کنید." #: https//support.torproject.org/connecting/connecting-2/ #: (content/connecting/connecting-2/contents+en.lrquestion.description) @@ -8352,6 +8367,10 @@ msgid "" " please consider [running a Tor " "relay](https://community.torproject.org/relay/)." msgstr "" +"ما به دنبال افرادی با اتصال اینترنتی به نسبت قابل اعتماد هستیم، که حداقل " +"دارای پهنای‌باند 10 مگابایت در ثانیه (Mbps) از هر جهت می باشند. اگر شما از " +"چنین ویژگی هایی برخوردار هستید، [اجرا یک رله " +"تور](https://community.torproject.org/relay/) را در نظر داشته باشید." #: https//support.torproject.org/operators/should-i-run-a-relay/ #: (content/operators/should-i-run-a-relay/contents+en.lrquestion.description) @@ -8361,6 +8380,10 @@ msgid "" "support](https://community.torproject.org/relay/setup/bridge). In that case " "you should have at least 1 MBit/s of available bandwidth." msgstr "" +"حتی اگر شما 10 مگابیت در ثانیه پهنای‌باند موجود ندارید می توانید به شبکه‌ی " +"تور با اجرای یک [پل تور با پشتیبانی " +"obfs4](https://community.torproject.org/relay/setup/bridge) کمک کنید. در این" +" مورد شما باید حداقل 1 مگابیت در ثانیه پهنای‌باند موجود داشته باشید." #: https//support.torproject.org/operators/upgrade-or-move/ #: (content/operators/upgrade-or-move/contents+en.lrquestion.title) @@ -8658,7 +8681,7 @@ msgstr "" #: https//support.torproject.org/metrics/2011-archive/ #: (content/metrics/2011-archive/contents+en.lrquestion.description) msgid "Please find the following tarball for more details:" -msgstr "" +msgstr "لطفاً تاربال زیر را برای جزئیات بیشتر پیدا کنید:" #: https//support.torproject.org/metrics/2011-archive/ #: (content/metrics/2011-archive/contents+en.lrquestion.description) @@ -8673,6 +8696,7 @@ msgid "" "And what about the advantage of the current approach over the old one when " "it comes to bridge users?" msgstr "" +"درباره مزیت رویکرد فعلی در مقایسه با رویکرد قبلی برای کاربران پل چطور؟" #: https//support.torproject.org/metrics/bridge-users/ #: (content/metrics/bridge-users/contents+en.lrquestion.description) @@ -8681,6 +8705,10 @@ msgid "" "report](https://research.torproject.org/techreports/counting-daily-bridge-" "users-2012-10-24.pdf) explaining the reasons for retiring the old approach." msgstr "" +"اوه، این یک داستان کاملاً متفاوت است. ما یک [گزارش " +"فنی](https://research.torproject.org/techreports/counting-daily-bridge-" +"users-2012-10-24.pdf) 13 صفحه ای در توضیح دلایل برای کنار گذاشتن رویکرد قبلی" +" نوشتیم." #: https//support.torproject.org/metrics/bridge-users/ #: (content/metrics/bridge-users/contents+en.lrquestion.description) @@ -8688,6 +8716,8 @@ msgid "" "tl;dr: in the old approach we measured the wrong thing, and now we measure " "the right thing." msgstr "" +"tl;dr: در رویکرد قبلی چیزی که اندازه گیری می گرفتیم صحیح نبود، ولی الان صحیح" +" می باشد" #: https//support.torproject.org/metrics/bridges-reports/ #: (content/metrics/bridges-reports/contents+en.lrquestion.title) @@ -8717,6 +8747,7 @@ msgstr "هر وقت پل‌ها این داده ها را گزارش کردند msgid "" "What are these red and blue dots indicating possible censorship events?" msgstr "" +"این نقاط آبی و قرمز که نشان دهنده اتفاق های سانسور احتمالی هستند چیستد؟" #: https//support.torproject.org/metrics/censorship-events/ #: (content/metrics/censorship-events/contents+en.lrquestion.description)

1 0

[snowflake-webext/master] Update .travis.yml and CONTRIBUTING.md
by cohosh＠torproject.org 22 Apr '20

22 Apr '20

commit 6b4ec94becd1ef51f7823115edaa359ce9a9f67b Author: Cecylia Bocovich <cohosh(a)torproject.org> Date: Thu Apr 16 10:20:09 2020 -0400 Update .travis.yml and CONTRIBUTING.md Remove the parts that apply only to the Go code and get CI working on the new directory structure. --- .travis.yml | 30 ++++-------------------------- CONTRIBUTING.md | 6 +----- 2 files changed, 5 insertions(+), 31 deletions(-) diff --git a/.travis.yml b/.travis.yml index 56d612c..cea2037 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,35 +1,13 @@ -language: go +language: node_js -dist: xenial - -go_import_path: git.torproject.org/pluggable-transports/snowflake.git - -addons: - apt: - sources: - - ubuntu-toolchain-r-test - packages: - - g++-5 - - gcc-5 +node_js: + - 8 -go: - - 1.13.x - -env: - - TRAVIS_NODE_VERSION="8" CC="gcc-5" CXX="g++-5" - -before_install: - - nvm install $TRAVIS_NODE_VERSION +dist: xenial install: - - pushd proxy - npm install - - popd script: - - test -z "$(go fmt ./...)" - - go vet ./... - - go test -v -race ./... - - cd proxy - npm run lint - npm test diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9f8bae0..cf31c7a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,5 +1 @@ - -- When editing Go, please run `go fmt` before every commit. - -- You may run tests locally with either `go test` or `npm test` for Go and - JavaScript, respectively. +- You may run tests locally with `npm test`

1 0

[snowflake-webext/master] Clean up .gitignore
by cohosh＠torproject.org 22 Apr '20

22 Apr '20

commit 891b826c8eec3d357d60b8e59269101dd80e3eeb Author: Cecylia Bocovich <cohosh(a)torproject.org> Date: Wed Apr 22 11:00:08 2020 -0400 Clean up .gitignore --- .gitignore | 7 ------- 1 file changed, 7 deletions(-) diff --git a/.gitignore b/.gitignore index 2d31939..448d843 100644 --- a/.gitignore +++ b/.gitignore @@ -3,13 +3,6 @@ *.swn *.swm .DS_Store -datadir/ -broker/broker -client/client -server-webrtc/server-webrtc -server/server -proxy-go/proxy-go -snowflake.log proxy/test proxy/build proxy/node_modules

1 0

[snowflake-webext/master] Move proxy code to top level directory
by cohosh＠torproject.org 22 Apr '20

22 Apr '20

commit 712def7a62a0ce0276782131be7df8216efa66d6 Author: Cecylia Bocovich <cohosh(a)torproject.org> Date: Thu Apr 16 10:17:31 2020 -0400 Move proxy code to top level directory --- proxy/.eslintignore => .eslintignore | 0 proxy/.eslintrc.json => .eslintrc.json | 0 .gitmodules | 2 +- README.md | 189 ++++++++++----------- proxy/broker.js => broker.js | 0 proxy/config.js => config.js | 0 proxy/init-badge.js => init-badge.js | 0 proxy/init-node.js => init-node.js | 0 proxy/init-testing.js => init-testing.js | 0 proxy/init-webext.js => init-webext.js | 0 proxy/make.js => make.js | 0 proxy/package.json => package.json | 0 proxy/README.md | 143 ---------------- proxy/proxypair.js => proxypair.js | 0 proxy/shims.js => shims.js | 0 proxy/snowflake.js => snowflake.js | 0 {proxy/spec => spec}/broker.spec.js | 0 {proxy/spec => spec}/init.spec.js | 0 {proxy/spec => spec}/proxypair.spec.js | 0 {proxy/spec => spec}/snowflake.spec.js | 0 {proxy/spec => spec}/ui.spec.js | 0 {proxy/spec => spec}/util.spec.js | 0 {proxy/spec => spec}/websocket.spec.js | 0 {proxy/static => static}/.htaccess | 0 {proxy/static => static}/SourceSansPro-Regular.ttf | Bin .../static => static}/_locales/en_US/messages.json | 0 .../assets/arrowhead-right-12.svg | 0 .../assets/arrowhead-right-dark-12.svg | 0 {proxy/static => static}/assets/favicon.ico | Bin .../static => static}/assets/status-off-dark.svg | 0 {proxy/static => static}/assets/status-off.svg | 0 {proxy/static => static}/assets/status-on-dark.svg | 0 {proxy/static => static}/assets/status-on.svg | 0 {proxy/static => static}/assets/status-running.svg | 0 {proxy/static => static}/assets/toolbar-off-48.png | Bin {proxy/static => static}/assets/toolbar-off-96.png | Bin {proxy/static => static}/assets/toolbar-off.ico | Bin {proxy/static => static}/assets/toolbar-off.svg | 0 {proxy/static => static}/assets/toolbar-on-48.png | Bin {proxy/static => static}/assets/toolbar-on-96.png | Bin {proxy/static => static}/assets/toolbar-on.ico | Bin {proxy/static => static}/assets/toolbar-on.svg | 0 .../assets/toolbar-running-48.png | Bin .../assets/toolbar-running-96.png | Bin .../static => static}/assets/toolbar-running.ico | Bin .../static => static}/assets/toolbar-running.svg | 0 {proxy/static => static}/bootstrap.css | 0 {proxy/static => static}/chrome150.jpg | Bin {proxy/static => static}/embed.css | 0 {proxy/static => static}/embed.html | 0 {proxy/static => static}/firefox150.jpg | Bin {proxy/static => static}/index.css | 0 {proxy/static => static}/index.html | 0 {proxy/static => static}/index.js | 0 {proxy/static => static}/popup.js | 0 {proxy/static => static}/screenshot.png | Bin {proxy/static => static}/tor-logo(a)2x.png | Bin proxy/translation => translation | 0 proxy/ui.js => ui.js | 0 proxy/util.js => util.js | 0 {proxy/webext => webext}/embed.js | 0 {proxy/webext => webext}/manifest.json | 0 proxy/websocket.js => websocket.js | 0 63 files changed, 93 insertions(+), 241 deletions(-) diff --git a/proxy/.eslintignore b/.eslintignore similarity index 100% rename from proxy/.eslintignore rename to .eslintignore diff --git a/proxy/.eslintrc.json b/.eslintrc.json similarity index 100% rename from proxy/.eslintrc.json rename to .eslintrc.json diff --git a/.gitmodules b/.gitmodules index 5f79304..fe6b18b 100644 --- a/.gitmodules +++ b/.gitmodules @@ -1,4 +1,4 @@ [submodule "proxy/translation"] - path = proxy/translation + path = translation url = https://git.torproject.org/translation.git branch = snowflakeaddon-messages.json_completed diff --git a/README.md b/README.md index 05fb5f7..33b8738 100644 --- a/README.md +++ b/README.md @@ -1,148 +1,143 @@ -# Snowflake +This is the browser proxy component of Snowflake. -[![Build Status](https://travis-ci.org/keroserene/snowflake.svg?branch=master)](http… +### Embedding -Pluggable Transport using WebRTC, inspired by Flashproxy. - - - -**Table of Contents** - -- [Usage](#usage) - - [Dependencies](#dependencies) - - [More Info](#more-info) - - [Building](#building) - - [Test Environment](#test-environment) -- [FAQ](#faq) -- [Appendix](#appendix) - - [-- Testing with Standalone Proxy --](#---testing-with-standalone-proxy---) - - +See https://snowflake.torproject.org/ for more info: +``` +<iframe src="https://snowflake.torproject.org/embed.html" width="88" height="16" frameborder="0" scrolling="no"></iframe> +``` -### Usage +### Building the badge / snowflake.torproject.org ``` -cd client/ -go get -go build -tor -f torrc +npm install +npm run build ``` -This should start the client plugin, bootstrapping to 100% using WebRTC. -#### Dependencies +which outputs to the `build/` directory. -Client: -- [pion/webrtc](https://github.com/pion/webrtc) -- Go 1.10+ +### Building the webextension -Proxy: -- JavaScript - ---- +``` +npm install +npm run webext +``` -#### More Info +and then load the `webext/` directory as an unpacked extension. + * https://developer.mozilla.org/en-US/docs/Tools/about:debugging#Loading_a_te… + * https://developer.chrome.com/extensions/getstarted#manifest -Tor can plug in the Snowflake client via a correctly configured `torrc`. -For example: +### Testing +Unit testing with Jasmine are available with: ``` -ClientTransportPlugin snowflake exec ./client \ --url https://snowflake-broker.azureedge.net/ \ --front ajax.aspnetcdn.com \ --ice stun:stun.l.google.com:19302 --max 3 +npm install +npm test ``` -The flags `-url` and `-front` allow the Snowflake client to speak to the Broker, -in order to get connected with some volunteer's browser proxy. `-ice` is a -comma-separated list of ICE servers, which are required for NAT traversal. +To run locally, start an http server in `build/` and navigate to `/embed.html`. -For logging, run `tail -F snowflake.log` in a second terminal. +### Preparing to deploy -You can modify the `torrc` to use your own broker: +Background information: + * https://bugs.torproject.org/23947#comment:8 + * https://help.torproject.org/tsa/doc/static-sites/ + * https://help.torproject.org/tsa/doc/ssh-jump-host/ + +You need to be in LDAP group "snowflake" and have set up an SSH key with your LDAP account. +In your ~/.ssh/config file, you should have something like: ``` -ClientTransportPlugin snowflake exec ./client --meek +Host staticiforme +HostName staticiforme.torproject.org +User <your user name> +ProxyJump people.torproject.org +IdentityFile ~/.ssh/tor ``` - -#### Building - -This describes how to build the in-browser snowflake. For the client, see Usage, -above. - -The client will only work if there are browser snowflakes available. -To run your own: +### Deploying ``` -cd proxy/ +npm install npm run build ``` -Then, start a local http server in the `proxy/build/` in any way you like. -For instance: +Do a "dry run" rsync with `-n` to check that only expected files are being changed. If you don't understand why a file would be updated, you can add the `-i` option to see the reason. ``` -cd build/ -python -m http.server +rsync -n --chown=:snowflake --chmod ug=rw,D+x --perms --delete -crv build/ staticiforme:/srv/snowflake.torproject.org/htdocs/ ``` -Then, open a browser tab to `http://127.0.0.1:8000/embed.html` to view -the debug-console of the snowflake., -So long as that tab is open, you are an ephemeral Tor bridge. +If it looks good, then repeat the rsync without `-n`. +``` +rsync --chown=:snowflake --chmod ug=rw,D+x --perms --delete -crv build/ staticiforme:/srv/snowflake.torproject.org/htdocs/ +``` -#### Test Environment +You can ignore errors of the form `rsync: failed to set permissions on "<dirname>/": Operation not permitted (1)`. -There is a Docker-based test environment at https://github.com/cohosh/snowbox. +Then run the command to copy the new files to the live web servers: +``` +ssh staticiforme 'static-update-component snowflake.torproject.org' +``` -### FAQ +### Parameters -**Q: How does it work?** +With no parameters, +snowflake uses the default relay `snowflake.freehaven.net:443` and +uses automatic signaling with the default broker at +`https://snowflake-broker.freehaven.net/`. -In the Tor use-case: +### Reuse as a library -1. Volunteers visit websites which host the "snowflake" proxy. (just -like flashproxy) -2. Tor clients automatically find available browser proxies via the Broker -(the domain fronted signaling channel). -3. Tor client and browser proxy establish a WebRTC peer connection. -4. Proxy connects to some relay. -5. Tor occurs. +The badge and the webextension make use of the same underlying library and +only differ in their UI. That same library can be produced for use with other +interfaces, such as [Cupcake][1], by running, -More detailed information about how clients, snowflake proxies, and the Broker -fit together on the way... +``` +npm install +npm run library +``` -**Q: What are the benefits of this PT compared with other PTs?** +which outputs a `./snowflake-library.js`. -Snowflake combines the advantages of flashproxy and meek. Primarily: +You'd then want to create a subclass of `UI` to perform various actions as +the state of the snowflake changes, -- It has the convenience of Meek, but can support magnitudes more -users with negligible CDN costs. (Domain fronting is only used for brief -signalling / NAT-piercing to setup the P2P WebRTC DataChannels which handle -the actual traffic.) +``` +class MyUI extends UI { + ... +} +``` -- Arbitrarily high numbers of volunteer proxies are possible like in -flashproxy, but NATs are no longer a usability barrier - no need for -manual port forwarding! +See `WebExtUI` in `init-webext.js` and `BadgeUI` in `init-badge.js` for +examples. -**Q: Why is this called Snowflake?** +Finally, initialize the snowflake with, -It utilizes the "ICE" negotiation via WebRTC, and also involves a great -abundance of ephemeral and short-lived (and special!) volunteer proxies... +``` +var log = function(msg) { + return console.log('Snowflake: ' + msg); +}; +var dbg = log; -### Appendix +var config = new Config("myui"); // NOTE: Set a unique proxy type for metrics +var ui = new MyUI(); // NOTE: Using the class defined above +var broker = new Broker(config.brokerUrl); -##### -- Testing with Standalone Proxy -- +var snowflake = new Snowflake(config, ui, broker); +snowflake.setRelayAddr(config.relayAddr); +snowflake.beginWebRTC(); ``` -cd proxy-go -go build -./proxy-go -``` -More documentation on the way. +This minimal setup is pretty much what's currently in `init-node.js`. + +When configuring the snowflake, set a unique `proxyType` (first argument +to `Config`) that will be used when recording metrics at the broker. Also, +it would be helpful to get in touch with the [Anti-Censorship Team][2] at the +Tor Project to let them know about your tool. -Also available at: -[torproject.org/pluggable-transports/snowflake](https://gitweb.torproject.org/pluggable-transports/snowflake.git/) +[1]: https://chrome.google.com/webstore/detail/cupcake/dajjbehmbnbppjkcnpdkaniap… +[2]: https://trac.torproject.org/projects/tor/wiki/org/teams/AntiCensorshipTeam diff --git a/proxy/broker.js b/broker.js similarity index 100% rename from proxy/broker.js rename to broker.js diff --git a/proxy/config.js b/config.js similarity index 100% rename from proxy/config.js rename to config.js diff --git a/proxy/init-badge.js b/init-badge.js similarity index 100% rename from proxy/init-badge.js rename to init-badge.js diff --git a/proxy/init-node.js b/init-node.js similarity index 100% rename from proxy/init-node.js rename to init-node.js diff --git a/proxy/init-testing.js b/init-testing.js similarity index 100% rename from proxy/init-testing.js rename to init-testing.js diff --git a/proxy/init-webext.js b/init-webext.js similarity index 100% rename from proxy/init-webext.js rename to init-webext.js diff --git a/proxy/make.js b/make.js similarity index 100% rename from proxy/make.js rename to make.js diff --git a/proxy/package.json b/package.json similarity index 100% rename from proxy/package.json rename to package.json diff --git a/proxy/README.md b/proxy/README.md deleted file mode 100644 index 33b8738..0000000 --- a/proxy/README.md +++ /dev/null @@ -1,143 +0,0 @@ -This is the browser proxy component of Snowflake. - -### Embedding - -See https://snowflake.torproject.org/ for more info: -``` -<iframe src="https://snowflake.torproject.org/embed.html" width="88" height="16" frameborder="0" scrolling="no"></iframe> -``` - -### Building the badge / snowflake.torproject.org - -``` -npm install -npm run build -``` - -which outputs to the `build/` directory. - -### Building the webextension - -``` -npm install -npm run webext -``` - -and then load the `webext/` directory as an unpacked extension. - * https://developer.mozilla.org/en-US/docs/Tools/about:debugging#Loading_a_te… - * https://developer.chrome.com/extensions/getstarted#manifest - -### Testing - -Unit testing with Jasmine are available with: -``` -npm install -npm test -``` - -To run locally, start an http server in `build/` and navigate to `/embed.html`. - -### Preparing to deploy - -Background information: - * https://bugs.torproject.org/23947#comment:8 - * https://help.torproject.org/tsa/doc/static-sites/ - * https://help.torproject.org/tsa/doc/ssh-jump-host/ - -You need to be in LDAP group "snowflake" and have set up an SSH key with your LDAP account. -In your ~/.ssh/config file, you should have something like: - -``` -Host staticiforme -HostName staticiforme.torproject.org -User <your user name> -ProxyJump people.torproject.org -IdentityFile ~/.ssh/tor -``` - -### Deploying - -``` -npm install -npm run build -``` - -Do a "dry run" rsync with `-n` to check that only expected files are being changed. If you don't understand why a file would be updated, you can add the `-i` option to see the reason. - -``` -rsync -n --chown=:snowflake --chmod ug=rw,D+x --perms --delete -crv build/ staticiforme:/srv/snowflake.torproject.org/htdocs/ -``` - -If it looks good, then repeat the rsync without `-n`. - -``` -rsync --chown=:snowflake --chmod ug=rw,D+x --perms --delete -crv build/ staticiforme:/srv/snowflake.torproject.org/htdocs/ -``` - -You can ignore errors of the form `rsync: failed to set permissions on "<dirname>/": Operation not permitted (1)`. - -Then run the command to copy the new files to the live web servers: - -``` -ssh staticiforme 'static-update-component snowflake.torproject.org' -``` - -### Parameters - -With no parameters, -snowflake uses the default relay `snowflake.freehaven.net:443` and -uses automatic signaling with the default broker at -`https://snowflake-broker.freehaven.net/`. - -### Reuse as a library - -The badge and the webextension make use of the same underlying library and -only differ in their UI. That same library can be produced for use with other -interfaces, such as [Cupcake][1], by running, - -``` -npm install -npm run library -``` - -which outputs a `./snowflake-library.js`. - -You'd then want to create a subclass of `UI` to perform various actions as -the state of the snowflake changes, - -``` -class MyUI extends UI { - ... -} -``` - -See `WebExtUI` in `init-webext.js` and `BadgeUI` in `init-badge.js` for -examples. - -Finally, initialize the snowflake with, - -``` -var log = function(msg) { - return console.log('Snowflake: ' + msg); -}; -var dbg = log; - -var config = new Config("myui"); // NOTE: Set a unique proxy type for metrics -var ui = new MyUI(); // NOTE: Using the class defined above -var broker = new Broker(config.brokerUrl); - -var snowflake = new Snowflake(config, ui, broker); - -snowflake.setRelayAddr(config.relayAddr); -snowflake.beginWebRTC(); -``` - -This minimal setup is pretty much what's currently in `init-node.js`. - -When configuring the snowflake, set a unique `proxyType` (first argument -to `Config`) that will be used when recording metrics at the broker. Also, -it would be helpful to get in touch with the [Anti-Censorship Team][2] at the -Tor Project to let them know about your tool. - -[1]: https://chrome.google.com/webstore/detail/cupcake/dajjbehmbnbppjkcnpdkaniap… -[2]: https://trac.torproject.org/projects/tor/wiki/org/teams/AntiCensorshipTeam diff --git a/proxy/proxypair.js b/proxypair.js similarity index 100% rename from proxy/proxypair.js rename to proxypair.js diff --git a/proxy/shims.js b/shims.js similarity index 100% rename from proxy/shims.js rename to shims.js diff --git a/proxy/snowflake.js b/snowflake.js similarity index 100% rename from proxy/snowflake.js rename to snowflake.js diff --git a/proxy/spec/broker.spec.js b/spec/broker.spec.js similarity index 100% rename from proxy/spec/broker.spec.js rename to spec/broker.spec.js diff --git a/proxy/spec/init.spec.js b/spec/init.spec.js similarity index 100% rename from proxy/spec/init.spec.js rename to spec/init.spec.js diff --git a/proxy/spec/proxypair.spec.js b/spec/proxypair.spec.js similarity index 100% rename from proxy/spec/proxypair.spec.js rename to spec/proxypair.spec.js diff --git a/proxy/spec/snowflake.spec.js b/spec/snowflake.spec.js similarity index 100% rename from proxy/spec/snowflake.spec.js rename to spec/snowflake.spec.js diff --git a/proxy/spec/ui.spec.js b/spec/ui.spec.js similarity index 100% rename from proxy/spec/ui.spec.js rename to spec/ui.spec.js diff --git a/proxy/spec/util.spec.js b/spec/util.spec.js similarity index 100% rename from proxy/spec/util.spec.js rename to spec/util.spec.js diff --git a/proxy/spec/websocket.spec.js b/spec/websocket.spec.js similarity index 100% rename from proxy/spec/websocket.spec.js rename to spec/websocket.spec.js diff --git a/proxy/static/.htaccess b/static/.htaccess similarity index 100% rename from proxy/static/.htaccess rename to static/.htaccess diff --git a/proxy/static/SourceSansPro-Regular.ttf b/static/SourceSansPro-Regular.ttf similarity index 100% rename from proxy/static/SourceSansPro-Regular.ttf rename to static/SourceSansPro-Regular.ttf diff --git a/proxy/static/_locales/en_US/messages.json b/static/_locales/en_US/messages.json similarity index 100% rename from proxy/static/_locales/en_US/messages.json rename to static/_locales/en_US/messages.json diff --git a/proxy/static/assets/arrowhead-right-12.svg b/static/assets/arrowhead-right-12.svg similarity index 100% rename from proxy/static/assets/arrowhead-right-12.svg rename to static/assets/arrowhead-right-12.svg diff --git a/proxy/static/assets/arrowhead-right-dark-12.svg b/static/assets/arrowhead-right-dark-12.svg similarity index 100% rename from proxy/static/assets/arrowhead-right-dark-12.svg rename to static/assets/arrowhead-right-dark-12.svg diff --git a/proxy/static/assets/favicon.ico b/static/assets/favicon.ico similarity index 100% rename from proxy/static/assets/favicon.ico rename to static/assets/favicon.ico diff --git a/proxy/static/assets/status-off-dark.svg b/static/assets/status-off-dark.svg similarity index 100% rename from proxy/static/assets/status-off-dark.svg rename to static/assets/status-off-dark.svg diff --git a/proxy/static/assets/status-off.svg b/static/assets/status-off.svg similarity index 100% rename from proxy/static/assets/status-off.svg rename to static/assets/status-off.svg diff --git a/proxy/static/assets/status-on-dark.svg b/static/assets/status-on-dark.svg similarity index 100% rename from proxy/static/assets/status-on-dark.svg rename to static/assets/status-on-dark.svg diff --git a/proxy/static/assets/status-on.svg b/static/assets/status-on.svg similarity index 100% rename from proxy/static/assets/status-on.svg rename to static/assets/status-on.svg diff --git a/proxy/static/assets/status-running.svg b/static/assets/status-running.svg similarity index 100% rename from proxy/static/assets/status-running.svg rename to static/assets/status-running.svg diff --git a/proxy/static/assets/toolbar-off-48.png b/static/assets/toolbar-off-48.png similarity index 100% rename from proxy/static/assets/toolbar-off-48.png rename to static/assets/toolbar-off-48.png diff --git a/proxy/static/assets/toolbar-off-96.png b/static/assets/toolbar-off-96.png similarity index 100% rename from proxy/static/assets/toolbar-off-96.png rename to static/assets/toolbar-off-96.png diff --git a/proxy/static/assets/toolbar-off.ico b/static/assets/toolbar-off.ico similarity index 100% rename from proxy/static/assets/toolbar-off.ico rename to static/assets/toolbar-off.ico diff --git a/proxy/static/assets/toolbar-off.svg b/static/assets/toolbar-off.svg similarity index 100% rename from proxy/static/assets/toolbar-off.svg rename to static/assets/toolbar-off.svg diff --git a/proxy/static/assets/toolbar-on-48.png b/static/assets/toolbar-on-48.png similarity index 100% rename from proxy/static/assets/toolbar-on-48.png rename to static/assets/toolbar-on-48.png diff --git a/proxy/static/assets/toolbar-on-96.png b/static/assets/toolbar-on-96.png similarity index 100% rename from proxy/static/assets/toolbar-on-96.png rename to static/assets/toolbar-on-96.png diff --git a/proxy/static/assets/toolbar-on.ico b/static/assets/toolbar-on.ico similarity index 100% rename from proxy/static/assets/toolbar-on.ico rename to static/assets/toolbar-on.ico diff --git a/proxy/static/assets/toolbar-on.svg b/static/assets/toolbar-on.svg similarity index 100% rename from proxy/static/assets/toolbar-on.svg rename to static/assets/toolbar-on.svg diff --git a/proxy/static/assets/toolbar-running-48.png b/static/assets/toolbar-running-48.png similarity index 100% rename from proxy/static/assets/toolbar-running-48.png rename to static/assets/toolbar-running-48.png diff --git a/proxy/static/assets/toolbar-running-96.png b/static/assets/toolbar-running-96.png similarity index 100% rename from proxy/static/assets/toolbar-running-96.png rename to static/assets/toolbar-running-96.png diff --git a/proxy/static/assets/toolbar-running.ico b/static/assets/toolbar-running.ico similarity index 100% rename from proxy/static/assets/toolbar-running.ico rename to static/assets/toolbar-running.ico diff --git a/proxy/static/assets/toolbar-running.svg b/static/assets/toolbar-running.svg similarity index 100% rename from proxy/static/assets/toolbar-running.svg rename to static/assets/toolbar-running.svg diff --git a/proxy/static/bootstrap.css b/static/bootstrap.css similarity index 100% rename from proxy/static/bootstrap.css rename to static/bootstrap.css diff --git a/proxy/static/chrome150.jpg b/static/chrome150.jpg similarity index 100% rename from proxy/static/chrome150.jpg rename to static/chrome150.jpg diff --git a/proxy/static/embed.css b/static/embed.css similarity index 100% rename from proxy/static/embed.css rename to static/embed.css diff --git a/proxy/static/embed.html b/static/embed.html similarity index 100% rename from proxy/static/embed.html rename to static/embed.html diff --git a/proxy/static/firefox150.jpg b/static/firefox150.jpg similarity index 100% rename from proxy/static/firefox150.jpg rename to static/firefox150.jpg diff --git a/proxy/static/index.css b/static/index.css similarity index 100% rename from proxy/static/index.css rename to static/index.css diff --git a/proxy/static/index.html b/static/index.html similarity index 100% rename from proxy/static/index.html rename to static/index.html diff --git a/proxy/static/index.js b/static/index.js similarity index 100% rename from proxy/static/index.js rename to static/index.js diff --git a/proxy/static/popup.js b/static/popup.js similarity index 100% rename from proxy/static/popup.js rename to static/popup.js diff --git a/proxy/static/screenshot.png b/static/screenshot.png similarity index 100% rename from proxy/static/screenshot.png rename to static/screenshot.png diff --git a/proxy/static/tor-logo(a)2x.png b/static/tor-logo(a)2x.png similarity index 100% rename from proxy/static/tor-logo(a)2x.png rename to static/tor-logo(a)2x.png diff --git a/proxy/translation b/translation similarity index 100% rename from proxy/translation rename to translation diff --git a/proxy/ui.js b/ui.js similarity index 100% rename from proxy/ui.js rename to ui.js diff --git a/proxy/util.js b/util.js similarity index 100% rename from proxy/util.js rename to util.js diff --git a/proxy/webext/embed.js b/webext/embed.js similarity index 100% rename from proxy/webext/embed.js rename to webext/embed.js diff --git a/proxy/webext/manifest.json b/webext/manifest.json similarity index 100% rename from proxy/webext/manifest.json rename to webext/manifest.json diff --git a/proxy/websocket.js b/websocket.js similarity index 100% rename from proxy/websocket.js rename to websocket.js

1 0

[snowflake-webext/master] Remove all Go pieces
by cohosh＠torproject.org 22 Apr '20

22 Apr '20

commit 03d7dd26d419bc4441e3eb06c11e1813bc8e9b3c Author: Cecylia Bocovich <cohosh(a)torproject.org> Date: Thu Apr 16 10:12:01 2020 -0400 Remove all Go pieces This repository is for the web proxy only --- broker/README.md | 49 -- broker/broker.go | 503 ----------- broker/geoip.go | 240 ------ broker/metrics.go | 196 ----- broker/snowflake-broker_test.go | 611 -------------- broker/snowflake-heap.go | 51 -- broker/test_geoip | 1236 ---------------------------- broker/test_geoip6 | 693 ---------------- client/README.md | 20 - client/client_test.go | 59 -- client/lib/interfaces.go | 55 -- client/lib/lib_test.go | 361 -------- client/lib/peers.go | 123 --- client/lib/rendezvous.go | 156 ---- client/lib/snowflake.go | 68 -- client/lib/util.go | 85 -- client/lib/webrtc.go | 380 --------- client/snowflake.go | 221 ----- client/torrc | 10 - client/torrc-localhost | 8 - common/messages/proxy.go | 222 ----- common/messages/proxy_test.go | 234 ------ common/safelog/log.go | 71 -- common/safelog/log_test.go | 148 ---- common/util/util.go | 103 --- common/util/util_test.go | 26 - common/websocketconn/websocketconn.go | 120 --- common/websocketconn/websocketconn_test.go | 263 ------ doc/broker-spec.txt | 69 -- go.mod | 15 - go.sum | 119 --- proxy-go/README.md | 3 - proxy-go/proxy-go_test.go | 397 --------- proxy-go/snowflake.go | 484 ----------- server/README.md | 61 -- server/server.go | 361 -------- server/server_test.go | 153 ---- server/stats.go | 44 - server/torrc | 7 - 39 files changed, 8025 deletions(-) diff --git a/broker/README.md b/broker/README.md deleted file mode 100644 index fb6181e..0000000 --- a/broker/README.md +++ /dev/null @@ -1,49 +0,0 @@ -This is the Broker component of Snowflake. - -### Overview - -The Broker handles the rendezvous by matching Snowflake -Clients with Proxies, and passing their WebRTC Session Descriptions -(the "signaling" step). This allows Clients and Proxies to establish -a Peer connection. - -It is analogous to Flashproxy's -[Facilitator](https://trac.torproject.org/projects/tor/wiki/FlashProxyFAQ), -but bidirectional and domain-fronted. - -The Broker expects: - -- Clients to send their SDP offer in a POST request, which will then block - until the Broker responds with the answer of the matched Proxy. -- Proxies to announce themselves with a POST request, to which the Broker - responds with some Client's SDP offer. The Proxy should then send a second - POST request soon after containing its SDP answer, which the Broker passes - back to the same Client. - -### Running your own - -The server uses TLS by default. -There is a `--disable-tls` option for testing purposes, -but you should use TLS in production. - -The server automatically fetches certificates -from [Let's Encrypt](https://en.wikipedia.org/wiki/Let's_Encrypt) as needed. -Use the `--acme-hostnames` option to tell the server -what hostnames it may request certificates for. -You can optionally provide a contact email address, -using the `--acme-email` option, -so that Let's Encrypt can inform you of any problems. - -In order to fetch certificates automatically, -the server needs to open an additional HTTP listener on port 80. -On Linux, you can use the `setcap` program, -part of libcap2, to enable the broker to bind to low-numbered ports -without having to run as root: -``` -setcap 'cap_net_bind_service=+ep' /usr/local/bin/broker -``` -You can control the listening broker port with the --addr option. -Port 443 is the default. - -You'll need to provide the URL of the custom broker -to the client plugin using the `--url $URL` flag. diff --git a/broker/broker.go b/broker/broker.go deleted file mode 100644 index d9ef111..0000000 --- a/broker/broker.go +++ /dev/null @@ -1,503 +0,0 @@ -/* -Broker acts as the HTTP signaling channel. -It matches clients and snowflake proxies by passing corresponding -SessionDescriptions in order to negotiate a WebRTC connection. -*/ -package main - -import ( - "container/heap" - "crypto/tls" - "flag" - "fmt" - "io" - "io/ioutil" - "log" - "net" - "net/http" - "os" - "os/signal" - "strings" - "sync" - "syscall" - "time" - - "git.torproject.org/pluggable-transports/snowflake.git/common/messages" - "git.torproject.org/pluggable-transports/snowflake.git/common/safelog" - "golang.org/x/crypto/acme/autocert" -) - -const ( - ClientTimeout = 10 - ProxyTimeout = 10 - readLimit = 100000 //Maximum number of bytes to be read from an HTTP request -) - -type BrokerContext struct { - snowflakes *SnowflakeHeap - // Map keeping track of snowflakeIDs required to match SDP answers from - // the second http POST. - idToSnowflake map[string]*Snowflake - // Synchronization for the snowflake map and heap - snowflakeLock sync.Mutex - proxyPolls chan *ProxyPoll - metrics *Metrics -} - -func NewBrokerContext(metricsLogger *log.Logger) *BrokerContext { - snowflakes := new(SnowflakeHeap) - heap.Init(snowflakes) - metrics, err := NewMetrics(metricsLogger) - - if err != nil { - panic(err.Error()) - } - - if metrics == nil { - panic("Failed to create metrics") - } - - return &BrokerContext{ - snowflakes: snowflakes, - idToSnowflake: make(map[string]*Snowflake), - proxyPolls: make(chan *ProxyPoll), - metrics: metrics, - } -} - -// Implements the http.Handler interface -type SnowflakeHandler struct { - *BrokerContext - handle func(*BrokerContext, http.ResponseWriter, *http.Request) -} - -// Implements the http.Handler interface -type MetricsHandler struct { - logFilename string - handle func(string, http.ResponseWriter, *http.Request) -} - -func (sh SnowflakeHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { - w.Header().Set("Access-Control-Allow-Origin", "*") - w.Header().Set("Access-Control-Allow-Headers", "Origin, X-Session-ID") - // Return early if it's CORS preflight. - if "OPTIONS" == r.Method { - return - } - sh.handle(sh.BrokerContext, w, r) -} - -func (mh MetricsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { - w.Header().Set("Access-Control-Allow-Origin", "*") - w.Header().Set("Access-Control-Allow-Headers", "Origin, X-Session-ID") - // Return early if it's CORS preflight. - if "OPTIONS" == r.Method { - return - } - mh.handle(mh.logFilename, w, r) -} - -// Proxies may poll for client offers concurrently. -type ProxyPoll struct { - id string - proxyType string - offerChannel chan []byte -} - -// Registers a Snowflake and waits for some Client to send an offer, -// as part of the polling logic of the proxy handler. -func (ctx *BrokerContext) RequestOffer(id string, proxyType string) []byte { - request := new(ProxyPoll) - request.id = id - request.proxyType = proxyType - request.offerChannel = make(chan []byte) - ctx.proxyPolls <- request - // Block until an offer is available, or timeout which sends a nil offer. - offer := <-request.offerChannel - return offer -} - -// goroutine which matches clients to proxies and sends SDP offers along. -// Safely processes proxy requests, responding to them with either an available -// client offer or nil on timeout / none are available. -func (ctx *BrokerContext) Broker() { - for request := range ctx.proxyPolls { - snowflake := ctx.AddSnowflake(request.id, request.proxyType) - // Wait for a client to avail an offer to the snowflake. - go func(request *ProxyPoll) { - select { - case offer := <-snowflake.offerChannel: - request.offerChannel <- offer - case <-time.After(time.Second * ProxyTimeout): - // This snowflake is no longer available to serve clients. - ctx.snowflakeLock.Lock() - defer ctx.snowflakeLock.Unlock() - if snowflake.index != -1 { - heap.Remove(ctx.snowflakes, snowflake.index) - delete(ctx.idToSnowflake, snowflake.id) - close(request.offerChannel) - } - } - }(request) - } -} - -// Create and add a Snowflake to the heap. -// Required to keep track of proxies between providing them -// with an offer and awaiting their second POST with an answer. -func (ctx *BrokerContext) AddSnowflake(id string, proxyType string) *Snowflake { - snowflake := new(Snowflake) - snowflake.id = id - snowflake.clients = 0 - snowflake.proxyType = proxyType - snowflake.offerChannel = make(chan []byte) - snowflake.answerChannel = make(chan []byte) - ctx.snowflakeLock.Lock() - heap.Push(ctx.snowflakes, snowflake) - ctx.snowflakeLock.Unlock() - ctx.idToSnowflake[id] = snowflake - return snowflake -} - -/* -For snowflake proxies to request a client from the Broker. -*/ -func proxyPolls(ctx *BrokerContext, w http.ResponseWriter, r *http.Request) { - body, err := ioutil.ReadAll(http.MaxBytesReader(w, r.Body, readLimit)) - if err != nil { - log.Println("Invalid data.") - w.WriteHeader(http.StatusBadRequest) - return - } - - sid, proxyType, err := messages.DecodePollRequest(body) - if err != nil { - w.WriteHeader(http.StatusBadRequest) - return - } - - // Log geoip stats - remoteIP, _, err := net.SplitHostPort(r.RemoteAddr) - if err != nil { - log.Println("Error processing proxy IP: ", err.Error()) - } else { - ctx.metrics.lock.Lock() - ctx.metrics.UpdateCountryStats(remoteIP, proxyType) - ctx.metrics.lock.Unlock() - } - - // Wait for a client to avail an offer to the snowflake, or timeout if nil. - offer := ctx.RequestOffer(sid, proxyType) - var b []byte - if nil == offer { - ctx.metrics.lock.Lock() - ctx.metrics.proxyIdleCount++ - ctx.metrics.lock.Unlock() - - b, err = messages.EncodePollResponse("", false) - if err != nil { - w.WriteHeader(http.StatusInternalServerError) - return - } - - w.Write(b) - return - } - b, err = messages.EncodePollResponse(string(offer), true) - if err != nil { - w.WriteHeader(http.StatusInternalServerError) - return - } - if _, err := w.Write(b); err != nil { - log.Printf("proxyPolls unable to write offer with error: %v", err) - } -} - -/* -Expects a WebRTC SDP offer in the Request to give to an assigned -snowflake proxy, which responds with the SDP answer to be sent in -the HTTP response back to the client. -*/ -func clientOffers(ctx *BrokerContext, w http.ResponseWriter, r *http.Request) { - startTime := time.Now() - offer, err := ioutil.ReadAll(http.MaxBytesReader(w, r.Body, readLimit)) - if nil != err { - log.Println("Invalid data.") - w.WriteHeader(http.StatusBadRequest) - return - } - // Immediately fail if there are no snowflakes available. - ctx.snowflakeLock.Lock() - numSnowflakes := ctx.snowflakes.Len() - ctx.snowflakeLock.Unlock() - if numSnowflakes <= 0 { - ctx.metrics.lock.Lock() - ctx.metrics.clientDeniedCount++ - ctx.metrics.lock.Unlock() - w.WriteHeader(http.StatusServiceUnavailable) - return - } - // Otherwise, find the most available snowflake proxy, and pass the offer to it. - // Delete must be deferred in order to correctly process answer request later. - ctx.snowflakeLock.Lock() - snowflake := heap.Pop(ctx.snowflakes).(*Snowflake) - ctx.snowflakeLock.Unlock() - snowflake.offerChannel <- offer - - // Wait for the answer to be returned on the channel or timeout. - select { - case answer := <-snowflake.answerChannel: - ctx.metrics.lock.Lock() - ctx.metrics.clientProxyMatchCount++ - ctx.metrics.lock.Unlock() - if _, err := w.Write(answer); err != nil { - log.Printf("unable to write answer with error: %v", err) - } - // Initial tracking of elapsed time. - ctx.metrics.clientRoundtripEstimate = time.Since(startTime) / - time.Millisecond - case <-time.After(time.Second * ClientTimeout): - log.Println("Client: Timed out.") - w.WriteHeader(http.StatusGatewayTimeout) - if _, err := w.Write([]byte("timed out waiting for answer!")); err != nil { - log.Printf("unable to write timeout error, failed with error: %v", err) - } - } - - ctx.snowflakeLock.Lock() - delete(ctx.idToSnowflake, snowflake.id) - ctx.snowflakeLock.Unlock() -} - -/* -Expects snowflake proxes which have previously successfully received -an offer from proxyHandler to respond with an answer in an HTTP POST, -which the broker will pass back to the original client. -*/ -func proxyAnswers(ctx *BrokerContext, w http.ResponseWriter, r *http.Request) { - - body, err := ioutil.ReadAll(http.MaxBytesReader(w, r.Body, readLimit)) - if nil != err || nil == body || len(body) <= 0 { - log.Println("Invalid data.") - w.WriteHeader(http.StatusBadRequest) - return - } - - answer, id, err := messages.DecodeAnswerRequest(body) - if err != nil || answer == "" { - w.WriteHeader(http.StatusBadRequest) - return - } - - var success = true - ctx.snowflakeLock.Lock() - snowflake, ok := ctx.idToSnowflake[id] - ctx.snowflakeLock.Unlock() - if !ok || nil == snowflake { - // The snowflake took too long to respond with an answer, so its client - // disappeared / the snowflake is no longer recognized by the Broker. - success = false - } - b, err := messages.EncodeAnswerResponse(success) - if err != nil { - log.Printf("Error encoding answer: %s", err.Error()) - w.WriteHeader(http.StatusInternalServerError) - return - } - w.Write(b) - - if success { - snowflake.answerChannel <- []byte(answer) - } - -} - -func debugHandler(ctx *BrokerContext, w http.ResponseWriter, r *http.Request) { - - var webexts, browsers, standalones, unknowns int - ctx.snowflakeLock.Lock() - s := fmt.Sprintf("current snowflakes available: %d\n", len(ctx.idToSnowflake)) - for _, snowflake := range ctx.idToSnowflake { - if snowflake.proxyType == "badge" { - browsers++ - } else if snowflake.proxyType == "webext" { - webexts++ - } else if snowflake.proxyType == "standalone" { - standalones++ - } else { - unknowns++ - } - - } - ctx.snowflakeLock.Unlock() - s += fmt.Sprintf("\tstandalone proxies: %d", standalones) - s += fmt.Sprintf("\n\tbrowser proxies: %d", browsers) - s += fmt.Sprintf("\n\twebext proxies: %d", webexts) - s += fmt.Sprintf("\n\tunknown proxies: %d", unknowns) - if _, err := w.Write([]byte(s)); err != nil { - log.Printf("writing proxy information returned error: %v ", err) - } -} - -func robotsTxtHandler(w http.ResponseWriter, r *http.Request) { - w.Header().Set("Content-Type", "text/plain; charset=utf-8") - if _, err := w.Write([]byte("User-agent: *\nDisallow: /\n")); err != nil { - log.Printf("robotsTxtHandler unable to write, with this error: %v", err) - } -} - -func metricsHandler(metricsFilename string, w http.ResponseWriter, r *http.Request) { - w.Header().Set("Content-Type", "text/plain; charset=utf-8") - - if metricsFilename == "" { - http.NotFound(w, r) - return - } - metricsFile, err := os.OpenFile(metricsFilename, os.O_RDONLY, 0644) - if err != nil { - log.Println("Error opening metrics file for reading") - http.NotFound(w, r) - return - } - - if _, err := io.Copy(w, metricsFile); err != nil { - log.Printf("copying metricsFile returned error: %v", err) - } -} - -func main() { - var acmeEmail string - var acmeHostnamesCommas string - var acmeCertCacheDir string - var addr string - var geoipDatabase string - var geoip6Database string - var disableTLS bool - var certFilename, keyFilename string - var disableGeoip bool - var metricsFilename string - var unsafeLogging bool - - flag.StringVar(&acmeEmail, "acme-email", "", "optional contact email for Let's Encrypt notifications") - flag.StringVar(&acmeHostnamesCommas, "acme-hostnames", "", "comma-separated hostnames for TLS certificate") - flag.StringVar(&certFilename, "cert", "", "TLS certificate file") - flag.StringVar(&keyFilename, "key", "", "TLS private key file") - flag.StringVar(&acmeCertCacheDir, "acme-cert-cache", "acme-cert-cache", "directory in which certificates should be cached") - flag.StringVar(&addr, "addr", ":443", "address to listen on") - flag.StringVar(&geoipDatabase, "geoipdb", "/usr/share/tor/geoip", "path to correctly formatted geoip database mapping IPv4 address ranges to country codes") - flag.StringVar(&geoip6Database, "geoip6db", "/usr/share/tor/geoip6", "path to correctly formatted geoip database mapping IPv6 address ranges to country codes") - flag.BoolVar(&disableTLS, "disable-tls", false, "don't use HTTPS") - flag.BoolVar(&disableGeoip, "disable-geoip", false, "don't use geoip for stats collection") - flag.StringVar(&metricsFilename, "metrics-log", "", "path to metrics logging output") - flag.BoolVar(&unsafeLogging, "unsafe-logging", false, "prevent logs from being scrubbed") - flag.Parse() - - var err error - var metricsFile io.Writer - var logOutput io.Writer = os.Stderr - if unsafeLogging { - log.SetOutput(logOutput) - } else { - // We want to send the log output through our scrubber first - log.SetOutput(&safelog.LogScrubber{Output: logOutput}) - } - - log.SetFlags(log.LstdFlags | log.LUTC) - - if metricsFilename != "" { - metricsFile, err = os.OpenFile(metricsFilename, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644) - - if err != nil { - log.Fatal(err.Error()) - } - } else { - metricsFile = os.Stdout - } - - metricsLogger := log.New(metricsFile, "", 0) - - ctx := NewBrokerContext(metricsLogger) - - if !disableGeoip { - err = ctx.metrics.LoadGeoipDatabases(geoipDatabase, geoip6Database) - if err != nil { - log.Fatal(err.Error()) - } - } - - go ctx.Broker() - - http.HandleFunc("/robots.txt", robotsTxtHandler) - - http.Handle("/proxy", SnowflakeHandler{ctx, proxyPolls}) - http.Handle("/client", SnowflakeHandler{ctx, clientOffers}) - http.Handle("/answer", SnowflakeHandler{ctx, proxyAnswers}) - http.Handle("/debug", SnowflakeHandler{ctx, debugHandler}) - http.Handle("/metrics", MetricsHandler{metricsFilename, metricsHandler}) - - server := http.Server{ - Addr: addr, - } - - sigChan := make(chan os.Signal, 1) - signal.Notify(sigChan, syscall.SIGHUP) - - // go routine to handle a SIGHUP signal to allow the broker operator to send - // a SIGHUP signal when the geoip database files are updated, without requiring - // a restart of the broker - go func() { - for { - signal := <-sigChan - log.Printf("Received signal: %s. Reloading geoip databases.", signal) - if err = ctx.metrics.LoadGeoipDatabases(geoipDatabase, geoip6Database); err != nil { - log.Fatalf("reload of Geo IP databases on signal %s returned error: %v", signal, err) - } - } - }() - - // Handle the various ways of setting up TLS. The legal configurations - // are: - // --acme-hostnames (with optional --acme-email and/or --acme-cert-cache) - // --cert and --key together - // --disable-tls - // The outputs of this block of code are the disableTLS, - // needHTTP01Listener, certManager, and getCertificate variables. - if acmeHostnamesCommas != "" { - acmeHostnames := strings.Split(acmeHostnamesCommas, ",") - log.Printf("ACME hostnames: %q", acmeHostnames) - - var cache autocert.Cache - if err = os.MkdirAll(acmeCertCacheDir, 0700); err != nil { - log.Printf("Warning: Couldn't create cache directory %q (reason: %s) so we're *not* using our certificate cache.", acmeCertCacheDir, err) - } else { - cache = autocert.DirCache(acmeCertCacheDir) - } - - certManager := autocert.Manager{ - Cache: cache, - Prompt: autocert.AcceptTOS, - HostPolicy: autocert.HostWhitelist(acmeHostnames...), - Email: acmeEmail, - } - go func() { - log.Printf("Starting HTTP-01 listener") - log.Fatal(http.ListenAndServe(":80", certManager.HTTPHandler(nil))) - }() - - server.TLSConfig = &tls.Config{GetCertificate: certManager.GetCertificate} - err = server.ListenAndServeTLS("", "") - } else if certFilename != "" && keyFilename != "" { - if acmeEmail != "" || acmeHostnamesCommas != "" { - log.Fatalf("The --cert and --key options are not allowed with --acme-email or --acme-hostnames.") - } - err = server.ListenAndServeTLS(certFilename, keyFilename) - } else if disableTLS { - err = server.ListenAndServe() - } else { - log.Fatal("the --acme-hostnames, --cert and --key, or --disable-tls option is required") - } - - if err != nil { - log.Fatal(err) - } -} diff --git a/broker/geoip.go b/broker/geoip.go deleted file mode 100644 index 708cdad..0000000 --- a/broker/geoip.go +++ /dev/null @@ -1,240 +0,0 @@ -/* -This code is for loading database data that maps ip addresses to countries -for collecting and presenting statistics on snowflake use that might alert us -to censorship events. - -The functions here are heavily based off of how tor maintains and searches their -geoip database - -The tables used for geoip data must be structured as follows: - -Recognized line format for IPv4 is: - INTIPLOW,INTIPHIGH,CC - where INTIPLOW and INTIPHIGH are IPv4 addresses encoded as big-endian 4-byte unsigned - integers, and CC is a country code. - -Note that the IPv4 line format - "INTIPLOW","INTIPHIGH","CC","CC3","COUNTRY NAME" -is not currently supported. - -Recognized line format for IPv6 is: - IPV6LOW,IPV6HIGH,CC - where IPV6LOW and IPV6HIGH are IPv6 addresses and CC is a country code. - -It also recognizes, and skips over, blank lines and lines that start -with '#' (comments). - -*/ -package main - -import ( - "bufio" - "bytes" - "crypto/sha1" - "encoding/hex" - "fmt" - "io" - "log" - "net" - "os" - "sort" - "strconv" - "strings" - "sync" -) - -type GeoIPTable interface { - parseEntry(string) (*GeoIPEntry, error) - Len() int - Append(GeoIPEntry) - ElementAt(int) GeoIPEntry - Lock() - Unlock() -} - -type GeoIPEntry struct { - ipLow net.IP - ipHigh net.IP - country string -} - -type GeoIPv4Table struct { - table []GeoIPEntry - - lock sync.Mutex // synchronization for geoip table accesses and reloads -} - -type GeoIPv6Table struct { - table []GeoIPEntry - - lock sync.Mutex // synchronization for geoip table accesses and reloads -} - -func (table *GeoIPv4Table) Len() int { return len(table.table) } -func (table *GeoIPv6Table) Len() int { return len(table.table) } - -func (table *GeoIPv4Table) Append(entry GeoIPEntry) { - (*table).table = append(table.table, entry) -} -func (table *GeoIPv6Table) Append(entry GeoIPEntry) { - (*table).table = append(table.table, entry) -} - -func (table *GeoIPv4Table) ElementAt(i int) GeoIPEntry { return table.table[i] } -func (table *GeoIPv6Table) ElementAt(i int) GeoIPEntry { return table.table[i] } - -func (table *GeoIPv4Table) Lock() { (*table).lock.Lock() } -func (table *GeoIPv6Table) Lock() { (*table).lock.Lock() } - -func (table *GeoIPv4Table) Unlock() { (*table).lock.Unlock() } -func (table *GeoIPv6Table) Unlock() { (*table).lock.Unlock() } - -// Convert a geoip IP address represented as a big-endian unsigned integer to net.IP -func geoipStringToIP(ipStr string) (net.IP, error) { - ip, err := strconv.ParseUint(ipStr, 10, 32) - if err != nil { - return net.IPv4(0, 0, 0, 0), fmt.Errorf("error parsing IP %s", ipStr) - } - var bytes [4]byte - bytes[0] = byte(ip & 0xFF) - bytes[1] = byte((ip >> 8) & 0xFF) - bytes[2] = byte((ip >> 16) & 0xFF) - bytes[3] = byte((ip >> 24) & 0xFF) - - return net.IPv4(bytes[3], bytes[2], bytes[1], bytes[0]), nil -} - -//Parses a line in the provided geoip file that corresponds -//to an address range and a two character country code -func (table *GeoIPv4Table) parseEntry(candidate string) (*GeoIPEntry, error) { - - if candidate[0] == '#' { - return nil, nil - } - - parsedCandidate := strings.Split(candidate, ",") - - if len(parsedCandidate) != 3 { - return nil, fmt.Errorf("provided geoip file is incorrectly formatted. Could not parse line:\n%s", parsedCandidate) - } - - low, err := geoipStringToIP(parsedCandidate[0]) - if err != nil { - return nil, err - } - high, err := geoipStringToIP(parsedCandidate[1]) - if err != nil { - return nil, err - } - - geoipEntry := &GeoIPEntry{ - ipLow: low, - ipHigh: high, - country: parsedCandidate[2], - } - - return geoipEntry, nil -} - -//Parses a line in the provided geoip file that corresponds -//to an address range and a two character country code -func (table *GeoIPv6Table) parseEntry(candidate string) (*GeoIPEntry, error) { - - if candidate[0] == '#' { - return nil, nil - } - - parsedCandidate := strings.Split(candidate, ",") - - if len(parsedCandidate) != 3 { - return nil, fmt.Errorf("") - } - - low := net.ParseIP(parsedCandidate[0]) - if low == nil { - return nil, fmt.Errorf("") - } - high := net.ParseIP(parsedCandidate[1]) - if high == nil { - return nil, fmt.Errorf("") - } - - geoipEntry := &GeoIPEntry{ - ipLow: low, - ipHigh: high, - country: parsedCandidate[2], - } - - return geoipEntry, nil -} - -//Loads provided geoip file into our tables -//Entries are stored in a table -func GeoIPLoadFile(table GeoIPTable, pathname string) error { - //open file - geoipFile, err := os.Open(pathname) - if err != nil { - return err - } - defer geoipFile.Close() - - hash := sha1.New() - - table.Lock() - defer table.Unlock() - - hashedFile := io.TeeReader(geoipFile, hash) - - //read in strings and call parse function - scanner := bufio.NewScanner(hashedFile) - for scanner.Scan() { - entry, err := table.parseEntry(scanner.Text()) - if err != nil { - return fmt.Errorf("provided geoip file is incorrectly formatted. Line is: %+q", scanner.Text()) - } - - if entry != nil { - table.Append(*entry) - } - - } - if err := scanner.Err(); err != nil { - return err - } - - sha1Hash := hex.EncodeToString(hash.Sum(nil)) - log.Println("Using geoip file ", pathname, " with checksum", sha1Hash) - log.Println("Loaded ", table.Len(), " entries into table") - - return nil -} - -//Returns the country location of an IPv4 or IPv6 address, and a boolean value -//that indicates whether the IP address was present in the geoip database -func GetCountryByAddr(table GeoIPTable, ip net.IP) (string, bool) { - - table.Lock() - defer table.Unlock() - - //look IP up in database - index := sort.Search(table.Len(), func(i int) bool { - entry := table.ElementAt(i) - return (bytes.Compare(ip.To16(), entry.ipHigh.To16()) <= 0) - }) - - if index == table.Len() { - return "", false - } - - // check to see if addr is in the range specified by the returned index - // search on IPs in invalid ranges (e.g., 127.0.0.0/8) will return the - //country code of the next highest range - entry := table.ElementAt(index) - if !(bytes.Compare(ip.To16(), entry.ipLow.To16()) >= 0 && - bytes.Compare(ip.To16(), entry.ipHigh.To16()) <= 0) { - return "", false - } - - return table.ElementAt(index).country, true - -} diff --git a/broker/metrics.go b/broker/metrics.go deleted file mode 100644 index ea4d220..0000000 --- a/broker/metrics.go +++ /dev/null @@ -1,196 +0,0 @@ -/* -We export metrics in the format specified in our broker spec: -https://gitweb.torproject.org/pluggable-transports/snowflake.git/tree/doc/broker-spec.txt -*/ - -package main - -import ( - "fmt" - "log" - "math" - "net" - "sync" - "time" -) - -var ( - once sync.Once -) - -const metricsResolution = 60 * 60 * 24 * time.Second //86400 seconds - -type CountryStats struct { - standalone map[string]bool - badge map[string]bool - webext map[string]bool - unknown map[string]bool - counts map[string]int -} - -// Implements Observable -type Metrics struct { - logger *log.Logger - tablev4 *GeoIPv4Table - tablev6 *GeoIPv6Table - - countryStats CountryStats - clientRoundtripEstimate time.Duration - proxyIdleCount uint - clientDeniedCount uint - clientProxyMatchCount uint - - //synchronization for access to snowflake metrics - lock sync.Mutex -} - -func (s CountryStats) Display() string { - output := "" - for cc, count := range s.counts { - output += fmt.Sprintf("%s=%d,", cc, count) - } - - // cut off trailing "," - if len(output) > 0 { - return output[:len(output)-1] - } - - return output -} - -func (m *Metrics) UpdateCountryStats(addr string, proxyType string) { - - var country string - var ok bool - - if proxyType == "standalone" { - if m.countryStats.standalone[addr] { - return - } - } else if proxyType == "badge" { - if m.countryStats.badge[addr] { - return - } - } else if proxyType == "webext" { - if m.countryStats.webext[addr] { - return - } - } else { - if m.countryStats.unknown[addr] { - return - } - } - - ip := net.ParseIP(addr) - if ip.To4() != nil { - //This is an IPv4 address - if m.tablev4 == nil { - return - } - country, ok = GetCountryByAddr(m.tablev4, ip) - } else { - if m.tablev6 == nil { - return - } - country, ok = GetCountryByAddr(m.tablev6, ip) - } - - if !ok { - country = "??" - } - - //update map of unique ips and counts - m.countryStats.counts[country]++ - if proxyType == "standalone" { - m.countryStats.standalone[addr] = true - } else if proxyType == "badge" { - m.countryStats.badge[addr] = true - } else if proxyType == "webext" { - m.countryStats.webext[addr] = true - } else { - m.countryStats.unknown[addr] = true - } - -} - -func (m *Metrics) LoadGeoipDatabases(geoipDB string, geoip6DB string) error { - - // Load geoip databases - log.Println("Loading geoip databases") - tablev4 := new(GeoIPv4Table) - err := GeoIPLoadFile(tablev4, geoipDB) - if err != nil { - m.tablev4 = nil - return err - } - m.tablev4 = tablev4 - - tablev6 := new(GeoIPv6Table) - err = GeoIPLoadFile(tablev6, geoip6DB) - if err != nil { - m.tablev6 = nil - return err - } - m.tablev6 = tablev6 - return nil -} - -func NewMetrics(metricsLogger *log.Logger) (*Metrics, error) { - m := new(Metrics) - - m.countryStats = CountryStats{ - counts: make(map[string]int), - standalone: make(map[string]bool), - badge: make(map[string]bool), - webext: make(map[string]bool), - unknown: make(map[string]bool), - } - - m.logger = metricsLogger - - // Write to log file every hour with updated metrics - go once.Do(m.logMetrics) - - return m, nil -} - -// Logs metrics in intervals specified by metricsResolution -func (m *Metrics) logMetrics() { - heartbeat := time.Tick(metricsResolution) - for range heartbeat { - m.printMetrics() - m.zeroMetrics() - } -} - -func (m *Metrics) printMetrics() { - m.lock.Lock() - m.logger.Println("snowflake-stats-end", time.Now().UTC().Format("2006-01-02 15:04:05"), fmt.Sprintf("(%d s)", int(metricsResolution.Seconds()))) - m.logger.Println("snowflake-ips", m.countryStats.Display()) - m.logger.Println("snowflake-ips-total", len(m.countryStats.standalone)+ - len(m.countryStats.badge)+len(m.countryStats.webext)+len(m.countryStats.unknown)) - m.logger.Println("snowflake-ips-standalone", len(m.countryStats.standalone)) - m.logger.Println("snowflake-ips-badge", len(m.countryStats.badge)) - m.logger.Println("snowflake-ips-webext", len(m.countryStats.webext)) - m.logger.Println("snowflake-idle-count", binCount(m.proxyIdleCount)) - m.logger.Println("client-denied-count", binCount(m.clientDeniedCount)) - m.logger.Println("client-snowflake-match-count", binCount(m.clientProxyMatchCount)) - m.lock.Unlock() -} - -// Restores all metrics to original values -func (m *Metrics) zeroMetrics() { - m.proxyIdleCount = 0 - m.clientDeniedCount = 0 - m.clientProxyMatchCount = 0 - m.countryStats.counts = make(map[string]int) - m.countryStats.standalone = make(map[string]bool) - m.countryStats.badge = make(map[string]bool) - m.countryStats.webext = make(map[string]bool) - m.countryStats.unknown = make(map[string]bool) -} - -// Rounds up a count to the nearest multiple of 8. -func binCount(count uint) uint { - return uint((math.Ceil(float64(count) / 8)) * 8) -} diff --git a/broker/snowflake-broker_test.go b/broker/snowflake-broker_test.go deleted file mode 100644 index 18b83dd..0000000 --- a/broker/snowflake-broker_test.go +++ /dev/null @@ -1,611 +0,0 @@ -package main - -import ( - "bytes" - "container/heap" - "io/ioutil" - "log" - "net" - "net/http" - "net/http/httptest" - "os" - "testing" - "time" - - . "github.com/smartystreets/goconvey/convey" -) - -func NullLogger() *log.Logger { - logger := log.New(os.Stdout, "", 0) - logger.SetOutput(ioutil.Discard) - return logger -} - -func TestBroker(t *testing.T) { - - Convey("Context", t, func() { - ctx := NewBrokerContext(NullLogger()) - - Convey("Adds Snowflake", func() { - So(ctx.snowflakes.Len(), ShouldEqual, 0) - So(len(ctx.idToSnowflake), ShouldEqual, 0) - ctx.AddSnowflake("foo", "") - So(ctx.snowflakes.Len(), ShouldEqual, 1) - So(len(ctx.idToSnowflake), ShouldEqual, 1) - }) - - Convey("Broker goroutine matches clients with proxies", func() { - p := new(ProxyPoll) - p.id = "test" - p.offerChannel = make(chan []byte) - go func(ctx *BrokerContext) { - ctx.proxyPolls <- p - close(ctx.proxyPolls) - }(ctx) - ctx.Broker() - So(ctx.snowflakes.Len(), ShouldEqual, 1) - snowflake := heap.Pop(ctx.snowflakes).(*Snowflake) - snowflake.offerChannel <- []byte("test offer") - offer := <-p.offerChannel - So(ctx.idToSnowflake["test"], ShouldNotBeNil) - So(offer, ShouldResemble, []byte("test offer")) - So(ctx.snowflakes.Len(), ShouldEqual, 0) - }) - - Convey("Request an offer from the Snowflake Heap", func() { - done := make(chan []byte) - go func() { - offer := ctx.RequestOffer("test", "") - done <- offer - }() - request := <-ctx.proxyPolls - request.offerChannel <- []byte("test offer") - offer := <-done - So(offer, ShouldResemble, []byte("test offer")) - }) - - Convey("Responds to client offers...", func() { - w := httptest.NewRecorder() - data := bytes.NewReader([]byte("test")) - r, err := http.NewRequest("POST", "snowflake.broker/client", data) - So(err, ShouldBeNil) - - Convey("with 503 when no snowflakes are available.", func() { - clientOffers(ctx, w, r) - So(w.Code, ShouldEqual, http.StatusServiceUnavailable) - So(w.Body.String(), ShouldEqual, "") - }) - - Convey("with a proxy answer if available.", func() { - done := make(chan bool) - // Prepare a fake proxy to respond with. - snowflake := ctx.AddSnowflake("fake", "") - go func() { - clientOffers(ctx, w, r) - done <- true - }() - offer := <-snowflake.offerChannel - So(offer, ShouldResemble, []byte("test")) - snowflake.answerChannel <- []byte("fake answer") - <-done - So(w.Body.String(), ShouldEqual, "fake answer") - So(w.Code, ShouldEqual, http.StatusOK) - }) - - Convey("Times out when no proxy responds.", func() { - if testing.Short() { - return - } - done := make(chan bool) - snowflake := ctx.AddSnowflake("fake", "") - go func() { - clientOffers(ctx, w, r) - // Takes a few seconds here... - done <- true - }() - offer := <-snowflake.offerChannel - So(offer, ShouldResemble, []byte("test")) - <-done - So(w.Code, ShouldEqual, http.StatusGatewayTimeout) - }) - }) - - Convey("Responds to proxy polls...", func() { - done := make(chan bool) - w := httptest.NewRecorder() - data := bytes.NewReader([]byte(`{"Sid":"ymbcCMto7KHNGYlp","Version":"1.0"}`)) - r, err := http.NewRequest("POST", "snowflake.broker/proxy", data) - So(err, ShouldBeNil) - - Convey("with a client offer if available.", func() { - go func(ctx *BrokerContext) { - proxyPolls(ctx, w, r) - done <- true - }(ctx) - // Pass a fake client offer to this proxy - p := <-ctx.proxyPolls - So(p.id, ShouldEqual, "ymbcCMto7KHNGYlp") - p.offerChannel <- []byte("fake offer") - <-done - So(w.Code, ShouldEqual, http.StatusOK) - So(w.Body.String(), ShouldEqual, `{"Status":"client match","Offer":"fake offer"}`) - }) - - Convey("return empty 200 OK when no client offer is available.", func() { - go func(ctx *BrokerContext) { - proxyPolls(ctx, w, r) - done <- true - }(ctx) - p := <-ctx.proxyPolls - So(p.id, ShouldEqual, "ymbcCMto7KHNGYlp") - // nil means timeout - p.offerChannel <- nil - <-done - So(w.Body.String(), ShouldEqual, `{"Status":"no match","Offer":""}`) - So(w.Code, ShouldEqual, http.StatusOK) - }) - }) - - Convey("Responds to proxy answers...", func() { - s := ctx.AddSnowflake("test", "") - w := httptest.NewRecorder() - data := bytes.NewReader([]byte(`{"Version":"1.0","Sid":"test","Answer":"test"}`)) - - Convey("by passing to the client if valid.", func() { - r, err := http.NewRequest("POST", "snowflake.broker/answer", data) - So(err, ShouldBeNil) - go func(ctx *BrokerContext) { - proxyAnswers(ctx, w, r) - }(ctx) - answer := <-s.answerChannel - So(w.Code, ShouldEqual, http.StatusOK) - So(answer, ShouldResemble, []byte("test")) - }) - - Convey("with client gone status if the proxy is not recognized", func() { - data = bytes.NewReader([]byte(`{"Version":"1.0","Sid":"invalid","Answer":"test"}`)) - r, err := http.NewRequest("POST", "snowflake.broker/answer", data) - So(err, ShouldBeNil) - proxyAnswers(ctx, w, r) - So(w.Code, ShouldEqual, http.StatusOK) - b, err := ioutil.ReadAll(w.Body) - So(err, ShouldBeNil) - So(b, ShouldResemble, []byte(`{"Status":"client gone"}`)) - - }) - - Convey("with error if the proxy gives invalid answer", func() { - data := bytes.NewReader(nil) - r, err := http.NewRequest("POST", "snowflake.broker/answer", data) - So(err, ShouldBeNil) - proxyAnswers(ctx, w, r) - So(w.Code, ShouldEqual, http.StatusBadRequest) - }) - - Convey("with error if the proxy writes too much data", func() { - data := bytes.NewReader(make([]byte, 100001)) - r, err := http.NewRequest("POST", "snowflake.broker/answer", data) - So(err, ShouldBeNil) - proxyAnswers(ctx, w, r) - So(w.Code, ShouldEqual, http.StatusBadRequest) - }) - - }) - - }) - - Convey("End-To-End", t, func() { - ctx := NewBrokerContext(NullLogger()) - - Convey("Check for client/proxy data race", func() { - proxy_done := make(chan bool) - client_done := make(chan bool) - - go ctx.Broker() - - // Make proxy poll - wp := httptest.NewRecorder() - datap := bytes.NewReader([]byte(`{"Sid":"ymbcCMto7KHNGYlp","Version":"1.0"}`)) - rp, err := http.NewRequest("POST", "snowflake.broker/proxy", datap) - So(err, ShouldBeNil) - - go func(ctx *BrokerContext) { - proxyPolls(ctx, wp, rp) - proxy_done <- true - }(ctx) - - // Client offer - wc := httptest.NewRecorder() - datac := bytes.NewReader([]byte("test")) - rc, err := http.NewRequest("POST", "snowflake.broker/client", datac) - So(err, ShouldBeNil) - - go func() { - clientOffers(ctx, wc, rc) - client_done <- true - }() - - <-proxy_done - So(wp.Code, ShouldEqual, http.StatusOK) - - // Proxy answers - wp = httptest.NewRecorder() - datap = bytes.NewReader([]byte(`{"Version":"1.0","Sid":"ymbcCMto7KHNGYlp","Answer":"test"}`)) - rp, err = http.NewRequest("POST", "snowflake.broker/answer", datap) - So(err, ShouldBeNil) - go func(ctx *BrokerContext) { - proxyAnswers(ctx, wp, rp) - proxy_done <- true - }(ctx) - - <-proxy_done - <-client_done - - }) - - Convey("Ensure correct snowflake brokering", func() { - done := make(chan bool) - polled := make(chan bool) - - // Proxy polls with its ID first... - dataP := bytes.NewReader([]byte(`{"Sid":"ymbcCMto7KHNGYlp","Version":"1.0"}`)) - wP := httptest.NewRecorder() - rP, err := http.NewRequest("POST", "snowflake.broker/proxy", dataP) - So(err, ShouldBeNil) - go func() { - proxyPolls(ctx, wP, rP) - polled <- true - }() - - // Manually do the Broker goroutine action here for full control. - p := <-ctx.proxyPolls - So(p.id, ShouldEqual, "ymbcCMto7KHNGYlp") - s := ctx.AddSnowflake(p.id, "") - go func() { - offer := <-s.offerChannel - p.offerChannel <- offer - }() - So(ctx.idToSnowflake["ymbcCMto7KHNGYlp"], ShouldNotBeNil) - - // Client request blocks until proxy answer arrives. - dataC := bytes.NewReader([]byte("fake offer")) - wC := httptest.NewRecorder() - rC, err := http.NewRequest("POST", "snowflake.broker/client", dataC) - So(err, ShouldBeNil) - go func() { - clientOffers(ctx, wC, rC) - done <- true - }() - - <-polled - So(wP.Code, ShouldEqual, http.StatusOK) - So(wP.Body.String(), ShouldResemble, `{"Status":"client match","Offer":"fake offer"}`) - So(ctx.idToSnowflake["ymbcCMto7KHNGYlp"], ShouldNotBeNil) - // Follow up with the answer request afterwards - wA := httptest.NewRecorder() - dataA := bytes.NewReader([]byte(`{"Version":"1.0","Sid":"ymbcCMto7KHNGYlp","Answer":"test"}`)) - rA, err := http.NewRequest("POST", "snowflake.broker/answer", dataA) - So(err, ShouldBeNil) - proxyAnswers(ctx, wA, rA) - So(wA.Code, ShouldEqual, http.StatusOK) - - <-done - So(wC.Code, ShouldEqual, http.StatusOK) - So(wC.Body.String(), ShouldEqual, "test") - }) - }) -} - -func TestSnowflakeHeap(t *testing.T) { - Convey("SnowflakeHeap", t, func() { - h := new(SnowflakeHeap) - heap.Init(h) - So(h.Len(), ShouldEqual, 0) - s1 := new(Snowflake) - s2 := new(Snowflake) - s3 := new(Snowflake) - s4 := new(Snowflake) - s1.clients = 4 - s2.clients = 5 - s3.clients = 3 - s4.clients = 1 - - heap.Push(h, s1) - So(h.Len(), ShouldEqual, 1) - heap.Push(h, s2) - So(h.Len(), ShouldEqual, 2) - heap.Push(h, s3) - So(h.Len(), ShouldEqual, 3) - heap.Push(h, s4) - So(h.Len(), ShouldEqual, 4) - - heap.Remove(h, 0) - So(h.Len(), ShouldEqual, 3) - - r := heap.Pop(h).(*Snowflake) - So(h.Len(), ShouldEqual, 2) - So(r.clients, ShouldEqual, 3) - So(r.index, ShouldEqual, -1) - - r = heap.Pop(h).(*Snowflake) - So(h.Len(), ShouldEqual, 1) - So(r.clients, ShouldEqual, 4) - So(r.index, ShouldEqual, -1) - - r = heap.Pop(h).(*Snowflake) - So(h.Len(), ShouldEqual, 0) - So(r.clients, ShouldEqual, 5) - So(r.index, ShouldEqual, -1) - }) -} - -func TestGeoip(t *testing.T) { - Convey("Geoip", t, func() { - tv4 := new(GeoIPv4Table) - err := GeoIPLoadFile(tv4, "test_geoip") - So(err, ShouldEqual, nil) - tv6 := new(GeoIPv6Table) - err = GeoIPLoadFile(tv6, "test_geoip6") - So(err, ShouldEqual, nil) - - Convey("IPv4 Country Mapping Tests", func() { - for _, test := range []struct { - addr, cc string - ok bool - }{ - { - "129.97.208.23", //uwaterloo - "CA", - true, - }, - { - "127.0.0.1", - "", - false, - }, - { - "255.255.255.255", - "", - false, - }, - { - "0.0.0.0", - "", - false, - }, - { - "223.252.127.255", //test high end of range - "JP", - true, - }, - { - "223.252.127.255", //test low end of range - "JP", - true, - }, - } { - country, ok := GetCountryByAddr(tv4, net.ParseIP(test.addr)) - So(country, ShouldEqual, test.cc) - So(ok, ShouldResemble, test.ok) - } - }) - - Convey("IPv6 Country Mapping Tests", func() { - for _, test := range []struct { - addr, cc string - ok bool - }{ - { - "2620:101:f000:0:250:56ff:fe80:168e", //uwaterloo - "CA", - true, - }, - { - "fd00:0:0:0:0:0:0:1", - "", - false, - }, - { - "0:0:0:0:0:0:0:0", - "", - false, - }, - { - "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff", - "", - false, - }, - { - "2a07:2e47:ffff:ffff:ffff:ffff:ffff:ffff", //test high end of range - "FR", - true, - }, - { - "2a07:2e40::", //test low end of range - "FR", - true, - }, - } { - country, ok := GetCountryByAddr(tv6, net.ParseIP(test.addr)) - So(country, ShouldEqual, test.cc) - So(ok, ShouldResemble, test.ok) - } - }) - - // Make sure things behave properly if geoip file fails to load - ctx := NewBrokerContext(NullLogger()) - if err := ctx.metrics.LoadGeoipDatabases("invalid_filename", "invalid_filename6"); err != nil { - log.Printf("loading geo ip databases returned error: %v", err) - } - ctx.metrics.UpdateCountryStats("127.0.0.1", "") - So(ctx.metrics.tablev4, ShouldEqual, nil) - - }) -} - -func TestMetrics(t *testing.T) { - - Convey("Test metrics...", t, func() { - done := make(chan bool) - buf := new(bytes.Buffer) - ctx := NewBrokerContext(log.New(buf, "", 0)) - - err := ctx.metrics.LoadGeoipDatabases("test_geoip", "test_geoip6") - So(err, ShouldEqual, nil) - - //Test addition of proxy polls - Convey("for proxy polls", func() { - w := httptest.NewRecorder() - data := bytes.NewReader([]byte("{\"Sid\":\"ymbcCMto7KHNGYlp\",\"Version\":\"1.0\"}")) - r, err := http.NewRequest("POST", "snowflake.broker/proxy", data) - r.RemoteAddr = "129.97.208.23:8888" //CA geoip - So(err, ShouldBeNil) - go func(ctx *BrokerContext) { - proxyPolls(ctx, w, r) - done <- true - }(ctx) - p := <-ctx.proxyPolls //manually unblock poll - p.offerChannel <- nil - <-done - - w = httptest.NewRecorder() - data = bytes.NewReader([]byte(`{"Sid":"ymbcCMto7KHNGYlp","Version":"1.0","Type":"standalone"}`)) - r, err = http.NewRequest("POST", "snowflake.broker/proxy", data) - r.RemoteAddr = "129.97.208.23:8888" //CA geoip - So(err, ShouldBeNil) - go func(ctx *BrokerContext) { - proxyPolls(ctx, w, r) - done <- true - }(ctx) - p = <-ctx.proxyPolls //manually unblock poll - p.offerChannel <- nil - <-done - - w = httptest.NewRecorder() - data = bytes.NewReader([]byte(`{"Sid":"ymbcCMto7KHNGYlp","Version":"1.0","Type":"badge"}`)) - r, err = http.NewRequest("POST", "snowflake.broker/proxy", data) - r.RemoteAddr = "129.97.208.23:8888" //CA geoip - So(err, ShouldBeNil) - go func(ctx *BrokerContext) { - proxyPolls(ctx, w, r) - done <- true - }(ctx) - p = <-ctx.proxyPolls //manually unblock poll - p.offerChannel <- nil - <-done - - w = httptest.NewRecorder() - data = bytes.NewReader([]byte(`{"Sid":"ymbcCMto7KHNGYlp","Version":"1.0","Type":"webext"}`)) - r, err = http.NewRequest("POST", "snowflake.broker/proxy", data) - r.RemoteAddr = "129.97.208.23:8888" //CA geoip - So(err, ShouldBeNil) - go func(ctx *BrokerContext) { - proxyPolls(ctx, w, r) - done <- true - }(ctx) - p = <-ctx.proxyPolls //manually unblock poll - p.offerChannel <- nil - <-done - ctx.metrics.printMetrics() - So(buf.String(), ShouldResemble, "snowflake-stats-end "+time.Now().UTC().Format("2006-01-02 15:04:05")+" (86400 s)\nsnowflake-ips CA=4\nsnowflake-ips-total 4\nsnowflake-ips-standalone 1\nsnowflake-ips-badge 1\nsnowflake-ips-webext 1\nsnowflake-idle-count 8\nclient-denied-count 0\nclient-snowflake-match-count 0\n") - - }) - - //Test addition of client failures - Convey("for no proxies available", func() { - w := httptest.NewRecorder() - data := bytes.NewReader([]byte("test")) - r, err := http.NewRequest("POST", "snowflake.broker/client", data) - So(err, ShouldBeNil) - - clientOffers(ctx, w, r) - - ctx.metrics.printMetrics() - So(buf.String(), ShouldResemble, "snowflake-stats-end "+time.Now().UTC().Format("2006-01-02 15:04:05")+" (86400 s)\nsnowflake-ips \nsnowflake-ips-total 0\nsnowflake-ips-standalone 0\nsnowflake-ips-badge 0\nsnowflake-ips-webext 0\nsnowflake-idle-count 0\nclient-denied-count 8\nclient-snowflake-match-count 0\n") - - // Test reset - buf.Reset() - ctx.metrics.zeroMetrics() - ctx.metrics.printMetrics() - So(buf.String(), ShouldResemble, "snowflake-stats-end "+time.Now().UTC().Format("2006-01-02 15:04:05")+" (86400 s)\nsnowflake-ips \nsnowflake-ips-total 0\nsnowflake-ips-standalone 0\nsnowflake-ips-badge 0\nsnowflake-ips-webext 0\nsnowflake-idle-count 0\nclient-denied-count 0\nclient-snowflake-match-count 0\n") - }) - //Test addition of client matches - Convey("for client-proxy match", func() { - w := httptest.NewRecorder() - data := bytes.NewReader([]byte("test")) - r, err := http.NewRequest("POST", "snowflake.broker/client", data) - So(err, ShouldBeNil) - - // Prepare a fake proxy to respond with. - snowflake := ctx.AddSnowflake("fake", "") - go func() { - clientOffers(ctx, w, r) - done <- true - }() - offer := <-snowflake.offerChannel - So(offer, ShouldResemble, []byte("test")) - snowflake.answerChannel <- []byte("fake answer") - <-done - - ctx.metrics.printMetrics() - So(buf.String(), ShouldResemble, "snowflake-stats-end "+time.Now().UTC().Format("2006-01-02 15:04:05")+" (86400 s)\nsnowflake-ips \nsnowflake-ips-total 0\nsnowflake-ips-standalone 0\nsnowflake-ips-badge 0\nsnowflake-ips-webext 0\nsnowflake-idle-count 0\nclient-denied-count 0\nclient-snowflake-match-count 8\n") - }) - //Test rounding boundary - Convey("binning boundary", func() { - w := httptest.NewRecorder() - data := bytes.NewReader([]byte("test")) - r, err := http.NewRequest("POST", "snowflake.broker/client", data) - So(err, ShouldBeNil) - - clientOffers(ctx, w, r) - clientOffers(ctx, w, r) - clientOffers(ctx, w, r) - clientOffers(ctx, w, r) - clientOffers(ctx, w, r) - clientOffers(ctx, w, r) - clientOffers(ctx, w, r) - clientOffers(ctx, w, r) - - ctx.metrics.printMetrics() - So(buf.String(), ShouldResemble, "snowflake-stats-end "+time.Now().UTC().Format("2006-01-02 15:04:05")+" (86400 s)\nsnowflake-ips \nsnowflake-ips-total 0\nsnowflake-ips-standalone 0\nsnowflake-ips-badge 0\nsnowflake-ips-webext 0\nsnowflake-idle-count 0\nclient-denied-count 8\nclient-snowflake-match-count 0\n") - - clientOffers(ctx, w, r) - buf.Reset() - ctx.metrics.printMetrics() - So(buf.String(), ShouldResemble, "snowflake-stats-end "+time.Now().UTC().Format("2006-01-02 15:04:05")+" (86400 s)\nsnowflake-ips \nsnowflake-ips-total 0\nsnowflake-ips-standalone 0\nsnowflake-ips-badge 0\nsnowflake-ips-webext 0\nsnowflake-idle-count 0\nclient-denied-count 16\nclient-snowflake-match-count 0\n") - }) - - //Test unique ip - Convey("proxy counts by unique ip", func() { - w := httptest.NewRecorder() - data := bytes.NewReader([]byte(`{"Sid":"ymbcCMto7KHNGYlp","Version":"1.0"}`)) - r, err := http.NewRequest("POST", "snowflake.broker/proxy", data) - r.RemoteAddr = "129.97.208.23:8888" //CA geoip - So(err, ShouldBeNil) - go func(ctx *BrokerContext) { - proxyPolls(ctx, w, r) - done <- true - }(ctx) - p := <-ctx.proxyPolls //manually unblock poll - p.offerChannel <- nil - <-done - - data = bytes.NewReader([]byte(`{"Sid":"ymbcCMto7KHNGYlp","Version":"1.0"}`)) - r, err = http.NewRequest("POST", "snowflake.broker/proxy", data) - if err != nil { - log.Printf("unable to get NewRequest with error: %v", err) - } - r.RemoteAddr = "129.97.208.23:8888" //CA geoip - go func(ctx *BrokerContext) { - proxyPolls(ctx, w, r) - done <- true - }(ctx) - p = <-ctx.proxyPolls //manually unblock poll - p.offerChannel <- nil - <-done - - ctx.metrics.printMetrics() - So(buf.String(), ShouldResemble, "snowflake-stats-end "+time.Now().UTC().Format("2006-01-02 15:04:05")+" (86400 s)\nsnowflake-ips CA=1\nsnowflake-ips-total 1\nsnowflake-ips-standalone 0\nsnowflake-ips-badge 0\nsnowflake-ips-webext 0\nsnowflake-idle-count 8\nclient-denied-count 0\nclient-snowflake-match-count 0\n") - }) - }) -} diff --git a/broker/snowflake-heap.go b/broker/snowflake-heap.go deleted file mode 100644 index 19a64b2..0000000 --- a/broker/snowflake-heap.go +++ /dev/null @@ -1,51 +0,0 @@ -/* -Keeping track of pending available snowflake proxies. -*/ - -package main - -/* -The Snowflake struct contains a single interaction -over the offer and answer channels. -*/ -type Snowflake struct { - id string - proxyType string - offerChannel chan []byte - answerChannel chan []byte - clients int - index int -} - -// Implements heap.Interface, and holds Snowflakes. -type SnowflakeHeap []*Snowflake - -func (sh SnowflakeHeap) Len() int { return len(sh) } - -func (sh SnowflakeHeap) Less(i, j int) bool { - // Snowflakes serving less clients should sort earlier. - return sh[i].clients < sh[j].clients -} - -func (sh SnowflakeHeap) Swap(i, j int) { - sh[i], sh[j] = sh[j], sh[i] - sh[i].index = i - sh[j].index = j -} - -func (sh *SnowflakeHeap) Push(s interface{}) { - n := len(*sh) - snowflake := s.(*Snowflake) - snowflake.index = n - *sh = append(*sh, snowflake) -} - -// Only valid when Len() > 0. -func (sh *SnowflakeHeap) Pop() interface{} { - flakes := *sh - n := len(flakes) - snowflake := flakes[n-1] - snowflake.index = -1 - *sh = flakes[0 : n-1] - return snowflake -} diff --git a/broker/test_geoip b/broker/test_geoip deleted file mode 100644 index 885c8ab..0000000 --- a/broker/test_geoip +++ /dev/null @@ -1,1236 +0,0 @@ -# Last updated based on February 7 2018 Maxmind GeoLite2 Country -# wget https://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.mmdb.gz -# gunzip GeoLite2-Country.mmdb.gz -# python mmdb-convert.py GeoLite2-Country.mmdb -16777216,16777471,AU -16777472,16778239,CN -16778240,16779263,AU -16779264,16781311,CN -16781312,16785407,JP -16785408,16793599,CN -16793600,16809983,JP -16809984,16842751,TH -16842752,16843007,CN -16843008,16843263,AU -16843264,16859135,CN -16859136,16875519,JP -16875520,16908287,TH -16908288,16909055,CN -16909056,16909311,US -16909312,16941055,CN -16941056,16973823,TH -16973824,17039359,CN -17039360,17039615,AU -2111307776,2111832063,CN -2111832064,2112487423,TW -2112487424,2112618495,VN -2112618496,2112880639,NZ -2112880640,2113560063,KR -2113560064,2113560319,SG -2113560320,2113683455,KR -2113683456,2113684607,JP -2113684608,2113684671,TW -2113684672,2113685663,JP -2113685664,2113685695,SG -2113685696,2113687999,JP -2113688000,2113688031,AU -2113688032,2113688959,JP -2113688960,2113688991,SG -2113688992,2113691135,JP -2113691136,2113691391,SG -2113691392,2113692415,JP -2113692416,2113692671,HK -2113692672,2113693599,JP -2113693600,2113693615,HK -2113693616,2113693879,JP -2113693880,2113693887,AU -2113693888,2113693951,JP -2113693952,2113694207,HK -2113694208,2113695279,JP -2113695280,2113695287,SG -2113695288,2113716223,JP -2113716224,2113724927,SG -2113724928,2113725183,IN -2113725184,2113728511,SG -2113728512,2113732607,JP -2113732608,2113761279,AU -2113761280,2113765375,VN -2113765376,2113798143,HK -2113798144,2113811455,AU -2113811456,2113812479,GB -2113812480,2113813503,JP -2113813504,2113830911,AU -2113830912,2113863679,CN -2113863680,2113929215,AU -2113929216,2130706431,JP -2147483648,2147483903,NL -2147483904,2147484671,RO -2147484672,2147485695,TR -2147485696,2147487743,DK -2147487744,2147489791,NO -2147489792,2147491839,RU -2147491840,2147494911,DE -2147494912,2147495167,RO -2147495168,2147495423,DE -2147495424,2147496959,RO -2147496960,2147497215,ES -2147497216,2147497471,RO -2147497472,2147497727,PL -2147497728,2147498239,DE -2147498240,2147498495,RO -2147498496,2147500031,DE -2147500032,2147501055,NL -2147501056,2147501311,SK -2147501312,2147501567,NL -2147501568,2147501823,GL -2147501824,2147502079,US -2147502080,2147504127,DK -2147504128,2147508223,RU -2147508224,2147510271,DE -2147510272,2147510783,UA -2147510784,2147511039,RU -2147511040,2147512319,CY -2147512320,2147514879,DE -2147514880,2147516415,IT -2147516416,2147520511,RU -2147520512,2147524607,DE -2147524608,2147526655,RU -2147526656,2147528703,UA -2147528704,2147532799,CZ -2147532800,2147534847,DE -2147534848,2147549183,CY -2147549184,2147557375,US -2147557376,2147557631,TW -2147557632,2147557887,SG -2147557888,2147558143,DE -2147558144,2147558399,TH -2147558400,2147558655,KR -2147558656,2147558911,TW -2147558912,2147559167,SG -2147559168,2147559423,TH -2147559424,2147559679,SG -2147559680,2147559935,US -2147559936,2147560191,DE -2147560192,2147560447,RU -2147560448,2147560703,TH -2147560704,2147560959,TW -2147560960,2147562239,US -2147562240,2147562495,RU -2147562496,2147563263,US -2147563264,2147563519,RU -2147563520,2147564287,US -2147564288,2147564543,AE -2147564544,2147564799,US -2147564800,2147565055,SG -2147565056,2147565311,HK -2147565312,2147565999,TW -2147566000,2147566047,JP -2147566048,2147566079,TW -2147566080,2147569407,US -2147569408,2147569663,TH -2147569664,2147570431,US -2147570432,2147570687,JP -2147570688,2147571455,US -2147571456,2147571711,SG -2147571712,2147573503,US -2147573504,2147573759,SG -2147573760,2147575039,US -2147575040,2147575551,TW -2147575552,2147575807,SG -2147575808,2147576575,US -2147576576,2147576831,TW -2147576832,2147577087,TH -2147577088,2147577599,ID -2147577600,2147579647,US -2147579648,2147579903,ID -2147579904,2147580927,US -2147580928,2147581183,ID -2147581184,2147581439,TH -2147581440,2147592703,US -2147592704,2147592959,HK -2147592960,2147600127,US -2147600128,2147600383,SG -2147600384,2147603711,US -2147603712,2147603967,IN -2147603968,2147942399,US -2147942400,2148007935,DE -2148007936,2148220515,US -2148220516,2148220535,AU -2148220536,2148229151,US -2148229152,2148229183,CA -2148229184,2148459007,US -2148459008,2148459519,TW -2148459520,2148532223,US -2148532224,2148597759,GB -2148597760,2148925439,US -2148925440,2148990975,JP -2148990976,2149253119,US -2149253120,2149384191,JP -2149384192,2150039551,US -2150039552,2150105087,NO -2150105088,2150203391,GB -2150203392,2150236159,AF -2150236160,2150301695,US -2150301696,2150367231,CA -2150367232,2150432767,US -2150432768,2150498303,IT -2150498304,2150957055,US -2150957056,2151022591,JP -2151022592,2151743487,US -2151743488,2151759871,BY -2151759872,2151768063,US -2151768064,2151770111,GB -2151770112,2151772159,BA -2151772160,2151776255,IT -2151776256,2151778303,AT -2151778304,2151780351,RU -2151780352,2151782399,DE -2151782400,2151784447,ES -2151784448,2151792639,IR -2151792640,2151794687,CH -2151794688,2151796735,IT -2151796736,2151800831,DE -2151800832,2151809023,PT -2151809024,2151940095,IT -2151940096,2152464383,RU -2152464384,2152529919,DK -2152529920,2152562687,NO -2152562688,2152595455,DK -2152595456,2152726527,FR -2152726528,2153119743,US -2153119744,2153185279,GB -2153185280,2153250815,SE -2153250816,2153381887,US -2153381888,2153382143,JP -2153382144,2153383679,US -2153383680,2153383935,HK -2153383936,2153384447,US -2153384448,2153385471,GB -2153385472,2153385599,AT -2153385600,2153385663,CZ -2153385664,2153385727,FI -2153385728,2153385791,PL -2153385792,2153385855,PT -2153385856,2153385919,TR -2153385920,2153385983,US -2153385984,2153387007,GB -2153387008,2153387263,CH -2153387264,2153387519,IS -2153387520,2153387775,IE -2153387776,2153388031,CH -2153388032,2153388287,ES -2153388288,2153388543,PL -2153388544,2153391615,US -2153391616,2153391871,HK -2153391872,2153394431,US -2153394432,2153394943,SG -2153394944,2153395455,US -2153395456,2153395711,VN -2153395712,2153396991,US -2153396992,2153397247,IL -2153397248,2153397503,IN -2153397504,2153397759,SA -2153397760,2153398015,QA -2153398016,2153398271,BH -2153398272,2153398783,JP -2153398784,2153399551,US -2153399552,2153399807,KR -2153399808,2153400319,HK -2153400320,2153401087,TW -2153401088,2153401599,MO -2153401600,2153402111,VN -2153402112,2153402367,PH -2153402368,2153403135,KR -2153403136,2153406463,US -2153406464,2153407487,JP -2153407488,2153407743,HK -2153407744,2153407999,AE -2153408000,2153408511,BR -2153408512,2153408767,AU -2153408768,2153409023,PA -2153409024,2153409279,AR -2153409280,2153409535,CR -2153409536,2153409791,CO -2153409792,2153410047,MX -2153410048,2153410303,CA -2153410304,2153410559,TW -2153410560,2153410815,PA -2153410816,2153411071,AR -2153411072,2153411327,CR -2153411328,2153411583,CO -2153411584,2153411839,MX -2153411840,2153412095,SV -2153412096,2153412351,TW -2153412352,2153412607,UY -2153412608,2153413119,AU -2153413120,2153413631,BR -2153413632,2153578495,US -2153578496,2153644031,FR -2153644032,2153906175,US -2153906176,2153971711,GB -2153971712,2154037247,US -2154037248,2154102783,CA -2154102784,2154430463,US -2154430464,2154495999,SG -2154496000,2154561535,US -2154561536,2154627071,CN -2154627072,2155610111,US -2155610112,2155675647,UA -2155675648,2155806719,US -2155806720,2155808767,IT -2155810816,2155812863,FR -2155812864,2155814911,GB -2155814912,2155819007,NL -2155819008,2155819519,DE -2155819520,2155821055,CH -2155821056,2155823103,IT -2155823104,2155825151,DE -2155825152,2155827199,AE -2155827200,2155831295,PL -2155831296,2155833343,RU -2155833344,2155833855,SE -2155833856,2155834623,NL -2155834624,2155834879,LU -2155834880,2155835391,NL -2155835392,2155839487,RO -2155839488,2155843583,FR -2155843584,2155845631,RU -2155845632,2155847679,DE -2155847680,2155849727,ES -2155849728,2155851775,TR -2155853824,2155855871,SE -2155855872,2155872255,SA -2155872256,2156003327,US -2156003328,2156134399,AT -2156134400,2156265471,US -2156265472,2156331007,KR -2156331008,2156593151,US -2156593152,2156658687,IL -2156658688,2156691455,IR -2156691456,2156697599,FR -2156697600,2156699647,GR -2156699648,2156703743,RU -2156703744,2156707839,BG -2156707840,2156709887,RU -2156709888,2156711935,ES -2156711936,2156713983,DE -2156713984,2156716031,NL -2156716032,2156718079,RO -2156718080,2156720127,IS -2156720128,2156724223,BY -2156724224,2156855295,CH -2156855296,2156920831,US -2156920832,2156986367,CA -2156986368,2159017983,US -2159017984,2159083519,DE -2159083520,2159149055,US -2159149056,2159280127,CH -2159280128,2159542271,US -2159542272,2159607807,AU -2159607808,2159673343,IN -2159673344,2159869951,US -2159869952,2159935487,CA -2159935488,2160525311,US -2160525312,2160533503,SG -2160533504,2160541695,NL -2160541696,2160590847,SG -2160590848,2160656383,US -2160656384,2160657407,BR -2160657408,2160658431,HN -2160658432,2160661503,BR -2160661504,2160662527,AR -2160662528,2160664575,BR -2160664576,2160666623,CL -2160666624,2160676863,BR -2160676864,2160677887,AR -2160677888,2160678911,BR -2160678912,2160679935,GF -2160679936,2160684031,BR -2160684032,2160685055,AR -2160685056,2160686079,DO -2160686080,2160687103,CL -2160687104,2160690175,BR -2160690176,2160691199,AR -2160691200,2160693247,BR -2160693248,2160694271,CR -2160694272,2160697343,BR -2160697344,2160698367,EC -2160698368,2160699391,BR -2160699392,2160700415,AR -2160700416,2160713727,BR -2160713728,2160714751,CL -2160714752,2160716799,BR -2160716800,2160717823,AR -2160717824,2160721919,BR -2160721920,2160852991,US -2160852992,2160885759,RU -2160885760,2160893951,AT -2160893952,2160902143,RU -2160902144,2160906239,NL -2160906240,2160908287,FR -2160908288,2160910335,PL -2160910336,2160914431,NL -2160914432,2160918527,SA -2160918528,2161508351,US -2161508352,2161573887,FI -2161573888,2162687999,US -2162688000,2162753535,GB -2162753536,2162819071,CA -2162819072,2162884607,SA -2162884608,2163212287,US -2163212288,2163277823,GB -2163277824,2163408895,US -2163408896,2163474431,GB -2163474432,2163605503,US -2163605504,2163638271,DE -2163638272,2163638527,US -2163638528,2163671039,DE -2163671040,2163867647,US -2163867648,2163933183,AU -2163933184,2164260863,US -2164260864,2164326399,CM -2164326400,2164981759,US -2164981760,2165112831,GB -2165112832,2165178367,DE -2165178368,2165309439,US -2165309440,2165374975,SE -2165374976,2165440511,US -2165440512,2165506047,NG -2165506048,2165571583,US -2165571584,2165637119,FR -2165637120,2165964799,US -2165964800,2166030335,DE -2166030336,2166095871,AT -2166095872,2166161407,CN -2166161408,2166292479,US -2166292480,2166358015,GB -2166358016,2166562559,US -2166562560,2166562815,FI -2166562816,2166571007,US -2166571008,2166575103,GB -2166575104,2166594559,US -2166594560,2166594815,PL -2166594816,2166729471,US -2166729472,2166729727,CA -2166729728,2167209983,US -2167209984,2167242751,DZ -2167242752,2167275519,BF -2167275520,2167930879,US -2167930880,2167996415,NG -2167996416,2168193023,US -2168193024,2168258559,JP -2168258560,2168651775,US -2168651776,2168717311,GB -2168717312,2168782847,US -2168782848,2168913919,DE -2168913920,2169372671,US -2169372672,2169438207,AU -2169438208,2170028031,US -2170028032,2170093567,FR -2170093568,2170159103,US -2170159104,2170224639,VE -2170224640,2170421247,US -2170421248,2170486783,AU -2170486784,2170552319,US -2170552320,2170617855,AU -2170617856,2170683391,CA -2170683392,2170814463,US -2170814464,2170879999,CA -2170880000,2170945535,US -2170945536,2171011071,FR -3652593408,3652593471,ES -3652593472,3652593511,FR -3652593512,3652593519,ES -3652593520,3652593631,FR -3652593632,3652593663,PT -3652593664,3652593943,FR -3652593944,3652593951,ES -3652593952,3652595007,FR -3652595008,3652595071,DE -3652595072,3652595167,FR -3652595168,3652595183,ES -3652595184,3652595871,FR -3652595872,3652595935,PL -3652595936,3652596351,FR -3652596352,3652596415,IT -3652596416,3652596479,FR -3652596480,3652596543,ES -3652596544,3652596799,FR -3652596800,3652596831,CZ -3652596832,3652597183,FR -3652597184,3652597247,DE -3652597248,3652597375,FR -3652597376,3652597383,ES -3652597384,3652597407,FR -3652597408,3652597439,PL -3652597440,3652597887,FR -3652597888,3652597903,GB -3652597904,3652599569,FR -3652599570,3652599570,PT -3652599571,3652599679,FR -3652599680,3652599743,IT -3652599744,3652601855,FR -3652601856,3652603903,PL -3652603904,3652608191,FR -3652608192,3652608223,PT -3652608224,3652608255,FR -3652608256,3652608511,GB -3652608512,3652608639,FR -3652608640,3652608767,GB -3652608768,3652609023,FR -3652609024,3652609279,GB -3652609280,3652609503,FR -3652609504,3652609535,FI -3652609536,3652609727,FR -3652609728,3652609759,PL -3652609760,3652609791,CZ -3652609792,3652609823,FR -3652609824,3652609855,CZ -3652609856,3652609919,FR -3652609920,3652609983,ES -3652609984,3652610047,BE -3652610048,3652611135,FR -3652611136,3652611199,ES -3652611200,3652611231,FR -3652611232,3652611263,PT -3652611264,3652611679,FR -3652611680,3652611711,PT -3652611712,3652611775,NL -3652611776,3652612223,FR -3652612224,3652612287,ES -3652612288,3652612351,FR -3652612352,3652612479,GB -3652612480,3652612543,IE -3652612544,3652612607,NL -3652612608,3652613335,FR -3652613336,3652613343,ES -3652613344,3652613375,FR -3652613376,3652613407,FI -3652613408,3652613615,FR -3652613616,3652613623,ES -3652613624,3652613679,FR -3652613680,3652613695,LT -3652613696,3652614015,FR -3652614016,3652614079,BE -3652614080,3652615871,FR -3652615872,3652615935,DE -3652615936,3652620639,FR -3652620640,3652620671,CZ -3652620672,3652620735,PT -3652620736,3652620799,FR -3652620800,3652620831,PT -3652620832,3652621247,FR -3652621248,3652621311,DE -3652621312,3652621375,FR -3652621376,3652621439,ES -3652621440,3652621503,FR -3652621504,3652621567,IT -3652621568,3652621631,FR -3652621632,3652621663,PT -3652621664,3652621823,FR -3652621824,3652621951,IE -3652621952,3652622271,FR -3652622272,3652622335,GB -3652622336,3652622879,FR -3652622880,3652622911,CZ -3652622912,3652623679,FR -3652623680,3652623807,NL -3652623808,3652624191,FR -3652624192,3652624319,IT -3652624320,3652628479,FR -3652628480,3652628543,IT -3652628544,3652628607,FR -3652628608,3652628639,PL -3652628640,3652628855,FR -3652628856,3652628863,ES -3652628864,3652629743,FR -3652629744,3652629759,ES -3652629760,3652630015,FR -3652630016,3652630031,ES -3652630032,3652630079,FR -3652630080,3652630111,PL -3652630112,3652631295,FR -3652631296,3652631359,BE -3652631360,3652631391,FR -3652631392,3652631407,CH -3652631408,3652631423,FR -3652631424,3652631455,PL -3652631456,3652631551,FR -3652631552,3652631583,CZ -3652631584,3652631807,FR -3652631808,3652631823,GB -3652631824,3652632031,FR -3652632032,3652632063,PT -3652632064,3652632303,FR -3652632304,3652632311,ES -3652632312,3652633599,FR -3652633600,3652634623,DE -3652634624,3652635647,PL -3652635648,3652638655,FR -3652638656,3652638719,ES -3652638720,3652638815,FR -3652638816,3652638847,FI -3652638848,3652638975,GB -3652638976,3652639359,FR -3652639360,3652639423,DE -3652639424,3652639679,FR -3652639680,3652639807,NL -3652639808,3652640575,FR -3652640576,3652640703,GB -3652640704,3652640711,FR -3652640712,3652640719,ES -3652640720,3652640767,FR -3652640768,3652640831,ES -3652640832,3652641727,FR -3652641728,3652641791,GB -3652641792,3652642111,FR -3652642112,3652642175,IE -3652642176,3652642239,FR -3652642240,3652642303,DE -3652642304,3652642367,FR -3652642368,3652642431,GB -3652642432,3652642719,FR -3652642720,3652642751,PT -3652642752,3652642975,FR -3652642976,3652643007,IE -3652643008,3652643375,FR -3652643376,3652643379,ES -3652643380,3652643519,FR -3652643520,3652643583,NL -3652643584,3652643647,ES -3652643648,3652644031,FR -3652644032,3652644063,BE -3652644064,3652644199,FR -3652644200,3652644215,ES -3652644216,3652644223,FR -3652644224,3652644239,NL -3652644240,3652644247,FR -3652644248,3652644255,ES -3652644256,3652644351,FR -3652644352,3652644383,FI -3652644384,3652644415,PL -3652644416,3652644575,FR -3652644576,3652644607,DE -3652644608,3652645119,FR -3652645120,3652645503,GB -3652645504,3652645663,FR -3652645664,3652645695,FI -3652645696,3652645887,FR -3652645888,3652646015,NL -3652646016,3652646079,ES -3652646080,3652646111,FR -3652646112,3652646143,CZ -3652646144,3652646271,NL -3652646272,3652646655,FR -3652646656,3652646719,ES -3652646720,3652646799,FR -3652646800,3652646815,PL -3652646816,3652646847,FR -3652646848,3652646863,FI -3652646864,3652648847,FR -3652648848,3652648863,LT -3652648864,3652648895,FI -3652648896,3652648959,DE -3652648960,3652714495,IE -3652714496,3653238783,DE -3653238784,3653369855,CH -3653369856,3653373951,IT -3653373952,3653378047,NL -3653378048,3653382143,DE -3653382144,3653386239,CH -3653386240,3653390335,DE -3653390336,3653394431,FR -3653394432,3653402623,NL -3653402624,3653406557,GB -3653406558,3653406558,GN -3653406559,3653406617,GB -3653406618,3653406618,GN -3653406619,3653407103,GB -3653407104,3653407111,UG -3653407112,3653408071,GB -3653408072,3653408079,NG -3653408080,3653408231,GB -3653408232,3653408239,KE -3653408240,3653410815,GB -3653410816,3653414911,CZ -3653414912,3653419007,IT -3653419008,3653423103,IL -3653423104,3653427199,GB -3653427200,3653431295,DE -3653431296,3653435391,RU -3653435392,3653439487,DE -3653439488,3653443583,FR -3653443584,3653447679,DE -3653447680,3653451775,LV -3653451776,3653464063,RU -3653464064,3653468159,NL -3653468160,3653472255,GR -3653476352,3653480447,CZ -3653480448,3653484543,DK -3653484544,3653488639,TR -3653488640,3653492735,RU -3653492736,3653500927,NL -3653500928,3653505023,GB -3653505024,3653509119,KZ -3653509120,3653513215,NL -3653513216,3653517311,NO -3653517312,3653525503,AT -3653525504,3653529599,RU -3653529600,3653533695,CZ -3653533696,3653537791,IT -3653537792,3653541887,AT -3653541888,3653545983,UA -3653545984,3653550079,CH -3653550080,3653554175,MK -3653554176,3653558271,CZ -3653558272,3653566463,GB -3653566464,3653570559,RU -3653570560,3653574655,ES -3653574656,3653578751,CZ -3653578752,3653582847,SE -3653582848,3653586943,PL -3653586944,3653591039,DE -3653591040,3653595135,LU -3653595136,3653599231,RU -3653599232,3653601279,CH -3653601280,3653603327,BA -3653603328,3653607423,CZ -3653611520,3653615615,HU -3653615616,3653619711,RU -3653619712,3653623807,CH -3653623808,3653636095,RU -3653636096,3653640191,NL -3653640192,3653648383,GB -3653648384,3653652479,SE -3653652480,3653656575,RU -3653656576,3653660671,GB -3653660672,3653664767,CZ -3653664768,3653668863,DE -3653668864,3653672959,SE -3653672960,3653681151,RU -3653681152,3653685247,ES -3653685248,3653689343,DK -3653689344,3653693439,LV -3653693440,3653697535,DE -3653697536,3653705727,IT -3653705728,3653708331,NO -3653708332,3653708332,FI -3653708333,3653713919,NO -3653713920,3653718015,DE -3653718016,3653722111,AT -3653722112,3653730303,LV -3653730304,3653734399,BA -3653734400,3653738495,KE -3653738496,3653746687,GB -3653746688,3653750783,DE -3653750784,3653754879,RU -3653754880,3653758975,UA -3653758976,3653763071,RU -3653763072,3654025215,IT -3654025216,3654287359,GB -3654287360,3654608404,SE -3654608405,3654608405,NO -3654608406,3654608895,SE -3654608896,3654609919,NO -3654609920,3654610431,SE -3654610432,3654610943,FR -3654610944,3654610951,SE -3654610952,3654610959,DE -3654610960,3654612231,SE -3654612232,3654612239,AT -3654612240,3654612271,SE -3654612272,3654612287,AT -3654612288,3654614047,SE -3654614048,3654614063,GB -3654614064,3654614079,SE -3654614080,3654614271,FI -3654614272,3654811647,SE -3654811648,3654942719,ES -3654942720,3655073791,IR -3655073792,3655335935,IT -3655335936,3657433087,DE -3657433088,3659415455,CN -3659415456,3659415487,SG -3659415488,3659530239,CN -3659530240,3659595775,TW -3659595776,3659628543,ID -3659628544,3659661311,JP -3659661312,3659792383,TW -3659792384,3660054527,KR -3660054528,3660578815,JP -3660578816,3661103103,KR -3661103104,3663986687,CN -3663986688,3663987711,AU -3663987712,3663987967,ID -3663987968,3663989247,JP -3663989248,3663989503,VN -3663989504,3663989759,ID -3663989760,3663990271,AU -3663990272,3663990527,VN -3663990528,3663990783,JP -3663990784,3663991295,HK -3663991296,3663991551,MY -3663991552,3663991807,AU -3663992064,3663992319,NZ -3663992320,3663992575,MY -3663992576,3663993599,NZ -3663993600,3663996159,ID -3663996160,3663996415,AU -3663996416,3663996671,TH -3663996672,3663997183,AU -3663997184,3663997439,ID -3663997440,3663997695,JP -3663997696,3663997951,AU -3663997952,3663998207,MY -3663998208,3663998463,JP -3663998464,3663998975,TH -3663998976,3663999487,IN -3663999488,3663999743,AU -3664000000,3664000767,AU -3664000768,3664001023,ID -3664001024,3664001279,NZ -3664001280,3664001535,LK -3664001536,3664001791,MY -3664002048,3664002303,VN -3664002304,3664002559,LK -3664002560,3664003327,ID -3664003328,3664003583,NZ -3664003584,3664003839,TH -3664003840,3664004095,JP -3664004352,3664004607,MY -3664004864,3664005119,KH -3664005120,3664005887,ID -3664005888,3664006143,MY -3664006144,3664006399,AU -3664006400,3664006655,PF -3664006656,3664006911,AU -3664007168,3664008191,AU -3664008192,3664008447,MN -3664008448,3664008703,PK -3664008960,3664010239,AU -3664010240,3664052223,CN -3664052224,3664084991,NZ -3664084992,3664117759,KR -3664117760,3664248831,HK -3664248832,3664642047,CN -3664642048,3664707583,JP -3664707584,3664773119,MY -3664773120,3666870271,JP -3666870272,3666960455,KR -3666960456,3666960456,US -3666960457,3667918847,KR -3667918848,3668967423,TW -3668967424,3669491711,JP -3669491712,3669557247,TW -3669557248,3669590015,AU -3669590016,3669606399,JP -3669606400,3669614591,CN -3669614592,3669616639,NZ -3669616640,3669618687,AU -3669618688,3669620735,CN -3669620736,3669622783,IN -3669622784,3669688319,SG -3669688320,3669753855,TW -3669753856,3670015999,HK -3670016000,3671064575,CN -3671064576,3671130111,MY -3671130112,3671195647,KR -3671195648,3671326719,TW -3671326720,3671392255,SG -3671392256,3671457791,HK -3671457792,3671588863,AU -3671588864,3672637439,JP -3672637440,3673161727,KR -3673161728,3673686015,CN -3673686016,3673751551,IN -3673751552,3673817087,CN -3673817088,3673882623,HK -3673882624,3673948159,JP -3673948160,3674210303,HK -3674210304,3678404607,JP -3678404608,3678535679,IN -3678535680,3678666751,JP -3678666752,3678928895,TW -3678928896,3678994431,CN -3678994432,3679027199,HK -3679027200,3679059967,JP -3679059968,3679158271,SG -3679158272,3679191039,JP -3679191040,3679453183,HK -3679453184,3679584255,TW -3679584256,3679649791,CN -3679649792,3679682559,ID -3679682560,3679715327,CN -3679715328,3679977471,TW -3679977472,3680108543,NZ -3680108544,3680124927,TW -3680124928,3680125951,IN -3680125952,3680129023,CN -3680129024,3680133119,PH -3680133120,3680137215,IN -3680137216,3680141311,HK -3680141312,3680174079,AU -3680174080,3680206847,TW -3680206848,3680239615,IN -3680239616,3680403455,MY -3680403456,3680436223,JP -3680436224,3680501759,MY -3680501760,3682598911,JP -3682598912,3684575268,CN -3684575269,3684575269,HK -3684575270,3684696063,CN -3684696064,3688366079,JP -3688366080,3689938943,CN -3689938944,3690070015,KR -3690070016,3690463231,CN -3690463232,3690987519,KR -3690987520,3695181823,JP -3695181824,3697278975,KR -3697278976,3697573887,JP -3697573888,3697582079,GB -3697582080,3697586175,SG -3697586176,3697606655,JP -3697606656,3697655807,AU -3697655808,3697672191,CN -3697672192,3697737727,JP -3697737728,3697803263,KR -3697803264,3698327551,JP -3698327552,3698589695,CN -3698589696,3699376127,KR -3699376128,3700424703,TW -3700424704,3700752383,JP -3700752384,3700817919,KR -3700817920,3700981759,JP -3700981760,3701014527,CN -3701014528,3701080063,JP -3701080064,3701211135,CN -3701211136,3701252095,JP -3701252096,3701256191,NC -3701256192,3701258239,SG -3701258240,3701260287,IN -3701260288,3701293055,JP -3701293056,3701301247,AU -3701301248,3701305343,ID -3701305344,3701309439,TW -3701309440,3701374975,JP -3701374976,3701375999,IN -3701376000,3701377023,HK -3701377024,3701380095,IN -3701380096,3701381119,KH -3701381120,3701390335,IN -3701390336,3701391359,AU -3701391360,3701392383,IN -3701392384,3701393407,HK -3701393408,3701394431,MY -3701394432,3701395455,BD -3701395456,3701396479,MY -3701396480,3701397247,NZ -3701397248,3701397503,AU -3701397504,3701398527,JP -3701398528,3701399551,MV -3701399552,3701400575,HK -3701400576,3701401599,TW -3701401600,3701402623,BD -3701402624,3701403647,BT -3701403648,3701404671,CN -3701404672,3701405695,HK -3701405696,3701406719,JP -3701406720,3701407743,HK -3701407744,3701473279,JP -3701473280,3704619007,CN -3704619008,3705667583,JP -3705667584,3705929727,IN -3705929728,3706060799,TW -3706060800,3706126335,KR -3706126336,3706142719,CN -3706142720,3706159103,VN -3706159104,3706191871,CN -3706191872,3706207107,SG -3706207108,3706207108,US -3706207109,3706208255,SG -3706208256,3706224639,CN -3706224640,3706225663,HK -3706225664,3706226687,JP -3706226688,3706231807,HK -3706231808,3706232831,JP -3706232832,3706233343,HK -3706233344,3706234367,JP -3706234368,3706237951,HK -3706237952,3706238975,JP -3706238976,3706244095,HK -3706244096,3706244863,JP -3706244864,3706245887,HK -3706245888,3706246143,JP -3706246144,3706253823,HK -3706253824,3706254335,JP -3706254336,3706256895,HK -3706256896,3706257151,JP -3706257152,3706257407,HK -3706257408,3706322943,AU -3706322944,3706388479,CN -3706388480,3706781695,AU -3706781696,3706847231,HK -3706847232,3706978303,CN -3706978304,3707109375,AU -3707109376,3707174911,HK -3707174912,3707207679,JP -3707207680,3707208703,BD -3707208704,3707209727,NZ -3707209728,3707211775,CN -3707211776,3707215871,NP -3707215872,3707217919,BD -3707217920,3707219967,ID -3707219968,3707222015,AU -3707222016,3707224063,JP -3707224064,3707240447,LK -3707240448,3707568127,CN -3707568128,3707633663,AU -3707633664,3707699199,JP -3707699200,3707764735,SG -3707764736,3708600319,CN -3708600320,3708616703,JP -3708616704,3708813311,CN -3708813312,3715629055,JP -3715629056,3715653631,TW -3715653632,3715655679,BD -3715655680,3715657727,IN -3715657728,3715661823,SG -3715661824,3715670015,AU -3715670016,3715671039,KH -3715671040,3715672063,AU -3715672064,3715674111,JP -3715674112,3715678207,HK -3715678208,3715694591,PK -3715694592,3715710975,VN -3715710976,3715719167,AU -3715719168,3715727359,PH -3715727360,3715729151,AU -3715729152,3715729407,NZ -3715729408,3715735551,AU -3715735552,3715741695,JP -3715741696,3715743743,PH -3715743744,3715760127,JP -3715760128,3715891199,CN -3715891200,3716153343,HK -3716153344,3716170239,SG -3716170240,3716170494,TH -3716170495,3716171519,SG -3716171520,3716171775,JP -3716171776,3716172031,SG -3716172032,3716172287,JP -3716172288,3716173055,SG -3716173056,3716173311,JP -3716173312,3716173567,SG -3716173568,3716173823,JP -3716173824,3716174079,SG -3716174080,3716174083,TH -3716174084,3716174335,JP -3716174336,3716175615,SG -3716175616,3716176895,JP -3716176896,3716178175,SG -3716178176,3716178943,JP -3716178944,3716179967,SG -3716179968,3716181759,JP -3716181760,3716182783,SG -3716182784,3716183295,JP -3716183296,3716183551,SG -3716183552,3716184063,JP -3716184064,3716184319,SG -3716184320,3716184575,JP -3716184576,3716184831,SG -3716184832,3716185087,JP -3716185088,3716186111,SG -3716186112,3716415487,CN -3716415488,3716431871,VN -3716431872,3716440063,KR -3716440064,3716444159,JP -3716444160,3716446207,PK -3716446208,3716464639,JP -3716464640,3716481023,ID -3716481024,3716489215,VN -3716489216,3716493311,MY -3716493312,3716497407,KR -3716497408,3716513791,JP -3716513792,3716530175,KR -3716530176,3716538367,AU -3716538368,3716546559,CN -3716546560,3716677631,IN -3716677632,3716808703,CN -3716808704,3718840319,KR -3718840320,3718905855,TW -3718905856,3719036927,JP -3719036928,3719823359,CN -3719823360,3720347647,JP -3720347648,3720859647,CN -3720859648,3720863743,AU -3720863744,3723493375,CN -3723493376,3725590527,JP -3725590528,3730833407,CN -3730833408,3732602879,KR -3732602880,3732668415,TH -3732668416,3732733951,ID -3732733952,3732799487,CN -3732799488,3732832255,PH -3732832256,3732865023,CN -3732865024,3732930559,PH -3732930560,3733979135,CN -3733979136,3734503423,JP -3734503424,3734765567,NZ -3734765568,3734896639,TW -3734896640,3735027711,JP -3735027712,3735289855,CN -3735289856,3735388159,SG -3735388160,3735404543,LK -3735404544,3735420927,ID -3735420928,3735551999,HK -3735552000,3739222015,CN -3739222016,3739570175,JP -3739570176,3739572223,ID -3739572224,3739574271,AU -3739574272,3739680767,JP -3739680768,3739697151,KR -3739697152,3739746303,JP -3739746304,3740270591,KR -3740270592,3740925951,CN -3740925952,3741024255,TW -3741024256,3741057023,KR -3741057024,3741319167,VN -3741319168,3742367743,CN -3742367744,3742629887,HK -3742629888,3742760959,CN -3742760960,3742892031,TW -3742892032,3742957567,TH -3742957568,3742973951,PH -3742973952,3742982143,SG -3742982144,3742986239,ID -3742986240,3742988287,AU -3742988288,3742990335,VU -3742990336,3743006719,JP -3743006720,3743014911,TH -3743014912,3743016959,AU -3743016960,3743019007,SG -3743019008,3743022079,MY -3743022080,3743023103,BD -3743023104,3743027199,TW -3743027200,3743028223,IN -3743028224,3743029247,AF -3743029248,3743030271,NZ -3743030272,3743035391,IN -3743035392,3743039487,HK -3743039488,3743055871,TW -3743055872,3743088639,KR -3743088640,3743093647,AU -3743093648,3743093648,NZ -3743093649,3743096831,AU -3743096832,3743105023,TW -3743105024,3743106047,AU -3743106048,3743109119,JP -3743109120,3743113215,BD -3743113216,3743115263,AU -3743115264,3743117311,VN -3743117312,3743118335,BD -3743118336,3743119359,JP -3743119360,3743120383,IN -3743120384,3743121407,JP -3743121408,3743125503,MY -3743125504,3743129599,ID -3743129600,3743130623,HK -3743130624,3743130879,SG -3743130880,3743131135,HK -3743131136,3743133695,SG -3743133696,3743134719,AU -3743134720,3743135743,JP -3743135744,3743136767,CN -3743136768,3743137791,MY -3743137792,3743154175,TH -3743154176,3743186943,MY -3743186944,3743219711,KR -3743219712,3743252479,JP -3743252480,3743264767,NC -3743264768,3743268863,JP -3743268864,3743272959,IN -3743272960,3743273983,CN -3743273984,3743275007,BD -3743275008,3743276031,HK -3743276032,3743277055,IN -3743277056,3743281151,PK -3743281152,3743282175,AU -3743282176,3743283199,JP -3743283200,3743284223,HK -3743284224,3743285247,CN -3743285248,3743416319,IN -3743416320,3745513471,KR -3745513472,3749052415,CN -3749052416,3749183487,HK -3749183488,3749838847,CN -3749838848,3749839871,SG -3749839872,3749840895,IN -3749840896,3749841919,CN -3749841920,3749842943,AU -3749842944,3749843967,PH -3749843968,3749844991,ID -3749844992,3749846015,AU -3749846016,3749847039,IN -3749847040,3749855231,HK -3749855232,3749969919,KR -3749969920,3750232063,JP -3750232064,3750756351,TW -3750756352,3752067071,CN -3752067072,3752132607,ID -3752132608,3752133631,BD -3752133632,3752134655,ID -3752134656,3752136703,TW -3752136704,3752137727,NZ -3752137728,3752138751,JP -3752138752,3752140799,IN -3752140800,3752148991,JP -3752148992,3752153087,NZ -3752153088,3752157183,JP -3752157184,3752165375,AU -3752165376,3752198143,KR -3752198144,3752329215,CN -3752329216,3752853503,KR -3752853504,3753902079,IN -3753902080,3754033151,CN -3754033152,3754164223,KR -3754164224,3754229759,IN -3754229760,3754295295,HK -3754295296,3754426367,CN -3754426368,3754491903,TW -3754491904,3754688511,CN -3754688512,3754950655,TH -3754950656,3755474943,CN -3755474944,3755737087,JP -3755737088,3755868159,CN -3755868160,3755933695,KR -3755933696,3755966463,JP -3755966464,3755974655,IN -3755974656,3755976703,JP -3755976704,3755978751,KH -3755978752,3755986943,CN -3755986944,3755988991,JP -3755988992,3755990015,HK -3755990016,3755991039,SG -3755991040,3755999231,JP -3755999232,3757047807,IN -3757047808,3757834239,CN -3757834240,3757850623,AU -3757850624,3757858815,JP -3757858816,3757862911,AU -3757862912,3757867007,JP -3757867008,3757875519,CN -3757875520,3757875583,HK -3757875584,3757899775,CN -3757899776,3757965311,KR -3757965312,3758063615,CN -3758063616,3758079999,HK -3758080000,3758088191,KR -3758088192,3758090239,ID -3758090240,3758091263,AU -3758091264,3758092287,CN -3758092288,3758093311,HK -3758093312,3758094335,IN -3758094336,3758095359,HK -3758095360,3758095871,CN -3758095872,3758096127,SG -3758096128,3758096383,AU diff --git a/broker/test_geoip6 b/broker/test_geoip6 deleted file mode 100644 index 80279f3..0000000 --- a/broker/test_geoip6 +++ /dev/null @@ -1,693 +0,0 @@ -# Last updated based on February 7 2018 Maxmind GeoLite2 Country -# wget https://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.mmdb.gz -# gunzip GeoLite2-Country.mmdb.gz -# python mmdb-convert.py GeoLite2-Country.mmdb -600:8801:9400:5a1:948b:ab15:dde3:61a3,600:8801:9400:5a1:948b:ab15:dde3:61a3,US -2001:200::,2001:200:ffff:ffff:ffff:ffff:ffff:ffff,JP -2001:208::,2001:208:ffff:ffff:ffff:ffff:ffff:ffff,SG -2001:218::,2001:218:ffff:ffff:ffff:ffff:ffff:ffff,JP -2001:220::,2001:220:ffff:ffff:ffff:ffff:ffff:ffff,KR -2001:230::,2001:230:ffff:ffff:ffff:ffff:ffff:ffff,KR -2001:238::,2001:238:ffff:ffff:ffff:ffff:ffff:ffff,TW -2001:240::,2001:240:ffff:ffff:ffff:ffff:ffff:ffff,JP -2620:21:2000::,2620:21:2000:ffff:ffff:ffff:ffff:ffff,US -2620:21:4000::,2620:21:4000:ffff:ffff:ffff:ffff:ffff,US -2620:21:6000::,2620:21:600f:ffff:ffff:ffff:ffff:ffff,US -2620:21:8000::,2620:21:8000:ffff:ffff:ffff:ffff:ffff,US -2620:21:a000::,2620:21:a000:ffff:ffff:ffff:ffff:ffff,US -2620:21:c000::,2620:21:c000:ffff:ffff:ffff:ffff:ffff,CA -2620:21:e000::,2620:21:e000:ffff:ffff:ffff:ffff:ffff,US -2620:22::,2620:22::ffff:ffff:ffff:ffff:ffff,US -2620:22:2000::,2620:22:2000:ffff:ffff:ffff:ffff:ffff,US -2620:22:4000::,2620:22:4000:ffff:ffff:ffff:ffff:ffff,CA -2620:22:6000::,2620:22:6000:ffff:ffff:ffff:ffff:ffff,US -2620:c2:8000::,2620:c2:8000:ffff:ffff:ffff:ffff:ffff,US -2620:c2:c000::,2620:c2:c000:ffff:ffff:ffff:ffff:ffff,US -2620:c3::,2620:c3::ffff:ffff:ffff:ffff:ffff,US -2620:c3:4000::,2620:c3:4000:ffff:ffff:ffff:ffff:ffff,US -2620:c3:8000::,2620:c3:8000:ffff:ffff:ffff:ffff:ffff,US -2620:c3:c000::,2620:c3:c00f:ffff:ffff:ffff:ffff:ffff,US -2620:c4::,2620:c4::ffff:ffff:ffff:ffff:ffff,US -2620:c4:4000::,2620:c4:4000:ffff:ffff:ffff:ffff:ffff,US -2620:c4:8000::,2620:c4:8000:ffff:ffff:ffff:ffff:ffff,US -2620:c4:c000::,2620:c4:c000:ffff:ffff:ffff:ffff:ffff,CA -2620:c5::,2620:c5::ffff:ffff:ffff:ffff:ffff,US -2620:c5:4000::,2620:c5:4000:ffff:ffff:ffff:ffff:ffff,US -2620:c5:c000::,2620:c5:c000:ffff:ffff:ffff:ffff:ffff,US -2620:c6::,2620:c6::ffff:ffff:ffff:ffff:ffff,US -2620:c6:4000::,2620:c6:4000:ffff:ffff:ffff:ffff:ffff,US -2620:c6:8000::,2620:c6:8000:ffff:ffff:ffff:ffff:ffff,US -2620:c6:c000::,2620:c6:c000:ffff:ffff:ffff:ffff:ffff,US -2620:c7::,2620:c7::ffff:ffff:ffff:ffff:ffff,US -2620:c7:4000::,2620:c7:4000:ffff:ffff:ffff:ffff:ffff,US -2620:c7:8000::,2620:c7:8000:ffff:ffff:ffff:ffff:ffff,US -2620:c7:c000::,2620:c7:c000:ffff:ffff:ffff:ffff:ffff,US -2620:c8::,2620:c8::ffff:ffff:ffff:ffff:ffff,US -2620:c8:4000::,2620:c8:4000:ffff:ffff:ffff:ffff:ffff,US -2620:c8:c000::,2620:c8:c00f:ffff:ffff:ffff:ffff:ffff,US -2620:c9::,2620:c9::ffff:ffff:ffff:ffff:ffff,US -2620:c9:4000::,2620:c9:4000:ffff:ffff:ffff:ffff:ffff,US -2620:c9:8000::,2620:c9:8000:ffff:ffff:ffff:ffff:ffff,US -2620:c9:c000::,2620:c9:c000:ffff:ffff:ffff:ffff:ffff,US -2620:ca::,2620:ca::ffff:ffff:ffff:ffff:ffff,US -2620:ca:4000::,2620:ca:4000:ffff:ffff:ffff:ffff:ffff,US -2620:ca:8000::,2620:ca:8000:ffff:ffff:ffff:ffff:ffff,US -2620:ca:c000::,2620:ca:c000:ffff:ffff:ffff:ffff:ffff,US -2620:cb::,2620:cb:f:ffff:ffff:ffff:ffff:ffff,US -2620:cb:4000::,2620:cb:4000:ffff:ffff:ffff:ffff:ffff,US -2620:cb:8000::,2620:cb:8000:ffff:ffff:ffff:ffff:ffff,US -2620:cb:c000::,2620:cb:c000:ffff:ffff:ffff:ffff:ffff,US -2620:cc::,2620:cc::ffff:ffff:ffff:ffff:ffff,US -2620:cc:4000::,2620:cc:4000:ffff:ffff:ffff:ffff:ffff,US -2620:cc:8000::,2620:cc:8000:ffff:ffff:ffff:ffff:ffff,US -2620:cc:c000::,2620:cc:c000:ffff:ffff:ffff:ffff:ffff,US -2620:cd::,2620:cd::ffff:ffff:ffff:ffff:ffff,US -2620:cd:4000::,2620:cd:4000:ffff:ffff:ffff:ffff:ffff,US -2620:cd:8000::,2620:cd:8000:ffff:ffff:ffff:ffff:ffff,US -2620:cd:c000::,2620:cd:c000:ffff:ffff:ffff:ffff:ffff,US -2620:ce::,2620:ce::ffff:ffff:ffff:ffff:ffff,US -2620:ce:4000::,2620:ce:4000:ffff:ffff:ffff:ffff:ffff,US -2620:ce:8000::,2620:ce:8000:ffff:ffff:ffff:ffff:ffff,US -2620:ce:c000::,2620:ce:c000:ffff:ffff:ffff:ffff:ffff,US -2620:cf:4000::,2620:cf:4000:ffff:ffff:ffff:ffff:ffff,US -2620:cf:8000::,2620:cf:8000:ffff:ffff:ffff:ffff:ffff,US -2620:cf:c000::,2620:cf:c00f:ffff:ffff:ffff:ffff:ffff,US -2620:d0::,2620:d0::ffff:ffff:ffff:ffff:ffff,US -2620:d0:4000::,2620:d0:4000:ffff:ffff:ffff:ffff:ffff,US -2620:d0:8000::,2620:d0:8000:ffff:ffff:ffff:ffff:ffff,US -2620:d0:c000::,2620:d0:c000:ffff:ffff:ffff:ffff:ffff,US -2620:d1::,2620:d1::ffff:ffff:ffff:ffff:ffff,US -2620:d1:4000::,2620:d1:4000:ffff:ffff:ffff:ffff:ffff,US -2620:d1:8000::,2620:d1:8000:ffff:ffff:ffff:ffff:ffff,US -2620:d1:c000::,2620:d1:c000:ffff:ffff:ffff:ffff:ffff,US -2620:d2::,2620:d2::ffff:ffff:ffff:ffff:ffff,US -2620:d2:4000::,2620:d2:4000:ffff:ffff:ffff:ffff:ffff,US -2620:d2:8000::,2620:d2:8000:ffff:ffff:ffff:ffff:ffff,US -2620:d2:c000::,2620:d2:c000:ffff:ffff:ffff:ffff:ffff,CA -2620:d3:4000::,2620:d3:4000:ffff:ffff:ffff:ffff:ffff,US -2620:d3:8000::,2620:d3:8000:ffff:ffff:ffff:ffff:ffff,US -2620:d3:c000::,2620:d3:c000:ffff:ffff:ffff:ffff:ffff,US -2620:d4::,2620:d4::ffff:ffff:ffff:ffff:ffff,US -2620:d4:4000::,2620:d4:4000:ffff:ffff:ffff:ffff:ffff,US -2620:d4:8000::,2620:d4:8000:ffff:ffff:ffff:ffff:ffff,US -2620:d5::,2620:d5::ffff:ffff:ffff:ffff:ffff,US -2620:d5:4000::,2620:d5:4000:ffff:ffff:ffff:ffff:ffff,US -2620:d5:8000::,2620:d5:8000:ffff:ffff:ffff:ffff:ffff,US -2620:d5:c000::,2620:d5:c000:ffff:ffff:ffff:ffff:ffff,US -2620:d6::,2620:d6::ffff:ffff:ffff:ffff:ffff,US -2620:d6:4000::,2620:d6:4000:ffff:ffff:ffff:ffff:ffff,US -2620:d6:8000::,2620:d6:8000:ffff:ffff:ffff:ffff:ffff,US -2620:d6:c000::,2620:d6:c000:ffff:ffff:ffff:ffff:ffff,US -2620:d7::,2620:d7::ffff:ffff:ffff:ffff:ffff,US -2620:d7:4000::,2620:d7:4000:ffff:ffff:ffff:ffff:ffff,CA -2620:d7:8000::,2620:d7:8000:ffff:ffff:ffff:ffff:ffff,US -2620:d7:c000::,2620:d7:c000:ffff:ffff:ffff:ffff:ffff,US -2620:d8::,2620:d8::ffff:ffff:ffff:ffff:ffff,US -2620:d8:4000::,2620:d8:4000:ffff:ffff:ffff:ffff:ffff,US -2620:d8:8000::,2620:d8:8000:ffff:ffff:ffff:ffff:ffff,US -2620:d8:c000::,2620:d8:c000:ffff:ffff:ffff:ffff:ffff,US -2620:d9::,2620:d9::ffff:ffff:ffff:ffff:ffff,US -2620:d9:4000::,2620:d9:4000:ffff:ffff:ffff:ffff:ffff,US -2620:d9:8000::,2620:d9:8000:ffff:ffff:ffff:ffff:ffff,US -2620:d9:c000::,2620:d9:c000:ffff:ffff:ffff:ffff:ffff,US -2620:da::,2620:da::ffff:ffff:ffff:ffff:ffff,US -2620:da:4000::,2620:da:4000:ffff:ffff:ffff:ffff:ffff,US -2620:da:c000::,2620:da:c000:ffff:ffff:ffff:ffff:ffff,US -2620:db::,2620:db::ffff:ffff:ffff:ffff:ffff,US -2620:db:4000::,2620:db:4000:ffff:ffff:ffff:ffff:ffff,CA -2620:db:8000::,2620:db:8000:ffff:ffff:ffff:ffff:ffff,US -2620:db:c000::,2620:db:c000:ffff:ffff:ffff:ffff:ffff,US -2620:dc::,2620:dc::ffff:ffff:ffff:ffff:ffff,US -2620:dc:8::,2620:dc:8:ffff:ffff:ffff:ffff:ffff,US -2620:dc:4000::,2620:dc:40ff:ffff:ffff:ffff:ffff:ffff,US -2620:dc:8000::,2620:dc:8000:ffff:ffff:ffff:ffff:ffff,CA -2620:dc:c000::,2620:dc:c000:ffff:ffff:ffff:ffff:ffff,US -2620:dd::,2620:dd::ffff:ffff:ffff:ffff:ffff,CA -2620:dd:4000::,2620:dd:4000:ffff:ffff:ffff:ffff:ffff,US -2620:dd:8000::,2620:dd:8000:ffff:ffff:ffff:ffff:ffff,US -2620:dd:c000::,2620:dd:c000:ffff:ffff:ffff:ffff:ffff,US -2620:de::,2620:de::ffff:ffff:ffff:ffff:ffff,US -2620:de:4000::,2620:de:4000:ffff:ffff:ffff:ffff:ffff,US -2620:de:8000::,2620:de:8000:ffff:ffff:ffff:ffff:ffff,US -2620:de:c000::,2620:de:c000:ffff:ffff:ffff:ffff:ffff,US -2620:df::,2620:df::ffff:ffff:ffff:ffff:ffff,US -2620:df:4000::,2620:df:400f:ffff:ffff:ffff:ffff:ffff,US -2620:df:8000::,2620:df:8000:ffff:ffff:ffff:ffff:ffff,US -2620:df:c000::,2620:df:c000:ffff:ffff:ffff:ffff:ffff,US -2620:e0::,2620:e0::ffff:ffff:ffff:ffff:ffff,US -2620:e0:4000::,2620:e0:4000:ffff:ffff:ffff:ffff:ffff,US -2620:e0:8000::,2620:e0:8000:ffff:ffff:ffff:ffff:ffff,US -2620:e0:c000::,2620:e0:c000:ffff:ffff:ffff:ffff:ffff,US -2620:e1::,2620:e1::ffff:ffff:ffff:ffff:ffff,US -2620:e1:4000::,2620:e1:4000:ffff:ffff:ffff:ffff:ffff,US -2620:e1:8000::,2620:e1:8000:ffff:ffff:ffff:ffff:ffff,US -2620:e1:c000::,2620:e1:c000:ffff:ffff:ffff:ffff:ffff,VG -2620:e2::,2620:e2::ffff:ffff:ffff:ffff:ffff,US -2620:e2:4000::,2620:e2:4000:ffff:ffff:ffff:ffff:ffff,US -2620:e2:8000::,2620:e2:8000:ffff:ffff:ffff:ffff:ffff,US -2620:e2:c000::,2620:e2:c000:ffff:ffff:ffff:ffff:ffff,US -2620:e3::,2620:e3::ffff:ffff:ffff:ffff:ffff,US -2620:e3:4000::,2620:e3:4000:ffff:ffff:ffff:ffff:ffff,US -2620:e3:8000::,2620:e3:8000:ffff:ffff:ffff:ffff:ffff,US -2620:e3:c000::,2620:e3:c000:ffff:ffff:ffff:ffff:ffff,US -2620:e4::,2620:e4::ffff:ffff:ffff:ffff:ffff,US -2620:e4:4000::,2620:e4:4000:ffff:ffff:ffff:ffff:ffff,US -2620:e4:8000::,2620:e4:8000:ffff:ffff:ffff:ffff:ffff,US -2620:e4:c000::,2620:e4:c000:ffff:ffff:ffff:ffff:ffff,US -2620:e5::,2620:e5::ffff:ffff:ffff:ffff:ffff,US -2620:e5:4000::,2620:e5:4000:ffff:ffff:ffff:ffff:ffff,US -2620:e5:8000::,2620:e5:8000:ffff:ffff:ffff:ffff:ffff,US -2620:e5:c000::,2620:e5:c000:ffff:ffff:ffff:ffff:ffff,US -2620:e6::,2620:e6::ffff:ffff:ffff:ffff:ffff,US -2620:e6:4000::,2620:e6:4000:ffff:ffff:ffff:ffff:ffff,US -2620:e6:8000::,2620:e6:8000:ffff:ffff:ffff:ffff:ffff,US -2620:e6:c000::,2620:e6:c000:ffff:ffff:ffff:ffff:ffff,US -2620:e7::,2620:e7::ffff:ffff:ffff:ffff:ffff,US -2620:e7:4000::,2620:e7:4000:ffff:ffff:ffff:ffff:ffff,US -2620:e7:8000::,2620:e7:8000:ffff:ffff:ffff:ffff:ffff,CA -2620:e7:c000::,2620:e7:c000:ffff:ffff:ffff:ffff:ffff,US -2620:e8::,2620:e8::ffff:ffff:ffff:ffff:ffff,US -2620:e8:4000::,2620:e8:4000:ffff:ffff:ffff:ffff:ffff,US -2620:e8:8000::,2620:e8:8000:ffff:ffff:ffff:ffff:ffff,US -2620:e8:c000::,2620:e8:c000:ffff:ffff:ffff:ffff:ffff,US -2620:e9::,2620:e9::ffff:ffff:ffff:ffff:ffff,US -2620:e9:4000::,2620:e9:4000:ffff:ffff:ffff:ffff:ffff,US -2620:e9:8000::,2620:e9:8000:ffff:ffff:ffff:ffff:ffff,US -2620:e9:c000::,2620:e9:c000:ffff:ffff:ffff:ffff:ffff,US -2620:ea::,2620:ea:f:ffff:ffff:ffff:ffff:ffff,US -2620:ea:4000::,2620:ea:4000:ffff:ffff:ffff:ffff:ffff,US -2620:ea:8000::,2620:ea:8000:ffff:ffff:ffff:ffff:ffff,US -2620:eb::,2620:eb::ffff:ffff:ffff:ffff:ffff,US -2620:eb:4000::,2620:eb:4000:ffff:ffff:ffff:ffff:ffff,US -2620:eb:8000::,2620:eb:8000:ffff:ffff:ffff:ffff:ffff,US -2620:eb:c000::,2620:eb:c000:ffff:ffff:ffff:ffff:ffff,US -2620:ec::,2620:ec::ffff:ffff:ffff:ffff:ffff,US -2620:ec:4000::,2620:ec:4000:ffff:ffff:ffff:ffff:ffff,US -2620:ec:8000::,2620:ec:8000:ffff:ffff:ffff:ffff:ffff,US -2620:ec:c000::,2620:ec:c000:ffff:ffff:ffff:ffff:ffff,US -2620:ed::,2620:ed::ffff:ffff:ffff:ffff:ffff,US -2620:ed:4000::,2620:ed:4000:ffff:ffff:ffff:ffff:ffff,CA -2620:ed:8000::,2620:ed:8000:ffff:ffff:ffff:ffff:ffff,US -2620:ed:c000::,2620:ed:c000:ffff:ffff:ffff:ffff:ffff,US -2620:ee::,2620:ee::ffff:ffff:ffff:ffff:ffff,US -2620:ee:4000::,2620:ee:4000:ffff:ffff:ffff:ffff:ffff,US -2620:ee:8000::,2620:ee:8000:ffff:ffff:ffff:ffff:ffff,US -2620:ee:c000::,2620:ee:c00f:ffff:ffff:ffff:ffff:ffff,US -2620:ef:4000::,2620:ef:4000:ffff:ffff:ffff:ffff:ffff,US -2620:ef:8000::,2620:ef:8000:ffff:ffff:ffff:ffff:ffff,US -2620:ef:c000::,2620:ef:c000:ffff:ffff:ffff:ffff:ffff,US -2620:f0::,2620:f0::ffff:ffff:ffff:ffff:ffff,US -2620:f0:4000::,2620:f0:400f:ffff:ffff:ffff:ffff:ffff,US -2620:f0:8000::,2620:f0:8000:ffff:ffff:ffff:ffff:ffff,US -2620:f0:c000::,2620:f0:c002:ffff:ffff:ffff:ffff:ffff,US -2620:f0:c003::,2620:f0:c003:ffff:ffff:ffff:ffff:ffff,NL -2620:f0:c004::,2620:f0:c004:ffff:ffff:ffff:ffff:ffff,US -2620:f0:c005::,2620:f0:c005:ffff:ffff:ffff:ffff:ffff,SG -2620:f0:c006::,2620:f0:c009:ffff:ffff:ffff:ffff:ffff,US -2620:f0:c00a::,2620:f0:c00a:ffff:ffff:ffff:ffff:ffff,CA -2620:f0:c00b::,2620:f0:c00f:ffff:ffff:ffff:ffff:ffff,US -2620:f1:4000::,2620:f1:4000:ffff:ffff:ffff:ffff:ffff,CA -2620:f1:8000::,2620:f1:8000:ffff:ffff:ffff:ffff:ffff,US -2620:f1:c000::,2620:f1:c000:ffff:ffff:ffff:ffff:ffff,US -2620:f2::,2620:f2::ffff:ffff:ffff:ffff:ffff,CA -2620:f2:4000::,2620:f2:4000:ffff:ffff:ffff:ffff:ffff,US -2620:f2:8000::,2620:f2:8000:ffff:ffff:ffff:ffff:ffff,US -2620:f2:c000::,2620:f2:c000:ffff:ffff:ffff:ffff:ffff,US -2620:f3::,2620:f3::ffff:ffff:ffff:ffff:ffff,US -2620:f3:4000::,2620:f3:4000:ffff:ffff:ffff:ffff:ffff,US -2620:f3:8000::,2620:f3:8000:ffff:ffff:ffff:ffff:ffff,US -2620:f3:c000::,2620:f3:c000:ffff:ffff:ffff:ffff:ffff,US -2620:f4::,2620:f4::ffff:ffff:ffff:ffff:ffff,US -2620:f4:4000::,2620:f4:40ff:ffff:ffff:ffff:ffff:ffff,US -2620:f4:8000::,2620:f4:8000:ffff:ffff:ffff:ffff:ffff,CA -2620:f4:c000::,2620:f4:c000:ffff:ffff:ffff:ffff:ffff,US -2620:f5::,2620:f5::ffff:ffff:ffff:ffff:ffff,US -2620:f5:4000::,2620:f5:4000:ffff:ffff:ffff:ffff:ffff,US -2620:f5:8000::,2620:f5:8000:ffff:ffff:ffff:ffff:ffff,US -2620:f5:c000::,2620:f5:c000:ffff:ffff:ffff:ffff:ffff,US -2620:f6::,2620:f6::ffff:ffff:ffff:ffff:ffff,CA -2620:f6:4000::,2620:f6:400f:ffff:ffff:ffff:ffff:ffff,US -2620:f6:8000::,2620:f6:8000:ffff:ffff:ffff:ffff:ffff,US -2620:f6:c000::,2620:f6:c000:ffff:ffff:ffff:ffff:ffff,CA -2620:f7::,2620:f7::ffff:ffff:ffff:ffff:ffff,US -2620:f7:4000::,2620:f7:4000:ffff:ffff:ffff:ffff:ffff,US -2620:f7:8000::,2620:f7:8000:ffff:ffff:ffff:ffff:ffff,US -2620:f7:c000::,2620:f7:c000:ffff:ffff:ffff:ffff:ffff,US -2620:f8::,2620:f8::ffff:ffff:ffff:ffff:ffff,US -2620:f8:4000::,2620:f8:4000:ffff:ffff:ffff:ffff:ffff,US -2620:f8:8000::,2620:f8:8000:ffff:ffff:ffff:ffff:ffff,US -2620:f8:c000::,2620:f8:c000:ffff:ffff:ffff:ffff:ffff,US -2620:f9::,2620:f9:f:ffff:ffff:ffff:ffff:ffff,US -2620:f9:4000::,2620:f9:4000:ffff:ffff:ffff:ffff:ffff,US -2620:f9:8000::,2620:f9:8000:ffff:ffff:ffff:ffff:ffff,US -2620:f9:c000::,2620:f9:c000:ffff:ffff:ffff:ffff:ffff,US -2620:fa::,2620:fa::ffff:ffff:ffff:ffff:ffff,US -2620:fa:4000::,2620:fa:4000:ffff:ffff:ffff:ffff:ffff,US -2620:fa:8000::,2620:fa:8000:ffff:ffff:ffff:ffff:ffff,CA -2620:fa:c000::,2620:fa:c000:ffff:ffff:ffff:ffff:ffff,US -2620:fb::,2620:fb::ffff:ffff:ffff:ffff:ffff,US -2620:fb:4000::,2620:fb:4000:ffff:ffff:ffff:ffff:ffff,US -2620:fb:8000::,2620:fb:8000:ffff:ffff:ffff:ffff:ffff,US -2620:fc::,2620:fc::ffff:ffff:ffff:ffff:ffff,CA -2620:fc:4000::,2620:fc:4000:ffff:ffff:ffff:ffff:ffff,CA -2620:fc:8000::,2620:fc:8000:ffff:ffff:ffff:ffff:ffff,US -2620:fc:c000::,2620:fc:c000:ffff:ffff:ffff:ffff:ffff,US -2620:fd::,2620:fd::ffff:ffff:ffff:ffff:ffff,CA -2620:fd:4000::,2620:fd:4000:ffff:ffff:ffff:ffff:ffff,US -2620:fd:8000::,2620:fd:8000:ffff:ffff:ffff:ffff:ffff,US -2620:fd:c000::,2620:fd:c000:ffff:ffff:ffff:ffff:ffff,CA -2620:fe::,2620:fe::ffff:ffff:ffff:ffff:ffff,US -2620:fe:2040::,2620:fe:2040:ffff:ffff:ffff:ffff:ffff,US -2620:fe:8000::,2620:fe:8000:ffff:ffff:ffff:ffff:ffff,US -2620:fe:c000::,2620:fe:c000:ffff:ffff:ffff:ffff:ffff,US -2620:ff::,2620:ff::ffff:ffff:ffff:ffff:ffff,US -2620:ff:4000::,2620:ff:4000:ffff:ffff:ffff:ffff:ffff,US -2620:ff:8000::,2620:ff:8000:ffff:ffff:ffff:ffff:ffff,US -2620:ff:c000::,2620:ff:c000:ffff:ffff:ffff:ffff:ffff,US -2620:100::,2620:100:f:ffff:ffff:ffff:ffff:ffff,US -2620:100:3000::,2620:100:3007:ffff:ffff:ffff:ffff:ffff,US -2620:100:4000::,2620:100:403f:ffff:ffff:ffff:ffff:ffff,US -2620:100:5000::,2620:100:5007:ffff:ffff:ffff:ffff:ffff,US -2620:100:6000::,2620:100:60ff:ffff:ffff:ffff:ffff:ffff,US -2620:100:7000::,2620:100:700f:ffff:ffff:ffff:ffff:ffff,US -2620:100:8000::,2620:100:8003:ffff:ffff:ffff:ffff:ffff,US -2620:100:9000::,2620:100:900f:ffff:ffff:ffff:ffff:ffff,US -2620:100:a000::,2620:100:a00f:ffff:ffff:ffff:ffff:ffff,US -2620:100:c000::,2620:100:c03f:ffff:ffff:ffff:ffff:ffff,US -2620:100:d000::,2620:100:d00f:ffff:ffff:ffff:ffff:ffff,US -2620:100:e000::,2620:100:e00f:ffff:ffff:ffff:ffff:ffff,US -2620:100:f000::,2620:100:f00f:ffff:ffff:ffff:ffff:ffff,US -2620:101::,2620:101:3:ffff:ffff:ffff:ffff:ffff,US -2620:101:1000::,2620:101:103f:ffff:ffff:ffff:ffff:ffff,US -2620:101:2000::,2620:101:201f:ffff:ffff:ffff:ffff:ffff,US -2620:101:3000::,2620:101:303f:ffff:ffff:ffff:ffff:ffff,US -2620:101:4000::,2620:101:403f:ffff:ffff:ffff:ffff:ffff,US -2620:101:5000::,2620:101:503f:ffff:ffff:ffff:ffff:ffff,US -2620:101:6000::,2620:101:6001:ffff:ffff:ffff:ffff:ffff,US -2620:101:7000::,2620:101:7001:ffff:ffff:ffff:ffff:ffff,US -2620:101:8000::,2620:101:80f1:ffff:ffff:ffff:ffff:ffff,US -2620:101:80f2::,2620:101:80f2:7fff:ffff:ffff:ffff:ffff,CA -2620:101:80f2:8000::,2620:101:80ff:ffff:ffff:ffff:ffff:ffff,US -2620:101:9000::,2620:101:900f:ffff:ffff:ffff:ffff:ffff,US -2620:101:b000::,2620:101:b07f:ffff:ffff:ffff:ffff:ffff,US -2620:101:c000::,2620:101:c0ff:ffff:ffff:ffff:ffff:ffff,CA -2620:101:d000::,2620:101:d007:ffff:ffff:ffff:ffff:ffff,US -2620:101:e000::,2620:101:e00f:ffff:ffff:ffff:ffff:ffff,US -2620:101:f000::,2620:101:f001:ffff:ffff:ffff:ffff:ffff,CA -2620:102::,2620:102:f:ffff:ffff:ffff:ffff:ffff,US -2620:102:2000::,2620:102:200f:ffff:ffff:ffff:ffff:ffff,US -2620:102:3000::,2620:102:300f:ffff:ffff:ffff:ffff:ffff,US -2620:102:4000::,2620:102:403f:ffff:ffff:ffff:ffff:ffff,US -2a07:14c0::,2a07:14c7:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:1500::,2a07:1507:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:1540::,2a07:1547:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:1580::,2a07:1587:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:15c0::,2a07:15c7:ffff:ffff:ffff:ffff:ffff:ffff,TR -2a07:1600::,2a07:1607:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:1640::,2a07:1647:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:1680::,2a07:1687:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:16c0::,2a07:16c7:ffff:ffff:ffff:ffff:ffff:ffff,ES -2a07:1700::,2a07:1707:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:1740::,2a07:1747:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:1780::,2a07:1787:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:17c0::,2a07:17c7:ffff:ffff:ffff:ffff:ffff:ffff,UA -2a07:1800::,2a07:1807:ffff:ffff:ffff:ffff:ffff:ffff,AT -2a07:1840::,2a07:1847:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:1880::,2a07:1887:ffff:ffff:ffff:ffff:ffff:ffff,IT -2a07:18c0::,2a07:18c7:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:1900::,2a07:1907:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:1940::,2a07:1947:ffff:ffff:ffff:ffff:ffff:ffff,IT -2a07:1980::,2a07:1987:ffff:ffff:ffff:ffff:ffff:ffff,IL -2a07:19c0::,2a07:19c7:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:1a00::,2a07:1a07:ffff:ffff:ffff:ffff:ffff:ffff,ES -2a07:1a40::,2a07:1a47:ffff:ffff:ffff:ffff:ffff:ffff,PL -2a07:1a80::,2a07:1a80:6fff:ffff:ffff:ffff:ffff:ffff,SE -2a07:1a80:7000::,2a07:1a80:70ff:ffff:ffff:ffff:ffff:ffff,AT -2a07:1a80:7100::,2a07:1a87:ffff:ffff:ffff:ffff:ffff:ffff,SE -2a07:1ac0::,2a07:1ac7:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:1b00::,2a07:1b07:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:1b40::,2a07:1b47:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:1b80::,2a07:1b87:ffff:ffff:ffff:ffff:ffff:ffff,AT -2a07:1bc0::,2a07:1bc7:ffff:ffff:ffff:ffff:ffff:ffff,PL -2a07:1c00::,2a07:1c07:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:1c40::,2a07:1c44:3ff:ffff:ffff:ffff:ffff:ffff,AT -2a07:1c44:400::,2a07:1c44:4ff:ffff:ffff:ffff:ffff:ffff,DE -2a07:1c44:500::,2a07:1c44:609:ffff:ffff:ffff:ffff:ffff,AT -2a07:1c44:60a::,2a07:1c44:60a:ffff:ffff:ffff:ffff:ffff,DE -2a07:1c44:60b::,2a07:1c44:619:ffff:ffff:ffff:ffff:ffff,AT -2a07:1c44:61a::,2a07:1c44:61a:ffff:ffff:ffff:ffff:ffff,KR -2a07:1c44:61b::,2a07:1c44:67f:ffff:ffff:ffff:ffff:ffff,AT -2a07:1c44:680::,2a07:1c44:6bf:ffff:ffff:ffff:ffff:ffff,KR -2a07:1c44:6c0::,2a07:1c44:6ff:ffff:ffff:ffff:ffff:ffff,DE -2a07:1c44:700::,2a07:1c44:70f:ffff:ffff:ffff:ffff:ffff,US -2a07:1c44:710::,2a07:1c44:1800:ffff:ffff:ffff:ffff:ffff,AT -2a07:1c44:1801::,2a07:1c44:1802:ffff:ffff:ffff:ffff:ffff,US -2a07:1c44:1803::,2a07:1c44:35ff:ffff:ffff:ffff:ffff:ffff,AT -2a07:1c44:3600::,2a07:1c44:36ff:ffff:ffff:ffff:ffff:ffff,GB -2a07:1c44:3700::,2a07:1c44:3fff:ffff:ffff:ffff:ffff:ffff,AT -2a07:1c44:4000::,2a07:1c44:40ff:ffff:ffff:ffff:ffff:ffff,US -2a07:1c44:4100::,2a07:1c44:42ff:ffff:ffff:ffff:ffff:ffff,AT -2a07:1c44:4300::,2a07:1c44:43ff:ffff:ffff:ffff:ffff:ffff,HR -2a07:1c44:4400::,2a07:1c44:4fff:ffff:ffff:ffff:ffff:ffff,AT -2a07:1c44:5000::,2a07:1c44:51ff:ffff:ffff:ffff:ffff:ffff,US -2a07:1c44:5200::,2a07:1c47:ffff:ffff:ffff:ffff:ffff:ffff,AT -2a07:1c80::,2a07:1c87:ffff:ffff:ffff:ffff:ffff:ffff,SE -2a07:1cc0::,2a07:1cc7:ffff:ffff:ffff:ffff:ffff:ffff,PL -2a07:1d00::,2a07:1d07:ffff:ffff:ffff:ffff:ffff:ffff,IR -2a07:1d40::,2a07:1d47:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:1d80::,2a07:1d87:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:1dc0::,2a07:1dc7:ffff:ffff:ffff:ffff:ffff:ffff,PL -2a07:1e00::,2a07:1e07:ffff:ffff:ffff:ffff:ffff:ffff,KZ -2a07:1e40::,2a07:1e47:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:1e80::,2a07:1e87:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:1ec0::,2a07:1ec7:ffff:ffff:ffff:ffff:ffff:ffff,IT -2a07:1f00::,2a07:1f07:ffff:ffff:ffff:ffff:ffff:ffff,CH -2a07:1f40::,2a07:1f47:ffff:ffff:ffff:ffff:ffff:ffff,CZ -2a07:1f80::,2a07:1f87:ffff:ffff:ffff:ffff:ffff:ffff,US -2a07:1fc0::,2a07:1fc7:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:2000::,2a07:2007:ffff:ffff:ffff:ffff:ffff:ffff,IQ -2a07:2040::,2a07:2047:ffff:ffff:ffff:ffff:ffff:ffff,ES -2a07:2080::,2a07:2087:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:20c0::,2a07:20c7:ffff:ffff:ffff:ffff:ffff:ffff,CZ -2a07:2100::,2a07:2107:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:2140::,2a07:2147:ffff:ffff:ffff:ffff:ffff:ffff,CH -2a07:2180::,2a07:2187:ffff:ffff:ffff:ffff:ffff:ffff,SE -2a07:21c0::,2a07:21c7:ffff:ffff:ffff:ffff:ffff:ffff,TR -2a07:2200::,2a07:2207:ffff:ffff:ffff:ffff:ffff:ffff,IR -2a07:2240::,2a07:2247:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:2280::,2a07:2287:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:2300::,2a07:2307:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:2340::,2a07:2347:ffff:ffff:ffff:ffff:ffff:ffff,SE -2a07:2380::,2a07:2387:ffff:ffff:ffff:ffff:ffff:ffff,ES -2a07:23c0::,2a07:23c7:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:2400::,2a07:2407:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:2440::,2a07:2447:ffff:ffff:ffff:ffff:ffff:ffff,SE -2a07:2480::,2a07:2487:ffff:ffff:ffff:ffff:ffff:ffff,IR -2a07:24c0::,2a07:24c7:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:2500::,2a07:2507:ffff:ffff:ffff:ffff:ffff:ffff,DK -2a07:2540::,2a07:2547:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:2580::,2a07:2587:ffff:ffff:ffff:ffff:ffff:ffff,IT -2a07:25c0::,2a07:25c7:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:2600::,2a07:2607:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:2640::,2a07:2647:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:2680::,2a07:2687:ffff:ffff:ffff:ffff:ffff:ffff,DK -2a07:26c0::,2a07:26c7:ffff:ffff:ffff:ffff:ffff:ffff,BE -2a07:2700::,2a07:2707:ffff:ffff:ffff:ffff:ffff:ffff,TR -2a07:2740::,2a07:2747:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:2780::,2a07:2787:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:27c0::,2a07:27c7:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:2800::,2a07:2807:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:2840::,2a07:2847:ffff:ffff:ffff:ffff:ffff:ffff,IT -2a07:2880::,2a07:2887:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:28c0::,2a07:28c7:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:2900::,2a07:291f:ffff:ffff:ffff:ffff:ffff:ffff,CH -2a07:2a00::,2a07:2a07:ffff:ffff:ffff:ffff:ffff:ffff,ES -2a07:2a40::,2a07:2a47:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:2a80::,2a07:2a87:ffff:ffff:ffff:ffff:ffff:ffff,SE -2a07:2ac0::,2a07:2ac7:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:2b00::,2a07:2b07:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:2b40::,2a07:2b47:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:2b80::,2a07:2b87:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:2bc0::,2a07:2bc7:ffff:ffff:ffff:ffff:ffff:ffff,ES -2a07:2c00::,2a07:2c07:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:2c40::,2a07:2c47:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:2c80::,2a07:2c87:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:2cc0::,2a07:2cc7:ffff:ffff:ffff:ffff:ffff:ffff,IT -2a07:2d00::,2a07:2d07:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:2d40::,2a07:2d47:ffff:ffff:ffff:ffff:ffff:ffff,CH -2a07:2d80::,2a07:2d87:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:2dc0::,2a07:2dc7:ffff:ffff:ffff:ffff:ffff:ffff,FI -2a07:2e00::,2a07:2e07:ffff:ffff:ffff:ffff:ffff:ffff,CH -2a07:2e40::,2a07:2e47:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:2e80::,2a07:2e87:ffff:ffff:ffff:ffff:ffff:ffff,IT -2a07:2ec0::,2a07:2ec7:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:2f00::,2a07:2f07:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:2f40::,2a07:2f47:ffff:ffff:ffff:ffff:ffff:ffff,UA -2a07:2f80::,2a07:2f87:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:2fc0::,2a07:2fc7:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:3000::,2a07:3007:ffff:ffff:ffff:ffff:ffff:ffff,ES -2a07:3040::,2a07:3047:ffff:ffff:ffff:ffff:ffff:ffff,PL -2a07:3080::,2a07:3087:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:30c0::,2a07:30c7:ffff:ffff:ffff:ffff:ffff:ffff,CZ -2a07:3100::,2a07:3107:ffff:ffff:ffff:ffff:ffff:ffff,RO -2a07:3140::,2a07:3147:ffff:ffff:ffff:ffff:ffff:ffff,BE -2a07:3180::,2a07:3187:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:31c0::,2a07:31c7:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:3200::,2a07:3207:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:3240::,2a07:3247:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:3280::,2a07:3287:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:32c0::,2a07:32c7:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:3300::,2a07:3307:ffff:ffff:ffff:ffff:ffff:ffff,TR -2a07:3340::,2a07:3347:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:3380::,2a07:3387:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:33c0::,2a07:33c7:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:3400::,2a07:3407:ffff:ffff:ffff:ffff:ffff:ffff,UA -2a07:3440::,2a07:3447:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:3480::,2a07:3487:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:3500::,2a07:3507:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:3540::,2a07:3547:ffff:ffff:ffff:ffff:ffff:ffff,IT -2a07:3580::,2a07:3587:ffff:ffff:ffff:ffff:ffff:ffff,ES -2a07:35c0::,2a07:35c7:ffff:ffff:ffff:ffff:ffff:ffff,UA -2a07:3600::,2a07:3607:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:3640::,2a07:3647:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:3680::,2a07:3687:ffff:ffff:ffff:ffff:ffff:ffff,LB -2a07:36c0::,2a07:36c7:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:3700::,2a07:3707:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:3740::,2a07:3747:ffff:ffff:ffff:ffff:ffff:ffff,AT -2a07:3780::,2a07:3787:ffff:ffff:ffff:ffff:ffff:ffff,IS -2a07:37c0::,2a07:37c7:ffff:ffff:ffff:ffff:ffff:ffff,BE -2a07:3800::,2a07:3807:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:3840::,2a07:3847:ffff:ffff:ffff:ffff:ffff:ffff,HR -2a07:3880::,2a07:3887:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:38c0::,2a07:38c7:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:3900::,2a07:3907:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:3940::,2a07:3947:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:3980::,2a07:3987:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:39c0::,2a07:39c7:ffff:ffff:ffff:ffff:ffff:ffff,DK -2a07:3a00::,2a07:3a07:ffff:ffff:ffff:ffff:ffff:ffff,ES -2a07:3a80::,2a07:3a87:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:3ac0::,2a07:3ac7:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:3b00::,2a07:3b07:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:3b40::,2a07:3b47:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:3b80::,2a07:3b87:ffff:ffff:ffff:ffff:ffff:ffff,GI -2a07:3bc0::,2a07:3bc7:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:3c00::,2a07:3c07:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:3c40::,2a07:3c47:ffff:ffff:ffff:ffff:ffff:ffff,DE -2a07:3c80::,2a07:3c87:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:3d00::,2a07:3d07:ffff:ffff:ffff:ffff:ffff:ffff,IT -2a07:3d80::,2a07:3d87:ffff:ffff:ffff:ffff:ffff:ffff,CZ -2a07:3dc0::,2a07:3dc7:ffff:ffff:ffff:ffff:ffff:ffff,DK -2a07:3e00::,2a07:3e07:ffff:ffff:ffff:ffff:ffff:ffff,CH -2a07:3e40::,2a07:3e47:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:3e80::,2a07:3e87:ffff:ffff:ffff:ffff:ffff:ffff,NL -2a07:3ec0::,2a07:3ec7:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:3f00::,2a07:3f07:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:3f40::,2a07:3f47:ffff:ffff:ffff:ffff:ffff:ffff,IE -2a07:3f80::,2a07:3f87:ffff:ffff:ffff:ffff:ffff:ffff,SK -2a07:3fc0::,2a07:3fc7:ffff:ffff:ffff:ffff:ffff:ffff,SE -2a07:4000::,2a07:4007:ffff:ffff:ffff:ffff:ffff:ffff,NO -2a07:4040::,2a07:4047:ffff:ffff:ffff:ffff:ffff:ffff,SE -2a07:4080::,2a07:4087:ffff:ffff:ffff:ffff:ffff:ffff,AT -2a07:40c0::,2a07:40c7:ffff:ffff:ffff:ffff:ffff:ffff,IL -2a07:4100::,2a07:4107:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:4140::,2a07:4147:ffff:ffff:ffff:ffff:ffff:ffff,MD -2a07:4180::,2a07:4187:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:41c0::,2a07:41c7:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:4200::,2a07:4207:ffff:ffff:ffff:ffff:ffff:ffff,FR -2a07:4240::,2a07:4247:ffff:ffff:ffff:ffff:ffff:ffff,RU -2a07:4280::,2a07:4287:ffff:ffff:ffff:ffff:ffff:ffff,GB -2a07:42c0::,2a07:42c7:ffff:ffff:ffff:ffff:ffff:ffff,DK -2a07:4340::,2a07:4347:ffff:ffff:ffff:ffff:ffff:ffff,AE -2a0c:af80::,2a0c:af87:ffff:ffff:ffff:ffff:ffff:ffff,GB -2c0f:f950::,2c0f:f950:ffff:ffff:ffff:ffff:ffff:ffff,SS -2c0f:f958::,2c0f:f958:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:f960::,2c0f:f960:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:f968::,2c0f:f968:ffff:ffff:ffff:ffff:ffff:ffff,MZ -2c0f:f970::,2c0f:f970:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:f978::,2c0f:f978:ffff:ffff:ffff:ffff:ffff:ffff,CD -2c0f:f980::,2c0f:f980:ffff:ffff:ffff:ffff:ffff:ffff,NA -2c0f:f988::,2c0f:f988:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:f990::,2c0f:f990:ffff:ffff:ffff:ffff:ffff:ffff,GN -2c0f:f998::,2c0f:f998:ffff:ffff:ffff:ffff:ffff:ffff,MR -2c0f:f9a0::,2c0f:f9a0:ffff:ffff:ffff:ffff:ffff:ffff,MW -2c0f:f9a8::,2c0f:f9a8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:f9b0::,2c0f:f9b0:ffff:ffff:ffff:ffff:ffff:ffff,GA -2c0f:f9b8::,2c0f:f9b8:1:ffff:ffff:ffff:ffff:ffff,MU -2c0f:f9b8:2::,2c0f:f9b8:2:ffff:ffff:ffff:ffff:ffff,US -2c0f:f9b8:3::,2c0f:f9b8:ffff:ffff:ffff:ffff:ffff:ffff,MU -2c0f:f9c0::,2c0f:f9c0:ffff:ffff:ffff:ffff:ffff:ffff,BW -2c0f:f9c8::,2c0f:f9c8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:f9d0::,2c0f:f9d0:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:f9d8::,2c0f:f9d8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:f9e0::,2c0f:f9e0:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:f9e8::,2c0f:f9e8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:f9f0::,2c0f:f9f0:ffff:ffff:ffff:ffff:ffff:ffff,MG -2c0f:f9f8::,2c0f:f9f8:ffff:ffff:ffff:ffff:ffff:ffff,BJ -2c0f:fa00::,2c0f:fa00:ffff:ffff:ffff:ffff:ffff:ffff,GH -2c0f:fa08::,2c0f:fa08:ffff:ffff:ffff:ffff:ffff:ffff,CD -2c0f:fa10::,2c0f:fa10:fffc:ffff:ffff:ffff:ffff:ffff,MU -2c0f:fa10:fffd::,2c0f:fa10:fffd:7fff:ffff:ffff:ffff:ffff,ZM -2c0f:fa10:fffd:8000::,2c0f:fa10:ffff:ffff:ffff:ffff:ffff:ffff,MU -2c0f:fa18::,2c0f:fa18:ffff:ffff:ffff:ffff:ffff:ffff,MA -2c0f:fa20::,2c0f:fa20:ffff:ffff:ffff:ffff:ffff:ffff,SS -2c0f:fa28::,2c0f:fa28:ffff:ffff:ffff:ffff:ffff:ffff,MG -2c0f:fa38::,2c0f:fa38:ffff:ffff:ffff:ffff:ffff:ffff,AO -2c0f:fa40::,2c0f:fa40:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fa48::,2c0f:fa48:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fa58::,2c0f:fa58:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fa60::,2c0f:fa60:ffff:ffff:ffff:ffff:ffff:ffff,AO -2c0f:fa68::,2c0f:fa68:ffff:ffff:ffff:ffff:ffff:ffff,GH -2c0f:fa70::,2c0f:fa70:ffff:ffff:ffff:ffff:ffff:ffff,AO -2c0f:fa78::,2c0f:fa78:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fa80::,2c0f:fa80:ffff:ffff:ffff:ffff:ffff:ffff,AO -2c0f:fa88::,2c0f:fa88:ffff:ffff:ffff:ffff:ffff:ffff,ST -2c0f:fa90::,2c0f:fa90:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fa98::,2c0f:fa98:ffff:ffff:ffff:ffff:ffff:ffff,ZW -2c0f:faa0::,2c0f:faa7:ffff:ffff:ffff:ffff:ffff:ffff,SD -2c0f:fab0::,2c0f:fabf:ffff:ffff:ffff:ffff:ffff:ffff,TN -2c0f:fac0::,2c0f:fac0:ffff:ffff:ffff:ffff:ffff:ffff,MW -2c0f:fac8::,2c0f:fac8:ffff:ffff:ffff:ffff:ffff:ffff,BW -2c0f:fad8::,2c0f:fad8:ffff:ffff:ffff:ffff:ffff:ffff,CM -2c0f:fae0::,2c0f:fae0:ffff:ffff:ffff:ffff:ffff:ffff,CM -2c0f:fae8::,2c0f:fae8:ffff:ffff:ffff:ffff:ffff:ffff,KE -2c0f:faf0::,2c0f:faf0:ffff:ffff:ffff:ffff:ffff:ffff,AO -2c0f:faf8::,2c0f:faf8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fb00::,2c0f:fb00:ffff:ffff:ffff:ffff:ffff:ffff,UG -2c0f:fb08::,2c0f:fb08:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fb10::,2c0f:fb10:ffff:ffff:ffff:ffff:ffff:ffff,LY -2c0f:fb18::,2c0f:fb18:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fb20::,2c0f:fb20:ffff:ffff:ffff:ffff:ffff:ffff,MA -2c0f:fb30::,2c0f:fb30:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fb38::,2c0f:fb38:ffff:ffff:ffff:ffff:ffff:ffff,SO -2c0f:fb40::,2c0f:fb40:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fb48::,2c0f:fb48:ffff:ffff:ffff:ffff:ffff:ffff,MZ -2c0f:fb50::,2c0f:fb50:ffff:ffff:ffff:ffff:ffff:ffff,KE -2c0f:fb58::,2c0f:fb58:ffff:ffff:ffff:ffff:ffff:ffff,AO -2c0f:fb60::,2c0f:fb60:ffff:ffff:ffff:ffff:ffff:ffff,KE -2c0f:fb68::,2c0f:fb68:ffff:ffff:ffff:ffff:ffff:ffff,LS -2c0f:fb70::,2c0f:fb70:ffff:ffff:ffff:ffff:ffff:ffff,AO -2c0f:fb78::,2c0f:fb78:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:fb80::,2c0f:fb80:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fb88::,2c0f:fb88:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:fb90::,2c0f:fb90:ffff:ffff:ffff:ffff:ffff:ffff,MZ -2c0f:fb98::,2c0f:fb98:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:fba0::,2c0f:fba0:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fba8::,2c0f:fba8:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:fbb0::,2c0f:fbb0:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fbb8::,2c0f:fbb8:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:fbc0::,2c0f:fbc0:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fbc8::,2c0f:fbc8:ffff:ffff:ffff:ffff:ffff:ffff,UG -2c0f:fbd0::,2c0f:fbd0:ffff:ffff:ffff:ffff:ffff:ffff,GN -2c0f:fbd8::,2c0f:fbd8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fbe0::,2c0f:fc1f:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fc40::,2c0f:fc40:ffff:ffff:ffff:ffff:ffff:ffff,EG -2c0f:fc48::,2c0f:fc48:ffff:ffff:ffff:ffff:ffff:ffff,MW -2c0f:fc58::,2c0f:fc58:ffff:ffff:ffff:ffff:ffff:ffff,MW -2c0f:fc60::,2c0f:fc61:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:fc68::,2c0f:fc68:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fc70::,2c0f:fc70:ffff:ffff:ffff:ffff:ffff:ffff,KE -2c0f:fc80::,2c0f:fc80:ffff:ffff:ffff:ffff:ffff:ffff,KE -2c0f:fc88::,2c0f:fc89:ffff:ffff:ffff:ffff:ffff:ffff,EG -2c0f:fc90::,2c0f:fc90:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:fc98::,2c0f:fc98:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:fca0::,2c0f:fca0:ffff:ffff:ffff:ffff:ffff:ffff,GH -2c0f:fca8::,2c0f:fca8:ffff:ffff:ffff:ffff:ffff:ffff,GH -2c0f:fcb0::,2c0f:fcb0:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fcb8::,2c0f:fcb8:ffff:ffff:ffff:ffff:ffff:ffff,GM -2c0f:fcc8::,2c0f:fcc8:ffff:ffff:ffff:ffff:ffff:ffff,ZM -2c0f:fcd0::,2c0f:fcd0:ffff:ffff:ffff:ffff:ffff:ffff,ZM -2c0f:fcd8::,2c0f:fcd8:ffff:ffff:ffff:ffff:ffff:ffff,SO -2c0f:fce0::,2c0f:fce0:ffff:ffff:ffff:ffff:ffff:ffff,KE -2c0f:fce8::,2c0f:fce8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fcf0::,2c0f:fcf0:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:fcf8::,2c0f:fcf8:ffff:ffff:ffff:ffff:ffff:ffff,GH -2c0f:fd00::,2c0f:fd00:ffff:ffff:ffff:ffff:ffff:ffff,LS -2c0f:fd08::,2c0f:fd08:ffff:ffff:ffff:ffff:ffff:ffff,GM -2c0f:fd10::,2c0f:fd10:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:fd18::,2c0f:fd18:ffff:ffff:ffff:ffff:ffff:ffff,SC -2c0f:fd20::,2c0f:fd20:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:fd28::,2c0f:fd28:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:fd30::,2c0f:fd30:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:fd38::,2c0f:fd38:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:fd40::,2c0f:fd40:ffff:ffff:ffff:ffff:ffff:ffff,ZM -2c0f:fd48::,2c0f:fd48:ffff:ffff:ffff:ffff:ffff:ffff,ZW -2c0f:fd50::,2c0f:fd50:ffff:ffff:ffff:ffff:ffff:ffff,MW -2c0f:fd58::,2c0f:fd58:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fd60::,2c0f:fd60:ffff:ffff:ffff:ffff:ffff:ffff,UG -2c0f:fd68::,2c0f:fd68:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fd78::,2c0f:fd78:ffff:ffff:ffff:ffff:ffff:ffff,BI -2c0f:fd80::,2c0f:fd80:ffff:ffff:ffff:ffff:ffff:ffff,BF -2c0f:fd88::,2c0f:fd88:ffff:ffff:ffff:ffff:ffff:ffff,GH -2c0f:fd90::,2c0f:fd90:ffff:ffff:ffff:ffff:ffff:ffff,ZM -2c0f:fd98::,2c0f:fd98:ffff:ffff:ffff:ffff:ffff:ffff,ZM -2c0f:fda0::,2c0f:fda0:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fda8::,2c0f:fda8:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:fdb0::,2c0f:fdb0:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:fdb8::,2c0f:fdb8:ffff:ffff:ffff:ffff:ffff:ffff,UG -2c0f:fdc0::,2c0f:fdc0:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:fdc8::,2c0f:fdc8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fdd0::,2c0f:fdd0:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fdd8::,2c0f:fdd8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fde8::,2c0f:fde8:ffff:ffff:ffff:ffff:ffff:ffff,MW -2c0f:fdf0::,2c0f:fdf0:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fdf8::,2c0f:fdf8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fe08::,2c0f:fe08:ffff:ffff:ffff:ffff:ffff:ffff,KE -2c0f:fe10::,2c0f:fe10:ffff:ffff:ffff:ffff:ffff:ffff,UG -2c0f:fe18::,2c0f:fe18:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fe20::,2c0f:fe20:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fe28::,2c0f:fe28:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fe30::,2c0f:fe30:ffff:ffff:ffff:ffff:ffff:ffff,MU -2c0f:fe38::,2c0f:fe38:ffff:ffff:ffff:ffff:ffff:ffff,KE -2c0f:fe40::,2c0f:fe40:8001:f:ffff:ffff:ffff:ffff,MU -2c0f:fe40:8001:10::,2c0f:fe40:8001:10:ffff:ffff:ffff:ffff,KE -2c0f:fe40:8001:11::,2c0f:fe40:80fe:ffff:ffff:ffff:ffff:ffff,MU -2c0f:fe40:80ff::,2c0f:fe40:80ff:7fff:ffff:ffff:ffff:ffff,KE -2c0f:fe40:80ff:8000::,2c0f:fe40:ffff:ffff:ffff:ffff:ffff:ffff,MU -2c0f:fe50::,2c0f:fe50:ffff:ffff:ffff:ffff:ffff:ffff,DZ -2c0f:fe58::,2c0f:fe58:ffff:ffff:ffff:ffff:ffff:ffff,LS -2c0f:fe60::,2c0f:fe60:ffff:ffff:ffff:ffff:ffff:ffff,RW -2c0f:fe68::,2c0f:fe68:ffff:ffff:ffff:ffff:ffff:ffff,MU -2c0f:fe70::,2c0f:fe70:ffff:ffff:ffff:ffff:ffff:ffff,UG -2c0f:fe78::,2c0f:fe78:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fe88::,2c0f:fe88:ffff:ffff:ffff:ffff:ffff:ffff,KE -2c0f:fe90::,2c0f:fe90:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fe98::,2c0f:fe98:ffff:ffff:ffff:ffff:ffff:ffff,TZ -2c0f:fea0::,2c0f:fea0:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:fea8::,2c0f:fea8:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:feb0::,2c0f:feb0:16:ffff:ffff:ffff:ffff:ffff,MU -2c0f:feb0:17::,2c0f:feb0:17:7fff:ffff:ffff:ffff:ffff,KE -2c0f:feb0:17:8000::,2c0f:feb0:1e:ffff:ffff:ffff:ffff:ffff,MU -2c0f:feb0:1f::,2c0f:feb0:1f:7fff:ffff:ffff:ffff:ffff,ZA -2c0f:feb0:1f:8000::,2c0f:feb0:1f:ffff:ffff:ffff:ffff:ffff,MU -2c0f:feb0:20::,2c0f:feb0:20:7fff:ffff:ffff:ffff:ffff,ZA -2c0f:feb0:20:8000::,2c0f:feb0:2f:7fff:ffff:ffff:ffff:ffff,MU -2c0f:feb0:2f:8000::,2c0f:feb0:2f:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:feb0:30::,2c0f:feb1:ffff:ffff:ffff:ffff:ffff:ffff,MU -2c0f:feb8::,2c0f:feb8:ffff:ffff:ffff:ffff:ffff:ffff,ZM -2c0f:fec0::,2c0f:fec0:ffff:ffff:ffff:ffff:ffff:ffff,UG -2c0f:fec8::,2c0f:fec8:ffff:ffff:ffff:ffff:ffff:ffff,SD -2c0f:fed8::,2c0f:fed8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:fee0::,2c0f:fee0:ffff:ffff:ffff:ffff:ffff:ffff,EG -2c0f:fef0::,2c0f:fef0:ffff:ffff:ffff:ffff:ffff:ffff,SC -2c0f:fef8::,2c0f:fef8:ffff:ffff:ffff:ffff:ffff:ffff,KE -2c0f:ff00::,2c0f:ff00:ffff:ffff:ffff:ffff:ffff:ffff,BW -2c0f:ff08::,2c0f:ff08:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:ff10::,2c0f:ff10:ffff:ffff:ffff:ffff:ffff:ffff,CD -2c0f:ff18::,2c0f:ff18:ffff:ffff:ffff:ffff:ffff:ffff,KE -2c0f:ff20::,2c0f:ff20:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:ff28::,2c0f:ff28:ffff:ffff:ffff:ffff:ffff:ffff,SD -2c0f:ff30::,2c0f:ff30:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:ff40::,2c0f:ff80:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:ff88::,2c0f:ff88:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:ff90::,2c0f:ff90:ffff:ffff:ffff:ffff:ffff:ffff,KE -2c0f:ff98::,2c0f:ff98:ffff:ffff:ffff:ffff:ffff:ffff,UG -2c0f:ffa0::,2c0f:ffa0:ffff:ffff:ffff:ffff:ffff:ffff,UG -2c0f:ffa8::,2c0f:ffa8:ffff:ffff:ffff:ffff:ffff:ffff,LS -2c0f:ffb0::,2c0f:ffb0:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:ffb8::,2c0f:ffb8:ffff:ffff:ffff:ffff:ffff:ffff,SD -2c0f:ffc0::,2c0f:ffc0:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:ffc8::,2c0f:ffc8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:ffd0::,2c0f:ffd0:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:ffd8::,2c0f:ffd8:ffff:ffff:ffff:ffff:ffff:ffff,ZA -2c0f:ffe8::,2c0f:ffe8:ffff:ffff:ffff:ffff:ffff:ffff,NG -2c0f:fff0::,2c0f:fff0:ffff:ffff:ffff:ffff:ffff:ffff,NG diff --git a/client/README.md b/client/README.md deleted file mode 100644 index 50bdba3..0000000 --- a/client/README.md +++ /dev/null @@ -1,20 +0,0 @@ -This is the Tor client component of Snowflake. - -It is based on goptlib. - -### Flags - -The client uses these following `torrc` options by default: -``` -ClientTransportPlugin snowflake exec ./client \ --url https://snowflake-broker.azureedge.net/ \ --front ajax.aspnetcdn.com \ --ice stun:stun.l.google.com:19302 -``` - -`-url` should be the URL of a Broker instance. - -`-front` is an optional front domain for the Broker request. - -`-ice` is a comma-separated list of ICE servers. These can be STUN or TURN -servers. diff --git a/client/client_test.go b/client/client_test.go deleted file mode 100644 index aeaf979..0000000 --- a/client/client_test.go +++ /dev/null @@ -1,59 +0,0 @@ -package main - -import ( - "testing" - - . "github.com/smartystreets/goconvey/convey" -) - -func TestICEServerParser(t *testing.T) { - Convey("Test parsing of ICE servers", t, func() { - for _, test := range []struct { - input string - urls [][]string - length int - }{ - { - "", - nil, - 0, - }, - { - " ", - nil, - 0, - }, - { - "stun:stun.l.google.com:19302", - [][]string{[]string{"stun:stun.l.google.com:19302"}}, - 1, - }, - { - "stun:stun.l.google.com:19302,stun.ekiga.net", - [][]string{[]string{"stun:stun.l.google.com:19302"}, []string{"stun.ekiga.net"}}, - 2, - }, - { - "stun:stun.l.google.com:19302, stun.ekiga.net", - [][]string{[]string{"stun:stun.l.google.com:19302"}, []string{"stun.ekiga.net"}}, - 2, - }, - } { - servers := parseIceServers(test.input) - - if test.urls == nil { - So(servers, ShouldBeNil) - } else { - So(servers, ShouldNotBeNil) - } - - So(len(servers), ShouldEqual, test.length) - - for i, server := range servers { - So(server.URLs, ShouldResemble, test.urls[i]) - } - - } - - }) -} diff --git a/client/lib/interfaces.go b/client/lib/interfaces.go deleted file mode 100644 index f6e8240..0000000 --- a/client/lib/interfaces.go +++ /dev/null @@ -1,55 +0,0 @@ -package lib - -import ( - "io" - "net" -) - -type Connector interface { - Connect() error -} - -type Resetter interface { - Reset() - WaitForReset() -} - -// Interface for a single remote WebRTC peer. -// In the Client context, "Snowflake" refers to the remote browser proxy. -type Snowflake interface { - io.ReadWriteCloser - Resetter - Connector -} - -// Interface for catching Snowflakes. (aka the remote dialer) -type Tongue interface { - Catch() (Snowflake, error) -} - -// Interface for collecting some number of Snowflakes, for passing along -// ultimately to the SOCKS handler. -type SnowflakeCollector interface { - // Add a Snowflake to the collection. - // Implementation should decide how to connect and maintain the webRTCConn. - Collect() (Snowflake, error) - - // Remove and return the most available Snowflake from the collection. - Pop() Snowflake - - // Signal when the collector has stopped collecting. - Melted() <-chan struct{} -} - -// Interface to adapt to goptlib's SocksConn struct. -type SocksConnector interface { - Grant(*net.TCPAddr) error - Reject() error - net.Conn -} - -// Interface for the Snowflake's transport. (Typically just webrtc.DataChannel) -type SnowflakeDataChannel interface { - io.Closer - Send([]byte) error -} diff --git a/client/lib/lib_test.go b/client/lib/lib_test.go deleted file mode 100644 index 1cdc2c6..0000000 --- a/client/lib/lib_test.go +++ /dev/null @@ -1,361 +0,0 @@ -package lib - -import ( - "bytes" - "fmt" - "io/ioutil" - "net" - "net/http" - "sync" - "testing" - - "git.torproject.org/pluggable-transports/snowflake.git/common/util" - "github.com/pion/webrtc/v2" - . "github.com/smartystreets/goconvey/convey" -) - -type MockDataChannel struct { - destination bytes.Buffer - done chan bool -} - -func (m *MockDataChannel) Send(data []byte) error { - m.destination.Write(data) - m.done <- true - return nil -} - -func (*MockDataChannel) Close() error { return nil } - -type MockResponse struct{} - -func (m *MockResponse) Read(p []byte) (int, error) { - p = []byte(`{"type":"answer","sdp":"fake"}`) - return 0, nil -} -func (m *MockResponse) Close() error { return nil } - -type MockTransport struct { - statusOverride int - body []byte -} - -// Just returns a response with fake SDP answer. -func (m *MockTransport) RoundTrip(req *http.Request) (*http.Response, error) { - s := ioutil.NopCloser(bytes.NewReader(m.body)) - r := &http.Response{ - StatusCode: m.statusOverride, - Body: s, - } - return r, nil -} - -type FakeDialer struct{} - -func (w FakeDialer) Catch() (Snowflake, error) { - fmt.Println("Caught a dummy snowflake.") - return &WebRTCPeer{}, nil -} - -type FakeSocksConn struct { - net.Conn - rejected bool -} - -func (f FakeSocksConn) Reject() error { - f.rejected = true - return nil -} -func (f FakeSocksConn) Grant(addr *net.TCPAddr) error { return nil } - -type FakePeers struct{ toRelease *WebRTCPeer } - -func (f FakePeers) Collect() (Snowflake, error) { return &WebRTCPeer{}, nil } -func (f FakePeers) Pop() Snowflake { return nil } -func (f FakePeers) Melted() <-chan struct{} { return nil } - -const sampleSDP = `"v=0\r\no=- 4358805017720277108 2 IN IP4 8.8.8.8\r\ns=-\r\nt=0 0\r\na=group:BUNDLE data\r\na=msid-semantic: WMS\r\nm=application 56688 DTLS/SCTP 5000\r\nc=IN IP4 8.8.8.8\r\na=candidate:3769337065 1 udp 2122260223 8.8.8.8 56688 typ host generation 0 network-id 1 network-cost 50\r\na=candidate:2921887769 1 tcp 1518280447 8.8.8.8 35441 typ host tcptype passive generation 0 network-id 1 network-cost 50\r\na=ice-ufrag:aMAZ\r\na=ice-pwd:jcHb08Jjgrazp2dzjdrvPPvV\r\na=ice-options:trickle\r\na=fingerprint:sha-256 C8:88:EE:B9:E7:02:2E:21:37:ED:7A:D1:EB:2B:A3:15:A2:3B:5B:1C:3D:D4:D5:1F:06:CF:52:40:03:F8:DD:66\r\na=setup:actpass\r\na=mid:data\r\na=sctpmap:5000 webrtc-datachannel 1024\r\n"` - -const sampleAnswer = `{"type":"answer","sdp":` + sampleSDP + `}` - -func TestSnowflakeClient(t *testing.T) { - - Convey("Peers", t, func() { - Convey("Can construct", func() { - p := NewPeers(1) - So(p.capacity, ShouldEqual, 1) - So(p.snowflakeChan, ShouldNotBeNil) - So(cap(p.snowflakeChan), ShouldEqual, 1) - }) - - Convey("Collecting a Snowflake requires a Tongue.", func() { - p := NewPeers(1) - _, err := p.Collect() - So(err, ShouldNotBeNil) - So(p.Count(), ShouldEqual, 0) - // Set the dialer so that collection is possible. - p.Tongue = FakeDialer{} - _, err = p.Collect() - So(err, ShouldBeNil) - So(p.Count(), ShouldEqual, 1) - // S - _, err = p.Collect() - }) - - Convey("Collection continues until capacity.", func() { - c := 5 - p := NewPeers(c) - p.Tongue = FakeDialer{} - // Fill up to capacity. - for i := 0; i < c; i++ { - fmt.Println("Adding snowflake ", i) - _, err := p.Collect() - So(err, ShouldBeNil) - So(p.Count(), ShouldEqual, i+1) - } - // But adding another gives an error. - So(p.Count(), ShouldEqual, c) - _, err := p.Collect() - So(err, ShouldNotBeNil) - So(p.Count(), ShouldEqual, c) - - // But popping and closing allows it to continue. - s := p.Pop() - s.Close() - So(s, ShouldNotBeNil) - So(p.Count(), ShouldEqual, c-1) - - _, err = p.Collect() - So(err, ShouldBeNil) - So(p.Count(), ShouldEqual, c) - }) - - Convey("Count correctly purges peers marked for deletion.", func() { - p := NewPeers(4) - p.Tongue = FakeDialer{} - p.Collect() - p.Collect() - p.Collect() - p.Collect() - So(p.Count(), ShouldEqual, 4) - s := p.Pop() - s.Close() - So(p.Count(), ShouldEqual, 3) - s = p.Pop() - s.Close() - So(p.Count(), ShouldEqual, 2) - }) - - Convey("End Closes all peers.", func() { - cnt := 5 - p := NewPeers(cnt) - for i := 0; i < cnt; i++ { - p.activePeers.PushBack(&WebRTCPeer{}) - } - So(p.Count(), ShouldEqual, cnt) - p.End() - <-p.Melted() - So(p.Count(), ShouldEqual, 0) - }) - - Convey("Pop skips over closed peers.", func() { - p := NewPeers(4) - p.Tongue = FakeDialer{} - wc1, _ := p.Collect() - wc2, _ := p.Collect() - wc3, _ := p.Collect() - So(wc1, ShouldNotBeNil) - So(wc2, ShouldNotBeNil) - So(wc3, ShouldNotBeNil) - wc1.Close() - r := p.Pop() - So(p.Count(), ShouldEqual, 2) - So(r, ShouldEqual, wc2) - wc4, _ := p.Collect() - wc2.Close() - wc3.Close() - r = p.Pop() - So(r, ShouldEqual, wc4) - }) - - }) - - Convey("Snowflake", t, func() { - - SkipConvey("Handler Grants correctly", func() { - socks := &FakeSocksConn{} - snowflakes := &FakePeers{} - - So(socks.rejected, ShouldEqual, false) - snowflakes.toRelease = nil - Handler(socks, snowflakes) - So(socks.rejected, ShouldEqual, true) - }) - - Convey("WebRTC Connection", func() { - c := NewWebRTCPeer(nil, nil) - So(c.buffer.Bytes(), ShouldEqual, nil) - - Convey("Can construct a WebRTCConn", func() { - s := NewWebRTCPeer(nil, nil) - So(s, ShouldNotBeNil) - So(s.offerChannel, ShouldNotBeNil) - So(s.answerChannel, ShouldNotBeNil) - s.Close() - }) - - Convey("Write buffers when datachannel is nil", func() { - c.Write([]byte("test")) - c.transport = nil - So(c.buffer.Bytes(), ShouldResemble, []byte("test")) - }) - - Convey("Write sends to datachannel when not nil", func() { - mock := new(MockDataChannel) - c.transport = mock - mock.done = make(chan bool, 1) - c.Write([]byte("test")) - <-mock.done - So(c.buffer.Bytes(), ShouldEqual, nil) - So(mock.destination.Bytes(), ShouldResemble, []byte("test")) - }) - - Convey("Exchange SDP sets remote description", func() { - c.offerChannel = make(chan *webrtc.SessionDescription, 1) - c.answerChannel = make(chan *webrtc.SessionDescription, 1) - - c.config = &webrtc.Configuration{} - c.pc, _ = webrtc.NewPeerConnection(*c.config) - offer, _ := c.pc.CreateOffer(nil) - err := c.pc.SetLocalDescription(offer) - So(err, ShouldBeNil) - - c.offerChannel <- nil - answer := util.DeserializeSessionDescription(sampleAnswer) - So(answer, ShouldNotBeNil) - c.answerChannel <- answer - err = c.exchangeSDP() - So(err, ShouldBeNil) - }) - - Convey("Exchange SDP keeps trying on nil answer", func(ctx C) { - var wg sync.WaitGroup - wg.Add(1) - - c.offerChannel = make(chan *webrtc.SessionDescription, 1) - c.answerChannel = make(chan *webrtc.SessionDescription, 1) - c.config = &webrtc.Configuration{} - c.pc, _ = webrtc.NewPeerConnection(*c.config) - offer, _ := c.pc.CreateOffer(nil) - c.pc.SetLocalDescription(offer) - - c.offerChannel <- nil - c.answerChannel <- nil - go func() { - err := c.exchangeSDP() - ctx.So(err, ShouldBeNil) - wg.Done() - }() - answer := util.DeserializeSessionDescription(sampleAnswer) - c.answerChannel <- answer - wg.Wait() - }) - - }) - }) - - Convey("Dialers", t, func() { - Convey("Can construct WebRTCDialer.", func() { - broker := &BrokerChannel{Host: "test"} - d := NewWebRTCDialer(broker, nil) - So(d, ShouldNotBeNil) - So(d.BrokerChannel, ShouldNotBeNil) - So(d.BrokerChannel.Host, ShouldEqual, "test") - }) - SkipConvey("WebRTCDialer can Catch a snowflake.", func() { - broker := &BrokerChannel{Host: "test"} - d := NewWebRTCDialer(broker, nil) - conn, err := d.Catch() - So(conn, ShouldBeNil) - So(err, ShouldNotBeNil) - }) - }) - - Convey("Rendezvous", t, func() { - transport := &MockTransport{ - http.StatusOK, - []byte(`{"type":"answer","sdp":"fake"}`), - } - fakeOffer := util.DeserializeSessionDescription(`{"type":"offer","sdp":"test"}`) - - Convey("Construct BrokerChannel with no front domain", func() { - b, err := NewBrokerChannel("test.broker", "", transport, false) - So(b.url, ShouldNotBeNil) - So(err, ShouldBeNil) - So(b.url.Path, ShouldResemble, "test.broker") - So(b.transport, ShouldNotBeNil) - }) - - Convey("Construct BrokerChannel *with* front domain", func() { - b, err := NewBrokerChannel("test.broker", "front", transport, false) - So(b.url, ShouldNotBeNil) - So(err, ShouldBeNil) - So(b.url.Path, ShouldResemble, "test.broker") - So(b.url.Host, ShouldResemble, "front") - So(b.transport, ShouldNotBeNil) - }) - - Convey("BrokerChannel.Negotiate responds with answer", func() { - b, err := NewBrokerChannel("test.broker", "", transport, false) - So(err, ShouldBeNil) - answer, err := b.Negotiate(fakeOffer) - So(err, ShouldBeNil) - So(answer, ShouldNotBeNil) - So(answer.SDP, ShouldResemble, "fake") - }) - - Convey("BrokerChannel.Negotiate fails with 503", func() { - b, err := NewBrokerChannel("test.broker", "", - &MockTransport{http.StatusServiceUnavailable, []byte("\n")}, - false) - So(err, ShouldBeNil) - answer, err := b.Negotiate(fakeOffer) - So(err, ShouldNotBeNil) - So(answer, ShouldBeNil) - So(err.Error(), ShouldResemble, BrokerError503) - }) - - Convey("BrokerChannel.Negotiate fails with 400", func() { - b, err := NewBrokerChannel("test.broker", "", - &MockTransport{http.StatusBadRequest, []byte("\n")}, - false) - So(err, ShouldBeNil) - answer, err := b.Negotiate(fakeOffer) - So(err, ShouldNotBeNil) - So(answer, ShouldBeNil) - So(err.Error(), ShouldResemble, BrokerError400) - }) - - Convey("BrokerChannel.Negotiate fails with large read", func() { - b, err := NewBrokerChannel("test.broker", "", - &MockTransport{http.StatusOK, make([]byte, 100001, 100001)}, - false) - So(err, ShouldBeNil) - answer, err := b.Negotiate(fakeOffer) - So(err, ShouldNotBeNil) - So(answer, ShouldBeNil) - So(err.Error(), ShouldResemble, "unexpected EOF") - }) - - Convey("BrokerChannel.Negotiate fails with unexpected error", func() { - b, err := NewBrokerChannel("test.broker", "", - &MockTransport{123, []byte("")}, false) - So(err, ShouldBeNil) - answer, err := b.Negotiate(fakeOffer) - So(err, ShouldNotBeNil) - So(answer, ShouldBeNil) - So(err.Error(), ShouldResemble, BrokerErrorUnexpected) - }) - }) - -} diff --git a/client/lib/peers.go b/client/lib/peers.go deleted file mode 100644 index d385971..0000000 --- a/client/lib/peers.go +++ /dev/null @@ -1,123 +0,0 @@ -package lib - -import ( - "container/list" - "errors" - "fmt" - "log" -) - -// Container which keeps track of multiple WebRTC remote peers. -// Implements |SnowflakeCollector|. -// -// Maintaining a set of pre-connected Peers with fresh but inactive datachannels -// allows allows rapid recovery when the current WebRTC Peer disconnects. -// -// Note: For now, only one remote can be active at any given moment. -// This is a property of Tor circuits & its current multiplexing constraints, -// but could be updated if that changes. -// (Also, this constraint does not necessarily apply to the more generic PT -// version of Snowflake) -type Peers struct { - Tongue - BytesLogger - - snowflakeChan chan Snowflake - activePeers *list.List - capacity int - - melt chan struct{} -} - -// Construct a fresh container of remote peers. -func NewPeers(max int) *Peers { - p := &Peers{capacity: max} - // Use buffered go channel to pass snowflakes onwards to the SOCKS handler. - p.snowflakeChan = make(chan Snowflake, max) - p.activePeers = list.New() - p.melt = make(chan struct{}) - return p -} - -// As part of |SnowflakeCollector| interface. -func (p *Peers) Collect() (Snowflake, error) { - cnt := p.Count() - s := fmt.Sprintf("Currently at [%d/%d]", cnt, p.capacity) - if cnt >= p.capacity { - return nil, fmt.Errorf("At capacity [%d/%d]", cnt, p.capacity) - } - log.Println("WebRTC: Collecting a new Snowflake.", s) - // Engage the Snowflake Catching interface, which must be available. - if nil == p.Tongue { - return nil, errors.New("missing Tongue to catch Snowflakes with") - } - // BUG: some broker conflict here. - connection, err := p.Tongue.Catch() - if nil != err { - return nil, err - } - // Track new valid Snowflake in internal collection and pass along. - p.activePeers.PushBack(connection) - p.snowflakeChan <- connection - return connection, nil -} - -// As part of |SnowflakeCollector| interface. -func (p *Peers) Pop() Snowflake { - // Blocks until an available, valid snowflake appears. - var snowflake Snowflake - var ok bool - for snowflake == nil { - snowflake, ok = <-p.snowflakeChan - if !ok { - return nil - } - conn := snowflake.(*WebRTCPeer) - if conn.closed { - snowflake = nil - } - } - // Set to use the same rate-limited traffic logger to keep consistency. - snowflake.(*WebRTCPeer).BytesLogger = p.BytesLogger - return snowflake -} - -// As part of |SnowflakeCollector| interface. -func (p *Peers) Melted() <-chan struct{} { - return p.melt -} - -// Returns total available Snowflakes (including the active one) -// The count only reduces when connections themselves close, rather than when -// they are popped. -func (p *Peers) Count() int { - p.purgeClosedPeers() - return p.activePeers.Len() -} - -func (p *Peers) purgeClosedPeers() { - for e := p.activePeers.Front(); e != nil; { - next := e.Next() - conn := e.Value.(*WebRTCPeer) - // Purge those marked for deletion. - if conn.closed { - p.activePeers.Remove(e) - } - e = next - } -} - -// Close all Peers contained here. -func (p *Peers) End() { - close(p.snowflakeChan) - close(p.melt) - cnt := p.Count() - for e := p.activePeers.Front(); e != nil; { - next := e.Next() - conn := e.Value.(*WebRTCPeer) - conn.Close() - p.activePeers.Remove(e) - e = next - } - log.Printf("WebRTC: melted all %d snowflakes.", cnt) -} diff --git a/client/lib/rendezvous.go b/client/lib/rendezvous.go deleted file mode 100644 index c82fc9e..0000000 --- a/client/lib/rendezvous.go +++ /dev/null @@ -1,156 +0,0 @@ -// WebRTC rendezvous requires the exchange of SessionDescriptions between -// peers in order to establish a PeerConnection. -// -// This file contains the one method currently available to Snowflake: -// -// - Domain-fronted HTTP signaling. The Broker automatically exchange offers -// and answers between this client and some remote WebRTC proxy. - -package lib - -import ( - "bytes" - "errors" - "io" - "io/ioutil" - "log" - "net/http" - "net/url" - - "git.torproject.org/pluggable-transports/snowflake.git/common/util" - "github.com/pion/webrtc/v2" -) - -const ( - BrokerError503 string = "No snowflake proxies currently available." - BrokerError400 string = "You sent an invalid offer in the request." - BrokerErrorUnexpected string = "Unexpected error, no answer." - readLimit = 100000 //Maximum number of bytes to be read from an HTTP response -) - -// Signalling Channel to the Broker. -type BrokerChannel struct { - // The Host header to put in the HTTP request (optional and may be - // different from the host name in URL). - Host string - url *url.URL - transport http.RoundTripper // Used to make all requests. - keepLocalAddresses bool -} - -// We make a copy of DefaultTransport because we want the default Dial -// and TLSHandshakeTimeout settings. But we want to disable the default -// ProxyFromEnvironment setting. -func CreateBrokerTransport() http.RoundTripper { - transport := http.DefaultTransport.(*http.Transport) - transport.Proxy = nil - return transport -} - -// Construct a new BrokerChannel, where: -// |broker| is the full URL of the facilitating program which assigns proxies -// to clients, and |front| is the option fronting domain. -func NewBrokerChannel(broker string, front string, transport http.RoundTripper, keepLocalAddresses bool) (*BrokerChannel, error) { - targetURL, err := url.Parse(broker) - if err != nil { - return nil, err - } - log.Println("Rendezvous using Broker at:", broker) - bc := new(BrokerChannel) - bc.url = targetURL - if front != "" { // Optional front domain. - log.Println("Domain fronting using:", front) - bc.Host = bc.url.Host - bc.url.Host = front - } - - bc.transport = transport - bc.keepLocalAddresses = keepLocalAddresses - return bc, nil -} - -func limitedRead(r io.Reader, limit int64) ([]byte, error) { - p, err := ioutil.ReadAll(&io.LimitedReader{R: r, N: limit + 1}) - if err != nil { - return p, err - } else if int64(len(p)) == limit+1 { - return p[0:limit], io.ErrUnexpectedEOF - } - return p, err -} - -// Roundtrip HTTP POST using WebRTC SessionDescriptions. -// -// Send an SDP offer to the broker, which assigns a proxy and responds -// with an SDP answer from a designated remote WebRTC peer. -func (bc *BrokerChannel) Negotiate(offer *webrtc.SessionDescription) ( - *webrtc.SessionDescription, error) { - log.Println("Negotiating via BrokerChannel...\nTarget URL: ", - bc.Host, "\nFront URL: ", bc.url.Host) - // Ideally, we could specify an `RTCIceTransportPolicy` that would handle - // this for us. However, "public" was removed from the draft spec. - // See https://developer.mozilla.org/en-US/docs/Web/API/RTCConfiguration#RTCIceTra… - if !bc.keepLocalAddresses { - offer = &webrtc.SessionDescription{ - Type: offer.Type, - SDP: util.StripLocalAddresses(offer.SDP), - } - } - data := bytes.NewReader([]byte(util.SerializeSessionDescription(offer))) - // Suffix with broker's client registration handler. - clientURL := bc.url.ResolveReference(&url.URL{Path: "client"}) - request, err := http.NewRequest("POST", clientURL.String(), data) - if nil != err { - return nil, err - } - if "" != bc.Host { // Set true host if necessary. - request.Host = bc.Host - } - resp, err := bc.transport.RoundTrip(request) - if nil != err { - return nil, err - } - defer resp.Body.Close() - log.Printf("BrokerChannel Response:\n%s\n\n", resp.Status) - - switch resp.StatusCode { - case http.StatusOK: - body, err := limitedRead(resp.Body, readLimit) - if nil != err { - return nil, err - } - answer := util.DeserializeSessionDescription(string(body)) - return answer, nil - case http.StatusServiceUnavailable: - return nil, errors.New(BrokerError503) - case http.StatusBadRequest: - return nil, errors.New(BrokerError400) - default: - return nil, errors.New(BrokerErrorUnexpected) - } -} - -// Implements the |Tongue| interface to catch snowflakes, using BrokerChannel. -type WebRTCDialer struct { - *BrokerChannel - webrtcConfig *webrtc.Configuration -} - -func NewWebRTCDialer(broker *BrokerChannel, iceServers []webrtc.ICEServer) *WebRTCDialer { - config := webrtc.Configuration{ - ICEServers: iceServers, - } - return &WebRTCDialer{ - BrokerChannel: broker, - webrtcConfig: &config, - } -} - -// Initialize a WebRTC Connection by signaling through the broker. -func (w WebRTCDialer) Catch() (Snowflake, error) { - // TODO: [#25591] Fetch ICE server information from Broker. - // TODO: [#25596] Consider TURN servers here too. - connection := NewWebRTCPeer(w.webrtcConfig, w.BrokerChannel) - err := connection.Connect() - return connection, err -} diff --git a/client/lib/snowflake.go b/client/lib/snowflake.go deleted file mode 100644 index 2065f73..0000000 --- a/client/lib/snowflake.go +++ /dev/null @@ -1,68 +0,0 @@ -package lib - -import ( - "errors" - "io" - "log" - "net" - "sync" - "time" -) - -const ( - ReconnectTimeout = 10 * time.Second - SnowflakeTimeout = 30 * time.Second -) - -// Given an accepted SOCKS connection, establish a WebRTC connection to the -// remote peer and exchange traffic. -func Handler(socks SocksConnector, snowflakes SnowflakeCollector) error { - // Obtain an available WebRTC remote. May block. - snowflake := snowflakes.Pop() - if nil == snowflake { - if err := socks.Reject(); err != nil { - log.Printf("socks.Reject returned error: %v", err) - } - - return errors.New("handler: Received invalid Snowflake") - } - defer snowflake.Close() - log.Println("---- Handler: snowflake assigned ----") - err := socks.Grant(&net.TCPAddr{IP: net.IPv4zero, Port: 0}) - if err != nil { - return err - } - - go func() { - // When WebRTC resets, close the SOCKS connection too. - snowflake.WaitForReset() - socks.Close() - }() - - // Begin exchanging data. Either WebRTC or localhost SOCKS will close first. - // In eithercase, this closes the handler and induces a new handler. - copyLoop(socks, snowflake) - log.Println("---- Handler: closed ---") - return nil -} - -// Exchanges bytes between two ReadWriters. -// (In this case, between a SOCKS and WebRTC connection.) -func copyLoop(socks, webRTC io.ReadWriter) { - var wg sync.WaitGroup - wg.Add(2) - go func() { - if _, err := io.Copy(socks, webRTC); err != nil { - log.Printf("copying WebRTC to SOCKS resulted in error: %v", err) - } - wg.Done() - }() - go func() { - if _, err := io.Copy(webRTC, socks); err != nil { - log.Printf("copying SOCKS to WebRTC resulted in error: %v", err) - } - wg.Done() - }() - wg.Wait() - log.Println("copy loop ended") -} diff --git a/client/lib/util.go b/client/lib/util.go deleted file mode 100644 index cacf1d7..0000000 --- a/client/lib/util.go +++ /dev/null @@ -1,85 +0,0 @@ -package lib - -import ( - "log" - "time" -) - -const ( - LogTimeInterval = 5 -) - -type BytesLogger interface { - Log() - AddOutbound(int) - AddInbound(int) -} - -// Default BytesLogger does nothing. -type BytesNullLogger struct{} - -func (b BytesNullLogger) Log() {} -func (b BytesNullLogger) AddOutbound(amount int) {} -func (b BytesNullLogger) AddInbound(amount int) {} - -// BytesSyncLogger uses channels to safely log from multiple sources with output -// occuring at reasonable intervals. -type BytesSyncLogger struct { - OutboundChan chan int - InboundChan chan int - Outbound int - Inbound int - OutEvents int - InEvents int - IsLogging bool -} - -func (b *BytesSyncLogger) Log() { - b.IsLogging = true - var amount int - output := func() { - log.Printf("Traffic Bytes (in|out): %d | %d -- (%d OnMessages, %d Sends)", - b.Inbound, b.Outbound, b.InEvents, b.OutEvents) - b.Outbound = 0 - b.OutEvents = 0 - b.Inbound = 0 - b.InEvents = 0 - } - last := time.Now() - for { - select { - case amount = <-b.OutboundChan: - b.Outbound += amount - b.OutEvents++ - if time.Since(last) > time.Second*LogTimeInterval { - last = time.Now() - output() - } - case amount = <-b.InboundChan: - b.Inbound += amount - b.InEvents++ - if time.Since(last) > time.Second*LogTimeInterval { - last = time.Now() - output() - } - case <-time.After(time.Second * LogTimeInterval): - if b.InEvents > 0 || b.OutEvents > 0 { - output() - } - } - } -} - -func (b *BytesSyncLogger) AddOutbound(amount int) { - if !b.IsLogging { - return - } - b.OutboundChan <- amount -} - -func (b *BytesSyncLogger) AddInbound(amount int) { - if !b.IsLogging { - return - } - b.InboundChan <- amount -} diff --git a/client/lib/webrtc.go b/client/lib/webrtc.go deleted file mode 100644 index 5aa7aec..0000000 --- a/client/lib/webrtc.go +++ /dev/null @@ -1,380 +0,0 @@ -package lib - -import ( - "bytes" - "crypto/rand" - "encoding/hex" - "errors" - "io" - "log" - "sync" - "time" - - "github.com/pion/webrtc/v2" -) - -// Remote WebRTC peer. -// Implements the |Snowflake| interface, which includes -// |io.ReadWriter|, |Resetter|, and |Connector|. -// -// Handles preparation of go-webrtc PeerConnection. Only ever has -// one DataChannel. -type WebRTCPeer struct { - id string - config *webrtc.Configuration - pc *webrtc.PeerConnection - transport SnowflakeDataChannel // Holds the WebRTC DataChannel. - broker *BrokerChannel - - offerChannel chan *webrtc.SessionDescription - answerChannel chan *webrtc.SessionDescription - errorChannel chan error - recvPipe *io.PipeReader - writePipe *io.PipeWriter - lastReceive time.Time - buffer bytes.Buffer - reset chan struct{} - - closed bool - - lock sync.Mutex // Synchronization for DataChannel destruction - once sync.Once // Synchronization for PeerConnection destruction - - BytesLogger -} - -// Construct a WebRTC PeerConnection. -func NewWebRTCPeer(config *webrtc.Configuration, - broker *BrokerChannel) *WebRTCPeer { - connection := new(WebRTCPeer) - { - var buf [8]byte - if _, err := rand.Read(buf[:]); err != nil { - panic(err) - } - connection.id = "snowflake-" + hex.EncodeToString(buf[:]) - } - connection.config = config - connection.broker = broker - connection.offerChannel = make(chan *webrtc.SessionDescription, 1) - connection.answerChannel = make(chan *webrtc.SessionDescription, 1) - // Error channel is mostly for reporting during the initial SDP offer - // creation & local description setting, which happens asynchronously. - connection.errorChannel = make(chan error, 1) - connection.reset = make(chan struct{}, 1) - - // Override with something that's not NullLogger to have real logging. - connection.BytesLogger = &BytesNullLogger{} - - // Pipes remain the same even when DataChannel gets switched. - connection.recvPipe, connection.writePipe = io.Pipe() - return connection -} - -// Read bytes from local SOCKS. -// As part of |io.ReadWriter| -func (c *WebRTCPeer) Read(b []byte) (int, error) { - return c.recvPipe.Read(b) -} - -// Writes bytes out to remote WebRTC. -// As part of |io.ReadWriter| -func (c *WebRTCPeer) Write(b []byte) (int, error) { - c.lock.Lock() - defer c.lock.Unlock() - c.BytesLogger.AddOutbound(len(b)) - // TODO: Buffering could be improved / separated out of WebRTCPeer. - if nil == c.transport { - log.Printf("Buffered %d bytes --> WebRTC", len(b)) - c.buffer.Write(b) - } else { - c.transport.Send(b) - } - return len(b), nil -} - -// As part of |Snowflake| -func (c *WebRTCPeer) Close() error { - c.once.Do(func() { - c.closed = true - c.cleanup() - c.Reset() - log.Printf("WebRTC: Closing") - }) - return nil -} - -// As part of |Resetter| -func (c *WebRTCPeer) Reset() { - if nil == c.reset { - return - } - c.reset <- struct{}{} -} - -// As part of |Resetter| -func (c *WebRTCPeer) WaitForReset() { <-c.reset } - -// Prevent long-lived broken remotes. -// Should also update the DataChannel in underlying go-webrtc's to make Closes -// more immediate / responsive. -func (c *WebRTCPeer) checkForStaleness() { - c.lastReceive = time.Now() - for { - if c.closed { - return - } - if time.Since(c.lastReceive) > SnowflakeTimeout { - log.Printf("WebRTC: No messages received for %v -- closing stale connection.", - SnowflakeTimeout) - c.Close() - return - } - <-time.After(time.Second) - } -} - -// As part of |Connector| interface. -func (c *WebRTCPeer) Connect() error { - log.Println(c.id, " connecting...") - // TODO: When go-webrtc is more stable, it's possible that a new - // PeerConnection won't need to be re-prepared each time. - err := c.preparePeerConnection() - if err != nil { - return err - } - err = c.establishDataChannel() - if err != nil { - // nolint: golint - return errors.New("WebRTC: Could not establish DataChannel") - } - err = c.exchangeSDP() - if err != nil { - return err - } - go c.checkForStaleness() - return nil -} - -// Create and prepare callbacks on a new WebRTC PeerConnection. -func (c *WebRTCPeer) preparePeerConnection() error { - if nil != c.pc { - if err := c.pc.Close(); err != nil { - log.Printf("c.pc.Close returned error: %v", err) - } - c.pc = nil - } - - s := webrtc.SettingEngine{} - s.SetTrickle(true) - api := webrtc.NewAPI(webrtc.WithSettingEngine(s)) - pc, err := api.NewPeerConnection(*c.config) - if err != nil { - log.Printf("NewPeerConnection ERROR: %s", err) - return err - } - // Prepare PeerConnection callbacks. - // Allow candidates to accumulate until ICEGatheringStateComplete. - pc.OnICECandidate(func(candidate *webrtc.ICECandidate) { - if candidate == nil { - log.Printf("WebRTC: Done gathering candidates") - } else { - log.Printf("WebRTC: Got ICE candidate: %s", candidate.String()) - } - }) - pc.OnICEGatheringStateChange(func(state webrtc.ICEGathererState) { - if state == webrtc.ICEGathererStateComplete { - log.Println("WebRTC: ICEGatheringStateComplete") - c.offerChannel <- pc.LocalDescription() - } - }) - // This callback is not expected, as the Client initiates the creation - // of the data channel, not the remote peer. - pc.OnDataChannel(func(channel *webrtc.DataChannel) { - log.Println("OnDataChannel") - panic("Unexpected OnDataChannel!") - }) - c.pc = pc - go func() { - offer, err := pc.CreateOffer(nil) - // TODO: Potentially timeout and retry if ICE isn't working. - if err != nil { - c.errorChannel <- err - return - } - log.Println("WebRTC: Created offer") - err = pc.SetLocalDescription(offer) - if err != nil { - c.errorChannel <- err - return - } - log.Println("WebRTC: Set local description") - }() - log.Println("WebRTC: PeerConnection created.") - return nil -} - -// Create a WebRTC DataChannel locally. -func (c *WebRTCPeer) establishDataChannel() error { - c.lock.Lock() - defer c.lock.Unlock() - if c.transport != nil { - panic("Unexpected datachannel already exists!") - } - ordered := true - dataChannelOptions := &webrtc.DataChannelInit{ - Ordered: &ordered, - } - dc, err := c.pc.CreateDataChannel(c.id, dataChannelOptions) - // Triggers "OnNegotiationNeeded" on the PeerConnection, which will prepare - // an SDP offer while other goroutines operating on this struct handle the - // signaling. Eventually fires "OnOpen". - if err != nil { - log.Printf("CreateDataChannel ERROR: %s", err) - return err - } - dc.OnOpen(func() { - c.lock.Lock() - defer c.lock.Unlock() - log.Println("WebRTC: DataChannel.OnOpen") - if nil != c.transport { - panic("WebRTC: transport already exists.") - } - // Flush buffered outgoing SOCKS data if necessary. - if c.buffer.Len() > 0 { - dc.Send(c.buffer.Bytes()) - log.Println("Flushed", c.buffer.Len(), "bytes.") - c.buffer.Reset() - } - // Then enable the datachannel. - c.transport = dc - }) - dc.OnClose(func() { - c.lock.Lock() - // Future writes will go to the buffer until a new DataChannel is available. - if nil == c.transport { - // Closed locally, as part of a reset. - log.Println("WebRTC: DataChannel.OnClose [locally]") - c.lock.Unlock() - return - } - // Closed remotely, need to reset everything. - // Disable the DataChannel as a write destination. - log.Println("WebRTC: DataChannel.OnClose [remotely]") - c.transport = nil - dc.Close() - // Unlock before Close'ing, since it calls cleanup and asks for the - // lock to check if the transport needs to be be deleted. - c.lock.Unlock() - c.Close() - }) - dc.OnMessage(func(msg webrtc.DataChannelMessage) { - if len(msg.Data) <= 0 { - log.Println("0 length message---") - } - c.BytesLogger.AddInbound(len(msg.Data)) - n, err := c.writePipe.Write(msg.Data) - if err != nil { - // TODO: Maybe shouldn't actually close. - log.Println("Error writing to SOCKS pipe") - if inerr := c.writePipe.CloseWithError(err); inerr != nil { - log.Printf("c.writePipe.CloseWithError returned error: %v", inerr) - } - } - if n != len(msg.Data) { - log.Println("Error: short write") - panic("short write") - } - c.lastReceive = time.Now() - }) - log.Println("WebRTC: DataChannel created.") - return nil -} - -func (c *WebRTCPeer) sendOfferToBroker() { - if nil == c.broker { - return - } - offer := c.pc.LocalDescription() - answer, err := c.broker.Negotiate(offer) - if nil != err || nil == answer { - log.Printf("BrokerChannel Error: %s", err) - answer = nil - } - c.answerChannel <- answer -} - -// Block until an SDP offer is available, send it to either -// the Broker or signal pipe, then await for the SDP answer. -func (c *WebRTCPeer) exchangeSDP() error { - select { - case <-c.offerChannel: - case err := <-c.errorChannel: - log.Println("Failed to prepare offer", err) - c.Close() - return err - } - // Keep trying the same offer until a valid answer arrives. - var ok bool - var answer *webrtc.SessionDescription - for nil == answer { - go c.sendOfferToBroker() - answer, ok = <-c.answerChannel // Blocks... - if !ok || nil == answer { - log.Printf("Failed to retrieve answer. Retrying in %v", ReconnectTimeout) - <-time.After(ReconnectTimeout) - answer = nil - } - } - log.Printf("Received Answer.\n") - err := c.pc.SetRemoteDescription(*answer) - if nil != err { - log.Println("WebRTC: Unable to SetRemoteDescription:", err) - return err - } - return nil -} - -// Close all channels and transports -func (c *WebRTCPeer) cleanup() { - if nil != c.offerChannel { - close(c.offerChannel) - } - if nil != c.answerChannel { - close(c.answerChannel) - } - if nil != c.errorChannel { - close(c.errorChannel) - } - // Close this side of the SOCKS pipe. - if nil != c.writePipe { - c.writePipe.Close() - c.writePipe = nil - } - c.lock.Lock() - if nil != c.transport { - log.Printf("WebRTC: closing DataChannel") - dataChannel := c.transport - // Setting transport to nil *before* dc Close indicates to OnClose that - // this was locally triggered. - c.transport = nil - // Release the lock before calling DeleteDataChannel (which in turn - // calls Close on the dataChannel), but after nil'ing out the transport, - // since otherwise we'll end up in the onClose handler in a deadlock. - c.lock.Unlock() - if c.pc == nil { - panic("DataChannel w/o PeerConnection, not good.") - } - dataChannel.(*webrtc.DataChannel).Close() - } else { - c.lock.Unlock() - } - if nil != c.pc { - log.Printf("WebRTC: closing PeerConnection") - err := c.pc.Close() - if nil != err { - log.Printf("Error closing peerconnection...") - } - c.pc = nil - } -} diff --git a/client/snowflake.go b/client/snowflake.go deleted file mode 100644 index af8447c..0000000 --- a/client/snowflake.go +++ /dev/null @@ -1,221 +0,0 @@ -// Client transport plugin for the Snowflake pluggable transport. -package main - -import ( - "flag" - "io" - "io/ioutil" - "log" - "net" - "os" - "os/signal" - "path/filepath" - "strings" - "syscall" - "time" - - pt "git.torproject.org/pluggable-transports/goptlib.git" - sf "git.torproject.org/pluggable-transports/snowflake.git/client/lib" - "git.torproject.org/pluggable-transports/snowflake.git/common/safelog" - "github.com/pion/webrtc/v2" -) - -const ( - DefaultSnowflakeCapacity = 1 -) - -// Maintain |SnowflakeCapacity| number of available WebRTC connections, to -// transfer to the Tor SOCKS handler when needed. -func ConnectLoop(snowflakes sf.SnowflakeCollector) { - for { - // Check if ending is necessary. - _, err := snowflakes.Collect() - if err != nil { - log.Printf("WebRTC: %v Retrying in %v...", - err, sf.ReconnectTimeout) - } - select { - case <-time.After(sf.ReconnectTimeout): - continue - case <-snowflakes.Melted(): - log.Println("ConnectLoop: stopped.") - return - } - } -} - -// Accept local SOCKS connections and pass them to the handler. -func socksAcceptLoop(ln *pt.SocksListener, snowflakes sf.SnowflakeCollector) { - defer ln.Close() - for { - conn, err := ln.AcceptSocks() - if err != nil { - if err, ok := err.(net.Error); ok && err.Temporary() { - continue - } - log.Printf("SOCKS accept error: %s", err) - break - } - log.Printf("SOCKS accepted: %v", conn.Req) - go func() { - defer conn.Close() - err = sf.Handler(conn, snowflakes) - if err != nil { - log.Printf("handler error: %s", err) - } - }() - } -} - -// s is a comma-separated list of ICE server URLs. -func parseIceServers(s string) []webrtc.ICEServer { - var servers []webrtc.ICEServer - s = strings.TrimSpace(s) - if len(s) == 0 { - return nil - } - urls := strings.Split(s, ",") - for _, url := range urls { - url = strings.TrimSpace(url) - servers = append(servers, webrtc.ICEServer{ - URLs: []string{url}, - }) - } - return servers -} - -func main() { - iceServersCommas := flag.String("ice", "", "comma-separated list of ICE servers") - brokerURL := flag.String("url", "", "URL of signaling broker") - frontDomain := flag.String("front", "", "front domain") - logFilename := flag.String("log", "", "name of log file") - logToStateDir := flag.Bool("log-to-state-dir", false, "resolve the log file relative to tor's pt state dir") - keepLocalAddresses := flag.Bool("keep-local-addresses", false, "keep local LAN address ICE candidates") - unsafeLogging := flag.Bool("unsafe-logging", false, "prevent logs from being scrubbed") - max := flag.Int("max", DefaultSnowflakeCapacity, - "capacity for number of multiplexed WebRTC peers") - - // Deprecated - oldLogToStateDir := flag.Bool("logToStateDir", false, "use -log-to-state-dir instead") - oldKeepLocalAddresses := flag.Bool("keepLocalAddresses", false, "use -keep-local-addresses instead") - - flag.Parse() - - log.SetFlags(log.LstdFlags | log.LUTC) - - // Don't write to stderr; versions of tor earlier than about 0.3.5.6 do - // not read from the pipe, and eventually we will deadlock because the - // buffer is full. - // https://bugs.torproject.org/26360 - // https://bugs.torproject.org/25600#comment:14 - var logOutput = ioutil.Discard - if *logFilename != "" { - if *logToStateDir || *oldLogToStateDir { - stateDir, err := pt.MakeStateDir() - if err != nil { - log.Fatal(err) - } - *logFilename = filepath.Join(stateDir, *logFilename) - } - logFile, err := os.OpenFile(*logFilename, - os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0600) - if err != nil { - log.Fatal(err) - } - defer logFile.Close() - logOutput = logFile - } - if *unsafeLogging { - log.SetOutput(logOutput) - } else { - // We want to send the log output through our scrubber first - log.SetOutput(&safelog.LogScrubber{Output: logOutput}) - } - - log.Println("\n\n\n --- Starting Snowflake Client ---") - - iceServers := parseIceServers(*iceServersCommas) - log.Printf("Using ICE servers:") - for _, server := range iceServers { - log.Printf("url: %v", strings.Join(server.URLs, " ")) - } - - // Prepare to collect remote WebRTC peers. - snowflakes := sf.NewPeers(*max) - - // Use potentially domain-fronting broker to rendezvous. - broker, err := sf.NewBrokerChannel( - *brokerURL, *frontDomain, sf.CreateBrokerTransport(), - *keepLocalAddresses || *oldKeepLocalAddresses) - if err != nil { - log.Fatalf("parsing broker URL: %v", err) - } - snowflakes.Tongue = sf.NewWebRTCDialer(broker, iceServers) - - // Use a real logger to periodically output how much traffic is happening. - snowflakes.BytesLogger = &sf.BytesSyncLogger{ - InboundChan: make(chan int, 5), - OutboundChan: make(chan int, 5), - Inbound: 0, - Outbound: 0, - InEvents: 0, - OutEvents: 0, - } - go snowflakes.BytesLogger.Log() - - go ConnectLoop(snowflakes) - - // Begin goptlib client process. - ptInfo, err := pt.ClientSetup(nil) - if err != nil { - log.Fatal(err) - } - if ptInfo.ProxyURL != nil { - pt.ProxyError("proxy is not supported") - os.Exit(1) - } - listeners := make([]net.Listener, 0) - for _, methodName := range ptInfo.MethodNames { - switch methodName { - case "snowflake": - // TODO: Be able to recover when SOCKS dies. - ln, err := pt.ListenSocks("tcp", "127.0.0.1:0") - if err != nil { - pt.CmethodError(methodName, err.Error()) - break - } - log.Printf("Started SOCKS listener at %v.", ln.Addr()) - go socksAcceptLoop(ln, snowflakes) - pt.Cmethod(methodName, ln.Version(), ln.Addr()) - listeners = append(listeners, ln) - default: - pt.CmethodError(methodName, "no such method") - } - } - pt.CmethodsDone() - - sigChan := make(chan os.Signal, 1) - signal.Notify(sigChan, syscall.SIGTERM) - - if os.Getenv("TOR_PT_EXIT_ON_STDIN_CLOSE") == "1" { - // This environment variable means we should treat EOF on stdin - // just like SIGTERM: https://bugs.torproject.org/15435. - go func() { - if _, err := io.Copy(ioutil.Discard, os.Stdin); err != nil { - log.Printf("calling io.Copy(ioutil.Discard, os.Stdin) returned error: %v", err) - } - log.Printf("synthesizing SIGTERM because of stdin close") - sigChan <- syscall.SIGTERM - }() - } - - // Wait for a signal. - <-sigChan - - // Signal received, shut down. - for _, ln := range listeners { - ln.Close() - } - snowflakes.End() - log.Println("snowflake is done.") -} diff --git a/client/torrc b/client/torrc deleted file mode 100644 index 9e3946e..0000000 --- a/client/torrc +++ /dev/null @@ -1,10 +0,0 @@ -UseBridges 1 -DataDirectory datadir - -ClientTransportPlugin snowflake exec ./client \ --url https://snowflake-broker.azureedge.net/ \ --front ajax.aspnetcdn.com \ --ice stun:stun.l.google.com:19302 \ --max 3 - -Bridge snowflake 192.0.2.3:1 diff --git a/client/torrc-localhost b/client/torrc-localhost deleted file mode 100644 index b2a6d05..0000000 --- a/client/torrc-localhost +++ /dev/null @@ -1,8 +0,0 @@ -UseBridges 1 -DataDirectory datadir - -ClientTransportPlugin snowflake exec ./client \ --url http://localhost:8080/ \ --keep-local-addresses - -Bridge snowflake 192.0.2.3:1 diff --git a/common/messages/proxy.go b/common/messages/proxy.go deleted file mode 100644 index 89dd43c..0000000 --- a/common/messages/proxy.go +++ /dev/null @@ -1,222 +0,0 @@ -//Package for communication with the snowflake broker - -//import "git.torproject.org/pluggable-transports/snowflake.git/common/messages" -package messages - -import ( - "encoding/json" - "fmt" - "strings" -) - -const version = "1.1" - -/* Version 1.1 specification: - -== ProxyPollRequest == -{ - Sid: [generated session id of proxy], - Version: 1.1, - Type: ["badge"|"webext"|"standalone"] -} - -== ProxyPollResponse == -1) If a client is matched: -HTTP 200 OK -{ - Status: "client match", - { - type: offer, - sdp: [WebRTC SDP] - } -} - -2) If a client is not matched: -HTTP 200 OK - -{ - Status: "no match" -} - -3) If the request is malformed: -HTTP 400 BadRequest - -== ProxyAnswerRequest == -{ - Sid: [generated session id of proxy], - Version: 1.1, - Answer: - { - type: answer, - sdp: [WebRTC SDP] - } -} - -== ProxyAnswerResponse == -1) If the client retrieved the answer: -HTTP 200 OK - -{ - Status: "success" -} - -2) If the client left: -HTTP 200 OK - -{ - Status: "client gone" -} - -3) If the request is malformed: -HTTP 400 BadRequest - -*/ - -type ProxyPollRequest struct { - Sid string - Version string - Type string -} - -func EncodePollRequest(sid string, proxyType string) ([]byte, error) { - return json.Marshal(ProxyPollRequest{ - Sid: sid, - Version: version, - Type: proxyType, - }) -} - -// Decodes a poll message from a snowflake proxy and returns the -// sid and proxy type of the proxy on success and an error if it failed -func DecodePollRequest(data []byte) (string, string, error) { - var message ProxyPollRequest - - err := json.Unmarshal(data, &message) - if err != nil { - return "", "", err - } - - majorVersion := strings.Split(message.Version, ".")[0] - if majorVersion != "1" { - return "", "", fmt.Errorf("using unknown version") - } - - // Version 1.x requires an Sid - if message.Sid == "" { - return "", "", fmt.Errorf("no supplied session id") - } - - return message.Sid, message.Type, nil -} - -type ProxyPollResponse struct { - Status string - Offer string -} - -func EncodePollResponse(offer string, success bool) ([]byte, error) { - if success { - return json.Marshal(ProxyPollResponse{ - Status: "client match", - Offer: offer, - }) - - } - return json.Marshal(ProxyPollResponse{ - Status: "no match", - }) -} - -// Decodes a poll response from the broker and returns an offer -// If there is a client match, the returned offer string will be non-empty -func DecodePollResponse(data []byte) (string, error) { - var message ProxyPollResponse - - err := json.Unmarshal(data, &message) - if err != nil { - return "", err - } - if message.Status == "" { - return "", fmt.Errorf("received invalid data") - } - - if message.Status == "client match" { - if message.Offer == "" { - return "", fmt.Errorf("no supplied offer") - } - } else { - message.Offer = "" - } - - return message.Offer, nil -} - -type ProxyAnswerRequest struct { - Version string - Sid string - Answer string -} - -func EncodeAnswerRequest(answer string, sid string) ([]byte, error) { - return json.Marshal(ProxyAnswerRequest{ - Version: "1.1", - Sid: sid, - Answer: answer, - }) -} - -// Returns the sdp answer and proxy sid -func DecodeAnswerRequest(data []byte) (string, string, error) { - var message ProxyAnswerRequest - - err := json.Unmarshal(data, &message) - if err != nil { - return "", "", err - } - - majorVersion := strings.Split(message.Version, ".")[0] - if majorVersion != "1" { - return "", "", fmt.Errorf("using unknown version") - } - - if message.Sid == "" || message.Answer == "" { - return "", "", fmt.Errorf("no supplied sid or answer") - } - - return message.Answer, message.Sid, nil -} - -type ProxyAnswerResponse struct { - Status string -} - -func EncodeAnswerResponse(success bool) ([]byte, error) { - if success { - return json.Marshal(ProxyAnswerResponse{ - Status: "success", - }) - - } - return json.Marshal(ProxyAnswerResponse{ - Status: "client gone", - }) -} - -func DecodeAnswerResponse(data []byte) (bool, error) { - var message ProxyAnswerResponse - var success bool - - err := json.Unmarshal(data, &message) - if err != nil { - return success, err - } - if message.Status == "" { - return success, fmt.Errorf("received invalid data") - } - - if message.Status == "success" { - success = true - } - - return success, nil -} diff --git a/common/messages/proxy_test.go b/common/messages/proxy_test.go deleted file mode 100644 index 1570d4f..0000000 --- a/common/messages/proxy_test.go +++ /dev/null @@ -1,234 +0,0 @@ -package messages - -import ( - "encoding/json" - "fmt" - "testing" - - . "github.com/smartystreets/goconvey/convey" -) - -func TestDecodeProxyPollRequest(t *testing.T) { - Convey("Context", t, func() { - for _, test := range []struct { - sid string - proxyType string - data string - err error - }{ - { - //Version 1.0 proxy message - "ymbcCMto7KHNGYlp", - "", - `{"Sid":"ymbcCMto7KHNGYlp","Version":"1.0"}`, - nil, - }, - { - //Version 1.1 proxy message - "ymbcCMto7KHNGYlp", - "standalone", - `{"Sid":"ymbcCMto7KHNGYlp","Version":"1.1","Type":"standalone"}`, - nil, - }, - { - //Version 0.X proxy message: - "", - "", - "ymbcCMto7KHNGYlp", - &json.SyntaxError{}, - }, - { - "", - "", - `{"Sid":"ymbcCMto7KHNGYlp"}`, - fmt.Errorf(""), - }, - { - "", - "", - "{}", - fmt.Errorf(""), - }, - { - "", - "", - `{"Version":"1.0"}`, - fmt.Errorf(""), - }, - { - "", - "", - `{"Version":"2.0"}`, - fmt.Errorf(""), - }, - } { - sid, proxyType, err := DecodePollRequest([]byte(test.data)) - So(sid, ShouldResemble, test.sid) - So(proxyType, ShouldResemble, test.proxyType) - So(err, ShouldHaveSameTypeAs, test.err) - } - - }) -} - -func TestEncodeProxyPollRequests(t *testing.T) { - Convey("Context", t, func() { - b, err := EncodePollRequest("ymbcCMto7KHNGYlp", "standalone") - So(err, ShouldEqual, nil) - sid, proxyType, err := DecodePollRequest(b) - So(sid, ShouldEqual, "ymbcCMto7KHNGYlp") - So(proxyType, ShouldEqual, "standalone") - So(err, ShouldEqual, nil) - }) -} - -func TestDecodeProxyPollResponse(t *testing.T) { - Convey("Context", t, func() { - for _, test := range []struct { - offer string - data string - err error - }{ - { - "fake offer", - `{"Status":"client match","Offer":"fake offer"}`, - nil, - }, - { - "", - `{"Status":"no match"}`, - nil, - }, - { - "", - `{"Status":"client match"}`, - fmt.Errorf("no supplied offer"), - }, - { - "", - `{"Test":"test"}`, - fmt.Errorf(""), - }, - } { - offer, err := DecodePollResponse([]byte(test.data)) - So(offer, ShouldResemble, test.offer) - So(err, ShouldHaveSameTypeAs, test.err) - } - - }) -} - -func TestEncodeProxyPollResponse(t *testing.T) { - Convey("Context", t, func() { - b, err := EncodePollResponse("fake offer", true) - So(err, ShouldEqual, nil) - offer, err := DecodePollResponse(b) - So(offer, ShouldEqual, "fake offer") - So(err, ShouldEqual, nil) - - b, err = EncodePollResponse("", false) - So(err, ShouldEqual, nil) - offer, err = DecodePollResponse(b) - So(offer, ShouldEqual, "") - So(err, ShouldEqual, nil) - }) -} -func TestDecodeProxyAnswerRequest(t *testing.T) { - Convey("Context", t, func() { - for _, test := range []struct { - answer string - sid string - data string - err error - }{ - { - "test", - "test", - `{"Version":"1.0","Sid":"test","Answer":"test"}`, - nil, - }, - { - "", - "", - `{"type":"offer","sdp":"v=0\r\no=- 4358805017720277108 2 IN IP4 [scrubbed]\r\ns=-\r\nt=0 0\r\na=group:BUNDLE data\r\na=msid-semantic: WMS\r\nm=application 56688 DTLS/SCTP 5000\r\nc=IN IP4 [scrubbed]\r\na=candidate:3769337065 1 udp 2122260223 [scrubbed] 56688 typ host generation 0 network-id 1 network-cost 50\r\na=candidate:2921887769 1 tcp 1518280447 [scrubbed] 35441 typ host tcptype passive generation 0 network-id 1 network-cost 50\r\na=ice-ufrag:aMAZ\r\na=ice-pwd:jcHb08Jjgrazp2dzjdrvPPvV\r\na=ice-options:trickle\r\na=fingerprint:sha-256 C8:88:EE:B9:E7:02:2E:21:37:ED:7A:D1:EB:2B:A3:15:A2:3B:5B:1C:3D:D4:D5:1F:06:CF:52:40:03:F8:DD:66\r\na=setup:actpass\r\na=mid:data\r\na=sctpmap:5000 webrtc-datachannel 1024\r\n"}`, - fmt.Errorf(""), - }, - { - "", - "", - `{"Version":"1.0","Answer":"test"}`, - fmt.Errorf(""), - }, - { - "", - "", - `{"Version":"1.0","Sid":"test"}`, - fmt.Errorf(""), - }, - } { - answer, sid, err := DecodeAnswerRequest([]byte(test.data)) - So(answer, ShouldResemble, test.answer) - So(sid, ShouldResemble, test.sid) - So(err, ShouldHaveSameTypeAs, test.err) - } - - }) -} - -func TestEncodeProxyAnswerRequest(t *testing.T) { - Convey("Context", t, func() { - b, err := EncodeAnswerRequest("test answer", "test sid") - So(err, ShouldEqual, nil) - answer, sid, err := DecodeAnswerRequest(b) - So(answer, ShouldEqual, "test answer") - So(sid, ShouldEqual, "test sid") - So(err, ShouldEqual, nil) - }) -} - -func TestDecodeProxyAnswerResponse(t *testing.T) { - Convey("Context", t, func() { - for _, test := range []struct { - success bool - data string - err error - }{ - { - true, - `{"Status":"success"}`, - nil, - }, - { - false, - `{"Status":"client gone"}`, - nil, - }, - { - false, - `{"Test":"test"}`, - fmt.Errorf(""), - }, - } { - success, err := DecodeAnswerResponse([]byte(test.data)) - So(success, ShouldResemble, test.success) - So(err, ShouldHaveSameTypeAs, test.err) - } - - }) -} - -func TestEncodeProxyAnswerResponse(t *testing.T) { - Convey("Context", t, func() { - b, err := EncodeAnswerResponse(true) - So(err, ShouldEqual, nil) - success, err := DecodeAnswerResponse(b) - So(success, ShouldEqual, true) - So(err, ShouldEqual, nil) - - b, err = EncodeAnswerResponse(false) - So(err, ShouldEqual, nil) - success, err = DecodeAnswerResponse(b) - So(success, ShouldEqual, false) - So(err, ShouldEqual, nil) - }) -} diff --git a/common/safelog/log.go b/common/safelog/log.go deleted file mode 100644 index 9148e53..0000000 --- a/common/safelog/log.go +++ /dev/null @@ -1,71 +0,0 @@ -//Package for a safer logging wrapper around the standard logging package - -//import "git.torproject.org/pluggable-transports/snowflake.git/common/safelog" -package safelog - -import ( - "bytes" - "io" - "regexp" - "sync" -) - -const ipv4Address = `\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}` -const ipv6Address = `([0-9a-fA-F]{0,4}:){5,7}([0-9a-fA-F]{0,4})?` -const ipv6Compressed = `([0-9a-fA-F]{0,4}:){0,5}([0-9a-fA-F]{0,4})?(::)([0-9a-fA-F]{0,4}:){0,5}([0-9a-fA-F]{0,4})?` -const ipv6Full = `(` + ipv6Address + `(` + ipv4Address + `))` + - `|(` + ipv6Compressed + `(` + ipv4Address + `))` + - `|(` + ipv6Address + `)` + `|(` + ipv6Compressed + `)` -const optionalPort = `(:\d{1,5})?` -const addressPattern = `((` + ipv4Address + `)|(\[(` + ipv6Full + `)\])|(` + ipv6Full + `))` + optionalPort -const fullAddrPattern = `(^|\s|[^\w:])` + addressPattern + `(\s|(:\s)|[^\w:]|$)` - -var scrubberPatterns = []*regexp.Regexp{ - regexp.MustCompile(fullAddrPattern), -} - -var addressRegexp = regexp.MustCompile(addressPattern) - -// An io.Writer that can be used as the output for a logger that first -// sanitizes logs and then writes to the provided io.Writer -type LogScrubber struct { - Output io.Writer - buffer []byte - - lock sync.Mutex -} - -func (ls *LogScrubber) Lock() { (*ls).lock.Lock() } -func (ls *LogScrubber) Unlock() { (*ls).lock.Unlock() } - -func scrub(b []byte) []byte { - scrubbedBytes := b - for _, pattern := range scrubberPatterns { - // this is a workaround since go does not yet support look ahead or look - // behind for regular expressions. - scrubbedBytes = pattern.ReplaceAllFunc(scrubbedBytes, func(b []byte) []byte { - return addressRegexp.ReplaceAll(b, []byte("[scrubbed]")) - }) - } - return scrubbedBytes -} - -func (ls *LogScrubber) Write(b []byte) (n int, err error) { - ls.Lock() - defer ls.Unlock() - - n = len(b) - ls.buffer = append(ls.buffer, b...) - for { - i := bytes.LastIndexByte(ls.buffer, '\n') - if i == -1 { - return - } - fullLines := ls.buffer[:i+1] - _, err = ls.Output.Write(scrub(fullLines)) - if err != nil { - return - } - ls.buffer = ls.buffer[i+1:] - } -} diff --git a/common/safelog/log_test.go b/common/safelog/log_test.go deleted file mode 100644 index 16edbc9..0000000 --- a/common/safelog/log_test.go +++ /dev/null @@ -1,148 +0,0 @@ -package safelog - -import ( - "bytes" - "log" - "testing" -) - -//Check to make sure that addresses split across calls to write are still scrubbed -func TestLogScrubberSplit(t *testing.T) { - input := []byte("test\nhttp2: panic serving [2620:101:f000:780:9097:75b1:519f:dbb8]:58344: interface conversion: *http2.responseWriter is not http.Hijacker: missing method Hijack\n") - - expected := "test\nhttp2: panic serving [scrubbed]: interface conversion: *http2.responseWriter is not http.Hijacker: missing method Hijack\n" - - var buff bytes.Buffer - scrubber := &LogScrubber{Output: &buff} - n, err := scrubber.Write(input[:12]) //test\nhttp2: - if n != 12 { - t.Errorf("wrong number of bytes %d", n) - } - if err != nil { - t.Errorf("%q", err) - } - if buff.String() != "test\n" { - t.Errorf("Got %q, expected %q", buff.String(), "test\n") - } - - n, err = scrubber.Write(input[12:30]) //panic serving [2620:101:f - if n != 18 { - t.Errorf("wrong number of bytes %d", n) - } - if err != nil { - t.Errorf("%q", err) - } - if buff.String() != "test\n" { - t.Errorf("Got %q, expected %q", buff.String(), "test\n") - } - - n, err = scrubber.Write(input[30:]) //000:780:9097:75b1:519f:dbb8]:58344: interface conversion: *http2.responseWriter is not http.Hijacker: missing method Hijack\n - if n != (len(input) - 30) { - t.Errorf("wrong number of bytes %d", n) - } - if err != nil { - t.Errorf("%q", err) - } - if buff.String() != expected { - t.Errorf("Got %q, expected %q", buff.String(), expected) - } - -} - -//Test the log scrubber on known problematic log messages -func TestLogScrubberMessages(t *testing.T) { - for _, test := range []struct { - input, expected string - }{ - { - "http: TLS handshake error from 129.97.208.23:38310: ", - "http: TLS handshake error from [scrubbed]: \n", - }, - { - "http2: panic serving [2620:101:f000:780:9097:75b1:519f:dbb8]:58344: interface conversion: *http2.responseWriter is not http.Hijacker: missing method Hijack", - "http2: panic serving [scrubbed]: interface conversion: *http2.responseWriter is not http.Hijacker: missing method Hijack\n", - }, - { - //Make sure it doesn't scrub fingerprint - "a=fingerprint:sha-256 33:B6:FA:F6:94:CA:74:61:45:4A:D2:1F:2C:2F:75:8A:D9:EB:23:34:B2:30:E9:1B:2A:A6:A9:E0:44:72:CC:74", - "a=fingerprint:sha-256 33:B6:FA:F6:94:CA:74:61:45:4A:D2:1F:2C:2F:75:8A:D9:EB:23:34:B2:30:E9:1B:2A:A6:A9:E0:44:72:CC:74\n", - }, - { - //try with enclosing parens - "(1:2:3:4:c:d:e:f) {1:2:3:4:c:d:e:f}", - "([scrubbed]) {[scrubbed]}\n", - }, - { - //Make sure it doesn't scrub timestamps - "2019/05/08 15:37:31 starting", - "2019/05/08 15:37:31 starting\n", - }, - } { - var buff bytes.Buffer - log.SetFlags(0) //remove all extra log output for test comparisons - log.SetOutput(&LogScrubber{Output: &buff}) - log.Print(test.input) - if buff.String() != test.expected { - t.Errorf("%q: got %q, expected %q", test.input, buff.String(), test.expected) - } - } - -} - -func TestLogScrubberGoodFormats(t *testing.T) { - for _, addr := range []string{ - // IPv4 - "1.2.3.4", - "255.255.255.255", - // IPv4 with port - "1.2.3.4:55", - "255.255.255.255:65535", - // IPv6 - "1:2:3:4:c:d:e:f", - "1111:2222:3333:4444:CCCC:DDDD:EEEE:FFFF", - // IPv6 with brackets - "[1:2:3:4:c:d:e:f]", - "[1111:2222:3333:4444:CCCC:DDDD:EEEE:FFFF]", - // IPv6 with brackets and port - "[1:2:3:4:c:d:e:f]:55", - "[1111:2222:3333:4444:CCCC:DDDD:EEEE:FFFF]:65535", - // compressed IPv6 - "::f", - "::d:e:f", - "1:2:3::", - "1:2:3::d:e:f", - "1:2:3:d:e:f::", - "::1:2:3:d:e:f", - "1111:2222:3333::DDDD:EEEE:FFFF", - // compressed IPv6 with brackets - "[::d:e:f]", - "[1:2:3::]", - "[1:2:3::d:e:f]", - "[1111:2222:3333::DDDD:EEEE:FFFF]", - "[1:2:3:4:5:6::8]", - "[1::7:8]", - // compressed IPv6 with brackets and port - "[1::]:58344", - "[::d:e:f]:55", - "[1:2:3::]:55", - "[1:2:3::d:e:f]:55", - "[1111:2222:3333::DDDD:EEEE:FFFF]:65535", - // IPv4-compatible and IPv4-mapped - "::255.255.255.255", - "::ffff:255.255.255.255", - "[::255.255.255.255]", - "[::ffff:255.255.255.255]", - "[::255.255.255.255]:65535", - "[::ffff:255.255.255.255]:65535", - "[::ffff:0:255.255.255.255]", - "[2001:db8:3:4::192.0.2.33]", - } { - var buff bytes.Buffer - log.SetFlags(0) //remove all extra log output for test comparisons - log.SetOutput(&LogScrubber{Output: &buff}) - log.Print(addr) - if buff.String() != "[scrubbed]\n" { - t.Errorf("%q: Got %q, expected %q", addr, buff.String(), "[scrubbed]\n") - } - } -} diff --git a/common/util/util.go b/common/util/util.go deleted file mode 100644 index fa62fd7..0000000 --- a/common/util/util.go +++ /dev/null @@ -1,103 +0,0 @@ -package util - -import ( - "encoding/json" - "log" - "net" - - "github.com/pion/sdp/v2" - "github.com/pion/webrtc/v2" -) - -func SerializeSessionDescription(desc *webrtc.SessionDescription) string { - bytes, err := json.Marshal(*desc) - if nil != err { - log.Println(err) - return "" - } - return string(bytes) -} - -func DeserializeSessionDescription(msg string) *webrtc.SessionDescription { - var parsed map[string]interface{} - err := json.Unmarshal([]byte(msg), &parsed) - if nil != err { - log.Println(err) - return nil - } - if _, ok := parsed["type"]; !ok { - log.Println("Cannot deserialize SessionDescription without type field.") - return nil - } - if _, ok := parsed["sdp"]; !ok { - log.Println("Cannot deserialize SessionDescription without sdp field.") - return nil - } - - var stype webrtc.SDPType - switch parsed["type"].(string) { - default: - log.Println("Unknown SDP type") - return nil - case "offer": - stype = webrtc.SDPTypeOffer - case "pranswer": - stype = webrtc.SDPTypePranswer - case "answer": - stype = webrtc.SDPTypeAnswer - case "rollback": - stype = webrtc.SDPTypeRollback - } - - if err != nil { - log.Println(err) - return nil - } - return &webrtc.SessionDescription{ - Type: stype, - SDP: parsed["sdp"].(string), - } -} - -// Stolen from https://github.com/golang/go/pull/30278 -func IsLocal(ip net.IP) bool { - if ip4 := ip.To4(); ip4 != nil { - // Local IPv4 addresses are defined in https://tools.ietf.org/html/rfc1918 - return ip4[0] == 10 || - (ip4[0] == 172 && ip4[1]&0xf0 == 16) || - (ip4[0] == 192 && ip4[1] == 168) - } - // Local IPv6 addresses are defined in https://tools.ietf.org/html/rfc4193 - return len(ip) == net.IPv6len && ip[0]&0xfe == 0xfc -} - -// Removes local LAN address ICE candidates -func StripLocalAddresses(str string) string { - var desc sdp.SessionDescription - err := desc.Unmarshal([]byte(str)) - if err != nil { - return str - } - for _, m := range desc.MediaDescriptions { - attrs := make([]sdp.Attribute, 0) - for _, a := range m.Attributes { - if a.IsICECandidate() { - ice, err := a.ToICECandidate() - if err == nil && ice.Typ == "host" { - ip := net.ParseIP(ice.Address) - if ip != nil && (IsLocal(ip) || ip.IsUnspecified() || ip.IsLoopback()) { - /* no append in this case */ - continue - } - } - } - attrs = append(attrs, a) - } - m.Attributes = attrs - } - bts, err := desc.Marshal() - if err != nil { - return str - } - return string(bts) -} diff --git a/common/util/util_test.go b/common/util/util_test.go deleted file mode 100644 index 271619a..0000000 --- a/common/util/util_test.go +++ /dev/null @@ -1,26 +0,0 @@ -package util - -import ( - "testing" - - . "github.com/smartystreets/goconvey/convey" -) - -func TestUtil(t *testing.T) { - Convey("Strip", t, func() { - const offerStart = "v=0\r\no=- 4358805017720277108 2 IN IP4 8.8.8.8\r\ns=-\r\nt=0 0\r\na=group:BUNDLE data\r\na=msid-semantic: WMS\r\nm=application 56688 DTLS/SCTP 5000\r\nc=IN IP4 8.8.8.8\r\n" - const goodCandidate = "a=candidate:3769337065 1 udp 2122260223 8.8.8.8 56688 typ host generation 0 network-id 1 network-cost 50\r\n" - const offerEnd = "a=ice-ufrag:aMAZ\r\na=ice-pwd:jcHb08Jjgrazp2dzjdrvPPvV\r\na=ice-options:trickle\r\na=fingerprint:sha-256 C8:88:EE:B9:E7:02:2E:21:37:ED:7A:D1:EB:2B:A3:15:A2:3B:5B:1C:3D:D4:D5:1F:06:CF:52:40:03:F8:DD:66\r\na=setup:actpass\r\na=mid:data\r\na=sctpmap:5000 webrtc-datachannel 1024\r\n" - - offer := offerStart + goodCandidate + - "a=candidate:3769337065 1 udp 2122260223 192.168.0.100 56688 typ host generation 0 network-id 1 network-cost 50\r\n" + // IsLocal IPv4 - "a=candidate:3769337065 1 udp 2122260223 fdf8:f53b:82e4::53 56688 typ host generation 0 network-id 1 network-cost 50\r\n" + // IsLocal IPv6 - "a=candidate:3769337065 1 udp 2122260223 0.0.0.0 56688 typ host generation 0 network-id 1 network-cost 50\r\n" + // IsUnspecified IPv4 - "a=candidate:3769337065 1 udp 2122260223 :: 56688 typ host generation 0 network-id 1 network-cost 50\r\n" + // IsUnspecified IPv6 - "a=candidate:3769337065 1 udp 2122260223 127.0.0.1 56688 typ host generation 0 network-id 1 network-cost 50\r\n" + // IsLoopback IPv4 - "a=candidate:3769337065 1 udp 2122260223 ::1 56688 typ host generation 0 network-id 1 network-cost 50\r\n" + // IsLoopback IPv6 - offerEnd - - So(StripLocalAddresses(offer), ShouldEqual, offerStart+goodCandidate+offerEnd) - }) -} diff --git a/common/websocketconn/websocketconn.go b/common/websocketconn/websocketconn.go deleted file mode 100644 index c745522..0000000 --- a/common/websocketconn/websocketconn.go +++ /dev/null @@ -1,120 +0,0 @@ -package websocketconn - -import ( - "io" - "time" - - "github.com/gorilla/websocket" -) - -// An abstraction that makes an underlying WebSocket connection look like a -// net.Conn. -type Conn struct { - *websocket.Conn - Reader io.Reader - Writer io.Writer -} - -func (conn *Conn) Read(b []byte) (n int, err error) { - return conn.Reader.Read(b) -} - -func (conn *Conn) Write(b []byte) (n int, err error) { - return conn.Writer.Write(b) -} - -func (conn *Conn) Close() error { - conn.Reader.(*io.PipeReader).Close() - conn.Writer.(*io.PipeWriter).Close() - // Ignore any error in trying to write a Close frame. - _ = conn.Conn.WriteControl(websocket.CloseMessage, []byte{}, time.Now().Add(time.Second)) - return conn.Conn.Close() -} - -func (conn *Conn) SetDeadline(t time.Time) error { - errRead := conn.Conn.SetReadDeadline(t) - errWrite := conn.Conn.SetWriteDeadline(t) - err := errRead - if err == nil { - err = errWrite - } - return err -} - -func readLoop(w io.Writer, ws *websocket.Conn) error { - for { - messageType, r, err := ws.NextReader() - if err != nil { - return err - } - if messageType != websocket.BinaryMessage && messageType != websocket.TextMessage { - continue - } - _, err = io.Copy(w, r) - if err != nil { - return err - } - } -} - -func writeLoop(ws *websocket.Conn, r io.Reader) error { - for { - var buf [2048]byte - n, err := r.Read(buf[:]) - if err != nil { - return err - } - data := buf[:n] - w, err := ws.NextWriter(websocket.BinaryMessage) - if err != nil { - return err - } - n, err = w.Write(data) - if err != nil { - return err - } - err = w.Close() - if err != nil { - return err - } - } -} - -// websocket.Conn methods start returning websocket.CloseError after the -// connection has been closed. We want to instead interpret that as io.EOF, just -// as you would find with a normal net.Conn. This only converts -// websocket.CloseErrors with known codes; other codes like CloseProtocolError -// and CloseAbnormalClosure will still be reported as anomalous. -func closeErrorToEOF(err error) error { - if websocket.IsCloseError(err, websocket.CloseNormalClosure, websocket.CloseNoStatusReceived) { - err = io.EOF - } - return err -} - -// Create a new Conn. -func New(ws *websocket.Conn) *Conn { - // Set up synchronous pipes to serialize reads and writes to the - // underlying websocket.Conn. - // - // https://godoc.org/github.com/gorilla/websocket#hdr-Concurrency - // "Connections support one concurrent reader and one concurrent writer. - // Applications are responsible for ensuring that no more than one - // goroutine calls the write methods (NextWriter, etc.) concurrently and - // that no more than one goroutine calls the read methods (NextReader, - // etc.) concurrently. The Close and WriteControl methods can be called - // concurrently with all other methods." - pr1, pw1 := io.Pipe() - go func() { - pw1.CloseWithError(closeErrorToEOF(readLoop(pw1, ws))) - }() - pr2, pw2 := io.Pipe() - go func() { - pr2.CloseWithError(closeErrorToEOF(writeLoop(ws, pr2))) - }() - return &Conn{ - Conn: ws, - Reader: pr1, - Writer: pw2, - } -} diff --git a/common/websocketconn/websocketconn_test.go b/common/websocketconn/websocketconn_test.go deleted file mode 100644 index 92774d4..0000000 --- a/common/websocketconn/websocketconn_test.go +++ /dev/null @@ -1,263 +0,0 @@ -package websocketconn - -import ( - "bytes" - "fmt" - "io" - "io/ioutil" - "net" - "net/http" - "net/url" - "sync" - "testing" - "time" - - "github.com/gorilla/websocket" -) - -// Returns a (server, client) pair of websocketconn.Conns. -func connPair() (*Conn, *Conn, error) { - // Will be assigned inside server.Handler. - var serverConn *Conn - - // Start up a web server to receive the request. - ln, err := net.Listen("tcp", "127.0.0.1:0") - if err != nil { - return nil, nil, err - } - defer ln.Close() - errCh := make(chan error) - server := http.Server{ - Handler: http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) { - upgrader := websocket.Upgrader{ - CheckOrigin: func(*http.Request) bool { return true }, - } - ws, err := upgrader.Upgrade(rw, req, nil) - if err != nil { - errCh <- err - return - } - serverConn = New(ws) - close(errCh) - }), - } - defer server.Close() - go func() { - err := server.Serve(ln) - if err != nil && err != http.ErrServerClosed { - errCh <- err - } - }() - - // Make a request to the web server. - urlStr := (&url.URL{Scheme: "ws", Host: ln.Addr().String()}).String() - ws, _, err := (&websocket.Dialer{}).Dial(urlStr, nil) - if err != nil { - return nil, nil, err - } - clientConn := New(ws) - - // The server is finished when errCh is written to or closed. - err = <-errCh - if err != nil { - return nil, nil, err - } - return serverConn, clientConn, nil -} - -// Test that you can write in chunks and read the result concatenated. -func TestWrite(t *testing.T) { - tests := [][][]byte{ - {}, - {[]byte("foo")}, - {[]byte("foo"), []byte("bar")}, - {{}, []byte("foo"), {}, {}, []byte("bar")}, - } - - for _, test := range tests { - s, c, err := connPair() - if err != nil { - t.Fatal(err) - } - - // This is a little awkward because we need to read to and write - // from both ends of the Conn, and we need to do it in separate - // goroutines because otherwise a Write may block waiting for - // someone to Read it. Here we set up a loop in a separate - // goroutine, reading from the Conn s and writing to the dataCh - // and errCh channels, whose ultimate effect in the select loop - // below is like - // data, err := ioutil.ReadAll(s) - dataCh := make(chan []byte) - errCh := make(chan error) - go func() { - for { - var buf [1024]byte - n, err := s.Read(buf[:]) - if err != nil { - errCh <- err - return - } - p := make([]byte, n) - copy(p, buf[:]) - dataCh <- p - } - }() - - // Write the data to the client side of the Conn, one chunk at a - // time. - for i, chunk := range test { - n, err := c.Write(chunk) - if err != nil || n != len(chunk) { - t.Fatalf("%+q Write chunk %d: got (%d, %v), expected (%d, %v)", - test, i, n, err, len(chunk), nil) - } - } - // We cannot immediately c.Close here, because that closes the - // connection right away, without waiting for buffered data to - // be sent. - - // Pull data and err from the server goroutine above. - var data []byte - err = nil - loop: - for { - select { - case p := <-dataCh: - data = append(data, p...) - case err = <-errCh: - break loop - case <-time.After(100 * time.Millisecond): - break loop - } - } - s.Close() - c.Close() - - // Now data and err contain the result of reading everything - // from s. - expected := bytes.Join(test, []byte{}) - if err != nil || !bytes.Equal(data, expected) { - t.Fatalf("%+q ReadAll: got (%+q, %v), expected (%+q, %v)", - test, data, err, expected, nil) - } - } -} - -// Test that multiple goroutines may call Read on a Conn simultaneously. Run -// this with -// go test -race -func TestConcurrentRead(t *testing.T) { - s, c, err := connPair() - if err != nil { - t.Fatal(err) - } - defer s.Close() - - // Set up multiple threads reading from the same conn. - errCh := make(chan error, 2) - var wg sync.WaitGroup - wg.Add(2) - for i := 0; i < 2; i++ { - go func() { - defer wg.Done() - _, err := io.Copy(ioutil.Discard, s) - if err != nil { - errCh <- err - } - }() - } - - // Write a bunch of data to the other end. - for i := 0; i < 2000; i++ { - _, err := fmt.Fprintf(c, "%d", i) - if err != nil { - c.Close() - t.Fatalf("Write: %v", err) - } - } - c.Close() - - wg.Wait() - close(errCh) - - err = <-errCh - if err != nil { - t.Fatalf("Read: %v", err) - } -} - -// Test that multiple goroutines may call Write on a Conn simultaneously. Run -// this with -// go test -race -func TestConcurrentWrite(t *testing.T) { - s, c, err := connPair() - if err != nil { - t.Fatal(err) - } - - // Set up multiple threads writing to the same conn. - errCh := make(chan error, 3) - var wg sync.WaitGroup - wg.Add(2) - for i := 0; i < 2; i++ { - go func() { - defer wg.Done() - for j := 0; j < 1000; j++ { - _, err := fmt.Fprintf(s, "%d", j) - if err != nil { - errCh <- err - break - } - } - }() - } - go func() { - wg.Wait() - err := s.Close() - if err != nil { - errCh <- err - } - close(errCh) - }() - - // Read from the other end. - _, err = io.Copy(ioutil.Discard, c) - c.Close() - if err != nil { - t.Fatalf("Read: %v", err) - } - - err = <-errCh - if err != nil { - t.Fatalf("Write: %v", err) - } -} - -// Test that Read and Write methods return errors after Close. -func TestClose(t *testing.T) { - s, c, err := connPair() - if err != nil { - t.Fatal(err) - } - defer c.Close() - - err = s.Close() - if err != nil { - t.Fatal(err) - } - - var buf [10]byte - n, err := s.Read(buf[:]) - if n != 0 || err == nil { - t.Fatalf("Read after Close returned (%v, %v), expected (%v, non-nil)", n, err, 0) - } - - _, err = s.Write([]byte{1, 2, 3}) - // Here we break the abstraction a little and look for a specific error, - // io.ErrClosedPipe. This is because we know the Conn uses an io.Pipe - // internally. - if err != io.ErrClosedPipe { - t.Fatalf("Write after Close returned %v, expected %v", err, io.ErrClosedPipe) - } -} diff --git a/doc/broker-spec.txt b/doc/broker-spec.txt deleted file mode 100644 index eba3347..0000000 --- a/doc/broker-spec.txt +++ /dev/null @@ -1,69 +0,0 @@ - - - Snowflake broker protocol - -0. Scope and Preliminaries - -The Snowflake broker is used to hand out Snowflake proxies to clients using the Snowflake pluggable transport. There are some similarities to the function of the broker and how BridgeDB hands out Tor bridges. - -This document specifies how the Snowflake broker interacts with other parts of the Tor ecosystem, starting with the metrics CollecTor module and to be expanded upon later. - -1. Metrics Reporting (version 1.0) - -Metrics data from the Snowflake broker can be retrieved by sending an HTTP GET request to https://[Snowflake broker URL]/metrics and consists of the following items: - - "snowflake-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL - [At start, exactly once.] - - YYYY-MM-DD HH:MM:SS defines the end of the included measurement - interval of length NSEC seconds (86400 seconds by default). - - "snowflake-ips" [CC=NUM,CC=NUM,...,CC=NUM] NL - [At most once.] - - List of mappings from two-letter country codes to the number of - unique IP addresses of Snowflake proxies that have polled. Each - country code only appears once. - - "snowflake-ips-total" NUM NL - [At most once.] - - A count of the total number of unique IP addresses of Snowflake - proxies that have polled. - - "snowflake-ips-standalone" NUM NL - [At most once.] - - A count of the total number of unique IP addresses of snowflake - proxies of type "standalone" that have polled. - - "snowflake-ips-badge" NUM NL - [At most once.] - - A count of the total number of unique IP addresses of snowflake - proxies of type "badge" that have polled. - - "snowflake-ips-webext" NUM NL - [At most once.] - - A count of the total number of unique IP addresses of snowflake - proxies of type "webext" that have polled. - - "snowflake-idle-count" NUM NL - [At most once.] - - A count of the number of times a proxy has polled but received - no client offer, rounded up to the nearest multiple of 8. - - "client-denied-count" NUM NL - [At most once.] - - A count of the number of times a client has requested a proxy - from the broker but no proxies were available, rounded up to - the nearest multiple of 8. - - "client-snowflake-match-count" NUM NL - [At most once.] - - A count of the number of times a client successfully received a - proxy from the broker, rounded up to the nearest multiple of 8. diff --git a/go.mod b/go.mod deleted file mode 100644 index 4366d6a..0000000 --- a/go.mod +++ /dev/null @@ -1,15 +0,0 @@ -module git.torproject.org/pluggable-transports/snowflake.git - -go 1.13 - -require ( - git.torproject.org/pluggable-transports/goptlib.git v1.1.0 - github.com/golang/protobuf v1.3.1 // indirect - github.com/gorilla/websocket v1.4.1 - github.com/pion/sdp/v2 v2.3.4 - github.com/pion/webrtc/v2 v2.2.2 - github.com/smartystreets/goconvey v1.6.4 - golang.org/x/crypto v0.0.0-20200128174031-69ecbb4d6d5d - golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa - golang.org/x/text v0.3.2 // indirect -) diff --git a/go.sum b/go.sum deleted file mode 100644 index 3708fc0..0000000 --- a/go.sum +++ /dev/null @@ -1,119 +0,0 @@ -git.torproject.org/pluggable-transports/goptlib.git v1.1.0 h1:LMQAA8pAho+QtYrrVNimJQiINNEwcwuuD99vezD/PAo= -git.torproject.org/pluggable-transports/goptlib.git v1.1.0/go.mod h1:YT4XMSkuEXbtqlydr9+OxqFAyspUv0Gr9qhM3B++o/Q= -github.com/cheekybits/genny v1.0.0 h1:uGGa4nei+j20rOSeDeP5Of12XVm7TGUd4dJA9RDitfE= -github.com/cheekybits/genny v1.0.0/go.mod h1:+tQajlRqAUrPI7DOSpB0XAqZYtQakVtB7wXkRAgjxjQ= -github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= -github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I= -github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= -github.com/golang/mock v1.2.0 h1:28o5sBqPkBsMGnC6b4MvE2TzSr5/AT4c/1fLqVGIwlk= -github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= -github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM= -github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.1 h1:YF8+flBXS5eO826T4nzqPrxfhQThhXl0YzfuUPu4SBg= -github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8= -github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= -github.com/gorilla/websocket v1.4.1 h1:q7AeDBpnBk8AogcD4DSag/Ukw/KV+YhzLj2bP5HvKCM= -github.com/gorilla/websocket v1.4.1/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= -github.com/hpcloud/tail v1.0.0 h1:nfCOvKYfkgYP8hkirhJocXT2+zOD8yUNjXaWfTlyFKI= -github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= -github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= -github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= -github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI= -github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= -github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= -github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= -github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= -github.com/lucas-clemente/quic-go v0.7.1-0.20190401152353-907071221cf9 h1:tbuodUh2vuhOVZAdW3NEUvosFHUMJwUNl7jk/VSEiwc= -github.com/lucas-clemente/quic-go v0.7.1-0.20190401152353-907071221cf9/go.mod h1:PpMmPfPKO9nKJ/psF49ESTAGQSdfXxlg1otPbEB2nOw= -github.com/marten-seemann/qtls v0.2.3 h1:0yWJ43C62LsZt08vuQJDK1uC1czUc3FJeCLPoNAI4vA= -github.com/marten-seemann/qtls v0.2.3/go.mod h1:xzjG7avBwGGbdZ8dTGxlBnLArsVKLvwmjgmPuiQEcYk= -github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= -github.com/onsi/ginkgo v1.7.0 h1:WSHQ+IS43OoUrWtD1/bbclrwK8TTH5hzp+umCiuxHgs= -github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= -github.com/onsi/gomega v1.4.3 h1:RE1xgDvH7imwFD45h+u2SgIfERHlS2yNG4DObb5BSKU= -github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= -github.com/pion/datachannel v1.4.15 h1:DrizUL97f9evDyoskyMLrFFFmOWCfXFGiGSbxmQyYt4= -github.com/pion/datachannel v1.4.15/go.mod h1:yixWvOWPime7vRVuihP1GzZPBELQkO/ZM1mrBc2BNM8= -github.com/pion/dtls/v2 v2.0.0-rc.7 h1:LDAIQDt1pcuAIJs7Q2EZ3PSl8MseCFA2nCW0YYSYCx0= -github.com/pion/dtls/v2 v2.0.0-rc.7/go.mod h1:U199DvHpRBN0muE9+tVN4TMy1jvEhZIZ63lk4xkvVSk= -github.com/pion/ice v0.7.9 h1:RKol/0RFu3TIE8ZLIFV1A1e/QW22B6BZKvSG9sfawEM= -github.com/pion/ice v0.7.9/go.mod h1:8BCwuq/EqAKhtUb8CIw2fWjVLotWOu13XJY09H3RVxA= -github.com/pion/logging v0.2.2 h1:M9+AIj/+pxNsDfAT64+MAVgJO0rsyLnoJKCqf//DoeY= -github.com/pion/logging v0.2.2/go.mod h1:k0/tDVsRCX2Mb2ZEmTqNa7CWsQPc+YYCB7Q+5pahoms= -github.com/pion/mdns v0.0.4 h1:O4vvVqr4DGX63vzmO6Fw9vpy3lfztVWHGCQfyw0ZLSY= -github.com/pion/mdns v0.0.4/go.mod h1:R1sL0p50l42S5lJs91oNdUL58nm0QHrhxnSegr++qC0= -github.com/pion/quic v0.1.1 h1:D951FV+TOqI9A0rTF7tHx0Loooqz+nyzjEyj8o3PuMA= -github.com/pion/quic v0.1.1/go.mod h1:zEU51v7ru8Mp4AUBJvj6psrSth5eEFNnVQK5K48oV3k= -github.com/pion/rtcp v1.2.1 h1:S3yG4KpYAiSmBVqKAfgRa5JdwBNj4zK3RLUa8JYdhak= -github.com/pion/rtcp v1.2.1/go.mod h1:a5dj2d6BKIKHl43EnAOIrCczcjESrtPuMgfmL6/K6QM= -github.com/pion/rtp v1.3.0/go.mod h1:q9wPnA96pu2urCcW/sK/RiDn597bhGoAQQ+y2fDwHuY= -github.com/pion/rtp v1.3.2 h1:Yfzf1mU4Zmg7XWHitzYe2i+l+c68iO+wshzIUW44p1c= -github.com/pion/rtp v1.3.2/go.mod h1:q9wPnA96pu2urCcW/sK/RiDn597bhGoAQQ+y2fDwHuY= -github.com/pion/sctp v1.7.5 h1:ognJDlxP7dN2xMUEHEea5pqjdD78o5UAMcLoP1JIp1g= -github.com/pion/sctp v1.7.5/go.mod h1:ichkYQ5tlgCQwEwvgfdcAolqx1nHbYCxo4D7zK/K0X8= -github.com/pion/sdp/v2 v2.3.4 h1:+f3F5Xl7ynVhc9Il8Dc7BFroYJWG3PMbfWtwFlVI+kg= -github.com/pion/sdp/v2 v2.3.4/go.mod h1:jccXVYW0fuK6ds2pwKr89SVBDYlCjhgMI6nucl5R5rA= -github.com/pion/srtp v1.2.7 h1:UYyLs5MXwbFtXWduBA5+RUWhaEBX7GmetXDZSKP+uPM= -github.com/pion/srtp v1.2.7/go.mod h1:KIgLSadhg/ioogO/LqIkRjZrwuJo0c9RvKIaGQj4Yew= -github.com/pion/stun v0.3.3 h1:brYuPl9bN9w/VM7OdNzRSLoqsnwlyNvD9MVeJrHjDQw= -github.com/pion/stun v0.3.3/go.mod h1:xrCld6XM+6GWDZdvjPlLMsTU21rNxnO6UO8XsAvHr/M= -github.com/pion/transport v0.6.0/go.mod h1:iWZ07doqOosSLMhZ+FXUTq+TamDoXSllxpbGcfkCmbE= -github.com/pion/transport v0.8.10 h1:lTiobMEw2PG6BH/mgIVqTV2mBp/mPT+IJLaN8ZxgdHk= -github.com/pion/transport v0.8.10/go.mod h1:tBmha/UCjpum5hqTWhfAEs3CO4/tHSg0MYRhSzR+CZ8= -github.com/pion/turn/v2 v2.0.3 h1:SJUUIbcPoehlyZgMyIUbBBDhI03sBx32x3JuSIBKBWA= -github.com/pion/turn/v2 v2.0.3/go.mod h1:kl1hmT3NxcLynpXVnwJgObL8C9NaCyPTeqI2DcCpSZs= -github.com/pion/webrtc/v2 v2.2.2 h1:ace9itTe8YND8m3lv5ndQurfk/DsChj+4pBzVJeBA04= -github.com/pion/webrtc/v2 v2.2.2/go.mod h1:oftEPcdfIvZVC1J0VP1OpyVCwB9tDkRXSYAszkL/2k4= -github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= -github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= -github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/sclevine/agouti v3.0.0+incompatible/go.mod h1:b4WX9W9L1sfQKXeJf1mUTLZKJ48R1S7H23Ji7oFO5Bw= -github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d h1:zE9ykElWQ6/NYmHa3jpm/yHnI4xSofP+UP6SpjHcSeM= -github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= -github.com/smartystreets/goconvey v1.6.4 h1:fv0U8FUIMPNf1L9lnHLvLhgicrIVChEkdzIKYqbNC9s= -github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= -github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= -github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= -github.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4= -github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= -golang.org/x/crypto v0.0.0-20190228161510-8dd112bcdc25/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= -golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= -golang.org/x/crypto v0.0.0-20200128174031-69ecbb4d6d5d h1:9FCpayM9Egr1baVnV1SX0H87m+XB0B8S0hAMi99X/3U= -golang.org/x/crypto v0.0.0-20200128174031-69ecbb4d6d5d/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/net v0.0.0-20191126235420-ef20fe5d7933/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa h1:F+8P+gmewFQYRk6JoLQLwjBCTu3mcIURZfNkVweuRKA= -golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f h1:wMNYb4v58l5UBM7MYRLPG6ZhfOqbKu7X5eyFl8ZhKvA= -golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190228124157-a34e9553db1e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190412213103-97732733099d h1:+R4KGOnez64A81RvjARKc4UT5/tI9ujCIVX+P5KiHuI= -golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg= -golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= -golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs= -golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= -golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= -golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4= -golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo= -gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/fsnotify.v1 v1.4.7 h1:xOHLXZwVvI9hhs+cLKq5+I5onOuwQLhQwiu63xxlHs4= -gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= -gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ= -gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= -gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw= -gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= diff --git a/proxy-go/README.md b/proxy-go/README.md deleted file mode 100644 index 264fc4f..0000000 --- a/proxy-go/README.md +++ /dev/null @@ -1,3 +0,0 @@ -This is a standalone (not browser-based) version of the Snowflake proxy. - -Usage: ./proxy-go diff --git a/proxy-go/proxy-go_test.go b/proxy-go/proxy-go_test.go deleted file mode 100644 index bed00f2..0000000 --- a/proxy-go/proxy-go_test.go +++ /dev/null @@ -1,397 +0,0 @@ -package main - -import ( - "bytes" - "fmt" - "io" - "io/ioutil" - "net" - "net/http" - "net/url" - "strconv" - "strings" - "testing" - - "git.torproject.org/pluggable-transports/snowflake.git/common/messages" - "git.torproject.org/pluggable-transports/snowflake.git/common/util" - "github.com/pion/webrtc/v2" - . "github.com/smartystreets/goconvey/convey" -) - -// Set up a mock broker to communicate with -type MockTransport struct { - statusOverride int - body []byte -} - -// Just returns a response with fake SDP answer. -func (m *MockTransport) RoundTrip(req *http.Request) (*http.Response, error) { - s := ioutil.NopCloser(bytes.NewReader(m.body)) - r := &http.Response{ - StatusCode: m.statusOverride, - Body: s, - } - return r, nil -} - -// Set up a mock faulty transport -type FaultyTransport struct { - statusOverride int - body []byte -} - -// Just returns a response with fake SDP answer. -func (f *FaultyTransport) RoundTrip(req *http.Request) (*http.Response, error) { - return nil, fmt.Errorf("TransportFailed") -} - -func TestRemoteIPFromSDP(t *testing.T) { - tests := []struct { - sdp string - expected net.IP - }{ - // https://tools.ietf.org/html/rfc4566#section-5 - {`v=0 -o=jdoe 2890844526 2890842807 IN IP4 10.47.16.5 -s=SDP Seminar -i=A Seminar on the session description protocol -u=http://www.example.com/seminars/sdp.pdf -e=j.doe(a)example.com (Jane Doe) -c=IN IP4 224.2.17.12/127 -t=2873397496 2873404696 -a=recvonly -m=audio 49170 RTP/AVP 0 -m=video 51372 RTP/AVP 99 -a=rtpmap:99 h263-1998/90000 -`, net.ParseIP("224.2.17.12")}, - // Missing c= line - {`v=0 -o=jdoe 2890844526 2890842807 IN IP4 10.47.16.5 -s=SDP Seminar -i=A Seminar on the session description protocol -u=http://www.example.com/seminars/sdp.pdf -e=j.doe(a)example.com (Jane Doe) -t=2873397496 2873404696 -a=recvonly -m=audio 49170 RTP/AVP 0 -m=video 51372 RTP/AVP 99 -a=rtpmap:99 h263-1998/90000 -`, nil}, - // Single line, IP address only - {`c=IN IP4 224.2.1.1 -`, net.ParseIP("224.2.1.1")}, - // Same, with TTL - {`c=IN IP4 224.2.1.1/127 -`, net.ParseIP("224.2.1.1")}, - // Same, with TTL and multicast addresses - {`c=IN IP4 224.2.1.1/127/3 -`, net.ParseIP("224.2.1.1")}, - // IPv6, address only - {`c=IN IP6 FF15::101 -`, net.ParseIP("ff15::101")}, - // Same, with multicast addresses - {`c=IN IP6 FF15::101/3 -`, net.ParseIP("ff15::101")}, - // Multiple c= lines - {`c=IN IP4 1.2.3.4 -c=IN IP4 5.6.7.8 -`, net.ParseIP("1.2.3.4")}, - // Modified from SDP sent by snowflake-client. - {`v=0 -o=- 7860378660295630295 2 IN IP4 127.0.0.1 -s=- -t=0 0 -a=group:BUNDLE data -a=msid-semantic: WMS -m=application 54653 DTLS/SCTP 5000 -c=IN IP4 1.2.3.4 -a=candidate:3581707038 1 udp 2122260223 192.168.0.1 54653 typ host generation 0 network-id 1 network-cost 50 -a=candidate:2617212910 1 tcp 1518280447 192.168.0.1 59673 typ host tcptype passive generation 0 network-id 1 network-cost 50 -a=candidate:2082671819 1 udp 1686052607 1.2.3.4 54653 typ srflx raddr 192.168.0.1 rport 54653 generation 0 network-id 1 network-cost 50 -a=ice-ufrag:IBdf -a=ice-pwd:G3lTrrC9gmhQx481AowtkhYz -a=fingerprint:sha-256 53:F8:84:D9:3C:1F:A0:44:AA:D6:3C:65:80:D3:CB:6F:23:90:17:41:06:F9:9C:10:D8:48:4A:A8:B6:FA:14:A1 -a=setup:actpass -a=mid:data -a=sctpmap:5000 webrtc-datachannel 1024 -`, net.ParseIP("1.2.3.4")}, - // Improper character within IPv4 - {`c=IN IP4 224.2z.1.1 -`, nil}, - // Improper character within IPv6 - {`c=IN IP6 ff15:g::101 -`, nil}, - // Bogus "IP7" addrtype - {`c=IN IP7 1.2.3.4 -`, nil}, - } - - for _, test := range tests { - // https://tools.ietf.org/html/rfc4566#section-5: "The sequence - // CRLF (0x0d0a) is used to end a record, although parsers - // SHOULD be tolerant and also accept records terminated with a - // single newline character." We represent the test cases with - // LF line endings for convenience, and test them both that way - // and with CRLF line endings. - lfSDP := test.sdp - crlfSDP := strings.Replace(lfSDP, "\n", "\r\n", -1) - - ip := remoteIPFromSDP(lfSDP) - if !ip.Equal(test.expected) { - t.Errorf("expected %q, got %q from %q", test.expected, ip, lfSDP) - } - ip = remoteIPFromSDP(crlfSDP) - if !ip.Equal(test.expected) { - t.Errorf("expected %q, got %q from %q", test.expected, ip, crlfSDP) - } - } -} - -func TestSessionDescriptions(t *testing.T) { - Convey("Session description deserialization", t, func() { - for _, test := range []struct { - msg string - ret *webrtc.SessionDescription - }{ - { - "test", - nil, - }, - { - `{"type":"answer"}`, - nil, - }, - { - `{"sdp":"test"}`, - nil, - }, - { - `{"type":"test", "sdp":"test"}`, - nil, - }, - { - `{"type":"answer", "sdp":"test"}`, - &webrtc.SessionDescription{ - Type: webrtc.SDPTypeAnswer, - SDP: "test", - }, - }, - { - `{"type":"pranswer", "sdp":"test"}`, - &webrtc.SessionDescription{ - Type: webrtc.SDPTypePranswer, - SDP: "test", - }, - }, - { - `{"type":"rollback", "sdp":"test"}`, - &webrtc.SessionDescription{ - Type: webrtc.SDPTypeRollback, - SDP: "test", - }, - }, - { - `{"type":"offer", "sdp":"test"}`, - &webrtc.SessionDescription{ - Type: webrtc.SDPTypeOffer, - SDP: "test", - }, - }, - } { - desc := util.DeserializeSessionDescription(test.msg) - So(desc, ShouldResemble, test.ret) - } - }) - Convey("Session description serialization", t, func() { - for _, test := range []struct { - desc *webrtc.SessionDescription - ret string - }{ - { - &webrtc.SessionDescription{ - Type: webrtc.SDPTypeOffer, - SDP: "test", - }, - `{"type":"offer","sdp":"test"}`, - }, - } { - msg := util.SerializeSessionDescription(test.desc) - So(msg, ShouldResemble, test.ret) - } - }) -} - -func TestBrokerInteractions(t *testing.T) { - const sampleSDP = `"v=0\r\no=- 4358805017720277108 2 IN IP4 8.8.8.8\r\ns=-\r\nt=0 0\r\na=group:BUNDLE data\r\na=msid-semantic: WMS\r\nm=application 56688 DTLS/SCTP 5000\r\nc=IN IP4 8.8.8.8\r\na=candidate:3769337065 1 udp 2122260223 8.8.8.8 56688 typ host generation 0 network-id 1 network-cost 50\r\na=candidate:2921887769 1 tcp 1518280447 8.8.8.8 35441 typ host tcptype passive generation 0 network-id 1 network-cost 50\r\na=ice-ufrag:aMAZ\r\na=ice-pwd:jcHb08Jjgrazp2dzjdrvPPvV\r\na=ice-options:trickle\r\na=fingerprint:sha-256 C8:88:EE:B9:E7:02:2E:21:37:ED:7A:D1:EB:2B:A3:15:A2:3B:5B:1C:3D:D4:D5:1F:06:CF:52:40:03:F8:DD:66\r\na=setup:actpass\r\na=mid:data\r\na=sctpmap:5000 webrtc-datachannel 1024\r\n"` - - const sampleOffer = `{"type":"offer","sdp":` + sampleSDP + `}` - const sampleAnswer = `{"type":"answer","sdp":` + sampleSDP + `}` - - Convey("Proxy connections to broker", t, func() { - broker := new(Broker) - broker.url, _ = url.Parse("localhost") - - //Mock peerConnection - config = webrtc.Configuration{ - ICEServers: []webrtc.ICEServer{ - { - URLs: []string{"stun:stun.l.google.com:19302"}, - }, - }, - } - pc, _ := webrtc.NewPeerConnection(config) - offer := util.DeserializeSessionDescription(sampleOffer) - pc.SetRemoteDescription(*offer) - answer, _ := pc.CreateAnswer(nil) - pc.SetLocalDescription(answer) - - Convey("polls broker correctly", func() { - var err error - - b, err := messages.EncodePollResponse(sampleOffer, true) - So(err, ShouldEqual, nil) - broker.transport = &MockTransport{ - http.StatusOK, - b, - } - - sdp := broker.pollOffer(sampleOffer) - expectedSDP, _ := strconv.Unquote(sampleSDP) - So(sdp.SDP, ShouldResemble, expectedSDP) - }) - Convey("handles poll error", func() { - var err error - - b := []byte("test") - So(err, ShouldEqual, nil) - broker.transport = &MockTransport{ - http.StatusOK, - b, - } - - sdp := broker.pollOffer(sampleOffer) - So(sdp, ShouldBeNil) - }) - Convey("sends answer to broker", func() { - var err error - - b, err := messages.EncodeAnswerResponse(true) - So(err, ShouldEqual, nil) - broker.transport = &MockTransport{ - http.StatusOK, - b, - } - - err = broker.sendAnswer(sampleAnswer, pc) - So(err, ShouldEqual, nil) - - b, err = messages.EncodeAnswerResponse(false) - So(err, ShouldEqual, nil) - broker.transport = &MockTransport{ - http.StatusOK, - b, - } - - err = broker.sendAnswer(sampleAnswer, pc) - So(err, ShouldNotBeNil) - }) - Convey("handles answer error", func() { - //Error if faulty transport - broker.transport = &FaultyTransport{} - err := broker.sendAnswer(sampleAnswer, pc) - So(err, ShouldNotBeNil) - - //Error if status code is not ok - broker.transport = &MockTransport{ - http.StatusGone, - []byte(""), - } - err = broker.sendAnswer("test", pc) - So(err, ShouldNotEqual, nil) - So(err.Error(), ShouldResemble, "broker returned 410") - - //Error if we can't parse broker message - broker.transport = &MockTransport{ - http.StatusOK, - []byte("test"), - } - err = broker.sendAnswer("test", pc) - So(err, ShouldNotBeNil) - - //Error if broker message surpasses read limit - broker.transport = &MockTransport{ - http.StatusOK, - make([]byte, 100001), - } - err = broker.sendAnswer("test", pc) - So(err, ShouldNotBeNil) - }) - }) -} - -func TestUtilityFuncs(t *testing.T) { - Convey("LimitedRead", t, func() { - c, s := net.Pipe() - Convey("Successful read", func() { - go func() { - bytes := make([]byte, 50) - c.Write(bytes) - c.Close() - }() - bytes, err := limitedRead(s, 60) - So(len(bytes), ShouldEqual, 50) - So(err, ShouldBeNil) - }) - Convey("Large read", func() { - go func() { - bytes := make([]byte, 50) - c.Write(bytes) - c.Close() - }() - bytes, err := limitedRead(s, 49) - So(len(bytes), ShouldEqual, 49) - So(err, ShouldEqual, io.ErrUnexpectedEOF) - }) - Convey("Failed read", func() { - s.Close() - bytes, err := limitedRead(s, 49) - So(len(bytes), ShouldEqual, 0) - So(err, ShouldEqual, io.ErrClosedPipe) - }) - }) - Convey("Tokens", t, func() { - tokens = make(chan bool, 2) - for i := uint(0); i < 2; i++ { - tokens <- true - } - So(len(tokens), ShouldEqual, 2) - getToken() - So(len(tokens), ShouldEqual, 1) - retToken() - So(len(tokens), ShouldEqual, 2) - }) - Convey("SessionID Generation", t, func() { - sid1 := genSessionID() - sid2 := genSessionID() - So(sid1, ShouldNotEqual, sid2) - }) - Convey("CopyLoop", t, func() { - c1, s1 := net.Pipe() - c2, s2 := net.Pipe() - go CopyLoop(s1, s2) - go func() { - bytes := []byte("Hello!") - c1.Write(bytes) - }() - bytes := make([]byte, 6) - n, err := c2.Read(bytes) - So(n, ShouldEqual, 6) - So(err, ShouldEqual, nil) - So(bytes, ShouldResemble, []byte("Hello!")) - s1.Close() - - //Check that copy loop has closed other connection - _, err = s2.Write(bytes) - So(err, ShouldNotBeNil) - }) -} diff --git a/proxy-go/snowflake.go b/proxy-go/snowflake.go deleted file mode 100644 index 422cf7e..0000000 --- a/proxy-go/snowflake.go +++ /dev/null @@ -1,484 +0,0 @@ -package main - -import ( - "bytes" - "crypto/rand" - "encoding/base64" - "flag" - "fmt" - "io" - "io/ioutil" - "log" - "net" - "net/http" - "net/url" - "os" - "regexp" - "strings" - "sync" - "time" - - "git.torproject.org/pluggable-transports/snowflake.git/common/messages" - "git.torproject.org/pluggable-transports/snowflake.git/common/safelog" - "git.torproject.org/pluggable-transports/snowflake.git/common/util" - "git.torproject.org/pluggable-transports/snowflake.git/common/websocketconn" - "github.com/gorilla/websocket" - "github.com/pion/webrtc/v2" -) - -const defaultBrokerURL = "https://snowflake-broker.bamsoftware.com/" -const defaultRelayURL = "wss://snowflake.bamsoftware.com/" -const defaultSTUNURL = "stun:stun.l.google.com:19302" -const pollInterval = 5 * time.Second - -//amount of time after sending an SDP answer before the proxy assumes the -//client is not going to connect -const dataChannelTimeout = 20 * time.Second - -const readLimit = 100000 //Maximum number of bytes to be read from an HTTP request - -var broker *Broker -var relayURL string - -const ( - sessionIDLength = 16 -) - -var ( - tokens chan bool - config webrtc.Configuration - client http.Client -) - -var remoteIPPatterns = []*regexp.Regexp{ - /* IPv4 */ - regexp.MustCompile(`(?m)^c=IN IP4 ([\d.]+)(?:(?:\/\d+)?\/\d+)?(:? |\r?\n)`), - /* IPv6 */ - regexp.MustCompile(`(?m)^c=IN IP6 ([0-9A-Fa-f:.]+)(?:\/\d+)?(:? |\r?\n)`), -} - -// https://tools.ietf.org/html/rfc4566#section-5.7 -func remoteIPFromSDP(sdp string) net.IP { - for _, pattern := range remoteIPPatterns { - m := pattern.FindStringSubmatch(sdp) - if m != nil { - // Ignore parsing errors, ParseIP returns nil. - return net.ParseIP(m[1]) - } - } - return nil -} - -type Broker struct { - url *url.URL - transport http.RoundTripper - keepLocalAddresses bool -} - -type webRTCConn struct { - dc *webrtc.DataChannel - pc *webrtc.PeerConnection - pr *io.PipeReader - - lock sync.Mutex // Synchronization for DataChannel destruction - once sync.Once // Synchronization for PeerConnection destruction -} - -func (c *webRTCConn) Read(b []byte) (int, error) { - return c.pr.Read(b) -} - -func (c *webRTCConn) Write(b []byte) (int, error) { - c.lock.Lock() - defer c.lock.Unlock() - if c.dc != nil { - c.dc.Send(b) - } - return len(b), nil -} - -func (c *webRTCConn) Close() (err error) { - c.once.Do(func() { - err = c.pc.Close() - }) - return -} - -func (c *webRTCConn) LocalAddr() net.Addr { - return nil -} - -func (c *webRTCConn) RemoteAddr() net.Addr { - //Parse Remote SDP offer and extract client IP - clientIP := remoteIPFromSDP(c.pc.RemoteDescription().SDP) - if clientIP == nil { - return nil - } - return &net.IPAddr{IP: clientIP, Zone: ""} -} - -func (c *webRTCConn) SetDeadline(t time.Time) error { - // nolint: golint - return fmt.Errorf("SetDeadline not implemented") -} - -func (c *webRTCConn) SetReadDeadline(t time.Time) error { - // nolint: golint - return fmt.Errorf("SetReadDeadline not implemented") -} - -func (c *webRTCConn) SetWriteDeadline(t time.Time) error { - // nolint: golint - return fmt.Errorf("SetWriteDeadline not implemented") -} - -func getToken() { - <-tokens -} - -func retToken() { - tokens <- true -} - -func genSessionID() string { - buf := make([]byte, sessionIDLength) - _, err := rand.Read(buf) - if err != nil { - panic(err.Error()) - } - return strings.TrimRight(base64.StdEncoding.EncodeToString(buf), "=") -} - -func limitedRead(r io.Reader, limit int64) ([]byte, error) { - p, err := ioutil.ReadAll(&io.LimitedReader{R: r, N: limit + 1}) - if err != nil { - return p, err - } else if int64(len(p)) == limit+1 { - return p[0:limit], io.ErrUnexpectedEOF - } - return p, err -} - -func (b *Broker) pollOffer(sid string) *webrtc.SessionDescription { - brokerPath := b.url.ResolveReference(&url.URL{Path: "proxy"}) - timeOfNextPoll := time.Now() - for { - // Sleep until we're scheduled to poll again. - now := time.Now() - time.Sleep(timeOfNextPoll.Sub(now)) - // Compute the next time to poll -- if it's in the past, that - // means that the POST took longer than pollInterval, so we're - // allowed to do another one immediately. - timeOfNextPoll = timeOfNextPoll.Add(pollInterval) - if timeOfNextPoll.Before(now) { - timeOfNextPoll = now - } - - body, err := messages.EncodePollRequest(sid, "standalone") - if err != nil { - log.Printf("Error encoding poll message: %s", err.Error()) - return nil - } - req, _ := http.NewRequest("POST", brokerPath.String(), bytes.NewBuffer(body)) - resp, err := b.transport.RoundTrip(req) - if err != nil { - log.Printf("error polling broker: %s", err) - } else { - defer resp.Body.Close() - if resp.StatusCode != http.StatusOK { - log.Printf("broker returns: %d", resp.StatusCode) - } else { - body, err := limitedRead(resp.Body, readLimit) - if err != nil { - log.Printf("error reading broker response: %s", err) - } else { - - offer, err := messages.DecodePollResponse(body) - if err != nil { - log.Printf("error reading broker response: %s", err.Error()) - log.Printf("body: %s", body) - return nil - } - if offer != "" { - return util.DeserializeSessionDescription(offer) - } - } - } - } - } -} - -func (b *Broker) sendAnswer(sid string, pc *webrtc.PeerConnection) error { - brokerPath := b.url.ResolveReference(&url.URL{Path: "answer"}) - ld := pc.LocalDescription() - if !b.keepLocalAddresses { - ld = &webrtc.SessionDescription{ - Type: ld.Type, - SDP: util.StripLocalAddresses(ld.SDP), - } - } - answer := string([]byte(util.SerializeSessionDescription(ld))) - body, err := messages.EncodeAnswerRequest(answer, sid) - if err != nil { - return err - } - req, _ := http.NewRequest("POST", brokerPath.String(), bytes.NewBuffer(body)) - resp, err := b.transport.RoundTrip(req) - if err != nil { - return err - } - if resp.StatusCode != http.StatusOK { - return fmt.Errorf("broker returned %d", resp.StatusCode) - } - - body, err = limitedRead(resp.Body, readLimit) - if err != nil { - return fmt.Errorf("error reading broker response: %s", err) - } - success, err := messages.DecodeAnswerResponse(body) - if err != nil { - return err - } - if !success { - return fmt.Errorf("broker returned client timeout") - } - - return nil -} - -func CopyLoop(c1 io.ReadWriteCloser, c2 io.ReadWriteCloser) { - var wg sync.WaitGroup - copyer := func(dst io.ReadWriteCloser, src io.ReadWriteCloser) { - defer wg.Done() - if _, err := io.Copy(dst, src); err != nil { - log.Printf("io.Copy inside CopyLoop generated an error: %v", err) - } - dst.Close() - src.Close() - } - wg.Add(2) - go copyer(c1, c2) - go copyer(c2, c1) - wg.Wait() -} - -// We pass conn.RemoteAddr() as an additional parameter, rather than calling -// conn.RemoteAddr() inside this function, as a workaround for a hang that -// otherwise occurs inside of conn.pc.RemoteDescription() (called by -// RemoteAddr). https://bugs.torproject.org/18628#comment:8 -func datachannelHandler(conn *webRTCConn, remoteAddr net.Addr) { - defer conn.Close() - defer retToken() - - u, err := url.Parse(relayURL) - if err != nil { - log.Fatalf("invalid relay url: %s", err) - } - - // Retrieve client IP address - if remoteAddr != nil { - // Encode client IP address in relay URL - q := u.Query() - clientIP := remoteAddr.String() - q.Set("client_ip", clientIP) - u.RawQuery = q.Encode() - } else { - log.Printf("no remote address given in websocket") - } - - ws, _, err := websocket.DefaultDialer.Dial(u.String(), nil) - if err != nil { - log.Printf("error dialing relay: %s", err) - return - } - wsConn := websocketconn.New(ws) - log.Printf("connected to relay") - defer wsConn.Close() - CopyLoop(conn, wsConn) - log.Printf("datachannelHandler ends") -} - -// Create a PeerConnection from an SDP offer. Blocks until the gathering of ICE -// candidates is complete and the answer is available in LocalDescription. -// Installs an OnDataChannel callback that creates a webRTCConn and passes it to -// datachannelHandler. -func makePeerConnectionFromOffer(sdp *webrtc.SessionDescription, config webrtc.Configuration, dataChan chan struct{}) (*webrtc.PeerConnection, error) { - pc, err := webrtc.NewPeerConnection(config) - if err != nil { - return nil, fmt.Errorf("accept: NewPeerConnection: %s", err) - } - pc.OnDataChannel(func(dc *webrtc.DataChannel) { - log.Println("OnDataChannel") - close(dataChan) - - pr, pw := io.Pipe() - conn := &webRTCConn{pc: pc, dc: dc, pr: pr} - - dc.OnOpen(func() { - log.Println("OnOpen channel") - }) - dc.OnClose(func() { - conn.lock.Lock() - defer conn.lock.Unlock() - log.Println("OnClose channel") - conn.dc = nil - dc.Close() - pw.Close() - }) - dc.OnMessage(func(msg webrtc.DataChannelMessage) { - var n int - n, err = pw.Write(msg.Data) - if err != nil { - if inerr := pw.CloseWithError(err); inerr != nil { - log.Printf("close with error generated an error: %v", inerr) - } - } - if n != len(msg.Data) { - panic("short write") - } - }) - - go datachannelHandler(conn, conn.RemoteAddr()) - }) - - err = pc.SetRemoteDescription(*sdp) - if err != nil { - if inerr := pc.Close(); inerr != nil { - log.Printf("unable to call pc.Close after pc.SetRemoteDescription with error: %v", inerr) - } - return nil, fmt.Errorf("accept: SetRemoteDescription: %s", err) - } - log.Println("sdp offer successfully received.") - - log.Println("Generating answer...") - answer, err := pc.CreateAnswer(nil) - // blocks on ICE gathering. we need to add a timeout if needed - // not putting this in a separate go routine, because we need - // SetLocalDescription(answer) to be called before sendAnswer - if err != nil { - if inerr := pc.Close(); inerr != nil { - log.Printf("ICE gathering has generated an error when calling pc.Close: %v", inerr) - } - return nil, err - } - - err = pc.SetLocalDescription(answer) - if err != nil { - if err = pc.Close(); err != nil { - log.Printf("pc.Close after setting local description returned : %v", err) - } - return nil, err - } - - return pc, nil -} - -func runSession(sid string) { - offer := broker.pollOffer(sid) - if offer == nil { - log.Printf("bad offer from broker") - retToken() - return - } - dataChan := make(chan struct{}) - pc, err := makePeerConnectionFromOffer(offer, config, dataChan) - if err != nil { - log.Printf("error making WebRTC connection: %s", err) - retToken() - return - } - err = broker.sendAnswer(sid, pc) - if err != nil { - log.Printf("error sending answer to client through broker: %s", err) - if inerr := pc.Close(); inerr != nil { - log.Printf("error calling pc.Close: %v", inerr) - } - retToken() - return - } - // Set a timeout on peerconnection. If the connection state has not - // advanced to PeerConnectionStateConnected in this time, - // destroy the peer connection and return the token. - select { - case <-dataChan: - log.Println("Connection successful.") - case <-time.After(dataChannelTimeout): - log.Println("Timed out waiting for client to open data channel.") - if err := pc.Close(); err != nil { - log.Printf("error calling pc.Close: %v", err) - } - retToken() - } -} - -func main() { - var capacity uint - var stunURL string - var logFilename string - var rawBrokerURL string - var unsafeLogging bool - var keepLocalAddresses bool - - flag.UintVar(&capacity, "capacity", 10, "maximum concurrent clients") - flag.StringVar(&rawBrokerURL, "broker", defaultBrokerURL, "broker URL") - flag.StringVar(&relayURL, "relay", defaultRelayURL, "websocket relay URL") - flag.StringVar(&stunURL, "stun", defaultSTUNURL, "stun URL") - flag.StringVar(&logFilename, "log", "", "log filename") - flag.BoolVar(&unsafeLogging, "unsafe-logging", false, "prevent logs from being scrubbed") - flag.BoolVar(&keepLocalAddresses, "keep-local-addresses", false, "keep local LAN address ICE candidates") - flag.Parse() - - var logOutput io.Writer = os.Stderr - log.SetFlags(log.LstdFlags | log.LUTC) - if logFilename != "" { - f, err := os.OpenFile(logFilename, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0600) - if err != nil { - log.Fatal(err) - } - defer f.Close() - logOutput = io.MultiWriter(os.Stderr, f) - } - if unsafeLogging { - log.SetOutput(logOutput) - } else { - // We want to send the log output through our scrubber first - log.SetOutput(&safelog.LogScrubber{Output: logOutput}) - } - - log.Println("starting") - - var err error - broker = new(Broker) - broker.keepLocalAddresses = keepLocalAddresses - broker.url, err = url.Parse(rawBrokerURL) - if err != nil { - log.Fatalf("invalid broker url: %s", err) - } - _, err = url.Parse(stunURL) - if err != nil { - log.Fatalf("invalid stun url: %s", err) - } - _, err = url.Parse(relayURL) - if err != nil { - log.Fatalf("invalid relay url: %s", err) - } - - broker.transport = http.DefaultTransport.(*http.Transport) - config = webrtc.Configuration{ - ICEServers: []webrtc.ICEServer{ - { - URLs: []string{stunURL}, - }, - }, - } - tokens = make(chan bool, capacity) - for i := uint(0); i < capacity; i++ { - tokens <- true - } - - for { - getToken() - sessionID := genSessionID() - runSession(sessionID) - } -} diff --git a/server/README.md b/server/README.md deleted file mode 100644 index 312a506..0000000 --- a/server/README.md +++ /dev/null @@ -1,61 +0,0 @@ -This is the server transport plugin for Snowflake. -The actual transport protocol it uses is -[WebSocket](https://tools.ietf.org/html/rfc6455). -In Snowflake, the client connects to the proxy using WebRTC, -and the proxy connects to the server (this program) using WebSocket. - - -# Setup - -The server needs to be able to listen on port 80 -in order to generate its TLS certificates. -On Linux, use the `setcap` program to enable -the server to listen on port 80 without running as root: -``` -setcap 'cap_net_bind_service=+ep' /usr/local/bin/snowflake-server -``` - -Here is a short example of configuring your torrc file -to run the Snowflake server under Tor: -``` -SocksPort 0 -ORPort 9001 -ExtORPort auto -BridgeRelay 1 - -ServerTransportListenAddr snowflake 0.0.0.0:443 -ServerTransportPlugin snowflake exec ./server --acme-hostnames snowflake.example --acme-email admin(a)snowflake.example --log /var/log/tor/snowflake-server.log -``` -The domain names given to the `--acme-hostnames` option -should resolve to the IP address of the server. -You can give more than one, separated by commas. - - -# TLS - -The server uses TLS WebSockets by default: wss:// not ws://. -There is a `--disable-tls` option for testing purposes, -but you should use TLS in production. - -The server automatically fetches certificates -from [Let's Encrypt](https://en.wikipedia.org/wiki/Let's_Encrypt) as needed. -Use the `--acme-hostnames` option to tell the server -what hostnames it may request certificates for. -You can optionally provide a contact email address, -using the `--acme-email` option, -so that Let's Encrypt can inform you of any problems. -The server will cache TLS certificate data in the directory -`pt_state/snowflake-certificate-cache` inside the tor state directory. - -In order to fetch certificates automatically, -the server needs to listen on port 80, -in addition to whatever ports it is listening on -for WebSocket connections. -This is a requirement of the ACME protocol used by Let's Encrypt. -The program will exit if it can't bind to port 80. -On Linux, you can use the `setcap` program, -part of libcap2, to enable the server to bind to low-numbered ports -without having to run as root: -``` -setcap 'cap_net_bind_service=+ep' /usr/local/bin/snowflake-server -``` diff --git a/server/server.go b/server/server.go deleted file mode 100644 index c03e41c..0000000 --- a/server/server.go +++ /dev/null @@ -1,361 +0,0 @@ -// Snowflake-specific websocket server plugin. It reports the transport name as -// "snowflake". -package main - -import ( - "crypto/tls" - "flag" - "fmt" - "io" - "io/ioutil" - "log" - "net" - "net/http" - "os" - "os/signal" - "path/filepath" - "strings" - "sync" - "syscall" - "time" - - pt "git.torproject.org/pluggable-transports/goptlib.git" - "git.torproject.org/pluggable-transports/snowflake.git/common/safelog" - "git.torproject.org/pluggable-transports/snowflake.git/common/websocketconn" - "github.com/gorilla/websocket" - "golang.org/x/crypto/acme/autocert" - "golang.org/x/net/http2" -) - -const ptMethodName = "snowflake" -const requestTimeout = 10 * time.Second - -// How long to wait for ListenAndServe or ListenAndServeTLS to return an error -// before deciding that it's not going to return. -const listenAndServeErrorTimeout = 100 * time.Millisecond - -var ptInfo pt.ServerInfo - -func usage() { - fmt.Fprintf(os.Stderr, `Usage: %s [OPTIONS] - -WebSocket server pluggable transport for Snowflake. Works only as a managed -proxy. Uses TLS with ACME (Let's Encrypt) by default. Set the certificate -hostnames with the --acme-hostnames option. Use ServerTransportListenAddr in -torrc to choose the listening port. When using TLS, this program will open an -additional HTTP listener on port 80 to work with ACME. - -`, os.Args[0]) - flag.PrintDefaults() -} - -// Copy from WebSocket to socket and vice versa. -func proxy(local *net.TCPConn, conn *websocketconn.Conn) { - var wg sync.WaitGroup - wg.Add(2) - - go func() { - if _, err := io.Copy(conn, local); err != nil { - log.Printf("error copying ORPort to WebSocket %v", err) - } - if err := local.CloseRead(); err != nil { - log.Printf("error closing read after copying ORPort to WebSocket %v", err) - } - conn.Close() - wg.Done() - }() - go func() { - if _, err := io.Copy(local, conn); err != nil { - log.Printf("error copying WebSocket to ORPort %v", err) - } - if err := local.CloseWrite(); err != nil { - log.Printf("error closing write after copying WebSocket to ORPort %v", err) - } - conn.Close() - wg.Done() - }() - - wg.Wait() -} - -// Return an address string suitable to pass into pt.DialOr. -func clientAddr(clientIPParam string) string { - if clientIPParam == "" { - return "" - } - // Check if client addr is a valid IP - clientIP := net.ParseIP(clientIPParam) - if clientIP == nil { - return "" - } - // Check if client addr is 0.0.0.0 or [::]. Some proxies erroneously - // report an address of 0.0.0.0: https://bugs.torproject.org/33157. - if clientIP.IsUnspecified() { - return "" - } - // Add a dummy port number. USERADDR requires a port number. - return (&net.TCPAddr{IP: clientIP, Port: 1, Zone: ""}).String() -} - -var upgrader = websocket.Upgrader{ - CheckOrigin: func(r *http.Request) bool { return true }, -} - -type HTTPHandler struct{} - -func (handler *HTTPHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { - ws, err := upgrader.Upgrade(w, r, nil) - if err != nil { - log.Println(err) - return - } - - conn := websocketconn.New(ws) - defer conn.Close() - - // Pass the address of client as the remote address of incoming connection - clientIPParam := r.URL.Query().Get("client_ip") - addr := clientAddr(clientIPParam) - statsChannel <- addr != "" - or, err := pt.DialOr(&ptInfo, addr, ptMethodName) - if err != nil { - log.Printf("failed to connect to ORPort: %s", err) - return - } - defer or.Close() - - proxy(or, conn) -} - -func initServer(addr *net.TCPAddr, - getCertificate func(*tls.ClientHelloInfo) (*tls.Certificate, error), - listenAndServe func(*http.Server, chan<- error)) (*http.Server, error) { - // We're not capable of listening on port 0 (i.e., an ephemeral port - // unknown in advance). The reason is that while the net/http package - // exposes ListenAndServe and ListenAndServeTLS, those functions never - // return, so there's no opportunity to find out what the port number - // is, in between the Listen and Serve steps. - // https://groups.google.com/d/msg/Golang-nuts/3F1VRCCENp8/3hcayZiwYM8J - if addr.Port == 0 { - return nil, fmt.Errorf("cannot listen on port %d; configure a port using ServerTransportListenAddr", addr.Port) - } - - var handler HTTPHandler - server := &http.Server{ - Addr: addr.String(), - Handler: &handler, - ReadTimeout: requestTimeout, - } - // We need to override server.TLSConfig.GetCertificate--but first - // server.TLSConfig needs to be non-nil. If we just create our own new - // &tls.Config, it will lack the default settings that the net/http - // package sets up for things like HTTP/2. Therefore we first call - // http2.ConfigureServer for its side effect of initializing - // server.TLSConfig properly. An alternative would be to make a dummy - // net.Listener, call Serve on it, and let it return. - // https://github.com/golang/go/issues/16588#issuecomment-237386446 - err := http2.ConfigureServer(server, nil) - if err != nil { - return server, err - } - server.TLSConfig.GetCertificate = getCertificate - - // Another unfortunate effect of the inseparable net/http ListenAndServe - // is that we can't check for Listen errors like "permission denied" and - // "address already in use" without potentially entering the infinite - // loop of Serve. The hack we apply here is to wait a short time, - // listenAndServeErrorTimeout, to see if an error is returned (because - // it's better if the error message goes to the tor log through - // SMETHOD-ERROR than if it only goes to the snowflake log). - errChan := make(chan error) - go listenAndServe(server, errChan) - select { - case err = <-errChan: - break - case <-time.After(listenAndServeErrorTimeout): - break - } - - return server, err -} - -func startServer(addr *net.TCPAddr) (*http.Server, error) { - return initServer(addr, nil, func(server *http.Server, errChan chan<- error) { - log.Printf("listening with plain HTTP on %s", addr) - err := server.ListenAndServe() - if err != nil { - log.Printf("error in ListenAndServe: %s", err) - } - errChan <- err - }) -} - -func startServerTLS(addr *net.TCPAddr, getCertificate func(*tls.ClientHelloInfo) (*tls.Certificate, error)) (*http.Server, error) { - return initServer(addr, getCertificate, func(server *http.Server, errChan chan<- error) { - log.Printf("listening with HTTPS on %s", addr) - err := server.ListenAndServeTLS("", "") - if err != nil { - log.Printf("error in ListenAndServeTLS: %s", err) - } - errChan <- err - }) -} - -func getCertificateCacheDir() (string, error) { - stateDir, err := pt.MakeStateDir() - if err != nil { - return "", err - } - return filepath.Join(stateDir, "snowflake-certificate-cache"), nil -} - -func main() { - var acmeEmail string - var acmeHostnamesCommas string - var disableTLS bool - var logFilename string - var unsafeLogging bool - - flag.Usage = usage - flag.StringVar(&acmeEmail, "acme-email", "", "optional contact email for Let's Encrypt notifications") - flag.StringVar(&acmeHostnamesCommas, "acme-hostnames", "", "comma-separated hostnames for TLS certificate") - flag.BoolVar(&disableTLS, "disable-tls", false, "don't use HTTPS") - flag.StringVar(&logFilename, "log", "", "log file to write to") - flag.BoolVar(&unsafeLogging, "unsafe-logging", false, "prevent logs from being scrubbed") - flag.Parse() - - log.SetFlags(log.LstdFlags | log.LUTC) - - var logOutput io.Writer = os.Stderr - if logFilename != "" { - f, err := os.OpenFile(logFilename, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0600) - if err != nil { - log.Fatalf("can't open log file: %s", err) - } - defer f.Close() - logOutput = f - } - if unsafeLogging { - log.SetOutput(logOutput) - } else { - // We want to send the log output through our scrubber first - log.SetOutput(&safelog.LogScrubber{Output: logOutput}) - } - - if !disableTLS && acmeHostnamesCommas == "" { - log.Fatal("the --acme-hostnames option is required") - } - acmeHostnames := strings.Split(acmeHostnamesCommas, ",") - - log.Printf("starting") - var err error - ptInfo, err = pt.ServerSetup(nil) - if err != nil { - log.Fatalf("error in setup: %s", err) - } - - go statsThread() - - var certManager *autocert.Manager - if !disableTLS { - log.Printf("ACME hostnames: %q", acmeHostnames) - - var cache autocert.Cache - var cacheDir string - cacheDir, err = getCertificateCacheDir() - if err == nil { - log.Printf("caching ACME certificates in directory %q", cacheDir) - cache = autocert.DirCache(cacheDir) - } else { - log.Printf("disabling ACME certificate cache: %s", err) - } - - certManager = &autocert.Manager{ - Prompt: autocert.AcceptTOS, - HostPolicy: autocert.HostWhitelist(acmeHostnames...), - Email: acmeEmail, - Cache: cache, - } - } - - // The ACME HTTP-01 responder only works when it is running on port 80. - // We actually open the port in the loop below, so that any errors can - // be reported in the SMETHOD-ERROR of some bindaddr. - // https://github.com/ietf-wg-acme/acme/blob/master/draft-ietf-acme-acme.md#ht… - needHTTP01Listener := !disableTLS - - servers := make([]*http.Server, 0) - for _, bindaddr := range ptInfo.Bindaddrs { - if bindaddr.MethodName != ptMethodName { - pt.SmethodError(bindaddr.MethodName, "no such method") - continue - } - - if needHTTP01Listener { - addr := *bindaddr.Addr - addr.Port = 80 - log.Printf("Starting HTTP-01 ACME listener") - var lnHTTP01 *net.TCPListener - lnHTTP01, err = net.ListenTCP("tcp", &addr) - if err != nil { - log.Printf("error opening HTTP-01 ACME listener: %s", err) - pt.SmethodError(bindaddr.MethodName, "HTTP-01 ACME listener: "+err.Error()) - continue - } - server := &http.Server{ - Addr: addr.String(), - Handler: certManager.HTTPHandler(nil), - } - go func() { - log.Fatal(server.Serve(lnHTTP01)) - }() - servers = append(servers, server) - needHTTP01Listener = false - } - - var server *http.Server - args := pt.Args{} - if disableTLS { - args.Add("tls", "no") - server, err = startServer(bindaddr.Addr) - } else { - args.Add("tls", "yes") - for _, hostname := range acmeHostnames { - args.Add("hostname", hostname) - } - server, err = startServerTLS(bindaddr.Addr, certManager.GetCertificate) - } - if err != nil { - log.Printf("error opening listener: %s", err) - pt.SmethodError(bindaddr.MethodName, err.Error()) - continue - } - pt.SmethodArgs(bindaddr.MethodName, bindaddr.Addr, args) - servers = append(servers, server) - } - pt.SmethodsDone() - - sigChan := make(chan os.Signal, 1) - signal.Notify(sigChan, syscall.SIGTERM) - - if os.Getenv("TOR_PT_EXIT_ON_STDIN_CLOSE") == "1" { - // This environment variable means we should treat EOF on stdin - // just like SIGTERM: https://bugs.torproject.org/15435. - go func() { - if _, err := io.Copy(ioutil.Discard, os.Stdin); err != nil { - log.Printf("error copying os.Stdin to ioutil.Discard: %v", err) - } - log.Printf("synthesizing SIGTERM because of stdin close") - sigChan <- syscall.SIGTERM - }() - } - - // Wait for a signal. - sig := <-sigChan - - // Signal received, shut down. - log.Printf("caught signal %q, exiting", sig) - for _, server := range servers { - server.Close() - } -} diff --git a/server/server_test.go b/server/server_test.go deleted file mode 100644 index ba00d16..0000000 --- a/server/server_test.go +++ /dev/null @@ -1,153 +0,0 @@ -package main - -import ( - "net" - "net/http" - "strconv" - "testing" - - "git.torproject.org/pluggable-transports/snowflake.git/common/websocketconn" - "github.com/gorilla/websocket" - . "github.com/smartystreets/goconvey/convey" -) - -func TestClientAddr(t *testing.T) { - Convey("Testing clientAddr", t, func() { - // good tests - for _, test := range []struct { - input string - expected net.IP - }{ - {"1.2.3.4", net.ParseIP("1.2.3.4")}, - {"1:2::3:4", net.ParseIP("1:2::3:4")}, - } { - useraddr := clientAddr(test.input) - host, port, err := net.SplitHostPort(useraddr) - if err != nil { - t.Errorf("clientAddr(%q) → SplitHostPort error %v", test.input, err) - continue - } - if !test.expected.Equal(net.ParseIP(host)) { - t.Errorf("clientAddr(%q) → host %q, not %v", test.input, host, test.expected) - } - portNo, err := strconv.Atoi(port) - if err != nil { - t.Errorf("clientAddr(%q) → port %q", test.input, port) - continue - } - if portNo == 0 { - t.Errorf("clientAddr(%q) → port %d", test.input, portNo) - } - } - - // bad tests - for _, input := range []string{ - "", - "abc", - "1.2.3.4.5", - "[12::34]", - "0.0.0.0", - "[::]", - } { - useraddr := clientAddr(input) - if useraddr != "" { - t.Errorf("clientAddr(%q) → %q, not %q", input, useraddr, "") - } - } - }) -} - -type StubHandler struct{} - -func (handler *StubHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { - ws, _ := upgrader.Upgrade(w, r, nil) - - conn := websocketconn.New(ws) - defer conn.Close() - - //dial stub OR - or, _ := net.DialTCP("tcp", nil, &net.TCPAddr{IP: net.ParseIP("localhost"), Port: 8889}) - - proxy(or, conn) -} - -func Test(t *testing.T) { - Convey("Websocket server", t, func() { - //Set up the snowflake web server - ipStr, portStr, _ := net.SplitHostPort(":8888") - port, _ := strconv.ParseUint(portStr, 10, 16) - addr := &net.TCPAddr{IP: net.ParseIP(ipStr), Port: int(port)} - Convey("We don't listen on port 0", func() { - addr = &net.TCPAddr{IP: net.ParseIP(ipStr), Port: 0} - server, err := initServer(addr, nil, - func(server *http.Server, errChan chan<- error) { - return - }) - So(err, ShouldNotBeNil) - So(server, ShouldBeNil) - }) - - Convey("Plain HTTP server accepts connections", func(c C) { - server, err := startServer(addr) - So(err, ShouldBeNil) - - ws, _, err := websocket.DefaultDialer.Dial("ws://localhost:8888", nil) - wsConn := websocketconn.New(ws) - So(err, ShouldEqual, nil) - So(wsConn, ShouldNotEqual, nil) - - server.Close() - wsConn.Close() - - }) - Convey("Handler proxies data", func(c C) { - - laddr := &net.TCPAddr{IP: net.ParseIP("localhost"), Port: 8889} - - go func() { - - //stub OR - listener, err := net.ListenTCP("tcp", laddr) - c.So(err, ShouldBeNil) - conn, err := listener.Accept() - c.So(err, ShouldBeNil) - - b := make([]byte, 5) - n, err := conn.Read(b) - c.So(err, ShouldBeNil) - c.So(n, ShouldEqual, 5) - c.So(b, ShouldResemble, []byte("Hello")) - - n, err = conn.Write([]byte("world!")) - c.So(n, ShouldEqual, 6) - c.So(err, ShouldBeNil) - }() - - //overwite handler - server, err := initServer(addr, nil, - func(server *http.Server, errChan chan<- error) { - server.ListenAndServe() - }) - So(err, ShouldBeNil) - - var handler StubHandler - server.Handler = &handler - - ws, _, err := websocket.DefaultDialer.Dial("ws://localhost:8888", nil) - So(err, ShouldEqual, nil) - wsConn := websocketconn.New(ws) - So(wsConn, ShouldNotEqual, nil) - - wsConn.Write([]byte("Hello")) - b := make([]byte, 6) - n, err := wsConn.Read(b) - So(n, ShouldEqual, 6) - So(b, ShouldResemble, []byte("world!")) - - wsConn.Close() - server.Close() - - }) - - }) -} diff --git a/server/stats.go b/server/stats.go deleted file mode 100644 index 47aefc6..0000000 --- a/server/stats.go +++ /dev/null @@ -1,44 +0,0 @@ -package main - -// This code handled periodic statistics logging. -// -// The only thing it keeps track of is how many connections had the client_ip -// parameter. Write true to statsChannel to record a connection with client_ip; -// write false for without. - -import ( - "log" - "time" -) - -const ( - statsInterval = 24 * time.Hour -) - -var ( - statsChannel = make(chan bool) -) - -func statsThread() { - var numClientIP, numConnections uint64 - prevTime := time.Now() - deadline := time.After(statsInterval) - for { - select { - case v := <-statsChannel: - if v { - numClientIP++ - } - numConnections++ - case <-deadline: - now := time.Now() - log.Printf("in the past %.f s, %d/%d connections had client_ip", - (now.Sub(prevTime)).Seconds(), - numClientIP, numConnections) - numClientIP = 0 - numConnections = 0 - prevTime = now - deadline = time.After(statsInterval) - } - } -} diff --git a/server/torrc b/server/torrc deleted file mode 100644 index 5dc2008..0000000 --- a/server/torrc +++ /dev/null @@ -1,7 +0,0 @@ -SocksPort 0 -ORPort 9001 -ExtORPort auto -BridgeRelay 1 - -ServerTransportListenAddr snowflake 0.0.0.0:443 -ServerTransportPlugin snowflake exec ./server --acme-hostnames snowflake.example --acme-email admin(a)snowflake.example --log /var/log/tor/snowflake-server.log

1 0

[snowflake/master] Remove proxy/translation submodule
by cohosh＠torproject.org 22 Apr '20

22 Apr '20

commit 6f89fc14f6aa8692b44dc71bd7b04cbe2cf20aa8 Author: Cecylia Bocovich <cohosh(a)torproject.org> Date: Thu Mar 19 12:17:04 2020 -0400 Remove proxy/translation submodule We're moving all web proxy code to another repository. --- .gitmodules | 4 ---- proxy/translation | 1 - 2 files changed, 5 deletions(-) diff --git a/.gitmodules b/.gitmodules index 5f79304..e69de29 100644 --- a/.gitmodules +++ b/.gitmodules @@ -1,4 +0,0 @@ -[submodule "proxy/translation"] - path = proxy/translation - url = https://git.torproject.org/translation.git - branch = snowflakeaddon-messages.json_completed diff --git a/proxy/translation b/proxy/translation deleted file mode 160000 index 9ce163a..0000000 --- a/proxy/translation +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 9ce163a282fcd35e3cc6b642b0236e237dc56159

1 0

[snowflake/master] Remove web proxy instructions from README.md
by cohosh＠torproject.org 22 Apr '20

22 Apr '20

commit da01bf232385cd1ae57e818d955990ced478b5df Author: Cecylia Bocovich <cohosh(a)torproject.org> Date: Thu Mar 19 12:21:04 2020 -0400 Remove web proxy instructions from README.md --- README.md | 29 ----------------------------- 1 file changed, 29 deletions(-) diff --git a/README.md b/README.md index 05fb5f7..7145248 100644 --- a/README.md +++ b/README.md @@ -35,9 +35,6 @@ Client: - [pion/webrtc](https://github.com/pion/webrtc) - Go 1.10+ -Proxy: -- JavaScript - --- #### More Info @@ -66,32 +63,6 @@ ClientTransportPlugin snowflake exec ./client --meek ``` -#### Building - -This describes how to build the in-browser snowflake. For the client, see Usage, -above. - -The client will only work if there are browser snowflakes available. -To run your own: - -``` -cd proxy/ -npm run build -``` - -Then, start a local http server in the `proxy/build/` in any way you like. -For instance: - -``` -cd build/ -python -m http.server -``` - -Then, open a browser tab to `http://127.0.0.1:8000/embed.html` to view -the debug-console of the snowflake., -So long as that tab is open, you are an ephemeral Tor bridge. - - #### Test Environment There is a Docker-based test environment at https://github.com/cohosh/snowbox.

1 0

[snowflake/master] Rename proxy-go/ directory to proxy/
by cohosh＠torproject.org 22 Apr '20

22 Apr '20

commit 20180dcb041929c1ddd37106e6e74ff8c847f2ee Author: Cecylia Bocovich <cohosh(a)torproject.org> Date: Thu Apr 16 10:02:11 2020 -0400 Rename proxy-go/ directory to proxy/ Now that the web proxies are in a different repository, no need to distinguish the two. --- README.md | 4 ++-- {proxy-go => proxy}/README.md | 2 +- {proxy-go => proxy}/proxy-go_test.go | 0 {proxy-go => proxy}/snowflake.go | 0 4 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 7145248..d9be45b 100644 --- a/README.md +++ b/README.md @@ -108,9 +108,9 @@ abundance of ephemeral and short-lived (and special!) volunteer proxies... ##### -- Testing with Standalone Proxy -- ``` -cd proxy-go +cd proxy go build -./proxy-go +./proxy ``` More documentation on the way. diff --git a/proxy-go/README.md b/proxy/README.md similarity index 80% rename from proxy-go/README.md rename to proxy/README.md index 264fc4f..381e3e5 100644 --- a/proxy-go/README.md +++ b/proxy/README.md @@ -1,3 +1,3 @@ This is a standalone (not browser-based) version of the Snowflake proxy. -Usage: ./proxy-go +Usage: ./proxy diff --git a/proxy-go/proxy-go_test.go b/proxy/proxy-go_test.go similarity index 100% rename from proxy-go/proxy-go_test.go rename to proxy/proxy-go_test.go diff --git a/proxy-go/snowflake.go b/proxy/snowflake.go similarity index 100% rename from proxy-go/snowflake.go rename to proxy/snowflake.go

1 0

Jump to page: