- tor-commits - lists.torproject.org

[newsletter/staging] Fix some formatting
by hiro＠torproject.org 27 Mar '19

27 Mar '19

commit 1c48c0bc62bf7bceabd75a51357fd637c5c4d57a Author: hiro <hiro(a)torproject.org> Date: Wed Feb 27 15:08:50 2019 +0100 Fix some formatting --- .../text/contents.lr | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/content/archive/tor-stories-sharing-files-over-tor-new-releases-events/text/contents.lr b/content/archive/tor-stories-sharing-files-over-tor-new-releases-events/text/contents.lr index 5a9f490..b1dcd0c 100644 --- a/content/archive/tor-stories-sharing-files-over-tor-new-releases-events/text/contents.lr +++ b/content/archive/tor-stories-sharing-files-over-tor-new-releases-events/text/contents.lr @@ -47,33 +47,44 @@ Learn more about OnionShare and its new features: https://blog.torproject.org/ne ## New Releases ## Tor Browser 8.0.6 + The main change in this new release is the update of Firefox to 60.5.1esr. Full changelog: https://blog.torproject.org/new-release-tor-browser-806 Tor Browser 8.5a8 + The main change in this new release is the update of Firefox to 60.5.1esr, fixing some vulnerabilities in the Skia library. Full changelog: https://blog.torproject.org/new-release-tor-browser-85a8 Tor 0.4.0.2-alpha, 0.3.5.8, 0.3.4.11, and 0.3.3.12 + These new source code releases all fix TROVE-2019-001. Full changelog: https://blog.torproject.org/new-releases-tor-0402-alpha-0358-03411-and-03312 ## Upcoming Events with Tor ## Nullcon. Goa, India. 1-3 March, 2019. https://blog.torproject.org/events/nullcon-goa + LibrePlanet. Boston, USA. 23-24 March, 2019. https://blog.torproject.org/events/libreplanet-boston + KNOW Conference. Las Vegas, USA. 24-27 March, 2019. https://blog.torproject.org/events/know-2019-vegas-0 + Internet Freedom Festival. Valencia, Spain. 1-5 April, 2019. https://blog.torproject.org/events/internet-freedom-festival-valencia + ## Join Our Community ## Getting involved with Tor is easy. Run a relay to make the network faster and more decentralized: https://trac.torproject.org/projects/tor/wiki/TorRelayGuide Learn about each of our teams and start collaborating: https://trac.torproject.org/projects/tor/wiki/WikiStart#Teams -Donate to help keep Tor fast, strong, and secure. https://donate.torproject.org +Donate to help keep Tor fast, strong, and secure. + +https://donate.torproject.org -- The Tor Project is a US 501(c)(3) non-profit organization advancing human rights and freedoms by creating and deploying free and open-source anonymity and privacy technologies, supporting their unrestricted availability and use, and furthering their scientific and popular understanding. Twitter: https://twitter.com/torproject + Facebook: https://facebook.com/torproject + Instagram: https://instagram.com/torproject

1 0

[newsletter/staging] Add htaccess
by hiro＠torproject.org 27 Mar '19

27 Mar '19

commit 055107aae9390d1edb3cf09faa0e02f938ada90e Author: hiro <hiro(a)torproject.org> Date: Tue Mar 26 18:27:31 2019 +0100 Add htaccess --- .htaccess | 0 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/.htaccess b/.htaccess new file mode 100644 index 0000000..e69de29

1 0

[newsletter/staging] Make .htaccess work
by hiro＠torproject.org 27 Mar '19

27 Mar '19

commit af15c3e82657ccb5ca021b172dfe18865f87cd28 Author: hiro <hiro(a)torproject.org> Date: Wed Mar 27 10:55:11 2019 +0100 Make .htaccess work --- .htaccess | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.htaccess b/.htaccess index e69de29..d99bfa2 100644 --- a/.htaccess +++ b/.htaccess @@ -0,0 +1,2 @@ + +

1 0

[newsletter/staging] Add newsletter for January
by hiro＠torproject.org 27 Mar '19

27 Mar '19

commit c6c795da7b4d66facbc5be75eb6bc2d61187730c Author: hiro <hiro(a)torproject.org> Date: Thu Jan 31 15:52:15 2019 +0100 Add newsletter for January --- content/archive/expect-more-from-tor/contents.lr | 187 +++++++++++++++++++++ .../archive/expect-more-from-tor/text/contents.lr | 97 +++++++++++ newsletter.lektorproject | 4 +- 3 files changed, 285 insertions(+), 3 deletions(-) diff --git a/content/archive/expect-more-from-tor/contents.lr b/content/archive/expect-more-from-tor/contents.lr new file mode 100644 index 0000000..2d81ad3 --- /dev/null +++ b/content/archive/expect-more-from-tor/contents.lr @@ -0,0 +1,187 @@ +_model: post +--- +_template: newsletter.html +--- +author: steph(a)torproject.org +--- +pub_date: 2019-01-31 +--- +title: Expect More from Tor in 2019 +--- +html_body: + +<table align="center" border="0" cellpadding="0" cellspacing="0" class="devicewidth" width="650"> + <tbody> + <tr> + <td width="100%"> + <table align="center" bgcolor="#ffffff" border="0" cellpadding="0" cellspacing="0" class="devicewidth" width="650"> + <tbody> + <tr> + <td height="20" width="100%"><a href="https://newsletter.torproject.org"><img alt="tor-news-logo" src="https://blog.torproject.org/sites/default/files/inline-images/tor-news-logo…" style="width: 250px; height: 75px;" /></a></td> + </tr> + <tr> + <td> + <table align="center" border="0" cellpadding="0" cellspacing="0" class="devicewidth" width="650"> + <tbody> + <tr> + <td width="100%"> + <table align="center" bgcolor="#ffffff" border="0" cellpadding="0" cellspacing="0" class="devicewidth" width="650"> + <tbody> + <tr> + <td style="padding:0 15px 15px 15px;"> + <hr /> + <h1>Expect More From Tor in 2019</h1> + + <p><a href="https://blog.torproject.org/expect-more-tor-2019"><img alt="tor onion " src="https://blog.torproject.org/sites/default/files/styles/full_width/public/im…" style="width: 650px; height: 325px;" /></a></p> + + <p>The Tor Project achieved a lot in the last year. We spent 2018 fighting for the fundamental human rights to privacy and freedom online and made our software more accessible than ever before.</p> + + <p>This is an important moment in time. More people are looking for solutions to effectively protect their privacy. <a href="https://blog.torproject.org/strength-numbers-usable-tools-dont-need-be-inva…">93% of the people we met</a> doing 1:1 usability studies said they knew they needed some protection online. More people need robust censorship circumvention tools as <a href="https://blog.torproject.org/strength-numbers-internet-freedom-line">internet freedom declines</a> around the world. More people understand the risks that come from <a href="https://www.theguardian.com/technology/2019/jan/20/shoshana-zuboff-age-of-s…">surveillance as the business model</a> of the internet. And we have been working hard to make Tor more accessible than ever before with the goal that anyone online can enjoy the protections our software provides.</p> + + <p>In 2018, we:</p> + + <ul> + <li> + <p>Gave Tor Browser a UX overhaul with the launch of <a href="https://blog.torproject.org/new-release-tor-browser-80">Tor Browser 8.0</a>, making it easier and friendlier to use than ever.</p> + </li> + <li> + <p>Made it easier for people in countries that censor the internet and censor Tor to circumvent censorship with the ability to fetch bridges from inside Tor Browser.</p> + </li> + <li> + <p><a href="https://blog.torproject.org/strength-numbers-internet-freedom-movement-must…">Localized</a> Tor Browser into 9 previously unsupported languages, bringing the number of available languages to 25.</p> + </li> + <li> + <p>Launched the alpha version of Tor Browser for Android.</p> + </li> + <li> + <p>Improved our Core Tor code for mobile devices, optimizing its performances and making it easier for third party mobile apps to embed Tor.</p> + </li> + <li> + <p>Traveled to meet our users <a href="https://blog.torproject.org/strength-numbers-usable-tools-dont-need-be-inva…">face-to-face</a> and get feedback without using popular and invasive data-collection practices.</p> + </li> + <li> + <p>Improved the security of v3 onion services with the <a href="https://blog.torproject.org/announcing-vanguards-add-onion-services">vanguards add-on</a>.</p> + </li> + <li> + <p>Published <a href="https://ooni.torproject.org/post/ooni-in-2018/">10 research reports through OONI</a> on censorship and network disruptions happening around the world.</p> + </li> + <li> + <p>Pulled in a <a href="https://blog.torproject.org/strength-numbers-final-count">record number of donations</a> as we <a href="https://techcrunch.com/2019/01/11/tor-lessens-reliance-us-grants/">reduced our reliance</a> on government funding. We received donations from 115 countries around the world.</p> + </li> + <li> + <p>Said <a href="https://blog.torproject.org/strength-numbers-onion-blooms">goodbye to Shari Steele</a>, who helped usher the Tor Project into a new stage of organizational maturity, and welcomed our new Executive Director, <a href="https://blog.torproject.org/strength-numbers-community-key">Isabela Bagueros</a>.</p> + </li> + </ul> + + <p>These developments, plus the reality of threats everyone faces online, make <a href="https://www.wired.com/story/tor-anonymity-easier-than-ever/">2019 the year to try Tor</a>.</p> + + <p><a href="https://blog.torproject.org/expect-more-tor-2019">Find out </a>what you can expect from us this year.</p> + + <hr /> + <h1><span class="quickedit-field" data-quickedit-field-id="node/1684/title/en/full" property="schema:name">Tor Browser at TPL: Defending Intellectual Freedom, and Winning Awards Doing So</span><span class="quickedit-field" data-quickedit-field-id="node/1578/title/en/full" property="schema:name"> </span></h1> + + <p><span class="quickedit-field" data-quickedit-field-id="node/1684/title/en/full" property="schema:name"><a href="https://blog.torproject.org/tor-browser-tpl-defending-intellectual-freedom-…" target="_blank"><img alt="toronto-library" src="https://blog.torproject.org/sites/default/files/styles/full_width/public/im…" style="width: 650px; height: 378px;" /></a></span></p> + + <p><em>Guest post by Jonathon Hodge, Digital Literacy Service Lead, Toronto Public Library</em></p> + + <p>Every public library worker will know that person: the one who is worried about being spied on.</p> + + <p>For a long time in public libraries, that person was treated with the kindness and respect we treat every person, regardless of whether we felt that their concerns may have been overblown. The difference between that bygone past and today, is that today, that person is right! The internet is <a href="https://www.newyorker.com/news/amy-davidson/the-n-s-a-verizon-scandal">spying on them</a>; it’s <a href="https://www.theatlantic.com/technology/archive/2014/08/google-knows-you-bet…">spying on all of us</a>. Even if we <a href="https://www.recode.net/2018/4/20/17254312/facebook-shadow-profiles-data-col…">don’t use it very much</a>. Public libraries have long offered effective guidance to the wealth of information society produces. So the question today is, ‘Are we doing enough for ‘that person’, and by extension, for all of us?’</p> + + <p>In Toronto, we felt that the answer was NO. Our communities let us know that they do not know enough about the actual threats they contend with on the internet, they do not know what tools to use or actions to take to protect themselves.</p> + + <p>Public librarians can do a great deal to arm our users with the knowledge, the tools, and the confidence to navigate the surveillance society online. We in Toronto felt that Tor Browser should be the centerpiece of a multi-vector <a href="https://torontopubliclibrary.typepad.com/news_releases/2016/11/toronto-publ…">Digital Privacy Initiative</a>, that combines privacy education, and technology training and providing privacy-enabling tools at the point of service.</p> + + <p><a href="https://blog.torproject.org/tor-browser-tpl-defending-intellectual-freedom-…">Find out more about TPL's initiative</a>.</p> + + <hr /> + <h1>New Releases</h1> + + <h2>Tor Browser 8.0.5</h2> + + <p>This new release updates Firefox to 60.5.0esr and Tor to the first stable release in the 0.3.5 series, 0.3.5.7. <a href="https://blog.torproject.org/new-release-tor-browser-805">Full changelog</a>.</p> + + <h2>Tor 0.4.01-alpha</h2> + + <p>Tor 0.4.0.1-alpha is the first release in the new 0.4.0.x series. It introduces improved features for power and bandwidth conservation, more accurate reporting of bootstrap progress for user interfaces, and an experimental backend for an exciting new adaptive padding feature. <a href="https://blog.torproject.org/new-release-tor-0401-alpha">Full changelog</a>.</p> + + <h2 class="title"><span class="quickedit-field" data-quickedit-field-id="node/1669/title/en/teaser" property="schema:name">Tor 0.3.5.7</span></h2> + + <p>The Tor 0.3.5 series includes several new features and performance improvements, including client authorization for v3 onion services, cleanups to bootstrap reporting, support for improved bandwidth measurement tools, experimental support for NSS in place of OpenSSL, and much more. <a href="https://blog.torproject.org/new-releases-tor-0357-03410-and-03311">Full changelog</a>.</p> + + <hr /> + <h1>Upcoming Events with Tor</h1> + + <p><a href="https://blog.torproject.org/events/fosdem-brussels">FOSDEM</a>. Brussels, Belgium. 2-3 February, 2019.</p> + + <p><a href="https://blog.torproject.org/events/tor-meetup-berlin">Tor Meetup</a>. Berlin, Germany. 2 February, 2019.</p> + + <p><a href="https://blog.torproject.org/events/it-defense-stuttgart">IT Defense</a>. Stuttgart, Germany. 6-8 February, 2019.</p> + + <p><a href="https://blog.torproject.org/events/libreplanet-boston">LibrePlanet</a>. Boston, USA. 23-24 March, 2019.</p> + + <p><a href="https://blog.torproject.org/events/know-2019-vegas-0">KNOW Conference</a>. Las Vegas, USA. 24-27 March, 2019.</p> + + <hr /> + <h1>Join Our Community</h1> + + <p>Getting involved with Tor is easy. <a href="https://trac.torproject.org/projects/tor/wiki/TorRelayGuide">Run a relay</a> to make the network faster and more decentralized.</p> + + <p>Learn about each of our <a href="https://trac.torproject.org/projects/tor/wiki/WikiStart#Teams">teams </a>and start collaborating.</p> + + <p><a href="https://torproject.org/donate/donate-sin-tn3">Donate</a> to help keep Tor fast, strong, and secure.</p> + + <div style="background-color: #68b030; padding: 6px 8px 6px 8px; +-webkit-border-radius:3px; border-radius:3px; margin: 0 auto; width:200px; text-align: center;"><a href="https://donate.torproject.org" style="font-size: 24px; font-family: Source sans pro, Helvetica, Arial, sans-serif; font-weight: bold; color: #ffffff; text-decoration: none; display:inline-block;" target="_blank">DONATE</a></div> + </td> + </tr> + <tr> + <td style="padding:0 15px;"> + <p>The Tor Project is a US 501(c)(3) non-profit organization advancing human rights and freedoms by creating and deploying free and open-source anonymity and privacy technologies, supporting their unrestricted availability and use, and furthering their scientific and popular understanding.</p> + + <hr /> + <p><a href="https://facebook.com/torproject"><img alt="" src="https://blog.torproject.org/sites/default/files/inline-images/tor-facebook.…" style="width: 25px; height: 25px; margin: 3px;" /></a><a href="https://twitter.com/torproject"> <img alt="tor-twitter" src="https://blog.torproject.org/sites/default/files/inline-images/tor-twitter.p…" style="width: 25px; height: 25px; margin: 3px;" /></a><a href="https://instagram.com/torproject"> <img alt="tor-insta" src="https://blog.torproject.org/sites/default/files/inline-images/instagram-tor…" style="width: 25px; height: 25px; margin: 3px;" /></a></p> + + <br /> + <a href="https://torproject.org">torproject.org</a></small></p> + </td> + </tr> + </tbody> + </table> + </td> + </tr> + </tbody> + </table> + </td> + </tr> + </tbody> + </table> + </td> + </tr> + </tbody> +</table> + +<table bgcolor="#ffffff" border="0" cellpadding="0" cellspacing="0" id="backgroundTable" st-sortable="left-image" width="100%"> + <tbody> + <tr> + <td> + <table align="center" border="0" cellpadding="0" cellspacing="0" class="devicewidth" width="650"> + <tbody> + <tr> + <td width="100%"> + <table align="center" bgcolor="#ffffff" border="0" cellpadding="0" cellspacing="0" class="devicewidth" width="650"> + <tbody> + <tr> + <td> </td> + </tr> + </tbody> + </table> + </td> + </tr> + </tbody> + </table> + </td> + </tr> + </tbody> +</table> + diff --git a/content/archive/expect-more-from-tor/text/contents.lr b/content/archive/expect-more-from-tor/text/contents.lr new file mode 100644 index 0000000..0f2938a --- /dev/null +++ b/content/archive/expect-more-from-tor/text/contents.lr @@ -0,0 +1,97 @@ +_model: post +--- +_template: post.html +--- +author: steph(a)torproject.org +--- +pub_date: 2019-01-31 +--- +title: Expect More from Tor in 2019 +--- +body: + +## Expect More from Tor in 2019 ## + +The Tor Project achieved a lot in the last year. We spent 2018 fighting for the fundamental human rights to privacy and freedom online and made our software more accessible than ever before. + +In the last year, a significant shift took place in the public understanding of how big tech handles our sensitive, personal information, and how these companies build tools that further censorship in repressive places. For instance, Google’s leaked plans to launch a censored search engine in China were met with global protest. The world is watching. + +At the Tor Project, this is an important moment in time. More people are looking for solutions to effectively protect their privacy. 93% of the people we met doing 1:1 usability studies said they knew they needed some protection online. More people need robust censorship circumvention tools as internet freedom declines around the world. More people understand the risks that come from surveillance as the business model of the internet. And we have been working hard to make Tor more accessible than ever before with the goal that anyone online can enjoy the protections our software provides. + +In 2018, we: + +- Gave Tor Browser a UX overhaul with the launch of Tor Browser 8.0, making it easier and friendlier to use than ever. + +- Made it easier for people in countries that censor the internet and censor Tor to circumvent censorship with the ability to fetch bridges from inside Tor Browser. + +- Localized Tor Browser into 9 previously unsupported languages, bringing the number of available languages to 25. + +- Launched the alpha version of Tor Browser for Android. + +- Improved our Core Tor code for mobile devices, optimizing its performances and making it easier for third party mobile apps to embed Tor. + +- Traveled to meet our users face-to-face and get feedback without using popular and invasive data-collection practices. + +- Improved the security of v3 onion services with the vanguards add-on. + +- Published 10 research reports through OONI on censorship and network disruptions happening around the world. + +- Pulled in a record number of donations as we reduced our reliance on government funding. We received donations from 115 countries around the world. + +- Said goodbye to Shari Steele, who helped usher the Tor Project into a new stage of organizational maturity, and welcomed our new Executive Director, Isabela Bagueros. + +These developments, plus the reality of threats everyone faces online, make 2019 the year to try Tor. + +Find out what you can expect from us this year: https://blog.torproject.org/expect-more-tor-2019 + +## Tor Browser at TPL: Defending Intellectual Freedom, and Winning Awards Doing So // + +Guest post by Jonathon Hodge, Digital Literacy Service Lead, Toronto Public Library + +Every public library worker will know that person: the one who is worried about being spied on. + +For a long time in public libraries, that person was treated with the kindness and respect we treat every person, regardless of whether we felt that their concerns may have been overblown. The difference between that bygone past and today, is that today, that person is right! The internet is spying on them; it’s spying on all of us. Even if we don’t use it very much. Public libraries have long offered effective guidance to the wealth of information society produces. So the question today is, ‘Are we doing enough for ‘that person’, and by extension, for all of us?’ + +In Toronto, we felt that the answer was NO. Our communities let us know that they do not know enough about the actual threats they contend with on the internet, they do not know what tools to use or actions to take to protect themselves. + +Public librarians can do a great deal to arm our users with the knowledge, the tools, and the confidence to navigate the surveillance society online. We in Toronto felt that Tor Browser should be the centerpiece of a multi-vector Digital Privacy Initiative, that combines privacy education, and technology training and providing privacy-enabling tools at the point of service. + +With the software being easy to install and maintain, and with a new public appetite for secure technology, I would encourage other public libraries to install Tor Browser. Our professional defense of intellectual freedom can no longer exist only at the realm of policy. In this age when our tech spies on us for the sake of massive internet companies and the State, our defense must be a technological one as well. That tech is Tor. + +Find out more about TPL's initiative: https://blog.torproject.org/tor-browser-tpl-defending-intellectual-freedom-… + +## New Releases ## + +Tor Browser 8.0.5 +This new release updates Firefox to 60.5.0esr and Tor to the first stable release in the 0.3.5 series, 0.3.5.7. Full changelog: https://blog.torproject.org/new-release-tor-browser-805 + +Tor 0.4.01-alpha +This is the first release in the new 0.4.0.x series. It introduces improved features for power and bandwidth conservation, more accurate reporting of bootstrap progress for user interfaces, and an experimental backend for an exciting new adaptive padding feature. Full changelog: https://blog.torproject.org/new-release-tor-0401-alpha + +Tor 0.3.5.7 +The Tor 0.3.5 series includes several new features and performance improvements, including client authorization for v3 onion services, cleanups to bootstrap reporting, support for improved bandwidth- measurement tools, experimental support for NSS in place of OpenSSL, and much more. Full changelog: https://blog.torproject.org/new-releases-tor-0357-03410-and-03311 + +## Upcoming Events with Tor ## + +FOSDEM. Brussels, Belgium. 2-3 February, 2019. https://blog.torproject.org/events/fosdem-brussels +Tor Meetup. Berlin, Germany. 2 February, 2019. https://blog.torproject.org/events/tor-meetup-berlin +IT Defense. Stuttgart, Germany. 6-8 February, 2019. https://blog.torproject.org/events/it-defense-stuttgart +LibrePlanet. Boston, USA. 23-24 March, 2019. https://blog.torproject.org/events/libreplanet-boston +KNOW Conference. Las Vegas, USA. 24-27 March, 2019. https://blog.torproject.org/events/know-2019-vegas-0 + + +## Join Our Community ## + +Getting involved with Tor is easy. Run a relay to make the network faster and more decentralized: https://trac.torproject.org/projects/tor/wiki/TorRelayGuide + +Learn about each of our teams and start collaborating: https://trac.torproject.org/projects/tor/wiki/WikiStart#Teams + +Donate to help keep Tor fast, strong, and secure. https://donate.torproject.org + +-- + +The Tor Project is a US 501(c)(3) non-profit organization advancing human rights and freedoms by creating and deploying free and open-source anonymity and privacy technologies, supporting their unrestricted availability and use, and furthering their scientific and popular understanding. + +Twitter: https://twitter.com/torproject +Facebook: https://facebook.com/torproject +Instagram: https://instagram.com/torproject diff --git a/newsletter.lektorproject b/newsletter.lektorproject index 3ecfebd..7e924e8 100644 --- a/newsletter.lektorproject +++ b/newsletter.lektorproject @@ -1,10 +1,8 @@ [project] -name = Tor Newsletter Archive +name = Tor Newsletter Archive [alternatives.en] name = English primary = yes url_prefix = / locale = en_US - -

1 0

[newsletter/staging] Update menu
by hiro＠torproject.org 27 Mar '19

27 Mar '19

commit d5fe8df3cebb78d3eaf6e67d6f281c24e2fdb434 Author: hiro <hiro(a)torproject.org> Date: Mon Feb 11 17:42:20 2019 +0100 Update menu --- databags/menu+en.json | 44 ++++++++++++++++++++++++++++---------------- 1 file changed, 28 insertions(+), 16 deletions(-) diff --git a/databags/menu+en.json b/databags/menu+en.json index d167ebd..94551ae 100644 --- a/databags/menu+en.json +++ b/databags/menu+en.json @@ -1,18 +1,30 @@ { - "archive": { - "path": "/archive", - "label": "Archive" - }, - "about": { - "path": "https://www.torproject.org/", - "label": "About Tor" - }, - "donate":{ - "path": "https://donate.torproject.org/", - "label": "Donate" - }, - "RSS":{ - "path": "/rss", - "label": "RSS" - } + "archive": { + "path": "/archive", + "label": "Archive" + }, + "about": { + "path": "https://www.torproject.org/", + "label": "About Tor" + }, + "documentation":{ + "path": "https://www.torproject.org/docs/documentation.html.en", + "label": "Documentation" + }, + "support":{ + "path": "https://support.torproject.org/", + "label": "Support" + }, + "blog":{ + "path": "https://blog.torproject.org/", + "label": "Blog" + }, + "donate":{ + "path": "https://donate.torproject.org/", + "label": "Donate" + }, + "RSS":{ + "path": "/rss", + "label": "RSS" + } }

1 0

[sbws/master] fix: relayprioritizer: number of prioritized relays
by juga＠torproject.org 27 Mar '19

27 Mar '19

commit 4fa29b025ea353a8cb55a2e81298c092297a3287 Author: juga0 <juga(a)riseup.net> Date: Fri Mar 22 09:17:42 2019 +0000 fix: relayprioritizer: number of prioritized relays The number of prioritized relays was set to the minimum, but in a small testing network this could be lesser than the number of relays in the network. --- sbws/lib/relayprioritizer.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/sbws/lib/relayprioritizer.py b/sbws/lib/relayprioritizer.py index bb64b06..4932ea8 100644 --- a/sbws/lib/relayprioritizer.py +++ b/sbws/lib/relayprioritizer.py @@ -143,7 +143,9 @@ class RelayPrioritizer: # Increment the number of times ``best_priority`` has been run. self.increment_priority_lists() # Increment the number of relays that have been "prioritized". - self.increment_priority_relays(upper_limit) + # Because in a small testing network the upper limit could be smaller + # than the number of relays in the network, use the length of the list. + self.increment_priority_relays(len(relays[0:upper_limit])) for relay in relays[0:upper_limit]: log.debug('Returning next relay %s with priority %f', relay.nickname, relay.priority)

1 0

[sbws/master] fix: destination: Multiply errors by the threads
by juga＠torproject.org 27 Mar '19

27 Mar '19

commit 612807e3155d4a85d004995af0dc06b45b22905d Author: juga0 <juga(a)riseup.net> Date: Mon Mar 25 14:28:12 2019 +0000 fix: destination: Multiply errors by the threads Since when a destination fails, all the threads using it will also fail that moment. For now not checking which threads are actually using it. Also lower the time to retry. Closes: #29891. --- sbws/core/scanner.py | 19 +++++++++++++++++-- sbws/globals.py | 4 +++- sbws/lib/destination.py | 35 ++++++++++++++++++++++++----------- tests/unit/lib/test_destination.py | 12 ++++++------ 4 files changed, 50 insertions(+), 20 deletions(-) diff --git a/sbws/core/scanner.py b/sbws/core/scanner.py index 32752ca..fdb44eb 100644 --- a/sbws/core/scanner.py +++ b/sbws/core/scanner.py @@ -29,7 +29,6 @@ from multiprocessing.dummy import Pool import time import os import logging -import requests import random from .. import settings @@ -367,6 +366,19 @@ def measure_relay(args, conf, destinations, cb, rl, relay): def dispatch_worker_thread(*a, **kw): + # If at the point where the relay is actually going to be measured there + # are not any functional destinations or the `end_event` is set, do not + # try to start measuring the relay, since it will fail anyway. + try: + # a[2] is the argument `destinations` + functional_destinations = a[2].functional_destinations + # In case the arguments or the method change, catch the possible exceptions + # but ignore here that there are not destinations. + except (IndexError, TypeError): + log.debug("Wrong argument or attribute.") + functional_destinations = True + if not functional_destinations or settings.end_event.is_set(): + return None return measure_relay(*a, **kw) @@ -530,7 +542,10 @@ def main_loop(args, conf, controller, relay_list, circuit_builder, result_dump, loop_tstop = time.time() loop_tdelta = (loop_tstop - loop_tstart) / 60 - log.debug("Measured %s relays in %s minutes", num_relays, loop_tdelta) + # At this point, we know the relays that were queued to be measured. + # That does not mean they were actually measured. + log.debug("Attempted to measure %s relays in %s minutes", + num_relays, loop_tdelta) # In a testing network, exit after first loop if controller.get_conf('TestingTorNetwork') == '1': log.info("In a testing network, exiting after the first loop.") diff --git a/sbws/globals.py b/sbws/globals.py index 5003973..723625d 100644 --- a/sbws/globals.py +++ b/sbws/globals.py @@ -127,7 +127,9 @@ DESTINATION_VERIFY_CERTIFICATE = True # whether the destination is functional or not. NUM_DESTINATION_ATTEMPTS_STORED = 10 # Time to wait before trying again a destination that wasn't functional. -DELTA_SECONDS_RETRY_DESTINATION = 60 * 60 * 3 +# Because intermitent failures with CDN destinations, start trying again +# after 5 min. +DELTA_SECONDS_RETRY_DESTINATION = 60 * 5 # Number of consecutive times a destination can fail before considering it # not functional. MAX_NUM_DESTINATION_FAILURES = 3 diff --git a/sbws/lib/destination.py b/sbws/lib/destination.py index 21a907c..3ccd94f 100644 --- a/sbws/lib/destination.py +++ b/sbws/lib/destination.py @@ -229,12 +229,19 @@ class Destination: the time to try again is incremented and resetted as soon as the destination does not fail. """ + # NOTE: does a destination fail because several threads are using + # it at the same time? + # If a destination fails for 1 minute and there're 3 threads, the + # 3 threads will fail. + # Failed the last X consecutive times if self._are_last_attempts_failures(): + # The log here will appear in all the the queued + # relays and threads. log.warning("The last %s times the destination %s failed." - "It will not be used again in %s hours.\n", + "Disabled for %s minutes.", self._max_num_failures, self.url, - self._delta_seconds_retry / 60 / 60) + self._delta_seconds_retry / 60) log.warning("Please, add more destinations or increment the " "number of maximum number of consecutive failures " "in the configuration.") @@ -285,19 +292,22 @@ class Destination: return p @staticmethod - def from_config(conf_section, max_dl): + def from_config(conf_section, max_dl, number_threads): assert 'url' in conf_section url = conf_section['url'] verify = _parse_verify_option(conf_section) try: - max_num_failures = conf_section.getint('max_num_failures') + # Because one a destination fails, all the threads that are using + # it at that moment will fail too, multiply by the number of + # threads. + max_num_failures = (conf_section.getint('max_num_failures') + or MAX_NUM_DESTINATION_FAILURES) except ValueError: - log.warning("Configuration max_num_failures is wrong, ignoring.") - max_num_failures = None - if max_num_failures: - return Destination(url, max_dl, verify, max_num_failures) - else: - return Destination(url, max_dl, verify) + # If the operator did not setup the number, set to the default. + max_num_failures = MAX_NUM_DESTINATION_FAILURES + + max_num_failures *= number_threads + return Destination(url, max_dl, verify, max_num_failures) class DestinationList: @@ -331,7 +341,10 @@ class DestinationList: log.debug('Loading info for destination %s', key) dests.append(Destination.from_config( conf[dest_sec], - conf.getint('scanner', 'max_download_size'))) + # Multiply by the number of threads since all the threads will + # fail at the same time. + conf.getint('scanner', 'max_download_size'), + conf.getint('scanner', 'measurement_threads'))) if len(dests) < 1: msg = 'No enabled destinations in config. Please see '\ 'docs/source/man_sbws.ini.rst" or "man 5 sbws.ini" ' \ diff --git a/tests/unit/lib/test_destination.py b/tests/unit/lib/test_destination.py index fc8b489..ca2ff21 100644 --- a/tests/unit/lib/test_destination.py +++ b/tests/unit/lib/test_destination.py @@ -5,9 +5,9 @@ from sbws.lib import destination def test_destination_is_functional(): - eight_hours_ago = datetime.utcnow() - timedelta(hours=8) - four_hours_ago = datetime.utcnow() - timedelta(hours=4) - two_hours_ago = datetime.utcnow() - timedelta(hours=2) + eleven_mins_ago = datetime.utcnow() - timedelta(minutes=11) + six_mins_ago = datetime.utcnow() - timedelta(minutes=6) + four_mins_ago = datetime.utcnow() - timedelta(minutes=4) d = destination.Destination('unexistenturl', 0, False) assert d.is_functional() @@ -29,19 +29,19 @@ def test_destination_is_functional(): d.add_failure() d.add_failure() # And last failure was 2h ago - d.add_failure(two_hours_ago) + d.add_failure(four_mins_ago) assert d._are_last_attempts_failures() assert not d._is_last_try_old_enough() assert not d.is_functional() # But if the last failure was 4h ago, try to use it again # And last failure was 4h ago - d.add_failure(four_hours_ago) + d.add_failure(six_mins_ago) assert d._is_last_try_old_enough() assert d.is_functional() # If last failure was 8h ago, try to use it again again - d.add_failure(eight_hours_ago) + d.add_failure(eleven_mins_ago) assert d._is_last_try_old_enough() assert d.is_functional()

1 0

[sbws/master] fix: Store the error when there is not destination
by juga＠torproject.org 27 Mar '19

27 Mar '19

commit e835710e4a59c7255211e5a67a4e2a3b5c831371 Author: juga0 <juga(a)riseup.net> Date: Fri Mar 22 16:15:56 2019 +0000 fix: Store the error when there is not destination --- sbws/core/scanner.py | 27 ++++++++++++++++----------- sbws/lib/resultdump.py | 9 +++++++++ 2 files changed, 25 insertions(+), 11 deletions(-) diff --git a/sbws/core/scanner.py b/sbws/core/scanner.py index d82aaca..5b28637 100644 --- a/sbws/core/scanner.py +++ b/sbws/core/scanner.py @@ -13,7 +13,7 @@ from ..lib.circuitbuilder import GapsCircuitBuilder as CB from ..lib.resultdump import ResultDump from ..lib.resultdump import ( ResultSuccess, ResultErrorCircuit, ResultErrorStream, - ResultErrorSecondRelay, ResultError, # ResultErrorDestination + ResultErrorSecondRelay, ResultError, ResultErrorDestination ) from ..lib.relaylist import RelayList from ..lib.relayprioritizer import RelayPrioritizer @@ -261,20 +261,25 @@ def measure_relay(args, conf, destinations, cb, rl, relay): ] # Pick a destionation dest = destinations.next() - # If there is no any destination at this point, it can not continue. + # When there're no any functional destinations. if not dest: - # XXX: this should return a ResultError - # instead of stopping the scanner once a destination can be recovered. + # NOTE: When there're still functional destinations but only one of + # them fail, the error will be included in `ResultErrorStream`. + # Since this is being executed in a thread, the scanner can not + # be stop here, but the `end_event` signal can be set so that the + # main thread stop the scanner. + # It might be useful to store the fact that the destinations fail, + # so store here the error, and set the signal once the error is stored + # (in `resultump`). log.critical("There are not any functional destinations.\n" "It is recommended to set several destinations so that " "the scanner can continue if one fails.") - # NOTE: Because this is executed in a thread, stop_threads can not - # be call from here, it has to be call from the main thread. - # Instead set the singleton end event, that will call stop_threads - # from the main process. - # Errors with only one destination are set in ResultErrorStream. - settings.end_event.set() - return None + reason = "No functional destinations" + # Resultdump will set end_event after storing the error + return [ + ResultErrorDestination(relay, [], '', our_nick, msg=reason), + ] + # Pick a relay to help us measure the given relay. If the given relay is an # exit, then pick a non-exit. Otherwise pick an exit. helper = None diff --git a/sbws/lib/resultdump.py b/sbws/lib/resultdump.py index 2ebfa20..f7ec55b 100644 --- a/sbws/lib/resultdump.py +++ b/sbws/lib/resultdump.py @@ -776,6 +776,15 @@ class ResultDump: "destination {}: {}".format( result.fingerprint, result.nickname, result.circ, result.dest_url, result.msg) + # When the error is that there are not more functional destinations. + if result.type == "error-destination": + log.info("Shutting down because there are not functional " + "destinations.") + # NOTE: Because this is executed in a thread, stop_threads can not + # be call from here, it has to be call from the main thread. + # Instead set the singleton end event, that will call stop_threads + # from the main process. + settings.end_event.set() log.info(msg) def enter(self):

1 0

[sbws/master] fix: docs: Remove contributing from readme
by juga＠torproject.org 27 Mar '19

27 Mar '19

commit a9dc014e3fc04d13d8595127a1a5dcbba66adb54 Author: juga0 <juga(a)riseup.net> Date: Tue Mar 26 08:39:30 2019 +0000 fix: docs: Remove contributing from readme and move contributing from the root directory to the docs/ one, since it links to testing but testing is not in the root directory. This way when a user only reads the readme locally, does not try to find testing in the root directory and instead will search for more documentation in docs/. --- README.md | 6 ------ docs/source/CONTRIBUTING.rst | 1 - CONTRIBUTING.rst => docs/source/contributing.rst | 0 docs/source/index.rst | 2 +- 4 files changed, 1 insertion(+), 8 deletions(-) diff --git a/README.md b/README.md index 8c57464..e796e46 100644 --- a/README.md +++ b/README.md @@ -42,12 +42,6 @@ Deploying and running See [./DEPLOY.rst](DEPLOY.rst) (in local directory or GitHub) or [DEPLOY.html](DEPLOY.html) (local build or Read the Docs). -Contributing --------------- - -See [./CONTRIBUTING.rst](CONTRIBUTING.rst) (in local directory or GitHub) or -[CONTRIBUTING.html](CONTRIBUTING.html) (local build or Read the Docs). - Changelog -------------- diff --git a/docs/source/CONTRIBUTING.rst b/docs/source/CONTRIBUTING.rst deleted file mode 120000 index e9a8ba6..0000000 --- a/docs/source/CONTRIBUTING.rst +++ /dev/null @@ -1 +0,0 @@ -../../CONTRIBUTING.rst \ No newline at end of file diff --git a/CONTRIBUTING.rst b/docs/source/contributing.rst similarity index 100% rename from CONTRIBUTING.rst rename to docs/source/contributing.rst diff --git a/docs/source/index.rst b/docs/source/index.rst index e6c5d6e..9806ef3 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -19,7 +19,6 @@ and in ``sbws`` Debian package: README INSTALL DEPLOY - CONTRIBUTING CHANGELOG AUTHORS man_sbws @@ -37,6 +36,7 @@ Included in the .. toctree:: :maxdepth: 1 + contributing testing documenting how_works

1 0

[sbws/master] fix: scanner: catch all possible exceptions
by juga＠torproject.org 27 Mar '19

27 Mar '19

commit eb749c09cc28ff5297b4ac770543a6bfbcb996ba Author: juga0 <juga(a)riseup.net> Date: Sat Mar 23 13:28:31 2019 +0000 fix: scanner: catch all possible exceptions calling requests.get, since RequestException does not catch all. --- sbws/core/scanner.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/sbws/core/scanner.py b/sbws/core/scanner.py index 5b28637..32752ca 100644 --- a/sbws/core/scanner.py +++ b/sbws/core/scanner.py @@ -98,8 +98,11 @@ def timed_recv_from_server(session, dest, byte_range): try: # headers are merged with the session ones, not overwritten. session.get(dest.url, headers=HTTP_GET_HEADERS, verify=dest.verify) - # Catch any `requests` exception, so that it can stored in the Result - except requests.exceptions.RequestException as e: + # All `requests` exceptions could be caught with + # `requests.exceptions.RequestException`, but it seems that `requests` + # does not catch all the ssl exceptions and urllib3 doesn't seem to have + # a base exception class. + except Exception as e: log.debug(e) return False, e end_time = time.time()

1 0