On 1/23/21 8:59 PM, Silvia Puglisi, Ph.D. wrote:
Hi Jacob,
On 2021-01-22 12:21, Jacob Hrbek wrote:
On 1/21/21 9:27 PM, Silvia wrote:
Exciting to see fsfe moving to onions. How can we help you guys with this?
Currently the main problem is with implementation as there is an issue with certificates using TLS-over-onions (Not economical for non-profit foundation) where it seems that using reverse proxy with currently used Apache or implementing EOTK is the way to go there?
Yes EOTK uses a TLS certificate. The idea behind this is that if the certificate belongs to fsfe, visitors of the onion service can be sure that the onion has been setup by fsfe. The certificate is not needed for any other reason than that.
If you are concerned about how people discover your onion you can use the onion-location header so that people visting fsfe.org over tor get the onion available button on the url bar and can get redirected to the onion (https://community.torproject.org/onion-services/advanced/onion-location/).
More options and way to configure EOTK (alec seems to be currently busy and unable to answer) appreciated.
EOTK is a tool that setup a few options for you in nginx and install required packages, but you can setup the onion also manually.
Here for example you will find a gist of the nginx config of the propubblica onion: https://gist.github.com/mtigas/9a7425dfdacda15790b2
Also brainstorm for the implementation as a whole would be appreciated the services seems to be mostly running in jail/VM which is favorable to be preserved for security reasons (e.g. in scenario where there is a major bug discovered in the wild to reduce the impact of one service on the system). So i am currently unsure whether we want to:
- run one tor daemon per system in jail/VM to provide the routing from
exposed ports from the services e.g. https://git.fsfe.org/kreyren/fsfe-planet/src/branch/onionz/docker-compose.ym... 2. implementing tor daemon within these jails/VMs with the service
srv/service1 (exposing port 12447) srv/service2 (exposing port 12448)
and setting tor as
HiddenServiceDir /var/lib/tor/service1 HiddenServicePort 12447 127.0.0.1:12447
HiddenServiceDir /var/lib/tor/service2 HiddenServicePort 12447 127.0.0.1:12447
I am not sure about the exact architecture here, but generally you need a master onion where you run onion balance and use it to scale horizontally with different backends (https://onionbalance-v3.readthedocs.io/en/latest/v3/tutorial-v3.html).
If you are concerned about DOS attacks you can also implement some more advanced web server configs. One of them is using captchas, another is to use cookies to filter out scripted clients. The idea in this case is that the web server sends the client a cookie and ask the client to verify it. Usually scripted clients don't set cookies so the verify fails and you find out that the client is malicious.
Nginx uses openresty and lua to implement captchas. This solution is usually highly scripted. With regards to cookies I can recommend this library from cloudflare for openresty https://github.com/cloudflare/lua-resty-cookie. I am sure there are equivalent solution in apache.
- implementing tor daemon on the router assuming all services being
routed through a routing server, but i am concerned about sanitization as if there is a bug in tor that could expose user traffic to bad actors. (currently being discussed)
- Implementing xen (https://en.wikipedia.org/wiki/Xen) which currently
not favorable as it would require lots of work on the backend.
- Other?
There is a tool called onionscan (https://onionscan.org/) that can help you find vulnerabilities on your onion. This also test things like bugs in your web service that might expose users data and information that you might prefer to keep secure.
I also assume that the fsfe onion isn't interested to be anonymous so you might consider setting up a 1-hop onion in this case (https://support.torproject.org/glossary/single-onion-service/).
Let me know if you need more help.
Cheers, -hiro
FWIW i would also like to provide something like https://onion.debian.org so that the website list is available to the end-user.
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
I am sorry for delay had to process lots of informations.
The onion-location HTTP header is already tracked.
Added propublica configuration in tracking as example configuration, thanks!
One-hop non-anonymous onion service noted in tracking.
OnionScan noted in tracking and recommended to be adapted as Continuous Integration.
Personal tracking updated in https://github.com/Kreyren/kreyren/issues/60 peer-review appreciated.
About the (D)DoS -> i was told that they are not a concern over onion routing due to the hard delay used for encryption. Can you confirm? Overall the (D)DoS is part of a threat model as FSFE has been (D)DoSed in the past where the main concern is Gitea as the static pages require an unreasonable amount of resources to be taken down atm.
About Apache -> This is currently used by the webserver and needs to be implemented with priority in https://git.fsfe.org/fsfe-system-hackers/webserver-bunsen/src/branch/master/... (currently tracked in https://git.fsfe.org/fsfe-system-hackers/webserver-bunsen/pulls/7). Any relevant information to the implementation is appreciated as i am currently struggling with making the TLS-over-onion work there where apache should strip the TLS when the website is accessed over onions. Any relevant information appreciated.
About Nginx -> Currently not an option as the website is already using Apache.
About EOTK -> I would like to implement this as an alternative configuration for system hackers to have a choice and do their research more efficiently and I assume that being a better implementation here as we currently want to onionize gitea service which based on my experience on https://git.dotya.ml requires changing of non-relative links within the webapp (which can be done in nginx/apache, but seems to be painful to maintain?).
FWIW i am also considering decentralization for the static pages where the theory is to reduce the system load, make the websites less vulnerable to (D)DoS and make the websites to load faster as the webserver would be closer to the client in question. I am not aware of sane configuration for this over onions, but any relevant info appreciated.
Hi Jacob,
On 2021-01-26 20:14, Jacob Hrbek wrote:
On 1/23/21 8:59 PM, Silvia Puglisi, Ph.D. wrote:
Hi Jacob,
On 2021-01-22 12:21, Jacob Hrbek wrote:
On 1/21/21 9:27 PM, Silvia wrote:
Exciting to see fsfe moving to onions. How can we help you guys with this?
Currently the main problem is with implementation as there is an issue with certificates using TLS-over-onions (Not economical for non-profit foundation) where it seems that using reverse proxy with currently used Apache or implementing EOTK is the way to go there?
Yes EOTK uses a TLS certificate. The idea behind this is that if the certificate belongs to fsfe, visitors of the onion service can be sure that the onion has been setup by fsfe. The certificate is not needed for any other reason than that.
If you are concerned about how people discover your onion you can use the onion-location header so that people visting fsfe.org over tor get the onion available button on the url bar and can get redirected to the onion (https://community.torproject.org/onion-services/advanced/onion-location/).
More options and way to configure EOTK (alec seems to be currently busy and unable to answer) appreciated.
EOTK is a tool that setup a few options for you in nginx and install required packages, but you can setup the onion also manually.
Here for example you will find a gist of the nginx config of the propubblica onion: https://gist.github.com/mtigas/9a7425dfdacda15790b2
Also brainstorm for the implementation as a whole would be appreciated the services seems to be mostly running in jail/VM which is favorable to be preserved for security reasons (e.g. in scenario where there is a major bug discovered in the wild to reduce the impact of one service on the system). So i am currently unsure whether we want to:
- run one tor daemon per system in jail/VM to provide the routing from
exposed ports from the services e.g. https://git.fsfe.org/kreyren/fsfe-planet/src/branch/onionz/docker-compose.ym... 2. implementing tor daemon within these jails/VMs with the service
srv/service1 (exposing port 12447) srv/service2 (exposing port 12448)
and setting tor as
HiddenServiceDir /var/lib/tor/service1 HiddenServicePort 12447 127.0.0.1:12447
HiddenServiceDir /var/lib/tor/service2 HiddenServicePort 12447 127.0.0.1:12447
I am not sure about the exact architecture here, but generally you need a master onion where you run onion balance and use it to scale horizontally with different backends (https://onionbalance-v3.readthedocs.io/en/latest/v3/tutorial-v3.html).
If you are concerned about DOS attacks you can also implement some more advanced web server configs. One of them is using captchas, another is to use cookies to filter out scripted clients. The idea in this case is that the web server sends the client a cookie and ask the client to verify it. Usually scripted clients don't set cookies so the verify fails and you find out that the client is malicious.
Nginx uses openresty and lua to implement captchas. This solution is usually highly scripted. With regards to cookies I can recommend this library from cloudflare for openresty https://github.com/cloudflare/lua-resty-cookie. I am sure there are equivalent solution in apache.
- implementing tor daemon on the router assuming all services being
routed through a routing server, but i am concerned about sanitization as if there is a bug in tor that could expose user traffic to bad actors. (currently being discussed)
- Implementing xen (https://en.wikipedia.org/wiki/Xen) which currently
not favorable as it would require lots of work on the backend.
- Other?
There is a tool called onionscan (https://onionscan.org/) that can help you find vulnerabilities on your onion. This also test things like bugs in your web service that might expose users data and information that you might prefer to keep secure.
I also assume that the fsfe onion isn't interested to be anonymous so you might consider setting up a 1-hop onion in this case (https://support.torproject.org/glossary/single-onion-service/).
Let me know if you need more help.
Cheers, -hiro
FWIW i would also like to provide something like https://onion.debian.org so that the website list is available to the end-user.
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
I am sorry for delay had to process lots of informations.
The onion-location HTTP header is already tracked.
Added propublica configuration in tracking as example configuration, thanks!
One-hop non-anonymous onion service noted in tracking.
OnionScan noted in tracking and recommended to be adapted as Continuous Integration.
Personal tracking updated in https://github.com/Kreyren/kreyren/issues/60 peer-review appreciated.
About the (D)DoS -> i was told that they are not a concern over onion routing due to the hard delay used for encryption. Can you confirm? Overall the (D)DoS is part of a threat model as FSFE has been (D)DoSed in the past where the main concern is Gitea as the static pages require an unreasonable amount of resources to be taken down atm.
So onion service DoS is a bit different than normal DoS, because the target in this case is the tor daemon. So while the website will still be reached on the normal internet in the case of onion service DoS it will be the onion that will not be reached.
Check out this blog post to have a better idea: https://blog.torproject.org/stop-the-onion-denial
About Apache -> This is currently used by the webserver and needs to be implemented with priority in https://git.fsfe.org/fsfe-system-hackers/webserver-bunsen/src/branch/master/... (currently tracked in https://git.fsfe.org/fsfe-system-hackers/webserver-bunsen/pulls/7). Any relevant information to the implementation is appreciated as i am currently struggling with making the TLS-over-onion work there where apache should strip the TLS when the website is accessed over onions. Any relevant information appreciated.
I am not particularly familiar with the specific Apache syntax but why do you want to strip the TLS? Can't you just specify a different configuration for the onion in Apache where TLS is not used? Like a different VirtualHost that use port 80 for the onion?
About Nginx -> Currently not an option as the website is already using Apache.
About EOTK -> I would like to implement this as an alternative configuration for system hackers to have a choice and do their research more efficiently and I assume that being a better implementation here as we currently want to onionize gitea service which based on my experience on https://git.dotya.ml requires changing of non-relative links within the webapp (which can be done in nginx/apache, but seems to be painful to maintain?).
What do you mean alternative configuration?
We are also working on a library of scripts for different configuration systems. For example: https://gitlab.torproject.org/hiro/terraform-onions for terraform.
FWIW i am also considering decentralization for the static pages where the theory is to reduce the system load, make the websites less vulnerable to (D)DoS and make the websites to load faster as the webserver would be closer to the client in question. I am not aware of sane configuration for this over onions, but any relevant info appreciated.
Do you mean serving the static pages from different mirrors? I think in this case you might want to consider using a v3 onionbalance master onion and implement a few fronts and then add all their onion addresses to onionbalance's configuration and run it. If DoS over onion is a big concern for you I think running 2/3 fronts is a good idea.
Let me know if this helps.
Cheers, -hiro
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
On 1/27/21 1:22 PM, Hiro/Silvia wrote:
Hi Jacob,
On 2021-01-26 20:14, Jacob Hrbek wrote:
On 1/23/21 8:59 PM, Silvia Puglisi, Ph.D. wrote:
Hi Jacob,
On 2021-01-22 12:21, Jacob Hrbek wrote:
On 1/21/21 9:27 PM, Silvia wrote:
Exciting to see fsfe moving to onions. How can we help you guys with this?
Currently the main problem is with implementation as there is an issue with certificates using TLS-over-onions (Not economical for non-profit foundation) where it seems that using reverse proxy with currently used Apache or implementing EOTK is the way to go there?
Yes EOTK uses a TLS certificate. The idea behind this is that if the certificate belongs to fsfe, visitors of the onion service can be sure that the onion has been setup by fsfe. The certificate is not needed for any other reason than that.
If you are concerned about how people discover your onion you can use the onion-location header so that people visting fsfe.org over tor get the onion available button on the url bar and can get redirected to the onion (https://community.torproject.org/onion-services/advanced/onion-location/).
More options and way to configure EOTK (alec seems to be currently busy and unable to answer) appreciated.
EOTK is a tool that setup a few options for you in nginx and install required packages, but you can setup the onion also manually.
Here for example you will find a gist of the nginx config of the propubblica onion: https://gist.github.com/mtigas/9a7425dfdacda15790b2
Also brainstorm for the implementation as a whole would be appreciated the services seems to be mostly running in jail/VM which is favorable to be preserved for security reasons (e.g. in scenario where there is a major bug discovered in the wild to reduce the impact of one service on the system). So i am currently unsure whether we want to:
- run one tor daemon per system in jail/VM to provide the routing from
exposed ports from the services e.g. https://git.fsfe.org/kreyren/fsfe-planet/src/branch/onionz/docker-compose.ym... 2. implementing tor daemon within these jails/VMs with the service
srv/service1 (exposing port 12447) srv/service2 (exposing port 12448)
and setting tor as
HiddenServiceDir /var/lib/tor/service1 HiddenServicePort 12447 127.0.0.1:12447
HiddenServiceDir /var/lib/tor/service2 HiddenServicePort 12447 127.0.0.1:12447
I am not sure about the exact architecture here, but generally you need a master onion where you run onion balance and use it to scale horizontally with different backends (https://onionbalance-v3.readthedocs.io/en/latest/v3/tutorial-v3.html).
If you are concerned about DOS attacks you can also implement some more advanced web server configs. One of them is using captchas, another is to use cookies to filter out scripted clients. The idea in this case is that the web server sends the client a cookie and ask the client to verify it. Usually scripted clients don't set cookies so the verify fails and you find out that the client is malicious.
Nginx uses openresty and lua to implement captchas. This solution is usually highly scripted. With regards to cookies I can recommend this library from cloudflare for openresty https://github.com/cloudflare/lua-resty-cookie. I am sure there are equivalent solution in apache.
- implementing tor daemon on the router assuming all services being
routed through a routing server, but i am concerned about sanitization as if there is a bug in tor that could expose user traffic to bad actors. (currently being discussed)
- Implementing xen (https://en.wikipedia.org/wiki/Xen) which currently
not favorable as it would require lots of work on the backend.
- Other?
There is a tool called onionscan (https://onionscan.org/) that can help you find vulnerabilities on your onion. This also test things like bugs in your web service that might expose users data and information that you might prefer to keep secure.
I also assume that the fsfe onion isn't interested to be anonymous so you might consider setting up a 1-hop onion in this case (https://support.torproject.org/glossary/single-onion-service/).
Let me know if you need more help.
Cheers, -hiro
FWIW i would also like to provide something like https://onion.debian.org so that the website list is available to the end-user.
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
I am sorry for delay had to process lots of informations.
The onion-location HTTP header is already tracked.
Added propublica configuration in tracking as example configuration, thanks!
One-hop non-anonymous onion service noted in tracking.
OnionScan noted in tracking and recommended to be adapted as Continuous Integration.
Personal tracking updated in https://github.com/Kreyren/kreyren/issues/60 peer-review appreciated.
About the (D)DoS -> i was told that they are not a concern over onion routing due to the hard delay used for encryption. Can you confirm? Overall the (D)DoS is part of a threat model as FSFE has been (D)DoSed in the past where the main concern is Gitea as the static pages require an unreasonable amount of resources to be taken down atm.
So onion service DoS is a bit different than normal DoS, because the target in this case is the tor daemon. So while the website will still be reached on the normal internet in the case of onion service DoS it will be the onion that will not be reached.
Check out this blog post to have a better idea: https://blog.torproject.org/stop-the-onion-denial
About Apache -> This is currently used by the webserver and needs to be implemented with priority in https://git.fsfe.org/fsfe-system-hackers/webserver-bunsen/src/branch/master/... (currently tracked in https://git.fsfe.org/fsfe-system-hackers/webserver-bunsen/pulls/7). Any relevant information to the implementation is appreciated as i am currently struggling with making the TLS-over-onion work there where apache should strip the TLS when the website is accessed over onions. Any relevant information appreciated.
I am not particularly familiar with the specific Apache syntax but why do you want to strip the TLS? Can't you just specify a different configuration for the onion in Apache where TLS is not used? Like a different VirtualHost that use port 80 for the onion?
About Nginx -> Currently not an option as the website is already using Apache.
About EOTK -> I would like to implement this as an alternative configuration for system hackers to have a choice and do their research more efficiently and I assume that being a better implementation here as we currently want to onionize gitea service which based on my experience on https://git.dotya.ml requires changing of non-relative links within the webapp (which can be done in nginx/apache, but seems to be painful to maintain?).
What do you mean alternative configuration?
We are also working on a library of scripts for different configuration systems. For example: https://gitlab.torproject.org/hiro/terraform-onions for terraform.
FWIW i am also considering decentralization for the static pages where the theory is to reduce the system load, make the websites less vulnerable to (D)DoS and make the websites to load faster as the webserver would be closer to the client in question. I am not aware of sane configuration for this over onions, but any relevant info appreciated.
Do you mean serving the static pages from different mirrors? I think in this case you might want to consider using a v3 onionbalance master onion and implement a few fronts and then add all their onion addresses to onionbalance's configuration and run it. If DoS over onion is a big concern for you I think running 2/3 fronts is a good idea.
Let me know if this helps.
Cheers, -hiro
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
How realistic is (D)DoS over onion on FSFE? Afaik onion-service (v3) should have a mitigation for that?
If the (D)DoS over onion on the tor daemon is a concern then: 1. How to prevent Onion (D)DoS on static pages? 2. How to handle (D)DoS on the Gitea instance? The Proof-of-work is currently not an option as the access to that is granted only to the supporters so i guess the concern would be git pulling? 3. The login prompt on https://id.fsfe.org/authorization seems to be a concern where captcha + rate limitation per IP seems a sufficient solution?
What do you mean alternative configuration?
Afaik the system hackers are currently working on a solution using apache where i understand that EOTK would be independent from apache as a standalone solution?
I don't have access to view that so waiting for an account atm
About decentralization the proof-of-concept is https://thedecentralizedweb.com/ (not necessary that this solution would be used) where the current theory of implementation is allowing anyone to provide a mirror of the website to join the decentralized stack so that from the point of view of the user the website is the same without any noticable changes, but reachable from multiple machines to do load balancing and routing optimization.
The current idea is using self-deployed DNS server that would do the load balancing with some logic to verify that the served website has not been altered from the main one where the onion-balance with few tweaks might be a good solution for onion services, but i don't know how painful it would be to implement it to allow painless integration of mirrors atm.
About apache i think i made it to work the way i want atm, will let you know about the result as i do more research and based on the input from system hackers (that are in charge of the implementation).
Hi Jacob, Sorry for the late reply.
On 1/27/21 9:57 PM, kreyren@rixotstudio.cz wrote:
On 1/27/21 1:22 PM, Hiro/Silvia wrote:
Hi Jacob,
On 2021-01-26 20:14, Jacob Hrbek wrote:
On 1/23/21 8:59 PM, Silvia Puglisi, Ph.D. wrote:
Hi Jacob,
On 2021-01-22 12:21, Jacob Hrbek wrote:
On 1/21/21 9:27 PM, Silvia wrote:
Exciting to see fsfe moving to onions. How can we help you guys with this?
Currently the main problem is with implementation as there is an issue with certificates using TLS-over-onions (Not economical for non-profit foundation) where it seems that using reverse proxy with currently used Apache or implementing EOTK is the way to go there?
Yes EOTK uses a TLS certificate. The idea behind this is that if the certificate belongs to fsfe, visitors of the onion service can be sure that the onion has been setup by fsfe. The certificate is not needed for any other reason than that.
If you are concerned about how people discover your onion you can use the onion-location header so that people visting fsfe.org over tor get the onion available button on the url bar and can get redirected to the onion (https://community.torproject.org/onion-services/advanced/onion-location/).
More options and way to configure EOTK (alec seems to be currently busy and unable to answer) appreciated.
EOTK is a tool that setup a few options for you in nginx and install required packages, but you can setup the onion also manually.
Here for example you will find a gist of the nginx config of the propubblica onion: https://gist.github.com/mtigas/9a7425dfdacda15790b2
Also brainstorm for the implementation as a whole would be appreciated the services seems to be mostly running in jail/VM which is favorable to be preserved for security reasons (e.g. in scenario where there is a major bug discovered in the wild to reduce the impact of one service on the system). So i am currently unsure whether we want to:
- run one tor daemon per system in jail/VM to provide the routing from
exposed ports from the services e.g. https://git.fsfe.org/kreyren/fsfe-planet/src/branch/onionz/docker-compose.ym... 2. implementing tor daemon within these jails/VMs with the service
srv/service1 (exposing port 12447) srv/service2 (exposing port 12448)
and setting tor as
HiddenServiceDir /var/lib/tor/service1 HiddenServicePort 12447 127.0.0.1:12447
HiddenServiceDir /var/lib/tor/service2 HiddenServicePort 12447 127.0.0.1:12447
I am not sure about the exact architecture here, but generally you need a master onion where you run onion balance and use it to scale horizontally with different backends (https://onionbalance-v3.readthedocs.io/en/latest/v3/tutorial-v3.html).
If you are concerned about DOS attacks you can also implement some more advanced web server configs. One of them is using captchas, another is to use cookies to filter out scripted clients. The idea in this case is that the web server sends the client a cookie and ask the client to verify it. Usually scripted clients don't set cookies so the verify fails and you find out that the client is malicious.
Nginx uses openresty and lua to implement captchas. This solution is usually highly scripted. With regards to cookies I can recommend this library from cloudflare for openresty https://github.com/cloudflare/lua-resty-cookie. I am sure there are equivalent solution in apache.
- implementing tor daemon on the router assuming all services being
routed through a routing server, but i am concerned about sanitization as if there is a bug in tor that could expose user traffic to bad actors. (currently being discussed)
- Implementing xen (https://en.wikipedia.org/wiki/Xen) which currently
not favorable as it would require lots of work on the backend.
- Other?
There is a tool called onionscan (https://onionscan.org/) that can help you find vulnerabilities on your onion. This also test things like bugs in your web service that might expose users data and information that you might prefer to keep secure.
I also assume that the fsfe onion isn't interested to be anonymous so you might consider setting up a 1-hop onion in this case (https://support.torproject.org/glossary/single-onion-service/).
Let me know if you need more help.
Cheers, -hiro
FWIW i would also like to provide something like https://onion.debian.org so that the website list is available to the end-user.
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
I am sorry for delay had to process lots of informations.
The onion-location HTTP header is already tracked.
Added propublica configuration in tracking as example configuration, thanks!
One-hop non-anonymous onion service noted in tracking.
OnionScan noted in tracking and recommended to be adapted as Continuous Integration.
Personal tracking updated in https://github.com/Kreyren/kreyren/issues/60 peer-review appreciated.
About the (D)DoS -> i was told that they are not a concern over onion routing due to the hard delay used for encryption. Can you confirm? Overall the (D)DoS is part of a threat model as FSFE has been (D)DoSed in the past where the main concern is Gitea as the static pages require an unreasonable amount of resources to be taken down atm.
So onion service DoS is a bit different than normal DoS, because the target in this case is the tor daemon. So while the website will still be reached on the normal internet in the case of onion service DoS it will be the onion that will not be reached.
Check out this blog post to have a better idea: https://blog.torproject.org/stop-the-onion-denial
About Apache -> This is currently used by the webserver and needs to be implemented with priority in https://git.fsfe.org/fsfe-system-hackers/webserver-bunsen/src/branch/master/... (currently tracked in https://git.fsfe.org/fsfe-system-hackers/webserver-bunsen/pulls/7). Any relevant information to the implementation is appreciated as i am currently struggling with making the TLS-over-onion work there where apache should strip the TLS when the website is accessed over onions. Any relevant information appreciated.
I am not particularly familiar with the specific Apache syntax but why do you want to strip the TLS? Can't you just specify a different configuration for the onion in Apache where TLS is not used? Like a different VirtualHost that use port 80 for the onion?
About Nginx -> Currently not an option as the website is already using Apache.
About EOTK -> I would like to implement this as an alternative configuration for system hackers to have a choice and do their research more efficiently and I assume that being a better implementation here as we currently want to onionize gitea service which based on my experience on https://git.dotya.ml requires changing of non-relative links within the webapp (which can be done in nginx/apache, but seems to be painful to maintain?).
What do you mean alternative configuration?
We are also working on a library of scripts for different configuration systems. For example: https://gitlab.torproject.org/hiro/terraform-onions for terraform.
FWIW i am also considering decentralization for the static pages where the theory is to reduce the system load, make the websites less vulnerable to (D)DoS and make the websites to load faster as the webserver would be closer to the client in question. I am not aware of sane configuration for this over onions, but any relevant info appreciated.
Do you mean serving the static pages from different mirrors? I think in this case you might want to consider using a v3 onionbalance master onion and implement a few fronts and then add all their onion addresses to onionbalance's configuration and run it. If DoS over onion is a big concern for you I think running 2/3 fronts is a good idea.
Let me know if this helps.
Cheers, -hiro
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
How realistic is (D)DoS over onion on FSFE? Afaik onion-service (v3) should have a mitigation for that?
If the (D)DoS over onion on the tor daemon is a concern then:
- How to prevent Onion (D)DoS on static pages?
- How to handle (D)DoS on the Gitea instance? The Proof-of-work is
currently not an option as the access to that is granted only to the supporters so i guess the concern would be git pulling? 3. The login prompt on https://id.fsfe.org/authorization seems to be a concern where captcha + rate limitation per IP seems a sufficient solution?
What do you mean alternative configuration?
Afaik the system hackers are currently working on a solution using apache where i understand that EOTK would be independent from apache as a standalone solution?
I am not sure I follow you with this. EOTK produces nginx configs as the result of running a number of scripts. Are you porting these to apache?
I don't have access to view that so waiting for an account atm
The link is public.
About decentralization the proof-of-concept is https://thedecentralizedweb.com/ (not necessary that this solution would be used) where the current theory of implementation is allowing anyone to provide a mirror of the website to join the decentralized stack so that from the point of view of the user the website is the same without any noticable changes, but reachable from multiple machines to do load balancing and routing optimization.
The current idea is using self-deployed DNS server that would do the load balancing with some logic to verify that the served website has not been altered from the main one where the onion-balance with few tweaks might be a good solution for onion services, but i don't know how painful it would be to implement it to allow painless integration of mirrors atm.
I don't have a solution for your decentralized setup at the moment. I think this problem is more related to decentralization that onions deployment and you might have more luck requesting for help somewhere else.
About apache i think i made it to work the way i want atm, will let you know about the result as i do more research and based on the input from system hackers (that are in charge of the implementation).
Ok good to know.
Cheers, -hiro
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
Onion-routing is now officially deployed on fsfe.org with HTTP header onion-location pointing to http://fsfeorg3hsfyuhmdylxrqdvgsmjeoxuuug5a4dv3c3grkxzsl33d3xyd.onion
Peer-review is appreciated for code in https://git.fsfe.org/fsfe-system-hackers/onionproxy ^-^
On 1/27/21 9:57 PM, kreyren@rixotstudio.cz wrote:
On 1/27/21 1:22 PM, Hiro/Silvia wrote:
Hi Jacob,
On 2021-01-26 20:14, Jacob Hrbek wrote:
On 1/23/21 8:59 PM, Silvia Puglisi, Ph.D. wrote:
Hi Jacob,
On 2021-01-22 12:21, Jacob Hrbek wrote:
On 1/21/21 9:27 PM, Silvia wrote:
Exciting to see fsfe moving to onions. How can we help you guys with this?
Currently the main problem is with implementation as there is an issue with certificates using TLS-over-onions (Not economical for non-profit foundation) where it seems that using reverse proxy with currently used Apache or implementing EOTK is the way to go there?
Yes EOTK uses a TLS certificate. The idea behind this is that if the certificate belongs to fsfe, visitors of the onion service can be sure that the onion has been setup by fsfe. The certificate is not needed for any other reason than that.
If you are concerned about how people discover your onion you can use the onion-location header so that people visting fsfe.org over tor get the onion available button on the url bar and can get redirected to the onion (https://community.torproject.org/onion-services/advanced/onion-location/).
More options and way to configure EOTK (alec seems to be currently busy and unable to answer) appreciated.
EOTK is a tool that setup a few options for you in nginx and install required packages, but you can setup the onion also manually.
Here for example you will find a gist of the nginx config of the propubblica onion: https://gist.github.com/mtigas/9a7425dfdacda15790b2
Also brainstorm for the implementation as a whole would be appreciated the services seems to be mostly running in jail/VM which is favorable to be preserved for security reasons (e.g. in scenario where there is a major bug discovered in the wild to reduce the impact of one service on the system). So i am currently unsure whether we want to:
- run one tor daemon per system in jail/VM to provide the routing from
exposed ports from the services e.g. https://git.fsfe.org/kreyren/fsfe-planet/src/branch/onionz/docker-compose.ym... 2. implementing tor daemon within these jails/VMs with the service
srv/service1 (exposing port 12447) srv/service2 (exposing port 12448)
and setting tor as
HiddenServiceDir /var/lib/tor/service1 HiddenServicePort 12447 127.0.0.1:12447
HiddenServiceDir /var/lib/tor/service2 HiddenServicePort 12447 127.0.0.1:12447
I am not sure about the exact architecture here, but generally you need a master onion where you run onion balance and use it to scale horizontally with different backends (https://onionbalance-v3.readthedocs.io/en/latest/v3/tutorial-v3.html).
If you are concerned about DOS attacks you can also implement some more advanced web server configs. One of them is using captchas, another is to use cookies to filter out scripted clients. The idea in this case is that the web server sends the client a cookie and ask the client to verify it. Usually scripted clients don't set cookies so the verify fails and you find out that the client is malicious.
Nginx uses openresty and lua to implement captchas. This solution is usually highly scripted. With regards to cookies I can recommend this library from cloudflare for openresty https://github.com/cloudflare/lua-resty-cookie. I am sure there are equivalent solution in apache.
- implementing tor daemon on the router assuming all services being
routed through a routing server, but i am concerned about sanitization as if there is a bug in tor that could expose user traffic to bad actors. (currently being discussed)
- Implementing xen (https://en.wikipedia.org/wiki/Xen) which currently
not favorable as it would require lots of work on the backend.
- Other?
There is a tool called onionscan (https://onionscan.org/) that can help you find vulnerabilities on your onion. This also test things like bugs in your web service that might expose users data and information that you might prefer to keep secure.
I also assume that the fsfe onion isn't interested to be anonymous so you might consider setting up a 1-hop onion in this case (https://support.torproject.org/glossary/single-onion-service/).
Let me know if you need more help.
Cheers, -hiro
FWIW i would also like to provide something like https://onion.debian.org so that the website list is available to the end-user.
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
I am sorry for delay had to process lots of informations.
The onion-location HTTP header is already tracked.
Added propublica configuration in tracking as example configuration, thanks!
One-hop non-anonymous onion service noted in tracking.
OnionScan noted in tracking and recommended to be adapted as Continuous Integration.
Personal tracking updated in https://github.com/Kreyren/kreyren/issues/60 peer-review appreciated.
About the (D)DoS -> i was told that they are not a concern over onion routing due to the hard delay used for encryption. Can you confirm? Overall the (D)DoS is part of a threat model as FSFE has been (D)DoSed in the past where the main concern is Gitea as the static pages require an unreasonable amount of resources to be taken down atm.
So onion service DoS is a bit different than normal DoS, because the target in this case is the tor daemon. So while the website will still be reached on the normal internet in the case of onion service DoS it will be the onion that will not be reached.
Check out this blog post to have a better idea: https://blog.torproject.org/stop-the-onion-denial
About Apache -> This is currently used by the webserver and needs to be implemented with priority in https://git.fsfe.org/fsfe-system-hackers/webserver-bunsen/src/branch/master/... (currently tracked in https://git.fsfe.org/fsfe-system-hackers/webserver-bunsen/pulls/7). Any relevant information to the implementation is appreciated as i am currently struggling with making the TLS-over-onion work there where apache should strip the TLS when the website is accessed over onions. Any relevant information appreciated.
I am not particularly familiar with the specific Apache syntax but why do you want to strip the TLS? Can't you just specify a different configuration for the onion in Apache where TLS is not used? Like a different VirtualHost that use port 80 for the onion?
About Nginx -> Currently not an option as the website is already using Apache.
About EOTK -> I would like to implement this as an alternative configuration for system hackers to have a choice and do their research more efficiently and I assume that being a better implementation here as we currently want to onionize gitea service which based on my experience on https://git.dotya.ml requires changing of non-relative links within the webapp (which can be done in nginx/apache, but seems to be painful to maintain?).
What do you mean alternative configuration?
We are also working on a library of scripts for different configuration systems. For example: https://gitlab.torproject.org/hiro/terraform-onions for terraform.
FWIW i am also considering decentralization for the static pages where the theory is to reduce the system load, make the websites less vulnerable to (D)DoS and make the websites to load faster as the webserver would be closer to the client in question. I am not aware of sane configuration for this over onions, but any relevant info appreciated.
Do you mean serving the static pages from different mirrors? I think in this case you might want to consider using a v3 onionbalance master onion and implement a few fronts and then add all their onion addresses to onionbalance's configuration and run it. If DoS over onion is a big concern for you I think running 2/3 fronts is a good idea.
Let me know if this helps.
Cheers, -hiro
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
How realistic is (D)DoS over onion on FSFE? Afaik onion-service (v3) should have a mitigation for that?
If the (D)DoS over onion on the tor daemon is a concern then:
- How to prevent Onion (D)DoS on static pages?
- How to handle (D)DoS on the Gitea instance? The Proof-of-work is
currently not an option as the access to that is granted only to the supporters so i guess the concern would be git pulling? 3. The login prompt on https://id.fsfe.org/authorization seems to be a concern where captcha + rate limitation per IP seems a sufficient solution?
What do you mean alternative configuration?
Afaik the system hackers are currently working on a solution using apache where i understand that EOTK would be independent from apache as a standalone solution?
I don't have access to view that so waiting for an account atm
About decentralization the proof-of-concept is https://thedecentralizedweb.com/ (not necessary that this solution would be used) where the current theory of implementation is allowing anyone to provide a mirror of the website to join the decentralized stack so that from the point of view of the user the website is the same without any noticable changes, but reachable from multiple machines to do load balancing and routing optimization.
The current idea is using self-deployed DNS server that would do the load balancing with some logic to verify that the served website has not been altered from the main one where the onion-balance with few tweaks might be a good solution for onion services, but i don't know how painful it would be to implement it to allow painless integration of mirrors atm.
About apache i think i made it to work the way i want atm, will let you know about the result as i do more research and based on the input from system hackers (that are in charge of the implementation).
Congrats Jacob!
-hiro
On 3/24/21 4:37 PM, Jacob Hrbek wrote:
Onion-routing is now officially deployed on fsfe.org with HTTP header onion-location pointing to http://fsfeorg3hsfyuhmdylxrqdvgsmjeoxuuug5a4dv3c3grkxzsl33d3xyd.onion
Peer-review is appreciated for code in https://git.fsfe.org/fsfe-system-hackers/onionproxy ^-^
On 1/27/21 9:57 PM, kreyren@rixotstudio.cz wrote:
On 1/27/21 1:22 PM, Hiro/Silvia wrote:
Hi Jacob,
On 2021-01-26 20:14, Jacob Hrbek wrote:
On 1/23/21 8:59 PM, Silvia Puglisi, Ph.D. wrote:
Hi Jacob,
On 2021-01-22 12:21, Jacob Hrbek wrote:
On 1/21/21 9:27 PM, Silvia wrote: > Exciting to see fsfe moving to onions. > How can we help you guys with this? Currently the main problem is with implementation as there is an issue with certificates using TLS-over-onions (Not economical for non-profit foundation) where it seems that using reverse proxy with currently used Apache or implementing EOTK is the way to go there?
Yes EOTK uses a TLS certificate. The idea behind this is that if the certificate belongs to fsfe, visitors of the onion service can be sure that the onion has been setup by fsfe. The certificate is not needed for any other reason than that.
If you are concerned about how people discover your onion you can use the onion-location header so that people visting fsfe.org over tor get the onion available button on the url bar and can get redirected to the onion (https://community.torproject.org/onion-services/advanced/onion-location/).
More options and way to configure EOTK (alec seems to be currently busy and unable to answer) appreciated.
EOTK is a tool that setup a few options for you in nginx and install required packages, but you can setup the onion also manually.
Here for example you will find a gist of the nginx config of the propubblica onion: https://gist.github.com/mtigas/9a7425dfdacda15790b2
Also brainstorm for the implementation as a whole would be appreciated the services seems to be mostly running in jail/VM which is favorable to be preserved for security reasons (e.g. in scenario where there is a major bug discovered in the wild to reduce the impact of one service on the system). So i am currently unsure whether we want to:
- run one tor daemon per system in jail/VM to provide the routing from
exposed ports from the services e.g. https://git.fsfe.org/kreyren/fsfe-planet/src/branch/onionz/docker-compose.ym... 2. implementing tor daemon within these jails/VMs with the service
srv/service1 (exposing port 12447) srv/service2 (exposing port 12448)
and setting tor as
HiddenServiceDir /var/lib/tor/service1 HiddenServicePort 12447 127.0.0.1:12447
HiddenServiceDir /var/lib/tor/service2 HiddenServicePort 12447 127.0.0.1:12447
I am not sure about the exact architecture here, but generally you need a master onion where you run onion balance and use it to scale horizontally with different backends (https://onionbalance-v3.readthedocs.io/en/latest/v3/tutorial-v3.html).
If you are concerned about DOS attacks you can also implement some more advanced web server configs. One of them is using captchas, another is to use cookies to filter out scripted clients. The idea in this case is that the web server sends the client a cookie and ask the client to verify it. Usually scripted clients don't set cookies so the verify fails and you find out that the client is malicious.
Nginx uses openresty and lua to implement captchas. This solution is usually highly scripted. With regards to cookies I can recommend this library from cloudflare for openresty https://github.com/cloudflare/lua-resty-cookie. I am sure there are equivalent solution in apache.
- implementing tor daemon on the router assuming all services being
routed through a routing server, but i am concerned about sanitization as if there is a bug in tor that could expose user traffic to bad actors. (currently being discussed)
- Implementing xen (https://en.wikipedia.org/wiki/Xen) which currently
not favorable as it would require lots of work on the backend.
- Other?
There is a tool called onionscan (https://onionscan.org/) that can help you find vulnerabilities on your onion. This also test things like bugs in your web service that might expose users data and information that you might prefer to keep secure.
I also assume that the fsfe onion isn't interested to be anonymous so you might consider setting up a 1-hop onion in this case (https://support.torproject.org/glossary/single-onion-service/).
Let me know if you need more help.
Cheers, -hiro
FWIW i would also like to provide something like https://onion.debian.org so that the website list is available to the end-user.
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
I am sorry for delay had to process lots of informations.
The onion-location HTTP header is already tracked.
Added propublica configuration in tracking as example configuration, thanks!
One-hop non-anonymous onion service noted in tracking.
OnionScan noted in tracking and recommended to be adapted as Continuous Integration.
Personal tracking updated in https://github.com/Kreyren/kreyren/issues/60 peer-review appreciated.
About the (D)DoS -> i was told that they are not a concern over onion routing due to the hard delay used for encryption. Can you confirm? Overall the (D)DoS is part of a threat model as FSFE has been (D)DoSed in the past where the main concern is Gitea as the static pages require an unreasonable amount of resources to be taken down atm.
So onion service DoS is a bit different than normal DoS, because the target in this case is the tor daemon. So while the website will still be reached on the normal internet in the case of onion service DoS it will be the onion that will not be reached.
Check out this blog post to have a better idea: https://blog.torproject.org/stop-the-onion-denial
About Apache -> This is currently used by the webserver and needs to be implemented with priority in https://git.fsfe.org/fsfe-system-hackers/webserver-bunsen/src/branch/master/... (currently tracked in https://git.fsfe.org/fsfe-system-hackers/webserver-bunsen/pulls/7). Any relevant information to the implementation is appreciated as i am currently struggling with making the TLS-over-onion work there where apache should strip the TLS when the website is accessed over onions. Any relevant information appreciated.
I am not particularly familiar with the specific Apache syntax but why do you want to strip the TLS? Can't you just specify a different configuration for the onion in Apache where TLS is not used? Like a different VirtualHost that use port 80 for the onion?
About Nginx -> Currently not an option as the website is already using Apache.
About EOTK -> I would like to implement this as an alternative configuration for system hackers to have a choice and do their research more efficiently and I assume that being a better implementation here as we currently want to onionize gitea service which based on my experience on https://git.dotya.ml requires changing of non-relative links within the webapp (which can be done in nginx/apache, but seems to be painful to maintain?).
What do you mean alternative configuration?
We are also working on a library of scripts for different configuration systems. For example: https://gitlab.torproject.org/hiro/terraform-onions for terraform.
FWIW i am also considering decentralization for the static pages where the theory is to reduce the system load, make the websites less vulnerable to (D)DoS and make the websites to load faster as the webserver would be closer to the client in question. I am not aware of sane configuration for this over onions, but any relevant info appreciated.
Do you mean serving the static pages from different mirrors? I think in this case you might want to consider using a v3 onionbalance master onion and implement a few fronts and then add all their onion addresses to onionbalance's configuration and run it. If DoS over onion is a big concern for you I think running 2/3 fronts is a good idea.
Let me know if this helps.
Cheers, -hiro
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
How realistic is (D)DoS over onion on FSFE? Afaik onion-service (v3) should have a mitigation for that?
If the (D)DoS over onion on the tor daemon is a concern then:
- How to prevent Onion (D)DoS on static pages?
- How to handle (D)DoS on the Gitea instance? The Proof-of-work is
currently not an option as the access to that is granted only to the supporters so i guess the concern would be git pulling? 3. The login prompt on https://id.fsfe.org/authorization seems to be a concern where captcha + rate limitation per IP seems a sufficient solution?
What do you mean alternative configuration?
Afaik the system hackers are currently working on a solution using apache where i understand that EOTK would be independent from apache as a standalone solution?
I don't have access to view that so waiting for an account atm
About decentralization the proof-of-concept is https://thedecentralizedweb.com/ (not necessary that this solution would be used) where the current theory of implementation is allowing anyone to provide a mirror of the website to join the decentralized stack so that from the point of view of the user the website is the same without any noticable changes, but reachable from multiple machines to do load balancing and routing optimization.
The current idea is using self-deployed DNS server that would do the load balancing with some logic to verify that the served website has not been altered from the main one where the onion-balance with few tweaks might be a good solution for onion services, but i don't know how painful it would be to implement it to allow painless integration of mirrors atm.
About apache i think i made it to work the way i want atm, will let you know about the result as i do more research and based on the input from system hackers (that are in charge of the implementation).
tor-onions mailing list tor-onions@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-onions
tor-onions@lists.torproject.org
-
Hiro/Silvia -
Jacob Hrbek -
kreyren＠rixotstudio.cz
-
Silvia/Hiro