[tor-talk] Ahmia search engine works normally again
scfith at riseup.net
Thu Dec 8 19:06:30 UTC 2016
First, not sure why you want to list .onion domains. The key here is that they are HIDDEN services. But I am sure you have reasons.
Second, if you do list .onion domains, know that they will be collected.
Third, provide a simple path of machine-readable content to download so that bots won’t kill your server, like a list.json file that is up to date so your server is not ruined.
my 2 cents.
> On Dec 7, 2016, at 10:40 AM, Nurmi, Juha <juha.nurmi at ahmia.fi> wrote:
> Ahmia.fi and msydqstlz2kzerdg.onion went offline for a short while. Someone
> flooded the onion site with multiple requests per second: with randomly
> selected user agent strings someone was generating GET
> msydqstlz2kzerdg.onion/address/<somethingherethatischanging> requests.
> Constant flood was similar to this:
> [07/Dec/2016:04:11:04 +0100] "GET /address/... HTTP/1.1" "OS"
> [07/Dec/2016:04:11:04 +0100] "GET /address/... HTTP/1.1" "like"
> [07/Dec/2016:04:11:04 +0100] "GET /address/... HTTP/1.1" "NT"
> As a results, Ahmia tried to load the page /address/ over and over again
> and this exhausted CPU of my front-end server.
> How I fixed this:
> 1) I separated ahmia.fi and msydqstlz2kzerdg.onion to run
> independently with their own processes
> 2) I disabled msydqstlz2kzerdg.onion/address/ completely,
> https://ahmia.fi/address is still available
> 3) I adjusted several performance settings and cache
> 4) I added one more CPU core to the front-end server
> Ahmia search engine works normally again. Still, I hope that this flood
> tor-talk mailing list - tor-talk at lists.torproject.org
> To unsubscribe or change other settings go to
More information about the tor-talk