[tor-dev] GSoC - Search Engine for Hidden services

[Giovanni `evilaliv3` Pellerano]

Hi George,

i answer for the two tor2web related questions:
1) where are the stats of Tor2web?
in collaboration with Ahmia i've decided to implement some stats, but
due to the fact that we don't to want to provide I) realtime data
useful for tracking II) so much information we decided to implement
the following:

tor2web logs, only in RAM and using a two days window, a cumulative
counter for hidden services and make it available for access only to
the yesterday stats in order to not provide this data in realtime so
that the data provided is simply:
hidden service 1: #22
hidden service 2: #333
...

here is the example: https://antani.tor2web.org/antanistaticmap/stats/yesterday

2) what about list of blocked hs?
tor2web also provides the configuration for blocked hs by simply using
MD5 of resources so that the url of the blocked page is not available
since the configuration time.
due to this simple implementation we make always a check for the MD5
of the hidden service and an MD5 of the full resource, if that matchs
it means that the page must be blocked.

tor2web also does currently fetch ahmia list in order to get automatic
updates of this hashed blocklist.

this is the list provided by ahmia: https://ahmia.fi/bannedMD5.txt
(numes it currently list only an entry why?)


George if you or anybody else have good suggestion for better
improval, you are welcome :)

Giovanni