Hi everyone,
I'm working on ahmia.fi, the hidden service search engine and you're reading status update #4.
The django part rewrite detailled in report #3 is nearly done. Next step is to synchronize the work done on crawler and index with the django app. I also want to add more tests and documentation.
Because we are not storing hidden service information on SQL anymore, the main challenge will be to rethink how to get statistics from index. I will write about this in a later report, but I think it is doable with elasticsearch's metrics aggregation.
A lot of code has been removed from the project (be it deprecated, duplicated, doable with django framework, or by elasticsearch). I think the project is going to be easier to improve, and the barrier to start being a contributor will be lower.
Changes are not yet available upstream because the other projects (crawler and index) need work as well. This is what the other contributors and I are going to do during the next two weeks.
See you in two weeks :) Ismael