[tor-commits] [metrics-web/master] Add a robots.txt to discourage indexing database lookups.

commit 46264713a550421c358e471a1dd1c50f20a46e5f Author: Karsten Loesing karsten.loesing@gmx.net Date: Mon Sep 12 08:21:42 2011 +0200

Add a robots.txt to discourage indexing database lookups. --- web/robots.txt | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/web/robots.txt b/web/robots.txt new file mode 100644 index 0000000..3b8e76e --- /dev/null +++ b/web/robots.txt @@ -0,0 +1,12 @@ +User-agent: * +Disallow: /networkstatus.html +Disallow: /routerdetail.html +Disallow: /relay.html +Disallow: /csv/ +Disallow: /serverdesc +Disallow: /extrainfodesc +Disallow: /descriptor.html +Disallow: /consensus +Disallow: /votes +Disallow: /consensus-health.html +