[tor-bugs] #32239 [Internal Services/Tor Sysadmin Team]: setup a cache frontend for the blog
Tor Bug Tracker & Wiki
blackhole at torproject.org
Thu Nov 7 21:49:01 UTC 2019
#32239: setup a cache frontend for the blog
-------------------------------------------------+-------------------------
Reporter: anarcat | Owner: anarcat
Type: task | Status:
| merge_ready
Priority: Medium | Milestone:
Component: Internal Services/Tor Sysadmin Team | Version:
Severity: Normal | Resolution:
Keywords: | Actual Points:
Parent ID: #32090 | Points:
Reviewer: | Sponsor:
-------------------------------------------------+-------------------------
Comment (by anarcat):
at first glance, our hit ratio is not great. i used the following lnav log
format parser:
{{{
{
"cache" : {
"title" : "Tor Nginx cache Log Format",
"description" : "Log format used by the Tor Project for its Nginx
cache servers.",
"url" : "http://example.com/log-format.html",
"regex" : {
"basic" : {
"pattern" : "^(?<client>[\\w\\d:.-]+) -
(?<user>[\\w\\d:.-]+)
\\[(?<timestamp>\\d{2}/\\w{3}/\\d{4}:\\d{2}:\\d{2}:\\d{2}(?:\\.\\d+)?(?:
[+-]\\d{4}))?\\] \"(?<method>\\w+) (?<path>\\S+) (?<transport>\\S+)\"
(?<status_code>\\d+) (?<resp_size>\\d+|-) \"(?<referer>[^\"]+)\" \"-\"
(?<upstream_addr>\\S+) (?<upstream_response_time>[\\d.-]+)
(?<upstream_cache_status>\\S+) (?<request_time>[\\d.-]+)$"
}
},
"level-field" : "upstream_cache_status",
"level" : {
"error" : "MISS",
"warning" : "BYPASS|EXPIRED|STALE|UPDATING|REVALIDATED",
"info" : "HIT"
},
"value" : {
"client" : { "kind" : "string", "identifier" : true,
"collate": "ipaddress" },
"user" : { "kind" : "string", "identifier" : true },
"method" : { "kind" : "string", "identifier" : true },
"path" : { "kind" : "string", "identifier" : true },
"transport" : { "kind" : "string", "identifier" : true },
"status_code" : { "kind" : "integer", "identifier" : true },
"resp_size" : { "kind" : "integer", "identifier" : true },
"referer" : { "kind" : "string", "identifier" : true },
"upstream_addr" : { "kind" : "string", "identifier" : true,
"collate": "ipaddress" },
"upstream_response_time" : { "kind" : "float", "identifier" :
true },
"upstream_cache_status" : { "kind" : "string", "identifier" :
true },
"request_time" : { "kind" : "float", "identifier" : true }
},
"sample" : [
{
"line" : "0.0.0.0 - - [07/Nov/2019:21:28:09 +0000] \"GET
/blog/feed HTTP/2.0\" 200 483311 \"-\" \"-\" - - HIT 0.373",
"level" : "info"
},
{
"line" : "0.0.0.0 - - [07/Nov/2019:21:28:10 +0000] \"GET
/new-release-tor-browser-90 HTTP/2.0\" 200 26764
\"https://www.google.com/\" \"-\" [2620:12a:8000::2]:443 0.416 MISS
0.420",
"level" : "error"
},
{
"line" : "0.0.0.0 - - [07/Nov/2019:21:28:10 +0000] \"GET
/robots.txt HTTP/1.1\" 200 1594 \"-\" \"-\" [2620:12a:8001::2]:443 0.020
MISS 0.020",
"level" : "error"
}
]
}
}
}}}
and then ran lnav on the access logfile, then ran the following query
(after hitting the `;` key):
{{{
select count(*), upstream_cache_status from logline where status_code <
300 group by upstream_cache_status;
}}}
which gave the following results:
{{{
count(*) upstream_cache_status
│
6│-
│
7│EXPIRED
│
2925│HIT
│
680│MISS
│
5│REVALIDATED
}}}
That's an 80% hit ratio, and that's when ignoring any status code above
2xx. If those are included, our hit ratio is around 72%.
So there is still some work to be done to improve the hit ratio.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/32239#comment:16>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
More information about the tor-bugs
mailing list