[tor-dev] Relay Database: Existing Schemas?
kostas at jakeliunas.com
Fri Feb 26 23:23:11 UTC 2016
On Thu, Apr 16, 2015 at 4:53 PM, Karsten Loesing <karsten at torproject.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> On 15/04/15 21:18, nusenu wrote:
>> I'm planing to store relay data in a database for analysis. I
>> assume others have done so as well, so before going ahead and
>> designing a db schema I'd like to make sure I didn't miss
>> pre-existing db schemas one could build on.
>> Data to be stored: - (most) descriptor fields - everything that
>> onionoo provides in a details record (geoip, asn, rdns, tordnsel,
>> cw, ...) - historic records
>> I didn't find something matching so far, so I'll go ahead, but if
>> you know of other existing relay db schemas I'd like to hear about
>> thanks, nusenu
>> "GSoC2013: Searchable Tor descriptor archive" (Kostas Jakeliunas)
>> l https://github.com/wfn/torsearch (btw, someone knows the license
>> of this?)
> Cc'ing Kostas for this question.
I've been going through old mail, and on 2015-04-16 you asked about
about a license (see above).
Just added a LICENSE file - can't hurt (standard BSD 3-clause).
If you're still by any chance collating (ha) and/or want to talk about
schema design for descriptors (I personally would not lose hope for
RDBMSes for large datasets - not until one gets into *actually* big
data - say, terabytes at least, or more - but of course it gets
nuanced real fast).
0x0e5dce45 @ pgp.mit.edu
>>> This is true: the summary/details documents (just like in Onionoo
>>> proper) deal with the *last* known info about relays.
>> (didn't find db/tordir.sql mentioned in the pdf)
> That file lives here now:
> A better schema might be the following one though. It's smaller, but
> it's better documented:
>> "Instructions for setting up relay descriptor database"
> five years old. I'd say ignore that one.
>> "Set up descriptor database for other researchers"
> Also five years old. Better ignore.
> Hope that helps.
> All the best,
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
> Comment: GPGTools - http://gpgtools.org
> -----END PGP SIGNATURE-----
More information about the tor-dev