<div dir="ltr">On Wed, Aug 14, 2013 at 1:33 PM, Karsten Loesing <span dir="ltr"><<a href="mailto:karsten@torproject.org" target="_blank">karsten@torproject.org</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><br>
</div>Looks like pg_trgm is contained in postgresql-contrib-9.1, so it's more<br>
likely that we can run something requiring this extension on a<br>
<a href="http://torproject.org" target="_blank">torproject.org</a> machine. Still, requiring extensions should be the last<br>
resort if no other solution can be found. Leaving out searches for<br>
nickname substrings is a valid solution for now.</blockquote><div><br></div><div>Got it. </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><div>
>> Do you have a list of searches you're planning to support?<br>
><br>
><br>
> These are the ones that should *really* be supported:<br>
><br>
</div></div>> - ?search=nickname<br>
> - ?search=fingerprint<br>
> - ?lookup=fingerprint<br>
> - ?search=address [done some limited testing, currently not focusing on<br>
> this]<br>
<br>
The lookup parameter is basically the same as search=fingerprint with<br>
the additional requirement that fingerprint must be 40 characters long.<br>
So, this is the current search parameter.<br>
<br>
I agree, these would be good to support.<br>
<br>
You might also add another parameter ?address=address for ExoneraTor.<br>
That should, in theory, be just a subset of the search parameter.<br></blockquote><div><br></div><div>Oh yes, makes a lot of sense, OK.</div><div><br></div><div>By the way: I considered having the last consensus (all the data for at least the /summary document, or /details as well) be stored in memory (this is possible) (probably as a hashtable where key = fingerprint, value = all the fields we'd need to return) so that when the backend is queried without any search criteria, it would be possible to avoid hitting the database (which is always nice), and just dump the last consensus. (There's also caching of course, which we could discuss at a (probably quite a bit) later point.)</div>
<div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
> - ?running=<boolean><br>
<br>
This one is tricky. So far, Onionoo looks only at the very latest<br>
consensus or bridge status to decide if a relay or bridge is running or not.<br>
<br>
But now you're adding archives to Onionoo, so that people can search for<br>
a certain consensus or certain bridge status in the past, or they can<br>
search for a time interval of consensuses or bridge statuses. How do<br>
you define that a relay or bridge is running, or more importantly<br>
included as not running?<br></blockquote><div><br></div><div>Agree, this is not clear. (And whatever ends up being done, this should be well documented and clearly articulated (of course.))</div><div><br></div><div>For me at least, 'running' implies the clause whether a given relay/bridge is running *right now*, i.e. whether it is present in the very last consensus. (Here's where that hashtable (with fingerprints as keys) in memory might be able to help: no need to run a separate query / do an inner join / whatnot; it would depend on whether there's a LIMIT involved though, etc.)</div>
<div><br></div><div>I'm not sure which one is more useful (intuitively for me, the "whether it is running *right now*" is more useful.) Do you mean that it might make sense to have a field (or have "running" be it) indicating whether a given relay/bridge was present in the last consensus in the specified date range? If this is what you meant, then the "return all that are/were not running" clause would indeed be kind of..peculiar (semantically - it wouldn't be very obvious what's it doing.)</div>
<div><br></div><div>Maybe it'd be simpler to first answer, what would be the most useful case?</div><div> </div><div>> How do you define that a relay or bridge [should be] included as not running?</div><div><br></div>
<div>Could you rephrase maybe? Do you mean that it might be difficult to construct sane queries to check for this condition? Or that the situation where</div><div><div><ul><li>a "from..to" date range is specified</li>
<li>?running=false is specified</li></ul><div>would be rather confusing ('exclude those nodes which are running *right now* ('now' possibly having nothing to do with the date range)?</div></div></div><div><br>
</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
> - ?flag=flag [every kind of clause which further narrows down the query<br>
<div>> is not bad; the current db model supports all the flags that Stem does, and<br>
> each flag has its own column]<br>
<br>
</div>I'd say leave this one out until there's an actual use case.<br></blockquote><div><br></div><div>Ok, I won't focus on these now; just wanted to say that these should be possible without much ado/problems.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
> - ?first_seen_days=range<br>
> - ?last_seen_days=range<br>
<div>><br>
> As per the plan, the db should be able to return a list of status entries /<br>
> validafter ranges (which can be used in {first,last}_seen_days) given some<br>
> fingerprint.<br>
<br>
</div>Oh, I think there's a misunderstanding of these two fields. These<br>
fields are only there to search for relays or bridges that have first<br>
appeared or were last seen on a given day.<br>
<br>
You'll need two new parameters, say, from=datetime and to=datetime (or<br>
start=datetime and end=datetime) to define a valid-after range for your<br>
search.<br></blockquote><div><br></div><div>Ah! I wasn't paying attention here. :) Ok, all good.</div><div> </div><div>Thanks as always!</div><div>Regards</div><div>Kostas.</div><div><br></div></div></div></div>