[tor-bugs] #15844 [Onionoo]: Develop database schema to support Onionoo's search parameter efficiently

[Tor Bug Tracker & Wiki]

#15844: Develop database schema to support Onionoo's search parameter efficiently
-----------------------------+-----------------
     Reporter:  karsten      |      Owner:
         Type:  enhancement  |     Status:  new
     Priority:  normal       |  Milestone:
    Component:  Onionoo      |    Version:
   Resolution:               |   Keywords:
Actual Points:               |  Parent ID:
       Points:               |
-----------------------------+-----------------

Comment (by karsten):

 teor, leeroy, that's very helpful feedback!

 But before going into the details, let's briefly talk about some
 simplifications:

  1. The only reason we need `LIKE '%foo%'` is to support substring
 searches in nicknames.  But if it's too hard to implement this in a
 database search, then maybe we'll have to sacrifice this feature.  I could
 imagine that we're only supporting starts-with searches if this makes this
 easier to build and more likely to stay efficient over time.
  2. We might as well talk about giving up case-sensitive searches for
 base64-encoded fingerprints and just make them case-insensitive as
 everything else.  Again, let's not do this easily, but if it reduces
 complexity and improves performance, let's consider it.
  3. And let me repeat my suggestion from above to require search terms to
 be at least 3 characters long.  It's not the case right now, but I could
 imagine adding that requirement if it helps us.

 However, note that there's another parameter, `contact`, that I didn't
 mention before, because it's simpler than `search`.  But it depends even
 more on substring searches, so even if we'd simplify the `search`
 parameter, we would still have to support substring searches in the
 `contact` parameter for it to be useful.  Unless there are better ideas
 for supporting searches in contact lines?

 What do you think, how would these three simplifications affect our
 chances to get this implemented cleanly and in an efficient way?

 And what would you say if we'd not only try to support searches for the
 15k relays and bridges that have been running in the past week, but for
 the currently 600k relays and bridges that have existed in the past 10
 years and another 1.4M that will exist in the next years?

 Again, thanks so much for your input!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/15844#comment:6>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online