[tor-bugs] #15522 [BridgeDB]: Write Protobufs for any BridgeDB data which must be sent over a network or IPC channel

[Tor Bug Tracker & Wiki]

#15522: Write Protobufs for any BridgeDB data which must be sent over a network or
IPC channel
-----------------------------+--------------------------------------------
     Reporter:  isis         |      Owner:  isis
         Type:  enhancement  |     Status:  new
     Priority:  normal       |  Milestone:
    Component:  BridgeDB     |    Version:
   Resolution:               |   Keywords:  bridgedb-db, metrics, protobuf
Actual Points:               |  Parent ID:
       Points:               |
-----------------------------+--------------------------------------------

Comment (by isis):

 Yawning mentioned on IRC that the designer of Protobufs quit Google and
 made a newer thing called [https://capnproto.org/ Cap'n Proto] (which is
 unfortunately [https://capnproto.org/install.html still beta] at this
 time), but claims to provide significant ("infinite percent", to use their
 words) speed increases, along with the following somewhat ridiculous but
 definitely-amusing-and-possibly-worth-looking-into claims:

 >
 > * '''Incremental reads''': It is easy to start processing a Cap’n Proto
 message before you have received all of it since outer objects appear
 entirely before inner objects (as opposed to most encodings, where outer
 objects encompass inner objects).
 > * '''Random access''': You can read just one field of a message without
 parsing the whole thing.
 > * '''mmap''': Read a large Cap’n Proto file by memory-mapping it. The OS
 won’t even read in the parts that you don’t access.
 > * '''Inter-language communication''': Calling C++ code from, say, Java
 or Python tends to be painful or slow. With Cap’n Proto, the two languages
 can easily operate on the same in-memory data structure.
 > * '''Inter-process communication''': Multiple processes running on the
 same machine can share a Cap’n Proto message via shared memory. No need to
 pipe data through the kernel. Calling another process can be just as fast
 and easy as calling another thread.
 > * '''Arena allocation''': Manipulating Protobuf objects tends to be
 bogged down by memory allocation, unless you are very careful about object
 reuse. Cap’n Proto objects are always allocated in an “arena” or “region”
 style, which is faster and promotes cache locality.
 > * '''Tiny generated code''': Protobuf generates dedicated parsing and
 serialization code for every message type, and this code tends to be
 enormous. Cap’n Proto generated code is smaller by an order of magnitude
 or more. In fact, usually it’s no more than some inline accessor methods!
 > * '''Tiny runtime library''': Due to the simplicity of the Cap’n Proto
 format, the runtime library can be much smaller.
 > * '''Time-traveling RPC''': Cap’n Proto features an RPC system that
 implements time travel such that call results are returned to the client
 before the request even arrives at the server!
 >

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/15522#comment:1>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online