Filename: ExtraRelayDescriptorFields.txt Title: Adding new X- fields to relay descriptor Author: Virgil Griffith, Nick Mathewson Created: 2014-06-03 Status: Open
1. Motivation We wish to allow developers to build new applications atop relays. Towards this end, we wish to add the ability for users to specify arbitrary new key-value entries under the "X-" namespace.
2. Proposal Allow optional key-value lines in the relay's torrc file. These lines will be mirrored in the relay's descriptor which is then published in the directory consensus.
For example: X-bitcoin 19mP9FKrXqL46Si58pHdhGKow88SUPy1V8 X-gravatar https://s.gravatar.com/avatar/d27fce46c9ac41a41bb52455ae75701d X-favoritequote Be excellent to each other. Party on dudes! X-foo bar
The value field must be printable ASCII (characters 32-126). The value must not under any condition contain a newline. The key may contain lowercase ASCII letters (a-z), digits, underscore, or dash. In regex, [-_0-9a-z].
There may need to be a maximum sum length of the X- entries. This is left to the developers. I propose a maximum sum length of 5 kilobytes.
To mitigate the chance of a malformed torrc file, I additionally propose there be a schema for the relay descriptor, and if the relay descriptor fails to match the schema it errors to the user to change her torrc file.
-V
This extension could be very useful for the Tor2web project, in order to easily introduce "directory" functionalities for join/leave/networking capabilities as per https://github.com/globaleaks/Tor2web-3.0/issues/41 .
That would be *amazing* .
I'm going to bring Italian Grappa with express-shipping as an alchoolic incentive to anyone that love to make it real! :-)
Just some spec-level nitpicking.
On Fri, Jul 04, 2014 at 10:22:59AM +0200, Virgil Griffith wrote:
The value field must be printable ASCII (characters 32-126).
Presumably all values are literal? There's no special meaning to ", , etc.?
The value must not under any condition contain a newline.
This is of course redundant to the above.
The key may contain
"may" contain? But it may not contain those stated chars, and may also contain other things?
lowercase ASCII letters (a-z), digits, underscore, or dash. In regex, [-_0-9a-z].
How many such characters must the key contain (after the "X-")? Is zero allowed, for example? A regex for the key would be helpful:
X-[-_0-9a-z]+
(if that's indeed what you meant). I'd probably remove the _, but that's just cosmetic: X-this_thing looks odder to me than X-this-thing.
There may need to be a maximum sum length of the X- entries. This is left to the developers. I propose a maximum sum length of 5 kilobytes.
Should there be an upper bound for individual keys, values, or key/value pairs?
- Ian
Ian Goldberg transcribed 1.2K bytes:
There may need to be a maximum sum length of the X- entries. This is left to the developers. I propose a maximum sum length of 5 kilobytes.
A couple questions:
0. Does this proposal include adding these additional `X-*` fields to the `@type bridge-server-descriptor`s as well? Or only to the non-bridge server-descriptors?
I ask because there have been times where BridgeDB appeared to be receiving around 17,000,000 bridge descriptors from Tonga (the BridgeAuth) per half hour. BridgeDB needs to parse all of these.
While the parsers can obviously skip any `X-*` lines, if, by some error somewhere in the network, BridgeDB were to start receiving seventeen million bridge descriptors again, except that now each descriptor contains your proposed maximum of 5KB extra text, that adds approximately 81GB every half hour. The previous case where the seventeen-million-bridge-descriptor bug happened nearly OOMed BridgeDB's machine continuously for two whole days, it ran out of disk space several times, and required constant babysitting from both me and sysrqb. If every bridge, or meerly a few hundred malicious bridges, were to send their `@type bridge-server-descriptor` every second with 5KB extra text, it would wreak havoc on several machines critical to the network's infrastructure.
Don't get me wrong, I like this proposal. I just don't want to say okay to something that'll wind up with me waking up to the Nagios instance screaming at me that BridgeDB's machine is down.
Which brings me to my next question:
1. Why 5KB?
Are we planning on allowing people to use Tor's consensus to pass encrypted emails around? I understand the desire to enable relays to be able to store *anything* in these fields, but this seems kind of crazy. Couldn't we specify the maximum length per line to something smaller? I also think that Ian's suggestion below to have an upper bound on the number of individual keys is a good idea.
Should there be an upper bound for individual keys, values, or key/value pairs?
Additionally, you might want to add something to the spec which states that duplicate keys aren't allowed, or that only the last such key will be parsed as valid, or they all get parsed, or however you want the parsers to behave.
- Does this proposal include adding these additional `X-*` fields to the `@type bridge-server-descriptor`s as well? Or only to the non-bridge server-descriptors?
This is for extrainfo descriptors, not server descriptors. Do bridges publish extrainfo descriptors?
- Why 5KB?
Are we planning on allowing people to use Tor's consensus to pass encrypted emails around? I understand the desire to enable relays to be able to store *anything* in these fields, but this seems kind of crazy. Couldn't we specify the maximum length per line to something smaller? I also think that Ian's suggestion below to have an upper bound on the number of individual keys is a good idea.
Should there be an upper bound for individual keys, values, or key/value pairs?
This is something Virgil and I disagree about. I wanted a maximum line length and count, while he likes the net 5kb threshold. I'll let him explain his reasoning.
Cheers! -Damian
Damian Johnson transcribed 1.1K bytes:
This is for extrainfo descriptors, not server descriptors. Do bridges publish extrainfo descriptors?
Oh. That wasn't clear to me from the proposal, but I must have simply missed it.
Yes, bridges do publish extrainfo descriptors. BridgeDB parses those as well, for now only to look for the `transport` lines.
- Why 5KB?
Are we planning on allowing people to use Tor's consensus to pass encrypted emails around?
Should there be an upper bound for individual keys, values, or key/value pairs?
This is something Virgil and I disagree about. I wanted a maximum line length and count, while he likes the net 5kb threshold. I'll let him explain his reasoning.
Encrypted emails... :)
Virgil, why does your below example have a bitcoin field? Perhaps you should rephrase your motivation more plainly to something like:
""" Hello, I'm a bitcoin capitalist profiteer and I wish to disturb the existing power structures within the Tor Project community with my capitalist bitcoin profiteering agenda. """
Sincerely,
David Stainton
On Fri, Jul 04, 2014 at 10:22:59AM +0200, Virgil Griffith wrote:
Filename: ExtraRelayDescriptorFields.txt Title: Adding new X- fields to relay descriptor Author: Virgil Griffith, Nick Mathewson Created: 2014-06-03 Status: Open
- Motivation
We wish to allow developers to build new applications atop relays. Towards this end, we wish to add the ability for users to specify arbitrary new key-value entries under the "X-" namespace.
- Proposal
Allow optional key-value lines in the relay's torrc file. These lines will be mirrored in the relay's descriptor which is then published in the directory consensus.
For example: X-bitcoin 19mP9FKrXqL46Si58pHdhGKow88SUPy1V8 X-gravatar https://s.gravatar.com/avatar/d27fce46c9ac41a41bb52455ae75701d X-favoritequote Be excellent to each other. Party on dudes! X-foo bar
The value field must be printable ASCII (characters 32-126). The value must not under any condition contain a newline. The key may contain lowercase ASCII letters (a-z), digits, underscore, or dash. In regex, [-_0-9a-z].
There may need to be a maximum sum length of the X- entries. This is left to the developers. I propose a maximum sum length of 5 kilobytes.
To mitigate the chance of a malformed torrc file, I additionally propose there be a schema for the relay descriptor, and if the relay descriptor fails to match the schema it errors to the user to change her torrc file.
-V
tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
-
Damian Johnson -
David Stainton -
Fabio Pietrosanti (naif) -
Ian Goldberg -
isis -
Virgil Griffith