<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><br><div>On 24 Dec 2017, at 08:16, Damian Johnson <<a href="mailto:atagar@torproject.org">atagar@torproject.org</a>> wrote:<br><br></div><blockquote type="cite"><div><span>Hi Tim, added preliminary Stem parsing support for the v2 fallback format...</span><br><span></span><br><span> <a href="https://gitweb.torproject.org/stem.git/commit/?id=ea55eaa">https://gitweb.torproject.org/stem.git/commit/?id=ea55eaa</a></span><br><span></span><br><span>Few thoughts are...</span><br><span></span><br><span> * It would be nice if the 'extrainfo=' lines were mandatory since I</span><br><span>need a delimiter between the entries.</span><br></div></blockquote><div><br></div>We have a few alternatives here:<br><div><br></div><div>C requires a comma as the delimiter between entries.</div><div>I can guarantee there will always be a comma delimiter after every entry,</div><div>Including the last entry. I will also make sure that any comment fields</div><div>come before this delimiter. I can guarantee there will never be a comma</div><div>inside the C string or comments in the entry.</div><div><br></div><div>Alternately, I can make extrainfo mandatory, and if I can't fetch a relay's</div><div>descriptor (a rare case?), I will mark is as 0.</div><div><br></div><div>But I'd like to be able to add extra fields in future without breaking</div><div>parsers, so I don't want parsers relying on field order.</div><div><br></div><div>Do you want me to add an explicit end of record comment, or is the</div><div>comma sufficient?</div><div><br></div><div><span style="background-color: rgba(255, 255, 255, 0);">Is there a delimiter you'd like me to add before the first entry?</span></div><br><blockquote type="cite"><div><span> * Maybe we should start the document with a format version number?</span><br></div></blockquote><div><br></div><div>I thought of that, too. Thanks for reminding me.</div><div><br></div><div><a href="https://trac.torproject.org/projects/tor/ticket/24725#ticket">https://trac.torproject.org/projects/tor/ticket/24725</a></div><br><blockquote type="cite"><div><span>Cheers! -Damian</span><br><span></span><br><span></span><br><span></span><br><span>On Fri, Dec 22, 2017 at 6:53 AM, teor <<a href="mailto:teor2345@gmail.com">teor2345@gmail.com</a>> wrote:</span><br><blockquote type="cite"><span>Hi all,</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>Some time in the next few weeks, the Tor fallback directory mirror file</span><br></blockquote><blockquote type="cite"><span>format will change. This affects stem and Relay Search, which parse this</span><br></blockquote><blockquote type="cite"><span>file.</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>Change Description</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>Here is a list of changes to the file format:</span><br></blockquote><blockquote type="cite"><span>* the "weight" line has been removed, and replaced with a Tor config</span><br></blockquote><blockquote type="cite"><span> default (#24679, #24681)</span><br></blockquote><blockquote type="cite"><span>* the comma that separates fallback C strings is now on its own line</span><br></blockquote><blockquote type="cite"><span>* a "nickname" comment has been added (#24600)</span><br></blockquote><blockquote type="cite"><span>* an optional "extrainfo" comment has been added (#22759)</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>The added fields will be populated with placeholders until the list is</span><br></blockquote><blockquote type="cite"><span>rebuilt (#22271). This will hopefully happen some time in the next few</span><br></blockquote><blockquote type="cite"><span>weeks.</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>Requesting More Extra Info Caches</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>There are only a few fallbacks that cache extra-info documents.</span><br></blockquote><blockquote type="cite"><span>I checked 67, and only 4 cached extra-info documents.</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>atagar, do you want me to ask some fallback operators to set</span><br></blockquote><blockquote type="cite"><span>DownloadExtraInfo 1?</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>What number or proportion would you like?</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>(We allow approximately 25% of fallbacks to go down before we start to rebuild</span><br></blockquote><blockquote type="cite"><span>the list. In the worst case, this can mean that ~40% are down at some point.)</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>Example Entries</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>A sample entry in the new format, using actual relay info:</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>"5.9.110.236:9030 orport=9001 id=0756B7CD4DFC8182BE23143FAC0642F515182CEB"</span><br></blockquote><blockquote type="cite"><span>" ipv6=[2a01:4f8:162:51e2::2]:9001"</span><br></blockquote><blockquote type="cite"><span>/* nickname=rueckgrat */</span><br></blockquote><blockquote type="cite"><span>/* extrainfo=1 */</span><br></blockquote><blockquote type="cite"><span>,</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>The current fallback file in the new format, with placeholders:</span><br></blockquote><blockquote type="cite"><span><a href="https://github.com/teor2345/tor/blob/ticket22759_tree/src/or/fallback_dirs.inc">https://github.com/teor2345/tor/blob/ticket22759_tree/src/or/fallback_dirs.inc</a></span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>A small sample fallback file in the new format, with actual relay info:</span><br></blockquote><blockquote type="cite"><span><a href="https://trac.torproject.org/projects/tor/attachment/ticket/22759/fallback_dirs_new_format.inc">https://trac.torproject.org/projects/tor/attachment/ticket/22759/fallback_dirs_new_format.inc</a></span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>Please let me know if you would like any changes to the format.</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>T</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>--</span><br></blockquote><blockquote type="cite"><span>Tim Wilson-Brown (teor)</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>teor2345 at gmail dot com</span><br></blockquote><blockquote type="cite"><span>PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B</span><br></blockquote><blockquote type="cite"><span>ricochet:ekmygaiu4rzgsk6n</span><br></blockquote><blockquote type="cite"><span>xmpp: teor at torproject dot org</span><br></blockquote><blockquote type="cite"><span>------------------------------------------------------------------------</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><span>_______________________________________________</span><br><span>tor-dev mailing list</span><br><span><a href="mailto:tor-dev@lists.torproject.org">tor-dev@lists.torproject.org</a></span><br><span><a href="https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev">https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev</a></span><br></div></blockquote></body></html>