[tor-dev] Revisiting prop224 time periods and HS descriptor upload/downloads

Wed Apr 13 12:34:54 UTC 2016

David Goulet <dgoulet at ev0ke.net> writes:

> [ text/plain ]
> On 12 Apr (16:01:32), George Kadianakis wrote:
>> David Goulet <dgoulet at ev0ke.net> writes:
>> 
>> > [ text/plain ]
>> > On 11 Apr (14:42:02), George Kadianakis wrote:
>> >> David Goulet <dgoulet at ev0ke.net> writes:
>> >> 
>> >> > [ text/plain ]
>> >> > On 04 Apr (19:13:39), George Kadianakis wrote:
>> >> >> Hello,
>> >> >> 
>> >> >> during March we discussed the cell formats of prop224:
>> >> >>   https://lists.torproject.org/pipermail/tor-dev/2016-March/010534.html
>> >> >> 
>> >> >> The prop224 topic for this month has to do with the way descriptors get
>> >> >> uploaded and downloaded, how this is scheduled using time periods and how the
>> >> >> shared randomness subsystem interacts with all that.
>> >> >> 
>> >> >> <snip>
>> >> >> 
>> >
>> > The logic I sketched out above makes it that we would need parameters (from
>> > the consensus) like so (or hardcode them):
>> >
>> > - TIME_PERIOD_ROTATION_TIME (currently 12:00)
>> >
>> 
>> [Second email with some more thoughts]
>> 
>> BTW, currently in prop224 the TIME_PERIOD_ROTATION_TIME is at 00:00 because of
>> the following paragraph:
>> 
>>    Time periods start with the Unix epoch (Jan 1, 1970), and are
>>    computed by taking the number of whole minutes since the epoch and
>>    dividing by the time period. So if the current time is 2013-11-12
>>    13:44:32 UTC, making the seconds since the epoch 1384281872, the
>>    number of minutes since the epoch is 23071364.  If the current time
>>    period length is 1500 (the default), then the current time period
>>    number is 15380. It began 15380*1500*60 seconds after the epoch at
>>    2013-11-11 20:00:00 UTC, and will end at (15380+1)*1500*60 seconds
>>    after the epoch at 2013-11-12 21:00:00 UTC.
>> 
>> I wonder what's the best way to change this to start at 12:00. 
>> 
>> We could in theory compute the "number of whole minutes since the epoch plus 12
>> hours" and use that in the division, but that would be a bit ugly... Is there a
>> more elegant thing to do?
>> 
>> We could also in theory change the shared random value generation to happen at
>> 12:00, and then have TIME_PERIOD_ROTATION_TIME naturally start at 00:00, but
>> this requires changing prop250. Could it be worth it? :/
>
> I think it's fine we keep the start time at 12:00 here. It's just an offset
> from the start of the epoch. Furthermore, adding a "rotation time" makes it
> that we we can control where everything started which doesn't have to be the
> epoch time at 00:00.
>
> We can find the start of the TP with those two (rotation time and lifetime)
> and then divide that time value by the lifetime to get the nth time period.
>
> Also, controling the rotation time is good to have for chutney testing with
> much more smaller timings.
>

OK, I posted a torspec branch with some initial changes based on the
discussions of this thread at 'prop224-timeperiods-1':
            https://gitweb.torproject.org/user/asn/torspec.git/log/?h=prop224-timeperiods-1

Specifically, wrt time periods and the start time, I introduced a "rotation
time offset" of 12 hours to the epoch calculation. You can see it here:
     https://gitweb.torproject.org/user/asn/torspec.git/commit/?h=prop224-timeperiods-1&id=6499cbf1023806f75d567503ad0ab8625a3876a7

Please let me know if the technique can be simplified or improved or needs better wording.
We need this mechanic to be clear and easy to understand and implement!

>> 
>> <snip>
>> 
>> 
>> Currently the proposal says:
>> 
>>    Hidden service directories should accept descriptors [...] and retain them
>>    for at least [TODO: how much?] minutes after the end of the period.
>> 
>> but that means that HSDirs need to keep track of when the period ends, and
>> whether a descriptor was uploaded for the current time period or for the
>> overlay period...
>
> I think this will make things much more complicated. IMO, the HSDir should
> _only_ rely on the revision counter and an expiry time and not trying to try
> and guess the lifetime of a descriptor from the service perspective.
>
> However, here is an idea. Considering teor's argument about HS fingerprinting,
> we should make the upload happen regurlarly so having RendPostPeriod
> customizable by an operator is probably a bad idea. We kind of need that _all_
> HS expect to behave the same in normal circumstances with their HS desc
> uploads. Thus, if the RendPostPeriod would become hardcoded (or consensus
> params), the HSDir would know that the HS is "gone" or have rotated HSDirs
> after let's say 4 * RendPostPeriod (4 here is abritrary) of not seeing a new
> revision counter. This makes a cache entry lifetime much smaller!
>
> In the end, I see either we use an expiry time that is the _maximum_ lifetime
> a descriptor can have or an expiry time that is based on the expected maximum
> time frame we should have received a new revision counter. The latter is fun
> because we don't need to consider client clock skew since if we happen to
> purge the descriptor from the cache it's because we think the service is gone
> or has rotated HSDir thus any client coming our way would fail in the first
> place to connect to the service by getting the descriptor that should have
> been updated X hours ago.
>

Here is another important commit that specifies the overlap period functionality:
    https://gitweb.torproject.org/user/asn/torspec.git/commit/?h=prop224-timeperiods-1&id=737b1f2bcccce1d632e43ba6e395f24f0af8a65c

It also adds the following section:

+   HSDirs MUST retain hidden service descriptors for 33 hours before expiring
+   them. That's 24 hours for the time period duration, plus 6 hours for the
+   maximum overlap period span, plus 3 hours for the maximum acceptable client
+   clock skew.

+   Hidden services should keep their old introduction circuits open for at
+   least 3 hours after descriptor expiration, so that clients with skewed
+   clocks can still visit them through outdated descriptors.

This implements the naive cache lifetime mechanic we discussed in this
thread. That's an improvement over the empty TODO section of the current
prop224 but maybe we can do better: we should think whether we want to do more
advanced HSDir heuristics like "If I'm an HSDir and I don't receive an HS
descriptor for N hours, consider that HS dead". Or maybe we should add
valid-until fields to hidden service descriptors. Thoughts?

Here are some things left to be done:

- Specify *when* hidden services upload descriptors. Do they do it hourly, or
  only when a change has happened. Both approaches leak information to the
  HSDir (the former leaks uptime, the latter leaks intro point changes).

- Specify behavior of hidden services and clients with regards to time periods
  and the use of SRVs as discussed in
    https://lists.torproject.org/pipermail/tor-dev/2016-April/010757.html

- Further specify descriptor caching behavior of HSDirs.

Did I forget anything?