Proposal 342: Decouple hs_interval and SRV lifetime
``` Filename: 342-decouple-hs-interval.md Title: Decoupling hs_interval and SRV lifetime Author: Nick Mathewson Created: 9 January 2023 Status: Draft ``` # Motivation and introduction Tor uses shared random values (SRVs) in the consensus to determine positions of relays within a hash ring. Which shared random value is to be used for a given time period depends upon the time at which that shared random value became valid. But right now, the consensus voting period is closely tied to the shared random value voting cycle: and clients need to understand both of these in order to determine when a shared random value became current. This creates tight coupling between: * The voting schedule * The SRV liveness schedule * The hsdir_interval parameter that determines the length of the an HSDIR index To decouple these values, this proposal describes a forward compatible change to how Tor reports SRVs in consensuses, and how Tor decides which hash ring to use when. ## Reporting SRV timestamps In consensus documents, parties should begin to accept `shared-rand-*-value` lines with an additional argument, in the format of an IsoTimeNospace timestamp (like "1985-10-26T00:00:00"). When present, this timestamp indicates the time at which the given shared random value first became the "current" SRV. Additionally, we define a new consensus method that adds these timestamps to the consensus. We specify that, in the absence of such a timestamp, parties are to assume that the `shared-rand-current-value` SRV became "current" at the first 00:00 UTC on the UTC day of the consensus's valid-after timestamp, and that the `shard-rand-previous-value` SRV became "current" at 00:00 UTC on the previous UTC day. ## Generalizing HSDir index scheduling. Under the current HSDir design, there is one SRV for each time period, and one time period for which each SRV is in use. Decoupling `hsdir_interval` from 24 hours will require that we change this notion slightly. We therefore propose this set of generalized directory behavior rules, which should be equivalent to the current rules under current parameters. The calculation of time periods remains the same (see `rend-spec-v3.txt` section `[TIME PERIODS]`). A single SRV is associated with each time period: specifically, the SRV that was "current" at the start of the time period. There is a separate hash ring associated with each time period and its SRV. Whenever fetching an onion service descriptor, the client uses the hash ring for the time period that contains the start of the liveness interval of the current consensus. Call this the "Consensus" time period. Whenever uploading an onion service descriptor, the service uses _two or three_ hash rings: * The "consensus" time period (see above). * The immediately preceding time period, if the SRV to calculate that hash ring is available in the consensus. * The immediately following time period, if the SRV to calculate that hash ring is available in the consensus. (Under the current parameters, where `hsdir_interval = SRV_interval`, there will never be more than two possible time periods for which the service can qualify.) ## Migration We declare that, for at least the lifetime of the C tor client, we will not make any changes to the voting interval, the SRV interval, or the `hsdir_interval`. As such, we do not need to prioritize implementing these changes in the C client: we can make them in Arti only. ## Issues left unsolved There are likely other lingering issues that would come up if we try to change the voting interval. This proposal does not attempt to solve them. This proposal does not attempt to add flexibility to the SRV voting algorithm itself. Changing `hsdir_interval` would create a flag day where everybody using old and new values of `hsdir_interval` would get different hash rings. We do not try to solve that here. ## Acknowledgments Thanks to David Goulet for explaining all of this stuff to me!
Apologies if I missed this in the proposal, but what are the benefits of this beyond decoupling? Will onion service users see benefits? Or is it just about simplifying the code? On Tue, Jan 10, 2023, 8:22 AM Nick Mathewson <nickm@torproject.org> wrote:
``` Filename: 342-decouple-hs-interval.md Title: Decoupling hs_interval and SRV lifetime Author: Nick Mathewson Created: 9 January 2023 Status: Draft ```
# Motivation and introduction
Tor uses shared random values (SRVs) in the consensus to determine positions of relays within a hash ring. Which shared random value is to be used for a given time period depends upon the time at which that shared random value became valid.
But right now, the consensus voting period is closely tied to the shared random value voting cycle: and clients need to understand both of these in order to determine when a shared random value became current.
This creates tight coupling between: * The voting schedule * The SRV liveness schedule * The hsdir_interval parameter that determines the length of the an HSDIR index
To decouple these values, this proposal describes a forward compatible change to how Tor reports SRVs in consensuses, and how Tor decides which hash ring to use when.
## Reporting SRV timestamps
In consensus documents, parties should begin to accept `shared-rand-*-value` lines with an additional argument, in the format of an IsoTimeNospace timestamp (like "1985-10-26T00:00:00"). When present, this timestamp indicates the time at which the given shared random value first became the "current" SRV.
Additionally, we define a new consensus method that adds these timestamps to the consensus.
We specify that, in the absence of such a timestamp, parties are to assume that the `shared-rand-current-value` SRV became "current" at the first 00:00 UTC on the UTC day of the consensus's valid-after timestamp, and that the `shard-rand-previous-value` SRV became "current" at 00:00 UTC on the previous UTC day.
## Generalizing HSDir index scheduling.
Under the current HSDir design, there is one SRV for each time period, and one time period for which each SRV is in use. Decoupling `hsdir_interval` from 24 hours will require that we change this notion slightly.
We therefore propose this set of generalized directory behavior rules, which should be equivalent to the current rules under current parameters.
The calculation of time periods remains the same (see `rend-spec-v3.txt` section `[TIME PERIODS]`).
A single SRV is associated with each time period: specifically, the SRV that was "current" at the start of the time period.
There is a separate hash ring associated with each time period and its SRV.
Whenever fetching an onion service descriptor, the client uses the hash ring for the time period that contains the start of the liveness interval of the current consensus. Call this the "Consensus" time period.
Whenever uploading an onion service descriptor, the service uses _two or three_ hash rings: * The "consensus" time period (see above). * The immediately preceding time period, if the SRV to calculate that hash ring is available in the consensus. * The immediately following time period, if the SRV to calculate that hash ring is available in the consensus.
(Under the current parameters, where `hsdir_interval = SRV_interval`, there will never be more than two possible time periods for which the service can qualify.)
## Migration
We declare that, for at least the lifetime of the C tor client, we will not make any changes to the voting interval, the SRV interval, or the `hsdir_interval`. As such, we do not need to prioritize implementing these changes in the C client: we can make them in Arti only.
## Issues left unsolved
There are likely other lingering issues that would come up if we try to change the voting interval. This proposal does not attempt to solve them.
This proposal does not attempt to add flexibility to the SRV voting algorithm itself.
Changing `hsdir_interval` would create a flag day where everybody using old and new values of `hsdir_interval` would get different hash rings. We do not try to solve that here.
## Acknowledgments
Thanks to David Goulet for explaining all of this stuff to me! _______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
On Tue, Jan 10, 2023 at 8:41 AM Holmes Wilson <h@zbay.llc> wrote:
Apologies if I missed this in the proposal, but what are the benefits of this beyond decoupling? Will onion service users see benefits? Or is it just about simplifying the code?
There's no immediate benefit to users; it's entirely about making the code more well-defined in the case where hs_interval is constant. I'm attempting to do this now since we're starting to write onion services in Arti, and it'll be easier if there are fewer ambiguities in the spec. cheers, -- Nick
Understood and thanks! On Tue, Jan 10, 2023, 11:24 AM Nick Mathewson <nickm@torproject.org> wrote:
On Tue, Jan 10, 2023 at 8:41 AM Holmes Wilson <h@zbay.llc> wrote:
Apologies if I missed this in the proposal, but what are the benefits of this beyond decoupling? Will onion service users see benefits? Or is it just about simplifying the code?
There's no immediate benefit to users; it's entirely about making the code more well-defined in the case where hs_interval is constant.
I'm attempting to do this now since we're starting to write onion services in Arti, and it'll be easier if there are fewer ambiguities in the spec.
cheers, -- Nick _______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
On Tue, Jan 10, 2023 at 8:22 AM Nick Mathewson <nickm@torproject.org> wrote:
``` Filename: 342-decouple-hs-interval.md Title: Decoupling hs_interval and SRV lifetime Author: Nick Mathewson Created: 9 January 2023 Status: Draft ```
# Motivation and introduction
I think there's another issue to address here too: the offset from the Unix Epoch at which the first Time Period begins. According to rend-spec-v3, "we want our time periods to start at 12:00UTC every day, so we subtract a "rotation time offset" of 12*60 minutes from the number of minutes since the epoch, before dividing by the time period (effectively making "our" epoch start at Jan 1, 1970 12:00UTC)." But this isn't exactly what the C Tor implementation does. In `hs_get_time_period_num(),` it defines the offset as `sr_state_get_phase_duration()`, which is tied to the voting interval and the constant SHARED_RANDOM_N_ROUNDS (which is 12). David, do you have any thoughts on the right solution here? Some options are: * We could document the current behavior. * We could add a consensus parameter for the time period offset. * We could define the time period offset as exactly 12 hours in all cases. (I guess this would break test networks though?) * Something else? best wishes, -- Nick
On 23 Jan (13:31:49), Nick Mathewson wrote:
On Tue, Jan 10, 2023 at 8:22 AM Nick Mathewson <nickm@torproject.org> wrote:
``` Filename: 342-decouple-hs-interval.md Title: Decoupling hs_interval and SRV lifetime Author: Nick Mathewson Created: 9 January 2023 Status: Draft ```
# Motivation and introduction
I think there's another issue to address here too: the offset from the Unix Epoch at which the first Time Period begins. According to rend-spec-v3,
"we want our time periods to start at 12:00UTC every day, so we subtract a "rotation time offset" of 12*60 minutes from the number of minutes since the epoch, before dividing by the time period (effectively making "our" epoch start at Jan 1, 1970 12:00UTC)."
But this isn't exactly what the C Tor implementation does. In `hs_get_time_period_num(),` it defines the offset as `sr_state_get_phase_duration()`, which is tied to the voting interval and the constant SHARED_RANDOM_N_ROUNDS (which is 12).
David, do you have any thoughts on the right solution here? Some options are: * We could document the current behavior. * We could add a consensus parameter for the time period offset. * We could define the time period offset as exactly 12 hours in all cases. (I guess this would break test networks though?) * Something else?
My intuition here would be to document current behavior. This shared random ritual is tied to the voting protocol (interval) because it has these commit + reveal phases thus using the voting interval between phase rounds is foundational. And so, I would keep those two tied and simply document. What we can think of is to add consensus parameters for how many rounds per phase instead of these hardcoded values in our C-tor code but unclear to me what it would give us in the long run. But for the interval, I would keep the voting one. We get TestingNetwork for free also with this. Whatever we do, the very important piece here is that we can't end up with a protocol taking more time than our MinUptimeHidServDirectoryV2 value (minimum uptime for a relay before becoming an HSDir). And so let say one day we change the voting interval to every 4 hours then we end up with 24 rounds of voting to complete the commit + reveal phases meaning a total of 96 hours (which is current value of MinUptimeHidServDirectoryV2) thus borderline no good. (Maybe something to keep in mind for arti-relay authorities to check on...). Hope this help with your question? Cheers! David -- pG2H247H2MnipNmet9NVk2ZChKK1OeFLQ50ZwK2ROe4=
On Tue, Jan 24, 2023 at 9:02 AM David Goulet <dgoulet@torproject.org> wrote:
On 23 Jan (13:31:49), Nick Mathewson wrote:
On Tue, Jan 10, 2023 at 8:22 AM Nick Mathewson <nickm@torproject.org> wrote:
``` Filename: 342-decouple-hs-interval.md Title: Decoupling hs_interval and SRV lifetime Author: Nick Mathewson Created: 9 January 2023 Status: Draft ```
# Motivation and introduction
I think there's another issue to address here too: the offset from the Unix Epoch at which the first Time Period begins. According to rend-spec-v3,
"we want our time periods to start at 12:00UTC every day, so we subtract a "rotation time offset" of 12*60 minutes from the number of minutes since the epoch, before dividing by the time period (effectively making "our" epoch start at Jan 1, 1970 12:00UTC)."
But this isn't exactly what the C Tor implementation does. In `hs_get_time_period_num(),` it defines the offset as `sr_state_get_phase_duration()`, which is tied to the voting interval and the constant SHARED_RANDOM_N_ROUNDS (which is 12).
David, do you have any thoughts on the right solution here? Some options are: * We could document the current behavior. * We could add a consensus parameter for the time period offset. * We could define the time period offset as exactly 12 hours in all cases. (I guess this would break test networks though?) * Something else?
My intuition here would be to document current behavior. This shared random ritual is tied to the voting protocol (interval) because it has these commit + reveal phases thus using the voting interval between phase rounds is foundational.
And so, I would keep those two tied and simply document.
What we can think of is to add consensus parameters for how many rounds per phase instead of these hardcoded values in our C-tor code but unclear to me what it would give us in the long run. But for the interval, I would keep the voting one. We get TestingNetwork for free also with this.
Whatever we do, the very important piece here is that we can't end up with a protocol taking more time than our MinUptimeHidServDirectoryV2 value (minimum uptime for a relay before becoming an HSDir).
And so let say one day we change the voting interval to every 4 hours then we end up with 24 rounds of voting to complete the commit + reveal phases meaning a total of 96 hours (which is current value of MinUptimeHidServDirectoryV2) thus borderline no good.
(Maybe something to keep in mind for arti-relay authorities to check on...).
Hope this help with your question?
Sure! It sounds to me that we should then change rend-spec to say something like this: "We want our time periods to start at a regular offset from the SRV voting schedule, so we subtract a "rotation time offset" of 12 voting periods from the number of minutes since the epoch, before dividing by the time period (effectively making "our" epoch start at Jan 1, 1970 12:00UTC when the voting period is 1 hour.)" How would that be? -- Nick
On 24 Jan (09:05:18), Nick Mathewson wrote:
On Tue, Jan 24, 2023 at 9:02 AM David Goulet <dgoulet@torproject.org> wrote:
On 23 Jan (13:31:49), Nick Mathewson wrote:
On Tue, Jan 10, 2023 at 8:22 AM Nick Mathewson <nickm@torproject.org> wrote:
``` Filename: 342-decouple-hs-interval.md Title: Decoupling hs_interval and SRV lifetime Author: Nick Mathewson Created: 9 January 2023 Status: Draft ```
# Motivation and introduction
I think there's another issue to address here too: the offset from the Unix Epoch at which the first Time Period begins. According to rend-spec-v3,
"we want our time periods to start at 12:00UTC every day, so we subtract a "rotation time offset" of 12*60 minutes from the number of minutes since the epoch, before dividing by the time period (effectively making "our" epoch start at Jan 1, 1970 12:00UTC)."
But this isn't exactly what the C Tor implementation does. In `hs_get_time_period_num(),` it defines the offset as `sr_state_get_phase_duration()`, which is tied to the voting interval and the constant SHARED_RANDOM_N_ROUNDS (which is 12).
David, do you have any thoughts on the right solution here? Some options are: * We could document the current behavior. * We could add a consensus parameter for the time period offset. * We could define the time period offset as exactly 12 hours in all cases. (I guess this would break test networks though?) * Something else?
My intuition here would be to document current behavior. This shared random ritual is tied to the voting protocol (interval) because it has these commit + reveal phases thus using the voting interval between phase rounds is foundational.
And so, I would keep those two tied and simply document.
What we can think of is to add consensus parameters for how many rounds per phase instead of these hardcoded values in our C-tor code but unclear to me what it would give us in the long run. But for the interval, I would keep the voting one. We get TestingNetwork for free also with this.
Whatever we do, the very important piece here is that we can't end up with a protocol taking more time than our MinUptimeHidServDirectoryV2 value (minimum uptime for a relay before becoming an HSDir).
And so let say one day we change the voting interval to every 4 hours then we end up with 24 rounds of voting to complete the commit + reveal phases meaning a total of 96 hours (which is current value of MinUptimeHidServDirectoryV2) thus borderline no good.
(Maybe something to keep in mind for arti-relay authorities to check on...).
Hope this help with your question?
Sure! It sounds to me that we should then change rend-spec to say something like this:
"We want our time periods to start at a regular offset from the SRV voting schedule, so we subtract a "rotation time offset" of 12 voting periods from the number of minutes since the epoch, before dividing by the time period (effectively making "our" epoch start at Jan 1, 1970 12:00UTC when the voting period is 1 hour.)"
Excellent. David -- pG2H247H2MnipNmet9NVk2ZChKK1OeFLQ50ZwK2ROe4=
participants (4)
-
David Goulet -
Holmes Wilson -
Nick Mathewson -
Nick Mathewson