[ooni-dev] ooni-pipeline using local time zone when parsing timestamps

[David Fifield]

On Thu, Mar 17, 2016 at 08:37:21PM +0100, Arturo Filastò wrote:
> 
> > On Mar 17, 2016, at 19:40, David Fifield <david at bamsoftware.com> wrote:
> > 
> > I don't understand your explanation that "utcfromtimestamp() leads it to
> > actually being converted back into the local timezone." That's what I
> > would expect of fromtimestamp, not utcfromtimestamp. Consider these:
> > 
> > # fromtimestamp gives different values according to TZ:
> > $ TZ=America/Denver python -c 'import datetime; print str(datetime.datetime.fromtimestamp(1458239719))'
> > 2016-03-17 12:35:19
> > $ TZ=Europe/Rome    python -c 'import datetime; print str(datetime.datetime.fromtimestamp(1458239719))'
> > 2016-03-17 19:35:19
> > $ TZ=UTC            python -c 'import datetime; print str(datetime.datetime.fromtimestamp(1458239719))'
> > 2016-03-17 18:35:19
> > 
> > # utcfromtimestamp gives the same value regardless of TZ:
> > $ TZ=America/Denver python -c 'import datetime; print str(datetime.datetime.utcfromtimestamp(1458239719))'
> > 2016-03-17 18:35:19
> > $ TZ=Europe/Rome    python -c 'import datetime; print str(datetime.datetime.utcfromtimestamp(1458239719))'
> > 2016-03-17 18:35:19
> > $ TZ=UTC            python -c 'import datetime; print str(datetime.datetime.utcfromtimestamp(1458239719))'
> > 2016-03-17 18:35:19
> > 
> 
> This is the problem, since you are setting the TZ to be Denver, Rome and then UTC, you would expect that
> the output of the datetime call would differ, since when you cast it to string it should evaluate to the time
> in the timezone of the caller.
> 
> You can better understand what I mean with this if you try to cast it to seconds since epoch again, as we
> used to do in ooni-probe < 1.4:
> 
> $ TZ=America/Denver python -c 'import datetime; print datetime.datetime.utcfromtimestamp(1458239719).strftime("%s”)’
> 1458261319
> 
> $ TZ=Europe/Rome python -c 'import datetime; print datetime.datetime.utcfromtimestamp(1458239719).strftime("%s")’
> 1458236119
> 
> $ TZ=UTC python -c 'import datetime; print datetime.datetime.utcfromtimestamp(1458239719).strftime("%s")’
> 1458239719

I have always had trouble with the Python date libraries. So I trust you
if you say it is working :)