[tor-project] [TSoP] stem.client - status report #5

[Dave Rolek]

Hi everyone!

This is my fifth status report for the stem.client TSoP project [1].
You may see my previous reports on the tor-project ml. [2][3][4][5]

This report is a few days late.


# Updates

It's been roughly a week and a half since my last update.

I've focused on some loose ends over this time. I've found it's bad if I
let things sit in a partial state too long, so I wanted to push a few
things forward.

## stem.client

Nothing directly on the stem.client codebase. Oi.

## tor-spec

Having been pretty deep in the spec, I've been noting a number of things
to address.

When tor-spec was being discussed on #tor-dev, I contributed some to the
conversation and resultant tickets.

The discussion spawned the following tickets, which I filed:
* #26859 - improve contextual description of EXTEND->CREATE /
           CREATED->EXTENDED handling and payloads [6]
* #26860 - decryption order appears to be wrongly specified [7]

I also took #26228, which I previously filed, and split it into smaller
parts, to move forward:
* #26228 - Clarify/determine specification for padding bytes, (formerly
           also PADDING cell) [8]
* #26870 - clarify inconsistency for [V]PADDING/DROP cell content vs.
           padding bytes [9]

In a bit of relation, another part discussed in #26228 - randomized
padding - made it into a ticket thanks to teor and prop#289 review:
* #26871 - prop289: randomize the unused part of relay payloads [10]

I submitted a PR for #26860 and another for #26228.

#26859 and #26860 were addressed together; now merged. [11]

#26228, #26870, and #26871 are also being addressed together, the others
mostly by teor; current status is a PR. [12]

## stem, general

I filed/implemented a few tox-related tickets to make
development/testing easier:
* #26811 - tox fails with pip 10 [13]
* #26916 - Make tox config more useful/friendly for running multiple
           interpreters/versions [14]
* #26822 - Investigate relying on tox's default install capabilities
         | instead of an explicit command [15]
         +- better tox stuff of #26811, filed only

## tor, general

I ran into a few tor bugs during the past few weeks:
* #26709 - Onion V3 addresses not always working [16]
         +- I reproduced this (or something similar) and have a log that
            I will post this week
* #26882 - ~one case of IP address not scrubbed in logs~ [17]
         +- filed (noticed in scrubbing a log manually)

I ran into an exit that seems to be undergoing general censorship in the
area (tpo unreachable). Will be emailing bad-relays at torproject.org with
the details. (Thanks teor!)

# other bookkeeping

A few other general cleanup things:
* closed out #26197 - GitHub mirroring for stem [18]
* closed out stem's GitHub PRs #1-#4 [19]
* various other Trac triage/bookkeeping, e.g. ...
  - making the current status of #24321 clear in the ticket [20]
  - closing out #26852 [21]
* (probably other stuff)


# What's next

There's a bit more cleanup I want to do, but I recognized this update
has been proportionally much more of that than I had intended.

So next I plan to focus as much as possible on stem.client.

## stem.client

1. refactor: Relay cell encryption/decryption/processing
2. centralize ORPort reads/sends (demux/mux) at connection level
3. implement required RelayCell subclasses (e.g. parsing of decrypted
   body)

## tor-spec

File tickets to track spec improvements resulting from TSoP experience.
(These are bookkeeping / awareness tickets for my WIP changes.)

## tor, general

Post log for #26709. [15]
Email about potentially censored exit.

## other

File a GitHub mirroring ticket for nyx.
A bit of TSoP administrativia.


# Other Tor things

Multiple in-person chats. Seems to be a decently consistent interest in
Tor. :)


# Closing

I'm proactively reading a *bit* of IRC scrollback again now, but please
still mention my nick (dmr) if you want to get my attention!

As with before, please don't hesitate to reach out to me via IRC or
email!


Thanks,
Dave


[1]
https://lists.torproject.org/pipermail/tor-dev/2018-April/013090.html
[2]
https://lists.torproject.org/pipermail/tor-project/2018-May/001811.html
[3]
https://lists.torproject.org/pipermail/tor-project/2018-June/001830.html
[4]
https://lists.torproject.org/pipermail/tor-project/2018-June/001858.html
[5]
https://lists.torproject.org/pipermail/tor-project/2018-July/001886.html
[6] https://trac.torproject.org/projects/tor/ticket/26859
[7] https://trac.torproject.org/projects/tor/ticket/26860
[8] https://trac.torproject.org/projects/tor/ticket/26228
[9] https://trac.torproject.org/projects/tor/ticket/26870
[10] https://trac.torproject.org/projects/tor/ticket/26871
[11]
https://gitweb.torproject.org/torspec.git/log/tor-spec.txt?id=b592584b6ae3365158d610288cad8324e41b19f8
[12] https://github.com/torproject/torspec/pull/28
[13] https://trac.torproject.org/projects/tor/ticket/26811
[14] https://trac.torproject.org/projects/tor/ticket/26916
[15] https://trac.torproject.org/projects/tor/ticket/26822
[16] https://trac.torproject.org/projects/tor/ticket/26709
[17] https://trac.torproject.org/projects/tor/ticket/26882
[18] https://trac.torproject.org/projects/tor/ticket/26197
[19] https://github.com/torproject/stem/pulls?q=is%3Aclosed
[20] https://trac.torproject.org/projects/tor/ticket/24321
[21] https://trac.torproject.org/projects/tor/ticket/26852