Hi,
I am running a relay and other servers. Sometimes doing SSHD over Tor via a hidden service in a VPS in openbsd.amsterdam. In all my other setups, in other providers, I do not see this problem happening.
Upon connecting for the first time, I do get a "banner line contains invalid characters" error, and the connection is terminated. (MSG1) Upon connecting for the second time, everything goes smooth. (MSG2)
Problem is happening on my 2 servers in openbsd.amsterdam.
Any directions into debugging this problem? Both via Tor or via SSHD?
Is it possible a bad guard is logging the IPs first connecting, then dropping the connection, and then allowing the connection to succeed later?
Any directions would be appreciated to solve the problem.
System: uname -a OpenBSD xxxx.yyyy.com 7.6 GENERIC#501 amd64 Tor version: Tor version 0.4.8.13.
MSG1 (BAD):
localhost:~# torify ssh -v ams02 OpenSSH_9.9p1, OpenSSL 3.3.2 3 Sep 2024 debug1: Reading configuration data /root/.ssh/config debug1: /root/.ssh/config line 284: Applying options for ams02 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 22: include /etc/ssh/ssh_config.d/*.conf matched no files debug1: Connecting to ngbjd5vpxamy5d63hcflinghioo2qa5e75uiysscmo4ml5qd4rpaobid.onion [127.42.42.0] port 60113. debug1: Connection established. debug1: identity file /root/.ssh/oams002_2 type 0 debug1: identity file /root/.ssh/oams002_2-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_9.9 kex_exchange_identification: banner line contains invalid characters banner exchange: Connection to 127.42.42.0 port 60113: invalid format
MSG2 (OK):
localhost:~# torify ssh -v ams02 OpenSSH_9.9p1, OpenSSL 3.3.2 3 Sep 2024 debug1: Reading configuration data /root/.ssh/config localhost:~# torify ssh -v ams02 OpenSSH_9.9p1, OpenSSL 3.3.2 3 Sep 2024 debug1: Reading configuration data /root/.ssh/config debug1: /root/.ssh/config line 284: Applying options for ams02 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 22: include /etc/ssh/ssh_config.d/*.conf matched no files debug1: Connecting to ngbjd5vpxamy5d63hcflinghioo2qa5e75uiysscmo4ml5qd4rpaobid.onion [127.42.42.0] port 60113. debug1: Connection established. debug1: identity file /root/.ssh/oams002_2 type 0 debug1: identity file /root/.ssh/oams002_2-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_9.9 debug1: Remote protocol version 2.0, remote software version OpenSSH_9.9 debug1: compat_banner: match: OpenSSH_9.9 pat OpenSSH* compat 0x04000000 debug1: Authenticating to ngbjd5vpxamy5d63hcflinghioo2qa5e75uiysscmo4ml5qd4rpaobid.onion:60113 as 'xxxxxxxxxx' debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: algorithm: sntrup761x25519-sha512 debug1: kex: host key algorithm: ssh-ed25519 debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
--x9p
Hi,
I do get a "banner line contains invalid characters" error
Imo, those characters are somehow related to the communication between the ssh client and the sshd on server. The onion skins should not be able to access the inner most layer, in this case the ssh communication.
Possible attack on servers via Tor Guard relays
With the written above, the Tor node attributes should not play a role.
the connection is terminated. (MSG1) Upon connecting for the second time, everything goes smooth. (MSG2)
Starting with MSG2 the ssh connection seems to work.
Something has happend. [1] reads:
SSHFP All SSH fingerprints (SSHFP) records of all the hosts are added to DNS. You can verify the SSH fingerprint by adding "-o VerifyHostKeyDNS=yes" to the ssh command. $ ssh -o VerifyHostKeyDNS=yes serverXX.openbsd.amsterdam The authenticity of host 'serverXX.openbsd.amsterdam' can't be established. ECDSA key fingerprint is SHA256:w3ZoL03eaY/2xdRd/7NvHHwfqIOjyv2O8xkvUnqEgps. Matching host key fingerprint found in DNS. Are you sure you want to continue connecting (yes/no)? yes ... serverXX$
Secondly, the MSG1/2 examples show ssh will speak to a host 'ams02'. The log shows the onion url 'ngb...bid.onion' that resolves to '127.42.42.0', which is not public routable. Confusing to me.
Any directions would be appreciated to solve the problem.
Maybe it's worth looking into local dns, host adresses and ssh fingerprint (known host). And what path the ssh communication takes, especially the difference between MSG1 and MSG2.
[1] https://openbsd.amsterdam/setup.html
- Cheers, Felix
On Wed, Jan 15, 2025 at 12:06:09PM -0300, x9p via tor-relays wrote:
I am running a relay and other servers. Sometimes doing SSHD over Tor via a hidden service in a VPS in openbsd.amsterdam. In all my other setups, in other providers, I do not see this problem happening.
Upon connecting for the first time, I do get a "banner line contains invalid characters" error, and the connection is terminated. (MSG1) Upon connecting for the second time, everything goes smooth. (MSG2)
What happens the third time? That is, does it happen consistently failing the first time and working after that, or what?
Is it possible a bad guard is logging the IPs first connecting, then dropping the connection, and then allowing the connection to succeed later?
As Felix said, there shouldn't be any way for your guard to get inside Tor's encryption, so no, that's unlikely to be what's going on.
And assuming you are indeed connecting to an onion service like it seems, there isn't any exit relay in the path that could be trying to modify or intercept the traffic either.
debug1: Connecting to ngbjd5vpxamy5d63hcflinghioo2qa5e75uiysscmo4ml5qd4rpaobid.onion [127.42.42.0] port 60113. debug1: Local version string SSH-2.0-OpenSSH_9.9 kex_exchange_identification: banner line contains invalid characters banner exchange: Connection to 127.42.42.0 port 60113: invalid format
Ok, it looks like that ssh server is still reachable at that onion service, so I connected to it myself and got similar results as you. The first time I connected, it did not send me any ssh banner. Eventually I hit enter and it gave me an error and hung up on me.
""" $ torify telnet ngbjd5vpxamy5d63hcflinghioo2qa5e75uiysscmo4ml5qd4rpaobid.onion 60113 Trying 127.42.42.0... Connected to ngbjd5vpxamy5d63hcflinghioo2qa5e75uiysscmo4ml5qd4rpaobid.onion. Escape character is '^]'.
Invalid SSH identification string. Connection closed by foreign host. """
Then I tried to connect again, and it gave me a banner that time, and several subsequent times:
""" $ torify telnet ngbjd5vpxamy5d63hcflinghioo2qa5e75uiysscmo4ml5qd4rpaobid.onion 60113 Trying 127.42.42.0... Connected to ngbjd5vpxamy5d63hcflinghioo2qa5e75uiysscmo4ml5qd4rpaobid.onion. Escape character is '^]'. SSH-2.0-OpenSSH_9.9
Invalid SSH identification string. Connection closed by foreign host. """
So: I think Tor is behaving correctly, and I think your ssh client is behaving correctly too. It is your sshd (ssh server) that is somehow not deciding to send a banner sometimes. That is where I would look.
--Roger
tor-relays@lists.torproject.org
-
Felix -
Roger Dingledine -
x9p