Segfault in upterm 0.7.3

[larsks]

If I start a session with upterm 0.7.3...

$ upterm version
Upterm version v0.7.3
$ upterm host
=== ...
Command:                /bin/bash
Force Command:          n/a
Host:                   ssh://uptermd.upterm.dev:22
SSH Session:            ssh ...@uptermd.upterm.dev
Press <q> or <ctrl-c> to continue...
$

And then connect to that session with ssh, the session crashes with:

[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x86fc92]

goroutine 47 [running]:
golang.org/x/crypto/ssh.(*curve25519sha256).Server(0xc000474390, {0x7f3cdbfc4f10, 0xc000164000}, {0xb94880, 0xc0000b8570}, 0x80, {0x0, 0x0})
/home/runner/work/upterm/upterm/vendor/golang.org/x/crypto/ssh/kex.go:515 +0x272
golang.org/x/crypto/ssh.(*handshakeTransport).server(0xc00017e2c0, {0xba06f8, 0x1038970}, 0xc000128500, 0xc0001645a0)
/home/runner/work/upterm/upterm/vendor/golang.org/x/crypto/ssh/handshake.go:643 +0xe3
golang.org/x/crypto/ssh.(*handshakeTransport).enterKeyExchange(0xc00017e2c0, {0xc000302700, 0x0, 0x0})
/home/runner/work/upterm/upterm/vendor/golang.org/x/crypto/ssh/handshake.go:592 +0x3eb
golang.org/x/crypto/ssh.(*handshakeTransport).kexLoop(0xc00017e2c0)
/home/runner/work/upterm/upterm/vendor/golang.org/x/crypto/ssh/handshake.go:301 +0xa6
created by golang.org/x/crypto/ssh.newServerTransport
/home/runner/work/upterm/upterm/vendor/golang.org/x/crypto/ssh/handshake.go:143 +0x13d

It also leaves the terminal in a bad state such that running stty sane is necessary (the traceback is also formatted very poorly due to the terminal state. I've just remove all leading whitespace to try to make it a little more legible).

[owenthereal]

I saw this error the other day. I think it's related to the RSA issue mentioned here: #93 (comment). Could you back up your ~/.ssh dir with something like mv ~/.ssh ~/.ssh.bak, regenerate an Ed25519 ssh key with ssh-keygen -o -a 100 -t ed25519, and try again? This would make sure your ssh key dir only has an Ed25519 ssh key.

[reidrac]

I'm also having this issue but it doesn't look like the workaround in #93 (comment) is working.

I'm running OpenSSH_8.4p1 Debian-5, OpenSSL 1.1.1k 25 Mar 2021.

Also:

$ ssh -Q PubkeyAcceptedKeyTypes | grep ^ssh-rsa$
ssh-rsa
$ ssh -Q HostKeyAlgorithms | grep ^ssh-rsa$
ssh-rsa

So it doesn't seem related?

EDIT: oh, and using latest upterm.

[owenthereal]

Could you try another key type, e.g. Ed25519, as described here: https://github.com/owenthereal/upterm#a-note-about-rsa-keys. Please make sure the rsa key is purged from ~/.ssh and the ssh agent.

[reidrac]

Could you try another key type, e.g. Ed25519, as described here: https://github.com/owenthereal/upterm#a-note-about-rsa-keys. Please make sure the rsa key is purged from ~/.ssh and the ssh agent.

I will, thank you!

I wonder if it would possible to prevent the SIGSEGV and report an error instead.

[reidrac]

No change: tried using an ED25519 identity and even with no ssh-agent involved, and I always get the same: SIGSEGV.

From GDB:

0x0000000000875652 in golang.org/x/crypto/ssh.(*curve25519sha256).Server (kex=<optimized out>, c=..., rand=..., magics=0xc00033c2a0,
    priv=<error reading variable: access outside bounds of object referenced via synthetic pointer>)
    at /home/reidrac/src/go/upterm/vendor/golang.org/x/crypto/ssh/kex.go:515
515             hostKeyBytes := priv.PublicKey().Marshal()

So it is using ED25519.

[owenthereal]

Could you share the output of the ssh command with the -vv flags? For example, ssh TOKEN@uptermd.upterm.dev -vv

[reidrac]

I run:

ssh -vvv -i id_ed25519.pub ...@uptermd.upterm.dev -o IdentityAgent=/dev/null 2> ssh.log

This should ensure that I use the right key and my ssh-agent is not getting in the way.

ssh.log

[owenthereal]

You are using the id_ed25519 public key (-i id_ed25519.pub) instead of the private key for the ssh command. This also reflects in your logs:

debug1: Will attempt key: id_ed25519.pub ED25519 SHA256:clJUbZ1WF6pTctuCA+q2xhql8S+EeStQOnMrM++hO3Q explicit

The -i flag takes the private key:

     -i identity_file
             Selects a file from which the identity (private key) for public key authentication is read.  You can also specify a public key
             file to use the corresponding private key that is loaded in ssh-agent(1) when the private key file is not present locally.  The
             default is ~/.ssh/id_rsa, ~/.ssh/id_ecdsa, ~/.ssh/id_ecdsa_sk, ~/.ssh/id_ed25519, ~/.ssh/id_ed25519_sk and ~/.ssh/id_dsa.
             Identity files may also be specified on a per-host basis in the configuration file.  It is possible to have multiple -i options
             (and multiple identities specified in configuration files).  If no certificates have been explicitly specified by the
             CertificateFile directive, ssh will also try to load certificate information from the filename obtained by appending -cert.pub to
             identity filenames.

Try with the private key ssh -i id_ed25519 ... instead.

However, upterm should handle this error more gracefully as mentioned here: #114 (comment)

[reidrac]

My bad, I have tried it so many times that I finally got it wrong.

ssh -vvv -i id_ed25519 ...@uptermd.upterm.dev -o IdentityAgent=/dev/null

Has the same results.

ssh.log

EDIT: what is confusing me is that I was expecting the keys to not be accepted in case ssh-rsa was not supported, but instead it crashes. Also this is OpenSSH_8.4p1 Debian-5, OpenSSL 1.1.1k 25 Mar 2021, that still supports ssh-rsa.

[reidrac]

The issue is fixed for me in 0.7.6 🎉

[larsks]

Seems to be working for me now as well!