Changing CTTY on OS X

The backstory

I’ve been working on a SSH tunneling program for OS X on and off for a while now. It basically fully works, but I realized part way through, particularly after I got proper keychain integration working, that I also wanted to use it for simple SSH session management. So from the menu bar icon I’d want to open a terminal session and not have to log in myself. Well, the entire process by which I even got keychain support working with SSH is a long one which I won’t go into here, but it involved creating my own PTY for SSH and pretending to be a TTY and logging in (ideally I’d use SSH libs, but one step at a time!). Great, so how do I connect that to a terminal? Well… I tried to think of various ways to do this.

Changing C-TTYs on OS X

Since SSH relies on the controlling terminal for input, I knew that whatever my solution, it would involve having the terminal the user typed into being the controlling terminal. This eliminated options like retty (which may or may not work on OS X), as it only changes the input/output file-descriptors for the process. I got a hacked solution using expect in a shell script, but I didn’t like the idea of expect sitting between all input/output with SSH.

So I continued searching.

During my search a stumbled upon a few posts (post 1, post 2) that talked about how to modify the controlling TTY of a process; and those posts led to programs that let you modify the controlling TTY of a process as explained in the posts (reptyr, injcode, neercs). This meant I could connect the SSH session normally, and then transfer the process afterwards to the terminal TTY. Great! However… these programs are linux only. It turns out, in fact, that the ptrace() functionality they rely on to change the CTTY doesn’t exist in the Mach version of ptrace() at all.

Luckily, I stumbled upon these articles which provided some insight on how I might be able to hack up one of the existing programs to work on OS X:

It seems that it isn’t that such functionality doesn’t exist, it is simply in a different place.

Update hopefully coming soon…

Interesting articles on changing TTYs