IKEv2 VPN on Ubuntu – IKE authentication credentials are unacceptable

I used these straightforward strongswan IKEv2 VPN setup instructions to set up a IKEv2 VPN on my Ubuntu server.

However, instead of self-generating my own certificate authority and having to deal with manually trusting this untrusted CA on every device I have to use VPN on, I decided that since I had letsencrypt in standalone mode set up on my server already (and the vpn subdomain properly covered under the certificate), I might as well use that certificate instead. I’m sure I’m committing half a dozen security sins, but it saves me time that I used to spend googling “how to trust certificate authority on [device name]”.

Setting up letsencrypt certificates with strongswan-based VPN is out of scope for this post, but in short, the difference is minimal. Instead of using the fake CA’s .pem, just symlink the letsencrypt certificate and key to /etc/ipsec.d/certs and /etc/ipsec.d/private. Remember to add a post_hook to /etc/letsencrypt/renewal/ to reboot strongswan after certificate renewal.

This worked great on macOS High Sierra and iOS 11. However, Windows 10 (Fall Creators) refused to connect to the VPN, stating that “IKE authentication credentials are unacceptable”.

The top google results for this were highly misleading for this particular context. It’s not about subjectAltName or Server Authentication flags or whatever.

It turns out that unlike macOS or iOS, Windows 10 wasn’t processing the full set of root and intermediate CA certificates, even though I symlinked the fullchain.pem to /etc/ipsec.d/certs. After throwing a symlink from /etc/letsencrypt/live/[domain]/chain.pem to /etc/ipsec.d/cacerts, Windows finally relented and let me connect to the VPN.

YMMV, because judging by the Google results, there are a lot of sources of error that yield the exact same, mildly useless “IKE authentication credentials are unacceptable” error message. However, this particular error is non-obvious, and worth looking at if you’re running into the same issue.

Setting up OpenSSH Server on Windows 10 Anniversary Update

UPDATE 2017-12-22

Microsoft now ships a real SSH server (and client!) with Windows Fall Creators Update, no Cygwin or Linux subsystem required. Finally.

This post remains for historical context only.

Microsoft has finally landed its anniversary update for Windows 10. Among all the random useless features, is an actual Ubuntu Linux subsystem within Windows, with the ridiculously silly name “Bash on Ubuntu on Windows”. Goodbye Cygwin?

One of the first things I wanted to try was to setup the SSH server, so I can remote-login from my real box. Getting one up was actually fairly easy, if you can deal with a few problems with weird, weird red-herring error messages.

The problem

To recap, installing the OpenSSH server is as easy as popping open a bash shell (I assume you figured out how to get the Linux subsystem installed already and popped open Ubuntu )

sudo apt-get install openssh-server

If you try to now start the server with

sudo service ssh start

It’ll respond with

initctl: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused

but subsequently

Starting OpenBSD Secure Shell server sshd [OK]

When you now try to connect in from a remote host, there are two outcomes:

  1. There is no SSH server on port 22.
  2. There is an SSH server on port 22, but it responds only to your Windows password and not your Ubuntu Linux user password. If login is successful, it drops you into a DOS prompt instead of a bash prompt. Trying to run bash within the prompt generates a response:
    "Error: 0x80070005".

Diagnosis

There are multiple underlying problems here.

  1. First, it’s important to note that there exists a separate Windows SSH server (separate from the OpenSSH server from the Ubuntu subsystem installation) now on port 22. If you telnet (yes, telnet) into port 22, and the host greeting is SSH-2.0-MS_1.100, you’ve run into the Microsoft SSH implementation (hence the MS part in the greeting).

    I’m not sure what this server is supposed to do (and, in fact, starting SSH servers on standard ports without explicitly telling the user seems like a potential security problem to me, in addition to being kind of jerk-ish). I do know it is launched when you reboot with Developer Mode on. You know, Developer Mode, the mode you have to turn on to run Bash on Ubuntu on Windows. This server is occupying port 22, so you cannot launch another SSH server to listen on this port. This is the problem that is causing the silent failure of sshd to start up, not a broken Upstart or any of that nonsense the error message being displayed is referring to.

  2. Apparently the Ubuntu compatibility layer on Windows does not implement chroot or one of the related system calls needed for OpenSSH privilege separation at the time of this post.
  3. If you launched bash from DOS cmd.exe, instead of the Bash on Ubuntu on Windows shortcut, /mnt/c/Users/[Windows Username] (the directory you start with in DOS) is not your Linux home directory. .ssh config files left in this directory will not configure your OpenSSH server.

Symptoms & Solutions

  1. To resolve the problem of conflicting SSH servers, set your OpenSSH server on a different port : one other than 22. This can be done by editing /etc/ssh/sshd_config and changing the Port configuration. Be sure to open this new port in your Windows Firewall for inbound connections, as this firewall configuration will not be automatically done for you. I set mine to 60022.
  2. To enable login, also change UsePrivilegeSeparation in the same config file to No. Failure to do so will cause the server to respond with Connection reset by nnn.nnn.nnn.nnn
  3. To enable public key auth and set other configurations, the Linux subsystem home directory is, as in normal Ubuntu, /home/<yourname>.

Having done this, now you can start or restart in the bash shell

sudo service ssh restart

And connect from remote host with your Ubuntu user credentials and/or public key:
$ ssh -p 60022 10.0.1.208
...
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 3.4.0+ x86_64)

(And if you telnet, the host message should be something like
SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.7
)

Building from source package on Debian / Ubuntu to fix sudo PATH issue

So I’ve been kicking around an Ubuntu installation, hoping to replace my aging Fedora 5 deployment. Last time I touched a Debian distro was…well…sufficiently long ago that it’s more or less all new to me.

What’s less new is the sudo path inheritance issue — this one’s been around. Ubuntu’s sudo hard-codes its PATH variable at compile-time with a --secure-path option. I’m sure this sounded like a good idea to the security goon who decided to fix this at fsckin’ COMPILE TIME with no way to override it in sudoers, or at runtime with -E after an env_reset. The policy may have been reasonable when it was set on a typical Debian stable server (where software is basically left to fossilize over decades), but certainly not on a constantly changing desktop distro. You can’t even sudo to any /opt/bin binaries! Read the Ubuntu bug report on sudo not preserving PATH.

Long story short, after a lot of experiments looking for workarounds (that won’t eventually take years off my life, one sudo command at a time), I decided to cut the Gordian knot and recompile sudo. Since I didn’t want to roll this from source (and incur all the maintenance hassle of removing/updating the software later on), this meant figuring out compiling source packages with dpkg — oh joy.

Debian source package compilation: the general process

It’s surprisingly non-painful compared with my RPM experience. The long way around:

  1. cd into a temp or source-keeping directory in your user account
  2. retrieve the source package: apt-get source [packagename]
  3. grab missing build dependencies: sudo apt-get build-dep [packagename]
  4. cd into the directory created for the package in your pwd (you can safely ignore the original tarball and the patch file, which have been untarred and applied for you already, respectively). Make edits to the source as needed.
  5. If you need to change configure options for the source package, look in the file debian/rules in the source directory
  6. when satisfied, build the binary package by issuing this incantation in the $PWD ( you’ll need the fakeroot package if you don’t already have it ):
    dpkg-buildpackage -rfakeroot -uc -b
    Use -nc if you mess up and need to continue a build.
  7. The completed .deb packages are placed in the parent directory, one level up from the source directory. cd back up one level.
  8. install: sudo dpkg -i [packagename].deb

If you’re screwing around with sudo, you will want to have a sudo tty session open before installing your replacement package, in case you screw up everything and lock yourself out.

A shortcut is potentially available using the -b switch to apt-get when you grab from source. However, I needed to look through configuration files and source code, so I took the long way around.

The easiest way to fix the sudo secure_path issue is to remove the --with-secure-path configuration option in debian/rules, in two places in that file. If you do this, pay attention to your $PATH and make sure they are sane (for example: it shouldn’t contain a globally writeable directory), as it will be inherited in sudo shells. In sudo 1.7, there is a runtime secure_path option for the sudoers file, so that would be the ideal, non-annoying solution to this issue.

Hard-coding the sudo PATH at compile-time tilts heavily toward security in the security/usability tradeoff — YMMV, but I find it entirely not worth it on a desktop distribution.