[remark] SSH authorization keys experiments

Last week, while brainstorming with a colleague, I discovered a few interesting options with regard to OpenSSH public key authentication mechanisms, specifically how OpenSSH resolves which public keys are allowed for a particular user.

First of all, we all know the ~/.ssh/authorized_keys file, but there is more. According to the sshd_config(5) manual page, we can control this resolution process through the following options:

• AuthorizedKeysFile:

  Specifies the file that contains the public keys used for user authentication.

  The format is described in the AUTHORIZED_KEYS FILE FORMAT section of sshd(8).

  Arguments to AuthorizedKeysFile accept the tokens described in the TOKENS section. After expansion, AuthorizedKeysFile is taken to be an absolute path or one relative to the user's home directory. Multiple files may be listed, separated by whitespace. Alternately this option may be set to none to skip checking for user keys in files.

  The default is .ssh/authorized_keys .ssh/authorized_keys2.

• AuthorizedKeysCommand:

  Specifies a program to be used to look up the user's public keys.

  The program must be owned by root, not writable by group or others and specified by an absolute path.

  Arguments to AuthorizedKeysCommand accept the tokens described in the TOKENS section. If no arguments are specified then the username of the target user is used.

  The program should produce on standard output zero or more lines of authorized_keys output (see AUTHORIZED_KEYS in sshd(8)). AuthorizedKeysCommand is tried after the usual AuthorizedKeysFile files and will not be executed if a matching key is found there.

  By default, no AuthorizedKeysCommand is run.

• AuthorizedKeysCommandUser:

  Specifies the user under whose account the AuthorizedKeysCommand is run.

  It is recommended to use a dedicated user that has no other role on the host than running authorized keys commands.

  If AuthorizedKeysCommand is specified but AuthorizedKeysCommandUser is not, then sshd(8) will refuse to start.

• TOKENS, the section referred by the previous options (I've removed non-applicable replacements):

  Arguments to some keywords can make use of tokens, which are expanded at runtime:

  • %% -- A literal %.
  • %f -- The fingerprint of the key or certificate.
  • %h -- The home directory of the user.
  • %k -- The base64-encoded key or certificate for authentication.
  • %t -- The key or certificate type.
  • %U -- The numeric user ID of the target user.
  • %u -- The username.

  AuthorizedKeysCommand accepts the tokens %%, %f, %h, %k, %t, %U, and %u.

  AuthorizedKeysFile accepts the tokens %%, %h, %U, and %u.

• (there are also similar options for SSH-certificate based authentication, which it seems are not actually X.509 certificates, but I'm not focusing on that in this article;)

Here are a few interesting use-cases for these options:

• master / backdoor SSH key -- have you locked-out yourself from your VM by accidentally removing or mangling ~/.ssh? (be aware that OpenSSH is very picky even with the home folder, ~/.ssh and ~/.ssh/authorized_keys permissions;) well you can now list something like /etc/ssh/authorized_keys--master ./.ssh/authorized_keys in the AuthorizedKeysFile, and gain access for any user, as a fallback for what one lists in ~/.ssh/authorized_keys; (please note that the global authorized keys file must be readable for everyone, thus chmod =rrr should be used;)

  AuthorizedKeysFile /etc/ssh/authorized_keys--master ./.ssh/authorized_keys
  

• tighter control on SSH keys -- instead of letting users manage their own authorized keys, one could manage them centrally with something like /etc/sshd/authorized_keys/%u, and deploy that folder with something like rsync or git; (please note that each file should be readable either by that user or by everyone;)

  AuthorizedKeysFile /etc/ssh/authorized_keys/%u
  

• delegate allowed public key resolution by fetching it from a URL, like for example with curl; (not particularly very safe, because whoever controls the server hosting of that URL, DNS resolution, or your network, could easily hijack that URL and gain access to your servers;)

  AuthorizedKeysCommand /usr/bin/curl -s -f -- https://operations.example.com/servers/2023a/authorized_keys/%u
  AuthorizedKeysCommandUser nobody
  

  (perhaps a wrapper script that percent-encodes %u, and verifies a minisign signature would be better;)

• delegate allowed public key resolution by using DNS resolution, something similar to how SMTP SPF records work; (as with the previous use-case, not particularly safe;)

• delegate allowed public key resolution by querying a local SQLite3 database;

!!!WARNING!!! -- when querying external sources, especially without additional authentication or encryption, care must be taken so that an attacker can't use this feature to explore and discover your internal infrastructure, or determine which employees have access to high-value targets.

For experimentation purposes, I've tried to implement a small proof-of-concept that verifies if a particular SSH public key, for a particular host and user, is allowed to authenticate, while taking into account the warning above about discovery attacks.

Although the code is not (yet) open-source, here are a few guide-lines based on its implementation:

• let's call the tool ssh-akc, which has at least a check subcommand;
• we configure OpenSSH with:

  AuthorizedKeysCommand /usr/local/bin/ssh-ack check https://operations.example.com/servers/authorized_keys/@{BUCKET_KEY} {shared-secret} %u %t %k
  AuthorizedKeysCommandUser nobody
  

• the tool computes two tokens based on:
  • the current host-name (for example by reading /etc/hostname);
  • the username, as received via %u;
  • the SSH public key type, as received via %t;
  • the SSH public key data, as received via %k;
  • the specified shared secret, {shared-secret} in the example above; (alternatively this could be read from a file;)
• one of the tokens is the "bucket key", and is replaced in the URL above;
• the other token is the "bucket data", and is the contents expected at the URL above;
• if, after successfully retrieving the contents of the URL, the response body is the same as the expected "bucket data", then it writes to /dev/stdout the string obtained by concatenating %t and %k (separated by a space), which is exactly what OpenSSH expects as a valid authorized keys file; (else just exit without writing anything, perhaps);
• the URL's can be served by anything, from a static HTTP server, to an S3 (or compliant) bucket, or even a static site host such as Netlify;

The following are the security properties of the whole system:

• it is important (but not critical) that write access to the URLs is permitted only to authorized employees;
• it is important (but not critical) that listing the available URLs is not permitted;
• it is important (but not critical) that the {shared-secret} is kept private, known only to authorized employees and the servers;
• however, if both the {shared-secret} is leaked, and the attacker can intercept or write to the URLs, then the attacker can own your servers;
• if the {shared-secret} token is kept secret, even if the attacker has write access to your URLs, he can only deny access, not grant himself access;
• gaining read-only access to all the URLs (even just a list of them without their contents), but provided that the {shared-secret} is not known, the attacker can't discover anything about your servers or users and their mappings; moreover, even if the shared secret is leaked, the attacker has to engage in a brute-force attack trying all combinations of possible hosts and users, but it's necessary he also knows the public keys (which can't be brute-forced);
• one can use a different secret for each host, thus increasing the strength against discovery attacks;

Care must be taken when computing the two bucket tokens, especially with regard to canonicalization attacks, for example I've used the following construct:

let secret_hash = blake3_derive_key (context = "ssh-ack v1 / secret", data = secret_string)
let context_data = join_strings (infix = "\0", strings = ["ssh-ack v1", host, user, ssh_key_type, ssh_key_data])
let context_hash = blake3_keyed_hash (key = secret_hash, data = context_data)
let bucket_key = blake3_derive_key (context = "ssh-ack v1 / bucket key", data = context_hash) .encode_to_hex
let bucket_data = blake3_derive_key (context = "ssh-ack v1 / bucket data", data = context_hash) .encode_to_hex

!!!WARNING!!! -- needless to say, I'm not a cryptographer, and I didn't do a thorough security analysis of this proposed scheme. Thus, use your common sense!