Secrets, the Boring Way

The failures are never the cryptography. They are a credential in a place nobody thought of as a place.

Security Christopher Roos 3 min read

Secret management attracts elaborate solutions, and most organizations would get more benefit from doing the boring parts consistently than from adopting anything sophisticated. The failures I have seen are almost never a failure of the cryptography. They are a secret in a place nobody thought of as a place.

The realistic threat

Worth being clear about what you are defending against, because it shapes the effort. For most teams it is not a determined attacker breaking a key. It is:

  • A credential committed to a repository, possibly years ago, still valid.
  • A key in an environment variable that gets printed by a debug log or an error report.
  • A shared account whose password is in a document, used by six people, one of whom left.
  • A long-lived access key on a laptop, still working, belonging to someone who changed teams.

None of these require sophistication to exploit and none are prevented by choosing a better vault.

The single highest-value change

Get rid of long-lived credentials wherever a short-lived one will do. A federated role that issues a token valid for an hour is not merely more secure than a stored key, it is categorically different: there is nothing durable to leak, so the whole class of "this key was in a file somewhere" disappears.

The common case is CI. A pipeline holding permanent cloud credentials as a stored variable is an obvious target and an unnecessary one, since every major CI system now supports issuing short-lived credentials through identity federation. Migrating that is usually an afternoon and removes a permanent liability.

Scope before rotation

Rotation gets most of the attention, and it matters, but scope matters more. A credential that can only read one bucket is a much smaller problem when it leaks than one that can do anything, regardless of how often either is rotated.

The practical version is to start every permission set at nothing and add what fails. It is slower than copying an existing policy, and the existing policy is almost always wider than it needs to be, because it was copied from something else.

Assume it will leak, and plan the response

The question that reveals whether a system is actually managed: if a credential leaked right now, how long would it take to invalidate it, and would anything break that you do not expect?

If nobody knows, the answer is longer than you would like. It is worth rehearsing once, in a low-stakes case, so that the procedure exists before it is needed. This is the same argument as restore drills, applied to a different failure.

Make the safe path the easy path

Every one of these failures happens because the insecure option was more convenient at the moment somebody was in a hurry. If retrieving a secret properly takes four steps and pasting it into a config file takes one, the config file will keep winning, and no amount of policy will change that. The fix is to make the correct path the shortest one.