Java Keystore Best Practices

A keystore is primarily a database for storing application secrets. Keystores can also be used for storing “trust certificates” and CA chains.

Trust certificates are public keys and they don’t necessarily need to be protected with the same rigor as private keys. However, they identify all trusted parties for an application. Java automatically looks for a certificate (or its root CA) in its keystores for all TLS requests. If it’s found, the TLS handshake will be successfully established.

Java comes bundled with the default keystore called “cacerts”. This keystore is pre-populated with many well-known root CAs. This means that any TLS call to a server whose certificate was issued by a well-known CA will be trusted.

Do not Use Default Keystores

Do not use “cacerts” or any other keystore bundled with Java. Instead, create the keystore specifically for your application. This keystore must only contain the keys/certs (including CAs) that are used by the application, it shall not contain any unused certs (including CAs), expired certs, etc.

Change the default password

Any Java developer knows that the default password for Java keystores is “changeit”. If you want to use the keystore files bundle with Java (which we do not recommend), please change it to a strong password.

keytool -storepasswd -keystore cacert

Same goes for IBM JDK and WebSphere products (including a more modern Liberty app server) – the default password (“WebAS”) is well-known. Changing this password is the first thing that any WebSphere administrator must do.

Change the Keystore Password on a Frequent Basis

As with any password, frequent password refresh is the key. Some organizations treat keystore passwords as service accounts and thus change them infrequently. This creates risks.

We recommend updating keystore password for every deployment/release.

Secure your Keystore and Key Passwords

Very often, keystore passwords are stored in open text in various config files.

E.g., here is a typical configuration of a Tomcat connector:

<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
    keystoreFile="${catalina.home}/conf/keystore.p12"
    keystorePass="mypass" keyPass="mypass"
    keyAlias="main-key"
    maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
    clientAuth="false" sslProtocol="TLS"
    />

This is obviously completely unacceptable from the security standpoint.

Save sensitive secrets elsewhere, ideally use a secret manager, like Hashicorp vault. At the very least, the master password could be stored in an environment variable for a properly protected account.

Keep Private Keys Separate

Keep the private keys in a separate keystore file, do not use the same keystore for trust certificates and for the keys. The keystore with the keys must be protected using the filesystem permission. It must also be secured using a complex password. The keystore with the keys must only contain the key/certificate pairs and CAs (if needed), nothing else.

Keep only Active Keys/Certs

Make sure that the keystores contain only active keys, certificates and CA chains. Expired certificates, unused CAS must be removed.

This requires implementing a process for cleaning keystores on a periodic basis or as part of every application release.

Do not Package Keystores inside Jar Files/Application Archive Files

Very often developers bundle keystore files with all the other application resources so they end up on the application classpath and then automatically picked up my Maven/Gradle build tools and added to Jar files.

This makes it difficult to update keys/certificates when needed. Keys/certificates can be (and should be) updated on a lifecycle which is different from the application’s lifecycle. Therefore, keeping them outside of application archives is a good idea – this allows for updating keystores completely independently from the application itself.

Even when there is a truly continuous delivery process in place and the application code is updated very frequently, it is still worth it to de-couple keystores from applications from the packaging standpoint so that the keystores could be quickly updated in case of a breach.

This also means that the deployment toolchain for keystores should be separate from the regular application deployment toolchain. In other words, keystores should be treated as a completely separate deployment artifact, independent of the application.

Do not Package Keystores inside Docker Containers

Keystores and PEM files should be placed on an external docker volume so they can be updated without modifying the docker image. This allows for updating keystores independently from docker images in case of a security breach or for a regular key/certificate rotation/refresh.

The files on the external volume could simply be replaced on the host and then the docker container (or multiple containers) can be restarted.

If there are multiple docker containers running on the same host, they could all share the same volume with keystores.

Do not Store Keystores in the Application Git Repo

All keys and secrets must be de-coupled from source code and stored separately.

Separate Keystores for Different Environments

The production environment must always have its own dedicated keystore file.

Use the PKCS12 Format for Keystores

PKCS12 is compatible with other tools, such as openSSL. It is the default for Java 9 and above but it must be explicitly set for previous Java versions.