Public key cryptography (asymmetric cryptography) is the foundation of the Internet and it is used for a variety of purposes.
Public and private keys can be stored in several different types of files. Each of these types can have its own encoding. The overall format of a file can be quite complex. It is important, however, to understand the purpose of these formats, and how they’re used.
This document can be used as a primer for understanding these file/encoding formats.
The actual structure (objects and fields) of public/private keys, including X.509 certificates, is specified in various RFCs using the ASN.1 notation.
For example, an RSA private key contains the following fields:
RSAPrivateKey ::= SEQUENCE {
version Version,
modulus INTEGER, -- n
publicExponent INTEGER, -- e
privateExponent INTEGER, -- d
prime1 INTEGER, -- p
prime2 INTEGER, -- q
exponent1 INTEGER, -- d mod (p-1)
exponent2 INTEGER, -- d mod (q-1)
coefficient INTEGER, -- (inverse of q) mod p
otherPrimeInfos OtherPrimeInfos OPTIONAL
}
The most widely used format is X.509 and it’s full syntax is defined by the RFC5280. X.509 provides the support for the “chain of trust” to verify a public key, as well as various extensions, primarily concerning the key’s usage. RFC5280 also documents formats for CSR, CLR, etc.
PKCS #8 specification defines the structure of private keys (PKCS stands for Public-Key Cryptography Standards).
These specifications only stipulate the structure (fields and objects), we still need to decide how to “encode” these fields when we want to save them on disk.
Security considerations aside, it would be nice if everything was stored in some sort of a structured format that we’re all used to, such as JSON or YAML, but this not the case for the majority of crypto formats, with the exception of JWK as explained later.
“DER” encoding, which is a variation of a more general X.690 encoding, is the de-facto standard for encoding ASN.1 structures. DER is a purely binary format, DER-encoded content can’t be inspected with a text editor. The basic idea of DER is to represent each field as a type+length+value triplet.
DER format is the one used for sending certificates over the wire — e.g., when a server presents its certificate to the client as part of the TLS handshake, the certificate bits are serialized using DER (theoretically, a certificate can be stored in a completely different format on disk).
DER can be further encoded using Base64, which creates an ASCII text that could be copied-pasted, sent over email and so forth. Base64-encoded DER files are usually saved with the “pem” or “crt” extension. PEM files also contain a header and footer to give an idea of the content of the file (sort of poor man metadata). X.509 files use “—–BEGIN CERTIFICATE—–“/”—–END CERTIFICATE—–“, but various other headers/footers can be used as well, such as “—–BEGIN PUBLIC KEY—–“/”—–END PUBLIC KEY—–“. In this last case, the file contains only the public key information and none of the additional X.509 fields.
Private keys can be saved in the PEM format as well, “—–BEGIN PRIVATE KEY—–“/”—–END PRIVATE KEY—–” is used to denote such files. All these headers as well the detailed PEM-encoding rules are documented in this specification.
ASN.1/DER/PEM is mostly used for TLS implementation and whenever X.509 certificates are involved.
Other public-key cryptography implementations can use different formats. Irrespective of the format, the underlying implementation and algorithms remain the same. E.g., an RSA public key must contain modulus and exponent fields, the only question is how to pass these fields around or how to store them in a file.
OAuth2 and JWT use JSON to exchange public/private keys.
Here is an example of a public key in the JWK format used by OAuth2 authorization servers:
{
"kty": "RSA",
"alg": "RS256",
"kid": "oViynWdKmd9m43BihjrQH9bHlp22fto0Nu-zwaBzUAs",
"use": "sig",
"e": "AQAB",
"n": "q8BD_0q9JQRnpZ5vLnBMEA03nUWmxE56nGvKFY8K0fOAHojFPExI0Il67NEv6TCPZaXiifT5p9N9DIQl-JaWNaQmDCvd5Hbeugqn05QGJ14E_ghTXA6iXsONnavri5qlgc5rPmAS9zkm755ID7mHnuskEMXJy929LlxFKHzDRTkN8Lf1hSVXG8Mdy0f1QW-01VNRE8ZW0Ar5vLLuGrDb8bg9fCZXA6CK7oVJHXzo6ajIgzpa86kpdvWOhhtYPCL9P9wNjt4kfX3LBb6_sl9s8lI0C0OWtoMyNtAbE4wFc08o0ZsW1UGQin5eFFBuH_zbaPwc7wvYw40bBw35U_V9Sw"
}
SSH keys (the ones that usually start with “ssh-rsa”) use their own form of encoding documented in the RFC4253 — each component of a key is stored as length+data and the entire key is Base64 encoded. Note that this is different from DER encoding, which is type+length+value. This post provides all the details.
It would obviously be a good idea to somehow encrypt and password-protect private keys. There are several formats that can be used for this purpose.
PKCS #8 files (usually encoded as PEM) files can be encrypted with a passphrase and various cyphers, in which case these file start with “—–BEGIN ENCRYPTED PRIVATE KEY—–” header.
The most widely used format for storing keys and certificates in an encrypted format is PKCS #12, defined by RFC7292. It can be used for storing certificates, public/private keys, and even arbitrary passwords. These files have “p12” or “pfx” extension (“pfx” is a PKCS #12 predecessor). Modern versions of Java use PKCS #12 as the default keystore format, although these files can still have the legacy “jks” extension.
Internally, PKCS #12 uses DER/ASN.1 structures for storing certificates and private keys.
For more recommendations on how to properly secure PKCS #12/JKS files, please refer to our keystore best practices post and the e-book.