Self-Signed Certificates Best Practices and How-to Guide

Posted on 08/10/2019 ,

Self-signed certificates are widely used for testing/development and sometimes in production for internal websites.

Self-signed certificates are created without any CA, thus they don't have a parent. The issuer is also the subject of the certificate.

In general, the use of self-signed certificates must be discouraged as they present an inherent security risk. For example, there is no way to revoke a self-signed cert. Using an internal CA for issuing all internal certificates is a much better option, we will cover it in a future post.

Self-signed certs come at a substantial maintenance cost -- issuing a cert for a long period of time is unsecure, but the short validity adds to the certificate renewal/distribution overhead.

The following best practices will help to make self-signed and internally-issued certificates more secure:

  • Limit the validity period, it should be as short as you can handle from the maintenance standpoint. Do not go beyond 12 months.
  • Do not use wildcards and limit the alt names, make it as specific as possible -- the certificate should only be issued for the exact hosts/domains where it is going to be used. Use the alternative names extension to list all the hostnames.
  • Use a meaningful and informative "subject" record. CN must match one of your domain names. You can use OU to specify the name of your application or service. Make sure that the full DN record used in the subject is unique across all of the certs. There are many attributes that can be used in the "subject" to make it more informative, you can find the full list here
  • Make sure that the algorithm used for the signature is at least SHA256WITHRSA (which is the default in OpenSSL)
  • Do not create unencrypted private keys. If a key is for a Java application, use the pkcs12 format, it can be used directly by any modern JVM. OpenSSL initially generates a DER encoded private key encrypted using DES, but it can be converted into pkcs12 using openssl pkcs12 command. PKCS12 is the implementation of the PKCS#11 format which is the universal, cross-platform cryptography interface, so it should be preferred over other formats.
  • Use elliptic curve keys as opposed to the default RSA ones, they provide a number of benefits over RSA
  • You can make your certificate more robust by specifying the certificate's purpose using extended key usage and "key usage" extensions. "TLS Web Server Authentication" should be the only allowed usage for a server. This will prevent unintended use of the certificate.
  • Make your extensions "critical". Self-signed certificates/keys are only used internally, so all clients should be able to support the usage extensions.
  • Create a repeatable/scriptable process for issuing certificates and keys. OpenSSL is a de-facto standard command line tool that can be used as the basis for this process.
    OpenSSL allows to externalize certificate information in a separate config file that can be version-controlled; this is highly recommended.

Here is a recommended OpenSSL configuration for certificate extensions:

[ extensions ]
basicConstraints = critical, CA:FALSE
keyUsage =critical, digitalSignature, keyEncipherment
extendedKeyUsage = critical, serverAuth
subjectKeyIdentifier = hash
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = host1
DNS.2 = host2

Here is the OpenSSL command for creating self-signed certificates:

openssl req -config cert_conf/self_signed_cert.conf -x509 -days 60 -new -newkey rsa:2048 -keyout certs/ss_keys/self_signed -out certs/ss_certs/self_signed.pem -verbose -passout pass:<your password>

"cert_conf/self_signed_cert.conf" is the certificate configuration file that contains the certificate information section as well as the extension section from above.

How to Troubleshoot and Fix Certificate Validation Issues in Java

Posted on 07/25/2019 ,

Certificate validation errors are a frequent cause of issues when dealing with APIs and Web services calls, especially when self-signed certificates are used.
The error message is usually javax.net.ssl.SSLHandshakeException: PKIX path building failed.

How to Troubleshoot

Run your code with the following system property: javax.net.debug="ssl,handshake,trustmanager". This trace will provide the most relevant output. You may want to save the entire output to a file.

At the top you will the exact trust store file used by the JVM as well as the available certificates:

javax.net.ssl|DEBUG|01|main|2019-07-24 16:11:55.923 EDT|TrustStoreManager.java:112|trustStore is: ./src/test/resources/com/myarch/sec/cryptofiles/app_truststore.pkcs12

Later on, after "ServerHello", you will see the certificate chain (could be a single certificate too, in the case of a self-signed certificate, it's always one cert) presented by the server:

javax.net.ssl|DEBUG|01|main|2019-07-24 16:11:56.511 EDT|CertificateMessage.java:358|Consuming server Certificate handshake message (
"Certificates": [
  "certificate" : {
    "version"            : "v3",
    "serial number"      : "00 FD FF 7C 02 1F DF B2 51",
    "signature algorithm": "SHA256withRSA",
    "issuer"             : "CN=Go Daddy Secure Certificate Authority - G2, OU=http://certs.godaddy.com/repository/, O="GoDaddy.com, Inc.", L=Scottsdale, ST=Arizona, C=US",
    "not before"         : "2017-10-27 19:32:01.000 EDT",
    "not  after"         : "2019-10-27 19:32:01.000 EDT",
    "subject"            : "CN=myarch.com, O=My Arch Inc., L=Centreville, ST=Virginia, C=US,

To see an abbreviated list of serial numbers of certificates in the trust store and certificates from the server, run:

cat log_file | grep  "serial\|ServerHello"

The output will look like this

   "serial number"      : "00 86 9A E3 5A 4D FE 72 BA",
javax.net.ssl|DEBUG|01|main|2019-07-24 16:11:56.415 EDT|ServerHello.java:866|Consuming ServerHello handshake message (
"ServerHello": {
javax.net.ssl|DEBUG|01|main|2019-07-24 16:11:56.416 EDT|ServerHello.java:962|Negotiated protocol version: TLSv1.2
    "serial number"      : "00 FD FF 7C 02 1F DF B2 51",
    "serial number"      : "07",
    "serial number"      : "1B E7 15",
    "serial number"      : "00",

One of the certificates (and one of the serials) in the trust store must match one of the certificates from the server.

How to Fix

You have a choice of adding the end entity's certificate (with the subject's CN usually matching the domain/hostname) or one of its issuers to the trust store.

Going with the end-entity certificate is more secure, however, you will need to update it when it changes/expires on the server. CA certificates have a much longer validity period, but having the CA cert in the trust store will make ALL of the certificates issued by that CA trusted by default (unless it was revoked by the CA).

The easiest is to obtain the certificates from the server is by using openssl:

openssl s_client -connect myarch.com:443  -showcerts

Copy the certificate that you want to import (starting with "-----BEGIN CERTIFICATE-----" and including "-----END CERTIFICATE-----") into a file. This gives you a PEM-encoded certificate.

Import the certificate into the truststore:

keytool -import -alias cert_alias -file cert_file -keystore truststore_file

Background

By default, Java uses a separate "truststore" (the same concept as the keystore, only for certificates, it does not contain any private keys), called "cacert". It is pre-populated with all known CAs (trust anchors). So all certificates issued by Verisign or Digicert will be trusted. As an aside, from the security standpoint, it is much better to use separate application-specific truststore.

You can see all the certificates in the default trustore by running

keytool -list -cacerts -storepass changeit

For a non-default trustore:

keytool -list -v -keystore trustore_file -storepass truststore_password

Certificates issued by internal CAs or self-issued certificates (a.k.a. self-signed) are never trusted by default.

During the TLS handshake a server presents its certificates (usually including the issuer's certificate) to the client as part of the "ServerHello" message. One of the certificates presented by the server must be in the truststore. This could be the end-entity cert (most secure) or one of its issuers.

The Java client is still going to validate the entire certificate chain, starting with the end-entity certificate, following the Certification path validation algorithm. This includes many additional checks, including the expiration date and the revocation (using OCSP starting with Java 9). If none of the certificate from the chain is in the truststore, the validation will fail in the very beginning with the "PKIX path building failed" exception.

Java Keystore Management Best Practices

Posted on 07/23/2019 ,

A keystore file is a database for storing application secrets (private keys), trust certificates and CA chains. Proper keystore/truststore management is extremely important for application security.

We’ve compiled a list of keystore-related best practices in our keystore management document.

Here is a brief summary of the document:

  • Do not use default keystore/truststore files (set “javax.net.ssl.trustStore” system property accordingly)
  • Change default keystore passwords
  • Update keystore (and keypair) passwords on a frequent basis
  • Properly secure your keystore passwords, don’t store them unencrypted
  • Keep private keys in a separate keystore file
  • Set permissions for keystore files to read-only. The account used to run the application should be the owner of the file.
  • Clean up your keystores, get rid of all the expired/unused keys, certs, and CAs
  • Do not package keystores inside jar/war files
  • Do not package keystores inside Docker containers
  • Do not store keystores in the application’s Git repo
  • Use different keystore files for different environments
  • Use the PKCS12 format

Certificate Management Best Practices Summary

Posted on 11/25/2018 ,

For more details, please refer to our certificate management document.

Best practices list:

  • Restrict certificate validity to short periods of time
  • Automate certificate renewal/refresh
  • Implement certificate validation/revocation mechanism (OSCP)
  • Do not use self-signed certs
  • Do not use wildcard certs
  • Establish and maintain a complete certificate inventory—you must know where each certificate is deployed, its expiration, etc.
  • Run frequent endpoint/port scans to detect self-signed and other out-of-policy certificates.
  • Go beyond HTTPS endpoints—also scan TCP endpoints, certs on disk, etc.
  • Minimize trust between system components, don’t blindly trust all certs/all CAs
  • Disable trusting to all public CAs by default
  • Use the internal CA for internal communications/calls
  • Implement certificate pinning
  • Implement a comprehensive approach to protecting private keys (passwords, keep the keys separate from public certificates, do not keep keys in the same git repo with the code)
  • Keep certificates outside of docker containers, put them on volumes where they can be easily updated

Certificate Management Best Practices Document

Posted on 11/08/2018 ,

We're incorporating more security reporting/compliance features into DPBuddy and we're also working on a new product related to certificate management.

As part of this work, we're attempting to compile and aggregate best practices related to certificates and key management.

A lot of it is just common sense, however, as we all know, even simple steps require some effort from developers and security professionals.

We're hoping that our document can be used as a checklist for everyone involved with the application security. We're also planning to automate some if not all of these guidelines in the new version of DPBuddy and in our upcoming new product.

Please review our security best practices document and be sure to register for updates.

DataPower Buddy Release 3.4

Posted on 05/30/2018 ,

We're pleased to announce the availability of DataPower Buddy 3.4.

This release provides support for DataPower firmware upgrades, extensive configuration reporting and diffing, DataPower operational analytics and many other features.

Please see release notes for more information.

Download PBuddy 3.4 from this page and follow our Quick Start Guide.

DataPower Buddy Roadmap

Posted on 05/29/2018 ,

Next DPBuddy release will provide improved operational analytics and certificate management. We're also working on improved configuration reports and configuration diff.

We're also planning on implementing native plugins for Maven and Gradle.

What else would you like to see in the upcoming versions of the product? Please let us know.

Running DPBuddy from Docker

Posted on 05/05/2018 ,

You can now use our DPBuddy docker image that comes pre-installed with Java, Apache Ant and DPBuddy.
Simply run "docker pull myarch/dpbuddy:3.4" and follow our documentation.

DPBuddy Cookbook

Posted on 10/09/2017 ,

Our cookbook contains quick examples/samples/code snippets to help with the most common DataPower development and administration tasks. The cookbook is a live document and it is frequently updated with new information.

Collecting and Analyzing DataPower Logs with DPBuddy and Elastic Stack

Posted on 08/30/2017 ,

Please follow this link.

Automating DataPower Firmware Upgrades with DPBuddy

Posted on 01/29/2017 ,

How to Deal with Generated DataPower Policies

Posted on 07/25/2016 ,

How to Manage and Remotely Tail DataPower Logs

Posted on 07/25/2016 ,

DataPower Buddy Release 3.3

Posted on 06/08/2016 ,

We're pleased to announce the availability of DataPower Buddy 3.3.

New Tasks/Commands and Notable New Feature

  • Support for encrypted properties and configuration settings. This can be used for encrypting DataPower account passwords. It can also be used for encrypting any sensitive data inside DataPower configuration/export files or inside any files that are copied to DataPower using DPBuddy's copy command.
  • PasswordAlias task/command to create DataPower password aliases. The password can be stored encrypted.
  • New format for configuration settings and properties/variables based on the open-source HOCON (Human-Optimized Config Object Notation) library. HOCON is a less restrictive JSON superset geared towards defining configuration settings. HOCON allows for comments, it does not require string quotes, it supports variable substitution and includes. For more details please refer to HOCON documentation. HOCON provides many benefits over the property/prefix-based format introduced in earlier DPBuddy releases. The properties format, however, is still fully supported, migration to HOCON is completely optional.
  • Support for DataPower firmware 7.5
  • Secure restore task/command. The command performs restoration of a secure backup and optionally waits for the completion of the secure restore process. This command can be used for automated sync of multiple appliances in production or for managing disaster recovery appliances.
  • Restore task/command. This command restores multiple domains (or all of the domains in an appliance) form the backup taken by the backup command. This command can also be used for appliance syncing and for DR.
  • RestartDP task/command. This command restarts the appliance and optionally waits for the completion of the restart process.
  • License command that prints DPBuddy's license information.

Configuration Transformation-Related Changes

  • Support for repeaters (loops). Repeaters allow for generating XML fragments in a loop based on an array or an object defined in a HOCON configuration file. For example, you may need to change the definition of a Load Balancer Group depending on the target environment (each environment can have a different number of LBG members). You can define your LBG members in the configuration file and then apply "update" action with "repeat" attribute.
  • "if" expressions are now supported at the "transform" container level (previously "if" was only supported at the action level).
  • "verbose" setting is now supported at the "transform" container and at the individual transformation action levels.
  • Improved logging of transformed element names.
  • Support for local Ant properties in Groovy expressions.
  • "antProject" is now part of the scope in all Groovy expressions.
  • Passwords are now automatically masked in all transformation logs (including verbose mode).

Changes to Existing Tasks/Commands

  • Import, copy and restore commands now stream files to DataPower, which speeds up file transfer and allows for supporting large files. Previously, DPBuddy loaded files in memory prior to import/copy.
  • Export task/command now supports sorting of the exported XML configuration file. DPBuddy will sort all DataPower configuration objects alphabetically according to their names and types. This feature can be used to make tracking of DataPower configuration changes easier since DataPower sometimes re-orders objects in the exported configuration.
  • Import task/command can now re-run import automatically in case of import errors. This feature should be enabled when importing the sorted configuration file created by the export command. DataPower requires configuration objects to be defined prior to being referenced at import. With the sorted export file, this is no longer the case, hence the import may need to be run twice.
  • Export command now automatically removes "export-manifest" from the exported xml files (can be controlled by the "defaultTransorm" option).
  • Import now supports "failOnError" option/attribute.
  • More efficient mkdir task/command -- a directory tree is now created in a single request.
  • assertState now prints error code/error description for objects in the "down" state.
  • Improved CLI help.
  • For DPBuddy CLI, config files can now reside in "/etc" directory.

To upgrade to this release, you can simply download and un-archive the distribution and point your DPBUDDY_HOME environment variable to the new location. If you're using DPBuddy from Apache Ant, you will also need to add <pathelement location="${dpbuddy.home}/conf"/> to the DPBuddy library's "taskdef" in your Ant files, otherwise, you will see verbose logging output in the console.

You can download DPBuddy 3.3 from this page.

DataPower Buddy Release 3.3 Beta

Posted on 03/15/2016 ,

We're pleased to announce the availability of DataPower Buddy 3.3 Beta.

This release introduces support for defining configuration properties/variables using HOCON (Human-Optimized Config Object Notation) format. HOCON is a superset of JSON, it is quite flexible (e.g., it supports comments, includes, substitutions) and it is more readable than raw JSON. HOCON provides a powerful alternative to defining environment-specific properties using prefix-based notation. The prefix-based mechanism, however, is still fully supported, so the use of HOCON is completely optional.

Other notable features of this release include:

  • Support for restore/import of multiple domains (or all of the domains). This could be useful for keeping multiple production appliances in sync.
  • Support for secure restore. This can also be used to maintain a DataPower cluster in sync or in a DR situation.
  • Support for appliance reboot/restart. Both secure restore and restart/reboot commands can optionally wait for the appliance to come back online.
  • Password encryption inside configuration files using open-source Jascrypt tool.
  • Under the hood, DPBuddy now streams files to the appliance during copy/import/restore, so these commands are now performed much faster and with lower memory requirements.
  • "Add" and "update" configuration transformation functions now support repeaters (loops). This can be used to generate environment-specific load-balancing group configuration with variable number of back-end servers.
  • Many minor changes and bug fixes. For example, passwords are now automatically masked when environment transformations run in verbose mode.
  • CLI help has been improved to make the use of CLI easier.

This release could also provide support for firmware 7.5; this feature will be finalized once 7.5 becomes available.

The general availability of DPBuddy 3.3 is expected in April 2016. Meanwhile, please let us know if you're interested in evaluating the beta version.