Understanding Docker Content Trust

Docker Content Trust (DCT) is a security mechanism that cryptographically signs container images and verifies those signatures before an image can be pulled or run. It prevents the execution of tampered, corrupted, or malicious images by establishing a chain of trust from the image publisher to the consumer. DCT is built on top of The Update Framework (TUF), a secure framework for metadata and content distribution, which is implemented by the Notary project. When DCT is enabled, Docker clients automatically enforce signature checks using encrypted keys stored in a Notary server, typically provided by Docker Hub or a private registry that supports Notary trust data.

This article provides a comprehensive guide to implementing image signing and verification with Docker Content Trust, including enabling DCT, signing images, verifying signatures, managing keys, integrating into CI/CD pipelines, and following best practices for production environments.

Prerequisites for Implementation

Before you begin, ensure your environment meets the following requirements:

  • Docker Engine 17.06 or later – DCT support is natively included. Earlier versions do not have integrated trust or require separate Notary client installation.
  • Docker Hub account or a private registry with Notary support – Both Docker Hub and many self-hosted registries (e.g., Docker Registry with Notary, Harbor, GitLab Container Registry) support DCT. For a private registry, ensure the Notary server is properly configured alongside the registry.
  • Signing keys – DCT uses a root key, an offline key responsible for signing the root metadata, and delegation keys. You will generate these keys during the signing process.
  • Notary CLI (optional) – While Docker CLI handles basic signing and verification, advanced key management (e.g., rotating keys, managing delegations) may require the notary command-line tool.
  • Network access to the trust server – Docker communicates with a Notary server (typically at notary.docker.io for Docker Hub) to upload and download trust data. Ensure your environment permits HTTPS traffic to the appropriate endpoint.

Enabling Docker Content Trust

Docker Content Trust is disabled by default. To enable it, set the environment variable DOCKER_CONTENT_TRUST to 1. This can be done globally or per-session:

export DOCKER_CONTENT_TRUST=1

Once enabled, every docker push command will automatically sign the image with your delegated signing key, and every docker pull and docker run command will verify that the image’s signatures and trust metadata are valid. If verification fails, the client refuses the operation and returns an error message.

You may also enable DCT for the entire Docker daemon by setting the variable in the Docker engine configuration file (/etc/docker/daemon.json) or by passing --content-trust=true when starting the daemon. However, the environment variable approach is more flexible and allows granular control per shell session or CI job.

It is important to note that enabling DCT does not require any changes to your image tags or registry configuration. The trust data is stored separately in the Notary server, keyed by image repository:tag.

Signing Images with Docker Content Trust

With DCT enabled, image signing occurs automatically during docker push. When you push an image, Docker generates a signing key (if one does not already exist) and uses it to sign the image manifest and any associated tags. The signed trust data, including cryptographic signatures and metadata, is uploaded to the Notary server along with the image layers.

To manually sign an existing image tag without pushing again, or to sign an image that was previously pushed without DCT, use the docker trust sign command:

docker trust sign :

This command creates a signature for the specified tag, associating it with your signing key. The signature is stored in Notary. You can verify the signing status with:

docker trust inspect --pretty :

Understanding signing keys: DCT uses a hierarchy of keys:

  • Root key – The most sensitive key; used to sign the root metadata. It should be kept offline and stored securely (e.g., on a hardware security module or encrypted offline storage).
  • Targets key – Used to sign target metadata, which maps tags to image digests. You typically use a delegation key for signing pushes.
  • Snapshot key – Used to sign snapshot metadata, which ensures the trust data is up-to-date.
  • Timestamp key – Used to sign timestamp metadata for freshness verification.

When you first enable DCT and push an image, Docker creates a local trust directory under ~/.docker/trust. This directory contains your private keys and trust data cache. It is recommended to back up the root key and store it offline to prevent loss of ability to sign future images.

Verifying Images

Automatic verification occurs on every docker pull or docker run when DOCKER_CONTENT_TRUST=1. The client performs the following steps:

  1. Fetches the trust data (root, targets, snapshot, timestamp) from the Notary server for the specified repository.
  2. Verifies the cryptographic signatures on each metadata file using the chain of keys.
  3. Checks that the tag requested matches a target in the target metadata.
  4. Resolves the tag to a digest and verifies that the digest matches the pulled image manifest.

If any step fails, Docker outputs an error such as Error: remote trust data does not exist or signature verification failed, and the image is not pulled or run.

To manually inspect trust data for an image, use:

docker trust inspect --pretty :

This command displays the list of signers, their roles, and the signed digests. You can also see the expiration dates of trust data. For deeper inspection, use the Notary CLI:

notary list /:

Verification is especially important in air-gapped or highly regulated environments where you must enforce that only images from specific, known signers are allowed.

Key Management and Rotation

Proper key management is critical for maintaining trust. Keys can be rotated to invalidate compromised keys or to adopt stronger cryptographic algorithms. Docker provides commands to manage keys:

  • Generate a new signing key: docker trust key generate – Creates a new delegation private key and stores it locally.
  • Load an existing key: docker trust key load --name – Imports a PEM-encoded private key.
  • Rotate the root key: docker trust key rotate – Replaces the root key and re-signs all metadata. This requires the old root key to be available.
  • Remove a delegation key: docker trust revoke --delegate – Invalidates a signing key by updating the targets metadata to exclude that key.

It is advisable to rotate keys at regular intervals (e.g., every 12 months) or immediately after a suspected compromise. When a key is rotated, all images signed with the old key remain valid if you also update the trust data to include the new key as a signer. However, to fully enforce new keys, you should re-sign all image tags using the new key.

For enterprise environments, consider using an external key management system (KMS) that integrates with Docker’s trust infrastructure, such as HashiCorp Vault or AWS KMS, to store root keys securely.

Integrating Docker Content Trust into CI/CD Pipelines

Automating image signing in CI/CD pipelines ensures that every built image is verified before deployment. Here is a typical workflow:

  1. Set up signing keys in CI environment: Inject the private signing key (targets key) into the CI runner via secure secrets. Avoid storing keys in the repository.
  2. Enable DCT: Set export DOCKER_CONTENT_TRUST=1 at the start of the pipeline.
  3. Build and push: Run docker build and docker push. Docker will automatically sign the image using the provided key.
  4. Verify in production: In the deployment stage (e.g., Kubernetes, Docker Swarm), ensure DCT is enabled on all nodes that pull images. This prevents running unsigned or compromised images.

Example GitHub Actions step:

steps: - name: Enable DCT run: echo "DOCKER_CONTENT_TRUST=1" >> $GITHUB_ENV - name: Push image run: docker push myregistry/myapp:latest env: DOCKER_CONTENT_TRUST: 1 (Note: when using YAML, set environment variable in the appropriate key).

For private registries, you may need to configure the Notary server URL. Set the environment variable DOCKER_CONTENT_TRUST_SERVER to the address of your Notary server (e.g., https://notary.mydomain.com).

Integrate trust verification into your deployment tools. For example, Kubernetes admission controllers can be configured to require images have valid trust data, using tools like Portieris or Grafeas.

Troubleshooting Common Issues

Even with a correct setup, you may encounter errors. Here are common problems and their solutions:

  • “No valid trust data for ” – This usually means the image was pushed without DCT enabled. Enable DCT and push again, or use docker trust sign to sign the existing tag.
  • “Failed to import trust key” – The private key file may be corrupted or not in PEM format. Ensure the key is generated with docker trust key generate or exported correctly from a KMS.
  • “Unable to reach trust server” – Network connectivity to the Notary server is blocked. Check firewall rules and ensure the trust server URL is correct. For Docker Hub, the server is notary.docker.io.
  • “Key not found: signing key is missing” – The local trust directory (~/.docker/trust) may have been deleted. Re-import the signing key from a backup or generate a new delegation key and re-sign the image.
  • “Snapshot or timestamp metadata is expired” – Notary metadata has a time-to-live. Use docker trust sign to refresh the snapshot and timestamp metadata, or set up a cron job to periodically rotate trust data.

If you are using a self-hosted Notary server, ensure it is running and accessible. Check the server logs for errors related to database storage or certificate issues.

Best Practices and Considerations

Implementing Docker Content Trust requires careful planning. Follow these best practices to maximize security:

  • Protect root keys offline – Store root keys on air-gapped machines or hardware security modules (HSMs). Do not store them on CI servers or development workstations.
  • Use delegation keys for signing – Assign separate delegation keys to different teams or pipelines. This allows granular revocation without affecting the entire repository.
  • Regularly rotate keys – Set a rotation policy (e.g., every 6 months). Automate the process using scripts that generate new keys and update trust data.
  • Enable DCT across the entire organization – Enforce DCT at the registry level (e.g., Docker Hub security settings) so that unsigned images cannot be stored or downloaded without explicit override.
  • Monitor trust data expiry – Notary metadata has expiration timestamps. Configure alerts to notify you before expiry, and re-sign images automatically to keep trust data fresh.
  • Integrate with vulnerability scanning – Combine image signing with container vulnerability scanning using tools like Trivy, Clair, or Docker Scout. Only sign images after a successful scan to ensure you are signing clean images.
  • Educate your team – Ensure developers understand the importance of signing and verification. Provide training on key management, the impact of enabling or disabling DCT, and how to handle signing errors.
  • Test your recovery procedures – Practice restoring from key backups and re-signing images after a simulated compromise. This ensures you can recover quickly from security incidents.

Conclusion

Docker Content Trust provides a robust mechanism for ensuring the integrity and authenticity of container images in your software supply chain. By enabling DCT, signing images with secure keys, and enforcing verification at pull and run time, you protect your environment from running tampered or malicious containers. Key management, while critical, can be handled effectively with automated rotation and offline storage. Integrating DCT into CI/CD pipelines ensures that every deployed image meets your organization’s security policy. As container adoption grows, implementing image signing and verification is no longer optional—it is a fundamental component of a secure container platform.