Skip to main content

Server 4.0.x-4.2.x Docker Layer Caching: EBS Volumes Are Not Deleting

How to Configure Docker Layer Caching Cleanup in AWS

Introduction

Docker Layer Caching (DLC) in Amazon Web Services (AWS) environments uses Elastic Block Store (EBS) volumes to store image layers for reuse across jobs, helping optimize build times. By default, these EBS volumes persist for 3 days after inactivity before being automatically cleaned up. This article explains how to customize the cleanup frequency for DLC volumes to better manage storage costs and resource usage in your CircleCI Server installation.

Prerequisites

Before configuring DLC cleanup settings, ensure you have:

  • CircleCI Server installed with AWS as your cloud provider

  • Administrative access to your CircleCI Server configuration

  • Projects currently using docker_layer_caching: true in their configuration

  • Access to modify your values.yaml file

Instructions

Step 1: Verify Docker Layer Caching Usage

First, confirm that your projects are actively using Docker Layer Caching:

  1. Check your project's .circleci/config.yml file

  2. Look for jobs that include docker_layer_caching: true in their configuration

  3. Verify that these jobs are using either machine executor or setup_remote_docker

Example configuration:

jobs:
  build:
    machine:
      image: ubuntu-2004:202010-01
      docker_layer_caching: true

Step 2: Access Your Values Configuration

  1. Locate your CircleCI Server values.yaml configuration file

  2. Open the file in your preferred text editor

  3. Find the vm_service block, or create one if it doesn't exist

Step 3: Configure DLC Cleanup Settings

Add or modify the dlc_lifespan_days setting under the vm_service block:

vm_service:
  dlc_lifespan_days: 1  # Set to desired number of days
  providers:
    # Your existing provider configuration

Note: Replace the number 1 with your preferred cleanup interval in days.

Step 4: Apply the Configuration Changes

  1. Save your values.yaml file

  2. Apply the updated configuration to your CircleCI Server installation

  3. Restart the relevant services as needed for your deployment method

Outcome

After completing these steps, your Docker Layer Caching volumes will be automatically cleaned up based on your specified timeframe. EBS volumes will be removed after the configured number of days of inactivity, helping you manage storage costs while maintaining caching benefits.

Additional Notes

Important Considerations:

  • Jobs that utilize DLC may experience fewer cache hits if you set cleanup to occur more frequently than cache creation

  • Shorter cleanup intervals reduce storage costs but may impact build performance

  • Consider your team's build frequency when setting the cleanup interval

  • Monitor your build times after making changes to ensure optimal performance

Recommended Settings:

  • For frequently building teams: 2-3 days

  • For occasional builds: 1-2 days

  • For storage cost optimization: 1 day

Additional Resources

Related Articles
How to Check Available Space on Remote Docker and Docker Layer Caching
How to delete a project's Docker Layer Cache
How to Debug Docker Layer Cache Misses in a Build
Docker Layer Caching (DLC) - Frequently Asked Questions
Docker Layer Caching (DLC)
Did this answer your question?