Manage GitHub Artifact Storage Quota

If you are reading this post, I hope you have already faced the error: “Create Artifact Container failed: Artifact storage quota has been hit. Unable to upload any new artifacts“. If not and if you are about to set up GitHub actions for your repository, read this article before you start. It is better to manage your GitHub artifact storage quota wisely. Otherwise you will have to wait for the next billing cycle for a quota reset and keep your CI/CD blocked.

What is GitHub Artifact Storage Quota?

GitHub: the famous code hosting platform stretches beyond being just a hosting platform. GitHub Actions is one such addition that automates workflows in a repository. Though GitHub is generous enough to provide free storage and computing power, there is a limit on your free meal. For more details about the quota, refer to the official document. You are going beyond your quota? you will get the abovementioned error.

How to Stay Within The Quota Limit?

This article focuses on the storage limitation though similar precaution has to be taken on computation too. Let’s say you have a pretty active development team or a community that keeps pushing changes every hour. Suppose your workflow builds an uber jar and packs them into docker containers using two workflow operations; you may choose to upload the uber jar to the artifact storage at the end of the build operation and download it in the deploy operation.

If your account is a GitHub Free account (with a 500 MB storage limitation) and your uber jar size is around 100 MB, you can only have five artifacts in the artifact storage. If you didn’t clear the previous builds, your sixth build would fail to upload the artifact with the following error: Create Artifact Container failed: Artifact storage quota has been hit. Unable to upload any new artifacts.

There are two different options to fix this problem automatically.

1. Setting up the maximum retention period for artifacts and logs
2. Delete old artifacts in your workflow before publishing new artifacts

Artifact and Log Retention

GitHub provides an option to set the retention period for artifacts and logs for each project. By default, GitHub keeps the artifacts and build logs for 90 days, but you can adjust this value based on your GitHub subscription. Based on the frequency of workflow execution, you may find it useful or not. For example, if your artifacts are small enough and your workflows are executed only once a day, deleting them after some days may be a good option. On the other hand, if your artifacts are too big and built hourly (or frequent enough to eat all your storage within a day), even one day retention period wouldn’t work.

Step 1:
To adjust the retention period, go to the Settings of your repository.

Step 2:
Expand the Actions section, and navigate to the sub-section: General.

Here you can change the retention period. If your repository is under an organization, you can set the retention period for the whole organization. In that case, the maximum retention period of any repository in that organization cannot exceed the retention period of the organization.

Delete Old Artifacts

Suppose this is your workflow definition. Note that the artifact is named “package” and uploaded to the artifact storage in the last step.

name: CI

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Set up JDK 8
        uses: actions/setup-java@v3
        with:
          java-version: "8"
          distribution: "adopt"
          cache: maven

      - name: Build with Maven
        run: mvn clean package

      - name: Copy Artifact
        run: mkdir staging && cp target/*.jar staging

      - name: Upload Artifact
        uses: actions/upload-artifact@v3
        with:
          name: package
          path: staging

Step 1:
To delete the old artifacts, add the following step just above the “Upload Artifact” step. This step lists all the artifacts from previous builds and deletes only the artifacts named “package”. Feel free to remove the filtering logic if you want to delete all the artifacts. Refer to the GitHub Actions API to learn more about the properties.

      - name: Delete Old Artifacts
        uses: actions/github-script@v6
        id: artifact
        with:
          script: |
            const res = await github.rest.actions.listArtifactsForRepo({
              owner: context.repo.owner,
              repo: context.repo.repo,
            })

            res.data.artifacts
              .filter(({ name }) => name === 'package')
              .forEach(({ id }) => {
                github.rest.actions.deleteArtifact({
                  owner: context.repo.owner,
                  repo: context.repo.repo,
                  artifact_id: id,
                })
              })

After making the changes, your workflow must look like this:

name: CI

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Set up JDK 8
        uses: actions/setup-java@v3
        with:
          java-version: "8"
          distribution: "adopt"
          cache: maven

      - name: Build with Maven
        run: mvn clean package

      - name: Delete Old Artifacts
        uses: actions/github-script@v6
        id: artifact
        with:
          script: |
            const res = await github.rest.actions.listArtifactsForRepo({
              owner: context.repo.owner,
              repo: context.repo.repo,
            })

            res.data.artifacts
              .filter(({ name }) => name === 'package')
              .forEach(({ id }) => {
                github.rest.actions.deleteArtifact({
                  owner: context.repo.owner,
                  repo: context.repo.repo,
                  artifact_id: id,
                })
              })

      - name: Copy Artifact
        run: mkdir staging && cp target/*.jar staging

      - name: Upload Artifact
        uses: actions/upload-artifact@v3
        with:
          name: package
          path: staging

Step 2:
Save the changes and push the changes to the repository.

Any workflow triggered after these changes will delete the previous artifacts but leave the build logs. Combining this with the retention period for build logs will keep your repository within the storage limit.

Have you found this article useful? Please let me know below in the comments. Knowing someone found my articles useful motivates me to write more. Also, comment below if you face any issues with following this article or getting it working. I will try my best to help you resolve the problem. The Java Helps community is also willing to help each other and grow together.