Skip to content

Add apt retry config to Dockerfile to handle transient mirror sync failures#155

Merged
vharseko merged 2 commits intocopilot/make-rest-context-path-configurablefrom
copilot/add-apt-retries-to-dockerfile
Apr 10, 2026
Merged

Add apt retry config to Dockerfile to handle transient mirror sync failures#155
vharseko merged 2 commits intocopilot/make-rest-context-path-configurablefrom
copilot/add-apt-retries-to-dockerfile

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 9, 2026

apt-get update fails intermittently on linux/ppc64le due to Ubuntu mirror sync races (File has unexpected size … Mirror sync in progress?), breaking the multi-platform Docker build.

Changes

  • Dockerfile: Write an apt config file before apt-get update to enable 3 automatic retries on fetch failures:
RUN echo 'Acquire::Retries "3";' > /etc/apt/apt.conf.d/80-retries \
 && apt-get update \
 && apt-get install -y --no-install-recommends curl unzip \
 ...
  • Dockerfile-alpine: No change — uses apk, unaffected.
Original prompt

Problem

The build-docker job is failing because apt-get update encounters a transient Ubuntu mirror sync issue on the linux/ppc64le platform. The error is:

E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/jammy-security/universe/binary-ppc64el/Packages.gz
File has unexpected size (1151119 != 1151100). Mirror sync in progress?

This causes the entire Docker multi-platform build to fail.

Failing job: https://github.com/OpenIdentityPlatform/OpenIDM/actions/runs/24202286864/job/70648875986?pr=142

Solution

Use apt's built-in Acquire::Retries option to add retry logic to the Dockerfile. This handles transient mirror issues gracefully without needing a shell retry loop.

Changes required in Dockerfile

On line 27, prepend a command that configures apt retries before running apt-get update:

Before:

RUN apt-get update \
 && apt-get install -y --no-install-recommends curl unzip \

After:

RUN echo 'Acquire::Retries "3";' > /etc/apt/apt.conf.d/80-retries \
 && apt-get update \
 && apt-get install -y --no-install-recommends curl unzip \

The rest of the RUN command (lines 29-38) should remain unchanged. Only add the echo 'Acquire::Retries "3";' > /etc/apt/apt.conf.d/80-retries && prefix before apt-get update.

Also apply the same fix to Dockerfile-alpine if it uses apt-get update

Check Dockerfile-alpine — if it uses apk instead of apt, no change is needed there.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: Please find a solution for this failing job. Use the logs, job definition, and any referenced files where the failure occurred. Keep your response focused on the solution and include code suggestions when appropriate.

Assistant:

I'll start by fetching the job logs and the workflow definition to understand the failure.

Assistant: ## Failing Job Analysis

The build-docker job is failing due to a transient Ubuntu mirror sync issue on the linux/ppc64le platform during the Docker multi-platform build.

Root Cause

The error is:

E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/jammy-security/universe/binary-ppc64el/Packages.gz
File has unexpected size (1151119 != 1151100). Mirror sync in progress?

This is a well-known transient issue where the Ubuntu package mirror is mid-sync and the apt-get update fails because the downloaded index file doesn't match the expected checksum/size. The apt-get update failure causes the entire RUN layer to fail, which causes the Docker build to fail for linux/ppc64le. Since the image wasn't pushed to the local registry, the subsequent docker run test also fails with manifest unknown.

Solution

There are a few approaches to make this more resilient:

Option 1: Add retry logic to apt-get update in the Dockerfile (Recommended)

In Dockerfile at line 27, replace the single apt-get update with a retry loop:

RUN for i in 1 2 3; do apt-get update && break || sleep 15; done \
 && apt-get install -y --no-install-recommends curl unzip \
 && bash -c 'if [ ! -z "$VERSION" ] ; then rm -rf ./*.zip ; curl -L https://github.com/OpenIdentityPlatform/OpenIDM/releases/download/$VERSION/openidm-$VERSION.zip --output openidm-$VERSION.zip ; fi' \
 && unzip openidm-*.zip && rm -rf *.zip \
 && apt-get remove -y --purge unzip \
 && rm -rf /var/lib/apt/lists/* \
 && useradd -m -r -u 1001 -g root $USER \
 && install -d -o $USER /opt/openidm \
 && chown -R $USER:root /opt/openidm \
 && chgrp -R 0 /opt/openidm \
 && chmod -R g=u /opt/openidm \
 && chmod +x /opt/openidm/*.sh

Option 2: Use Acquire::Retries for apt

A cleaner apt-native approach:

RUN echo 'Acquire::Retries "3";' > /etc/apt/apt.conf.d/80-retries \
 && apt-get update \
 && apt-get install -y --no-install-recommends curl unzip \
 ...

Option 3: Re-run the workflow

Since this is a transient mirror issue, simply re-running the failed job may succeed if the mirror has finished syncing. However, this doesn't prevent future occurrences.

Additional Note

The workflow at line 147 already has continue-on-error: true on the build step, but the subsequent Docker test step (line 157) does not have continue-on-error and doesn't check whether the image was actually built before trying to run it. You could guard the test:

      - name: Docker test
        shell: bash
        run: |
          docker pull localhost:5000/${GITHUB_REPOSITORY,,}:${{ env.release_version }} && \
          docker run --rm -it -d --memory="1g" --name=test localhost:5000/${GITHUB_REPOSITORY,,}:${{ env.release_version }} && \
          ...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

Copilot AI changed the title [WIP] Add apt retry logic to handle transient Ubuntu mirror issues Add apt retry config to Dockerfile to handle transient mirror sync failures Apr 9, 2026
Copilot AI requested a review from vharseko April 9, 2026 17:15
@vharseko vharseko changed the base branch from copilot/make-rest-context-path-configurable to master April 9, 2026 18:00
@vharseko vharseko changed the base branch from master to copilot/make-rest-context-path-configurable April 10, 2026 08:51
@vharseko vharseko marked this pull request as ready for review April 10, 2026 08:51
@vharseko vharseko requested review from maximthomas and removed request for vharseko April 10, 2026 08:52
@vharseko vharseko merged commit 5f5835a into copilot/make-rest-context-path-configurable Apr 10, 2026
14 checks passed
@vharseko vharseko deleted the copilot/add-apt-retries-to-dockerfile branch April 10, 2026 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants