Libelektra: Jenkins: docker pull fails with unexpected EOF

Ok, thank you for fixing it! Then it does not make sense for me to clear the docker registry.

I'll try rebuilding and see if everything works now.

I've seen this error before when I first tried to add a new agent. At the time it was the hetzner-jenkins1 agent. The mistake just went away after few retries, I had no idea why.

I cloned libelektra and executed the following on the jenkinsNode3:
docker build libelektra/scripts/docker/debian/stretch/.

Usually, agents are reusing the local image, once it is pulled, but I have no idea why would pulling from hub throw an occasional error.

The images are rebuilt every month to my knowledge (such that software inside the images is kept somewhat up-to-date). This happened also for December now, so it's possible that the newest image was not already cached locally.

I hope this (monthly rebuilding of all images) still happens, as @Mistreated did not add the libelektra-monthly build job.

@ingwinlu do you still know what the monthly build job did and why it did not have a Jenkinsfile?

I recreated the old Jenkins server. I can now recreate the monthly job.

I can now recreate the monthly job.

Done.
I do not know it's exact steps, but I will look into them in the evening.

I recreated the old Jenkins server. I can now recreate the monthly job.

Good job, it might be helpful to keep the old Jenkins server for a while. Simply stop the container after use and do not start it on bootup.

Now also push fails with device or resource busy (I have seen this before): https://build.libelektra.org/blue/organizations/jenkins/libelektra/detail/PR-3319/2/pipeline

On jenkinsNode3VM:

docker push hub.libelektra.org/build-elektra-website-backend:PR-3319_2
[...]
860ee8d82838: Retrying in 1 second
6bbb813c7d87: Retrying in 1 second
error creating overlay mount to /var/lib/docker/overlay2/8535169ca4de05e069978de34233d82158d1831fb4d980772411f59de2d370a5/merged: device or resource busy
script returned exit code 1

Maybe increasing the timeout in the nginx of a7 helps: https://github.com/moby/moby/issues/22188#issuecomment-328011573

I increased the timeout, let's see.

Increasing the timeout did not help. I noticed that (only) jenkinsNode3VM was running debian packaged docker 18.x so I upgraded to the upstream docker packaged 19.x version (which was used by all other nodes already).

jenkinsNode3VM was now able to push and pull images without a problem. I hope it remains like that.

Seems that this was fixed.