Yarn cache is not updated when npm updates are grouped with Docker image updates

[eslawski]

How are you running Renovate?

A Mend.io-hosted app

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

No response

Please tell us more about your question or problem

When updates to npm dependencies (using yarn zero-installs) are grouped with updates to docker images, we've observed that the yarn cache is not properly regenerated. This is troublesome as it requires the developer to pull down the branch, run a yarn install to generate the necessary cache files, and push back the changes.

Furthermore, we've noticed this bug can be intermittent. However, once a PR gets into this state, it seems to consistently exhibit the issue.

This can be observed in this minimal reproduction repo.

Skimming the logs, we do notice see Enabling global cache as zero-install is not detected which feels highly related.

Logs (if relevant)

No response

[rarkins]

Can you modify the reproduction that there's a standalone PR containing only an npm update which does have the cache updated?

[eslawski]

@rarkins Sure thing! Here is an example PR where yarn cache is correctly updated in that repo: eslawski/yarn-zero-install-and-docker-image-bug#8

[rarkins]

Here's the log from the branch which updates the offline cache:

DEBUG: Executing command (branch="renovate/example-pr-that-correctly-updates-yarn-cache")
{
  "command": "yarn install --mode=skip-build"
}

...

DEBUG: updateYarnOffline resolvedPaths (branch="renovate/example-pr-that-correctly-updates-yarn-cache")
{
  "resolvedPaths": [
    ".yarn/cache",
    ".pnp.cjs",
    ".pnp.js",
    ".pnp.loader.mjs"
  ]
}

DEBUG: Updated 22 lock files (branch="renovate/example-pr-that-correctly-updates-yarn-cache")
{
  "updatedArtifacts": [
    "yarn.lock",
    ".pnp.cjs",
    ".yarn/cache/@vue-compiler-core-npm-3.5.13-b0c91e9970-22f042bb47.zip",
    ".yarn/cache/@vue-compiler-dom-npm-3.5.13-8785a9ba19-5dc628c520.zip",
    ".yarn/cache/@vue-compiler-sfc-npm-3.5.13-3213a0466e-08d55bbdbe.zip",
    ".yarn/cache/@vue-compiler-ssr-npm-3.5.13-e1cdeef007-09f2706455.zip",
    ".yarn/cache/@vue-reactivity-npm-3.5.13-7bf539a4bd-e4db379fad.zip",
    ".yarn/cache/@vue-runtime-core-npm-3.5.13-6c90bdb869-55ef3ec9ef.zip",
    ".yarn/cache/@vue-runtime-dom-npm-3.5.13-d22a720eb3-f32e52b08c.zip",
    ".yarn/cache/@vue-server-renderer-npm-3.5.13-d23f3ccee8-1da86b265d.zip",
    ".yarn/cache/@vue-shared-npm-3.5.13-519178ab33-5c0c24f443.zip",
    ".yarn/cache/vue-npm-3.5.13-59cb3fae80-bcab4ca74c.zip",
    ".yarn/cache/@vue-compiler-core-npm-3.5.7-ad571caa0a-9f3aaa1b3e.zip (delete)",
    ".yarn/cache/@vue-compiler-dom-npm-3.5.7-6038a1699d-306c1c681b.zip (delete)",
    ".yarn/cache/@vue-compiler-sfc-npm-3.5.7-f97f728985-307071f21d.zip (delete)",
    ".yarn/cache/@vue-compiler-ssr-npm-3.5.7-1d2631336b-a4d437c7a7.zip (delete)",
    ".yarn/cache/@vue-reactivity-npm-3.5.7-23ff297b21-0ce685814f.zip (delete)",
    ".yarn/cache/@vue-runtime-core-npm-3.5.7-c79c928e83-90de256c6c.zip (delete)",
    ".yarn/cache/@vue-runtime-dom-npm-3.5.7-11aa5e2605-2af5cb3d6b.zip (delete)",
    ".yarn/cache/@vue-server-renderer-npm-3.5.7-ab53ee1962-860434a75f.zip (delete)",
    ".yarn/cache/@vue-shared-npm-3.5.7-757f98a0ac-efd70fac34.zip (delete)",
    ".yarn/cache/vue-npm-3.5.7-6cdd2a5d6f-9e019d7b78.zip (delete)"
  ]
}

Here's the other one:

DEBUG: Executing command (branch="renovate/example-pr-that-does-not-update-yarn-cache")
{
  "command": "yarn install --mode=update-lockfile"
}

...

DEBUG: updateYarnOffline resolvedPaths (branch="renovate/example-pr-that-does-not-update-yarn-cache")
{
  "resolvedPaths": [
    ".yarn/cache",
    ".pnp.cjs",
    ".pnp.js",
    ".pnp.loader.mjs"
  ]
}

DEBUG: Updated 1 lock files (branch="renovate/example-pr-that-does-not-update-yarn-cache")
{
  "updatedArtifacts": [
    "yarn.lock"
  ]
}

So this seems to be related to skipInstalls being overridden when the first upgrade in the branch doesn't have it set.

How many files do you expect to be updated in the reproduction branch? I tested a fix locally but only see one extra file: https://github.com/renovate-reproductions/35127/pull/2/files

[eslawski]

I expect about a dozen or so additional files under .yarn/cache directory and the .pnp.cjs file for the reproduction branch. Similar to how other branch works.

[eslawski]

I think I might have just come across a similar bug, and wanted to share.

Setup:

• Group yarn and helm dependencies
• The yarn dependencies are already within range so Renovate does not suggest updates
• There is an update for the helm dependency
• The resulting PR regens the yarn cache, but DOES NOT actually update the helm dependency

Can create another minimal reproduction repo if that is helpful.

I am not super familiar with skipInstalls, but is that specific to the dependency manager or the PR?

[rarkins]

skipInstalls is something specifically for npm and Yarn

[eslawski]

Alright false alarm on my latest message. I cannot reproduce what I described above. I must had had a faulty setup or looking at stale data or something. Will keep an eye on it. For posterity, this is the minimal reproduction repo was working in and unable to reproduce what I descibed.

My apologies for distracting from the original issue.

[rarkins]

As a workaround, please configure skipInstalls=false for all updates in this branch so that a full Yarn install is done and not a lockfile-only one. If problems still persist, let's solve them after

[eslawski]

Hey unfortunately it appears that skipInstalls=false does not work as a workaround for this issue. I updated the sample reproduction repo to use it, and I am not seeing the yarn cache generated like I would expect.

[eslawski]

Just wanted to bump this discussion and highlight that the skipInstalls=false workaround does not appear to resolve this issue.

[jamietanna]

@eslawski just to confirm, which PR(s) on the minimal repro issue are you seeing this not working with?

[eslawski]

@jamietanna This is the problematic PR from the reproduction repo I was working with most recently. Issue being, that no yarn cache files are generated. Compared to a PR like this that correctly generates the yarn cache files.

[drewgingerich]

Hey there! I'm taking over for Evan, and we're still seeing this issue. Any new developments on what might be happening?