Hm... i've got a heavy lockfile-using run, and caching works on linux and osx:

https://github.com/robots-from-jupyter/robotframework-jupyterlibrary/runs/1493514408?check_suite_focus=true

It doesn't work on windows, and while it makes me sad, it also doesn't surprise me.

If you have links to the repo in question, or logs, or any additional information you could provide other than "doesn't work" that would help!

Thanks @bollwyvl for your quick reply.

Sorry for not providing more details earlier. I'm experimenting on a toy repo, you can find a recent log here:

https://github.com/ma-sadeghi/xyztest/runs/1534789528?check_suite_focus=true

The cache size is reported to be ~0MB.

PS. I'm not using setup-miniconda for this repo, I was trying to make it work with the built-in conda on CI machines.

@bollwyvl

Update 1

Looking at your yml file with a little digging into setup-miniconda, it turned out I was doing a whole bunch of things wrong. The cache folder (~/conda_pkgs_dir) was nonexistent, so I had to manually create it, also I wasn't setting CONDA_PKGS_DIRS environment variable to point to ~/conda_pkgs_dir. Here's the most recent iteration:

https://github.com/ma-sadeghi/xyztest/runs/1535236467?check_suite_focus=true

which seems like caching is working fine and the "installing dependencies" step is now much faster.

Thanks for the insight.

Huzzah! Cache for everybody!

Note that caching the entire folder will not work as well as it could, as
the cache contains both the tarballs _and_ the unpacked contents... In our
post step step we (try) to clean some of that up.

Feel free to close if there's nothing else broken... especially as this
wasn't involving this action.

@bollwyvl Sorry I know this is entirely out of the scope of this repo, but may I ask what exactly is it that you do to optimize caching? Which files do you get rid of? the unpacked contents or the tarballs?

Yeah that's pretty much the size of it:

https://github.com/conda-incubator/setup-miniconda/blob/master/src/delete.ts

There's some more stuff in there, too, but it's piddly stuff in comparison
to, say, mkl or qt unpacked.

@bollwyvl Quick question: looking at delete.ts, it seems that you're deleting the uncompressed folders, rather than the .tar.bz2 compressed files. I'm probably missing something, but it seems that by doing so, conda needs to do an additional "decompress" step, whereas if .tar.bz2 are deleted, conda can simply reuse the already-decompressed folders. I tried it on my computer and it seemed that the latter led to faster installs.

I think you could run conda clean as well. @bollwyvl maybe we could use that instead of delete.ts, I did it like that because I was not aware of conda clean at that moment.

$ conda clean --help



usage: conda clean [-h] [-a] [-i] [-p] [-t] [-f]
                   [-c TEMPFILES [TEMPFILES ...]] [-d] [--json] [-q] [-v] [-y]

Remove unused packages and caches.

Options:

optional arguments:
  -h, --help            Show this help message and exit.

Removal Targets:
  -a, --all             Remove index cache, lock files, unused cache packages,
                        and tarballs.
  -i, --index-cache     Remove index cache.
  -p, --packages        Remove unused packages from writable package caches.
                        WARNING: This does not check for packages installed
                        using symlinks back to the package cache.
  -t, --tarballs        Remove cached package tarballs.
  -f, --force-pkgs-dirs
                        Remove *all* writable package caches. This option is
                        not included with the --all flag. WARNING: This will
                        break environments with packages installed using
                        symlinks back to the package cache.
  -c TEMPFILES [TEMPFILES ...], --tempfiles TEMPFILES [TEMPFILES ...]
                        Remove temporary files that could not be deleted
                        earlier due to being in-use. Argument is path(s) to
                        prefix(es) where files should be found and removed.

Output, Prompt, and Flow Control Options:
  -d, --dry-run         Only display what would have been done.
  --json                Report all output as json. Suitable for using conda
                        programmatically.
  -q, --quiet           Do not display progress bar.
  -v, --verbose         Can be used multiple times. Once for INFO, twice for
                        DEBUG, three times for TRACE.
  -y, --yes             Do not ask for confirmation.

Examples:

    conda clean --tarballs

Some more ideas for a smaller cache:

• https://jcristharif.com/conda-docker-tips.html
• https://pythonspeed.com/articles/conda-docker-image-size/