related request: #2566

I have started work on this.
Considering the comments on #2566, are we looking for --values or --set (or both?)

A nice use case is described in the issue I opened ( #3198 ), in which I'd like to be able to do:

helm package --version 1.2.3 --set image.tag 1.2.3

This way the chart version and the docker image version are one and the same.

I have just completed the first draft and posted a PR for this.

@ngeor , @sslavic the PR I submitted adds both --set and --values options, could you please take a look?

Hi @adshmh , looks good; unfortunately I don't know Go but I see you covered it with the test cases and all, so it probably does what it says :) Thanks!

When will this feature be available? In 2.9.0 ?

3471 missed the deadline for 2.9 so it'll be in 2.10.

Re-opening this, as #3471 needs to be backed out.

What does this do to the comments/etc in the archive package? eg, are they stripped or left alone or ??

Currently I'm trying to do the same thing with yq but it just dumps the final values after processing. ie, all the comments, blank lines for organisation, etc are gone.

3471 stripped all the comments out, which is why we needed to back that PR out. See https://github.com/kubernetes/helm/pull/3471#issuecomment-381779783 for more info

Reverted from 2.10 "feat: add --set and --values options to 'helm package'" https://github.com/helm/helm/commit/7b8aae466761448522acbd3beb2a16ad2283013a

Some news about this?
Thanks

Same as above; nobody's worked on a follow-up to fix the bugs in #3471 so this has not been implemented. You're more than welcome to fork Helm and compile against #3471 if you're okay with comments being stripped from the values file.

Is this done ? I tried on versions 2.11.0 and 2.12.0 none of them seems to have helm package --set...

+1. it will really help us in CI for stopping pasing and upgarding yamls of values before packing.

+1

--set-string can also be useful, why not allowing all of the override options that exists in install/upgrade.
Simply execute the same code used to override the values for the package command, it makes sense that the package will have the override values - those flags should be shared just like with install/upgrade.

2.13.1 does not have such feature either, is there any time table for it?

As mentioned earlier, nobody is currently working on this and there's no timetable for when a fix will be available. Feel free to take this one on. Have a look at #3471 for ideas.

implemented in Helm 3. Closing!

Looking at the code, it appears #3471 accidentally made its way into Helm 3, but was never pulled out as we did for Helm 2 due to this comment: https://github.com/helm/helm/pull/3471#issuecomment-381779783

I'm re-opening this feature request as we are removing the --set and --values flags in Helm 3 with https://github.com/helm/helm/pull/7201. They never worked anyways... Anything set via --set or --values were never injected into the package, and it introduced some fatal regressions with helm package, namely the fact that it stripped out comments from values.yaml, which several community members use as documentation.

Maybe I am misunderstanding, but I've happily been using --set with package to set values and it's worked great from 3.0.0 to 3.0.2, definitely not a "no-op" as described in #7201 .

My pipelines are broken since 3.0.3. This was working in 3.0.2. Why isn't this considered useful, if we have --version and --app-version. How are we supposed to edit the image and tag , to match --app-version? Bash ?

Looking at the code again, you are correct @lukaslarson and @maxhillaert. However, this was still a feature that was intended to be reverted as it stripped all comments from the package's values file. It was removed in Helm 2, but accidentally made its way into Helm 3. It was never intended to be shipped.

If someone wants to work on re-implementing #3471 that retains comments, we'd be happy to look at PRs.

@bacongobbler This broke some stuff for us. I'd be interested in working on a PR, however it seems non-trivial, so I'd like to check the approach.

It looks like gopkg.in/yaml.v3 supports retaining comments when going round trip. My proposal would be to make Values a yaml Node. Then, when coalescing values, walk the tree of parsed output rather than nested maps. This is certainly less ergonomic but the best way to preserve comments afaict.

This is a bit of a problem because we're using sigs.k8s.io/yaml everywhere which is pinned to v2 of the underlying lib. Minimally, this would require introducing it as a separate dependency, which is kind of yuck. Not sure what the best approach here is to avoid touching everything that deals with yaml.

The best option looks to be migrating to yaml.v3, so it sounds like you're on the right track.

If you want to start to work on a PoC/fork of Helm that uses yaml.v3 instead of sig.k8s.io/yaml, that would be a good path forward. That way we can begin to evaluate its functionality compared to what's in master.

Once we've established that yaml.v3 is the best path forward, we can figure out how to proceed from the SDK's standpoint.

Does that help unblock you?

@jmcelwain Are you actively working on this? I will gladly take this over if not.

@sreya92 Got busy at work. Here's where I left off: switching to yaml.v3 was mostlyeasy, with the exception of this bit here. Wasn't sure what the best approach to handle this was without going as far as vendoring the dependency and updating it to v3 there.

It looks like there was some work done to upgrade, but there's no other information in issues, etc., and I didn't bother to open an issue. The other option would be just keeping both dependencies, but that seems gross.

My personal opinion would be that this is a blocker for this issue, unless maintainers feel that carry around two copies of similar libs is worthwhile. Not sure what the LoE is here, but we should probably work to get github.com/ghodss/yaml to v3 and get the vendored version in k8s upgraded as well. That would make this all much easier.

Opened https://github.com/ghodss/yaml/issues/61 to inquire about possibility of an upgrade path there. Will wait to hear back from them before continuing anything else -- upgrading dependencies is way preferable to other options IMO.

@jmcelwain There is not a large amount of code in https://github.com/ghodss/yaml, there are unserviced pull requests from 2017: https://github.com/ghodss/yaml/pulls, and the code is MIT licensed.

Is this dependency really pulling its weight? It is a few marshalling helpers but is managing to hold up an issue which when resolved could improve the ability for many projects to dynamically set package defaults at build-time (our use case is to not have to maintain multiple divergent places to store version tags something like --set image.tag=${GIT_TAG} to lockstep chart and image versions for a project).

Can I suggest we do not wait any longer and just copy the necessary code into helm (I wouldn't bother vendoring it just absorb and adapt as necessary). I think it is a big enough project to handle the maintenance of this wrapper and YAML marshalling is a core competency (not saying you should maintain your own parser, but...!)

Since this is causing us pain right now (and stopping us upgrading from helm 3.0.2) I would be happy to take a stab at it.

I’d prefer to avoid that situation. There is not enough maintainers to maintain both a package manager AND a YAML parser. That would introduce a significant maintenance burden to the team.

@bacongobbler it is _not_ a yaml parser. It uses gopkg.in/yaml.v2 to do the actual parsing. It's about 900 lines of reflect wrapper code around that parser. The problem is that they dont' use gopkg.in/yaml.v3 that would resolve this issue.

It seems to me the maintenance burden of using this minor intermediate to your actual dependency is a greater burden that maintaining this bit of code.

As mentioned earlier in the thread, please feel free to propose a PR upstream that would update ghodss/yaml to yaml.v3. If they will not accept it, feel free to fork the project and maintain it if you feel you can take on the maintenance burden.

Let me be clear though: we _will not_ accept pull requests that fork and vendor ghodss/yaml into Helm. We cannot take on the additional maintenance burden of an entire Go library just to support one feature.

@bacongobbler please take a look at my PR here: https://github.com/helm/helm/pull/7963

It is 900 lines of code with the tests included and superfluous functionality removed.

Right now you are depending on what looks like a relatively unmaintained library with a much larger footprint than what I have added there.

an entire Go library just to support one feature

You use all of one function from this entire Go library, and in order to do that you have gone to the effort to create a permanent fork that further obscures where the code is coming from.

feel free to fork the project and maintain it if you feel you can take on the maintenance burden.

I mean you already run a single-purpose fork of this code. If the project is abandoned then you are not actually benefiting from any maintenance from the original author.

If you would like I can take that small amount of code in my PR and put it up in a repo and say I'll maintain it (and I will attempt to do so), but it seems rather silly.

Let me be clear though: we will not accept pull requests that fork and vendor ghodss/yaml into Helm.

For the record, I'd already pushed that PR by the time I read this, so I hope you will reconsider because I think you are having a cached thought about 'vendoring' and 'entire Go library' (which I might often agree with) without looking at the code involved. But let's discuss the actual code in question on the PR. I am happy to be a code owner for this code if that helps. Let's imagine I wrote it as a contribution... I sort of did.

Maybe they will merge it: https://github.com/ghodss/yaml/pull/62

The other alternative I briefly explored was the possibility of replacing the YAML to JSON conversion via some other dependency or serializing through some intermediate format. I'm not familiar enough with the Go ecosystem to know if this is a viable path -- I was somewhat surprised I couldn't find more serialization-type libs when I looked briefly.

Any idea on when we can have the --set back as we use this to automate helm packaging. It was in Helm 3.0.2 and now since updating is no longer supported.

We are still blocked on getting https://github.com/ghodss/yaml/pull/62 merged.

Looking at the code again, you are correct @lukaslarson and @maxhillaert. However, this was still a feature that was intended to be reverted as it stripped all comments from the package's values file. It was removed in Helm 2, but accidentally made its way into Helm 3. It was never intended to be shipped.

If someone wants to work on re-implementing #3471 that retains comments, we'd be happy to look at PRs.

Does it matter if comments are removed, If there is a caveat of using --set I don't care about comments.

We are still blocked on getting ghodss/yaml#62 merged.

I guess were are stuck with using 3.0.2 :(

3471 stripped comments regardless if you used --set or not, so it broke users who never used this feature. It broke backwards compatibility for everyone. Therefore it cannot be re-introduced with a caveat, and which is why we're waiting on a proper fix upstream.

which is why we have to stick on 3.0.2

We are still blocked on getting ghodss/yaml#62 merged.

But since it seems that this dependency is no more maintained, maybe it would be better to use a forked version, wouldn't it?

Switching that is a good idea -- but must be deferred to Helm 4. Someone should file a special issue for switching YAML parsers so we can track that for the Helm 4 process.

Let me be clear though: we will not accept pull requests that fork and vendor ghodss/yaml into Helm. We cannot take on the additional maintenance burden of an entire Go library just to support one feature.

Given it seems https://github.com/ghodss/yaml is no longer maintained, and there's no sign on life on https://github.com/ghodss/yaml/pull/62, is it worth reconsidering this position, and re-evaluating https://github.com/helm/helm/pull/7963 ?

We are not interested in taking on the maintenance burden of supporting an entire YAML parsing library just for one feature.

I would urge for the community to consider working with the existing maintainers of ghodss/yaml to find a new set of maintainers willing to support the project, fork the project, or find a new library that can preserve comments.

So what would it be?

• find new maintainers to support ghodss/YAML !?
• find someone to fork ghodss/YAML and use YAML.v3 (and continue to support it) !!
• use YAML.v3 directly
• propose to switch YAML parsers for helm 4

an entire YAML parsing library

This misrepresents the scope of the code involved here. It's a few wrappers around other parsers.

We are not interested in taking on the maintenance burden of supporting an entire YAML parsing library just for one feature.

Perhaps this confusion explains a lot - the parser is https://github.com/go-yaml/yaml, not ghodss/YAML. From https://github.com/ghodss/yaml:

A wrapper around go-yaml

So the simple solution is : just stop using the wrapper.

The "wrappers" have a profound influence on some of the behaviors. Having switched parsers (and wrapper libraries) a few times early in the project, the core maintainers are keenly aware of the small discrepancies between them, an how those produce high-level compatibility issues.

There is a possibility that we might consider using a fork of ghodss/yaml IF AND ONLY IF the owners of that fork were committed to continued maintenance.

Perhaps I can be more precise - stop using the abandoned wrapper library, and do the equivalent/identical wrapping behavior in helm, which is what https://github.com/helm/helm/pull/7963 does in about ~250 LOC plus tests.

Yes, ideally instead someone would fix in ghodss/yaml, but it's no longer maintained. Surely the alternative isn't waiting until Helm 4?

I think forking is probably the best option, IMO; @ghodss doesn't appear to have had any activity on GH since early 2019, and the last commit to the repo was a year and a half ago (by @bradfitz). The last (only) release was in 2017. If there are any latent security issues in the code, they're not going to get fixed on main.

I'd strongly advocate adopting a fork if someone is willing to commit to maintaining it; I don't see any downsides relative to continuing to use a moribund and abandoned package. I agree that moving to a different wrapper makes more sense as a major change (seems like a heavy enough lift that it could easily be a Helm 4 thing, but I'm not going to speculate further), but I think this could be solved with a replace line in go.mod.

I'm willing to commit to help co-maintaining a fork if someone else is as well; I don't have enough bandwidth to do it myself right now, but I do have enough to contribute.

Admittedly, I'm mostly interested in this so I can get rid of my pile of sed scripts for packaging up my Helm charts, but if that's what it takes, so be it.

Hello there
I have just created a plugin which allows to set values when packaging. for the moment it just supports the set flag (because I needed that :sweat_smile: ) but I intend to add other flags. do not hesitate to push PRs if you like or add remarks in order to improve it.
Thank you

Just in case anyone hasn't been following along, there's been some progress in ghodss/yaml#62 contacting a maintainer, so hopefully we'll hear some good news soon!

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.

Definitely not stale. I've been trying to find time to follow up with https://github.com/ghodss/yaml/pull/62 and see if we can help get it merged to unblock progress on this issue. As my org has increased our dependency on Helm, we continue to want to be able to bake values into a per-CI build packaged chart.

I've deleted any comments that have asked for a status update as they do not add anything to the conversation at-hand. @jmcelwain provided a status update no less than 5 days ago (thank you, btw) and is as good as any. As mentioned earlier, the solution to this problem is tricky and involves updating or forking several dependencies, so there's some due diligence required.

When we have more information to share, we'll update the thread.

As a workaround, I've used https://github.com/mikefarah/yq to update my helm chart prior to packaging:

yq w -i values.yaml version $(Build.BuildNumber)

In our CI we build app / docker image and the chart in the same pipeline (to ensure that the app & chart evolve together with the same versioning). The default values for the chart depends on the image version previously build.

The yq workaround can be easily integrated in a pipeline, but that seems strange that this use case is not correctly covered by helm !

So they are no integrated way with helm to set image.tag during/before the package phase ?

edit : found this solution in my case, we need to use the Chart.AppVersion for the image tag because we package the chart with the same versioning.

edit 2 : Chart.AppVersion / don't work with shared chart (exp a spring-boot chart used in multiple umbrella charts)

my conclusion : Really need to set image.tag during the package phase

Any news about this feature?