Enhancements: Server-side Apply

Created on 6 Apr 2018  ·  120Comments  ·  Source: kubernetes/enhancements

Feature Description

kinapi-change kinfeature siapi-machinery sicli stagbeta trackeno

Most helpful comment

We're still beta, we worked on new improvements but we're not ready for GA yet. We'll release a blog-post in a few days about this!

All 120 comments

/assign @apelisse

Looks like we still need some docs to get this feature ready for release @apelisse
Could I please get some help with that? If there’s anything I can do to assist please let me know

At a minimum we're looking to have a placeholder PR on the kubernetes/website repo. The process is fairly straightforward: checkout release-1.11 branch, make a placeholder commit, push it to your fork, and raise a PR between it and the release-1.11 branch, with /hold status.

THANKS SO MUCH!!!!!

@apelisse I noticed the milestone change. Are you planning on pulling this from the 1.11 release?

Yeah, not in 1.11. I'm updating the body.

Thanks @apelisse!

Unsure if it is the right place to comment about this, but we have been working on our own kind of "server side apply" by building https://github.com/seibert-media/dimios
While this is solely a slack time project which we hope to build our future interaction with Kubernetes on, we both stumbled into problems with the entire apply process (discussed on slack with some people) but also did some thinking on how to best approach this.

While I haven't worked on K8s directly myself yet, I could imagine to contribute to this idea/feature if possible/wanted.

Can you send this as an email to [email protected]? We can follow-up there.

@apelisse I am not allowed to post messages to said group ([email protected])

Sorry, you may need to join the group first: https://groups.google.com/forum/#!forum/kubernetes-wg-apply

Blue button, "Join group to post" (top left)

@apelisse @kubernetes/sig-api-machinery-feature-requests @kubernetes/sig-cli-feature-requests --

This feature was removed from the previous milestone, so we'd like to check in and see if there are any plans for this in Kubernetes 1.12.

If so, please ensure that this issue is up-to-date with ALL of the following information:

  • One-line feature description (can be used as a release note):
  • Primary contact (assignee):
  • Responsible SIGs:
  • Design proposal link (community repo):
  • Link to e2e and/or unit tests:
  • Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred:
  • Approver (likely from SIG/area to which feature belongs):
  • Feature target (which target equals to which milestone):

    • Alpha release target (x.y)

    • Beta release target (x.y)

    • Stable release target (x.y)

Set the following:

  • Description
  • Assignee(s)
  • Labels:

    • stage/{alpha,beta,stable}

    • sig/*

    • kind/feature

Please note that the Features Freeze is July 31st, after which any incomplete Feature issues will require an Exception request to be accepted into the milestone.

In addition, please be aware of the following relevant deadlines:

  • Docs deadline (open placeholder PRs): 8/21
  • Test case freeze: 8/28

Please make sure all PRs for features have relevant release notes included as well.

Happy shipping!

/cc @justaugustus @kacole2 @robertsandoval @rajendar38

I think everything is there @justaugustus! Thank you

Thanks for the update, @apelisse!

Hey there! @apelisse I'm the wrangler for the Docs this release. Is there any chance I could have you open up a docs PR against the release-1.12 branch as a placeholder? That gives us more confidence in the feature shipping in this release and gives me something to work with when we start doing reviews/edits. Thanks! If this feature does not require docs, could you please update the features tracking spreadsheet to reflect it?

@zparnold Changed the milestone from 1.12 to 1.13 :-). You can forget about this for now, thanks.

It'll be alpha, it would be good to document. Jenny said she'd try and put something together. [EDIT: wrong issue, ignore]

On Tue, Aug 21, 2018 at 12:44 PM Antoine Pelisse notifications@github.com
wrote:

@zparnold https://github.com/zparnold Changed the milestone from 1.12
to 1.13 :-). You can forget about this for now, thanks.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/features/issues/555#issuecomment-414797736,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAnglgxXgV0DuPpOq4P4UZZjMY8Fcx6Lks5uTGMDgaJpZM4TKb_j
.

Sorry, I saw this in my email and mistook it for the dry run feature issue. Yes, serverside apply is not launching in 1.12.

@lavalamp @apelisse I'll make the changes in the spreadsheet. thanks for making us aware! cc @justaugustus

Are there any issues or PR's we could track to better understand what work is being done to land what functionality in 1.13 as part of this?

@spiffxp We don't have an issue right now. we have a working-group for this, and we keep track of tasks in a spreadsheet. I'll try to create an issue to track the progress and increase visibility, thank you.

Isn't this the issue?

For PRs, look here: https://github.com/kubernetes-sigs/structured-merge-diff/pulls

Also look in the main repo, search for "feature branch" in the name (e.g., Jenny has an API change out right now).

We have notes from our bi-weekly meeting here: https://docs.google.com/document/d/1j_UqnJRvB7AywfZauHybVVNCOoDDA1AGWGxzPGE08ko/edit# (join the wg-apply mailing list if you want to read)

We also have meeting recordings although we might be behind publishing them.

@lavalamp @apelisse what is the confidence level of this feature making it to Alpha in 1.13? Are these 3 PRs the only ones pending?

@lavalamp right my Q was more whether any of the feature branch stuff was going to land in master, otherwise it doesn't seem like this should be on our radar; the wg-apply notes are pretty spartan in that regard

There's the roadmap doc mentioned in April, but it's unclear to me whether that's still valid, and if so, which Phase we're currently in vs. aiming for

Based on your latest comment in the sig-api-machinery meeting notes I'm trying to ascertain what work items we should be watching for "not sure we'll make it to alpha in time" and whether "alpha" implies landing the feature branch to master

@spiffxp Yes, making it to alpha is the same thing as merging the feature branch back into master.

@AishSundar No, there are many PRs yet to be written. Look at the closed PRs in that repo for an estimate of the recent velocity. Additionally we will need more PRs to the feature branch in the main repo.

Can you two help me understand why this level of detail is necessary/useful? We still have 3ish weeks before anything freezes or slushes, no? If the branch doesn't get merged by then, we will have to push this off.

@lavalamp since this is a short release cycle the release team is trying to assess, early on, the amount of work pending for each planned feature and chances of any possible slips. This will help us track these better through the cycle

There is definitely some work left to get this in 1.13 and chances that it slips are non-negligible. However, we still want to take our chance and hope that the few weeks left will be enough to get to alpha quality.

I'm happy to help you track that item in any way possible and can come back to you on a regular basis to give you a status. Hope that helps, thanks.

@apelisse for now if you can just point us to the list of open PRs and open issues (for unopened PRs) that will be great. we will touch base early November to see where we are. Thanks.

@kacole2 as FYI

We aren't tracking our work via specific issues that need specific PRs written, that would be fairly heavyweight for us. We are using this spreadsheet: https://docs.google.com/spreadsheets/d/1ZHTDyiQssYBzs7DQWQd70bNQybeyS0m-S1aIS_zo-78/edit?pli=1#gid=0

(you're probably already in some email group that has access)

In an offline chat with @spiffxp it seems you're actually trying to evaluate risk; I think the questions above don't actually make sense for this particular effort given that this is happening in a feature branch, which is probably why we're talking past each other.

  • In terms of risk to stability, we won't be merging it if it's not meeting the bar before the window closes, so there's very little risk. All of our changes are in the feature branch or other repos so it's not like we'd leave master in a half broken state if we don't make it.
  • In terms of risk of not getting to the finish line in time, we're well aware that time is short and there's a good chance we won't make the release.

Hi @lavalamp and @apelisse, I'm the docs wrangler for the 1.13 release. Could you please open a placeholder PR for the docs for this enhancement against the dev-1.13 branch of k/website and send me a link? If you already have a docs PR open, or if this doesn't require docs in k/website, please let me know.

The deadline for placeholder PRs for the 1.13 release is November 8. So it's important to make a docs PR as soon as possible.

If you have any questions about any of this, I'm happy to help. You can also message me on slack (I'm tfogo there too).

Thanks!

@tfogo Thanks for your help! I created it here: https://github.com/kubernetes/website/pull/10812

Hi @lavalamp and @apelisse - I'm Enhancements Shadow for 1.13. Reading through this thread and considering the method of tracking this feature, could you please update the release team with the likelihood of this enhancement making the 1.13 release?

Code slush begins on 11/9 and code freeze is 11/15.

Thank you!

Hi @guineveresaenger, there are still chances that we miss the release, but we're doing everything we can to have a high-quality alpha feature in 1.13!

@apelisse can we please time-box the Go/NoGo decision to Wednesday (11/14) next week. At this point in the cycle we expect all 1.13 enhancements to have all code and test PRs either merged or very close to merging. I understand this feature is being developed in a feature branch. But with Code freeze next Friday (11/16), we need to merge it into master atleast by early Wednesday to get stable CI signals for a couple of days from Master. If that's not possible we will have to push it to 1.14. Thanks

@kacole2

We found a couple bugs at the last second so we will merge this as the first thing for 1.14 when master opens.

Thanks for the update @lavalamp.

@kacole2 @tfogo @marpaia @kbarnard10 as FYI

@apelisse @lavalamp Hello - I’m the enhancement’s lead for 1.14 and I’m checking in on this issue to see what work (if any) is being planned for the 1.14 release. Enhancements freeze is Jan 29th.

Hello @claurence, we're planning to release this feature alpha this cycle! I just updated the description to reflect the new milestone. Anything else I need to update?

@apelisse sounds good! I'll add this to our 1.14 tracking sheet.

What would the API request done by the client look like after this? I assume it would send the full local resource definition, but would it still be a PATCH request? Or a PUT?

I'm really excited about this, it takes out a lot of complexity from custom clients.

Yes, it will receive the local object on the PATCH verb, with a specific content-type.

Btw the link to the design proposal is broken

Fixed, thanks, good catch.

Btw, all the design Google docs linked in there are not publicly viewable.

Hi Felix, that doc needs a massive clean-up, I'll take care of that right after code-freeze (or before if we have extra-time). Thanks!

I am back, maybe I can update the KEP.

/assign lavalamp

Hi @apelisse! I'm one of the v1.14 docs release shadows.

Does this enhancement require any new docs (or modifications)?

Just a friendly reminder we're looking for a PR against k/website (branch dev-1.14) due by Friday, March 1. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

Thanks for the reminder @cody-clark, we definitely need documentation!

Perfect! I'll mark it in the tracking sheet. Thanks for the quick reply, @apelisse!

@apelisse looking over the KEP for this enhancement I don't see any testing plans - can someone help PR in testing plans for this enhancement? This information is helpful for knowing readiness of this feature for the release and is specifically useful for CI Signal.

If we don't have testing plans this enhancement will be at risk for being included in the 1.14 release

@claurence I've updated the description, let me know if you expected something different! Thanks!

Hello, 1.14 enhancement shadow here. Code Freeze is March 7th and all PRs must be merged by then to your issue to make the 1.14 release. What open K/K PRs do you still have that need to merge? Thanks

Hey @lledru, we have a few PRs in flight that should get merged in the week or so. None of them is really blocking for the release and we absolutely could release in the current state!

hello @apelisse,
Can you please submit a PR against the KEP with the testing plan ?
This information is helpful for knowing readiness of this feature for the release and is specifically useful for CI Signal.
If we don't have testing plans this enhancement will be at risk for being included in the 1.14 release
thank you very much.

Hey @apelisse, the docs deadline was last Friday March 1st (to open a placeholder PR). Do you have any PRs in flight for dev-1.14 docs? Is this still targeting 1.14?

We've created the documentation place-holder: https://github.com/kubernetes/website/pull/12898. I'm not sure it's correct but at least it exists :-)

@lledru Thanks for the reminder, @jennybuckley is working on updating the KEP.

@lledru There it is: #878

@apelisse any work planned here for 1.15 or staying in Alpha?

Yep, it's going beta 1.15 hopefully

Thanks @apelisse. When the time is right, drop in the k/k PRs associated with the beta graduation criteria that's called out in the KEP

/milestone v1.15
/stage beta

Hey @apelisse. I can see you had a placeholder PR for 1.14 but we need a PR against k/website (branch dev-1.15) due by Thursday, May 30. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

@simplytunde @apelisse I'll take care of it, as I still got some stuff to put in the docs for my "clear managedFields" task.

Thanks @kwiesmueller let me know if I can help!

Done: https://github.com/kubernetes/website/pull/14300
We can use this as a placeholder PR and use it for other things we might want to add for 1.15

I'm beyond exited for this to land (next month?).
Is there a way to try it out already (in Minikube)?

Hi @apelisse @kwiesmueller . Code Freeze is Thursday, May 30th 2019 @ EOD PST. All enhancements going into the release must be code-complete, including tests, and have docs PRs open.

Please list all current k/k PRs so they can be tracked going into freeze. If the PRs aren't merged by freeze, this feature will slip for the 1.15 release cycle. Only release-blocking issues and PRs will be allowed in the milestone.

If you know this will slip, please reply back and let us know. Thanks!

I really don't see how this would be done by code-freeze. I think you can mark it for 1.16 :-/

😭

@felixfbecker You can use any kubernetes 1.14 or later with the ServerSideApply feature enabled to try it out.

In minikube, it would be something like ... minikube start --feature-gates=ServerSideApply=true as far as I understand!

Sorry we missed it, we have a scalability issue that we need to address first.

/milestone clear

It enables everything that has been released in 1.14 plus what we merged in master since then.
I'd love to learn more about the features you want, what you're building with it, and what you don't like about the feature! Feel free to ping me on slack (apelisse@)

Was that last comment directed at me?

@felixfbecker and someone else who deleted their post ;-)

@felixfbecker It was me. Sorry about that I deleted my post since I found the answer after stumbling across this thread 🙂

Hi @apelisse , I'm the 1.16 Enhancement Shadow. Is this feature going to be graduating alpha/beta/stable stages in 1.16? Please let me know so it can be added to the 1.16 Tracking Spreadsheet and I can update the milestone and stage tags

Once coding begins or if it already has, please list all relevant k/k PRs in this issue so they can be tracked properly.

Milestone dates are Enhancement Freeze 7/30 and Code Freeze 8/29.

Thank you.

Yeah, we're planning on releasing this feature as Beta.

/remove-label tracked/no
/label tracked/yes
/milestone v1.16

Hey, @apelisse @lavalamp I'm the v1.16 docs release lead.

Does this enhancement (or the work planned for v1.16) require any new docs (or modifications)?

Just a friendly reminder we're looking for a PR against k/website (branch dev-1.16) due by Friday,August 23rd. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

Is a limited version of server-side apply available in v 13 (that is where my cluster is at) ? Some exchanges above seemed to imply it may be. My use case is for creating pods/deplyments/services in the reconcile loop of an operator. Using the api/core<>Pod etc. is tedious and error prone and being able to use a server api to apply yaml would be a lot cleaner for me. Thanks for any suggestions..

@champak, you can try it out by turning on the feature-gate "ServerSideApply" your apiserver, but please keep in mind that this is still an alpha feature.

@apelisse code freeze for 1.16 is on Thursday 8/29. Are there any outstanding k/k PRs that still need to be merged for this to go Beta?

What are the criteria for something to enter beta? I've tried server side apply but it doesn't seem useable really rn as a replacement to client-side apply that kubectl does: https://github.com/kubernetes/kubernetes/issues/80916

@felixfbecker Sorry that slipped through the cracks! Thanks for trying it out. I agree something is wrong and we need to fix it for beta.

But in general, it sounds a bit like you have an expectation that this is an extension of client side apply, actually it's more like a parallel system--use one or the other. The general transition mechanism is to follow Jenny's advice in your bug. We did this to provide a very safe experience-- you can choose exactly which behavior you want.

We will provide a seamless transition experience when we deprecate client side apply.

Hi @apelisse it looks as though https://github.com/kubernetes/kubernetes/pull/81816 didn't merge before code freeze and it's not in the Tide Merge Pool. This feature is going to be bumped from v1.16. If you would still like to have this be a part of the 1.16 release, please file an exception

The mandatory prs merged, this is beta now.

On Fri, Aug 30, 2019, 4:58 AM Kendrick Coleman notifications@github.com
wrote:

Hi @apelisse https://github.com/apelisse it looks as though
kubernetes/kubernetes#81816
https://github.com/kubernetes/kubernetes/pull/81816 didn't merge before
code freeze and it's not in the Tide Merge Pool https://prow.k8s.io/tide.
This feature is going to be bumped from v1.16. If you would still like to
have this be a part of the 1.16 release, please file an exception
https://github.com/kubernetes/sig-release/blob/master/releases/EXCEPTIONS.md


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/enhancements/issues/555?email_source=notifications&email_token=AAE6BFRCLZC47AWM6AQMJE3QHEDOPA5CNFSM4EZJX7R2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5ROEEA#issuecomment-526574096,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE6BFRKMWNNIOAIINJGBELQHEDOPANCNFSM4EZJX7RQ
.

@lavalamp would you mind clarifying? are you saying https://github.com/kubernetes/kubernetes/pull/81816 wasn't mandatory for this feature to be in beta?

@guineveresaenger It's not mandatory for this feature to be in beta. That PR is to address scalability concerns and can be merged in a future release, it doesn't change the API and only improves performance in clusters where many things are managed by this new feature

@guineveresaenger Yes, it's opt-in on a per-object basis in beta 1, so that isn't needed as there's no longer a global performance impact.

thanks for letting me know @lavalamp

Hey there @lavalamp @apelisse -- 1.17 Enhancements lead here. I wanted to check in and see if you think this Enhancement will be graduating to alpha/beta/stable in 1.17?

The current release schedule is:

  • Monday, September 23 - Release Cycle Begins
  • Tuesday, October 15, EOD PST - Enhancements Freeze
  • Thursday, November 14, EOD PST - Code Freeze
  • Tuesday, November 19 - Docs must be completed and reviewed
  • Monday, December 9 - Kubernetes 1.17.0 Released

If you do, please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

Thanks!

/milestone clear

Hi @mrbobbytables, while we're planning great improvements to this feature for the next release, but we're not planning on moving to GA yet. Thanks.

@apelisse noted -- thanks for the quick response! 😄

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

/remove-lifecycle stale

We're working on a "beta 2" release of this feature for 1.18, not sure how/if we need to track that, but happy to follow a process.

Hey @apelisse, thanks for highlighting that! We'll track it as a change within beta in the enhancements sheet. Will there be any changes to the KEP associated with this? Or will this just be related to code changes? If you could link all the k/k PRs associated with this so we can track it, that would be great.

If you're making any KEP changes, Enhancements Freeze will be January 28th
Code Freeze will be March 5th.

Thanks!

/milestone v1.18

We might get https://github.com/kubernetes/enhancements/pull/1442 merged until the 28th.
Maybe even https://github.com/kubernetes/enhancements/pull/923.

And i just saw that https://github.com/kubernetes/enhancements/pull/1123 merged, so the implementation could make it into 1.18 as well.

Hello, @apelisse - I'm a Docs shadow on the 1.18 release team.

Does this enhancement work planned for 1.18 require any new docs or modifications to existing docs?

If not, can you please update the 1.18 Enhancement Tracker Sheet (or let me know and I'll do so)

If doc updates are required, reminder that the placeholder PRs against k/website (branch dev-1.18) are due by Friday, Feb 28th.

Let me know if you have any questions!

We're currently working on the docs, thanks for the reminder!

Hey @apelisse @kwiesmueller,

Just a friendly reminder that code freeze for 1.18 is March 05, 2020.

As we track toward code freeze, please list out/link to any PRs you are working on toward graduating this enhancement!

Thanks for the reminder!

Hey @apelisse @kwiesmueller,

Circling back as we are getting closer to code freeze. Please list any k/k PRs you are working on so we can better track this issue.

I think we won't make it with the stuff I'm working on.
@apelisse might have some and probably @julianvmodesto with dryrun and diff?

I've created the placeholder doc PR: https://github.com/kubernetes/website/pull/19286

Hey @apelisse friendly reminder that code freeze for 1.18 is March 5th. That's just a few days away. Thanks for opening the docs PR above....could you link to any relevant K/K PRs for this so we can better track the progress of the enhancement as we get closer to code freeze?

Thanks so much!

Hey @apelisse, unfortunately https://github.com/kubernetes/kubernetes/pull/88875 didn't merge before code freeze (looks like it needs to be approved still). At this point, you'll need to file an exception request for this one.

/milestone clear

What is happening with this?

We're still beta, we worked on new improvements but we're not ready for GA yet. We'll release a blog-post in a few days about this!

Hi @apelisse -- 1.19 Enhancements Lead here, I wanted to check in if you think this enhancement would graduate in 1.19?


The current release schedule is:

  • Monday, April 13: Week 1 - Release cycle begins
  • Tuesday, May 19: Week 6 - Enhancements Freeze
  • Thursday, June 25: Week 11 - Code Freeze
  • Thursday, July 9: Week 14 - Docs must be completed and reviewed
  • Tuesday, August 4: Week 17 - Kubernetes v1.19.0 released

Hi there, we're not planning on promoting this feature this cycle, thank you!

Thank you @apelisse for the updates. I will update the tracking sheet accordingly. :+1:

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

/remove-lifecycle stale

Hi @apelisse

Enhancements Lead here. Any plans to graduate this in 1.20?

Thanks,
Kirsten

Not really, we're planning to keep improving the existing feature, but there is no graduation scheduled for 1.20, thanks!

Was this page helpful?
0 / 5 - 0 ratings

Related issues

xing-yang picture xing-yang  ·  13Comments

euank picture euank  ·  13Comments

majgis picture majgis  ·  5Comments

wojtek-t picture wojtek-t  ·  12Comments

robscott picture robscott  ·  11Comments