Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce the number of expensive calls in the Windows stats queries for dockershim #104287

Merged
merged 1 commit into from Nov 16, 2021

Conversation

jsturtevant
Copy link
Contributor

@jsturtevant jsturtevant commented Aug 11, 2021

What type of PR is this?

/kind bug

What this PR does / why we need it:

This improves the response time and minimizes the calls to the system stats which reduces overall cpu for kubelet. This is done by passing the container object, which has the information required, instead of querying for it on every container via the dockerservice. It also caches some information like the runtime info so it is not queried on every container.

Dockershim create a new namepiped instance to docker every call causing reads and higher cpu usage. (Containerd re-uses the same namedpipe instance so doesn't see the same cpu spike).

Which issue(s) this PR fixes:

Fixes #104283

Special notes for your reviewer:

I initially included the commits from #105744 but moved it to a separate PR. That PR also improves the overall CPU used here but not opening and list all the containers twice.

Does this PR introduce a user-facing change?

Reduce the number of calls to docker for stats via dockershim. For Windows this reduces the latency when calling docker, for Linux this saves cpu cycles. 

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/bug Categorizes issue or PR as related to a bug. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/apiserver area/cloudprovider area/code-generation area/dependency Issues or PRs related to dependency changes area/kubectl area/kubelet sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/cli Categorizes an issue or PR as relevant to SIG CLI. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/storage Categorizes an issue or PR as relevant to SIG Storage. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Aug 11, 2021
@jsturtevant jsturtevant marked this pull request as draft August 11, 2021 17:48
@ehashman ehashman added this to Waiting on Author in SIG Node PR Triage Aug 12, 2021
@marosset
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 10, 2021
@jsturtevant
Copy link
Contributor Author

unrelated flake #106326
/retest

@jsturtevant
Copy link
Contributor Author

/assign @dchen1107

@marosset marosset moved this from In Progress (v1.23) to Reviewed - Needs Approval From Other SIGs in SIG-Windows Nov 11, 2021
@marosset marosset moved this from Waiting on Author to Needs Approver in SIG Node PR Triage Nov 11, 2021
@marosset
Copy link
Contributor

/assign @mrunalp

@varshaprasad96
Copy link
Contributor

Hi @jsturtevant,
k8s 1.23 bug triage here 👋,
Just wanted to follow up and check if this issue is on track for 1.23?

@marosset
Copy link
Contributor

Hi @jsturtevant, k8s 1.23 bug triage here 👋, Just wanted to follow up and check if this issue is on track for 1.23?

We would to have this merged for 1.23. We just need final approval from area owners.

@mrunalp mrunalp moved this from Needs Approver to Done in SIG Node PR Triage Nov 16, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jsturtevant, mrunalp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 16, 2021
@k8s-ci-robot k8s-ci-robot merged commit 1d1d462 into kubernetes:master Nov 16, 2021
SIG-Windows automation moved this from Reviewed - Needs Approval From Other SIGs to Done (v1.23) Nov 16, 2021
@marosset
Copy link
Contributor

/priority critical-urgent

@ehashman
Copy link
Member

/remove-priority backlog
/priority critical-urgent

@k8s-ci-robot k8s-ci-robot added priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. and removed priority/backlog Higher priority than priority/awaiting-more-evidence. labels Nov 29, 2021
k8s-ci-robot added a commit that referenced this pull request Dec 1, 2021
…#104287-upstream-release-1.20

Automated cherry pick of #104287: Reduce calls to docker from dockershim for stats
k8s-ci-robot added a commit that referenced this pull request Dec 1, 2021
…#104287-upstream-release-1.22

Automated cherry pick of #104287: Reduce calls to docker from dockershim for stats
k8s-ci-robot added a commit that referenced this pull request Dec 1, 2021
…#104287-upstream-release-1.21

Automated cherry pick of #104287: Reduce calls to docker from dockershim for stats
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubelet cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/windows Categorizes an issue or PR as relevant to SIG Windows. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
SIG-Windows
  
Done (v1.23)
Development

Successfully merging this pull request may close these issues.

Context Deadline exceeded when Metrics Server calls Windows Stats endpoint