New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrated scheduler files comparer.go
, dumper.go
, node_tree.go
to structured logging
#105968
Migrated scheduler files comparer.go
, dumper.go
, node_tree.go
to structured logging
#105968
Conversation
/sig instrumentation |
4eaaf6e
to
594ae56
Compare
@@ -57,7 +57,7 @@ func (d *CacheDumper) dumpSchedulingQueue() { | |||
for _, p := range pendingPods { | |||
podData.WriteString(printPod(p)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please don't use custom object serialization for pods. Please either use klog.KObjs(pendingPods)
to get basic information like pod name and namespace or propose custom struct that will include fields that should be printed.
cc @pohly
This is interesting case where we want to log slice of K8s object but with additional information, maybe you have suggestion how to best handle it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's possible to write a custom formatter like this:
type formatter *v1.Pod
func (f formatter) String() string {
return fmt.Sprintf("%s/%s ...", f.Namespace, f.Name, ....)
}
func (f formatter) MarshalLog interface{} {
return struct {
Namespace, Name string
<other data>
} {
Namespace: f.Namespace,
Name: f.Name,
<other data>
}
}
var _ logr.Marshaler = formatter{}
klog.Infos("some data about a pod", "pod", formatter(pod))
Text output via klog will call Stringer
and JSON output will call MarshalLog
, then render the struct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This example is for a single pod. The same works with []*v1.Pod
or any other type, it just gets a bit more complex.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So before this new helper lib is available, we keep the current code as-is?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this PR needs a replacement for podData.WriteString
and printPod
, then this PR also would be the right place to implement this custom formatter.
It could be local to the pkg/scheduler/internal/cache/debugger
package. Or is the same code also needed somewhere else?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this PR needs a replacement for podData.WriteString and printPod
I don't have a strong option on this as the debugger package is not extensively used, esp. not by the regular end-users.
Or is the same code also needed somewhere else?
I think the pattern is only used in the debugger pkg.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@shivanshu1333: do you intend to implement such a formatter as part of this PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, actually I've some doubts I need your help to figure out the right way to do this, I had posted a query in slack for the same https://kubernetes.slack.com/archives/C020CCMUEAX/p1636304943054700
/assign @serathius |
594ae56
to
cd6e185
Compare
/unassign |
This PR is needed to finish scheduler structured logging migration planned for 1.23 |
/retest |
cd6e185
to
99b42fc
Compare
The unit test to check the behaviour for |
Please let me know if any change is required, and also let’s think about adding the above unit test as well, please guide me about possibilities of introducing a unit test for |
99b42fc
to
3486dea
Compare
// printPod writes parts of a Pod object to a string. | ||
func printPod(p *v1.Pod) string { | ||
return fmt.Sprintf("name: %v, namespace: %v, uid: %v, phase: %v, nominated node: %v\n", p.Name, p.Namespace, p.UID, p.Status.Phase, p.Status.NominatedNodeName) | ||
klog.InfoS("Dump of scheduling queue", "podData", formatPod{pendingPods}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
klog.InfoS("Dump of scheduling queue", "podData", formatPod{pendingPods}) | |
klog.InfoS("Dump of scheduling queue", "podData", formatPods{pendingPods}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This of course then also needs to be updated in formatter.go.
func (f formatPod) String() string { | ||
var podInfo strings.Builder | ||
for _, v := range f.pods { | ||
podInfo.WriteString(fmt.Sprintf("Name: %v, Namespace: %v, UID: %v, Phase: %v, NominatedNode: %v\n", v.Name, v.Namespace, v.UID, v.Status.Phase, v.Status.NominatedNodeName)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In #104868 it was pointed out that multi-line strings do not actually get printed as multiple lines, making the text output fairly hard to read.
I wonder whether the same happens here... yes, it does, as your unit test in https://gist.github.com/shivanshu1333/3e7b7cda9fad55c2065900cd30f75781 shows.
I suspect the scheduler developers will consider this as a regression because it makes the text output less useful to them.
Your formatter looks (mostly) good to me and we should come back to it once we know how to emit multi-line strings in text format, but for now we should keep the original klog.Info(d.printNodeInfo(name, nodeInfo))
and
klog.Infof("Dump of scheduling queue:\n%s", podData.String())
.
}) | ||
} | ||
|
||
if nodeInfo.NominatedPods != 0 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This if check can be removed.
nodeInfo.WriteString(fmt.Sprintf("Name: %v, Namespace: %v, UID: %v, Phase: %v, NominatedNode: %v\n", v.Pod.Name, v.Pod.Namespace, v.Pod.UID, v.Pod.Status.Phase, v.Pod.Status.NominatedNodeName)) | ||
} | ||
|
||
if len(f.nominatedPodInfo) != 0 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This if check can be removed.
@pohly Does this mean that for the scope of this migration PR, let's not worry about the formatter in this PR and keep the original things as it is, and start new efforts (PR) to continue formatting work? |
Yes. It means the migration won't be complete, but that's better than migrating and producing output that is worse than what it is without the migration. |
3486dea
to
84a6435
Compare
Created #106276 to take care of custom formatter, I think now this PR is unblocked and we can work on the mentioned issue separately. |
84a6435
to
1a079d7
Compare
/approve Leaving /lgtm to @pohly. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/lgtm
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Huang-Wei, pohly, shivanshu1333 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/retest |
What type of PR is this?
/kind cleanup
What this PR does / why we need it:
Migrated
pkg/scheduler/internal/cache/debugger/comparer.go
,pkg/scheduler/internal/cache/debugger/dumper.go
, andpkg/scheduler/internal/cache/node_tree.go
to structured loggingWhich issue(s) this PR fixes:
Ref #105841
Special notes for your reviewer:
Added missing migration points, fixed previously migrated logs
Does this PR introduce a user-facing change?
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: