Commit Graph

9642 Commits

Author SHA1 Message Date
Kubernetes Prow Robot
c592bd40f2 Merge pull request #105609 from pohly/generic-ephemeral-volume-ga
generic ephemeral volume GA
2021-10-28 17:36:50 -07:00
Kubernetes Prow Robot
17da6a2345 Merge pull request #105699 from yuzhiquan/remove-format-pods
Remove format.pods func, instead with klog.Kobjs
2021-10-25 15:53:30 -07:00
Eric Ernst
2c0fad1f52 kuberuntime: populate sandbox resources, overhead
Populate Resources and Overhead fields which, are now part of
LinuxPodSandboxConfig.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2021-10-20 11:30:23 -07:00
Eric Ernst
ddcf815d12 kuberuntime: refactor linux resources for better reuse
Seperate the CPU/Memory req/limit -> linux resource conversion into its
own function for better reuse.

Elsewhere in kuberuntime pkg, we will want to leverage this
requests/limits to Linux Resource type conversion.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2021-10-20 11:30:23 -07:00
Eric Ernst
b1361aed93 kuberuntime: augment linux container config unit test
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2021-10-20 11:30:23 -07:00
Eric Ernst
a73502a0be kuberuntime: augment linux container config unit test
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2021-10-20 11:29:22 -07:00
Kubernetes Prow Robot
b2c4269992 Merge pull request #105631 from klueska/upstream-distribute-cpus-across-numa
Add CPUManager policy option to distribute CPUs across NUMA nodes instead of packing them
2021-10-19 11:40:24 -07:00
Kubernetes Prow Robot
1af8a8c026 Merge pull request #105465 from marosset/remove-host-process-contianer-kubelet-annotations
Stop passing WindowsHostProcessContainer annotations for CRI calls in kubelet
2021-10-18 15:50:02 -07:00
Kubernetes Prow Robot
e595d79dfc Merge pull request #104574 from 249043822/br-repeat-package
fix duplicate package import in pod_worker
2021-10-18 15:49:46 -07:00
Kubernetes Prow Robot
5889fb4fbc Merge pull request #105652 from wzshiming/feat/structure-shutdown-config
Refactor to use structure to pass parameters for GracefulNodeShutdown
2021-10-18 14:45:20 -07:00
Kevin Klues
86f9c266bc Add optimizations to reduce iterations in distributed NUMA algorithm
Signed-off-by: Kevin Klues <kklues@nvidia.com>
2021-10-18 08:53:25 +00:00
Kevin Klues
70e0f47191 Support full-pcpus-only with the new NUMA distribution policy option
Signed-off-by: Kevin Klues <kklues@nvidia.com>
2021-10-16 19:31:02 +00:00
Kevin Klues
d54445a84d Generalize the NUMA distribution algorithm to take cpuGroupSize
This parameter ensures that CPUs are always allocated in groups of size
'cpuGroupSize'. This is important, for example, to ensure that all CPUs (i.e.
hyperthreads) from the same core are handed out together.

Signed-off-by: Kevin Klues <kklues@nvidia.com>
2021-10-16 19:31:02 +00:00
Kevin Klues
1436e33642 Add more extensive testing for NUMA distribution algorithm in CPUManager
Signed-off-by: Kevin Klues <kklues@nvidia.com>
2021-10-16 19:31:02 +00:00
Kevin Klues
cf3afb8602 Add 2 distinguishing test cases between the 2 takeByTopology algorithms
Signed-off-by: Kevin Klues <kklues@nvidia.com>
2021-10-16 19:31:02 +00:00
Kevin Klues
eb78e2406b Add a new TestTakeByTopologyNUMADistributed() test to the CPUManager
As part of this, pull out all of the existing "TakeByTopology" tests and have
them be called by the original TestTakeByTopologyNUMAPacked() as well as the
new TestTakeByTopologyNUMADistributed() test. In a subsequent commit, we will
add some tests that should differ between these two algorithms.

Signed-off-by: Kevin Klues <kklues@nvidia.com>
2021-10-16 19:31:02 +00:00
Kevin Klues
876dd9b078 Added algorithm to CPUManager to distribute CPUs across NUMA nodes
Signed-off-by: Kevin Klues <kklues@nvidia.com>
2021-10-16 19:31:02 +00:00
Kevin Klues
462544d079 Split CPUManager takeByTopology() into two different algorithms
The first implements the original algorithm which packs CPUs onto NUMA nodes if
more than one NUMA node is required to satisfy the allocation. The second
disitributes CPUs across NUMA nodes if they can't all fit into one.

The "distributing" algorithm is currently a noop and just returns an error of
"unimplemented". A subsequent commit will add the logic to implement this
algorithm according to KEP 2902:

https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2902-cpumanager-distribute-cpus-policy-option

Signed-off-by: Kevin Klues <kklues@nvidia.com>
2021-10-16 14:46:19 +00:00
Kevin Klues
0e7928edce Add new CPUManager policy option for "distribute-cpus-across-numa"
This commit only adds the option to the policy options framework. A
subsequent commit will add the logic to utilize it.

The KEP describing this new option can be found here:
https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2902-cpumanager-distribute-cpus-policy-option

Signed-off-by: Kevin Klues <kklues@nvidia.com>
2021-10-16 14:46:19 +00:00
yuzhiquanlong
27fe56e916 remove unused import 2021-10-15 18:40:31 +08:00
Francesco Romani
4bae656835 cpumanager: test NUMA node support for CPU assign (2)
This batch of tests adds a fake topology on which each numa node
has multiple sockets. We didn't find yet a real HW topology in the wild
like this, but we need one to fully exercise the code.

So, until we find a HW topology, we add a fake one flipping
the NUMA/socket config of the existing xeon dual gold 6320.

Signed-off-by: Francesco Romani <fromani@redhat.com>
2021-10-15 10:29:21 +00:00
Francesco Romani
547996f3f6 cpumanager: test NUMA node support for CPU assign (1)
This batch of tests adds a real topology on which each physical socket
has multiple NUMA zones. Taken by a real dual xeon 6320 gold.

Signed-off-by: Francesco Romani <fromani@redhat.com>
2021-10-15 10:29:21 +00:00
Francesco Romani
f6ccc4426a cpumanager: test: use proper subtests
The exisiting unit tests where performing subtests without
actually using the full features of the testing package
(https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks)

Update them with fairly minimal changes. The patch is deceptively
large because we need to move the code inside a new block.

Signed-off-by: Francesco Romani <fromani@redhat.com>
2021-10-15 10:29:21 +00:00
Francesco Romani
15caa134b2 cpumanager: topology: use rich cmp package
User the `cmp.Diff` package in the unit tests, moving away from
`reflect.DeepEqual`. This gives us a clearer picture of the differences
when the tests fail.

Signed-off-by: Francesco Romani <fromani@redhat.com>
2021-10-15 10:29:21 +00:00
Kevin Klues
aff54a0914 Abstract out whether NUMA or Sockets come first in the memory hierarchy
This allows us to get rid of the check for determining which one is higher all
throughout the code. Now we just check once and instantiate an interface of the
appropriate type that makes sure the ordering in the hierarchy is preserved
through the appropriate calls.

Signed-off-by: Kevin Klues <kklues@nvidia.com>
2021-10-15 10:29:15 +00:00
yuzhiquanlong
be9e1fda5e remove format pods func, instead with klog.Kobjs 2021-10-15 18:26:02 +08:00
Kevin Klues
17c7e86c6d Add NUMA support to the CPU assignment algorithm in the CPUManager
Signed-off-by: Kevin Klues <kklues@nvidia.com>
2021-10-15 08:35:59 +00:00
Shiming Zhang
e47c78a354 Add log for creating node shutdown manager 2021-10-15 11:16:21 +08:00
Shiming Zhang
b468c24e85 Refactor to use structure to pass parameters 2021-10-15 11:16:21 +08:00
Kubernetes Prow Robot
a923852ba0 Merge pull request #105215 from rphillips/add_probe_shutdown
kubelet: add probe termination to graceful shutdowns
2021-10-11 21:19:46 -07:00
Patrick Ohly
a8c930ef46 generic ephemeral volume: graduation to GA
The feature gate gets locked to "true", with the goal to remove it in two
releases.

All code now can assume that the feature is enabled. Tests for "feature
disabled" are no longer needed and get removed.

Some code wasn't using the new helper functions yet. That gets changed while
touching those lines.
2021-10-11 20:54:20 +02:00
Kubernetes Prow Robot
dc9c571166 Merge pull request #105569 from pohly/generic-ephemeral-kubelet-volume-stats
kubelet: also provide filesystem stats for generic ephemeral volumes
2021-10-11 07:52:39 -07:00
Kubernetes Prow Robot
1f2813368e Merge pull request #105542 from pohly/generic-ephemeral-volume-util-kubelet
kubelet: use generic ephemeral volume helper functions
2021-10-11 02:16:40 -07:00
Kubernetes Prow Robot
fb82a0d7eb Merge pull request #104873 from pohly/json-output-stream
JSON output streams
2021-10-10 17:04:37 -07:00
Patrick Ohly
b22263d835 component-base: configurable JSON output
This implements the replacement of klog output to different files per level
with optionally splitting JSON output into two streams: one for info messages
on stdout, one for error messages on stderr. The info messages can get buffered
to increase performance. Because stdout and stderr might be merged by the
consumer, the info stream gets flushed before writing an error, to ensure that
the order of messages is preserved.

This also ensures that the following code pattern doesn't leak info messages:
   klog.ErrorS(err, ...)
   os.Exit(1)

Commands explicitly have to flush before exiting via logs.FlushLogs. Most
already do. But buffered info messages can still get lost during an unexpected
program termination, therefore buffering is off by default.

The new options get added to the v1alpha1 LoggingConfiguration with new command
line flags. Because it is an alpha field, changing it inside the v1beta kubelet
config should be okay as long as the fields are clearly marked as alpha.
2021-10-09 10:10:35 +02:00
Kubernetes Prow Robot
63f66e6c99 Merge pull request #105012 from fromanirh/cpumanager-policy-options-beta
node: graduate CPUManagerPolicyOptions to beta
2021-10-08 07:32:59 -07:00
Kubernetes Prow Robot
2face135c7 Merge pull request #97415 from AlexeyPerevalov/ExcludeSharedPoolFromPodResources
Return only isolated cpus in podresources interface
2021-10-08 05:58:58 -07:00
Patrick Ohly
b1ba381ef8 kubelet: also provide filesystem stats for generic ephemeral volumes
When checking for a reference to a PVC, the code also needs to consider that a
PVC might be referenced indirectly through an ephemeral volume source.
2021-10-08 12:11:52 +02:00
Kubernetes Prow Robot
dd650bd41f Merge pull request #105527 from rphillips/fixes/filter_terminated_pods
kubelet: set terminated podWorker status for terminated pods
2021-10-07 22:19:51 -07:00
Ryan Phillips
0166d446b9 kubelet: set terminated podWorker status for terminated pods 2021-10-07 16:18:59 -05:00
Patrick Ohly
844662e7fa kubelet: use generic ephemeral volume helper functions
The name concatenation and ownership check were originally considered small
enough to not warrant dedicated functions, but the intent of the code is more
readable with them.
2021-10-07 17:31:54 +02:00
Alexey Perevalov
5d9032007a Return only isolated cpus in podresources interface
Co-Authored-by: Swati Sehgal <swsehgal@redhat.com>
Signed-off-by: Alexey Perevalov <alexey.perevalov@huawei.com>
2021-10-07 15:34:08 +01:00
Kubernetes Prow Robot
c4d802b0b5 Merge pull request #103289 from AlexeyPerevalov/DoNotExportEmptyTopology
podresources: do not export empty NUMA topology
2021-10-07 07:11:46 -07:00
Kubernetes Prow Robot
907d62eac8 Merge pull request #105462 from ehashman/merge-terminal-phase
Ensure terminal pods maintain terminal status
2021-10-05 13:12:58 -07:00
Mark Rossetti
99e43bfa8c Stop passing WindowsHostProcessContainer annotations for CRI calls in kubelet
Signed-off-by: Mark Rossetti <marosset@microsoft.com>
2021-10-05 10:08:53 -07:00
Elana Hashman
3005ef34f2 Ensure terminal pods maintain terminal status 2021-10-05 09:26:27 -07:00
Kubernetes Prow Robot
c91f9bdc60 Merge pull request #104689 from cynepco3hahue/memory_manager_restricted_policy_fix
kubelet: memory manager: fix preferred topology hints calculation
2021-10-05 06:47:08 -07:00
Kubernetes Prow Robot
efa9029a0d Merge pull request #104920 from tkashem/response-writer-cleanup
apiserver: decorate http.ResponseWriter correctly
2021-10-05 00:53:09 -07:00
Elana Hashman
5ff6c2396d Do not sync Waiting statuses for Terminated pods 2021-10-04 11:05:54 -07:00
Abu Kashem
0d50c969c5 apiserver: wrap ResponseWriter using abstraction 2021-10-04 10:59:11 -04:00