github/kubernetes
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
e64169d6b5
kubernetes/test/images/resource-consumer
History
Claudiu Belu 4c51eb9063 test images: Image Promoter fixes 
Prior to the Image Centralization part 4 (https://github.com/kubernetes/kubernetes/pull/81170),
a PR merged that enables the Image Promoter to run on the k/k test images.

The Image Promoter currently only builds the Conformance-related images, but the
Image Centralization part 4 centralized some of those images into agnhost, so they
need to be removed from the conformance_images list.

Additionally, https://github.com/kubernetes/kubernetes/pull/81226 proposes mounttest-user
image to be removed, and RunAsUser to be used in tests instead.

The image used by the Image Promoter (gcr.io/k8s-testimages/gcb-docker-gcloud:v20190906-745fed4)
is based on busybox, and thus, the sed binary is actually busybox. image-util.sh calls
kube::util::ensure-gnu-sed several times, which ensures that a GNU sed binary exists
(it checks by greping GNU in its --help output). Obviously, it won't match the busybox sed
binary. But the sed usage in image-util.sh is fairly simple, and the busybox sed is sufficient.

Bumps image versions for: jessie-dnsutils, nonewprivs, resource-consumer, sample-apiserver. These
images are included in the conformance_images that are being built by the Image Promoter, so
we're bumping them just to make sure we're not breaking anything and cause all the CIs to fall.
We're going to bump the image versions used in tests in a subsequent PR. The image version was not
bumped for: agnhost, kitten, nautilus, as they were already bumped by the Image Centralization part 4
PR.
2020-01-06 09:08:51 -08:00
..
common Fixed resource consumer to expose /metrics instead of /Metrics 2019-02-05 10:15:42 +01:00
consume-cpu should use time.Since instead of time.Now().Sub 2018-04-10 12:05:51 +08:00
.gitignore Multi Arch test images 2017-06-26 12:49:45 +05:30
BASEIMAGE add s390x to test containers 2019-03-18 17:13:49 -04:00
BUILD Uses the resource-consumer/controller image instead of resource-consumer-controller 2019-04-25 05:17:25 -07:00
Dockerfile Adopt debian-base as baseimage 2017-07-14 15:08:54 +05:30
Makefile Move go build to image-utils 2017-06-28 19:19:25 +05:30
README.md update README.md by cleaning up command prompts 2019-04-10 22:35:39 +08:00
resource_consumer_handler.go Remove test/images/* from hack/.golint_failures 2018-11-12 20:51:41 -07:00
resource_consumer.go Remove "All rights reserved" from all the headers. 2016-06-29 17:47:36 -07:00
utils.go Remove test/images/* from hack/.golint_failures 2018-11-12 20:51:41 -07:00
VERSION test images: Image Promoter fixes 2020-01-06 09:08:51 -08:00

README.md

Resource Consumer

Overview

Resource Consumer is a tool which allows to generate cpu/memory utilization in a container. The reason why it was created is testing kubernetes autoscaling. Resource Consumer can help with autoscaling tests for:

  • cluster size autoscaling,
  • horizontal autoscaling of pod - changing the size of replication controller,
  • vertical autoscaling of pod - changing its resource limits.

Usage

Resource Consumer starts an HTTP server and handle sent requests. It listens on port given as a flag (default 8080). Action of consuming resources is send to the container by a POST http request. Each http request creates new process. Http request handler is in file resource_consumer_handler.go

The container consumes specified amount of resources:

  • CPU in millicores,
  • Memory in megabytes,
  • Fake custom metrics.

Consume CPU http request

  • suffix "ConsumeCPU",
  • parameters "millicores" and "durationSec".

Consumes specified amount of millicores for durationSec seconds. Consume CPU uses "./consume-cpu/consume-cpu" binary (file consume-cpu/consume_cpu.go). When CPU consumption is too low this binary uses cpu by calculating math.sqrt(0) 10^7 times and if consumption is too high binary sleeps for 10 millisecond. One replica of Resource Consumer cannot consume more that 1 cpu.

Consume Memory http request

  • suffix "ConsumeMem",
  • parameters "megabytes" and "durationSec".

Consumes specified amount of megabytes for durationSec seconds. Consume Memory uses stress tool (stress -m 1 --vm-bytes megabytes --vm-hang 0 -t durationSec). Request leading to consuming more memory then container limit will be ignored.

Bump value of a fake custom metric

  • suffix "BumpMetric",
  • parameters "metric", "delta" and "durationSec".

Bumps metric with given name by delta for durationSec seconds. Custom metrics in Prometheus format are exposed on "/metrics" endpoint.

CURL example

kubectl run resource-consumer --image=gcr.io/kubernetes-e2e-test-images/resource-consumer:1.4 --expose --service-overrides='{ "spec": { "type": "LoadBalancer" } }' --port 8080 --requests='cpu=500m,memory=256Mi'
kubectl get services resource-consumer

There are two IPs. The first one is internal, while the second one is the external load-balanced IP. Both serve port 8080. (Use second one)

curl --data "millicores=300&durationSec=600" http://<EXTERNAL-IP>:8080/ConsumeCPU

300 millicores will be consumed for 600 seconds.

Image

Docker image of Resource Consumer can be found in Google Container Registry as gcr.io/kubernetes-e2e-test-images/resource-consumer:1.4

Use cases

Cluster size autoscaling

  1. Consume more resources on each node that is specified for autoscaler
  2. Observe that cluster size increased

Horizontal autoscaling of pod

  1. Create consuming RC and start consuming appropriate amount of resources
  2. Observe that RC has been resized
  3. Observe that usage on each replica decreased

Vertical autoscaling of pod

  1. Create consuming pod and start consuming appropriate amount of resources
  2. Observed that limits has been increased

Analytics