The image "e2eteam/powershell-helper:6.2.7-linux-cache" is a Linux image. Because we're running "docker buildx build --platform windows/amd64", docker buildx will consider it as a Windows image unless we explicitly specify otherwise. If the image's platform is not correctly identified, we can run into problems when trying to build the image.
We are already doing something similar with the windows-servercore-cache image.
We can cache the powershell-helper image's results into a scratch Linux image using
docker buildx. This will allow us to spend less time pulling the data we need from the
powershell-helper image when we need it.
Additionally, docker buildx might have some issues with cross-registry images, so this
will allow us to circumvent it.
This is to consume the changes for binding the udp listeners of netexec
to specific addresses.
Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
The current udp implementation listens on any for tcp, udp and tcp. There
are some cases where it makes sense to listen on specific addresses
(especially udp, see https://github.com/kubernetes/kubernetes/issues/95565).
This is because UDP is connectionless, and in order to conntrack to
work, the application must ensure that the src of the reply is the same
as the dest of the request. The easiest way to do that is to bind
explicitly on an ip.
Here we pass an optional parameter that contains a comma separated list
of addresses.
Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
The same SHA cannot be pushed twice to the staging registry. Because some images were
mirrored, their SHAs remained unchanged. This addresses this issue.
nginx expects to find its conf and logs folder locally, and fails if it cannot find them.
cd-ing into the the nginx folder solves this issue. This is a similar approach to the
echoserver image, which also uses nginx.
A 32-bit php was included in the images, instead of the 64-bit one. The base image
is nanoserver-based, which does not support 32-bit apps. Because of this, httpd
fails to start.
Additionally, we've previously removed the busybox-helper dependency, but was
left in in the httpd images. This removes the dependency from the httpd images.
Due to the dockerhub rate limiting, we had to find an alternative solution. We've mirrored the dockerhub
images into our own.
Additionally, our own busybox, httpd, and nginx images also have Windows support.
We've added Windows support to the resource-consumer image and 1.8 tag is already promoted.
We need to bump the VERSION, so we can promote the new image.
The PHP release page tends to get updated and changed every time there's a
new release, removing the old ones. Because of this, the PHP link in the
httpd and httpd-new images may become invalid.
Updating the links to the archives solves this issue.
Adds the httpd, nginx, images that are used in tests.
Two different versions of nginx have to build, and thus, the have
different folders. An ALIAS file was added to nginx-new in order to
keep the same image name.
A few details about the image builder postsubmit jobs changed, so the README had to
be updated as well.
Added a few extra bits of information regarding the Windows images.
- reset `binfmt_misc` is needn't when the building platform is non-amd64 and the
target arch is the same as building platform
- non-amd64 platform doesn't supported cross-build well, and there is no binary of
`qemu-user-static` able to do that, and thus skip the cross-build on non-amd64
platform.
Signed-off-by: Dave Chen <dave.chen@arm.com>
We cannot have any RUN commands in the Windows stage when using docker buildx,
which is why we were using the busybox-helper image. The purpose of the image
was to contain a few things that we would obtain by running a few commands:
- symlinks for the busybox binary
- run vcredist_x64.exe which would also give us the vcruntime140.dll which is
necessary for dig or httpd.
There are alternatives to the commands above that can be achieved in a Linux stage
as well:
- we can create the symlinks in a Linux stage with ln -s. Copying them over to
Windows will allow them to work just as well as if they were being copied over
from a Windows image. The 'Files\' prefix issue to the symlink target still persists.
- we can download the vcruntime140.dll directly, allowing us to skip the vcredist_x64.exe
installation.
The default value for the progress is ``auto``, which will eat the output of RUN commands. This makes it a bit hard to debug when issues occur. Changing that option to ``plain`` will ensure that the output is properly kept.
Currently, the image is not working properly because of the apparmor_parser giving this error:
Error relocating /sbin/apparmor_parser: secure_getenv: symbol not found
Updating musl to 1.1.20 or newer will fix this problem.
The metadata-concealment image does not have any BASEIMAGE file, which means
that the image will be built from scratch. In this case, there are a few
fixes that need to be made in the image-build.sh script.
Some of these images didn't have any job run for them. Some of these
images previously failed due to an issue that has been addressed since.
Making a change into their image directory will spawn a postsubmit job
that will build that image.
For debugging purposes, it should be useful to run ``docker version`` and ``docker buildx version`` in order to more easily check and verify issues encountered with the Image Builder.
We are planing to test and support 20H2 release of Windows, thus,
we need to build test images for it as well. The busybox image already
has a BASEIMAGE entry for it, but we also need to add it to the image-util.sh's
windows_os_versions, so the OS Version can be properly included in the
manifest list.
We are planing to test and support 20H2 release of Windows, thus,
we need to build test images for it as well. The busybox image already
has a BASEIMAGE entry for it, but we also need to add it to the image-util.sh's
windows_os_versions, so the OS Version can be properly included in the
manifest list.
Dockerhub will introduce rate limiting in November, and a lot of E2E tests
are relying on the busybox image. It could potentially become an issue
causing jobs to fail because of this.
Ideally, we'd have the busybox image mirrored on gcr.io, but that could take
some time. Until then, we can just have the Image Builder mirror the image
for us in the staging registry and use that for tests until this issue is
solved. The busybox image should NOT be promoted out of staging.
During the sig-testing meeting, it was decided that we should do the same
for the other images are hosted on dockerhub.
Two different versions of httpd and nginx have to be built, and thus, the have
different folders. An ALIAS file was added to httpd-new and nginx-new in order
to keep the same image name.
Before creating and bootstrapping a docker buildx instance, we need to call
register.sh with the -p yes flag. Without this, the docker buildx will only
support linux/amd64 and linux/386 platforms, meaning that it will fail when
trying to build images for other architecture types.
Additionally, the builder has to have qemu and its qemu-* binaries installed
in order to properly build the images. The recently created image
gcr.io/k8s-testimages/gcb-docker-gcloud:v20201130-750d12f has those requirements met.
Many README files and other docs contained a link to a an appspot
tracking app that is no longer active. Following the links leads to an
error about Go 1.9 no longer being supported. Go 1.9 support was dropped
in appspot in 2019 and disabled June 2020.
This also resulted in a broken image link displaying when viewing these
files on GitHub. Since the app is no longer functioning, and since it
causes a potentially (but granted, minor) confusing error to display,
this just removes those links as I don't believe they are needed
anymore.
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Currently, the Image Builder job is failing as it cannot build images
for other architecture types. This happens because the Image Builder image
does not have any of the expected qemu-* binaries in /usr/bin/ needed in order to
run qemu-binfmt-conf.sh with the -p yes flag, so that flag is removed.
The default cloudbuild has HOME=/builder/home and docker buildx is in /root/.docker/cli-plugins/docker-buildx
We need to set the Home to /root explicitly since we're using docker buildx
docker buildx requires DOCKER_CLI_EXPERIMENTAL=enabled to be set
in order to be used.
This environment variable is not getting plumbed through from the
test/images/cloudbuild.yaml file, causing the docker buildx commands
to fail.
Provides a response that includes a body and a method. This response
will enable a client (e2e test) to confirm that a proxy did not alter
the http method.
Adds Windows support for most of the images.
Adds a README explaining the image building process, including the
Windows Container image building process.
Currently, some of the E2E test images have Windows support, and one of the goals is for most of
them to have Windows support. For that, the Image Builder is currently building those Windows
container images using a few Windows Server nodes (for 1809, 1903, 1909) with Remote Docker
enabled which are hosted on an azure subscription dedicated for CNCF.
With this, the Windows nodes dependency is removed entirely, as the images can be also built with
docker buildx. One additional benefit to this is that adding new supported Windows OS versions
to the E2E test images manifest lists becomes a lot easier (we wouldn't have to create a new Windows
Server node that matches that new OS version, assign DNS name, update certificates, etc.), and it
also becomes easier for other people to build their own E2E windows test images.
However, some dependencies are still required to run on a Windows machine. To solve this, we can
just pull helper images: e2eteam/powershell-helper:6.2.7 and e2eteam/busybox-helper:1.29.0. Their
Dockerfiles and a Makefile for them has been included in this commit. If any change is required to
them, then a new image will be built and tagged under a different version, but they are pretty
straight-forward and shouldn't require changes.
However, there is a small concern when it comes to the build time: Windows servercore images are
very large (for example, mcr.microsoft.com/windows/servercore:ltsc2019 is 4.99GB uncompressed, and
about ~2 GB compressed - those images are already cached on the Windows Server builder nodes, so
this isn't an issue there), and we currently support 1809, 1903, and 1909 (soon to add 2004).
This can lead to build times that are too big.
We have changed the base image to nanoserver (uncompressed size: 250MB), but some images still
require some DLLs or some other dependencies that can be fetched from a servercore image.
A separate job has been defined that would build a scratch windows-servercore-cache image monthly,
and then we can just get those dependencies from this cache, which will be very small.
This would be preferred, as the Windows images update periodically, and those dependencies
could be updated as well.
'agnhost' image uses hardcoded 'cluster.local' value for DNS domain.
It leads to failure of a bunch of HPA tests when test cluster is
configured to use custom DNS domain and there is no alias for
default 'cluster.local' one.
So, fix it by reusing it's own function for reading DNS domain suffixes.
Signed-off-by: Valerii Ponomarov <kiparis.kh@gmail.com>
Using Windows nanoserver container images as a base instead of the current
Windows servercore image will reduce the image size by about ~10x.
However, the nanoserver image lacks several things we need:
- netapi32.dll
- powershell
- certain powershell commands
- chocolatey cannot be used
When building the nanoserver images, we are going to use a Windows servercore helper,
in which we are going to install the necessary dependencies, and then copy them over
to our nanoserver image, including necessary DLLs.
Other notable changes include:
- switch from wget to curl (wget was a powershell alias).
- implement in code getting the DNS suffix list and DNS server list.
- reimplement getting file permissions for mounttest.
When trying to build the s390x image, it would fail when running the apk
command with the following error:
ERROR: Unable to open root: Bad address
ERROR: Failed to open apk database: Bad address
This can be fixed by updating the third_party/multiarch/qemu-user-static/register/register.sh
and third_party/multiarch/qemu-user-static/register/qemu-binfmt-conf.sh scripts
and their usage to a newer version [1].
Additionally, the packages nginx-mod-http-lua and nginx-mod-http-lua-upstream
cannot be found in the regular http://dl-cdn.alpinelinux.org/alpine/v3.9/main/s390x/
repository, but we can use an older one [2].
[1] https://github.com/qemu/qemu/blob/master/scripts/qemu-binfmt-conf.sh
[2] http://dl-cdn.alpinelinux.org/alpine/v3.8/main
There's currently no way to know whether an error is for SCTP or
UDP, for example:
Jul 24 09:55:54.469: INFO: netserver-0[e2e-nettest-3476].container[webserver].log
2020/07/24 09:53:52 Started UDP server
2020/07/24 09:53:52 Error occurred. error:protocol not supported
In this case the "Error occurred. error:protocol not supported" is
actually for the SCTP socket. Make that more apparent.
bprashanth hasn't be active since the release of v1.11. Removing them
from test/images/pets/OWNERS would leave mkumatag as the sole approver.
But mkumatag is also an approver for test/images/OWNERS so this commit
removes the test/images/pets/OWNERS completely.
Note: we should try to find more OWNERS for test/images/pets instead,
but this cleanup is a short term solution to avoid the bot suggesting
inactive members for reviews and approval.
Currently, the jessie-dnsutils image fails to build for arm64 arch with the following
error:
GPG error: http://archive.debian.org jessie Release: The following signatures were invalid: KEYEXPIRED 1587841717
We can bypass this issue by adding a --force-yes flag when installing the needed dnsutils.
Adds reviewers to the OWNERS files in the kubernetes/test/images folder.
The reviewers are added automatically, based on their contributions on
an image (>= 20% code churn).
Note that the code churn is taken into account for authors, and not committers.
Adds OWNERS files for: ipc-utils, node-perf, nonroot, regression-issue-74839,
resource-consumer, sample-device-plugin.
The google cloud builder job is launched without the required Windows Image Builder nodes
certificates that are needed for authentication when building the Windows container images.
Adds a step in test/images/cloudbuild.yaml that fetches a secret containing the certificates.
Removes the fatal error from getIP and moves it to
retry loop so that application will not immediately
crash on failure.
Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
This updates the error messages when registering a
node to be more explicit about what error occurred
and how long it will wait to retry.
Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
Currently the guestbook application will fail if unable
to resolve TCP address on first attempt. If pod networking
is not setup when the application starts then it will be
unable to resolve, leading to frequent failures. This moves
the address resolution into the retry block so it will try
again if unsuccessful on first attempt.
Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
The current /exit method is not sufficient to test graceful shutdown
behaviors within Kube that allow services to remain available during
rolling restarts. Add support for `wait=DURATION` and
`timeout=DURATION` to the exit handler and wire that to the Go http
server's graceful termination.
With these methods netexec can be used in a pod to simulate graceful
shutdown by adding a preStop handler that hits the exit endpoint with
a timeout and wait period.
The agnhost image used for testing has a `netexec` path which supports
two new flags, `--tls-cert-file` and `--tls-private-key-file`. If the
former is provided, the HTTP server will be upgraded to HTTPS, using the
certificate (and private key) provided.
By default, there are keys already mounted into the container at
`/localhost.crt` and `/localhost.key`, which contain PEM-encoded TLS
certs with IP SANs for `127.0.0.1` and `[::1]`.
Adds reviewers to the OWNERS files in the kubernetes/test/images folder.
The reviewers are added automatically, based on their contributions on
an image (>= 20% code churn).
Note that the code churn is taken into account for authors, and not committers.
Adds OWNERS files for: cuda-vector-add, nonewprivs, pets, redis, volume.
Adds reviewers to the OWNERS files in the kubernetes/test/images folder.
The reviewers are added automatically, based on their contributions on
an image (>= 20% code churn).
Note that the code churn is taken into account for authors, and not committers.
Adds ONWERS files for: apparmor-loader, echoserver, jessie-dnsutils, metadata-concealment,
sample-apiserver.
The build times are a bit high for the image builder (~50 minutes), and it will a bit more
when Windows support will be added to the other test images. This commit changes the
machineType to N1_HIGHCPU_8.
Reenables Windows test image building. Added DOCKER_CERT_BASE_PATH (default value: $HOME),
which will contain the path where the certificates needed for Remote Docker Connection can
be found.
If a REMOTE_DOCKER_URL was not set for a particular OS version, exclude that image from the
manifest list. This fixes an issue where, if REMOTE_DOCKER_URL was not set for Windows Server 1909,
the Windows were completely excluded from the manifest list, including for Windows Server 1809
and 1903 which could have been built and pushed.
Sets "test-webserver" as the default CMD for kitten and nautilus. Since they are now based on
agnhost, they should be set to run test-webserver to maintain previous behaviour.
Bumps the agnhost version to 2.13, as 2.12 has already been promoted. 2.13 will contain
Windows support.
Adds Windows support for the kitten and nautilus images, so they can promoted together
with agnhost (they were not previously promoted).
Adds OWNERS files to: agnhost, busybox, kitten, nautilus.
Adds splitOsArch function to image-util.sh, which makes the script DRY-er.
When building a Windows test image, if REMOTE_DOCKER_URL is not set, skip the rest of the
building process for that image, which will save some time (no need to build binaries).
If a REMOTE_DOCKER_URL was not set for a particular OS version, exclude that image from the
manifest list. This fixes an issue where, if REMOTE_DOCKER_URL was not set for Windows Server 1909,
the Windows were completely excluded from the manifest list, including for Windows Server 1809
and 1903 which could have been built and pushed.
Sets "test-webserver" as the default CMD for kitten and nautilus. Since they are now based on
agnhost, they should be set to run test-webserver to maintain previous behaviour.
The current agnhost version is 2.12, 2.11 was not previously built as the
VERSION bumps merged one after the other, and the Image Promoter did not get to
build the 2.11 image.
In the current version, due to how make works, when building all the conformance
images (make all-push WHAT=all-conformance), ALL the images are being built first
before being pushed.
This PR will allow images to be built and pushed immediately afterwards, so the first
images that have been succesfully built are already pushed and promotable, even if
the the task failed on the last image, or it timed out.
A previous PR (#76838) introduced the ability to build and publish
Windows Test Images to kubernetes/test/images/image-util.sh.
Additionally, that PR also configured the Image Promoter to use a
few Windows Remote Docker build nodes to build the Windows Test Images,
however, there is a minor issue: the build container has a different $HOME
folder than expected (is: /builder/home, expected: /root - since it's the
root user), and the Remote Docker credentials are mounted in /root.
Because of that, image-build.sh cannot find the credentials it needs.
This will have to be properly fixed, but for now, we can just skip
the Windows image building part.
Extends agnhost with the capability to validate a mounted token against
the API server's OIDC endpoints.
Co-authored-by: Michael Taufen <mtaufen@google.com>
