The Container Images for Windows Server 2022 have been published, and
we can start building test images using them, so we can start adding
jobs for them.
The image versions for the e2e test images have been bumped in a previous
commit, but haven't been promoted yet. We don't need to bump them here.
httpd-2.4.46-win64-VC15.zip no longer exists, so we have to use
httpd-2.4.48-win64-VC15.zip instead.
The Container Images for Windows Server 2022 have been published, and
we can start building test images using them, so we can start adding
jobs for them.
The image versions for the e2e test images have been bumped in a previous
commit, but haven't been promoted yet. We don't need to bump them here.
We're starting with windows-servercore-cache and busybox images, since
they are needed for the other images the most.
A previous added LD_FLAGS for the go binary compilation, but it's not
defined for all images.
In the test image build jobs, the image-util.sh script is not being run in a git
repository, which causes git log to fail.
In this case, we can use the PULL_BASE_SHA set in cloudbuild.yaml instead.
Currently, whenever agnhost/VERSION is bumped, the version in
agnhost/agnhost.go has to be bumped as well. This is also verified
on presubmit (build/dependencies.yaml).
This means that whenever we need to bump the agnhost image version,
someone has to approve the build/dependencies.yaml, which is not as
easy.
This commit removes the need for this check by automatically setting
the Version inside agnhost.go at build time, simplifying the process.
Removes any reference from the registry gcr.io/kubernetes-e2e-test-images in
kubernetes/kubernetes, replacing it with k8s.gcr.io/kubernetes-e2e-test-images.
In some cases, the images had to be updated since a few things have changed since
their original implementation, most notably being the fact that some of the images
have been centralized into the agnhost image.
Co-Authored-By: Claudiu Belu <cbelu@cloudbasesolutions.com>
For manifest lists containing Windows images, it is important to also have the "os.version"
annotation set, as it is needed by the Windows nodes, so they can pull the appropriate image
from the list.
Previously, the docker manifest CLI did not have the capability to set it, so, we had to set
it outselves in the manifest list's image JSON file. This is no longer necessary since
docker 20.10.0, which includes docker manifest annotate --os-version.
The docker installed in the image gcr.io/k8s-testimages/gcb-docker-gcloud:v20210622-762366a
satisfies this version requirement.
This image is the same as "gcr.io/authenticated-image-pulling/windows-nanoserver:v1"
that is used for the "should be able to pull from private registry with secret"
test on Windows.
Adding this image will allow other people to build and push their own
images to their own private registries.
We can indirectly retrieve the kube-cross version from the
`build/build-image/cross/VERSION` for the sample-apiserver. This allows
us to simplify the handling in `build/dependencies.yaml` as well as
the required approval (via `OWNERS`) if the kube-cross version changes.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
- ceph deploy on ARM64 depends on "libec_jerasure_neon.so" which is not included
in `ceph-base` package in fedora26 distro, updated the distro to fedora33 to
fix the issue
```
sh ./mon.sh "$(hostname -i)"
/usr/lib64/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file
```
- default pool `rbd` is not created on arm64, need to created this pool manually.
```
rbd import --image-feature layering block foo
rbd: error opening default pool 'rbd'
```
Signed-off-by: Dave Chen <dave.chen@arm.com>
Looking deeper into the logs there are a lot of errors like:
`script exited with error 1`
Initial reaction was that there was a problem with download, but it
looks like the script we use to register the qemu emulators may be at
fault, let's try this alternate mechanism.
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
The conformance test for ServiceAccountIssuerDiscovery is currently
configured with --in-cluster-discovery, which only supports token
validation against in-cluster endpoints. Many cloud providers provide
their own, external endpoints for OIDC discovery, and because the iss
claim in tokens will point to these endpoints, but the client in this
test only trusts the Cluster CA, it will fail to connect to the external
discovery endpoints when validating the token.
To ensure that the conformance test at least supports scenario where
both the discovery doc endpoint and JWKS endpoint are cluster-local and
the scenario where both endpoints are cluster-external, this PR has the
test try both and requires at least one to pass.
Caveat: The test still won't support a configuration where one
endpoint is cluster-local and the other is external. We don't yet have
evidence that this is a configuration that is used in practice, so this
initial hotfix will at least fix the conformance test for the "both
external" configuration we know providers already use. Note that if one
endpoint is cluster-local, and the other is cluster-external, tokens can
still only be validated in-cluster, because both endpoints must be
accessible to Relying Parties that validate tokens.
Currently, the only image left in gcr.io/kubernetes-e2e-test-images is the
cuda-vector-add:1.0 image.
According to 8408188cdf, the 1.0 image was based on CUDA 8.0,
while the 2.0 version is based on CUDA 10.0. We can simply rebuild the 1.0 image based on
the CUDA 8.0 image and then promote the new image.
Added ALIAS file, which specifies what the image name should be, similarly to how we build
multiple versions of nginx and httpd.
Note that the image CMD was changed from "./vectorAdd" to "nvidia-smi && ./vectorAdd" in 2.0.
The image "e2eteam/powershell-helper:6.2.7-linux-cache" is a Linux image. Because we're running "docker buildx build --platform windows/amd64", docker buildx will consider it as a Windows image unless we explicitly specify otherwise. If the image's platform is not correctly identified, we can run into problems when trying to build the image.
We are already doing something similar with the windows-servercore-cache image.
We can cache the powershell-helper image's results into a scratch Linux image using
docker buildx. This will allow us to spend less time pulling the data we need from the
powershell-helper image when we need it.
Additionally, docker buildx might have some issues with cross-registry images, so this
will allow us to circumvent it.
This is to consume the changes for binding the udp listeners of netexec
to specific addresses.
Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
The current udp implementation listens on any for tcp, udp and tcp. There
are some cases where it makes sense to listen on specific addresses
(especially udp, see https://github.com/kubernetes/kubernetes/issues/95565).
This is because UDP is connectionless, and in order to conntrack to
work, the application must ensure that the src of the reply is the same
as the dest of the request. The easiest way to do that is to bind
explicitly on an ip.
Here we pass an optional parameter that contains a comma separated list
of addresses.
Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
The same SHA cannot be pushed twice to the staging registry. Because some images were
mirrored, their SHAs remained unchanged. This addresses this issue.
nginx expects to find its conf and logs folder locally, and fails if it cannot find them.
cd-ing into the the nginx folder solves this issue. This is a similar approach to the
echoserver image, which also uses nginx.
A 32-bit php was included in the images, instead of the 64-bit one. The base image
is nanoserver-based, which does not support 32-bit apps. Because of this, httpd
fails to start.
Additionally, we've previously removed the busybox-helper dependency, but was
left in in the httpd images. This removes the dependency from the httpd images.
Due to the dockerhub rate limiting, we had to find an alternative solution. We've mirrored the dockerhub
images into our own.
Additionally, our own busybox, httpd, and nginx images also have Windows support.
We've added Windows support to the resource-consumer image and 1.8 tag is already promoted.
We need to bump the VERSION, so we can promote the new image.
The PHP release page tends to get updated and changed every time there's a
new release, removing the old ones. Because of this, the PHP link in the
httpd and httpd-new images may become invalid.
Updating the links to the archives solves this issue.
Adds the httpd, nginx, images that are used in tests.
Two different versions of nginx have to build, and thus, the have
different folders. An ALIAS file was added to nginx-new in order to
keep the same image name.
A few details about the image builder postsubmit jobs changed, so the README had to
be updated as well.
Added a few extra bits of information regarding the Windows images.
- reset `binfmt_misc` is needn't when the building platform is non-amd64 and the
target arch is the same as building platform
- non-amd64 platform doesn't supported cross-build well, and there is no binary of
`qemu-user-static` able to do that, and thus skip the cross-build on non-amd64
platform.
Signed-off-by: Dave Chen <dave.chen@arm.com>
We cannot have any RUN commands in the Windows stage when using docker buildx,
which is why we were using the busybox-helper image. The purpose of the image
was to contain a few things that we would obtain by running a few commands:
- symlinks for the busybox binary
- run vcredist_x64.exe which would also give us the vcruntime140.dll which is
necessary for dig or httpd.
There are alternatives to the commands above that can be achieved in a Linux stage
as well:
- we can create the symlinks in a Linux stage with ln -s. Copying them over to
Windows will allow them to work just as well as if they were being copied over
from a Windows image. The 'Files\' prefix issue to the symlink target still persists.
- we can download the vcruntime140.dll directly, allowing us to skip the vcredist_x64.exe
installation.
The default value for the progress is ``auto``, which will eat the output of RUN commands. This makes it a bit hard to debug when issues occur. Changing that option to ``plain`` will ensure that the output is properly kept.
Currently, the image is not working properly because of the apparmor_parser giving this error:
Error relocating /sbin/apparmor_parser: secure_getenv: symbol not found
Updating musl to 1.1.20 or newer will fix this problem.
The metadata-concealment image does not have any BASEIMAGE file, which means
that the image will be built from scratch. In this case, there are a few
fixes that need to be made in the image-build.sh script.
Some of these images didn't have any job run for them. Some of these
images previously failed due to an issue that has been addressed since.
Making a change into their image directory will spawn a postsubmit job
that will build that image.
For debugging purposes, it should be useful to run ``docker version`` and ``docker buildx version`` in order to more easily check and verify issues encountered with the Image Builder.
We are planing to test and support 20H2 release of Windows, thus,
we need to build test images for it as well. The busybox image already
has a BASEIMAGE entry for it, but we also need to add it to the image-util.sh's
windows_os_versions, so the OS Version can be properly included in the
manifest list.
We are planing to test and support 20H2 release of Windows, thus,
we need to build test images for it as well. The busybox image already
has a BASEIMAGE entry for it, but we also need to add it to the image-util.sh's
windows_os_versions, so the OS Version can be properly included in the
manifest list.
Dockerhub will introduce rate limiting in November, and a lot of E2E tests
are relying on the busybox image. It could potentially become an issue
causing jobs to fail because of this.
Ideally, we'd have the busybox image mirrored on gcr.io, but that could take
some time. Until then, we can just have the Image Builder mirror the image
for us in the staging registry and use that for tests until this issue is
solved. The busybox image should NOT be promoted out of staging.
During the sig-testing meeting, it was decided that we should do the same
for the other images are hosted on dockerhub.
Two different versions of httpd and nginx have to be built, and thus, the have
different folders. An ALIAS file was added to httpd-new and nginx-new in order
to keep the same image name.
Before creating and bootstrapping a docker buildx instance, we need to call
register.sh with the -p yes flag. Without this, the docker buildx will only
support linux/amd64 and linux/386 platforms, meaning that it will fail when
trying to build images for other architecture types.
Additionally, the builder has to have qemu and its qemu-* binaries installed
in order to properly build the images. The recently created image
gcr.io/k8s-testimages/gcb-docker-gcloud:v20201130-750d12f has those requirements met.
