- drop versions < 1.22 in the etcd map
- use 3.5.9-0 for >= 1.22 versions
- make the minimum version for external etcd 3.4.13-4 and max 3.5.9-0
- update images_test to not rely on a pinned etcd version in tests
note: the image 3.4.18-0 was never released in registry.k8s.io!
Since kube-proxy in LocalModeNodeCIDR needs to obtain the PodCIDR
assigned to the node it watches for the Node object.
However, kube-proxy startup process requires to have these watches in
different places, that opens the possibility of having a race condition
if the same node is recreated and a different PodCIDR is assigned.
Initializing the second watch with the value obtained in the first one
allows us to detect this situation.
Change-Id: I6adeedb6914ad2afd3e0694dcab619c2a66135f8
Signed-off-by: Antonio Ojea <aojea@google.com>
The command can be used to validate an input --config and throw
warnings and errors.
Add a new argument strctErrors to the functions
documentMapTo{Init|Join}Configuration(). This allows
to return errors from the calls to VerifyUnmarshalStrict().
Add a new function verifyKnownGVKs() in config/common.go
that is used to verify if a list of GVKs in a config file is
known. This function is used by the "validate" and "migrate"
commands.
Both commands now throw errors for unknown APIs or fields.
Move the Linux-specific conntrack setup code into a new
"platformSetup" rather than trying to fit it into the generic setup
code.
Also move metrics registration there.
TestBindToCommandArgRequirements has a .Pointer()
comparison that is not guaranteed to pass given
the reflected functions are closures.
This test is consistently failing on Go 1.21 rc1.
Remove said comparison from the test.
Before this commit, kubeadm starts kubelet before it creates
/etc/kubernetes/manifests. On boot, kubelet tries to load the
static pod manifests from this dir by calling `listConfig()`
7ad8303b96/pkg/kubelet/config/file.go (L97)
and it'll then try to start a file watcher every second for 20s
7ad8303b96/pkg/kubelet/config/file.go (L114)7ad8303b96/pkg/kubelet/config/file_linux.go (L51-L67)
If kubelet starts and calls `listConfig()` before kubeadm creates
`/etc/kubernetes/manifests` (while writing the static pods manifests),
the file watcher will be created less than a second after, but there
will be no changes to report, so the manifests will only be detected
on the next tick of `listTicker`, a bit less than 20s later
7ad8303b96/pkg/kubelet/config/file.go (L102-L103)
Even if we fixed the watch code to `listConfig()` just after starting the
inotify watch, watching source file is only supported on linux,
so moving the manifests generation before kubelet start fixes all
cases and make more sense IMO.
Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
If scheduler fails to be active (elected if leader election is enabled),
setting this option will not start informers so that to avoid memory
overhead.
Signed-off-by: Eric Lin <exlin@google.com>
kubelet option '--image-gc-low-threshold' description says it 'should not be larger than that of --image-gc-high-threshold'. But actually, it must less than '--image-gc-high-threshold', according to 02659772cb/pkg/kubelet/apis/config/validation/validation.go (L84)
If the etcd member is not found, then it has already been removed, and
kubeadm reset should immediately complete the 'remove-etcd-member'
phase. Previously, the phase would complete only once the
exponential-backoff retry expired, up to 3 minutes duration.
This commit also fixes a semantic error in etcd.GetMemberID. Previously,
the function returned 0 if no member was found, but 0 is not a valid
member ID.
* update serial number to a valid non-zero number in ca certificate
* fix the existing problem (0 SerialNumber in all certificate) as part of this PR in a separate commit
Rather than duplicating some of the KubeProxyConfiguration into
ProxyServer, just store the KubeProxyConfiguration itself so later
code can reference it directly.
For the fields that get platform-specific defaults (Mode,
DetectLocalMode), fill the defaults directly into the
KubeProxyConfiguration rather than keeping the original there and the
defaulted version in the ProxyServer.
Validate the --detect-local-mode value in the API object validation
rather than doing it separately later. Also, remove runtime checks and
unit tests for cases that would be blocked by validation
This touches cases where FromInt() is used on numeric constants, or
values which are already int32s, or int variables which are defined
close by and can be changed to int32s with little impact.
Signed-off-by: Stephen Kitt <skitt@redhat.com>
Rather than duplicating some of the KubeProxyConfiguration into
ProxyServer, just store the KubeProxyConfiguration itself so later
code can reference it directly.
For the fields that get platform-specific defaults (Mode,
DetectLocalMode), fill the defaults directly into the
KubeProxyConfiguration rather than keeping the original there and the
defaulted version in the ProxyServer.
Validate the --detect-local-mode value in the API object validation
rather than doing it separately later. Also, remove runtime checks and
unit tests for cases that would be blocked by validation
This commit syncs RBAC from coredns/deployment and removes a get nodes
RBAC.
Historically the federation CoreDNS plugin needed the nodes resource to
fetch zone and region labels.
However, the CoreDNS federation plugin was deprecated and cleaned up a
long time ago and removed the Nodes RBAC requirement here in
`coredns/deployment` coredns.yaml.sed:
https://github.com/coredns/deployment/pull/229
This change however, never made it to `kubernetes/kubernetes`.
Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
testing.T.Cleanup ensures the environment is restored after a test and
any of its parallel sub-tests. It's possible that some of these can be
simplified further to T.Setenv(key, ""), but I did not investigate.
T.Setenv ensures that the environment is returned to its prior state
when the test ends. It also panics when called from a parallel test to
prevent racy test interdependencies.
T.Setenv ensures that the environment is returned to its prior state
when the test ends. It also panics when called from a parallel test to
prevent racy test interdependencies.
The winkernel proxy was originally created by copying+pasting from the
iptables code, but some iptables-specific things were never removed
(and one function got left behind after its functionality was moved
into the shared proxy code).
This commit is the main API piece of KEP-3257 (ClusterTrustBundles).
This commit:
* Adds the certificates.k8s.io/v1alpha1 API group
* Adds the ClusterTrustBundle type.
* Registers the new type in kube-apiserver.
* Implements the type-specfic validation specified for
ClusterTrustBundles:
- spec.pemTrustAnchors must always be non-empty.
- spec.signerName must be either empty or a valid signer name.
- Changing spec.signerName is disallowed.
* Implements the "attest" admission check to restrict actions on
ClusterTrustBundles that include a signer name.
Because it wasn't specified in the KEP, I chose to make attempts to
update the signer name be validation errors, rather than silently
ignored.
I have tested this out by launching these changes in kind and
manipulating ClusterTrustBundle objects in the resulting cluster using
kubectl.
Add an arg to ParseNodeIPArgument saying whether to allow dual-stack
IPs for external cloud providers. Update kubelet for the new API, but
always pass "false" for now.
The same code should be used to parse the command-line argument and
the annotation. Unfortunately, for compatiblity reasons, they have to
handle invalid inputs differently...
(It doesn't make sense to put this code in cloud-provider, since
ParseNodeIPArgument is used for the non-cloud-provider case too.)
Most of the individual controllers were already converted earlier. Some log
calls were missed or added and then not updated during a rebase. Some of those
get updated here to fill those gaps.
Adding of the name to the logger used by each controller gets
consolidated in this commit. By using the name under which the
controller is registered we ensure that the names in the log
are consistent.
* scheduler(NodeResourcesFit): calculatePodResourceRequest in PreScore phase
* scheduler(NodeResourcesFit and NodeResourcesBalancedAllocation): calculatePodResourceRequest in PreScore phase
* modify the comments and tests.
* revert the tests.
* don't need consider nodes.
* use list instead of map.
* add comment for podRequests.
* avoid using negative wording in variable names.
* migrating pkg/controller/serviceaccount to contextual logging
Signed-off-by: Naman <namanlakhwani@gmail.com>
* small nit
Signed-off-by: Naman <namanlakhwani@gmail.com>
* capitalising first letter of error
Signed-off-by: Naman <namanlakhwani@gmail.com>
* addressed review comments
Signed-off-by: Naman <namanlakhwani@gmail.com>
* small nit to add key
Signed-off-by: Naman <namanlakhwani@gmail.com>
---------
Signed-off-by: Naman <namanlakhwani@gmail.com>
* migrated controller/replicaset to contextual logging
Signed-off-by: Naman <namanlakhwani@gmail.com>
* small nits
Signed-off-by: Naman <namanlakhwani@gmail.com>
* addressed changes
Signed-off-by: Naman <namanlakhwani@gmail.com>
* small nit
Signed-off-by: Naman <namanlakhwani@gmail.com>
* taking t as input
Signed-off-by: Naman <namanlakhwani@gmail.com>
---------
Signed-off-by: Naman <namanlakhwani@gmail.com>
Since Topology manager is graduating to GA, we remove
internal configuration variable names with `Experimental`
prefix.
There is no expected change in behavior, only trival
variable renaming.
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
data structure is what returned if everything okay, but this
structure is not validated at all both in `init` and `join` cmd.
Signed-off-by: Dave Chen <dave.chen@arm.com>
Updating the required code and docs for SeccompDefault to go GA, which
now means that we enable the feature per default.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
The manual conversion was needed to adapt `ClusterConfiguration.DNS` from v1beta2
to v1beta3, now that v1beta2 is removed from the code base, the manual conversion
is unnecessary as well.
The help to reduce the maintenance efforts.
Signed-off-by: Dave Chen <dave.chen@arm.com>
Back up kubelet config file for `kubeadm upgrade apply`, some code
refactoring is done to de-dup some redundant code logic.
Signed-off-by: Dave Chen <dave.chen@arm.com>
The root cause for that error is because `rename` doesn't work
across different mount points.
The kubelet config file and back up directory are mounted to
different file system in kinder environment.
```
df /var/lib/kubelet/config.yaml | tail -n1 | awk '{print $1}'
/dev/sda2
df /etc/kubernetes/tmp/kubeadm-kubelet-configxxx | tail -n1 | awk '{print $1}'
overlay
```
Call `cp` instead of `rename` to back up the kubelet file would fix
that issue.
Signed-off-by: Dave Chen <dave.chen@arm.com>
This addresses the TODO item so that the old kubelet config file could
be recovered if something goes wrong.
Signed-off-by: Dave Chen <dave.chen@arm.com>
Co-authored-by: Paco Xu <paco.xu@daocloud.io>
In fact, this actually uses pkg/util/node's GetHostname() but takes
the unit tests from cmd/kubeadm/app/util's private fork of that
function since they were more extensive. (Of course the fact that
kubeadm had a private fork of this function is a strong argument for
moving it to component-helpers.)
When trying to bring up a cluster via kubeadm with these feature gates enabled,
kube-proxy fails because it didn't know about them:
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker
featureGates: {"DynamicResourceAllocation":true,"ContextualLogging":true}
runtimeConfig: {"resource.k8s.io/v1alpha1":"true"}
=>
2023-01-20T07:07:54.474966617Z stderr F E0120 07:07:54.474846 1 run.go:74] "command failed" err="failed complete: unrecognized feature gate: ContextualLogging"
The effect of the logging feature gates is minor for kube-proxy, supporting
them is mostly useful for the sake of consistency and to support kubeadm.
ContextForChannel uses a goroutine to transform a channel close to
a context cancel. However, this exposes a synchronization issue if
we want to unify the underlying implementation between contextless
and with context - a ConditionFunc that closes the channel today
expects the behavior that no subsequent conditions will be invoked
(we have a test in wait_test.go TestUntilReturnsImmediately that
verifies this expectation). We can't unify the implementation
without ensuring this property holds.
To do that this commit changes from the goroutine propagation to
implementing context.Context and using stopCh as the Done(). We
then implement Err() by returning context.Canceled and stub the
other methods. Since our context cannot be explicitly cancelled
by users, we cease to return the cancelFn and callers that need
that behavior must wrap the context as normal.
This should be invisible to clients - they would already observe
the same behavior from the context, and the existing error
behavior of Poll* is preserved (which ignores ctx.Err()).
As a side effect, one less goroutine is created making it more
efficient.
run the testcase with `-v` flag will reveal the warning,
e.g.
`W1103 ... Usage of CRI endpoints without URL scheme is deprecated...`
Signed-off-by: Dave Chen <dave.chen@arm.com>
Set the `cri-socket` both in flags and config file will hit errors,
this should not be a valid case to validate in current testcases.
Signed-off-by: Dave Chen <dave.chen@arm.com>
enforceRequirements will run preflight checks, including whether the user
is privileged is not. Because of this, the test will make different assertions
based on the user's UID. However, we don't have UIDs on Windows, so we're asserting
the wrong thing.
This fix addresses the issue.
`genCSRConfig.kubeadmConfig` is possible to be nil if there any error
from the config loading, so access the field should only be done if
there is no error in the previous step.
Signed-off-by: Dave Chen <dave.chen@arm.com>
In the dual-stack case, iptables.NewDualStackProxier and
ipvs.NewDualStackProxier filtered the nodeport addresses values by IP
family before creating the single-stack proxiers. But in the
single-stack case, the kube-proxy startup code just passed the value
to the single-stack proxiers without validation, so they had to
re-check it themselves. Fix that.
Kube-proxy was checking that iptables supports both IPv4 and IPv6 and
falling back to single-stack if not. But it always fell back to the
primary IP family, regardless of which family iptables supported...
Fix it so that if the primary IP family isn't supported then it bails
out entirely.
It was just saying the copy of file failed with `exit status 1`,
no much details for what's going wrong.
Combine the stderr and stdout and show those info will be easier
for us to fix the problem.
Signed-off-by: Dave Chen <dave.chen@arm.com>
The path module has a few different functions:
Clean, Split, Join, Ext, Dir, Base, IsAbs. These functions do not
take into account the OS-specific path separator, meaning that they
won't behave as intended on Windows.
For example, Dir is supposed to return all but the last element of the
path. For the path "C:\some\dir\somewhere", it is supposed to return
"C:\some\dir\", however, it returns ".".
Instead of these functions, the ones in filepath should be used instead.
If we are dry-running, do not attempt to fetch the /version
resource and just return the stored FakeServerVersion,
which is done when constructing the dry-run client in
upgrade/common.go#getClient().
The problem here is that during upgrade
dry-run client reactors are backed by a dynamic client
via NewClientBackedDryRunGetterFromKubeconfig() and
for GetActions there seems to be no analog to
Discovery().Serverversion() resource for a dynamic client(?).
The kubeadm dry run client reactor code is flawed as it assumes
all invoked "get" verb actions can be casted to GetAction.
Apparently that is not the case when Discovery().ServerVersion()
and other discovery calls are made. In such cases the action
type is the bare ActionImpl.
Catch if an action can be casted to ActionImpl and construct a
GetAction from it. GetActionImpl only suppersets ActionImpl with
a Name field (empty string in this case).
Add unit test for Discovery().ServerVersion().
There seems to be a bug where it's not possible to write to
/etc/kubernetes/tmp... at the time of backing up the old kubelet
config.yaml
Also this kubelet config backup only targets "upgrade node"
and it should also target "upgrade apply".
Revert the related changes until a fully working feature
is implemented.
This addresses the TODO item so that the old kubelet config file could
be recovered if something goes wrong.
Signed-off-by: Dave Chen <dave.chen@arm.com>
The --contention-profiling flag enables block profiling by calling
SetBlockProfileRate(). It, however does not call SetMutexProfileFraction
which enables mutex profiling.
Mutex profiling gives us information about lock contention whereas
block profiling on the other hand gives us information on gorotuines
being blocked on sync primitives.
Updating the docs to "block profiling" in order to make it more accurate.
Eventhough a block profile may reflect contention points on locks to some
extent, it also talks about other sources where goroutines may be blocked.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Create a new conversion Factory interface for CRDs, and split out
NewDelegatingConverter as a standalone package-level function, instead
of being part of CRConverterFactory.
Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>
Priority is normally set by the admission controller when missing.
For static pods there seems to be some corner cases, as, if priority
is not explicitly set, Graceful Node Shutdown will shutdown
static pods with the other pods that have priority 0 or unset.
Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
These testcases are too vague, they are not test against the scenario
they want, but instead all of them are failed due to client cannot
be created.
`kubeconfig` file is created and mocked the function of `loadConfig` in
order to make those testcases valid.
Signed-off-by: Dave Chen <dave.chen@arm.com>
Dependencies need to be updated to use
github.com/container-orchestrated-devices/container-device-interface.
It's not decided yet whether we will implement Topology support
for DRA or not. Not having any toppology-related code
will help to avoid wrong impression that DRA is used as a hint
provider for the Topology Manager.
The controller uses the exact same logic as the generic ephemeral inline volume
controller, just for inline ResourceClaimTemplate -> ResourceClaim.
In addition, it supports removal of pods from the ReservedFor field when those
pods are known to not need the claim anymore. At the moment, only this special
case is supported. Removal of arbitrary objects would imply granting full read
access to all types to determine whether a) an object is gone and b) if the
current incarnation is the one which is listed in ReservedFor. This may get
added later.
This adds a new resource.k8s.io API group with v1alpha1 as version. It contains
four new types: resource.ResourceClaim, resource.ResourceClass, resource.ResourceClaimTemplate, and
resource.PodScheduling.
This change enables hot reload of encryption config file when api server
flag --encryption-provider-config-automatic-reload is set to true. This
allows the user to change the encryption config file without restarting
kube-apiserver. The change is detected by polling the file and is done
by using fsnotify watcher. When file is updated it's process to generate
new set of transformers and close the old ones.
Signed-off-by: Nilekh Chaudhari <1626598+nilekhc@users.noreply.github.com>
This change adds a flag --encryption-provider-config-automatic-reload
which will be used to drive automatic reloading of the encryption
config at runtime. While this flag is set to true, or when KMS v2
plugins are used without KMS v1 plugins, the /healthz endpoints
associated with said plugins are collapsed into a single endpoint at
/healthz/kms-providers - in this state, it is not possible to
configure exclusions for specific KMS providers while including the
remaining ones - ex: using /readyz?exclude=kms-provider-1 to exclude
a particular KMS is not possible. This single healthz check handles
checking all configured KMS providers. When reloading is enabled
but no KMS providers are configured, it is a no-op.
k8s.io/apiserver does not support dynamic addition and removal of
healthz checks at runtime. Reloading will instead have a single
static healthz check and swap the underlying implementation at
runtime when a config change occurs.
Signed-off-by: Monis Khan <mok@microsoft.com>
This patch adds new Kubelet option topologyManagerPolicyOptions.
To introduce new TopologyManager options, first we need to introduce new
flag called `topology-manager-policy-options` to allow users to modify
behaviour of best-effort and restricted policies.
Signed-off-by: PiotrProkop <pprokop@nvidia.com>
CPUManager is going GA, thus it makes little sense
to keep the names of the internal configuration
variables `Experimental*`.
Trivial rename only.
Signed-off-by: Francesco Romani <fromani@redhat.com>
With graduation of device plugins to GA in 1.26, the feature gate is
enabled by default so `devicePluginEnabled` field no longer needs to
be passed at the time of Container Manager creation.
In addition to that, we remove the `ManagerStub` as it is no longer
needed.
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
For parity with `init` and `join`, the test file only
covers some basic scenarios and is expected to be expanded
when the config file for `kubeadm reset` is ready.
Signed-off-by: Dave Chen <dave.chen@arm.com>
The init config file fetched from cluster won't have `ignorePreflightErrors`
set, and shouldn't try to reuse it even it's set with any value.
Instead, it should be defined in reset config file and fetch from it when the
reset config is ready.
Signed-off-by: Dave Chen <dave.chen@arm.com>
It was saying `discovery.bootstrapToken` is invalid with empty value, while
it is actually `discovery.bootstrapToken.caCertHashes` invalid.
Signed-off-by: Dave Chen <dave.chen@arm.com>
Currently, there are some unit tests that are failing on Windows due to
various reasons:
- config options not supported on Windows.
- files not closed, which means that they cannot be removed / renamed.
- paths not properly joined (filepath.Join should be used).
- time.Now() is not as precise on Windows, which means that 2
consecutive calls may return the same timestamp.
- different error messages on Windows.
- files have \r\n line endings on Windows.
- /tmp directory being used, which might not exist on Windows. Instead,
the OS-specific Temp directory should be used.
- the default value for Kubelet's EvictionHard field was containing
OS-specific fields. This is now moved, the field is now set during
Kubelet's initialization, after the config file is read.
Kube/proxy, in NodeCIDR local detector mode, uses the node.Spec.PodCIDRs
field to build the Services iptables rules.
The Node object depends on the kubelet, but if kube-proxy runs as a
static pods or as a standalone binary, it is not possible to guarantee
that the values obtained at bootsrap are valid, causing traffic outages.
Kube-proxy has to react on node changes to avoid this problems, it
simply restarts if detect that the node PodCIDRs have changed.
In case that the Node has been deleted, kube-proxy will only log an
error and keep working, since it may break graceful shutdowns of the
node.
Some of the unit tests cannot pass on Windows due to various reasons:
- fsnotify does not have a Windows implementation.
- Proxy Mode IPVS not supported on Windows.
- Seccomp not supported on Windows.
- VolumeMode=Block is not supported on Windows.
- iSCSI volumes are mounted differently on Windows, and iscsiadm is a
Linux utility.
- All certs will be created under the folder of `/etc/kubernetes/tmp/kubeadm-join-dryrunxxx`
if the `dry-run` mode is enabled.
- Try to make each phase idempotent by resetting the cert dir with `dry-run` mode
Signed-off-by: Dave Chen <dave.chen@arm.com>
Previously, the err msg will repeat the token ID of the form "[a-z0-9]{6}"
if the token doesn't match with the desired format.
Signed-off-by: Dave Chen <dave.chen@arm.com>
Fix conversion errors
Changed the order
update
update
fix manaul coversions
keep the global parameter for backward compatibility
Address Wei's comments
Fix an error
Fix issues
Add unit tests for validation
Fix a comment
Address comments
Update comments
fix verifiation errors
Add tests for scheme_test.go
Convert percentageOfNodesToScore to pointer
Fix errors
Resolve conflicts
Fix testing errors
Address Wei's comments
Revert IntPtr to Int changes
Address comments
Not overrite percentageOfNodesToScore
Fix a bug
Fix a bug
change errs to err
Fix a nit
Remove duplication
Address comments
Fix lint warning
Fix an issue
Update comments
Clean up
Address comments
Revert changes to defaults
fix unit test error
Update
Fix tests
Use default PluginConfigs
The phase `kubeadm init phase upload-certs` intends to be used if
secret "kubeadm-certs" is expired and deleted.
But the help msg was saying this command cannot run on its own.
It's fine to run this command on its own to upload local certs to secret.
Signed-off-by: Dave Chen <dave.chen@arm.com>
This change updates the API server code to load the encryption
config once at start up instead of multiple times. Previously the
code would set up the storage transformers and the etcd healthz
checks in separate parse steps. This is problematic for KMS v2 key
ID based staleness checks which need to be able to assert that the
API server has a single view into the KMS plugin's current key ID.
Signed-off-by: Monis Khan <mok@microsoft.com>
- The sub-phases like `kubeadm reset phase cleanup-node` which
could be run independently would be able to support the `dry-run`
mode as well.
- Consistent with the sub-phases which support the `dry-run` mode
already, such as `kubeadm init phase control-plane apiserver`.
- Prepare for the day when each of those sub-phases could be run
independently.
Signed-off-by: Dave Chen <dave.chen@arm.com>
phase `kubeadm init phase kubeconfig/*` must run after the phase
`kubeadm init phase certs/ca` to get the client cert and key singed
by the CA.
print out the way to generate the CA file is more useful than just say
the file cannot be loaded.
Signed-off-by: Dave Chen <dave.chen@arm.com>
wire up feature_gate.go with metrics via AddMetrics method
Change-Id: I9b4f6b04c0f4eb9bcb198b16284393d21c774ad8
wire in metrics to kubernetes components
Change-Id: I6d4ef8b26f149f62b03f32d1658f04f3056fe4dc
rename metric since we're using the value to determine if enabled is true or false
Change-Id: I13a6b6df90a5ffb4b9c5b34fa187562413bea029
Update staging/src/k8s.io/component-base/featuregate/feature_gate.go
Co-authored-by: Jordan Liggitt <jordan@liggitt.net>
The generation of `RunData` for init and join is relative time consuming,
the data should be shared / reused whenever possible.
Signed-off-by: Dave Chen <dave.chen@arm.com>
