Commit Graph

175 Commits

Author SHA1 Message Date
Andreea Florescu
1401badaaf fix test benchmark
With newer versions of git we need to add an exception
for the `workdir` which is owned by a different user
then the one that is running the git commands.

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2022-09-12 12:20:15 +03:00
Andreea Florescu
258161e88a update README with latest version
Also added a note that the version in the readme is not necessarily
the latest one.

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2022-08-26 19:48:52 +03:00
Andreea Florescu
6fbd7c7355 fix test commit format
The newer versions of git are checking the ownership of directories.
Since the `/workdir` is a shared directory with ownership of buildkite,
we need to add an exception for it.

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2022-08-26 19:48:52 +03:00
Andreea Florescu
b3ed1c9253 update to container version 16
This version adds some tooling for fuzzing and updated to Rust 1.63.0.

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2022-08-26 19:48:52 +03:00
Mathieu Poirier
99fe2eb2e0 Relax 50/72 rule for more descriptive titles
Expand the title and changelog description lines to 60 and 75
respectively.  That is in line with what other projects do and
allows for better narratives.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
2022-02-22 10:06:54 +02:00
Andreea Florescu
ee6be8ca18 make coverage test less sensitive to changes
Instead of accounting for any difference in coverage, only fail the test
if the difference between the current coverage and the new coverage is
higher than 0.5%.

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2022-02-14 10:36:49 +01:00
Sergii Glushchenko
d216a46879 Update to rustvmm/dev:v15 with Rust 1.58.1
Signed-off-by: Sergii Glushchenko <gsserge@amazon.com>
2022-01-21 17:04:21 +02:00
Sergey Glushchenko
d023262164 Use v14 container image with Rust 1.54
Signed-off-by: Sergii Glushchenko <gsserge@amazon.com>
2022-01-18 17:21:11 +02:00
Laura Loghin
b6858b232d run cargo audit with --deny warnings enabled
It is better to fail on warnings as well when running `cargo
audit` so we can catch problems in time.
Fixes https://github.com/rust-vmm/rust-vmm-ci/issues/52.

Signed-off-by: Laura Loghin <lauralg@amazon.com>
2022-01-11 15:11:28 +01:00
Sergey Glushchenko
aee82cf0a4 Add @gsserge to CODEOWNERS
Signed-off-by: Sergii Glushchenko <gsserge@amazon.com>
2021-11-01 16:50:08 +01:00
Andreea Florescu
b037be3396 update test_coverage to allow running it on crates
The test was assuming that we always want to run all the tests in an
workspace, but that is not the case when we have multiple crates that we
want to publish part of the same workspace.

To address this issue, we can now select to run the coverage test only
on the crate in scope. This can be achieved by passing the parameter
`--test-scope` with the value "crate". By default, tests are still
running as a workspace so that we do not need to update all the other
rust-vmm components that were counting on this feature.

The `test-scope` option can also be reused in other tests that are
making use of `--all`. For example, cargo build.

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2021-10-28 14:14:38 +03:00
Ahmed Abouzied
7931077cdc Assert no spaces in crate features JSON
Spaces in the `crate_features` JSON value can cause errors when running
kcov.

Fixes: https://github.com/rust-vmm/rust-vmm-ci/issues/74

Signed-off-by: Ahmed Abouzied <email@aabouzied.com>
2021-10-11 16:40:34 +03:00
Sergii Glushchenko
68d4dbf85c Check formatting of code examples in doc comments
Add format_code_in_doc_comments=true to the cargo fmt
config options.

Fixes: https://github.com/rust-vmm/rust-vmm-ci/issues/58

Signed-off-by: Sergii Glushchenko <gsserge@amazon.com>
2021-10-06 17:39:17 +03:00
Andreea Florescu
7f22582590 update container version to v13
In the new container version we're using ubuntu 20.04 as the base image
instead of ubuntu 18.04.

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2021-09-24 14:20:45 +03:00
alindima
ce13bfb1ef Make build warnings fatal, again
Add the RUSTFLAGS="-D warnings" to cargo build
commands

The yaml config file has been deleted in the meantime,
so we need to add this again.

Signed-off-by: alindima <alindima@amazon.com>
2021-09-23 19:19:35 +03:00
Catalin Dumitru
472dea07fb Document default timeout for pipeline step
Mentioned that each step in the autogenerated pipeline has a
default timeout of 5 minutes.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-09-23 17:11:18 +03:00
Catalin Dumitru
f67ef4c84d Update README.md
Included the new environment variable `TIMEOUTS_MIN` for overriding
step timeouts and updated the "Custom Pipeline" section to
reflect that we are also autogenerating the custom pipelines.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-09-16 17:11:28 +03:00
Catalin Dumitru
dd754bb88e Add timeout for Buildkite steps
Added a default timeout value for each Buildkite step that can be
overridden through the environment variable `TIMEOUTS_MIN`.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-09-16 17:11:28 +03:00
Catalin Dumitru
e58c6f698f Remove static pipeline
The use of the static file pipeline.yml is now deprecated. All
pipelines should use the dynamic script autogenerate_pipeline.py.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-09-16 16:34:19 +03:00
alindima
6e1bdaac8e Make build warnings fatal
Add the RUSTFLAGS="-D warnings" to cargo build
commands

Signed-off-by: alindima <alindima@amazon.com>
2021-09-15 10:24:01 +03:00
Catalin Dumitru
ae7db2d98a Rename master to main
Updated occurrences of "master" with "main".

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-09-02 12:33:44 +03:00
Catalin Dumitru
1311bfa03f Update clippy test command
Added the option --all-targets to the command in the clippy test for
clippy to also run on the integration tests.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-09-01 15:59:39 +03:00
Catalin Dumitru
d7ca3dc9f2 Add help option for scripts
Added help options -h, --help for the script that autogenerates
the pipeline and for the one that runs the tests locally. In the
former, replaced the module optparse (which is now deprecated)
with the module argparse.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-09-01 14:29:22 +03:00
Catalin Dumitru
8901e77522 Leftovers from #79
Added copyright info and fixed typos.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-08-26 12:34:31 +03:00
Catalin Dumitru
555474aaba Add script to run tests locally
The script uses the test_description.json file to get the test
names and the commands to be run.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-08-26 12:34:31 +03:00
Catalin Dumitru
cd3d97f923 Allow skipping tests when generating the pipeline
Added a new environment variable for skipping tests if needed.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-08-25 16:25:48 +03:00
Catalin Dumitru
cc6ed996a0 Format integration tests code
Split lines that were too long, adjusted intendation and fixed typos.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-08-25 16:25:48 +03:00
Andreea Florescu
ba2f45c3ac make the test description configurable
Make it possible for autogenerating pipelines that are not committed to
this repository, but that are passed as a parameter. The need for this
behavior comes from a problem we've been facing with repositories that
also have custom pipelines. It happens that the container version (which
impacts the rust version), and the docker plugin version are lagging
behind, and are making tests ocasionally fail. To make sure that both
custom pipelines and the rust-vmm-ci pipeline are using the same Rust &
other environment versions, we're using the same script for generating
those tests as well.

This cannot be applied to Windows pipelines as they're not using the
rust-vmm-container at the moment.

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2021-08-25 11:33:47 +03:00
Catalin Dumitru
877d1fb680 Update README.md
Updated the instructions with the new commands for the generation
of the dynamic pipeline.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-08-19 14:17:39 +03:00
Catalin Dumitru
b47488a9b9 Autogenerate pipeline fron json config file
Instead of having a static pipeline.yml, autogenerate it from a JSON
configuration file. The json approach is taken because we also want to
be able to run tests locally, and thus the json can be parsed once for
generating the pipeline, and once for generating a local test run.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-08-19 14:17:39 +03:00
Catalin Dumitru
a00c7d2f1b Update the pipeline steps that run the tests
In the other crates, the tests reside in rust-vmm-ci/intergation_tests.
When we test the CI itself, the tests reside in integration_tests.
Therefore we needed to make the command find the tests independently of
the crate it is run from.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-08-02 17:39:26 +03:00
Catalin Dumitru
0b9d283cff Create local rust crate and update config path
In order to be able to run the CI on rust-vmm-ci we needed to
make it a rust crate. Also the path to the coverage config
file needed to be updated in test_coverage.py so that the path
does not depend on the crate; a breadth-first search was used
to guarantee that the config file belongs to the crate that is
being tested.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-08-02 17:39:26 +03:00
Catalin Dumitru
7693628035 Update commit format test to work locally
Set default values for the remote name and
base branch when the test is run locally and
the Buildkite variables are not available and
the variables BASE_BRANCH and REMOTE are not set.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
2021-07-21 15:59:49 +03:00
Andreea Florescu
d2ab3c0908 use the latest docker plugin version
This version is needed because there are features (for example passing
devices to a container) that are not available in the old version.

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2021-06-22 16:22:43 +03:00
Keiichi Watanabe
9f7b0d9069 check commit authors for each commit
Although we check commit authors in test_commit_format.py to skip tests
for dependabot's commits, the current code only checks the author of the
head commit.
This PR make it check each commit's author so that we can test branches
where both a dependabot's commit and other commits exist.

Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org>
2021-06-22 11:18:24 +03:00
Alexandru Agache
6591890ee1 use rust-vmm-container v12
Compared to v11, the new version bumps the Rust toolchain version
from 1.46 to 1.52.1.

Signed-off-by: Alexandru Agache <aagch@amazon.com>
2021-06-14 11:20:58 +03:00
Andreea Florescu
24d66cdae6 update clippy check
We were not testing clippy for all features, the workspace, bins, and
examples.

Partially addresses: https://github.com/rust-vmm/rust-vmm-ci/issues/21

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2021-03-15 18:24:13 +02:00
Andreea Florescu
ebc701641f exclude dependabot from the 50/72 commit rule
When dependabot updates the rust-vmm-ci submodule, the git commit
sumamry is over 50 chars. To be able to merge PR, exclude the dependabot
user from this checks.

Fixes: https://github.com/rust-vmm/rust-vmm-ci/issues/61

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2021-02-23 12:50:55 +02:00
Andreea Florescu
63fdf0f635 update rust-vmm-container
This new version brings new utilities.

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2021-02-18 16:04:05 +02:00
Federico Ponzi
98a26fe4c2 Typo fix in README.md 2021-01-20 09:42:42 +02:00
Laura Loghin
631c82acd3 mount tmp in test pipelines
This is needed for running the integration tests in
vmm-reference. They need access to resources from tmp.

Signed-off-by: Laura Loghin <lauralg@amazon.com>
2021-01-06 14:58:04 +02:00
Laura Loghin
03fcf08e11 don't run commit test for repos specified with git
Until we find a safe way to run the commit format test for
repositories specified with git, we should skip that test
for those repos.
Added a conditional for the corresponding buildkite step
that will run the commit test only if BUILDKITE_REPO
doesn't start with `git@`.

Signed-off-by: Laura Loghin <lauralg@amazon.com>
2020-12-08 12:17:21 +02:00
Andreea Florescu
3b7377cb51 fixed typo in readme
Fixes: https://github.com/rust-vmm/rust-vmm-ci/issues/39

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2020-12-03 16:38:20 +02:00
Andreea Florescu
c9430ee33a add gitignore file
Add pycache to gitignore.

Signed-off-by: Andreea Florescu <fandree@amazon.com>
2020-11-27 11:22:45 +02:00
Alexandra Iordache
e1108f1c90 buildkite: re-enable cargo audit test
Signed-off-by: Alexandra Iordache <aghecen@amazon.com>
2020-10-16 08:53:15 +03:00
Henry Wang
02004b5bd1 Add a flag that saves the coverage output dir
This commit adds a pytest command line option `--no-cleanup` for
the coverage test. Running the coverage test with this option will
let users save the `kcov_output` directory so that it is easier to
inspect the coverage report.

Signed-off-by: Henry Wang <henry.wang@arm.com>
2020-10-15 09:40:21 +03:00
Samuel Ortiz
97025bdb9b buildkite: Skip lines should be shorter than 70 chars
Welcome back to the 80s.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-10-14 15:07:41 +03:00
Samuel Ortiz
c83003c634 buildkite: Skip cargo audit check temporarily
Until our container moves to cargo audit v0.12.0.
See https://github.com/RustSec/advisory-db/issues/414#issuecomment-702479926

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-10-14 15:00:21 +03:00
Samuel Ortiz
c8cf2b70dc buildkite: Fix audit label indentation
Commit b3acb307f add the cargo-audit label but indentation is wrong,
making it an invalid YAML, at least from buildkite's point of view.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-10-14 12:42:13 +03:00
Anthony Riley II
b3acb307f1 Fixes: https://github.com/rust-vmm/rust-vmm-ci/issues/8
Co-authored-by: Laura Loghin <46103760+lauralt@users.noreply.github.com>
Signed-off-by: Anthony Riley II <riley@anth1y.io>
2020-10-09 11:04:02 +03:00