Commit Graph

8706 Commits

Author SHA1 Message Date
Lantao Liu
3a7fef51fc Create etcd user in cloud init.
Signed-off-by: Lantao Liu <lantaol@google.com>
2020-05-08 10:14:53 -07:00
Sebastiaan van Stijn
dc92ad6520
Replace errors.Cause() with errors.Is()
Dependencies may be switching to use the new `%w` formatting
option to wrap errors; switching to use `errors.Is()` makes
sure that we are still able to unwrap the error and detect the
underlying cause.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-08 14:36:45 +02:00
Akihiro Suda
834f58bd0c
Merge pull request #4237 from thaJeztah/bump_pkg_errors
vendor: github.com/pkg/errors v0.9.1
2020-05-08 20:16:28 +09:00
Sebastiaan van Stijn
f212e7d1fe
vendor: github.com/pkg/errors v0.9.1
full diff: https://github.com/pkg/errors/compare/v0.8.1...v0.9.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-07 22:56:05 +02:00
Phil Estes
990076b731
Merge pull request #4228 from thaJeztah/refactor_reaper
Refactor reaper-related functionality to be in the sys/reaper package
2020-05-07 14:32:55 -04:00
Michael Crosby
b2aef14c0c
Merge pull request #4210 from cpuguy83/ctr_mount
Add commands to mount/unmount image from ref
2020-05-07 13:07:38 -04:00
Michael Crosby
35ffd2376e
Merge pull request #4232 from fahedouch/install_runc_with_gosu
[ci] install runc with gosu
2020-05-07 13:01:48 -04:00
Phil Estes
80859e8fd8
Merge pull request #4235 from renzhengeek/renzhen/fix-iohang
snapshots/devmapper: fix race windown causing IO hangup
2020-05-07 08:55:22 -04:00
Eric Ren
63b7587cd6 snapshots/devmapper: fix race windown causing IO hangup
The issue beblow happens several times beforing the root
cause found:

  1. A `fdisk -l` process has being hung up for a long time;
  2. A image layer snapshot device is visiable to dmsetup, which
       should *not* happen because it should be deactivated after
       `Commit()`;

The backtrace of `fdisk` is always the same over time:

```bash
[<ffffffff810bbc6a>] io_schedule+0x2a/0x80
[<ffffffff81295a3f>] do_blockdev_direct_IO+0x1e9f/0x2f10
[<ffffffff81296aea>] __blockdev_direct_IO+0x3a/0x40
[<ffffffff81290e43>] blkdev_direct_IO+0x43/0x50
[<ffffffff811b8a14>] generic_file_read_iter+0x374/0x960
[<ffffffff81291ad5>] blkdev_read_iter+0x35/0x40
[<ffffffff8125229b>] new_sync_read+0xfb/0x240
[<ffffffff81252406>] __vfs_read+0x26/0x40
[<ffffffff81252b96>] vfs_read+0x96/0x130
[<ffffffff812540e5>] SyS_read+0x55/0xc0
[<ffffffff81003c04>] do_syscall_64+0x74/0x180
```

The root cause is, in Commit(), there's a race window between
`SuspendDevice()` and `DeactivateDevice()`, which may cause the
IOs of a process or command like `fdisk` on the "suspended" device
hang up forever. It has twofold:

  1. The IOs suspends on the devices;
  2. The device is in `Suspended` state, because it's deactivated with
     `deferred` flag and without `force` flag;

So they cannot make progress.

One reproducer is:
 1. enlarge the race window by putting sleep seconds there;
 2. run `while true; do sudo fdisk -l; sleep 0.5; done` on one terminal;
 3. and pull image on another terminal;

Fixes it by:
 1. Resume the devices again after flushing IO by suspend;
 2. Remove device without `deferred` flag;

Fix: #4234
Signed-off-by: Eric Ren <renzhen@linux.alibaba.com>
2020-05-07 07:46:45 +08:00
fahedouch
24b331ad1e omit sudo when EUID eqto 0
Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>

omit sudo when EUID eqto 0

Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>

omit sudo when EUID eqto 0

Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>

omit sudo when EUID eqto 0

Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>

use gosu to omit sudo in GA

Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>

use gosu to omit sudo in GA

Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>

set working-dir for <<Setup gosu>> step

Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>

fix job permissions

Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>
2020-05-05 20:13:27 +02:00
Phil Estes
b1f514641f
Merge pull request #4230 from thaJeztah/remove_libcontainer_system
Remove remaining uses of libcontainer/system package
2020-05-04 15:18:57 -04:00
Sebastiaan van Stijn
0088c2de80
sys: RunningInUserNS(): use sync.Once
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-04 18:57:02 +02:00
Sebastiaan van Stijn
76c62f2722
sys: simplify RunningInUserNS to original implementation
Given that we're only interested in detecting if userns is
enabled, and no further details about the mapping, we can
revert this function to go back to its original implementation
in github.com/lxc/lxd/shared/util.go

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-04 18:56:59 +02:00
Sebastiaan van Stijn
6a9b94927f
Remove remaining uses of libcontainer/system package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-04 18:56:57 +02:00
Phil Estes
21fd2ccf8b
Merge pull request #4227 from thaJeztah/cleanup_sys
sys/proc: remove GetSystemCPUUsage as it is unused
2020-05-04 11:56:16 -04:00
Sebastiaan van Stijn
37c839e594
sys/proc: remove GetSystemCPUUsage as it is unused
After e115b52ce2 was merged, this
function was no longer in use, so we should be able to remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-04 13:40:19 +02:00
Sebastiaan van Stijn
1b66fecad3
Integrate sys.SetSubreaper, sys.GetSubreaper in sys/reaper package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-04 08:44:02 +02:00
Sebastiaan van Stijn
fffea312aa
move sys.Reap() into sys/reaper package
It was only used in sys/reaper, and after moving it, that package
no longer depends on the `sys` package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-04 08:44:00 +02:00
Sebastiaan van Stijn
295e74008f
sys: remove custom setChildSubreaper const
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-04 08:43:57 +02:00
Wei Fu
d7cad51275
Merge pull request #4229 from thaJeztah/bump_logrus
vendor: sirupsen/logrus v1.6.0
2020-05-04 08:59:53 +08:00
Wei Fu
b354188618
Merge pull request #4226 from thaJeztah/remove_libcontainer_from_shim
Remove libcontainer from containerd-shim
2020-05-04 08:51:02 +08:00
Sebastiaan van Stijn
ec12d3042f
vendor: sirupsen/logrus v1.6.0
full diff: https://github.com/sirupsen/logrus/compare/v1.5.0...v1.6.0

- Add flag to disable quotes in TextFormatter
- Revert "fix race conditions on entry"
    - fixes Deadlock during Entry.Infof after upgrade to v1.5.0
    - fixes Deadlock when using WithField inside of hook
    - fixes Overly-aggressive mutex locks

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-03 19:06:42 +02:00
Sebastiaan van Stijn
23aab35fdb
Remove libcontainer from containerd-shim
Replace the libcontainer variant with the one in our sys package

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-03 16:53:24 +02:00
Akihiro Suda
c668d7660d
Merge pull request #4225 from fuweid/update-cri
vendor: update containerd/cri with 65830369b6 from master
2020-05-03 20:53:51 +09:00
Wei Fu
ae7cdc814b vendor: update containerd/cri
Commit ID: 65830369b6 from master

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2020-05-03 10:44:49 +08:00
Mike Brown
65830369b6
Merge pull request #1468 from fuweid/me-change-rollback
RunPodSandbox: destroy network if fails or invalid
2020-05-01 18:09:07 -05:00
Wei Fu
7d7104c549
Merge pull request #4222 from dims/update-to-latest-hcsshim
Update to latest hcsshim - 0.8.9
2020-05-01 17:23:50 +08:00
Wei Fu
9c506d0af4
Merge pull request #1465 from dsyer/master
Clarify that plugin names are long in version = 2
2020-05-01 12:09:28 +08:00
Wei Fu
48e797c77f RunPodSandbox: destroy network if fails or invalid
Should destroy the pod network if fails to setup or return invalid
net interface, especially multiple CNI configurations.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2020-05-01 12:07:33 +08:00
Davanum Srinivas
c1096864a7
Update to latest hcsshim - 0.8.9
Switch to ad tagged version of hcsshim. Context here:
https://github.com/kubernetes/kubernetes/issues/87420#issuecomment-620837610

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2020-04-30 21:42:30 -04:00
Phil Estes
4be98fa28b
Merge pull request #4220 from thaJeztah/update_terminal_sequences
vendor konsorten/go-windows-terminal-sequences v1.0.3
2020-04-29 09:19:14 -04:00
Phil Estes
a10b502568
Merge pull request #4221 from thaJeztah/bump_logrus
vendor: sirupsen/logrus v1.5.0
2020-04-29 09:18:27 -04:00
Phil Estes
f3c03c0b81
Merge pull request #4219 from ktock/bc
vendor: update containerd/cri dc7afe8fbe
2020-04-29 09:16:18 -04:00
Sebastiaan van Stijn
ea182aec8a
vendor: sirupsen/logrus v1.5.0
full diff: https://github.com/sirupsen/logrus/compare/v1.4.1...v1.5.0

- Ability to DisableHTMLEscape when using the JSON formatter
- Support/fixes for go 1.14
- Many many bugfixes

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-29 12:21:43 +02:00
Sebastiaan van Stijn
68f1515932
vendor konsorten/go-windows-terminal-sequences v1.0.3
full diff: https://github.com/konsorten/go-windows-terminal-sequences/compare/v1.0.1...v1.0.3

- Fixes a panic when using logrus on Go 1.14
- Fix compile error on linux and darwin

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-29 12:12:03 +02:00
ktock
046a520c91 vendor: update containerd/cri dc7afe8fbe
full diff: 61b7af7564...dc7afe8fbe

This commit includes moving up to the latest critools(1.18.0).

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2020-04-29 14:49:54 +09:00
Mike Brown
dc7afe8fbe
Merge pull request #1466 from ktock/parallel-snapshot
Pass chained layer digests to snapshotter for parallel snapshot preparation
2020-04-28 11:12:37 -05:00
ktock
ca661c8dc9 Pass chained layer digests to snapshotter for parallel snapshot preparation
Currently, CRI plugin passes each layer digest to remote snapshotters
sequentially, which leads to sequential snapshots preparation. But it costs
extra time especially for remote snapshotters which need to connect to the
remote backend store (e.g. registries) for checking the snapshot existence on
each preparation.

This commit solves this problem by introducing new label
`containerd.io/snapshot/cri.chain` for passing all layer digests in an image to
snapshotters and by allowing them to prepare these snapshots in parallel, which
leads to speed up the preparation.

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2020-04-28 15:03:08 +09:00
Phil Estes
3f95bc5f18
Merge pull request #4213 from fahedouch/remove_unnecessary_condition
remove if condition because it is handled by the sdNotify daemon func
2020-04-27 10:57:56 -04:00
Dave Syer
5ac8515bf0 Clarify that plugin names are long in version = 2
Otherwise it's confusing for readers who just need quick reference
for plugin configurations.

Signed-off-by: Dave Syer <dsyer@pivotal.io>
2020-04-27 14:35:41 +01:00
Mike Brown
4ea4ca99c7
Merge pull request #1455 from 6WIND/master
fix incomplete host device for PrivilegedWithoutHostDevices
2020-04-26 22:28:20 -05:00
Wei Fu
197dca5a35
Merge pull request #1464 from mikebrow/test-apparmor-profile
move up to latest critools; add apparmor profile check
2020-04-27 11:16:28 +08:00
fahedouch
8e76d18eff remove if condition because it is handled by the sdNotify daemon func
Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>

lint code

Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>
2020-04-26 23:18:34 +02:00
Mike Brown
776c125e4f move up to latest critools; add apparmor profile check
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2020-04-26 16:16:48 -05:00
Brian Goff
1a10211e3f WithLease: always return context and done fn
We should never return a nil context because of the way this function is
typically used... e.g.

```
  ctx, done, err := containerd.WithLease(ctx)
```

If there is an error `ctx` will be nil and any error handling may cause
an NPE if it tries to use `ctx`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-04-25 21:16:43 -07:00
Brian Goff
98b30f4690 Add commands to mount/unmount image from ref
Example:

```terminal
$ mkdir /opt/busybox
$ ctr image mount docker.io/library/busybox:latest /opt/busybox
/opt/busybox
$ ls -lh /opt/busybox
total 40K
drwxr-xr-x 2 root   root     12K Apr 14 01:10 bin
drwxr-xr-x 2 root   root    4.0K Apr 14 01:10 dev
drwxr-xr-x 3 root   root    4.0K Apr 14 01:10 etc
drwxr-xr-x 2 nobody nogroup 4.0K Apr 14 01:10 home
drwx------ 2 root   root    4.0K Apr 14 01:10 root
drwxrwxrwt 2 root   root    4.0K Apr 14 01:10 tmp
drwxr-xr-x 3 root   root    4.0K Apr 14 01:10 usr
drwxr-xr-x 4 root   root    4.0K Apr 14 01:10 var
$ ctr image unmount /opt/busybox
$ ls -lh /opt/busybox
total 0
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-04-25 21:00:21 -07:00
Phil Estes
36952e989a
Merge pull request #4208 from estesp/moar-startup-time
Allow more time for containerd to start for integration
2020-04-23 14:35:25 -04:00
Phil Estes
5d4a9bae9f
Merge pull request #4199 from fuweid/update-go-version-zuul
.zuul: update go version to 1.13.10
2020-04-23 14:35:10 -04:00
Phil Estes
cd0e62f5c9
Allow more time for containerd to start for integration
Currently hardcoded to 2 seconds; in GitHub actions we see random
cancellation of our integration suite right at 2 seconds even
though containerd is within milliseconds of being ready.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2020-04-23 11:23:38 -04:00
Mike Brown
14c4b47bb1
Merge pull request #1459 from mikebrow/containerd-project-readme
link readme to containerd/project
2020-04-23 10:19:34 -05:00