Commit Graph

635 Commits

Author SHA1 Message Date
Michael Crosby
e68bdbe9d9 Capture more error locations during stress tests
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-02-07 15:50:29 -05:00
Lantao Liu
050ff32dea Support trace level.
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-02-06 02:30:05 +00:00
Michael Crosby
78bd07afef
Merge pull request #1746 from AkihiroSuda/split-differ-interface
diff: resplit Applier from Differ
2018-02-05 09:18:56 -05:00
Michael Crosby
8ee29a17e6 Bump gc threshold to 40%
Doing tests, this is a better balance for the threshold in reguards to
memory and cpu usage.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-02-02 11:00:34 -05:00
Michael Crosby
55aa0b415b Don't enable debug endpoints in default config
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-31 11:08:51 -05:00
Michael Crosby
98b53e0dd3 Allow tcp debug address
This uses a simple `IsAbs` check to see if we are using an on disk path
for a unix socket vs an address since we do not prefix addresses with
`unix://` or `tcp://`.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-31 10:19:33 -05:00
Michael Crosby
5f89502a24
Merge pull request #1790 from jessvalarezo/ctr-c-create
ctr: add container create cmd and config flag
2018-01-29 17:19:14 -05:00
Derek McGowan
b763777288
diff: rename differ to comparer
Remove combined interface and split implementations.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-01-26 16:32:09 -08:00
Phil Estes
d4fb0709c9
Add error return for missing params in ctr images
For missing required parameters adds error return before attempting any
actions to `ctr images` commands.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2018-01-26 09:14:13 -05:00
Jess Valarezo
2c9ce2e693 ctr: add container create, config flag for spec
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2018-01-25 17:16:13 -08:00
Derek McGowan
7e4403540d
Merge pull request #1969 from darrenstahlmsft/WindowsSnapshotter5
Implement Windows snapshotter and differ
2018-01-24 17:02:02 -08:00
Stephen Day
3fcc52b091
Merge pull request #2055 from stevvooe/aggressive-memory-shim
cmd/containerd-shim: aggressive memory reclamation
2018-01-24 16:06:57 -08:00
Stephen J Day
71e9f6dac2
cmd/containerd-shim, reaper: reduce channel allocation
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-01-24 15:14:08 -08:00
Stephen J Day
0e8f08476c
cmd/containerd-shim: aggressive memory reclamation
To avoid having the shim hold on to too much memory, we've made a few
adjustments to favor more aggressive reclamation of memory from the
operating system. Typically, this would be negligible, on the order of a
few megabytes, but this is impactful when running several containers.

The first fix is to lower the threshold used to determine when to run
the garbage collector. The second runs `runtime/debug.FreeOSMemory` at a
regular interval.

Under test, this result in a sustained memory usage of around 3.7 MB.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-01-24 14:51:13 -08:00
Darren Stahl
dcff993653 Update Windows runtime to use snapshotter and differ layers
This changes the Windows runtime to use the snapshotter and differ
created layers, and updates the ctr commands to use the snapshotter and differ.

Signed-off-by: Darren Stahl <darst@microsoft.com>
2018-01-23 14:40:24 -08:00
Darren Stahl
a5a9f91832 Implement Windows snapshotter and differ
This implements the Windows snapshotter and diff Apply function.
This allows for Windows layers to be created, and layers to be pulled
from the hub.

Signed-off-by: Darren Stahl <darst@microsoft.com>
2018-01-23 14:40:23 -08:00
Phil Estes
d7efcbc083
Merge pull request #2036 from stevvooe/use-buffer-pools
archive, cio, cmd, linux: use buffer pools
2018-01-23 15:00:41 -05:00
Phil Estes
f47f6af585
Remove unnecessary subreaper API from sys/
Given these same exact functions are both now available in
opencontainers/runc (libcontainer/system) package, and we only use the
`SetSubreaper` today from the shim, there seems to be no reason for
duplication.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2018-01-23 10:30:29 -05:00
Phil Estes
dc5964ccc2
Merge pull request #2037 from pauldotknopf/master
Made a public method to set environment variables and mounts for containers.
2018-01-22 17:05:26 -05:00
Stephen J Day
cd72819b53
archive, cio, cmd, linux: use buffer pools
To avoid buffer bloat in long running processes, we try to use buffer
pools where possible. This is meant to address shim memory usage issues,
but may not be the root cause.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-01-22 13:52:06 -08:00
Paul Knopf
b4c3cd7640
Add WithEnv and WithMount oci options
Signed-off-by: Paul Knopf <pauldotknopf@gmail.com>
2018-01-22 16:35:31 -05:00
Michael Crosby
e5740ca612
Merge pull request #2007 from AkihiroSuda/cio-fifo-path
cio: add WithFIFODir opt
2018-01-22 13:32:43 -05:00
Michael Crosby
4812f4be8f
Merge pull request #2030 from Random-Liu/print-plugin-default-config
Print default plugin config.
2018-01-22 11:12:20 -05:00
Lantao Liu
3d6fe5ad18 Print default plugin config.
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-01-19 22:51:32 +00:00
Michael Crosby
94602aea63 Add execs to stress tests
This improves the exec support so that they can run along with the
normal stress tests.  You don't have to pick exec stres or container
stress.  They both run at the same time and report the different values.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-19 13:44:54 -05:00
Stephen J Day
5cab90d270
log: remove log "module" system
After comtemplation, the complexity of the logging module system
outweighs its usefulness. This changeset removes the system and restores
lighter weight code paths. As a concession, we can always provide more
context when necessary to log messages to understand them without having
to fork the context for a certain set of calls.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-01-18 11:22:13 -08:00
Akihiro Suda
507a149488 cio: add WithFIFODir opt
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-01-18 14:33:56 +09:00
Akihiro Suda
1167035be3 ctr: promote cOpts over opts, as oci.WithImageConfig requires snapshot
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-01-11 14:04:11 +09:00
Michael Crosby
9f5182f394 Remove reaper from containerd daemon
This allows other packages and plugins to easily exec things without
racing with the reaper.

The reaper is mostly needed in the shim but can be removed in containerd
in favor of the `exec.Cmd` apis

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-10 11:51:58 -05:00
Daniel Nephin
184bc25629 Add unconvert linter
This linter checks for unnecessary type convertions.

Some convertions are whitelisted because their type is different
on 32bit platforms

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-01-09 17:36:44 -05:00
Kenfe-Mickael Laventure
0cc79a6ff6
Add no-pivot flag to ctr
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2018-01-09 07:48:30 -08:00
Michael Crosby
4e755ad383
Merge pull request #1947 from jessvalarezo/ctr-snapshot-diff
ctr: snapshots diff command
2018-01-03 11:34:29 -05:00
Jess Valarezo
625eb5e661 ctr: snapshots diff command
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2018-01-02 21:18:12 -05:00
Michael Crosby
399e3c57c3 Change ctr help for mount from dest to dst
dest is not valid, only destination and dst

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-12-15 16:40:25 -05:00
Phil Estes
afbbe43745
Merge pull request #1895 from dnephin/refactor-cio
Refactor cio package
2017-12-15 14:44:44 -05:00
Michael Crosby
0b318b476a Fix gauge constant
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-12-15 14:20:34 -05:00
Michael Crosby
0725b60402 Add binary sizes to stress test metrics
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-12-15 12:49:59 -05:00
Phil Estes
7c37625af4
Merge pull request #1914 from stevvooe/stabilize-snapshot-tree-output
cmd/ctr: stablize output of snapshot tree
2017-12-15 12:22:26 -05:00
Akihiro Suda
fad72b6ae4
Merge pull request #1910 from crosbymichael/stress-json
Add metrics endpoint to stress test tool
2017-12-15 19:00:17 +09:00
Michael Crosby
6ae0f5f7e2 Add error metric for stress tests
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-12-13 13:15:07 -05:00
Stephen J Day
9aeeefae55
cmd/ctr: stablize output of snapshot tree
Preserves the order of the tree output between each execution. Slightly
refactored the behavior to be more "object oriented".

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-12-12 14:50:05 -08:00
Michael Crosby
652e078078 Add commit to stress metric
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-12-12 14:14:39 -05:00
Michael Crosby
4d55298aab Add prom timer to stress
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-12-12 14:05:14 -05:00
Lantao Liu
0eebf9051b Fix containerd deadlock.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-12-12 01:45:14 +00:00
Daniel Nephin
7d4337e738 Reduce the number of IO constructors
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-12-11 15:07:09 -05:00
Michael Crosby
ca5f16c33e Move stress worker to new file
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-12-11 10:36:19 -05:00
Michael Crosby
9fcca96771 Add json output to stress test tool
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-12-11 10:33:55 -05:00
Stephen J Day
4d5c2860d2
release: prepare 1.0.0
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-12-04 21:14:02 -08:00
Derek McGowan
40c67fdf78
Merge pull request #1880 from AkihiroSuda/refactor-importer
importer: refactor and fix GC
2017-12-04 20:56:26 -08:00
Akihiro Suda
63401970c7 importer: refactor
- Use lease API (previoisly, GC was not supported)
- Refactored interfaces for ease of future Docker v1 importer support

For usage, please refer to `ctr images import --help`.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-05 12:48:32 +09:00