Because `shutdownErr` will likely be `nil` in the close select branch,
returning it to waiters will result in the waiting `(*Client).Call`
returning `(nil, nil)`. This should take whatever is set for the client
as the exit condition, which is likely to be `ErrClosed`.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
To gracefully handle scenarios where the connection is closed or the
client is closed, we now set the final error to be `ErrClosed`. Callers
can resolve it through using `errors.Cause` to detect this condition.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
BSD does have LOCAL_PEERCRED but x/sys.unix does not yet have support for it.
I will add support and do a proper fix later.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Due to a performance drop when calling `(*net.UnixConn).File`, this
change uses the `syscall.RawConn` directly to call `Control` on the
socket. The result is there is no longer a performance penalty to using
unix socket credentials.
The benchmarks after this change are as follows:
```
goos: linux
goarch: amd64
pkg: github.com/stevvooe/ttrpc
BenchmarkRoundTrip-8 50000 22474 ns/op 2593 B/op 43 allocs/op
BenchmarkRoundTripUnixSocketCreds-8 100000 22120 ns/op 2593 B/op 43 allocs/op
PASS
ok github.com/stevvooe/ttrpc 4.049s
```
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Because of issues with glibc, using the `os/user` package can cause when
calling `user.Current()`. Neither the Go maintainers or glibc developers
could be bothered to fix it, so we have to work around it by calling the
uid and gid functions directly. This is probably better because we don't
actually use much of the data provided in the `user.User` struct.
This required some refactoring to have better control over when the uid
and gid are resolved. Rather than checking the current user on every
connection, we now resolve it once at initialization. To test that this
provided an improvement in performance, a benchmark was added.
Unfortunately, this exposed a regression in the performance of unix
sockets in Go when `(*UnixConn).File` is called. The underlying culprit
of this performance regression is still at large.
The following open issues describe the underlying problem in more
detail:
https://github.com/golang/go/issues/13470https://sourceware.org/bugzilla/show_bug.cgi?id=19341
In better news, I now have an entire herd of shaved yaks.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Because ttrpc can be used with abstract sockets, it is critical to
ensure that only certain users can connect to the unix socket. This is
of particular interest in the primary use case of containerd, where a
shim may run as root and any user can connection.
With this, we get a few nice features. The first is the concept of a
`Handshaker` that allows one to intercept each connection and replace it
with one of their own. The enables credential checks and other measures,
such as tls. The second is that servers now support configuration. This
allows one to inject a handshaker for each connection. Other options
will be added in the future.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
The request and response requests opened up a nasty race condition where
waiters could find themselves either blocked or receiving errant errors.
The result was low performance and inadvertent busy waits. This
refactors the client to have a single request into the main client loop,
eliminating the race.
The reason for the original design was to allow a sender to control
request and response individually to make unit testing easier. The unit
test has now been refactored to use a channel to ensure that requests
are serviced on graceful shutdown.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This change increases the maximum message size to 4MB to be inline
with the grpc default. The buffer management approach has been changed
to use a pool to minimize allocations and keep memory usage low.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This apples logic to correctly Close a server, as well as implements
graceful shutdown. This ensures that inflight requests are not
interrupted and works similar to the functionality in `net/http`.
This required a fair bit of refactoring around how the connection is
managed. The connection now has an explicit wrapper object, ensuring
that shutdown happens in a coordinated fashion, whether or not a
forceful close or graceful shutdown is called.
In addition to the above, hardening around the accept loop has been
added. We now correctly exit on non-temporary errors and debounce the
accept call when encountering repeated errors. This should address some
issues where `SIGTERM` was not honored when dropping into the accept
spin.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Improve the test to conduct two concurrent calls. The test server now
just doubles the input and we use a unix socket to better represent what
will be used in production.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Following the convention of http2, we now use odd stream ids for client
initiated streams. This makes it easier to tell who initiates the
stream. We enforce the convention on the server-side.
This allows us to upgrade the protocol in the future to have server
initiated streams.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
With this changeset, ttrpc can now handle mutliple outstanding requests
and responses on the same connection without blocking. On the
server-side, we dispatch a goroutine per outstanding reequest. On the
client side, a management goroutine dispatches responses to blocked
waiters.
The protocol has been changed to support this behavior by including a
"stream id" that can used to identify which request a response belongs
to on the client-side of the connection. With these changes, we should
also be able to support streams in the future.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Rather than employ the typeurl package, we now generate code to
correctly allocate the incoming types from the caller. As a side-effect
of this activity, the services definitions have been split out into a
separate type that handles the full resolution and dispatch of the
method, incuding correctly mapping the RPC status.
This work is a pre-cursor to larger protocol change that will allow us
to handle multiple, concurrent requests.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
With this changeset, we have the first end to end generated server and
client. Most of the necessary types are in place and working. There are
a lot of details to work out, but this would be enough for to exercise
the end to end flow.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
The server unit test now manually mocks much of the generated code for a
given service, including back registration of the service. We avoid
having a package-level global descriptor in favor of a closures for the
handler dispatch function.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
