Portworx Volume Driver in Kubernetes

- Add a new type PortworxVolumeSource
- Implement the kubernetes volume plugin for Portworx Volumes under pkg/volume/portworx
- The Portworx Volume Driver uses the libopenstorage/openstorage specifications and apis for volume operations.

Changes for k8s configuration and examples for portworx volumes.

- Add PortworxVolume hooks in kubectl, kube-controller-manager and validation.
- Add a README for PortworxVolume usage as PVs, PVCs and StorageClass.
- Add example spec files

Handle code review comments.

- Modified READMEs to incorporate to suggestions.
- Add a test for ReadWriteMany access mode.
- Use util.UnmountPath in TearDown.
- Add ReadOnly flag to PortworxVolumeSource
- Use hostname:port instead of unix sockets
- Delete the mount dir in TearDown.
- Fix link issue in persistentvolumes README
- In unit test check for mountpath after Setup is done.
- Add PVC Claim Name as a Portworx Volume Label

Generated code and documentation.
- Updated swagger spec
- Updated api-reference docs
- Updated generated code under pkg/api/v1

Godeps update for Portworx Volume Driver
- Adds github.com/libopenstorage/openstorage
- Adds go.pedge.io/pb/go/google/protobuf
- Updates Godep Licenses
This commit is contained in:
Aditya Dani
2016-12-19 23:17:11 +00:00
parent dba0af3675
commit 28df55fc31
84 changed files with 14212 additions and 3090 deletions

191
vendor/github.com/libopenstorage/openstorage/LICENSE generated vendored Normal file
View File

@@ -0,0 +1,191 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
Copyright 2015 Openstorage.org.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

208
vendor/github.com/libopenstorage/openstorage/api/api.go generated vendored Normal file
View File

@@ -0,0 +1,208 @@
package api
import (
"fmt"
"strconv"
"strings"
"time"
)
// Strings for VolumeSpec
const (
Name = "name"
SpecEphemeral = "ephemeral"
SpecShared = "shared"
SpecSize = "size"
SpecScale = "scale"
SpecFilesystem = "fs"
SpecBlockSize = "block_size"
SpecHaLevel = "repl"
SpecPriority = "io_priority"
SpecSnapshotInterval = "snap_interval"
SpecAggregationLevel = "aggregation_level"
SpecDedupe = "dedupe"
SpecPassphrase = "passphrase"
)
// OptionKey specifies a set of recognized query params.
const (
// OptName query parameter used to lookup volume by name.
OptName = "Name"
// OptVolumeID query parameter used to lookup volume by ID.
OptVolumeID = "VolumeID"
// OptLabel query parameter used to lookup volume by set of labels.
OptLabel = "Label"
// OptConfigLabel query parameter used to lookup volume by set of labels.
OptConfigLabel = "ConfigLabel"
// OptCumulative query parameter used to request cumulative stats.
OptCumulative = "Cumulative"
)
// Api client-server Constants
const (
OsdVolumePath = "osd-volumes"
OsdSnapshotPath = "osd-snapshot"
)
// Node describes the state of a node.
// It includes the current physical state (CPU, memory, storage, network usage) as
// well as the containers running on the system.
type Node struct {
Id string
Cpu float64 // percentage.
MemTotal uint64
MemUsed uint64
MemFree uint64
Avgload int
Status Status
GenNumber uint64
Disks map[string]StorageResource
MgmtIp string
DataIp string
Timestamp time.Time
StartTime time.Time
Hostname string
NodeData map[string]interface{}
// User defined labels for node. Key Value pairs
NodeLabels map[string]string
}
// Cluster represents the state of the cluster.
type Cluster struct {
Status Status
// Id is the ID of the cluster.
Id string
// NodeId is the ID of the node on which this cluster object
// is initialized
NodeId string
// Nodes is an array of all the nodes in the cluster.
Nodes []Node
}
// StatPoint represents the basic structure of a single Stat reported
// TODO: This is the first step to introduce stats in openstorage.
// Follow up task is to introduce an API for logging stats
type StatPoint struct {
// Name of the Stat
Name string
// Tags for the Stat
Tags map[string]string
// Fields and values of the stat
Fields map[string]interface{}
// Timestamp in Unix format
Timestamp int64
}
func DriverTypeSimpleValueOf(s string) (DriverType, error) {
obj, err := simpleValueOf("driver_type", DriverType_value, s)
return DriverType(obj), err
}
func (x DriverType) SimpleString() string {
return simpleString("driver_type", DriverType_name, int32(x))
}
func FSTypeSimpleValueOf(s string) (FSType, error) {
obj, err := simpleValueOf("fs_type", FSType_value, s)
return FSType(obj), err
}
func (x FSType) SimpleString() string {
return simpleString("fs_type", FSType_name, int32(x))
}
func CosTypeSimpleValueOf(s string) (CosType, error) {
obj, err := simpleValueOf("cos_type", CosType_value, s)
return CosType(obj), err
}
func (x CosType) SimpleString() string {
return simpleString("cos_type", CosType_name, int32(x))
}
func GraphDriverChangeTypeSimpleValueOf(s string) (GraphDriverChangeType, error) {
obj, err := simpleValueOf("graph_driver_change_type", GraphDriverChangeType_value, s)
return GraphDriverChangeType(obj), err
}
func (x GraphDriverChangeType) SimpleString() string {
return simpleString("graph_driver_change_type", GraphDriverChangeType_name, int32(x))
}
func VolumeActionParamSimpleValueOf(s string) (VolumeActionParam, error) {
obj, err := simpleValueOf("volume_action_param", VolumeActionParam_value, s)
return VolumeActionParam(obj), err
}
func (x VolumeActionParam) SimpleString() string {
return simpleString("volume_action_param", VolumeActionParam_name, int32(x))
}
func VolumeStateSimpleValueOf(s string) (VolumeState, error) {
obj, err := simpleValueOf("volume_state", VolumeState_value, s)
return VolumeState(obj), err
}
func (x VolumeState) SimpleString() string {
return simpleString("volume_state", VolumeState_name, int32(x))
}
func VolumeStatusSimpleValueOf(s string) (VolumeStatus, error) {
obj, err := simpleValueOf("volume_status", VolumeStatus_value, s)
return VolumeStatus(obj), err
}
func (x VolumeStatus) SimpleString() string {
return simpleString("volume_status", VolumeStatus_name, int32(x))
}
func simpleValueOf(typeString string, valueMap map[string]int32, s string) (int32, error) {
obj, ok := valueMap[strings.ToUpper(fmt.Sprintf("%s_%s", typeString, s))]
if !ok {
return 0, fmt.Errorf("no openstorage.%s for %s", strings.ToUpper(typeString), s)
}
return obj, nil
}
func simpleString(typeString string, nameMap map[int32]string, v int32) string {
s, ok := nameMap[v]
if !ok {
return strconv.Itoa(int(v))
}
return strings.TrimPrefix(strings.ToLower(s), fmt.Sprintf("%s_", strings.ToLower(typeString)))
}
func toSec(ms uint64) uint64 {
return ms / 1000
}
func (v *Stats) WriteThroughput() uint64 {
if v.IntervalMs == 0 {
return 0
}
return (v.WriteBytes) / toSec(v.IntervalMs)
}
func (v *Stats) ReadThroughput() uint64 {
if v.IntervalMs == 0 {
return 0
}
return (v.ReadBytes) / toSec(v.IntervalMs)
}
func (v *Stats) Latency() uint64 {
ops := v.Writes + v.Reads
if ops == 0 {
return 0
}
return (uint64)((v.IoMs * 1000) / (v.Writes + v.Reads))
}
func (v *Stats) Iops() uint64 {
if v.IntervalMs == 0 {
return 0
}
return (v.Writes + v.Reads) / toSec(v.IntervalMs)
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,398 @@
syntax = "proto3";
import "google/protobuf/timestamp.proto";
package openstorage.api;
option go_package = "api";
option java_multiple_files = true;
option java_package = "com.openstorage.api";
enum Status {
STATUS_NONE = 0;
STATUS_INIT = 1;
STATUS_OK = 2;
STATUS_OFFLINE = 3;
STATUS_ERROR = 4;
STATUS_NOT_IN_QUORUM = 5;
STATUS_DECOMMISSION = 6;
STATUS_MAINTENANCE = 7;
STATUS_STORAGE_DOWN = 8;
STATUS_STORAGE_DEGRADED = 9;
STATUS_NEEDS_REBOOT = 10;
STATUS_STORAGE_REBALANCE = 11;
STATUS_STORAGE_DRIVE_REPLACE = 12;
// Add statuses before MAX and update the number for MAX
STATUS_MAX = 13;
}
enum DriverType {
DRIVER_TYPE_NONE = 0;
DRIVER_TYPE_FILE = 1;
DRIVER_TYPE_BLOCK = 2;
DRIVER_TYPE_OBJECT = 3;
DRIVER_TYPE_CLUSTERED = 4;
DRIVER_TYPE_GRAPH = 5;
}
enum FSType {
FS_TYPE_NONE = 0;
FS_TYPE_BTRFS = 1;
FS_TYPE_EXT4 = 2;
FS_TYPE_FUSE = 3;
FS_TYPE_NFS = 4;
FS_TYPE_VFS = 5;
FS_TYPE_XFS = 6;
FS_TYPE_ZFS = 7;
}
enum GraphDriverChangeType {
GRAPH_DRIVER_CHANGE_TYPE_NONE = 0;
GRAPH_DRIVER_CHANGE_TYPE_MODIFIED = 1;
GRAPH_DRIVER_CHANGE_TYPE_ADDED = 2;
GRAPH_DRIVER_CHANGE_TYPE_DELETED = 3;
}
enum SeverityType {
SEVERITY_TYPE_NONE = 0;
SEVERITY_TYPE_ALARM = 1;
SEVERITY_TYPE_WARNING = 2;
SEVERITY_TYPE_NOTIFY = 3;
}
enum ResourceType {
RESOURCE_TYPE_NONE = 0;
RESOURCE_TYPE_VOLUME = 1;
RESOURCE_TYPE_NODE = 2;
RESOURCE_TYPE_CLUSTER = 3;
RESOURCE_TYPE_DRIVE = 4;
}
enum AlertActionType {
ALERT_ACTION_TYPE_NONE = 0;
ALERT_ACTION_TYPE_DELETE = 1;
ALERT_ACTION_TYPE_CREATE = 2;
ALERT_ACTION_TYPE_UPDATE = 3;
}
enum VolumeActionParam {
VOLUME_ACTION_PARAM_NONE = 0;
// Maps to the boolean value false
VOLUME_ACTION_PARAM_OFF = 1;
// Maps to the boolean value true.
VOLUME_ACTION_PARAM_ON = 2;
}
enum CosType {
NONE = 0;
LOW = 1;
MEDIUM = 2;
HIGH = 3;
}
// VolumeState represents the state of a volume.
enum VolumeState {
VOLUME_STATE_NONE = 0;
// Volume is transitioning to new state
VOLUME_STATE_PENDING = 1;
// Volume is ready to be assigned to a container
VOLUME_STATE_AVAILABLE = 2;
// Volume is attached to container
VOLUME_STATE_ATTACHED = 3;
// Volume is detached but associated with a container
VOLUME_STATE_DETACHED = 4;
// Volume detach is in progress
VOLUME_STATE_DETATCHING = 5;
// Volume is in error state
VOLUME_STATE_ERROR = 6;
// Volume is deleted, it will remain in this state
// while resources are asynchronously reclaimed
VOLUME_STATE_DELETED = 7;
}
// VolumeStatus represents a health status for a volume.
enum VolumeStatus {
VOLUME_STATUS_NONE = 0;
// Volume is not present
VOLUME_STATUS_NOT_PRESENT = 1;
// Volume is healthy
VOLUME_STATUS_UP = 2;
// Volume is in fail mode
VOLUME_STATUS_DOWN = 3;
// Volume is up but with degraded performance
// In a RAID group, this may indicate a problem with one or more drives
VOLUME_STATUS_DEGRADED = 4;
}
enum StorageMedium {
// Magnetic spinning disk.
STORAGE_MEDIUM_MAGNETIC = 0;
// SSD disk
STORAGE_MEDIUM_SSD = 1;
// NVME disk
STORAGE_MEDIUM_NVME = 2;
}
enum ClusterNotify {
// Node is down
CLUSTER_NOTIFY_DOWN = 0;
}
// StorageResource groups properties of a storage device.
message StorageResource {
// Id is the LUN identifier.
string id = 1;
// Path device path for this storage resource.
string path = 2;
// Storage medium.
StorageMedium medium = 3;
// True if this device is online.
bool online = 4;;
// IOPS
uint64 iops = 5;;
// SeqWrite
double seq_write = 6;
// SeqRead
double seq_read = 7;
// RandRW
double randRW = 8;
// Total size in bytes.
uint64 size = 9;;
// Physical Bytes used.
uint64 used = 10;
// True if this device is rotational.
string rotation_speed = 11;
// Timestamp of last time this device was scanned.
google.protobuf.Timestamp last_scan = 12;
}
// VolumeLocator is a structure that is attached to a volume
// and is used to carry opaque metadata.
message VolumeLocator {
// User friendly identifier
string name = 1;
// A set of name-value pairs that acts as search filters
map<string, string> volume_labels = 2;
}
message Source {
// A volume id, if specified will create a clone of the parent.
string parent = 1;
// Seed will seed the volume from the specified URI
// Any additional config for the source comes from the labels in the spec
string seed = 2;
}
// VolumeSpec has the properties needed to create a volume.
message VolumeSpec {
// Ephemeral storage
bool ephemeral = 1;
// Thin provisioned volume size in bytes
uint64 size = 2;
// Format disk with this FSType
FSType format = 3;
// Block size for filesystem
int64 block_size = 4;
// Specifies the number of nodes that are
// allowed to fail, and yet data is available
// A value of 0 implies that data is not erasure coded,
// a failure of a node will lead to data loss
int64 ha_level = 5;
// The COS, 1 to 9
CosType cos = 6;
// Perform dedupe on this disk
bool dedupe = 7;
// SnapshotInterval in minutes, set to 0 to disable snapshots
uint32 snapshot_interval = 8;
// Volume configuration labels
map<string, string> volume_labels = 9;
// Shared is true if this volume can be remotely accessed.
bool shared = 10;
// ReplicaSet is the desired replicaSet the volume want to be placed.
ReplicaSet replica_set = 11;
// Specifies the number of parts the volume can be aggregated from.
uint32 aggregation_level = 12;
// Encrypted is true if this volume will be cryptographically secured.
bool encrypted = 13;
// User passphrase if this is an encrypted volume
string passphrase = 14;
// SnapshotSchedule
string snapshot_schedule = 15;
// Scale allows autocreation of volumes.
uint32 scale = 16;
}
// Set of machine IDs (nodes) to which part of this volume is erasure coded - for clustered storage arrays
message ReplicaSet {
repeated string nodes = 1;
}
// List of name value mapping of driver specific runtime information.
message RuntimeStateMap {
map<string, string> runtime_state = 1;
}
// Volume represents a live, created volume.
message Volume {
// Self referential volume ID
string id = 1;
Source source = 2;
bool readonly = 3;
// User specified locator
VolumeLocator locator = 4;
// Volume creation time
google.protobuf.Timestamp ctime = 5;
// User specified VolumeSpec
VolumeSpec spec = 6;
// Volume usage
uint64 usage = 7;
// Time when an integrity check for run
google.protobuf.Timestamp last_scan = 8;
// Format FSType type if any
FSType format = 9;
VolumeStatus status = 10;
VolumeState state = 11;
// Machine ID (node) on which this volume is attached
// Machine ID is a node instance identifier for clustered systems.
string attached_on = 12;
string device_path = 14;
repeated string attach_path = 15;
// List of ReplicaSets which provide storage for this volume, for clustered storage arrays
repeated ReplicaSet replica_sets = 16;
// Last recorded error
string error = 17;
// List of name value mapping of driver specific runtime information.
repeated RuntimeStateMap runtime_state = 18;
string secure_device_path = 19;
// BackgroundProcessing is true if volume is attached but not by the user
bool background_processing = 20;
}
message Stats {
// Reads completed successfully
uint64 reads = 1;
// Time spent in reads in ms
uint64 read_ms = 2;
uint64 read_bytes = 3;
// Writes completed successfully
uint64 writes = 4;
// Time spent in writes in ms
uint64 write_ms = 5;
uint64 write_bytes = 6;
// IOs curently in progress
uint64 io_progress = 7;
// Time spent doing IOs ms
uint64 io_ms = 8;
// BytesUsed
uint64 bytes_used = 9;
// Interval in ms during which stats were collected
uint64 interval_ms = 10;
}
message Alert {
// Id for Alert
int64 id = 1;
// Severity of the Alert
SeverityType severity = 2;
// AlertType user defined alert type
int64 alert_type = 3;
// Message describing the Alert
string message = 4;
//Timestamp when Alert occured
google.protobuf.Timestamp timestamp = 5;
// ResourceId where Alert occured
string resource_id = 6;
// Resource where Alert occured
ResourceType resource = 7;
// Cleared Flag
bool cleared = 8;
// TTL in seconds for this Alert
uint64 ttl = 9;
}
message Alerts {
repeated Alert alert = 1;
}
message VolumeCreateRequest {
// User specified volume name and labels
VolumeLocator locator = 1;
// Source to create volume
Source source = 2;
// The storage spec for the volume
VolumeSpec spec = 3;
}
message VolumeResponse {
string error = 1;
}
message VolumeCreateResponse {
// ID of the newly created volume
string id = 1;
VolumeResponse volume_response = 2;
}
// VolumeStateAction specifies desired actions.
message VolumeStateAction {
// Attach or Detach volume
VolumeActionParam attach = 1;
// Mount or unmount volume
VolumeActionParam mount = 2;
string mount_path = 3;
// Device path returned in attach
string device_path = 4;
}
message VolumeSetRequest {
// User specified volume name and labels
VolumeLocator locator = 1;
// The storage spec for the volume
VolumeSpec spec = 2;
// State modification on this volume.
VolumeStateAction action = 3;
}
message VolumeSetResponse {
Volume volume = 1;
VolumeResponse volume_response = 2;
}
message SnapCreateRequest {
// volume id
string id = 1;
VolumeLocator locator = 2;
bool readonly = 3;
}
message SnapCreateResponse {
VolumeCreateResponse volume_create_response = 1;
}
message VolumeInfo {
string volume_id = 1;
string path = 2;
VolumeSpec storage = 3;
}
// GraphDriverChanges represent a list of changes between the filesystem layers
// specified by the ID and Parent. // Parent may be an empty string, in which
// case there is no parent.
// Where the Path is the filesystem path within the layered filesystem
message GraphDriverChanges {
string path = 1;
GraphDriverChangeType kind = 2;
}
message ClusterResponse {
string error = 1;
}
message ActiveRequest {
map<int64, string> ReqestKV = 1;
}
message ActiveRequests {
int64 RequestCount = 1;
repeated ActiveRequest ActiveRequest = 2;
}

View File

@@ -0,0 +1,141 @@
package client
import (
"crypto/tls"
"fmt"
"net"
"net/http"
"net/url"
"sync"
"time"
)
var (
httpCache = make(map[string]*http.Client)
cacheLock sync.Mutex
)
// NewClient returns a new REST client for specified server.
func NewClient(host string, version string) (*Client, error) {
baseURL, err := url.Parse(host)
if err != nil {
return nil, err
}
if baseURL.Path == "" {
baseURL.Path = "/"
}
unix2HTTP(baseURL)
c := &Client{
base: baseURL,
version: version,
httpClient: getHttpClient(host),
}
return c, nil
}
func GetUnixServerPath(socketName string, paths ...string) string {
serverPath := "unix://"
for _, path := range paths {
serverPath = serverPath + path
}
serverPath = serverPath + socketName + ".sock"
return serverPath
}
// Client is an HTTP REST wrapper. Use one of Get/Post/Put/Delete to get a request
// object.
type Client struct {
base *url.URL
version string
httpClient *http.Client
}
// Status sends a Status request at the /status REST endpoint.
func (c *Client) Status() (*Status, error) {
status := &Status{}
err := c.Get().UsePath("/status").Do().Unmarshal(status)
return status, err
}
// Version send a request at the /versions REST endpoint.
func (c *Client) Versions(endpoint string) ([]string, error) {
versions := []string{}
err := c.Get().Resource(endpoint + "/versions").Do().Unmarshal(&versions)
return versions, err
}
// Get returns a Request object setup for GET call.
func (c *Client) Get() *Request {
return NewRequest(c.httpClient, c.base, "GET", c.version)
}
// Post returns a Request object setup for POST call.
func (c *Client) Post() *Request {
return NewRequest(c.httpClient, c.base, "POST", c.version)
}
// Put returns a Request object setup for PUT call.
func (c *Client) Put() *Request {
return NewRequest(c.httpClient, c.base, "PUT", c.version)
}
// Put returns a Request object setup for DELETE call.
func (c *Client) Delete() *Request {
return NewRequest(c.httpClient, c.base, "DELETE", c.version)
}
func unix2HTTP(u *url.URL) {
if u.Scheme == "unix" {
// Override the main URL object so the HTTP lib won't complain
u.Scheme = "http"
u.Host = "unix.sock"
u.Path = ""
}
}
func newHTTPClient(u *url.URL, tlsConfig *tls.Config, timeout time.Duration) *http.Client {
httpTransport := &http.Transport{
TLSClientConfig: tlsConfig,
}
switch u.Scheme {
case "unix":
socketPath := u.Path
unixDial := func(proto, addr string) (net.Conn, error) {
ret, err := net.DialTimeout("unix", socketPath, timeout)
return ret, err
}
httpTransport.Dial = unixDial
unix2HTTP(u)
default:
httpTransport.Dial = func(proto, addr string) (net.Conn, error) {
return net.DialTimeout(proto, addr, timeout)
}
}
return &http.Client{Transport: httpTransport}
}
func getHttpClient(host string) *http.Client {
c, ok := httpCache[host]
if !ok {
cacheLock.Lock()
defer cacheLock.Unlock()
c, ok = httpCache[host]
if !ok {
u, err := url.Parse(host)
if err != nil {
// TODO(pedge): clean up
fmt.Println("Failed to parse into url", host)
return nil
}
if u.Path == "" {
u.Path = "/"
}
c = newHTTPClient(u, nil, 10*time.Second)
httpCache[host] = c
}
}
return c
}

View File

@@ -0,0 +1,304 @@
package client
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"path"
"strconv"
"strings"
"time"
)
// Request is contructed iteratively by the client and finally dispatched.
// A REST endpoint is accessed with the following convention:
// base_url/<version>/<resource>/[<instance>]
type Request struct {
client *http.Client
version string
verb string
path string
base *url.URL
params url.Values
headers http.Header
resource string
instance string
err error
body []byte
req *http.Request
resp *http.Response
timeout time.Duration
}
// Response is a representation of HTTP response received from the server.
type Response struct {
status string
statusCode int
err error
body []byte
}
// Status upon error, attempts to parse the body of a response into a meaningful status.
type Status struct {
Message string
ErrorCode int
}
// NewRequest instance
func NewRequest(client *http.Client, base *url.URL, verb string, version string) *Request {
return &Request{
client: client,
verb: verb,
base: base,
path: base.Path,
version: version,
}
}
func checkExists(mustExist string, before string) error {
if len(mustExist) == 0 {
return fmt.Errorf("%q should be set before setting %q", mustExist, before)
}
return nil
}
func checkSet(name string, s *string, newval string) error {
if len(*s) != 0 {
return fmt.Errorf("%q already set to %q, cannot change to %q",
name, *s, newval)
}
*s = newval
return nil
}
// Resource specifies the resource to be accessed.
func (r *Request) Resource(resource string) *Request {
if r.err == nil {
r.err = checkSet("resource", &r.resource, resource)
}
return r
}
// Instance specifies the instance of the resource to be accessed.
func (r *Request) Instance(instance string) *Request {
if r.err == nil {
r.err = checkExists("resource", "instance")
if r.err == nil {
r.err = checkSet("instance", &r.instance, instance)
}
}
return r
}
// UsePath use the specified path and don't build up a request.
func (r *Request) UsePath(path string) *Request {
if r.err == nil {
r.err = checkSet("path", &r.path, path)
}
return r
}
// QueryOption adds specified options to query.
func (r *Request) QueryOption(key string, value string) *Request {
if r.err != nil {
return r
}
if r.params == nil {
r.params = make(url.Values)
}
r.params.Add(string(key), value)
return r
}
// QueryOptionLabel adds specified label to query.
func (r *Request) QueryOptionLabel(key string, labels map[string]string) *Request {
if r.err != nil {
return r
}
if b, err := json.Marshal(labels); err != nil {
r.err = err
} else {
if r.params == nil {
r.params = make(url.Values)
}
r.params.Add(string(key), string(b))
}
return r
}
// SetHeader adds specified header values to query.
func (r *Request) SetHeader(key, value string) *Request {
if r.headers == nil {
r.headers = http.Header{}
}
r.headers.Set(key, value)
return r
}
// Timeout makes the request use the given duration as a timeout. Sets the "timeout"
// parameter.
func (r *Request) Timeout(d time.Duration) *Request {
if r.err != nil {
return r
}
r.timeout = d
return r
}
// Body sets the request Body.
func (r *Request) Body(v interface{}) *Request {
var err error
if r.err != nil {
return r
}
r.body, err = json.Marshal(v)
if err != nil {
r.err = err
return r
}
return r
}
// URL returns the current working URL.
func (r *Request) URL() *url.URL {
u := *r.base
p := r.path
if len(r.version) != 0 {
p = path.Join(p, strings.ToLower(r.version))
}
if len(r.resource) != 0 {
p = path.Join(p, strings.ToLower(r.resource))
if len(r.instance) != 0 {
p = path.Join(p, r.instance)
}
}
u.Path = p
query := url.Values{}
for key, values := range r.params {
for _, value := range values {
query.Add(key, value)
}
}
if r.timeout != 0 {
query.Set("timeout", r.timeout.String())
}
u.RawQuery = query.Encode()
return &u
}
// headerVal for key as an int. Return false if header is not present or valid.
func headerVal(key string, resp *http.Response) (int, bool) {
if h := resp.Header.Get(key); len(h) > 0 {
if i, err := strconv.Atoi(h); err == nil {
return i, true
}
}
return 0, false
}
func parseHTTPStatus(resp *http.Response, body []byte) error {
var (
status *Status
err error
)
httpOK := resp.StatusCode >= http.StatusOK && resp.StatusCode <= http.StatusPartialContent
hasStatus := false
if body != nil {
err = json.Unmarshal(body, status)
if err == nil && status.Message != "" {
hasStatus = true
}
}
// If the status is NG, return an error regardless of HTTP status.
if hasStatus && status.ErrorCode != 0 {
return fmt.Errorf("Error %v : %v", status.ErrorCode, status.Message)
}
// Status is good and HTTP status is good, everything is good
if httpOK {
return nil
}
// If HTTP status is NG, return an error.
return fmt.Errorf("HTTP error %d", resp.StatusCode)
}
// Do executes the request and returns a Response.
func (r *Request) Do() *Response {
var (
err error
req *http.Request
resp *http.Response
url string
body []byte
)
if r.err != nil {
return &Response{err: r.err}
}
url = r.URL().String()
req, err = http.NewRequest(r.verb, url, bytes.NewBuffer(r.body))
if err != nil {
return &Response{err: err}
}
if r.headers == nil {
r.headers = http.Header{}
}
req.Header = r.headers
req.Header.Set("Content-Type", "application/json")
resp, err = r.client.Do(req)
if err != nil {
return &Response{err: err}
}
if resp.Body != nil {
defer resp.Body.Close()
body, err = ioutil.ReadAll(resp.Body)
}
if err != nil {
return &Response{err: err}
}
return &Response{
status: resp.Status,
statusCode: resp.StatusCode,
body: body,
err: parseHTTPStatus(resp, body),
}
}
// Body return http body, valid only if there is no error
func (r Response) Body() ([]byte, error) {
return r.body, r.err
}
// StatusCode HTTP status code returned.
func (r Response) StatusCode() int {
return r.statusCode
}
// Unmarshal result into obj
func (r Response) Unmarshal(v interface{}) error {
if r.err != nil {
return r.err
}
return json.Unmarshal(r.body, v)
}
// Error executing the request.
func (r Response) Error() error {
return r.err
}
func (r Response) FormatError() error {
if len(r.body) == 0 {
return fmt.Errorf("Error: %v", r.err)
} else {
return fmt.Errorf("HTTP-%d: %s", r.statusCode, string(r.body))
}
}

View File

@@ -0,0 +1,387 @@
package volume
import (
"bytes"
"errors"
"fmt"
"io"
"io/ioutil"
"strconv"
"github.com/libopenstorage/openstorage/api"
"github.com/libopenstorage/openstorage/api/client"
"github.com/libopenstorage/openstorage/volume"
)
const (
graphPath = "/graph"
volumePath = "/osd-volumes"
snapPath = "/osd-snapshot"
)
type volumeClient struct {
volume.IODriver
c *client.Client
}
func newVolumeClient(c *client.Client) volume.VolumeDriver {
return &volumeClient{volume.IONotSupported, c}
}
// String description of this driver.
func (v *volumeClient) Name() string {
return "VolumeDriver"
}
func (v *volumeClient) Type() api.DriverType {
// Block drivers implement the superset.
return api.DriverType_DRIVER_TYPE_BLOCK
}
func (v *volumeClient) GraphDriverCreate(id string, parent string) error {
response := ""
if err := v.c.Put().Resource(graphPath + "/create").Instance(id).Do().Unmarshal(&response); err != nil {
return err
}
if response != id {
return fmt.Errorf("Invalid response: %s", response)
}
return nil
}
func (v *volumeClient) GraphDriverRemove(id string) error {
response := ""
if err := v.c.Put().Resource(graphPath + "/remove").Instance(id).Do().Unmarshal(&response); err != nil {
return err
}
if response != id {
return fmt.Errorf("Invalid response: %s", response)
}
return nil
}
func (v *volumeClient) GraphDriverGet(id string, mountLabel string) (string, error) {
response := ""
if err := v.c.Get().Resource(graphPath + "/inspect").Instance(id).Do().Unmarshal(&response); err != nil {
return "", err
}
return response, nil
}
func (v *volumeClient) GraphDriverRelease(id string) error {
response := ""
if err := v.c.Put().Resource(graphPath + "/release").Instance(id).Do().Unmarshal(&response); err != nil {
return err
}
if response != id {
return fmt.Errorf("Invalid response: %v", response)
}
return nil
}
func (v *volumeClient) GraphDriverExists(id string) bool {
response := false
v.c.Get().Resource(graphPath + "/exists").Instance(id).Do().Unmarshal(&response)
return response
}
func (v *volumeClient) GraphDriverDiff(id string, parent string) io.Writer {
body, _ := v.c.Get().Resource(graphPath + "/diff?id=" + id + "&parent=" + parent).Do().Body()
return bytes.NewBuffer(body)
}
func (v *volumeClient) GraphDriverChanges(id string, parent string) ([]api.GraphDriverChanges, error) {
var changes []api.GraphDriverChanges
err := v.c.Get().Resource(graphPath + "/changes").Instance(id).Do().Unmarshal(&changes)
return changes, err
}
func (v *volumeClient) GraphDriverApplyDiff(id string, parent string, diff io.Reader) (int, error) {
b, err := ioutil.ReadAll(diff)
if err != nil {
return 0, err
}
response := 0
if err = v.c.Put().Resource(graphPath + "/diff?id=" + id + "&parent=" + parent).Instance(id).Body(b).Do().Unmarshal(&response); err != nil {
return 0, err
}
return response, nil
}
func (v *volumeClient) GraphDriverDiffSize(id string, parent string) (int, error) {
size := 0
err := v.c.Get().Resource(graphPath + "/diffsize").Instance(id).Do().Unmarshal(&size)
return size, err
}
// Create a new Vol for the specific volume spev.c.
// It returns a system generated VolumeID that uniquely identifies the volume
func (v *volumeClient) Create(locator *api.VolumeLocator, source *api.Source,
spec *api.VolumeSpec) (string, error) {
response := &api.VolumeCreateResponse{}
request := &api.VolumeCreateRequest{
Locator: locator,
Source: source,
Spec: spec,
}
if err := v.c.Post().Resource(volumePath).Body(request).Do().Unmarshal(response); err != nil {
return "", err
}
if response.VolumeResponse != nil && response.VolumeResponse.Error != "" {
return "", errors.New(response.VolumeResponse.Error)
}
return response.Id, nil
}
// Status diagnostic information
func (v *volumeClient) Status() [][2]string {
return [][2]string{}
}
// Inspect specified volumes.
// Errors ErrEnoEnt may be returned.
func (v *volumeClient) Inspect(ids []string) ([]*api.Volume, error) {
if len(ids) == 0 {
return nil, nil
}
var volumes []*api.Volume
request := v.c.Get().Resource(volumePath)
for _, id := range ids {
request.QueryOption(api.OptVolumeID, id)
}
if err := request.Do().Unmarshal(&volumes); err != nil {
return nil, err
}
return volumes, nil
}
// Delete volume.
// Errors ErrEnoEnt, ErrVolHasSnaps may be returned.
func (v *volumeClient) Delete(volumeID string) error {
response := &api.VolumeResponse{}
if err := v.c.Delete().Resource(volumePath).Instance(volumeID).Do().Unmarshal(response); err != nil {
return err
}
if response.Error != "" {
return errors.New(response.Error)
}
return nil
}
// Snap specified volume. IO to the underlying volume should be quiesced before
// calling this function.
// Errors ErrEnoEnt may be returned
func (v *volumeClient) Snapshot(volumeID string, readonly bool,
locator *api.VolumeLocator) (string, error) {
response := &api.SnapCreateResponse{}
request := &api.SnapCreateRequest{
Id: volumeID,
Readonly: readonly,
Locator: locator,
}
if err := v.c.Post().Resource(snapPath).Body(request).Do().Unmarshal(response); err != nil {
return "", err
}
// TODO(pedge): this probably should not be embedded in this way
if response.VolumeCreateResponse != nil &&
response.VolumeCreateResponse.VolumeResponse != nil &&
response.VolumeCreateResponse.VolumeResponse.Error != "" {
return "", errors.New(
response.VolumeCreateResponse.VolumeResponse.Error)
}
if response.VolumeCreateResponse != nil {
return response.VolumeCreateResponse.Id, nil
}
return "", nil
}
// Stats for specified volume.
// Errors ErrEnoEnt may be returned
func (v *volumeClient) Stats(
volumeID string,
cumulative bool,
) (*api.Stats, error) {
stats := &api.Stats{}
req := v.c.Get().Resource(volumePath + "/stats").Instance(volumeID)
req.QueryOption(api.OptCumulative, strconv.FormatBool(cumulative))
if err := req.Do().Unmarshal(stats); err != nil {
return nil, err
}
return stats, nil
}
// Alerts on this volume.
// Errors ErrEnoEnt may be returned
func (v *volumeClient) Alerts(volumeID string) (*api.Alerts, error) {
alerts := &api.Alerts{}
if err := v.c.Get().Resource(volumePath + "/alerts").Instance(volumeID).Do().Unmarshal(alerts); err != nil {
return nil, err
}
return alerts, nil
}
// Active Requests on all volume.
func (v *volumeClient) GetActiveRequests() (*api.ActiveRequests, error) {
requests := &api.ActiveRequests{}
resp := v.c.Get().Resource(volumePath + "/requests").Instance("vol_id").Do()
if resp.Error() != nil {
return nil, resp.FormatError()
}
if err := resp.Unmarshal(requests); err != nil {
return nil, err
}
return requests, nil
}
// Shutdown and cleanup.
func (v *volumeClient) Shutdown() {}
// Enumerate volumes that map to the volumeLocator. Locator fields may be regexp.
// If locator fields are left blank, this will return all volumes.
func (v *volumeClient) Enumerate(locator *api.VolumeLocator,
labels map[string]string) ([]*api.Volume, error) {
var volumes []*api.Volume
req := v.c.Get().Resource(volumePath)
if locator.Name != "" {
req.QueryOption(api.OptName, locator.Name)
}
if len(locator.VolumeLabels) != 0 {
req.QueryOptionLabel(api.OptLabel, locator.VolumeLabels)
}
if len(labels) != 0 {
req.QueryOptionLabel(api.OptConfigLabel, labels)
}
resp := req.Do()
if resp.Error() != nil {
return nil, resp.FormatError()
}
if err := resp.Unmarshal(&volumes); err != nil {
return nil, err
}
return volumes, nil
}
// Enumerate snaps for specified volume
// Count indicates the number of snaps populated.
func (v *volumeClient) SnapEnumerate(ids []string,
snapLabels map[string]string) ([]*api.Volume, error) {
var volumes []*api.Volume
request := v.c.Get().Resource(snapPath)
for _, id := range ids {
request.QueryOption(api.OptVolumeID, id)
}
if len(snapLabels) != 0 {
request.QueryOptionLabel(api.OptLabel, snapLabels)
}
if err := request.Do().Unmarshal(&volumes); err != nil {
return nil, err
}
return volumes, nil
}
// Attach map device to the host.
// On success the devicePath specifies location where the device is exported
// Errors ErrEnoEnt, ErrVolAttached may be returned.
func (v *volumeClient) Attach(volumeID string) (string, error) {
response, err := v.doVolumeSetGetResponse(
volumeID,
&api.VolumeSetRequest{
Action: &api.VolumeStateAction{
Attach: api.VolumeActionParam_VOLUME_ACTION_PARAM_ON,
},
},
)
if err != nil {
return "", err
}
if response.Volume != nil {
if response.Volume.Spec.Encrypted {
return response.Volume.SecureDevicePath, nil
} else {
return response.Volume.DevicePath, nil
}
}
return "", nil
}
// Detach device from the host.
// Errors ErrEnoEnt, ErrVolDetached may be returned.
func (v *volumeClient) Detach(volumeID string) error {
return v.doVolumeSet(
volumeID,
&api.VolumeSetRequest{
Action: &api.VolumeStateAction{
Attach: api.VolumeActionParam_VOLUME_ACTION_PARAM_OFF,
},
},
)
}
func (v *volumeClient) MountedAt(mountPath string) string {
return ""
}
// Mount volume at specified path
// Errors ErrEnoEnt, ErrVolDetached may be returned.
func (v *volumeClient) Mount(volumeID string, mountPath string) error {
return v.doVolumeSet(
volumeID,
&api.VolumeSetRequest{
Action: &api.VolumeStateAction{
Mount: api.VolumeActionParam_VOLUME_ACTION_PARAM_ON,
MountPath: mountPath,
},
},
)
}
// Unmount volume at specified path
// Errors ErrEnoEnt, ErrVolDetached may be returned.
func (v *volumeClient) Unmount(volumeID string, mountPath string) error {
return v.doVolumeSet(
volumeID,
&api.VolumeSetRequest{
Action: &api.VolumeStateAction{
Mount: api.VolumeActionParam_VOLUME_ACTION_PARAM_OFF,
MountPath: mountPath,
},
},
)
}
// Update volume
func (v *volumeClient) Set(volumeID string, locator *api.VolumeLocator,
spec *api.VolumeSpec) error {
return v.doVolumeSet(
volumeID,
&api.VolumeSetRequest{
Locator: locator,
Spec: spec,
},
)
}
func (v *volumeClient) doVolumeSet(volumeID string,
request *api.VolumeSetRequest) error {
_, err := v.doVolumeSetGetResponse(volumeID, request)
return err
}
func (v *volumeClient) doVolumeSetGetResponse(volumeID string,
request *api.VolumeSetRequest) (*api.VolumeSetResponse, error) {
response := &api.VolumeSetResponse{}
if err := v.c.Put().Resource(volumePath).Instance(volumeID).Body(request).Do().Unmarshal(response); err != nil {
return nil, err
}
if response.VolumeResponse != nil && response.VolumeResponse.Error != "" {
return nil, errors.New(response.VolumeResponse.Error)
}
return response, nil
}

View File

@@ -0,0 +1,50 @@
package volume
import (
"fmt"
"github.com/libopenstorage/openstorage/api/client"
"github.com/libopenstorage/openstorage/volume"
"github.com/libopenstorage/openstorage/api"
)
// VolumeDriver returns a REST wrapper for the VolumeDriver interface.
func VolumeDriver(c *client.Client) volume.VolumeDriver {
return newVolumeClient(c)
}
// NewDriver returns a new REST client of the supplied version for specified driver.
// host: REST endpoint [http://<ip>:<port> OR unix://<path-to-unix-socket>]. default: [unix:///var/lib/osd/<driverName>.sock]
// version: Volume API version
func NewDriverClient(host, driverName, version string) (*client.Client, error) {
if driverName == "" {
return nil, fmt.Errorf("Driver Name cannot be empty")
}
if host == "" {
host = client.GetUnixServerPath(driverName, volume.DriverAPIBase)
}
if version == "" {
// Set the default version
version = volume.APIVersion
}
return client.NewClient(host, version)
}
// GetSupportedDriverVersions returns a list of supported versions
// for the provided driver. It uses the given server endpoint or the
// standard unix domain socket
func GetSupportedDriverVersions(driverName, host string) ([]string, error) {
// Get a client handler
if host == "" {
host = client.GetUnixServerPath(driverName, volume.DriverAPIBase)
}
client, err := client.NewClient(host, "")
if err != nil {
return []string{}, err
}
versions, err := client.Versions(api.OsdVolumePath)
if err != nil {
return []string{}, err
}
return versions, nil
}

View File

@@ -0,0 +1,192 @@
package spec
import (
"fmt"
"regexp"
"strconv"
"github.com/libopenstorage/openstorage/api"
"github.com/libopenstorage/openstorage/pkg/units"
)
// SpecHandler provides conversion function from what gets passed in over the
// plugin API to an api.VolumeSpec object.
type SpecHandler interface {
// SpecFromString parses options from the name.
// If the scheduler was unable to pass in the volume spec via the API,
// the spec can be passed in via the name in the format:
// "key=value;key=value;name=volname"
// If the spec was parsed, it returns:
// (true, parsed_spec, parsed_name)
// If the input string didn't contain the string, it returns:
// (false, DefaultSpec(), inputString)
SpecFromString(inputString string) (bool, *api.VolumeSpec, string)
// SpecFromOpts parses in docker options passed in the the docker run
// command of the form --opt name=value
// If the options are validated then it returns:
// (resultant_VolumeSpec, nil)
// If the options have invalid values then it returns:
// (nil, error)
SpecFromOpts(opts map[string]string) (*api.VolumeSpec, error)
// Returns a default VolumeSpec if no docker options or string encoding
// was provided.
DefaultSpec() *api.VolumeSpec
}
var (
nameRegex = regexp.MustCompile(api.Name + "=([0-9A-Za-z]+),?")
sizeRegex = regexp.MustCompile(api.SpecSize + "=([0-9A-Za-z]+),?")
scaleRegex = regexp.MustCompile(api.SpecScale + "=([0-9A-Za-z]+),?")
fsRegex = regexp.MustCompile(api.SpecFilesystem + "=([0-9A-Za-z]+),?")
bsRegex = regexp.MustCompile(api.SpecBlockSize + "=([0-9]+),?")
haRegex = regexp.MustCompile(api.SpecHaLevel + "=([0-9]+),?")
cosRegex = regexp.MustCompile(api.SpecPriority + "=([A-Za-z]+),?")
sharedRegex = regexp.MustCompile(api.SpecShared + "=([A-Za-z]+),?")
passphraseRegex = regexp.MustCompile(api.SpecPassphrase + "=([0-9A-Za-z_@./#&+-]+),?")
)
type specHandler struct {
}
func NewSpecHandler() SpecHandler {
return &specHandler{}
}
func (d *specHandler) cosLevel(cos string) (uint32, error) {
switch cos {
case "high", "3":
return uint32(api.CosType_HIGH), nil
case "medium", "2":
return uint32(api.CosType_MEDIUM), nil
case "low", "1", "":
return uint32(api.CosType_LOW), nil
}
return uint32(api.CosType_LOW),
fmt.Errorf("Cos must be one of %q | %q | %q", "high", "medium", "low")
}
func (d *specHandler) getVal(r *regexp.Regexp, str string) (bool, string) {
found := r.FindString(str)
if found == "" {
return false, ""
}
submatches := r.FindStringSubmatch(str)
if len(submatches) < 2 {
return false, ""
}
val := submatches[1]
return true, val
}
func (d *specHandler) DefaultSpec() *api.VolumeSpec {
return &api.VolumeSpec{
VolumeLabels: make(map[string]string),
Format: api.FSType_FS_TYPE_EXT4,
HaLevel: 1,
}
}
func (d *specHandler) SpecFromOpts(
opts map[string]string,
) (*api.VolumeSpec, error) {
spec := d.DefaultSpec()
for k, v := range opts {
switch k {
case api.SpecEphemeral:
spec.Ephemeral, _ = strconv.ParseBool(v)
case api.SpecSize:
if size, err := units.Parse(v); err != nil {
return nil, err
} else {
spec.Size = uint64(size)
}
case api.SpecFilesystem:
if value, err := api.FSTypeSimpleValueOf(v); err != nil {
return nil, err
} else {
spec.Format = value
}
case api.SpecBlockSize:
if blockSize, err := units.Parse(v); err != nil {
return nil, err
} else {
spec.BlockSize = blockSize
}
case api.SpecHaLevel:
haLevel, _ := strconv.ParseInt(v, 10, 64)
spec.HaLevel = haLevel
case api.SpecPriority:
cos, _ := api.CosTypeSimpleValueOf(v)
spec.Cos = cos
case api.SpecDedupe:
spec.Dedupe, _ = strconv.ParseBool(v)
case api.SpecSnapshotInterval:
snapshotInterval, _ := strconv.ParseUint(v, 10, 32)
spec.SnapshotInterval = uint32(snapshotInterval)
case api.SpecAggregationLevel:
aggregationLevel, _ := strconv.ParseUint(v, 10, 32)
spec.AggregationLevel = uint32(aggregationLevel)
case api.SpecShared:
if shared, err := strconv.ParseBool(v); err != nil {
return nil, err
} else {
spec.Shared = shared
}
case api.SpecPassphrase:
spec.Encrypted = true
spec.Passphrase = v
default:
spec.VolumeLabels[k] = v
}
}
return spec, nil
}
func (d *specHandler) SpecFromString(
str string,
) (bool, *api.VolumeSpec, string) {
// If we can't parse the name, the rest of the spec is invalid.
ok, name := d.getVal(nameRegex, str)
if !ok {
return false, d.DefaultSpec(), str
}
opts := make(map[string]string)
if ok, sz := d.getVal(sizeRegex, str); ok {
opts[api.SpecSize] = sz
}
if ok, scale := d.getVal(scaleRegex, str); ok {
opts[api.SpecScale] = scale
}
if ok, fs := d.getVal(fsRegex, str); ok {
opts[api.SpecFilesystem] = fs
}
if ok, bs := d.getVal(bsRegex, str); ok {
opts[api.SpecBlockSize] = bs
}
if ok, ha := d.getVal(haRegex, str); ok {
opts[api.SpecHaLevel] = ha
}
if ok, priority := d.getVal(cosRegex, str); ok {
opts[api.SpecPriority] = priority
}
if ok, shared := d.getVal(sharedRegex, str); ok {
opts[api.SpecShared] = shared
}
if ok, passphrase := d.getVal(passphraseRegex, str); ok {
opts[api.SpecPassphrase] = passphrase
}
spec, err := d.SpecFromOpts(opts)
if err != nil {
return false, d.DefaultSpec(), name
}
return true, spec, name
}

View File

@@ -0,0 +1,49 @@
package api
type StatusKind int32
const (
// StatusSeverityLow indicates an OK status
StatusSeverityLow StatusKind = iota
// StatusSeverityMedium indicates a status which is in transition from OK to BAD or vice versa
StatusSeverityMedium
// StatusSeverityHigh indicates a BAD status
StatusSeverityHigh
)
var statusToStatusKind = map[Status]StatusKind{
Status_STATUS_NONE: StatusSeverityHigh,
Status_STATUS_INIT: StatusSeverityMedium,
Status_STATUS_OK: StatusSeverityLow,
Status_STATUS_OFFLINE: StatusSeverityHigh,
Status_STATUS_ERROR: StatusSeverityHigh,
Status_STATUS_NOT_IN_QUORUM: StatusSeverityHigh,
Status_STATUS_DECOMMISSION: StatusSeverityHigh,
Status_STATUS_MAINTENANCE: StatusSeverityHigh,
Status_STATUS_STORAGE_DOWN: StatusSeverityHigh,
Status_STATUS_STORAGE_DEGRADED: StatusSeverityHigh,
Status_STATUS_NEEDS_REBOOT: StatusSeverityHigh,
Status_STATUS_STORAGE_REBALANCE: StatusSeverityMedium,
Status_STATUS_STORAGE_DRIVE_REPLACE: StatusSeverityMedium,
// Add statuses before MAX
Status_STATUS_MAX: StatusSeverityHigh,
}
func StatusSimpleValueOf(s string) (Status, error) {
obj, err := simpleValueOf("status", Status_value, s)
return Status(obj), err
}
func (x Status) SimpleString() string {
return simpleString("status", Status_name, int32(x))
}
func (x Status) StatusKind() StatusKind {
statusType, _ := statusToStatusKind[x]
return statusType
}
// StatusKindMapLength used only for unit testing
func StatusKindMapLength() int {
return len(statusToStatusKind)
}

View File

@@ -0,0 +1,139 @@
package units
import (
"errors"
"fmt"
"regexp"
"strconv"
"strings"
)
const (
_ = iota
// KiB 1024 bytes
KiB = 1 << (10 * iota)
// MiB 1024 KiB
MiB
// GiB 1024 MiB
GiB
// TiB 1024 GiB
TiB
// PiB 1024 TiB
PiB
)
const (
// KB 1000 bytes
KB = 1000
// MB 1000 KB
MB = KB * 1000
// GB 1000 MB
GB = MB * 1000
// TB 1000 GB
TB = GB * 1000
// PB 1000 TB
PB = TB * 1000
)
var (
unitMap = map[string]int64{
"B": 1,
"b": 1,
"KB": KB,
"kb": KB,
"MB": MB,
"mb": MB,
"GB": GB,
"gb": GB,
"TB": TB,
"tb": TB,
"PB": PB,
"pb": PB,
"K": KiB,
"k": KiB,
"M": MiB,
"m": MiB,
"G": GiB,
"g": GiB,
"T": TiB,
"t": TiB,
"P": PiB,
"p": PiB,
"KiB": KiB,
"MiB": MiB,
"GiB": GiB,
"TiB": TiB,
"PiB": PiB,
"Mi": MiB,
"Gi": GiB,
"Ti": TiB,
"Pi": PiB,
}
)
var unitPattern = regexp.MustCompile(
"([0-9]+)(.[0-9]+)*\\s*(B|b|K|k|M|m|G|g|T|t|P|p|KB|kb|KiB|MB|mb|MiB|Mi|GB|gb|GiB|Gi|TB|tb|TiB|Ti|PB|pb|PiB|Pi|)")
var BadUnit = errors.New("Bad unit")
func String(b uint64) string {
if b > PiB {
return fmt.Sprintf("%.2f PiB", float64(b)/float64(PiB))
}
if b > TiB {
return fmt.Sprintf("%.2f TiB", float64(b)/float64(TiB))
}
if b > GiB {
return fmt.Sprintf("%.1f TiB", float64(b)/float64(GiB))
}
if b > MiB {
return fmt.Sprintf("%v MiB", b/MiB)
}
if b > KiB {
return fmt.Sprintf("%v KiB", b/KiB)
}
return fmt.Sprintf("%v bytes", b)
}
func Parse(bUnit string) (int64, error) {
ustring := strings.TrimSpace(bUnit)
unitPattern.Longest()
if !unitPattern.MatchString(ustring) {
return -1, fmt.Errorf("Unit parse error: %s", bUnit)
}
matches := unitPattern.FindStringSubmatch(ustring)
if len(matches) == 0 || len(matches) > 4 {
return -1, fmt.Errorf(
"Unit parse error: invalid count of fields (%v)",
len(matches))
}
if len(matches) == 1 {
return strconv.ParseInt(ustring, 10, 64)
}
shift := 0
if len(matches) == 4 {
shift = 1
}
if len(matches) == 2 {
return -1, fmt.Errorf("Unit parse error: invalid fields %v",
matches)
}
if ustring != matches[0] {
return -1, fmt.Errorf("Unit parse error: invalid fields %v",
matches)
}
multiplier, ok := unitMap[matches[2+shift]]
if !ok {
multiplier = unitMap["G"]
}
base, err := strconv.ParseInt(matches[1], 10, 64)
if err != nil {
return -1, fmt.Errorf("Invalid number")
}
return base * multiplier, nil
}

View File

@@ -0,0 +1,41 @@
## Volume Drivers
Volume drivers implement the [Volume Plugin Interface](https://docs.docker.com/engine/extend/plugins_volume/).
This provides an interface to register a volume driver and advertise the driver to Docker. Registering a driver with this volume interface will cause Docker to be able to communicate with the driver to create and assign volumes to a container.
A volume spec is needed to create a volume. A volume spec looks like:
```
// VolumeSpec has the properties needed to create a volume.
type VolumeSpec struct {
// Ephemeral storage
Ephemeral bool
// Thin provisioned volume size in bytes
Size uint64
// Format disk with this FileSystem
Format Filesystem
// BlockSize for file system
BlockSize int
// HA Level specifies the number of nodes that are
// allowed to fail, and yet data is availabel.
// A value of 0 implies that data is not erasure coded,
// a failure of a node will lead to data loss.
HALevel int
// This disk's CoS
Cos VolumeCos
// Perform dedupe on this disk
Dedupe bool
// SnapshotInterval in minutes, set to 0 to disable Snapshots
SnapshotInterval int
// Volume configuration labels
ConfigLabels Labels
}
```
Various volume driver implementations can be found in the `drivers` directory.
### Block Drivers
Block drivers operate at the block layer. They provide raw volumes formatted with a user specified filesystem. This volume is then mounted into the container at a path specified using the `docker run -v` option.
### File Drivers
File drivers operate at the filesystem layer.

View File

@@ -0,0 +1,171 @@
package volume
import (
"errors"
"github.com/libopenstorage/openstorage/api"
)
var (
ErrAlreadyShutdown = errors.New("VolumeDriverProvider already shutdown")
ErrExist = errors.New("Driver already exists")
ErrDriverNotFound = errors.New("Driver implementation not found")
ErrDriverInitializing = errors.New("Driver is initializing")
ErrEnoEnt = errors.New("Volume does not exist.")
ErrEnomem = errors.New("Out of memory.")
ErrEinval = errors.New("Invalid argument")
ErrVolDetached = errors.New("Volume is detached")
ErrVolAttached = errors.New("Volume is attached")
ErrVolAttachedOnRemoteNode = errors.New("Volume is attached on another node")
ErrVolAttachedScale = errors.New("Volume is attached but can be scaled")
ErrVolHasSnaps = errors.New("Volume has snapshots associated")
ErrNotSupported = errors.New("Operation not supported")
)
// Constants used by the VolumeDriver
const (
APIVersion = "v1"
PluginAPIBase = "/run/docker/plugins/"
DriverAPIBase = "/var/lib/osd/driver/"
MountBase = "/var/lib/osd/mounts/"
VolumeBase = "/var/lib/osd/"
)
type Store interface {
// Lock volume specified by volumeID.
Lock(volumeID string) (interface{}, error)
// Lock volume with token obtained from call to Lock.
Unlock(token interface{}) error
// CreateVol returns error if volume with the same ID already existe.
CreateVol(vol *api.Volume) error
// GetVol from volumeID.
GetVol(volumeID string) (*api.Volume, error)
// UpdateVol with vol
UpdateVol(vol *api.Volume) error
// DeleteVol. Returns error if volume does not exist.
DeleteVol(volumeID string) error
}
// VolumeDriver is the main interface to be implemented by any storage driver.
// Every driver must at minimum implement the ProtoDriver sub interface.
type VolumeDriver interface {
IODriver
ProtoDriver
BlockDriver
Enumerator
}
// IODriver interfaces applicable to object store interfaces.
type IODriver interface {
// Read sz bytes from specified volume at specified offset.
// Return number of bytes read and error.
Read(volumeID string, buf []byte, sz uint64, offset int64) (int64, error)
// Write sz bytes from specified volume at specified offset.
// Return number of bytes written and error.
Write(volumeID string, buf []byte, sz uint64, offset int64) (int64, error)
// Flush writes to stable storage.
// Return error.
Flush(volumeID string) error
}
type SnapshotDriver interface {
// Snapshot create volume snapshot.
// Errors ErrEnoEnt may be returned
Snapshot(volumeID string, readonly bool, locator *api.VolumeLocator) (string, error)
}
// ProtoDriver must be implemented by all volume drivers. It specifies the
// most basic functionality, such as creating and deleting volumes.
type ProtoDriver interface {
SnapshotDriver
// Name returns the name of the driver.
Name() string
// Type of this driver
Type() api.DriverType
// Create a new Vol for the specific volume spec.
// It returns a system generated VolumeID that uniquely identifies the volume
Create(locator *api.VolumeLocator, Source *api.Source, spec *api.VolumeSpec) (string, error)
// Delete volume.
// Errors ErrEnoEnt, ErrVolHasSnaps may be returned.
Delete(volumeID string) error
// Mount volume at specified path
// Errors ErrEnoEnt, ErrVolDetached may be returned.
Mount(volumeID string, mountPath string) error
// MountedAt return volume mounted at specified mountpath.
MountedAt(mountPath string) string
// Unmount volume at specified path
// Errors ErrEnoEnt, ErrVolDetached may be returned.
Unmount(volumeID string, mountPath string) error
// Update not all fields of the spec are supported, ErrNotSupported will be thrown for unsupported
// updates.
Set(volumeID string, locator *api.VolumeLocator, spec *api.VolumeSpec) error
// Stats for specified volume.
// cumulative stats are /proc/diskstats style stats.
// nonCumulative stats are stats for specific duration.
// Errors ErrEnoEnt may be returned
Stats(volumeID string, cumulative bool) (*api.Stats, error)
// Alerts on this volume.
// Errors ErrEnoEnt may be returned
Alerts(volumeID string) (*api.Alerts, error)
// GetActiveRequests get active requests
GetActiveRequests() (*api.ActiveRequests, error)
// Status returns a set of key-value pairs which give low
// level diagnostic status about this driver.
Status() [][2]string
// Shutdown and cleanup.
Shutdown()
}
// Enumerator provides a set of interfaces to get details on a set of volumes.
type Enumerator interface {
// Inspect specified volumes.
// Returns slice of volumes that were found.
Inspect(volumeIDs []string) ([]*api.Volume, error)
// Enumerate volumes that map to the volumeLocator. Locator fields may be regexp.
// If locator fields are left blank, this will return all volumes.
Enumerate(locator *api.VolumeLocator, labels map[string]string) ([]*api.Volume, error)
// Enumerate snaps for specified volumes
SnapEnumerate(volID []string, snapLabels map[string]string) ([]*api.Volume, error)
}
type StoreEnumerator interface {
Store
Enumerator
}
// BlockDriver needs to be implemented by block volume drivers. Filesystem volume
// drivers can ignore this interface and include the builtin DefaultBlockDriver.
type BlockDriver interface {
// Attach map device to the host.
// On success the devicePath specifies location where the device is exported
// Errors ErrEnoEnt, ErrVolAttached may be returned.
Attach(volumeID string) (string, error)
// Detach device from the host.
// Errors ErrEnoEnt, ErrVolDetached may be returned.
Detach(volumeID string) error
}
// VolumeDriverProvider provides VolumeDrivers.
type VolumeDriverProvider interface {
// Get gets the VolumeDriver for the given name.
// If a VolumeDriver was not created for the given name, the error ErrDriverNotFound is returned.
Get(name string) (VolumeDriver, error)
// Shutdown shuts down all volume drivers.
Shutdown() error
}
// VolumeDriverRegistry registers VolumeDrivers.
type VolumeDriverRegistry interface {
VolumeDriverProvider
// New creates the VolumeDriver for the given name.
// If a VolumeDriver was already created for the given name, the error ErrExist is returned.
Register(name string, params map[string]string) error
// Add inserts a new VolumeDriver provider with a well known name.
Add(name string, init func(map[string]string) (VolumeDriver, error)) error
}
// VolumeDriverRegistry constructs a new VolumeDriverRegistry.
func NewVolumeDriverRegistry(nameToInitFunc map[string]func(map[string]string) (VolumeDriver, error)) VolumeDriverRegistry {
return newVolumeDriverRegistry(nameToInitFunc)
}

View File

@@ -0,0 +1,71 @@
package volume
import "sync"
type volumeDriverRegistry struct {
nameToInitFunc map[string]func(map[string]string) (VolumeDriver, error)
nameToVolumeDriver map[string]VolumeDriver
lock *sync.RWMutex
isShutdown bool
}
func newVolumeDriverRegistry(nameToInitFunc map[string]func(map[string]string) (VolumeDriver, error)) *volumeDriverRegistry {
return &volumeDriverRegistry{
nameToInitFunc,
make(map[string]VolumeDriver),
&sync.RWMutex{},
false,
}
}
func (v *volumeDriverRegistry) Get(name string) (VolumeDriver, error) {
v.lock.RLock()
defer v.lock.RUnlock()
if v.isShutdown {
return nil, ErrAlreadyShutdown
}
volumeDriver, ok := v.nameToVolumeDriver[name]
if !ok {
return nil, ErrDriverNotFound
}
return volumeDriver, nil
}
func (v *volumeDriverRegistry) Add(name string, init func(map[string]string) (VolumeDriver, error)) error {
v.nameToInitFunc[name] = init
return nil
}
func (v *volumeDriverRegistry) Register(name string, params map[string]string) error {
initFunc, ok := v.nameToInitFunc[name]
if !ok {
return ErrNotSupported
}
v.lock.Lock()
defer v.lock.Unlock()
if v.isShutdown {
return ErrAlreadyShutdown
}
if _, ok := v.nameToVolumeDriver[name]; ok {
return ErrExist
}
volumeDriver, err := initFunc(params)
if err != nil {
return err
}
v.nameToVolumeDriver[name] = volumeDriver
return nil
}
func (v *volumeDriverRegistry) Shutdown() error {
v.lock.Lock()
if v.isShutdown {
return ErrAlreadyShutdown
}
for _, volumeDriver := range v.nameToVolumeDriver {
volumeDriver.Shutdown()
}
v.isShutdown = true
return nil
}

View File

@@ -0,0 +1,45 @@
package volume
import (
"github.com/libopenstorage/openstorage/api"
)
var (
// BlockNotSupported is a default (null) block driver implementation. This can be
// used by drivers that do not want to (or care about) implementing the attach,
// format and detach interfaces.
BlockNotSupported = &blockNotSupported{}
SnapshotNotSupported = &snapshotNotSupported{}
IONotSupported = &ioNotSupported{}
)
type blockNotSupported struct{}
func (b *blockNotSupported) Attach(volumeID string) (string, error) {
return "", ErrNotSupported
}
func (b *blockNotSupported) Detach(volumeID string) error {
return ErrNotSupported
}
type snapshotNotSupported struct{}
func (s *snapshotNotSupported) Snapshot(volumeID string, readonly bool, locator *api.VolumeLocator) (string, error) {
return "", ErrNotSupported
}
type ioNotSupported struct{}
func (i *ioNotSupported) Read(volumeID string, buffer []byte, size uint64, offset int64) (int64, error) {
return 0, ErrNotSupported
}
func (i *ioNotSupported) Write(volumeID string, buffer []byte, size uint64, offset int64) (int64, error) {
return 0, ErrNotSupported
}
func (i *ioNotSupported) Flush(volumeID string) error {
return ErrNotSupported
}