Commit Graph

190 Commits

Author SHA1 Message Date
Michal Mielewczyk
f226f978f0 Unify inactive cores stats.
Inactive core stats should be caluculated and returned to adapter in unified
from, just like all stats are.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-16 02:49:42 -04:00
Jan Musial
703a757db1 Fix minor bugs in promotion policy
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-09-11 10:31:14 +02:00
Jan Musial
633f31716e Make NHIT API naming convention similar to cleaning
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-09-11 07:31:50 +02:00
Michal Mielewczyk
0d4f4d3294 Added API for updating statistics.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-10 08:01:09 -04:00
Michal Mielewczyk
42d6dbbf11 Remove low-level stats getters from public API.
Since stats builder is implemented for retrieving cache, core and ioclass stats,
adapters should use it instead.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-10 08:01:09 -04:00
Michal Mielewczyk
89de6038c1 Split retrieving core stats and config info into two functions.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-10 08:01:09 -04:00
Michal Mielewczyk
f9da89263b Stats builder for ioclass statistics.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-10 04:32:21 -04:00
Michal Mielewczyk
8304ed84b8 More precise percentage value in stats builder.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-09 02:46:45 -04:00
Jan Musial
a1d3cf0c4d Switching mechanism for promotion policies
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-08-29 13:24:26 +02:00
Michal Rakowski
e3a20c0da0
Merge pull request #239 from imjfckm/pp-implementation
nhit promotion policy implementation
2019-08-27 12:42:03 +02:00
Jan Musial
62a0ccdd9f nhit promotion policy implementation
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-08-22 13:34:38 +02:00
Michal Mielewczyk
2c27e9a433 Store name for newly initialized cache in array instead of passing pointer.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-08-20 01:43:11 -04:00
Robert Baldyga
9730b2bd1f Remove remaining core_id occurrences from public API
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-08-07 16:12:06 +02:00
Michal Mielewczyk
80f82cab2b Deinitialize volume types in ocf internally.
Adapter can provide ops for data type deinitialization, if any additional
operations are required on deinitialization procedure.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-08-05 06:16:46 -04:00
Robert Baldyga
721fb68d6e Fix ocf_volume_ops documentation
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-31 15:48:31 +02:00
Robert Baldyga
92c7e12549 Remove core id from public API
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-30 09:26:26 +02:00
Robert Baldyga
1100cb0b4f Remove cache id from public API
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-30 09:26:26 +02:00
Robert Baldyga
4f0735b503 Ensure that core name is set and unique
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-29 17:37:48 +02:00
Robert Baldyga
901b39031f Ensure that cache name is set and unique
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-29 17:37:43 +02:00
Jan Musiał
232f0cd8d7
Merge pull request #216 from robertbaldyga/io-and-req-in-single-allocation
Allocate io and req in single allocation
2019-07-23 11:40:30 +02:00
Jan Musial
917cbd859a Add promotion policy API and use it in I/O path
Promotion policy is supposed to perform ALRU noise filtering by
eliminating one-hit wonders being added to cache and polluting it.

Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-07-19 13:52:00 +02:00
Robert Baldyga
61414f889e Introduce OCF IO allocator
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-17 10:31:23 +02:00
Robert Baldyga
e254c9b587 Merge new_io and configure into one function
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-17 10:31:23 +02:00
Robert Baldyga
4d2d31ff76 Introduce ocf_io_internal
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-17 09:54:51 +02:00
Michal Mielewczyk
636834dae3 Check for existing metadata when attaching cache.
Instead of overriding existing metadata silently, return appropriate error.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-07-08 09:23:32 -04:00
Michal Rakowski
b1a6c467a0 Introduce core_is_dirty mngt method
Signed-off-by: Michal Rakowski <michal.rakowski@intel.com>
2019-06-25 09:12:44 +02:00
Michal Rakowski
a818ffabfd Move ocf_mngt_cache_is_dirty to public mngt interface 2019-06-24 15:23:06 +02:00
Adam Rutkowski
b97bb6f53b Add function to check for lazy write cache mode (WO or WB)
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-06-12 12:07:02 -04:00
Adam Rutkowski
82e8c55f4a Write-only cache mode
Write-only cache mode is similar to writeback, however read
operations do not promote data to cache. Reads are mostly serviced
by the core device, only dirty sectors are fetched from the cache.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-06-12 12:07:02 -04:00
Michal Rakowski
9f4536c6e3 Error codes in IO path changed to OCF-specific
Signed-off-by: Michal Rakowski <michal.rakowski@intel.com>
2019-06-05 09:10:54 +02:00
Michal Wysoczanski
9e75edd18c Explicitly cast void pointer to const char *.
Signed-off-by: Michal Wysoczanski <michal.wysoczanski@intel.com>
2019-05-30 00:08:58 +02:00
Robert Baldyga
46ae8b2ff1 Remove unused OCF_ERR_CACHE_IN_USE error code
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-05-28 11:20:48 +02:00
Robert Baldyga
8a82be339f Introduce asynchronous cache lock
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-05-28 11:20:48 +02:00
Robert Baldyga
cda536a14a Remove mpool from OCF utils
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-05-27 13:04:40 +02:00
Jan Musiał
93a06686a7
Merge pull request #165 from robertbaldyga/ctx-get-put
Give ocf_ctx get/put semantics
2019-05-24 14:29:04 +02:00
Robert Baldyga
c903d13ad2 Give ocf_ctx get/put semantics
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-05-23 15:50:10 +02:00
Robert Baldyga
0526239baf Introduce helpers for setting config structures to defaults
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-05-23 13:29:49 +02:00
Robert Baldyga
65a6fc0468 Remove unused cache_id field from core config
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-05-23 13:29:49 +02:00
Robert Baldyga
efa96ef7bd Add ocf_cache_line_size_none
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-05-23 13:29:49 +02:00
Robert Baldyga
2cf2a59584 Add open_cores parameter to ocf_mngt_cache_device_config
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-05-21 18:18:33 +02:00
Robert Baldyga
92d5ad8075 Remove unused parameter from ocf_mngt_cache_device_config
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-05-21 18:17:00 +02:00
Michal Mielewczyk
7165bc16c3 Add kick function for cleaner
ocf_kick_cleaner() allows to perfom cleaning immediately.

Nop cleaning policy now returns new 'OCF_CLEANER_DISABLE' macro which indicates
that cleaing shouldn't be performed. To enable it back, ocf_kick_cleaner()
should be called.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-05-10 08:09:56 -04:00
Jan Musiał
8e4a1f10f0
Merge pull request #146 from robertbaldyga/init-core-pool-with-ctx
Initialize and deinitialize core pool with ctx
2019-05-09 12:41:00 +02:00
Robert Baldyga
6c7df8abf6 Initialize and deinitialize core pool with ctx
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-05-09 11:22:06 +02:00
Robert Bałdyga
1c9fe96663
Merge pull request #133 from arutk/ajrutkow_async_counters
Extended reference counting
2019-05-08 14:23:50 +02:00
Robert Baldyga
7b88aac56f Remove "interruption" argument from flush() functions
As non-interruptible flushes are no longer triggered from OCF
internals, we can get rid of "interruption" argument and let
adapters handle it themselves.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-05-07 17:23:19 +02:00
Adam Rutkowski
348b0f9ab8 Async wait for cleaner completion
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-05-06 19:37:49 -04:00
Adam Rutkowski
962f9d17d1 Remove functions to wait for cache pending requests
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-05-06 19:23:11 -04:00
Katarzyna Łapińska
c80ca4a9a8
Merge pull request #122 from KamilLepek/security_tests
Add security fuzzy tests and improve pyocf api
2019-04-18 07:33:11 +02:00
Michal Mielewczyk
5e52ac4ef2 Rename 'printf' field name in logger ops to 'print'.
Due the aggresive security checks in compiler 'printf' might be substituded with
'__printf_chk'. However it does not differentiate whether substituted string is
library function call whether field in structure.

By renaming field we prevent it to be unintentionally subustituted by the
preprocessor.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-04-17 11:37:47 -04:00
Jan Musiał
121b3233e4
Merge pull request #120 from arutk/core-query-api
Add API to query for core devices associated with cache
2019-04-17 13:33:04 +02:00
Adam Rutkowski
dfd89dd43a Probe core UUIDs from metadata
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-17 14:07:42 -04:00
Kamil Lepek
d55195ecf5 Add security fuzzy tests and improve pyocf api
Fuzzy tests added for:
changing cache mode
attaching cache device with fuzzed cache line size
setting cleaning policy
setting alru params
setting acp params
setting seq cut off policy per cache
setting seq cut off policy per core
2019-04-17 12:46:25 +02:00
Jan Musial
003d6505aa Fix leaky tests
Fix some instance tracking to prevent Python-side objects from leaking
buffers.
Also, reduce min size of Cache instance (real minimum should be around ~~19MiB,
but we need to make it more deterministic and 20 MiB seems to be reasonable).

Still some stuff left to do, but it needs more investigation and, for
now, this should suffice just to enable some form of CI.

Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-04-16 09:55:02 +02:00
Adam Rutkowski
2aba5a2843 Add volume_params argument to bottom volume open
This pointer is used to provide optional volume specific data
from the user down to bottom volume open callback. volume_params
is provided to OCF in ocf_mngt_cache_device_config.volume_params.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-01 19:28:14 -04:00
Michal Wysoczanski
901bbc5ebf Make ioclass name bigger
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-03-27 17:09:01 -04:00
Robert Baldyga
5699422586 Introduce ocf_cache_has_pending_cleaning() function
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-03-26 15:09:52 +01:00
Vitaliy Mysak
b2b0a56fbd Remove undefined declaration: ocf_cleaner_set_io_queue
Function ocf_cleaner_set_io_queue is not used and not defined
But is declared

This patch removes declaration

Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
2019-03-21 09:17:44 +01:00
Robert Baldyga
91e0345b78 Implement asynchronous attach, load, detach and stop
NOTE: This is still not the real asynchronism. Metadata interfaces
are still not fully asynchronous.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-03-18 15:06:57 +01:00
Robert Baldyga
c5df82f2cb Make management API asynchronous
NOTE: This patch only changes API that pretends to be asynchronous.
Most of management operations are still performed synchronously.
The real asynchronism will be introduced in the next patches.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-03-12 10:33:26 +01:00
Michal Rakowski
e5227cef89 Implement pyocf adapter with sample OCF test
PyOCF is a tool written with testing OCF functionality in mind.
It is a Python3 (3.6 version required) package which wraps OCF
by providing Python objects in place of OCF objects (volumes, queues,
etc). Thin layer of translation between OCF objects and PyOCF objects
enables using customized behaviors for OCF primitives by subclassing
PyOCF classes.

This initial version implements only WT and WI modes and single,
synchronously operating Queue.

TO DO:

  - Queues/Cleaner/MetadataUpdater implemented as Python threads
  - Loading of caches from PyOCF Volumes (fix bugs in OCF)
  - Make sure it works multi-threaded for more sophisticated tests

Co-authored-by: Jan Musial <jan.musial@intel.com>
Signed-off-by: Michal Rakowski <michal.rakowski@intel.com>
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-03-07 12:48:40 +01:00
Michal Rakowski
794b008127
Merge pull request #66 from robertbaldyga/metadata-probe-asynchronous
Make metadata probe asynchronous
2019-03-04 12:26:34 +01:00
Robert Baldyga
9b9b965b55 Add cache locking functions in trylock variant
- Add cache trylock and read trylock functions.
- Introduce new error code -OCF_ERR_NO_LOCK.
- Change trylock functions in env to return this code in case of
  lock contention.

[ENV CHANGES REQUIRED]
Following functions should return 0 on success or -OCF_ERR_NO_LOCK
in case of lock contention:
- env_mutex_trylock()
- env_rwsem_up_read_trylock()
- env_rwsem_up_write_trylock()

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-03-01 17:31:32 +01:00
Robert Baldyga
9c0390a6ce Make metadata probe asynchronous
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-03-01 15:46:16 +01:00
Robert Baldyga
c1028b2b89 Add ocf_cache_set_flush_queue() function
This is needed to enable flushing with dynamic queues.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-02-28 10:51:51 +01:00
Jan Musiał
d5ac3035d7
Merge pull request #64 from robertbaldyga/add-has-pending-reqs-func
Add ocf_cache_has_pending_requests() function
2019-02-27 12:22:16 +01:00
Michal Mielewczyk
e53944d472 Dynamic I/O queue management
- Queue allocation is now separated from starting cache.
- Queue can be created and destroyed in runtime.
- All queue ops accept queue handle instead of queue id.
- Cache stores queues as list instead of array.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-02-26 17:36:19 +01:00
Robert Baldyga
03bd6d6c49 Add ocf_cache_has_pending_requests() function
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-02-26 17:23:07 +01:00
Robert Baldyga
44ed250d41 Add API function ocf_cache_wait_for_io_finish()
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-02-20 11:16:06 +01:00
Robert Baldyga
404b976109 Make cleaner asynchronous
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-02-19 16:10:24 +01:00
Michal Mielewczyk
f8037cb349 Added function to hanlde single request from queue. 2019-02-13 12:02:59 -05:00
Robert Baldyga
97300b1137 Rename "data object" to "volume"
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-02-08 14:13:05 +01:00
Michal Mielewczyk
2dfd6248c9 Configure all io classes at once.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-01-28 08:51:26 -05:00
Robert Baldyga
3b4a666942 Enable passing priv to logger during context initialization
This may be used in logger implementations that need file
name or descriptor to initialize properly.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-01-24 10:54:50 +01:00
Robert Baldyga
6860cdbd2c Reorganize context ops
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-01-22 13:14:35 +01:00
Robert Baldyga
03c2f142b3 Cache lock cleanup and API improvements
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-01-21 13:46:28 +01:00
Robert Baldyga
7352f3092d Fix leak in core pool
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-01-16 11:53:54 +01:00
Robert Baldyga
d9cfab67b8 Fix compilation errors after tracing introduction
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-01-14 17:28:51 +01:00
Robert Bałdyga
dd6b074258
Merge pull request #3 from tomaszrybicki/add-tracing-support
Added tracing functionality
2019-01-14 16:37:34 +01:00
Michał Mielewczyk
f11db9c31a
Merge pull request #38 from robertbaldyga/remove-name_size
Remove name_size field from cache and core config
2019-01-14 16:04:16 +01:00
Robert Baldyga
322f1aeffd Remove name_size field from cache and core config
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-01-14 15:09:09 +01:00
Robert Baldyga
afb63790f2 Add ocf_uuid_to_str() function
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-01-14 15:07:31 +01:00
Robert Baldyga
749f6906bb Fix function initializing UUID from string
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-01-14 15:07:15 +01:00
Tomasz Rybicki
1156531590 Added tracing functionality
Signed-off-by: Tomasz Rybicki <tomasz.rybicki@intel.com>
2019-01-14 14:17:48 +01:00
Robert Baldyga
4cf4d6d7c7 Simplify data object API
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-01-04 17:34:39 +01:00
Robert Baldyga
4be6761a18 Core data object
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-01-04 08:33:34 +01:00
Robert Baldyga
848cd9112b Add str_to_uuid function
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2018-12-19 13:30:17 +01:00
Robert Baldyga
db92083432 Unify req naming convention (rq -> req)
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2018-12-12 13:36:34 +01:00
Robert Baldyga
d862778e2b Remove legacy io completion API
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2018-12-12 11:20:41 +01:00
Robert Baldyga
a8e1ce8cc5 Initial commit
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2018-11-29 15:14:21 +01:00