Commit Graph

365 Commits

Author SHA1 Message Date
Robert Baldyga
96a1fdb17e Deinitialize locks on cache stop instead of put
Cache lock waiters hold cache refcount. Because of that,
if there were some waiters, deinitialization of cache
lock on the last put did never happen and putting the
cache was effectively impossible.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-09-27 12:02:04 +02:00
Robert Bałdyga
75569ecaba
Merge pull request #284 from mmichal10/prevent-cache-name-duplicate
Prevent cache name duplicate
2019-09-25 15:36:08 +02:00
Michal Mielewczyk
6c076a7c07 Remove set_cache_name() from public API.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-25 09:21:42 -04:00
Michał Mielewczyk
9e707d81b5
Merge pull request #287 from micrakow/nhit_fixe
Nhit fixes
2019-09-25 15:11:47 +02:00
Michal Rakowski
5efa5ac414 nhit PP: Prevent setting nhit policy again if it was already set 2019-09-25 14:59:23 +02:00
Michal Rakowski
547306efea nhit PP: change trigger_threshold to percent value
Signed-off-by: Michal Rakowski <michal.rakowski@intel.com>
2019-09-25 14:59:23 +02:00
Michał Mielewczyk
9613c325fc
Merge pull request #285 from arutk/fix_io_class_configure
Fix error handling in IO ocf_mngt_cache_io_classes_configure
2019-09-25 14:42:06 +02:00
Michał Wysoczański
aafe870e44
Merge pull request #280 from arutk/metadata_sync_2
Additional metadata synchronization
2019-09-25 14:11:59 +02:00
Adam Rutkowski
a934b43aec Add missing error handling in hash bucket locks initialization
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-25 14:26:17 -04:00
Adam Rutkowski
6de280283a Fix hash_table_entries param type in ocf_metadata_concurrency_attached_init
Number of hash buckets is 32 bit integer.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-25 14:26:17 -04:00
Adam Rutkowski
937b010ef6 Synchronize access to cleaner shared structures
Cleaning policy callbacks are typically  called with hash buckets or
cache lines locked. However cleaning policies maintain structures
which are shared across multiple cache lines (e.g. ALRU list).
Additional synchronization is required for theese structures to
maintain integrity.

ACP already implements hash bucket locks. This patch is adding
ALRU list mutex.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-25 14:26:17 -04:00
Adam Rutkowski
5e28474322 Adding partition locks
Adding locks to assure partition list consistency. Partition
lists is typically modified under cacheline or hash bucket lock.
This is not sufficient synchronization as adding/removing cacheline
from partition list affects neighbouring cachelines state as well.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-25 14:26:17 -04:00
Adam Rutkowski
41d3542952 Lock collision page in metadata flush
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-25 14:26:15 -04:00
Michal Mielewczyk
f461f3c62e Extend probe informations with cache name.
Since ocf requires loading cache with the same name as it was stopped, it should
also allow to read name from metadata.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-25 07:59:50 -04:00
Michal Mielewczyk
39c5819a51 Set cache name before adding it to context list.
This change allows to check if specified cache name name is unique. To prevent
adding cache instance with the same name, context lock is acquired until name
isn't set.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-25 07:59:50 -04:00
Michal Mielewczyk
c04ea4898f Check if loaded cache name is valid.
When loading cache, it's name should be the same as the loaded one.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-25 05:06:37 -04:00
Michal Rakowski
23aba6a9f3 nhit PP: Added info about setting nhit params
Signed-off-by: Michal Rakowski <michal.rakowski@intel.com>
2019-09-25 11:04:51 +02:00
Adam Rutkowski
be3b402162 Synchronization of collision table
Adding synchronization around metadata collision segment pages.
This part of metadata is modified when cacheline is mapped/unmapped
and when dirty status changes.

Synchronization on page level is required on top of cacheline
and hash bucket locks to assure metadata flush always reads
consistent state when copying entire collision table memory
page.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-25 00:26:29 -04:00
Adam Rutkowski
5684b53d9b Adding collision table locks
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-24 22:59:35 -04:00
Adam Rutkowski
727a6d2e4b Fix error handling in IO ocf_mngt_cache_io_classes_configure
Modifying _ocf_mngt_io_class_configure and _ocf_mngt_io_class_remove
to never return -OCF_ERR_IO_CLASS_NOT_EXIST error code. This
return code was ignored by the caller anyway. In _ocf_mngt_io_class_remove
-OCF_ERR_IO_CLASS_NOT_EXIST indicated the IO class is already
removed, which is not an error. In _ocf_mngt_io_class_configure
-OCF_ERR_IO_CLASS_NOT_EXIST indicated empty IO class name, which
is actualy invalid input. This change made it possible to remove
erroneous error handling for -OCF_ERR_IO_CLASS_NOT_EXIST case in
ocf_mngt_cache_io_classes_configure.

This change fixes IO class configuration.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-24 21:51:43 -04:00
Michał Wysoczański
07aa29fc56
Merge pull request #283 from rafalste/fix_nhit_param_value
Accept max values of nhit PP as valid.
2019-09-24 10:46:38 +02:00
Rafal Stefanowski
9cb5c60c80 Accept max values of nhit PP as valid.
Signed-off-by: Rafal Stefanowski <rafal.stefanowski@intel.com>
2019-09-23 15:58:57 +02:00
Robert Baldyga
b8f5f135fe ocf_async_lock: Replace mutex with spinlocks
The ocf_async_lock may be used in atomic context, thus we need
to replace synchronization primitives to non-sleeping variants.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-09-23 15:54:25 +02:00
Robert Baldyga
dd0a39eea7 Create new volume instead of using non-allocated one
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-09-23 14:01:17 +02:00
Adam Rutkowski
30f22d4f47 Optimize cacheline locking in ocf_engine_prepare_clines
Hash bucket read/write lock is sufficient to safely attempt
cacheline trylock/lock. This change removes cacheline lock
global RW semaprhore and moves cacheline trylock/lock under
hash bucket read/write lock respectively.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-20 17:21:00 -04:00
Adam Rutkowski
5248093e1f Move common mapping and locking logic to dedicated function
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-20 17:21:00 -04:00
Adam Rutkowski
d2bd807e49 Remove calls to OCF_METADATA_(UN)LOCK_WR(RD)
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-20 17:21:00 -04:00
Adam Rutkowski
2333d837fb Add single hash bucket lock interface
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-20 17:21:00 -04:00
Adam Rutkowski
3a70d68d38 Switch from global metadata locks to hash-bucket locks in engines
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-20 17:21:00 -04:00
Adam Rutkowski
b39bcf86d4 Separate engine map/evict (refactoring)
This temporarily increases amount of boiler-plate code, but
this is going to be mitigated in the following commits.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-20 17:21:00 -04:00
Adam Rutkowski
d91012f4b4 Introduce hash bucket locks
There is one RW lock per hash bucket. Write lock is required
to map cacheline, read lock is sufficient for traversing.
Hash bucket locks are always acquired under global metadata
read lock. This assures mutual exclusion with eviction and
management paths, where global metadata write lock is held.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-20 17:21:00 -04:00
Adam Rutkowski
42f65c3fbb Change ocf_metadata_(un)lock -> OCF_METADATA_(UN)LOCK
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-20 17:21:00 -04:00
Adam Rutkowski
f34cacf150 Move resume callback to async lock function params (refactoring)
This is a step towards common async lock interface in OCF.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-20 17:21:00 -04:00
Jan Musial
827273645c Use better function for calculating occupancy 2019-09-20 14:56:17 +02:00
Kamil Łepek
7131178e71
Merge pull request #272 from imjfckm/fix-pp-validation
Add PP type validation
2019-09-18 17:30:23 +02:00
Jan Musial
0c1ccddf8a Add PP type validation
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-09-18 15:12:23 +02:00
Jan Musial
0e85ebe4a3 Get PP params in line with rest of OCF
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-09-18 11:15:41 +02:00
Jan Musial
e9bd139349 Add validation of PP for cache start config
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-09-18 09:53:13 +02:00
Jan Musial
e8fc2c24f1 Add missing stuff from get_param in PP
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-09-17 15:05:29 +02:00
Michał Mielewczyk
f86287ef06
Merge pull request #261 from micrakow/coverity_19_9
Fixed some bugs found by the coverity tool
2019-09-17 09:25:18 +02:00
Michal Rakowski
83e23c5593 Fixed some bugs found by the coverity tool
Signed-off-by: Michal Rakowski <michal.rakowski@intel.com>
2019-09-16 15:49:37 +02:00
Michal Mielewczyk
c5edc60345 Fix stats update in cleaner.
Core is not assigned to request in cleaner, so to increase it's stats it has to
be retrieved from mapping.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-16 05:59:28 -04:00
Michał Mielewczyk
0391fc17b7
Merge pull request #255 from imjfckm/pp-tests
Add promotion policies functional tests
2019-09-16 09:34:16 +02:00
Jan Musiał
58012cd14b
Merge pull request #260 from mmichal10/unify-inactive-cores-stats
Unify inactive cores stats
2019-09-16 09:03:08 +02:00
Michal Mielewczyk
f226f978f0 Unify inactive cores stats.
Inactive core stats should be caluculated and returned to adapter in unified
from, just like all stats are.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-16 02:49:42 -04:00
Michal Mielewczyk
494a1ccc79 Extract stats builder utils to separate file.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-12 05:21:17 -04:00
Jan Musial
703a757db1 Fix minor bugs in promotion policy
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-09-11 10:31:14 +02:00
Jan Musial
633f31716e Make NHIT API naming convention similar to cleaning
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-09-11 07:31:50 +02:00
Michal Rakowski
29c1c7f9e8
Merge pull request #253 from mmichal10/stats-refactor
Stats builder for ioclasses
2019-09-10 14:56:26 +02:00
Michal Mielewczyk
e6f032cae9 Fix typo.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-10 08:01:10 -04:00
Michal Mielewczyk
01ce586e6a Use API instead of raw variables to update block stats.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-10 08:01:09 -04:00
Michal Mielewczyk
51c9c516a4 Use API instead of raw variables to update req stats.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-10 08:01:09 -04:00
Michal Mielewczyk
b4c384eb2d Use API instead of raw variables to update error stats.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-10 08:01:09 -04:00
Michal Mielewczyk
0d4f4d3294 Added API for updating statistics.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-10 08:01:09 -04:00
Michal Mielewczyk
42d6dbbf11 Remove low-level stats getters from public API.
Since stats builder is implemented for retrieving cache, core and ioclass stats,
adapters should use it instead.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-10 08:01:09 -04:00
Michal Mielewczyk
89de6038c1 Split retrieving core stats and config info into two functions.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-10 08:01:09 -04:00
Michal Mielewczyk
f9da89263b Stats builder for ioclass statistics.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-10 04:32:21 -04:00
Adam Rutkowski
13cf871a13 Per-execution-context freelists
Global free cacheline list is divided into a set of freelists, one
per execution context. When attempting to map addres to cache, first
the freelist for current execution context is considered (fast path).
If current execution context freelist is empty (fast path failure),
mapping function attempts to get freelist from other execution context
list (slow path).

The purpose of this change is improve concurrency in freelist access.
It is part of fine granularity metadata lock implementation.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-09-09 16:19:52 -04:00
Michal Mielewczyk
2450d3da4b Move block stats counters to ioclass section.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-09 02:46:45 -04:00
Michal Mielewczyk
8304ed84b8 More precise percentage value in stats builder.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-09-09 02:46:45 -04:00
Robert Bałdyga
5fd239d2bb
Merge pull request #245 from imjfckm/switching-pp
Switching mechanism for promotion policies
2019-08-30 11:35:16 +02:00
Robert Bałdyga
1f9f79f11d
Merge pull request #250 from robertbaldyga/parts-meta-new-section
Move parts metadata to separate section
2019-08-29 14:15:31 +02:00
Jan Musial
ee3fdb0506 Change nhit_hash names to avoid collision with kernel
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-08-29 13:24:54 +02:00
Jan Musial
a1d3cf0c4d Switching mechanism for promotion policies
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-08-29 13:24:26 +02:00
Robert Baldyga
d10c618065 Move parts metadata to separate section
Fix problem introduced by increasing partition name size to 1024 bytes,
which effectively made superblock bigger than one page. Due to this
flushing superblock required more than one io, which in case of dirty
shutdown between these ios resulted in CRC missmatch and made cache
recovery impossible.

Moving parts metadata to separate sections makes superblock fitting
in one page, effectively solving described problem.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-08-29 10:23:13 +02:00
Jan Musiał
0a9f197b10
Merge pull request #238 from arutk/split_cacheline_lock_trylock
Split cacheline locking implementation into trylock and lock
2019-08-29 09:49:53 +02:00
Adam Rutkowski
6dd1af7df7 Split cacheline locking implementation into trylock and lock
This change refactors the code in order to prepare for removing
global concurrency lock, which won't be needed after per-bucket
metadata locking is in place.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-08-28 18:54:19 -04:00
Robert Bałdyga
358c529eff
Merge pull request #243 from arutk/simplify_freelist_striping_init
Initialize freelist using metadata layout implementation
2019-08-27 17:12:04 +02:00
Michal Rakowski
e3a20c0da0
Merge pull request #239 from imjfckm/pp-implementation
nhit promotion policy implementation
2019-08-27 12:42:03 +02:00
Adam Rutkowski
860c8c7a0f Initialize freelist using metadata layout implementation
Metadata layout (seq / striping) is already encapsulated
in ocf_metadata_layout_iface, no need to double this logic
in freelist initialization.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-08-26 15:48:11 -04:00
Jan Musial
62a0ccdd9f nhit promotion policy implementation
Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-08-22 13:34:38 +02:00
Michal Mielewczyk
2c27e9a433 Store name for newly initialized cache in array instead of passing pointer.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-08-20 01:43:11 -04:00
Robert Bałdyga
8ed525ae7f
Merge pull request #234 from mdnfiras/sync-prim-destroyers
Adding synchronization primitives destroyers
2019-08-13 15:46:26 +02:00
Firas Medini
33c96d226e Verifying IO address and size
IO address and size must be multiple of sector size. If not, ocf_io_new() will return NULL.

Signed-off-by: Firas Medini <mdnfiras@yahoo.com>
2019-08-13 05:37:14 -07:00
Firas Medini
1f979f630b Adding synchronization primitives destroyers
Environment should provide calls for destroying primitives (i.e. env_mutex_destroy()) and OCF should call these functions in its cleanup paths.

Signed-off-by: Firas Medini <mdnfiras@yahoo.com>
2019-08-13 05:13:11 -07:00
Robert Baldyga
9730b2bd1f Remove remaining core_id occurrences from public API
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-08-07 16:12:06 +02:00
Michal Rakowski
d499ef1c05
Merge pull request #227 from mmichal10/mark-core-added-false
Don't try to remove invalid cores

If valid cache metadata was read, but environment has changed (i.e. number of cache lines has changed) ocf (in error handling path) was trying to close cores which were not opened. It happened due to cores were marked in cache metadata as added, but any cache inserting operation didn't take place.

In this patch 'added' flag in cache metadata was replaced with more meaningful 'valid' - it is set if given core is stored in cache metadata. Moreover, new 'added' flag was added to core run-time metadata and it is set if given core is added to cache.
2019-08-07 13:37:59 +02:00
Michal Mielewczyk
cdc0a575a1 Added 'added' flag in core runtime data.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-08-06 07:36:39 -04:00
Michal Mielewczyk
433a25a1b2 Renamed 'added' flag in core metadata to 'valid'.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-08-06 07:08:12 -04:00
Michal Mielewczyk
5dd09e823b Initialize promotion policy before cleaning policy.
If cleaning policy didn't have init() function,
   '_ocf_mngt_init_instance_load_complete' returned early and promotion policy
   wasn't initialized at all.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-08-06 06:37:14 -04:00
Adam Rutkowski
9649f25415
Merge pull request #229 from robertbaldyga/cleanup-metadata
Metadata interface cleanup
2019-08-05 14:11:04 +02:00
Michal Mielewczyk
340ca76159 Move check for valid cache size in load.
Loaded cache size should be compared with acutal cache size after loading
superblock.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-08-05 07:58:38 -04:00
Michal Mielewczyk
80f82cab2b Deinitialize volume types in ocf internally.
Adapter can provide ops for data type deinitialization, if any additional
operations are required on deinitialization procedure.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-08-05 06:16:46 -04:00
Robert Baldyga
fdf6d07b85 Fix typo "collisioin" -> "collision"
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-08-03 20:04:23 +02:00
Robert Baldyga
4b1a3fa4b5 Simplify metadata_raw interface
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-08-03 20:04:23 +02:00
Robert Baldyga
a31ddd44cb Remove ocf_metadata_hash_entries_hash()
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-08-03 20:04:23 +02:00
Robert Baldyga
ff7583c6da Simplify partition metadata interface
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-08-03 20:04:23 +02:00
Robert Baldyga
3108f36e42 Simplify collision matadata interface
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-08-03 20:04:23 +02:00
Robert Baldyga
0b0546302f Remove ocf_metadata_get_core_sector()
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-08-03 20:04:23 +02:00
Robert Baldyga
0ca0ea1343 Remove ocf_metadata_line_size_is_valid()
This function is no longer used.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-08-03 20:04:23 +02:00
Robert Baldyga
92c7e12549 Remove core id from public API
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-30 09:26:26 +02:00
Robert Baldyga
1100cb0b4f Remove cache id from public API
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-30 09:26:26 +02:00
Robert Baldyga
259df7ace9 Store core name in metadata
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-30 09:26:26 +02:00
Robert Baldyga
eb4272afa9 Store cache name in metadata
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-30 09:26:17 +02:00
Robert Baldyga
4f0735b503 Ensure that core name is set and unique
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-29 17:37:48 +02:00
Robert Baldyga
901b39031f Ensure that cache name is set and unique
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-29 17:37:43 +02:00
Robert Baldyga
331b99397f Change ctx lock to rmutex
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-29 15:08:44 +02:00
Kamil Łepek
05be67a72b
Merge pull request #220 from arutk/metadata_offset_hash
New hash function formula
2019-07-29 13:03:04 +02:00
Jan Musiał
fa14d6a4b5
Merge pull request #221 from arutk/metadata_rename_concurrency
Restrutcure concurrency code
2019-07-29 09:42:37 +02:00
Robert Baldyga
3355a6cd56 engine: discard: Add missing put on error path
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-07-27 19:06:11 +02:00