Commit Graph

253 Commits

Author SHA1 Message Date
Robert Baldyga
8b93b699c3 Eliminate queue -> cache mapping
Eliminate need to resolve cache based on the queue. This allows to share
the queue between cache instances. The queue still holds pointer to
a cache that owns the queue, but no management or io path relies on the
queue -> cache mapping.

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-09 12:45:51 +02:00
Robert Baldyga
460cd461d3 Allocate requests for management path separately
Management path does not benefit much from mpools, as number of requests
allocated is very small. It's less restrictive (mngt_queue does not have
single-CPU affinity) thus avoiding mpool usage in management path allows
to introduce additional restrictions on mpool, leading to I/O performance
improvement.

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-09 12:45:51 +02:00
Robert Baldyga
5b2f26decf
Merge pull request #800 from robertbaldyga/redesign-queue-api
Redesign queue API
2024-08-02 14:43:52 +02:00
Ian Levine
ac1b6b774a Added a priority queue for the request instead of push front
Now the request can be pushed to a high priority queue (instead of ocf_queue_push_req_front)
and to a low priority queue (instead of ocf_queue_push_req_back).
Both functions were merged into one function (ocf_queue_push_req) and instead of the
allow_sync parameter there is now a flags parameter that can be an OR combination of
OCF_QUEUE_ALLOW_SYNC and OCF_QUEUE_PRIO_HIGH

Signed-off-by: Ian Levine <ian.levine@huawei.com>
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
2024-08-02 12:53:16 +02:00
Ian Levine
038126e9ab Move and rename ocf_engine_push_req_* from engine_common to ocf_queue_push_req_* in ocf_queue
Signed-off-by: Ian Levine <ian.levine@huawei.com>
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
2024-08-02 12:53:16 +02:00
Robert Baldyga
40ff7d2dcf
Merge pull request #799 from Open-CAS/cache_detach
Implement cache detach/attach
2024-07-31 06:54:08 +02:00
Robert Baldyga
43cc487c40 lru: Move partition runtime structures outside of metadata
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
2024-07-12 13:23:29 +02:00
Michal Mielewczyk
83ec255458 Disable changing cache params for detached cache
Majority of management operations should be blocked for detached cache,
although adding and removing cores should be possible.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-07-10 16:19:37 +02:00
Robert Baldyga
168ecd0075 Add missing "static" to the local function
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
2024-05-11 00:59:39 +02:00
Robert Baldyga
253734b160 Move misplaced function declaration to the appropriate header
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
2024-05-11 00:49:52 +02:00
Michal Mielewczyk
8faf74169a Parallelize initializing hash table
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2022-09-26 14:14:40 +02:00
Michal Mielewczyk
4dbf740f5b Parallelize initializing collision section
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2022-09-26 14:14:40 +02:00
Michal Mielewczyk
b50bd1b506 Initialize metadata structures in pipelines
Initializing metadata in an asynchronous manner will allow to use
parallelization utilities in the future commits

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2022-09-26 14:06:40 +02:00
Michal Mielewczyk
ef997b47fa Fix whitespaces
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2022-09-23 07:09:41 +02:00
Robert Baldyga
1c701e4101
Merge pull request #750 from robertbaldyga/remove-req-io-if
Get rid of req->io_if
2022-09-08 22:59:57 +02:00
Robert Baldyga
228c5fc891 Get rid of req->io_if
Remove one callback indirection level. I/O never changes it's direction
so there is no point in storing both read and write callbacks for each
request.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-09-07 23:07:04 +02:00
Adam Rutkowski
0a09d05a8b Add missing ocf_metadata_read_sb error handling
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-09-06 13:24:05 +02:00
Krzysztof Majzerowicz-Jaszcz
e12803f547 Fix for bad metadata capacity reported by dmesg
Metadata capacity reported by dmesg was actually a memory footprint.

A proper size of metadata is now reported.

Signed-off-by: Krzysztof Majzerowicz-Jaszcz <krzysztof.majzerowicz-jaszcz@intel.com>
2022-07-06 14:30:39 +02:00
Robert Baldyga
f0f6ff219b Set core volume type in metadata on core insert
ocf_metadata_flush_superblock() is being called on the cache stop, after
deinitialization of the cores (and their volumes), thus accessing core
volume in superblock flushing procedure leads to use-after-free bug.

Fix this by moving volume type setting to the core insertion code.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-28 07:59:43 +02:00
Robert Baldyga
8822094f14 Fix metadata on disk size calculation when cleaner is disabled
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-21 09:33:42 +02:00
Piotr Debski
c448043b42 Conditional pipeline step for filtering invalid segments
Signed-off-by: Piotr Debski <piotr.debski@intel.com>
2022-06-16 09:33:09 +02:00
Adam Rutkowski
1a27b07f72 Pipeline conditional step
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Signed-off-by: Piotr Debski <piotr.debski@intel.com>
2022-06-16 09:33:09 +02:00
Robert Baldyga
d4df912f46 Add option to disable cleaner
This allows to avoid allocating cleaner metadata section and effectively
save up to 20% of metadata memory footprint.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-04-28 13:04:27 +02:00
Krzysztof Majzerowicz-Jaszcz
1b3f0d44a8 Fix error code for superblock checksum mismatch
Fix error code for superblock checksum mismatch.
Superblock validation now returns a proper error on checksum check fail.

Signed-off-by: Krzysztof Majzerowicz-Jaszcz <krzysztof.majzerowicz-jaszcz@intel.com>
2022-04-01 07:23:49 +00:00
Robert Baldyga
25434cb8d1 Explicitly validate valid_core_bitmap consistency
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-03-30 23:46:06 +02:00
Jan Musial
d1bd32add9 Fix potential unsigned overflow in calculations
Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-03-30 08:24:39 +02:00
Robert Baldyga
af43a240d3 Return more specific error on CRC mismatch
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-03-28 22:42:59 +02:00
Adam Rutkowski
6b6300c646 Add extra data seek before data fill in mio
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 14:26:51 +02:00
Robert Baldyga
4fc3f8f0d1 Remove extra whitespace
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-03-09 11:44:27 +01:00
Robert Baldyga
d5b2c65a39 Remove "metadata_layout" parameter of the cache
This feature is replaced with LRU list shuffling.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-03-07 17:48:25 +01:00
Adam Rutkowski
689c44c76b Remove ocf_metadata_probe_cores() implementation
This function must be fixed to work with metadata flapping. Until then
mark as not supported

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Signed-off-by: Krzysztof Majzerowicz-Jaszcz <krzysztof.majzerowicz-jaszcz@intel.com>
2022-03-04 19:13:40 +01:00
Adam Rutkowski
866bba72bf Explicitly validate superblock after load
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>

Additional changes - load sb recovery CRC check

Signed-off-by: Krzysztof Majzerowicz-Jaszcz <krzysztof.majzerowicz-jaszcz@intel.com>
2022-03-04 19:12:51 +01:00
Robert Baldyga
90ff4afcda Check superblock CRC before it is used
Superblock can be used during load of other sections, so we need to check
its CRC before other sections are loaded.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
Signed-off-by: Krzysztof Majzerowicz-Jaszcz <krzysztof.majzerowicz-jaszcz@intel.com>
2022-03-04 19:12:08 +01:00
Krzysztof Majzerowicz-Jaszcz
06f2140090 Removing ocf_metadata_sb_crc_recovery
Removing ocf_metadata_sb_crc_recovery - not used

Signed-off-by: Krzysztof Majzerowicz-Jaszcz <krzysztof.majzerowicz-jaszcz@intel.com>
2022-03-04 19:10:47 +01:00
Robert Baldyga
805ea14529 Remove runtime recovery in standby mode
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-02-01 03:11:50 +01:00
Robert Baldyga
e30fd48338
Merge pull request #656 from jfckm/extend-metadata-probe
Include cache mode and cache line size in metadata probe
2022-02-04 13:01:10 +01:00
Jan Musial
8522b0b6e6 Include cache mode and cache line size in metadata probe
Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-02-04 08:15:05 +01:00
Michal Mielewczyk
5d74aec921 Add missing return in raw_ram_zero() in error path
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2022-01-12 07:46:49 +01:00
Adam Rutkowski
294e02bc1b Fail cache recovery in case of erroneous mapping
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-01-10 11:10:02 +01:00
Adam Rutkowski
9693b82cf9 Only flush superblock at the end of cache attach
The purpose of this change is not to write superblock to the cache
drive untill all other sections are initilized on disk in attach()
path. Combined with superblock clearing at the erarlier stage of
attach(), this assures there are no residual mappings in the collision
section in case of power failure during attach with pre-existing
metadata.

This is implemented by removing ocf_metadata_flush_all_set_status() step
at the beginning of ocf_metadata_flush_all().
ocf_metadata_flush_all() is called, except for the attach() case described
above, in two cases:
1. at the end of cache load - potentially after cache recovery
2. during detaching cache drive in cache stop.

To make sure there are no regressions in the first case, an explicit
_ocf_mngt_attach_shutdown_status() is added to load pipeline before
ocf_metadata_flush_all(). The second case is always ran after cache
drive is attached, so dirty status bit must have already be written to
the disk.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-01-05 13:06:59 +01:00
Adam Rutkowski
196437f9bc Zero superblock before writing metadata
This is the first step towards atomic initialization of metadata
on cache disk.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-01-05 13:06:59 +01:00
Robert Baldyga
c6644116ae
Merge pull request #614 from robertbaldyga/redesign-standby
Redesign failover standby API
2022-01-04 14:07:05 +01:00
Jan Musial
ae18ce274e Fix cache size requirements and some logging
Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-01-03 14:30:07 +01:00
Robert Baldyga
b25cd91b86 Remove unused ocf_metadata_load_unsafe()
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2021-12-29 20:31:43 +01:00
Robert Baldyga
4cabc60d40 Avoid loading runtime metadata sections during recovery
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2021-12-29 14:04:19 +01:00
Robert Baldyga
4625763df5 Return error on CRC mismatch during recovery
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2021-12-29 14:04:19 +01:00
Robert Baldyga
0751b2c0c0 Fix metadata flapping
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2021-12-15 22:21:57 +01:00
Robert Baldyga
df9a9f2722 Read superblock sections from cache volume during activate
Because of metadata flapping it is much more complicated to capture those
sections in flight in standby mode, so we read them directly from the cache
volume during the activate.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2021-12-15 15:30:34 +01:00
Robert Baldyga
99c8c05f3f Introduce flapping of metadata config sections
This feature provides double buffering of config sections to prevent
situation when power failure during metadata flush leads to partially
updated metadata. Flapping mechanism makes it always possible to perform
graceful rollback to previous config metadata content in such situation.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2021-12-15 15:30:34 +01:00
Michal Mielewczyk
655f732748 Don't access freed memory
Instead of accessing memory of a freed IO, redo size calculations

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2021-12-08 14:16:49 +01:00