Commit Graph

1945 Commits

Author SHA1 Message Date
Robert Baldyga
c7e47e226a
Merge pull request #873 from mmichal10/fix-detach-cmpl
Fix detach cmpl
2025-03-27 08:59:32 +01:00
Michal Mielewczyk
e762a5a128 pyocf: Test for IO in D2C
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-26 16:05:31 +01:00
Michal Mielewczyk
a955f70978 Track allocation size for D2C requests
Otherwise the D2C requests can't be returned to the proper memory pool,
which effectively leds to a memory leak

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-26 16:05:29 +01:00
Michal Mielewczyk
f1e25c923b D2C: Prevent use after free
Request could be completed and freed before the statistics were updated

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-26 15:53:51 +01:00
Michal Mielewczyk
baccc5560b pyocf: Add tests for detaching core
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-26 15:39:55 +01:00
Michal Mielewczyk
047b276fc8 pyocf: API for detaching core
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-26 11:39:05 +01:00
Michal Mielewczyk
62307d82b8 Don't destroy stop pipeline if detach cache failed
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-26 11:39:05 +01:00
Robert Baldyga
a03cfacd5e
Merge pull request #872 from mmichal10/fix-cleaning-deinit
Fix cleaning deinit
2025-03-26 11:34:36 +01:00
Michal Mielewczyk
dedfa5eed9 pytest: Add test for loading cores with cleaning
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-26 08:53:30 +01:00
Michal Mielewczyk
4a2597dc25 stop: Don't deinitialize cleaning policy twice
The cleaning metadata has been deinitialized in the previous pipeline
step together with other services

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-26 08:53:30 +01:00
Robert Baldyga
11c5a45bff
Merge pull request #871 from mmichal10/deinit-d2c-refcnt
Add missing d2c refcnt deinit
2025-03-26 08:21:25 +01:00
Michal Mielewczyk
6f7c0c6c51 Add missing d2c refcnt deinit
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-25 15:39:42 +01:00
Robert Baldyga
71be2d5c1e
Merge pull request #870 from mmichal10/fix-attach-refcnt
Fix attach refcnt
2025-03-25 09:55:55 +01:00
Michal Mielewczyk
a38341389a Microoptimization for resolving cache mode
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-25 09:32:39 +01:00
Michal Mielewczyk
5d0608de2c Don't unfreeze refcnt in atomic context
Unfreezing a refcnt might be a sleeping operation so it can't be called
from completion context. Moving D2C refcnt unfreeze to a separate step
in pipeline ensures that it will always be called from process context

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-25 09:32:39 +01:00
Robert Baldyga
28c6790f94
Merge pull request #869 from robertbaldyga/volume-fix-memleak
Fix refcnt memleak in ocf_volume
2025-03-25 08:36:57 +01:00
Robert Baldyga
db134d12e7 Fix refcnt memleak in ocf_volume
Avoid double initialization of volume refcnt.

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
2025-03-24 21:47:35 +01:00
Robert Baldyga
541fafd492
Merge pull request #868 from mmichal10/fix-occupancy-accounting
Fix occupancy accounting
2025-03-24 20:50:36 +01:00
Michal Mielewczyk
295b3949bc backfill: Update occupancy only if BF succeeded
Increasing the occupancy before the backfill request has completed leads
to incorrect statistics

test_lazy_engine_errors() detected the bug in the following scenario:
1. Rio submitted a read request
2. The read turned out to be a full miss. After a successful read from
   the core device, engine_read set the metadata bits to valid and
   submitted a backfill request
3. In meantime, Rio submitted a write request to the same cache lines
4. Since the test uses an error device as cache (all sectors are
   erroneous) the backfill request failed and was redirected to
   engine_invalidate
5. engine_invalidate reset the metadata bits to invalid, but since cache
   lines had waiters registered, the occupancy wasn't decremented as
   there was an assumption that the new owner (in this case, the write
   request) would do the bookkeeping
6. Upon receiving cache line locks, the write request detected that
   the mapping changed so it was completed with an error without
   decrementing the occupancy
7. The incorrect value of cached cache lines was persisted for the whole
   lifetime of the cache

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-24 12:22:25 +01:00
Michal Mielewczyk
7f55116b5e pyocf: settle cache before testing occupancy
The next commit will move occupancy accounting to backfill which makes
testing statistics value even more time dependent. Settling cache before
cache.get_stats() prevents this error-inducing race conditions

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-24 12:22:25 +01:00
Michal Mielewczyk
96e527049a Refactor __set_cache_line_invalid() pt.2
Get rid of nested ifs

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-24 12:02:34 +01:00
Michal Mielewczyk
4a01ff957d Refactor __set_cache_line_invalid() pt.1
Better naming

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-24 12:02:34 +01:00
Michal Mielewczyk
67eb940589 Refactor metadata collision API
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-24 12:02:32 +01:00
Michal Mielewczyk
e1d2ff12d8 FIX metadata collision API
If `is_valid` was NULL, OCF would crash

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-24 11:38:32 +01:00
Robert Baldyga
f43d53be19
Merge pull request #867 from mmichal10/detach_vs_acp
Detach vs acp
2025-03-17 16:20:21 +01:00
Michal Mielewczyk
98a3aaea89 pyocf: Test detaching cache with cleaning policy
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-17 15:28:09 +01:00
Michal Mielewczyk
69987bd20c detach: Don't deinitialize cleaning policy twice
The cleaning metadata has been deinitialized in the previous pipeline step
together with other services

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-17 15:28:09 +01:00
Michal Mielewczyk
1e75a9826d stop&detach: Deinit services before removing cores
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-17 15:28:09 +01:00
Michal Mielewczyk
9c2da08ffe stop pipeline: flush_metadata as a separate step
Flushing metadata has nothing to do with dinitializing services so it
should be a separate step in the stop pipeline

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-17 15:28:07 +01:00
Michal Mielewczyk
75c4c9b473 Move _ocf_mngt_cache_deinit_services() definition
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-17 15:27:17 +01:00
Michal Mielewczyk
942825146a Simplify _ocf_mngt_cache_deinit_services()
Move flushing metadata outside cache_detinit_services(), so the function
can be shared between stop() and detach() without redundant ifs.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-17 15:27:16 +01:00
Michal Mielewczyk
eca7872299 detach: Rename update_metadata() to remove_cores()
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-17 15:19:55 +01:00
Michal Mielewczyk
03338d798c detach pipeline: remove unused completion
The completion callback is called only in the cache stop scenario, after
flushing the metadata

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-17 15:19:55 +01:00
Michal Mielewczyk
5e55bf78b4 Rename cache_unplug() to deinit_services()
The function doesn't unplug anything, it just deinitializes cleaning and
promotion.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-17 15:19:53 +01:00
Robert Baldyga
fba384d23f
Merge pull request #815 from Kamoppl/kamilg/add_checkpatch
github-actions: add checkpatch
2025-03-17 08:16:48 +01:00
Robert Baldyga
bddc18f891
Merge pull request #862 from mmichal10/sanitizers
Enable compiling pyocf with sanitizers
2025-03-17 08:16:06 +01:00
Krzysztof Majzerowicz-Jaszcz
23abad76a8 Add script for running PyOCF with sanitization
GCC/Clang sanitizer can be used together with PyOCF to catch some errors during
testing.

CC was purposely removed from the Makefile. It always points to GCC on Linux
by default. This allows to change the compiler and its options during the run
of the script

Signed-off-by: Krzysztof Majzerowicz-Jaszcz <krzysztof.majzerowicz-jaszcz@intel.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-14 16:54:30 +01:00
Michal Mielewczyk
8dc462e08e Fix using request after free
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-14 13:36:55 +01:00
Michal Mielewczyk
495c67e74e posix env: Implement missing atomic operations
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-03-14 13:36:55 +01:00
Robert Baldyga
a2272266f6
Merge pull request #864 from Open-CAS/fix_detached_check
Check if cache is attached before setting promotion and seqcutoff params
2025-03-12 11:05:40 +01:00
Rafal Stefanowski
794fa125e8 Check if cache is attached before setting promotion and seqcutoff params
Fixes null pointer dereference if setting those params is attempted
on detached cache.

Signed-off-by: Rafal Stefanowski <rafal.stefanowski@huawei.com>
2025-03-11 16:47:53 +01:00
Robert Baldyga
a12496320c
Merge pull request #849 from mmichal10/refcnt-per-cpu
Refcnt per cpu
2025-02-06 17:28:07 +01:00
Robert Baldyga
bb4d67397c
Merge pull request #861 from Deixx/deinit-variable-metadata
Split count_pages and update on metadata deinit
2025-02-06 16:06:33 +01:00
Roel Apfelbaum
b02481cf74 A utility to continue pipeline on zero refcnt
Signed-off-by: Roel Apfelbaum <roel.apfelbaum@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-02-06 12:04:40 +01:00
Robert Baldyga
7685b70810 Protect cache lock against locking during lock deinitialization
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2025-02-06 12:04:37 +01:00
Adam Rutkowski
53ee7c1d3a Per-cpu refcounters
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Signed-off-by: Jan Musial <jan.musial@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
Signed-off-by: Rafal Stefanowski <rafal.stefanowski@huawei.com>
2025-02-06 12:04:34 +01:00
Daniel Madej
d8feef301d Change type of pages count variables to uint32_t
Signed-off-by: Daniel Madej <daniel.madej@huawei.com>
2025-02-06 11:54:53 +01:00
Daniel Madej
a15003d43e Split count_pages and update on metadata deinit
This resets count_pages_variable on cache-detach, so during the following
cache-attach metadata size is calculated properly.

Signed-off-by: Daniel Madej <daniel.madej@huawei.com>
2025-02-06 11:54:53 +01:00
Robert Baldyga
ba4b81a94c
Merge pull request #863 from Deixx/detach-zero-superblock
Zero metadata superblock on detach
2025-02-06 11:22:55 +01:00
Daniel Madej
3263503de1 pyocf: Add test for re-attaching device w/o force
Signed-off-by: Daniel Madej <daniel.madej@huawei.com>
2025-02-06 09:24:40 +01:00