Commit Graph

1915 Commits

Author SHA1 Message Date
Robert Baldyga
1fbb00de8f
Merge pull request #811 from mmichal10/req-rewrq
Request improvements
2024-09-09 14:37:03 +02:00
Robert Baldyga
dc58eeae9b Introduce d2c request
This avoids unnecessary map allocation and initialization of unused fields of
request structure. It also allows to track thier number separately from
the regular requests

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-09 12:45:51 +02:00
Robert Baldyga
8b93b699c3 Eliminate queue -> cache mapping
Eliminate need to resolve cache based on the queue. This allows to share
the queue between cache instances. The queue still holds pointer to
a cache that owns the queue, but no management or io path relies on the
queue -> cache mapping.

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-09 12:45:51 +02:00
Robert Baldyga
460cd461d3 Allocate requests for management path separately
Management path does not benefit much from mpools, as number of requests
allocated is very small. It's less restrictive (mngt_queue does not have
single-CPU affinity) thus avoiding mpool usage in management path allows
to introduce additional restrictions on mpool, leading to I/O performance
improvement.

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-09 12:45:51 +02:00
Robert Baldyga
6cd5a27ea9
Merge pull request #812 from Kamoppl/kamilg/fix_tj_actions
github-actions: update checkout version
2024-09-09 12:39:31 +02:00
Kamil Gierszewski
0f60163a46
github-actions: update checkout version
Signed-off-by: Kamil Gierszewski <kamil.gierszewski@huawei.com>
2024-09-09 12:29:01 +02:00
Robert Baldyga
bfd28af1cc
Merge pull request #781 from Open-CAS/dependabot/github_actions/dot-github/workflows/tj-actions/changed-files-41
build(deps): bump tj-actions/changed-files from 18.2 to 41 in /.github/workflows
2024-09-09 11:19:04 +02:00
Robert Baldyga
193410e7b2
Merge pull request #810 from mmichal10/pt-stats
Introduce pass-through block stats
2024-09-06 15:33:23 +02:00
Sara Merzel
835eb708b5 Introduce pass-through block stats
Signed-off-by: Sara Merzel <sara.merzel@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-06 14:47:02 +02:00
Robert Baldyga
3ebf6e64c1
Merge pull request #808 from mmichal10/vol_and_req_fixes
Volume improvements
2024-09-06 14:24:48 +02:00
Robert Baldyga
5a1f6f1f8a
Merge pull request #809 from robertbaldyga/ram-needed-no-fail
Modify ocf_mngt_get_ram_needed to never fail
2024-09-06 14:20:24 +02:00
Amir Haroush
ed62866324 Modify ocf_mngt_get_ram_needed to never fail
Signed-off-by: Amir Haroush <amir.haroush@huawei.com>
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
2024-09-05 15:41:54 +02:00
Gershon Geva
2096e34489 Pass user's params when opening a core volume
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-04 20:29:41 +02:00
Robert Baldyga
87b16aef6a Do not deinit user volume
The user is supposed to deinit/destroy it.

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-03 16:10:56 +02:00
Robert Baldyga
3d99a2c938 Add missing ocf_volume_init() calls
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-03 16:10:56 +02:00
Robert Baldyga
8aa2d0fb63 Remove unused attach context property
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-03 16:10:56 +02:00
Robert Baldyga
01902e1206
Merge pull request #807 from mmichal10/parallelize-fixes
Parallelize fixes
2024-09-03 12:49:54 +02:00
Robert Baldyga
5e6a90a293 parallelize: Fix race condition
In situation when all the shards finish their work before parallelize
loop does it's final loop condition check, which involves access to
parallelize object, it's possible that parallelize object will be
deinitialized before this final access.

Increasing refcount by 1 before running parallelize and decreasing it
only after the loop is finished addresses this problem.

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-02 12:22:11 +02:00
Robert Baldyga
55b99518ed parallelize: Create number of shards requested by user
In some scenarios running the exact number of shards, regardless of
number of available queues is crucial for correctness of operation.

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-02 12:22:11 +02:00
Robert Baldyga
5c714cb3de parallelize: Use mngt_queue only as a fallback
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-02 12:22:11 +02:00
Robert Baldyga
83d5642547
Merge pull request #806 from mmichal10/deffered-miss
Count deferred requests as full miss
2024-09-02 12:18:31 +02:00
Michael Lyulko
470204ac70 Count deferred requests as full miss
Otherwise, it may increase the number of hits, while the overall performance
has not been improved. This way, the hit rate is more correlated with
the performance changes.

Signed-off-by: Michael Lyulko <michael.lyulko@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-09-02 09:21:28 +02:00
Robert Baldyga
35bf43b2e5
Merge pull request #805 from mmichal10/locking-fixes
Locking and queue improvements
2024-08-30 11:03:58 +02:00
Michal Mielewczyk
53756e81be Queue visitor API
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 10:45:57 +02:00
Robert Baldyga
b00ab08473 Introduce io_queues_lock
The queues can be created and destroyed dynamically at any point in
the cache lifetime, and this can happen from different execution contexts,
thus there is a need to protect the queue_list with a lock.

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 10:45:36 +02:00
Robert Baldyga
8db93260ae Avoid adding mngt_queue to io_queues list
Previously every created queue was added to io_queues list, which
made mngt_queue being used in ocf_parallelize. Change mngt_queue creation
API so that mngt_queue is not added to the list and doesn't have
unnecessary functionalities initialized.

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 10:45:26 +02:00
Michael Lyulko
1d903f4038 Fix debug stats compilation
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 10:16:26 +02:00
Michal Mielewczyk
0a9a173f33 Add missing flush_mutext destroy
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 10:15:08 +02:00
Michal Mielewczyk
2221a7bdf8 Add missing cache lock deinit
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 09:52:05 +02:00
Michal Mielewczyk
28f679cc91 Add missing cache unlock
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 09:52:05 +02:00
Michal Mielewczyk
a542cfa690 Refactor cache_trylock()
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 09:52:05 +02:00
Robert Baldyga
421d3f03ea
Merge pull request #804 from mmichal10/pytest-fixes
(Mostly) pyocf fixes
2024-08-29 09:36:58 +02:00
Michael Lyulko
59227999da pyocf: fix standby detach completion
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 08:39:55 +02:00
Robert Baldyga
640ee86663 pyocf: Introduce discard tests
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 08:39:55 +02:00
Roel Apfelbaum
9f0147ab1e pyocf: refactor sync io operations
Replace the pattern:
	completion = OcfCompletion([("err", c_int)])
	io.callback = completion.callback
	io.submit()
	completion.wait()
with:
	completion = Sync(io).submit()
Also, remove some redundant imports.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 08:39:53 +02:00
Robert Baldyga
8d4661bdc9 pyocf: rio: Introduce copy()
This allows to create a Rio prototype object and the copy it for every
invocations so that some parameters can be overwritten without modifying
the prototype object itself.

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 08:29:43 +02:00
Robert Baldyga
d892e3b0d5 pyocf: rio: Fix offset
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 08:29:42 +02:00
Michal Mielewczyk
16830cd3c3 pyocf: rio: Fix the default JobSpec
Make JobSpec dataclass' default values immutable.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 08:29:10 +02:00
Robert Baldyga
8668ae9c37 pyocf: Fix test_io_propagation_multiple_subvolumes
Make sure end_offset is never generated as 0. Otherwise the last volume
in the range will not get any io, leading false negative test result.

Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 08:28:13 +02:00
Michal Mielewczyk
1c7de189e2 Get rid of non-OCF error codes
Let's put an end to random crashes and vague error messages in pyocf!

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-29 08:27:51 +02:00
Robert Baldyga
476573bffa
Merge pull request #803 from mmichal10/fix-cleaner-req-refcnt
Fix cleaner req refcnt
2024-08-27 17:02:21 +02:00
Michal Mielewczyk
d883a40dbc cleaner: refactor completion function
The completion function should be the same either when it is called from
the queue context or from currnet context

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-27 15:20:02 +02:00
Michal Mielewczyk
736665af47 cleaner: Fix master request refcnt
Commit db6b009ef introduced changes in managing the master request life cycle,
but apparently not all paths have been updated. This change removes a redundant
ocf_req_get() before sending the requets into a queue

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-27 07:44:09 +02:00
Michał Mielewczyk
63f4a373ec
Merge pull request #802 from mmichal10/fix-cleaner-refcnt
cleaner: Fix tracking in-flight cache requests
2024-08-20 13:02:27 +02:00
Michal Mielewczyk
312b32ad61 cleaner: Fix tracking in-flight cache requests
When flushing a request, the number of cache reads is unknown until all cache
lines are locked and the IOs are actually submitted.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@huawei.com>
2024-08-19 22:10:45 +02:00
Robert Baldyga
5b2f26decf
Merge pull request #800 from robertbaldyga/redesign-queue-api
Redesign queue API
2024-08-02 14:43:52 +02:00
Ian Levine
ac1b6b774a Added a priority queue for the request instead of push front
Now the request can be pushed to a high priority queue (instead of ocf_queue_push_req_front)
and to a low priority queue (instead of ocf_queue_push_req_back).
Both functions were merged into one function (ocf_queue_push_req) and instead of the
allow_sync parameter there is now a flags parameter that can be an OR combination of
OCF_QUEUE_ALLOW_SYNC and OCF_QUEUE_PRIO_HIGH

Signed-off-by: Ian Levine <ian.levine@huawei.com>
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
2024-08-02 12:53:16 +02:00
Ian Levine
4f2d5c22d6 Move and rename ocf_engine_pop_req from cache_engine to ocf_queue_pop_req in ocf_queue
Signed-off-by: Ian Levine <ian.levine@huawei.com>
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
2024-08-02 12:53:16 +02:00
Ian Levine
038126e9ab Move and rename ocf_engine_push_req_* from engine_common to ocf_queue_push_req_* in ocf_queue
Signed-off-by: Ian Levine <ian.levine@huawei.com>
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
2024-08-02 12:53:16 +02:00
Ian Levine
de32a9649a Rename ocf_engine_cb to ocf_req_cb and move it from engine_common.h to ocf_request.h
Signed-off-by: Ian Levine <ian.levine@huawei.com>
Signed-off-by: Robert Baldyga <robert.baldyga@huawei.com>
2024-08-02 12:53:10 +02:00