Commit Graph

1637 Commits

Author SHA1 Message Date
Adam Rutkowski
0030ebdecc Handle already opened volume in volume open
Volumes are now exposed in OCF API and we should gracefully handle
attempt to open already opened volume (instead of ENV_BUG).

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-06-13 11:40:08 +02:00
Adam Rutkowski
a1e9303189
Merge pull request #730 from jfckm/metadata-helpers
pyocf: Extend metadata helpers to work for all metadata segments
2022-06-13 11:07:29 +02:00
Jan Musial
3f777ea1cf pyocf: extend metadata helpers
Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-06-10 12:47:48 +02:00
Jan Musial
6193530415 pyocf: rename C collision helpers file
Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-06-10 12:47:48 +02:00
Jan Musial
97942c6277 pyocf: Helper for finding metadata sections in log
Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-06-10 12:47:48 +02:00
Adam Rutkowski
b053f7925a
Merge pull request #702 from robertbaldyga/v22.6-composite-volume
Introduce composite volume
2022-06-02 13:36:21 +02:00
Adam Rutkowski
5f767dd618
Merge pull request #726 from arutk/fipm
flush handling fixes and enhanced tests
2022-06-02 10:46:36 +02:00
Adam Rutkowski
fcfdd2692e [pyocf] add missing volume registration in tests
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-06-02 09:49:39 +02:00
Adam Rutkowski
9739a442b6 [pyocf] properly resolve volume type
OCF volume type is different for each context in which given
type (Volume subclass) is registered. So OCF volume type should
not be a Volume subclass member.

Adding map in Context class to store OCF volume type for each
registered volume type.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-06-02 09:49:39 +02:00
Robert Baldyga
6a9436740b Reflect cache attach API changes in pyocf
The uuid/volume_type pair has been replaced with pointer to volume object.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-02 09:49:39 +02:00
Robert Baldyga
bc0e28f1c6 Reflect cache attach API changes in example
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-02 09:49:39 +02:00
Robert Baldyga
b847fa9a61 Introduce composite volume
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-02 09:49:39 +02:00
Robert Baldyga
5012dcd7d5 pyocf: Add test designs for composite volume
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-02 09:49:39 +02:00
Robert Baldyga
097bc989a9 doc: Add composite volume requirements documentation
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-02 09:49:39 +02:00
Robert Baldyga
8858e7344d Replace uuid/type pair with volume object in the device config
It makes it possible to attach/load cache using volume types that have
non-standard constructors.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-02 09:49:39 +02:00
Robert Baldyga
54b951fcdf Make default io allocators part of internal API
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-02 09:49:39 +02:00
Robert Baldyga
16e824affd pyocf: Add on_init/on_deinit to VolumeOps
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-02 09:49:39 +02:00
Robert Baldyga
c9ea68f3bf Introduce on_init/on_deinit ops in ocf_volume interface
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-02 09:49:39 +02:00
Jan Musial
67024ebdb3 pyocf: Add argtype/restype for ocf_volume_create
Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-06-02 09:49:39 +02:00
Robert Baldyga
15cef7aea9 pyocf: Add missing volume open parameter
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-02 09:49:39 +02:00
Robert Baldyga
af62d14f02 Set priv to NULL on volume deinit
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-02 09:49:39 +02:00
Robert Baldyga
70a410b2fe Improve error handling in ocf_volume_init()
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-06-02 09:49:39 +02:00
Michał Mielewczyk
71e056e662
Merge pull request #723 from jfckm/standby-io-tests
Tests for standby I/O
2022-06-02 09:17:43 +02:00
Adam Rutkowski
28f99ad7a5 pyocf: flush engine logic test
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-06-01 22:35:21 +02:00
Adam Rutkowski
f0b8815429 pyocf: reintroduce trace device
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-06-01 22:35:17 +02:00
Adam Rutkowski
ad2a583f43 pyocf: test for large I/O
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-06-01 22:33:35 +02:00
Adam Rutkowski
df7ed6920c Fix ops(flush) engine
Flush I/O should be forwarded to core and cache device. In case of core
this is simple - just mirror the I/O from the top volume. Since
cache data is owned by OCF it makes sense to send a simple flush I/O
with 0 address and size.

Current implementation attempts to use cache data I/O interface
(ocf_submit_cache_reqs function) instead of submitting empty flush to
the underlying cache device. This function is designed to read/write
from mapped cachelines while there is no traversation/mapping
performed on flush I/O.

If request map allocation succeeds, this results in sending I/O to
addres 0 with size and flags inherited from the top adapter I/O.
This doesn't make any sense, and can even result in invalid I/O if the
size is greater than cache device size.

Even worse, if flush request map allocation fails (which happens
always in case of large flush requests) then the erroneous call to
ocf_submit_cache_reqs results in NULL pointer dereference.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-06-01 22:33:35 +02:00
Adam Rutkowski
8e16a26b6a pyocf: add volume submit_flush() and submit_discard()
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-06-01 22:33:35 +02:00
Adam Rutkowski
07c7db81f4 pyocf: add FLUSH flag
Flush I/O must be recognized by the bottom adapter by inspecting
adapter specific flags

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-06-01 22:33:35 +02:00
Adam Rutkowski
1992bfc410
Merge pull request #710 from pdebski21/cache_line_size_mismatch
Explicit check for cacheline size mismatch during cache activation
2022-06-01 18:07:36 +02:00
Piotr Debski
0b9104e8d5 Cache metadata and superblock cache line size mismatch check
Signed-off-by: Piotr Debski <piotr.debski@intel.com>
2022-05-23 15:20:35 +02:00
Adam Rutkowski
9646df431f
Merge pull request #712 from arutk/black
pyocf: format all .py files with black -l 100
2022-05-19 16:42:24 +02:00
Adam Rutkowski
8c220bf3f5 pyocf: standby io tests
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-05-18 15:40:55 +02:00
Jan Musial
3b7d25737b pyocf: Various improvements in RIO
Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-05-18 15:40:55 +02:00
Jan Musial
438b6d158e pyocf: Delete Io object in pyocf before putting in OCF
To avoid Io reuse (new io with same pointer could be allocated before
deleting from pyocf instance tracking dictionary).

Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-05-18 15:40:55 +02:00
Jan Musiał
0357e49896
Merge pull request #716 from jfckm/alock-count-locks
Extend alock API with entries_count method
2022-05-18 15:33:32 +02:00
Jan Musial
6016a6f4c7 Mark unlikely branches in pio_concurrency
Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-05-18 11:56:06 +02:00
Adam Rutkowski
83bb7317bf pyocf: format all .py files with black -l 100
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 16:44:19 +02:00
Adam Rutkowski
3a1b6fd718
Merge pull request #695 from arutk/failover_test_5
pyocf: failover functional and power failure recovery tests
2022-05-16 16:37:45 +02:00
Jan Musial
60a6da7ee6 Extend alock API with entries_count method
Right now alock assumes that number of locks taken will equal number of
core lines. This is not the case in pio, where only parts of metadata
are under locks. If pio request overlaps locked and not-locked metadata
section it will have different core lines number and awaited locks
number. To remedy this discrepancy additional method which gets count of
locks that will be taken/waited on is added to alock API.

Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-05-16 16:21:08 +02:00
Adam Rutkowski
2721378942 failover tests
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:42:17 +02:00
Adam Rutkowski
fcfbd860c4 pyocf: fix error volume stats reset
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
a504821302 pyocf: separate get_conf() from stats in Cache
Configuration parameters are available in standby mode, while
stats are not. Need to separate the two.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
3d83e1f004 pyocf: more generic erorr device volume
Make ErrorDevice a wraper over any type of volume rather than
inheriting from RamVolume. This way error injection layer can be added
over any type of volume.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
d8e519855b pyocf: more options to standby load
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
b78c36ccdd pyocf: add missing import Volume in cache
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
6a03145acf pyocf: cache/core object volume fixes
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
ebb107f1a1 posix env: print call stack in case of BUG()
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
2dde108cd9 pyocf: collision addr helper
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
978620f9e3 pyocf: move C wrappers to newly added "c" diretory
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00