github/ocf
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,830 Commits 1 Branch 1 Tag 4.1 MiB
9b7f476470
Commit Graph

466 Commits

Author SHA1 Message Date
Adam Rutkowski
e2ea2b41e6 pyocf: extend ErrorDevice to inject error on flush and discard 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-29 12:30:21 +02:00
Robert Baldyga
7cf58bb326
Merge pull request #733 from pdebski21/composite_volume_test2 
Composite volume tests load and attach
 2022-06-27 13:40:40 +02:00
Piotr Debski
f4eb291235 pyocf: Composite volume tests 
Signed-off-by: Piotr Debski <piotr.debski@intel.com>
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-22 17:31:46 +02:00
Robert Baldyga
641649f6a9 pyocf: Refactor _cvol_io() function 
Signed-off-by: Piotr Debski <piotr.debski@intel.com>
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-22 17:31:27 +02:00
Jan Musial
b39161f1aa pyocf: implement test_attach_cleaner_disabled_non_default 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-22 14:02:14 +02:00
Jan Musial
b898f5c336 pyocf: implement test_cleaner_disabled_nop 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-22 14:02:14 +02:00
Jan Musial
fb83a182ee pyocf: API for getting cleaning policy from cache 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-22 14:02:14 +02:00
Jan Musial
2f289bd873 pyocf: update OcfErrorCode with new code 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-22 14:02:14 +02:00
Robert Baldyga
9a22ef0917
Merge pull request #736 from pdebski21/tests_disable_cleaner2 
Tests disable cleaner
 2022-06-22 13:56:43 +02:00
Robert Baldyga
ef4bfc9ac3 pyocf: Make cache lookup by name a common utility 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-22 12:01:54 +02:00
Piotr Debski
5441cdb50a Cache API changes for disable_cleaner option 
Signed-off-by: Piotr Debski <piotr.debski@intel.com>
 2022-06-22 12:01:54 +02:00
Piotr Debski
b4f2866155 Tests cleaner disabled 
Signed-off-by: Piotr Debski <piotr.debski@intel.com>
 2022-06-22 12:01:54 +02:00
Robert Baldyga
597eb0c24c pyocf: Metadata volatile tests 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-22 09:03:38 +02:00
Adam Rutkowski
ae735b4434 pyocf: composite volume tests 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
96c5ef1d01 pyocf: register composite volume type id in ctx 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
8f2fac399e pyocf: composite volume class 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
0f8183157d pyocf: rename ExpObjVolume to OcfInternalVolume 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
6195967483 pyocf: use device.handle for device config if available 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
eb3439666d pyocf: destroy ocf_volume_t after attaching cache device 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
2976c77c4c pyocf: allow no queue in I/O allocation 
.. this is useful for testing I/O API without any cache

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
bd20236444 pyocf: add volume::close() 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
6c9f558b62 pyocf: manage volume.opened in s_open 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
8e1fe9a263 pyocf: generic open() for exported object volumes 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
b4dd5dac00 pyocf: methods to get ocf_volume_t from cache/core exp obj vol 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
db2e4ecb68 pyocf: rename static Volume::open() to s_open() 
.. to distinguish from exported object open() instance method

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
59d1905139 pyocf: function to register internal volume type id in ctx 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
6b9719907a pyocf: helper to get composite volume type id 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
03dbbd4471 pyocf: fix discard I/O tracing 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:08 +02:00
Adam Rutkowski
19dff9d4a2 pyocf: fix standby I/O test 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:08 +02:00
Jan Musial
3f777ea1cf pyocf: extend metadata helpers 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-10 12:47:48 +02:00
Jan Musial
6193530415 pyocf: rename C collision helpers file 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-10 12:47:48 +02:00
Jan Musial
97942c6277 pyocf: Helper for finding metadata sections in log 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-10 12:47:48 +02:00
Adam Rutkowski
b053f7925a
Merge pull request #702 from robertbaldyga/v22.6-composite-volume 
Introduce composite volume
 2022-06-02 13:36:21 +02:00
Adam Rutkowski
5f767dd618
Merge pull request #726 from arutk/fipm 
flush handling fixes and enhanced tests
 2022-06-02 10:46:36 +02:00
Adam Rutkowski
fcfdd2692e [pyocf] add missing volume registration in tests 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-02 09:49:39 +02:00
Adam Rutkowski
9739a442b6 [pyocf] properly resolve volume type 
OCF volume type is different for each context in which given
type (Volume subclass) is registered. So OCF volume type should
not be a Volume subclass member.

Adding map in Context class to store OCF volume type for each
registered volume type.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-02 09:49:39 +02:00
Robert Baldyga
6a9436740b Reflect cache attach API changes in pyocf 
The uuid/volume_type pair has been replaced with pointer to volume object.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-02 09:49:39 +02:00
Robert Baldyga
5012dcd7d5 pyocf: Add test designs for composite volume 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-02 09:49:39 +02:00
Robert Baldyga
16e824affd pyocf: Add on_init/on_deinit to VolumeOps 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-02 09:49:39 +02:00
Jan Musial
67024ebdb3 pyocf: Add argtype/restype for ocf_volume_create 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-02 09:49:39 +02:00
Robert Baldyga
15cef7aea9 pyocf: Add missing volume open parameter 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-02 09:49:39 +02:00
Michał Mielewczyk
71e056e662
Merge pull request #723 from jfckm/standby-io-tests 
Tests for standby I/O
 2022-06-02 09:17:43 +02:00
Adam Rutkowski
28f99ad7a5 pyocf: flush engine logic test 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-01 22:35:21 +02:00
Adam Rutkowski
f0b8815429 pyocf: reintroduce trace device 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-01 22:35:17 +02:00
Adam Rutkowski
ad2a583f43 pyocf: test for large I/O 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-01 22:33:35 +02:00
Adam Rutkowski
8e16a26b6a pyocf: add volume submit_flush() and submit_discard() 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-01 22:33:35 +02:00
Adam Rutkowski
07c7db81f4 pyocf: add FLUSH flag 
Flush I/O must be recognized by the bottom adapter by inspecting
adapter specific flags

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-01 22:33:35 +02:00
Adam Rutkowski
9646df431f
Merge pull request #712 from arutk/black 
pyocf: format all .py files with black -l 100
 2022-05-19 16:42:24 +02:00
Adam Rutkowski
8c220bf3f5 pyocf: standby io tests 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-05-18 15:40:55 +02:00
Jan Musial
3b7d25737b pyocf: Various improvements in RIO 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-05-18 15:40:55 +02:00
Jan Musial
438b6d158e pyocf: Delete Io object in pyocf before putting in OCF 
To avoid Io reuse (new io with same pointer could be allocated before
deleting from pyocf instance tracking dictionary).

Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-05-18 15:40:55 +02:00
Jan Musiał
0357e49896
Merge pull request #716 from jfckm/alock-count-locks 
Extend alock API with entries_count method
 2022-05-18 15:33:32 +02:00
Adam Rutkowski
83bb7317bf pyocf: format all .py files with black -l 100 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 16:44:19 +02:00
Adam Rutkowski
3a1b6fd718
Merge pull request #695 from arutk/failover_test_5 
pyocf: failover functional and power failure recovery tests
 2022-05-16 16:37:45 +02:00
Jan Musial
60a6da7ee6 Extend alock API with entries_count method 
Right now alock assumes that number of locks taken will equal number of
core lines. This is not the case in pio, where only parts of metadata
are under locks. If pio request overlaps locked and not-locked metadata
section it will have different core lines number and awaited locks
number. To remedy this discrepancy additional method which gets count of
locks that will be taken/waited on is added to alock API.

Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-05-16 16:21:08 +02:00
Adam Rutkowski
2721378942 failover tests 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:42:17 +02:00
Adam Rutkowski
fcfbd860c4 pyocf: fix error volume stats reset 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
a504821302 pyocf: separate get_conf() from stats in Cache 
Configuration parameters are available in standby mode, while
stats are not. Need to separate the two.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
3d83e1f004 pyocf: more generic erorr device volume 
Make ErrorDevice a wraper over any type of volume rather than
inheriting from RamVolume. This way error injection layer can be added
over any type of volume.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
d8e519855b pyocf: more options to standby load 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
b78c36ccdd pyocf: add missing import Volume in cache 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
6a03145acf pyocf: cache/core object volume fixes 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
2dde108cd9 pyocf: collision addr helper 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
978620f9e3 pyocf: move C wrappers to newly added "c" diretory 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
f66eefb3bd pyocf: remember queue name 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
7c186f5d0e pyocf: make queues a required rio param 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
c37ffea4c2 pyocf: add from_page() method to Size class 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:16 +02:00
Robert Baldyga
ed012411d3 Add disable_cleaner field to CacheAttachConfig 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-04-28 13:04:27 +02:00
Robert Baldyga
94aca1e8e4 pyocf: Remove non-existing field from CacheAttachConfig 
This field has been moved to CacheDeviceConfig.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-04-28 13:04:27 +02:00
Robert Baldyga
761ff2f053 pyocf: Add test designs for disable_cleaner option 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-04-28 13:04:27 +02:00
Adam Rutkowski
650511df4e pyocf: remove Queue objects from Cache::start() params 
... since a started cache is required to construct a queue object.
Instead add params to control whether to construct and register
management and default I/O queue in start().

Additionally introducing Cache method to register additional I/O
queues.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-04-13 10:38:57 +02:00
Adam Rutkowski
1f9c3dd24f pyocf: add test fixture with 2 ctxts 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-04-13 10:38:56 +02:00
Michal Mielewczyk
09d2843468 pyocf: minor security test fixes 
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
 2022-03-31 10:01:05 +02:00
Robert Baldyga
a64fc61413
Merge pull request #666 from arutk/pyocf_vol_cleaniup 
pyocf: volume changes required for failover tests
 2022-03-29 10:02:01 +02:00
Robert Baldyga
cf2d4327a1
Merge pull request #667 from arutk/pyocf_failover_api 
pyocf: failover API in Cache
 2022-03-29 09:32:00 +02:00
Robert Baldyga
af43a240d3 Return more specific error on CRC mismatch 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-03-28 22:42:59 +02:00
Adam Rutkowski
5c39af81ca pyocf: failover api 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 16:24:43 +02:00
Robert Baldyga
223ba0b025
Merge pull request #678 from arutk/pes 
Verify standby/active cache state in OCF entry points
 2022-03-28 12:07:06 +02:00
Adam Rutkowski
0e8d02235a pyocf: Move core/cache md5() logic to volume 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:37:11 +02:00
Adam Rutkowski
16c85c1560 Remove ocf_core_new_io 
This function is equivalent to calling ocf_volume_new_io()
on core front volume

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:36:19 +02:00
Adam Rutkowski
05f5d49be9 switch to volume I/O interface in functional tests 
... instead of core::new_io()

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:36:19 +02:00
Adam Rutkowski
9e88afad62 pyocf: cache and core exported object volumes 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:35:18 +02:00
Adam Rutkowski
d2164e4ffd pyocf: replicated volume 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
dcf3129ff8 Extract Volume::open() class method 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
d60e48ee26 pyocf: ocf_volume_new_io wrapper 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
e0c762387c pyocf: implement get_(front_)volume() in Cache and Core 
This common interface is later going to be used for generic code
implementing cache/core exported object I/O.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
2672f5460a pyocf: unique volume ops for each volume class 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
7cb9523d8c pyocf: volume offlining 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
33b134242b pyocf: remove trace volume 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
838870fa10 pyocf: Use abstract Volume in generic code paths 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
9e13364896 pyocf: Extract generic volume logic from RamVolume 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
16f9d58f28 pyocf: Rename Volume to RamVolume 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
062f63e4ff pyocf: Volume not to inherit from ctypes.Structure 
Volume object is never referenced from C code, so there
is no need to inherit from ctypes.Structure.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
9975706488 pyocf: Move poison out of Volume class 
.. so that abstract Volume does not need to be imported to access
the poison value.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
232133302b pyocf: use generic volume submit in io class 
.. instead of submit_core(). This will enable to reuse Io class
for cache exported object I/O.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
2de7b9d8f2 pyocf: submit_flush() and submit_dicard() 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Adam Rutkowski
efd6389e74 pyocf: add get_default_queue to core object 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:29:02 +02:00
Robert Baldyga
2cba3f4473
Merge pull request #673 from jfckm/settle-in-tests 
Settle in start_stop tests before checking stats
 2022-03-28 09:47:18 +02:00
Adam Rutkowski
4a839cd332 Verify standby/active cache state in OCF entry points 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 09:42:02 +02:00
Robert Baldyga
aa4622cc01 Make core remove error recoverable 
First try to clean only the mapping. This operation does not require any
rollback, so even if flushing collision fails, core object is still
intact. In case of error we inform user that core was not removed by
returning new error code (-OCF_ERR_CORE_NOT_REMOVED).

After flushing collision succeeds we remove core from metadata and
flush superblock at the end. At that point the core is fully removed
from OCF and even if superblock flush error occurs there is nothing we
can do about it, so we just return the error code.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-03-25 21:32:55 +01:00