github/ocf
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,939 Commits 1 Branch 1 Tag 4.1 MiB
62307d82b8
Commit Graph

484 Commits

Author SHA1 Message Date
Robert Baldyga
675cd11f40 Add missing license headers 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-07-22 08:33:55 +02:00
Jan Musial
f1a8bac1db pyocf: Improve logs readability for corruption tests 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
159fda9356 Security test for metadata corruption in standby activate 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
e8b46f09ee Security tests for metadata corruption 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
01166ba55e Security test for cache exported object 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
b3bd778a78 pyocf: simplify volume open/close API 
Make Volumes usable by both pyocf and OCF with clear open/_open split
and clean-up of instance/uuid tracking on C interface only.

Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
b1b3e134cf pyocf: clean-up hanging device reference 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
a0c6995189 pyocf: make open/close explicit 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
abc726d7f8 pyocf: warn about not-closed Volumes 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
4328fd77b0 pyocf: don't store UUIDs in Cache object 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
cfd8eea0c0 pyocf: fix cache device references 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
1bdf4a13ae pyocf: Improve volume instance lifecycle handling 
We need a strong reference to volume for as long as OCF has it open.
For that I changed instance tracking dictionary from weakvalue to
normal. This way I made sure that GC won't clean up Volume before its
closed.

Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
83a28825d2 pyocf: Load standby from device 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
d87bdc3a43 pyocf: Improve buffer logger to also print on console 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
2aaa69b33b pyocf: Add helper for superblock size 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Jan Musial
eac7d64456 pyocf: Parse size from string 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-07-19 10:50:11 +02:00
Adam Rutkowski
69ef673bd1 pyocf: composite volume tests 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-29 12:30:21 +02:00
Adam Rutkowski
81396681f4 pyocf: extend OcfCompletion 
1. add optional timeout and return value to wait()
2. add method to check whether completion was triggered

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-29 12:30:21 +02:00
Adam Rutkowski
e2ea2b41e6 pyocf: extend ErrorDevice to inject error on flush and discard 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-29 12:30:21 +02:00
Robert Baldyga
7cf58bb326
Merge pull request #733 from pdebski21/composite_volume_test2 
Composite volume tests load and attach
 2022-06-27 13:40:40 +02:00
Piotr Debski
f4eb291235 pyocf: Composite volume tests 
Signed-off-by: Piotr Debski <piotr.debski@intel.com>
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-22 17:31:46 +02:00
Robert Baldyga
641649f6a9 pyocf: Refactor _cvol_io() function 
Signed-off-by: Piotr Debski <piotr.debski@intel.com>
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-22 17:31:27 +02:00
Jan Musial
b39161f1aa pyocf: implement test_attach_cleaner_disabled_non_default 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-22 14:02:14 +02:00
Jan Musial
b898f5c336 pyocf: implement test_cleaner_disabled_nop 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-22 14:02:14 +02:00
Jan Musial
fb83a182ee pyocf: API for getting cleaning policy from cache 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-22 14:02:14 +02:00
Jan Musial
2f289bd873 pyocf: update OcfErrorCode with new code 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-22 14:02:14 +02:00
Robert Baldyga
9a22ef0917
Merge pull request #736 from pdebski21/tests_disable_cleaner2 
Tests disable cleaner
 2022-06-22 13:56:43 +02:00
Robert Baldyga
ef4bfc9ac3 pyocf: Make cache lookup by name a common utility 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-22 12:01:54 +02:00
Piotr Debski
5441cdb50a Cache API changes for disable_cleaner option 
Signed-off-by: Piotr Debski <piotr.debski@intel.com>
 2022-06-22 12:01:54 +02:00
Piotr Debski
b4f2866155 Tests cleaner disabled 
Signed-off-by: Piotr Debski <piotr.debski@intel.com>
 2022-06-22 12:01:54 +02:00
Robert Baldyga
597eb0c24c pyocf: Metadata volatile tests 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-22 09:03:38 +02:00
Adam Rutkowski
ae735b4434 pyocf: composite volume tests 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
96c5ef1d01 pyocf: register composite volume type id in ctx 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
8f2fac399e pyocf: composite volume class 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
0f8183157d pyocf: rename ExpObjVolume to OcfInternalVolume 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
6195967483 pyocf: use device.handle for device config if available 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
eb3439666d pyocf: destroy ocf_volume_t after attaching cache device 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
2976c77c4c pyocf: allow no queue in I/O allocation 
.. this is useful for testing I/O API without any cache

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
bd20236444 pyocf: add volume::close() 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
6c9f558b62 pyocf: manage volume.opened in s_open 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
8e1fe9a263 pyocf: generic open() for exported object volumes 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
b4dd5dac00 pyocf: methods to get ocf_volume_t from cache/core exp obj vol 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
db2e4ecb68 pyocf: rename static Volume::open() to s_open() 
.. to distinguish from exported object open() instance method

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
59d1905139 pyocf: function to register internal volume type id in ctx 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
6b9719907a pyocf: helper to get composite volume type id 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:41 +02:00
Adam Rutkowski
03dbbd4471 pyocf: fix discard I/O tracing 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:08 +02:00
Adam Rutkowski
19dff9d4a2 pyocf: fix standby I/O test 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-13 11:40:08 +02:00
Jan Musial
3f777ea1cf pyocf: extend metadata helpers 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-10 12:47:48 +02:00
Jan Musial
6193530415 pyocf: rename C collision helpers file 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-10 12:47:48 +02:00
Jan Musial
97942c6277 pyocf: Helper for finding metadata sections in log 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-10 12:47:48 +02:00
Adam Rutkowski
b053f7925a
Merge pull request #702 from robertbaldyga/v22.6-composite-volume 
Introduce composite volume
 2022-06-02 13:36:21 +02:00
Adam Rutkowski
5f767dd618
Merge pull request #726 from arutk/fipm 
flush handling fixes and enhanced tests
 2022-06-02 10:46:36 +02:00
Adam Rutkowski
fcfdd2692e [pyocf] add missing volume registration in tests 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-02 09:49:39 +02:00
Adam Rutkowski
9739a442b6 [pyocf] properly resolve volume type 
OCF volume type is different for each context in which given
type (Volume subclass) is registered. So OCF volume type should
not be a Volume subclass member.

Adding map in Context class to store OCF volume type for each
registered volume type.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-02 09:49:39 +02:00
Robert Baldyga
6a9436740b Reflect cache attach API changes in pyocf 
The uuid/volume_type pair has been replaced with pointer to volume object.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-02 09:49:39 +02:00
Robert Baldyga
5012dcd7d5 pyocf: Add test designs for composite volume 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-02 09:49:39 +02:00
Robert Baldyga
16e824affd pyocf: Add on_init/on_deinit to VolumeOps 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-02 09:49:39 +02:00
Jan Musial
67024ebdb3 pyocf: Add argtype/restype for ocf_volume_create 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-06-02 09:49:39 +02:00
Robert Baldyga
15cef7aea9 pyocf: Add missing volume open parameter 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-06-02 09:49:39 +02:00
Michał Mielewczyk
71e056e662
Merge pull request #723 from jfckm/standby-io-tests 
Tests for standby I/O
 2022-06-02 09:17:43 +02:00
Adam Rutkowski
28f99ad7a5 pyocf: flush engine logic test 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-01 22:35:21 +02:00
Adam Rutkowski
f0b8815429 pyocf: reintroduce trace device 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-01 22:35:17 +02:00
Adam Rutkowski
ad2a583f43 pyocf: test for large I/O 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-01 22:33:35 +02:00
Adam Rutkowski
8e16a26b6a pyocf: add volume submit_flush() and submit_discard() 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-01 22:33:35 +02:00
Adam Rutkowski
07c7db81f4 pyocf: add FLUSH flag 
Flush I/O must be recognized by the bottom adapter by inspecting
adapter specific flags

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-06-01 22:33:35 +02:00
Adam Rutkowski
9646df431f
Merge pull request #712 from arutk/black 
pyocf: format all .py files with black -l 100
 2022-05-19 16:42:24 +02:00
Adam Rutkowski
8c220bf3f5 pyocf: standby io tests 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-05-18 15:40:55 +02:00
Jan Musial
3b7d25737b pyocf: Various improvements in RIO 
Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-05-18 15:40:55 +02:00
Jan Musial
438b6d158e pyocf: Delete Io object in pyocf before putting in OCF 
To avoid Io reuse (new io with same pointer could be allocated before
deleting from pyocf instance tracking dictionary).

Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-05-18 15:40:55 +02:00
Jan Musiał
0357e49896
Merge pull request #716 from jfckm/alock-count-locks 
Extend alock API with entries_count method
 2022-05-18 15:33:32 +02:00
Adam Rutkowski
83bb7317bf pyocf: format all .py files with black -l 100 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 16:44:19 +02:00
Adam Rutkowski
3a1b6fd718
Merge pull request #695 from arutk/failover_test_5 
pyocf: failover functional and power failure recovery tests
 2022-05-16 16:37:45 +02:00
Jan Musial
60a6da7ee6 Extend alock API with entries_count method 
Right now alock assumes that number of locks taken will equal number of
core lines. This is not the case in pio, where only parts of metadata
are under locks. If pio request overlaps locked and not-locked metadata
section it will have different core lines number and awaited locks
number. To remedy this discrepancy additional method which gets count of
locks that will be taken/waited on is added to alock API.

Signed-off-by: Jan Musial <jan.musial@intel.com>
 2022-05-16 16:21:08 +02:00
Adam Rutkowski
2721378942 failover tests 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:42:17 +02:00
Adam Rutkowski
fcfbd860c4 pyocf: fix error volume stats reset 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
a504821302 pyocf: separate get_conf() from stats in Cache 
Configuration parameters are available in standby mode, while
stats are not. Need to separate the two.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
3d83e1f004 pyocf: more generic erorr device volume 
Make ErrorDevice a wraper over any type of volume rather than
inheriting from RamVolume. This way error injection layer can be added
over any type of volume.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
d8e519855b pyocf: more options to standby load 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
b78c36ccdd pyocf: add missing import Volume in cache 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
6a03145acf pyocf: cache/core object volume fixes 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
2dde108cd9 pyocf: collision addr helper 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
978620f9e3 pyocf: move C wrappers to newly added "c" diretory 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
f66eefb3bd pyocf: remember queue name 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
7c186f5d0e pyocf: make queues a required rio param 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:18 +02:00
Adam Rutkowski
c37ffea4c2 pyocf: add from_page() method to Size class 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-05-16 15:40:16 +02:00
Robert Baldyga
ed012411d3 Add disable_cleaner field to CacheAttachConfig 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-04-28 13:04:27 +02:00
Robert Baldyga
94aca1e8e4 pyocf: Remove non-existing field from CacheAttachConfig 
This field has been moved to CacheDeviceConfig.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-04-28 13:04:27 +02:00
Robert Baldyga
761ff2f053 pyocf: Add test designs for disable_cleaner option 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-04-28 13:04:27 +02:00
Adam Rutkowski
650511df4e pyocf: remove Queue objects from Cache::start() params 
... since a started cache is required to construct a queue object.
Instead add params to control whether to construct and register
management and default I/O queue in start().

Additionally introducing Cache method to register additional I/O
queues.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-04-13 10:38:57 +02:00
Adam Rutkowski
1f9c3dd24f pyocf: add test fixture with 2 ctxts 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-04-13 10:38:56 +02:00
Michal Mielewczyk
09d2843468 pyocf: minor security test fixes 
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
 2022-03-31 10:01:05 +02:00
Robert Baldyga
a64fc61413
Merge pull request #666 from arutk/pyocf_vol_cleaniup 
pyocf: volume changes required for failover tests
 2022-03-29 10:02:01 +02:00
Robert Baldyga
cf2d4327a1
Merge pull request #667 from arutk/pyocf_failover_api 
pyocf: failover API in Cache
 2022-03-29 09:32:00 +02:00
Robert Baldyga
af43a240d3 Return more specific error on CRC mismatch 
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
 2022-03-28 22:42:59 +02:00
Adam Rutkowski
5c39af81ca pyocf: failover api 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 16:24:43 +02:00
Robert Baldyga
223ba0b025
Merge pull request #678 from arutk/pes 
Verify standby/active cache state in OCF entry points
 2022-03-28 12:07:06 +02:00
Adam Rutkowski
0e8d02235a pyocf: Move core/cache md5() logic to volume 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:37:11 +02:00
Adam Rutkowski
16c85c1560 Remove ocf_core_new_io 
This function is equivalent to calling ocf_volume_new_io()
on core front volume

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:36:19 +02:00
Adam Rutkowski
05f5d49be9 switch to volume I/O interface in functional tests 
... instead of core::new_io()

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:36:19 +02:00
Adam Rutkowski
9e88afad62 pyocf: cache and core exported object volumes 
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
 2022-03-28 10:35:18 +02:00