Commit Graph

180 Commits

Author SHA1 Message Date
Adam Rutkowski
f0b8815429 pyocf: reintroduce trace device
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-06-01 22:35:17 +02:00
Adam Rutkowski
8e16a26b6a pyocf: add volume submit_flush() and submit_discard()
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-06-01 22:33:35 +02:00
Adam Rutkowski
07c7db81f4 pyocf: add FLUSH flag
Flush I/O must be recognized by the bottom adapter by inspecting
adapter specific flags

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-06-01 22:33:35 +02:00
Adam Rutkowski
83bb7317bf pyocf: format all .py files with black -l 100
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 16:44:19 +02:00
Adam Rutkowski
3a1b6fd718
Merge pull request #695 from arutk/failover_test_5
pyocf: failover functional and power failure recovery tests
2022-05-16 16:37:45 +02:00
Adam Rutkowski
fcfbd860c4 pyocf: fix error volume stats reset
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
a504821302 pyocf: separate get_conf() from stats in Cache
Configuration parameters are available in standby mode, while
stats are not. Need to separate the two.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
3d83e1f004 pyocf: more generic erorr device volume
Make ErrorDevice a wraper over any type of volume rather than
inheriting from RamVolume. This way error injection layer can be added
over any type of volume.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
d8e519855b pyocf: more options to standby load
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
b78c36ccdd pyocf: add missing import Volume in cache
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
6a03145acf pyocf: cache/core object volume fixes
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
2dde108cd9 pyocf: collision addr helper
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
978620f9e3 pyocf: move C wrappers to newly added "c" diretory
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
f66eefb3bd pyocf: remember queue name
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
7c186f5d0e pyocf: make queues a required rio param
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:18 +02:00
Adam Rutkowski
c37ffea4c2 pyocf: add from_page() method to Size class
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-05-16 15:40:16 +02:00
Robert Baldyga
ed012411d3 Add disable_cleaner field to CacheAttachConfig
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-04-28 13:04:27 +02:00
Robert Baldyga
94aca1e8e4 pyocf: Remove non-existing field from CacheAttachConfig
This field has been moved to CacheDeviceConfig.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-04-28 13:04:27 +02:00
Adam Rutkowski
650511df4e pyocf: remove Queue objects from Cache::start() params
... since a started cache is required to construct a queue object.
Instead add params to control whether to construct and register
management and default I/O queue in start().

Additionally introducing Cache method to register additional I/O
queues.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-04-13 10:38:57 +02:00
Robert Baldyga
a64fc61413
Merge pull request #666 from arutk/pyocf_vol_cleaniup
pyocf: volume changes required for failover tests
2022-03-29 10:02:01 +02:00
Robert Baldyga
cf2d4327a1
Merge pull request #667 from arutk/pyocf_failover_api
pyocf: failover API in Cache
2022-03-29 09:32:00 +02:00
Robert Baldyga
af43a240d3 Return more specific error on CRC mismatch
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-03-28 22:42:59 +02:00
Adam Rutkowski
5c39af81ca pyocf: failover api
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 16:24:43 +02:00
Adam Rutkowski
0e8d02235a pyocf: Move core/cache md5() logic to volume
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:37:11 +02:00
Adam Rutkowski
16c85c1560 Remove ocf_core_new_io
This function is equivalent to calling ocf_volume_new_io()
on core front volume

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:36:19 +02:00
Adam Rutkowski
9e88afad62 pyocf: cache and core exported object volumes
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:35:18 +02:00
Adam Rutkowski
d2164e4ffd pyocf: replicated volume
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
dcf3129ff8 Extract Volume::open() class method
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
d60e48ee26 pyocf: ocf_volume_new_io wrapper
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
e0c762387c pyocf: implement get_(front_)volume() in Cache and Core
This common interface is later going to be used for generic code
implementing cache/core exported object I/O.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
2672f5460a pyocf: unique volume ops for each volume class
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
7cb9523d8c pyocf: volume offlining
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
33b134242b pyocf: remove trace volume
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
838870fa10 pyocf: Use abstract Volume in generic code paths
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
9e13364896 pyocf: Extract generic volume logic from RamVolume
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
16f9d58f28 pyocf: Rename Volume to RamVolume
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
062f63e4ff pyocf: Volume not to inherit from ctypes.Structure
Volume object is never referenced from C code, so there
is no need to inherit from ctypes.Structure.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
9975706488 pyocf: Move poison out of Volume class
.. so that abstract Volume does not need to be imported to access
the poison value.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
232133302b pyocf: use generic volume submit in io class
.. instead of submit_core(). This will enable to reuse Io class
for cache exported object I/O.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
2de7b9d8f2 pyocf: submit_flush() and submit_dicard()
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
efd6389e74 pyocf: add get_default_queue to core object
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 10:29:02 +02:00
Adam Rutkowski
4a839cd332 Verify standby/active cache state in OCF entry points
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-03-28 09:42:02 +02:00
Robert Baldyga
aa4622cc01 Make core remove error recoverable
First try to clean only the mapping. This operation does not require any
rollback, so even if flushing collision fails, core object is still
intact. In case of error we inform user that core was not removed by
returning new error code (-OCF_ERR_CORE_NOT_REMOVED).

After flushing collision succeeds we remove core from metadata and
flush superblock at the end. At that point the core is fully removed
from OCF and even if superblock flush error occurs there is nothing we
can do about it, so we just return the error code.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-03-25 21:32:55 +01:00
Robert Baldyga
d46845b97a Remove unused error code
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-03-09 11:43:35 +01:00
Robert Baldyga
d5b2c65a39 Remove "metadata_layout" parameter of the cache
This feature is replaced with LRU list shuffling.

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-03-07 17:48:25 +01:00
Robert Baldyga
765d1c5d77 rio: fix size comparison in
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2022-02-24 15:02:46 +01:00
Adam Rutkowski
a6d8bd0470 rio: accept queue list in Rio constructor
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Signed-off-by: Jan Musial <jan.musial@intel.com>
2022-02-24 14:56:09 +01:00
Adam Rutkowski
372b1f1e1c remove unnecessary callback decorator in rio
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-02-24 14:56:09 +01:00
Adam Rutkowski
9baf9a3876 rio: convert Size to bytes for extent calculation
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-02-24 14:56:09 +01:00
Adam Rutkowski
17eddab094 pyocf: queue settle functionality
Queue settle is a mechanism to wait for all OCF processing
on a given queue to finish.

In some tests simply waiting for I/O to finish is not enough. Most
notably some statistics are potentially incremented after user triggered
I/O is finished. This is due to asynchronous nature of I/O operations
and OCF approach to statistics update, where only eventual consistency
is guaranteed without explicit mechanism available to query whether
the final state is reached yet. However it is fully in the adapter power
to determine this, as OCF executes in context of API calls from the
adapter (like I/O submission, ocf_queue_run, ocf_cleaner_run, management
operations) and I/O completion callbacks. Queue settle is a mechanism to
assure ocf_queue_run is not being executed by the thread associated with
a queue.

With queue settle mechanism it is straightforward for the adapter to
wait for cache statistics to reach fixed values:
1. wait for all I/O to OCF to finish
2. settle all I/O queues
3. make sure background cleaner is not active

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2022-02-24 14:56:09 +01:00