Volume object is never referenced from C code, so there
is no need to inherit from ctypes.Structure.
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
.. instead of submit_core(). This will enable to reuse Io class
for cache exported object I/O.
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
First try to clean only the mapping. This operation does not require any
rollback, so even if flushing collision fails, core object is still
intact. In case of error we inform user that core was not removed by
returning new error code (-OCF_ERR_CORE_NOT_REMOVED).
After flushing collision succeeds we remove core from metadata and
flush superblock at the end. At that point the core is fully removed
from OCF and even if superblock flush error occurs there is nothing we
can do about it, so we just return the error code.
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
Queue settle is a mechanism to wait for all OCF processing
on a given queue to finish.
In some tests simply waiting for I/O to finish is not enough. Most
notably some statistics are potentially incremented after user triggered
I/O is finished. This is due to asynchronous nature of I/O operations
and OCF approach to statistics update, where only eventual consistency
is guaranteed without explicit mechanism available to query whether
the final state is reached yet. However it is fully in the adapter power
to determine this, as OCF executes in context of API calls from the
adapter (like I/O submission, ocf_queue_run, ocf_cleaner_run, management
operations) and I/O completion callbacks. Queue settle is a mechanism to
assure ocf_queue_run is not being executed by the thread associated with
a queue.
With queue settle mechanism it is straightforward for the adapter to
wait for cache statistics to reach fixed values:
1. wait for all I/O to OCF to finish
2. settle all I/O queues
3. make sure background cleaner is not active
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
rio stands for Rigid IO tester and is a simple mechanism for testing
OCF cache IO.
Signed-off-by: Jan Musial <jan.musial@intel.com>
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
... this is useful to workaround current pyocf limitations and
load cache with manual core insertion
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Adding option to
1. inject error based on I/O number
2. arm/disarm error injection for easier testing
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Reformat function that calculates how long cache/core is dirty
Update `dirty_for` types in functional tests
Values stored in info structs fields (both in cache and core structs)
are unsigned 64-bits ints but `dirty_for`s were unsigned 32-bits ints.
Use existing function to transform returned value to seconds.
Replace seconds stored in metadata with seconds.
Replacement was done if old value of replaced field was equal to zero.
Acquiring monotonic high precission timestamp is potentially
slow and it makes sense to compare the field's value
to zero before calling atomic function.
Signed-off-by: Slawomir Jankowski <slawomir.jankowski@intel.com>
Change 'OCF_ERR_START_CACHE_FAIL' to 'OCF_ERR_NO_MEM' while CAS fails in case of memory lack on device.
Add new error code for case, when device doesn't satisfy CAS requirements - 'OCF_ERR_INVAL_CACHE_DEV'.
Use 'OCF_ERR_INVAL_CACHE_DEV' in code.
Update error code match in test.
closes#317 issue
Signed-off-by: Ostrokrzew <slawomir.jankowski@intel.com>
pointer type to array which is 32 characters long;
**core.py**: Add missing import and modify class' field type
to keep consistency;
**ocf_mngt_core**: Remove local variable 'name';
remove env_vmalloc for 'name' - isn't no longer needed;
remove initialization 'name' - as above;
remove env_vfree for context->cfg.name - variable isn't no allocated
in memory;
check if cfg->name exists;
change label in goto from deleted err_name to the closest err_pipeline.
Signed-off-by: Slawomir_Jankowski <slawomir.jankowski@intel.com>
It allows to modify and retrieve particular PP params event if it isn't active
and store values between cache stop and load.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
Promotion policy is supposed to perform ALRU noise filtering by
eliminating one-hit wonders being added to cache and polluting it.
Signed-off-by: Jan Musial <jan.musial@intel.com>
Flush/load metadata paths are heavily dependend on Data behaving
correctly in terms of seeks/position and that needed to be fixed.
Signed-off-by: Jan Musial <jan.musial@intel.com>
ocf_kick_cleaner() allows to perfom cleaning immediately.
Nop cleaning policy now returns new 'OCF_CLEANER_DISABLE' macro which indicates
that cleaing shouldn't be performed. To enable it back, ocf_kick_cleaner()
should be called.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
As non-interruptible flushes are no longer triggered from OCF
internals, we can get rid of "interruption" argument and let
adapters handle it themselves.
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
Queue is stopped when reference counter drops to 0, so no need to
manually stop management queue after cache stop - simple put is
enough.
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
Tests for starting cache and checking if mode works properly
Tests for stopping cache
Negative and stress tests
Signed-off-by: Daniel Madej <daniel.madej@intel.com>
* Add references to Cores in Cache, ctx holds caches, caches hold cores,
everything gets cleaned up nicely
* GC in Python seems to be a bit lazy, if we want to run CI on
low-memory machines we need to make sure it does run in between tests
'cause we don't want no huge Volumes hanging around for long
Signed-off-by: Jan Musial <jan.musial@intel.com>
Implement simple secure erase tests. Perform IO that will trigger
copying of Data buffers and make sure OCF calls secure erase on them.
Signed-off-by: Jan Musial <jan.musial@intel.com>
Due the aggresive security checks in compiler 'printf' might be substituded with
'__printf_chk'. However it does not differentiate whether substituted string is
library function call whether field in structure.
By renaming field we prevent it to be unintentionally subustituted by the
preprocessor.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
Fix some instance tracking to prevent Python-side objects from leaking
buffers.
Also, reduce min size of Cache instance (real minimum should be around ~~19MiB,
but we need to make it more deterministic and 20 MiB seems to be reasonable).
Still some stuff left to do, but it needs more investigation and, for
now, this should suffice just to enable some form of CI.
Signed-off-by: Jan Musial <jan.musial@intel.com>
PyOCF is a tool written with testing OCF functionality in mind.
It is a Python3 (3.6 version required) package which wraps OCF
by providing Python objects in place of OCF objects (volumes, queues,
etc). Thin layer of translation between OCF objects and PyOCF objects
enables using customized behaviors for OCF primitives by subclassing
PyOCF classes.
This initial version implements only WT and WI modes and single,
synchronously operating Queue.
TO DO:
- Queues/Cleaner/MetadataUpdater implemented as Python threads
- Loading of caches from PyOCF Volumes (fix bugs in OCF)
- Make sure it works multi-threaded for more sophisticated tests
Co-authored-by: Jan Musial <jan.musial@intel.com>
Signed-off-by: Michal Rakowski <michal.rakowski@intel.com>
Signed-off-by: Jan Musial <jan.musial@intel.com>
