Commit Graph

245 Commits

Author SHA1 Message Date
Robert Baldyga
2a1161e384 Update README.md
Add information about "Contibuting" page

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2019-05-07 09:16:36 +02:00
Michał Wysoczański
bb6fe41b9d
Merge pull request #136 from arutk/unplug_error_handling
Refactor cache mngt stop/detach for better error handling
2019-04-23 15:33:16 +02:00
Adam Rutkowski
26579e53cd Refactor cache mngt stop/detach for better error handling
_ocf_mngt_cache_unplug context is now provided by the caller.
This way _ocf_mngt_cache_unplug returns only non-critical (cache write)
errors, allowing stop/detach operation to always proceed and optionally
finish with error. This eliminates the need for rolling back previous
stop/detach operations, which might turn out to be impossible e.g.
under memory pressure.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-23 16:16:18 -04:00
Katarzyna Łapińska
bf879c4b09
Merge pull request #113 from Deixx/tests_start_stop
pyocf tests for start/stop
2019-04-23 12:10:17 +02:00
Michał Wysoczański
190812e84a
Merge pull request #135 from arutk/rename_env_ocf_env
Rename makefile ENV* variables to OCF_ENV*
2019-04-23 11:22:39 +02:00
Adam Rutkowski
e11edcacd8 Rename makefile ENV* variables to OCF_ENV*
'ENV' variable is set and used by OS utilities  on some Linux
distros, interfering with OCF build system. Renaming ENV and
other ENV* variables to OCF_ENV* to avoid conflicts.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-20 16:54:04 +02:00
Michał Wysoczański
4f2efb0ae7
Merge pull request #134 from Donaim/fix-scanbuild
Fix immediate reassign
2019-04-19 15:30:58 +02:00
Vitaliy Mysak
bf2242fade Fix immediate reassign
Don't reassign value of cache without any previus use.
It produced warnings when analyzing with scanbuild.

Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
2019-04-19 09:25:23 -04:00
Adam Rutkowski
c7df9161be Use secure alloc in _raw_dynamic_get_item
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-18 23:11:12 -04:00
Michal Rakowski
9f927de841
Merge pull request #132 from arutk/secure_alloc_dynamic
Use secure alloc in _raw_dynamic_get_item
2019-04-18 21:17:03 +02:00
Daniel Madej
1aa5e99173 Pyocf tests for start/stop
Tests for starting cache and checking if mode works properly
Tests for stopping cache
Negative and stress tests

Signed-off-by: Daniel Madej <daniel.madej@intel.com>
2019-04-18 17:00:14 +02:00
Michał Mielewczyk
d3bfbf6200
Merge pull request #130 from micrakow/kw-fix-ww16_19
Added missing return in metadata_probe_cores
2019-04-18 16:40:36 +02:00
Michal Rakowski
3d6b3ea2e8 Added missing return in metadata_probe_cores
Signed-off-by: Michal Rakowski <michal.rakowski@intel.com>
2019-04-18 16:35:05 +02:00
Michał Wysoczański
0b098ddb80
Merge pull request #128 from arutk/secure_alloc
Use env_secure_(alloc/free) macro for metadata allocations
2019-04-18 15:56:57 +02:00
Adam Rutkowski
cf24b46a58 posix env: evaluate ENV_BUG_ON condition unconditionally
OCF depends on ENV_BUG_ON condition to be evaluated as it
may have side effects. So simple implementation with
"assert(!cond)" is not good enough as it will likely be noop
in release build.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-18 17:54:05 -04:00
Adam Rutkowski
c5a80cc488 Use env_secure_(alloc/free) macro for metadata allocations
Adapter can opt to take additional steps to securely allocate
memory used by OCF to store cache metadata. Typically this would
involve mlocking pages and zeroing memory before deallocation.

Memory allocated using secure_alloc is not expected to be zeroed
or physically continous.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-18 17:54:05 -04:00
Adam Rutkowski
9528d1bf64 Add secure alloc/free to posix env
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-18 17:53:59 -04:00
Adam Rutkowski
79cb39e273
Merge pull request #129 from imjfckm/master
Fix cores volumes dropping before close
2019-04-18 09:41:33 -04:00
Jan Musial
be88300071 Fix cores volumes dropping before close
* Add references to Cores in Cache, ctx holds caches, caches hold cores,
  everything gets cleaned up nicely

* GC in Python seems to be a bit lazy, if we want to run CI on
  low-memory machines we need to make sure it does run in between tests
  'cause we don't want no huge Volumes hanging around for long

Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-04-18 14:11:59 +02:00
Daniel Madej
99c448aae6
Merge pull request #124 from katlapinka/security_config
Security tests for start cache command
2019-04-18 14:07:32 +02:00
Kamil Łepek
92837ec2f3
Merge pull request #119 from Deixx/tests_neg_io
Negative tests for IO path
2019-04-18 14:06:15 +02:00
klapinsk
f9eab4c749 Security tests for start cache command 2019-04-18 13:43:27 +02:00
Michal Rakowski
f66579ac8b
Merge pull request #123 from imjfckm/master
Secure erase tests
2019-04-18 13:43:12 +02:00
Daniel Madej
876294f603 Negative tests for IO path
Signed-off-by: Daniel Madej <daniel.madej@intel.com>
2019-04-18 12:58:02 +02:00
Jan Musial
75c3948f6d Secure erase tests
Implement simple secure erase tests. Perform IO that will trigger
copying of Data buffers and make sure OCF calls secure erase on them.

Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-04-18 12:51:59 +02:00
Michał Mielewczyk
d4e929140e
Merge pull request #127 from arutk/pytest_wait_io
pyocf: wait for pending I/O before cache stop
2019-04-18 09:21:50 +02:00
Michał Mielewczyk
2af7e7b4de
Merge pull request #125 from arutk/vol_ref_cnt
Wait for IO put in volume close
2019-04-18 09:02:58 +02:00
Katarzyna Łapińska
c80ca4a9a8
Merge pull request #122 from KamilLepek/security_tests
Add security fuzzy tests and improve pyocf api
2019-04-18 07:33:11 +02:00
Adam Rutkowski
1efb301462 Wait for IO put in volume close
Volume close should not close underlying device until all
I/O targeting this volume are deallocated. To achieve this
a reference counter is added to volume. Counter value
matches number of I/O objects associated with volume. Counter
is freezed when volume is closed, blocking allocation of new
I/O objects.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-17 23:12:04 -04:00
Adam Rutkowski
407470c25d Add reference counter init routine
This is useful when reference counter is initialized in non-zeroed
memory (or assuming atomic variable is not properly initialized by
memseting to zero).

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-17 23:12:04 -04:00
Adam Rutkowski
d607402f2b pyocf: wait for pending I/O before cache stop
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-17 18:12:56 -04:00
Adam Rutkowski
c8fbb99284
Merge pull request #126 from mmichal10/rename-printf
Rename 'printf' field name in logger ops to 'print'.
2019-04-17 15:24:23 -04:00
Michal Mielewczyk
5e52ac4ef2 Rename 'printf' field name in logger ops to 'print'.
Due the aggresive security checks in compiler 'printf' might be substituded with
'__printf_chk'. However it does not differentiate whether substituted string is
library function call whether field in structure.

By renaming field we prevent it to be unintentionally subustituted by the
preprocessor.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-04-17 11:37:47 -04:00
Michal Mielewczyk
ff079aa5b4 Initialize variable meant for keeping ocf_mngt_start_trace() result.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-04-17 11:28:05 -04:00
Jan Musiał
121b3233e4
Merge pull request #120 from arutk/core-query-api
Add API to query for core devices associated with cache
2019-04-17 13:33:04 +02:00
Adam Rutkowski
dfd89dd43a Probe core UUIDs from metadata
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-17 14:07:42 -04:00
Kamil Lepek
d55195ecf5 Add security fuzzy tests and improve pyocf api
Fuzzy tests added for:
changing cache mode
attaching cache device with fuzzed cache line size
setting cleaning policy
setting alru params
setting acp params
setting seq cut off policy per cache
setting seq cut off policy per core
2019-04-17 12:46:25 +02:00
Kamil Łepek
057777e91d
Merge pull request #121 from micrakow/seq_cutoff_policy_check
Added seq_cutoff_policy check
2019-04-16 15:02:25 +02:00
Michal Rakowski
4652b009d6 Added seq_cutoff_policy check
Signed-off-by: Michal Rakowski <michal.rakowski@intel.com>
2019-04-16 14:48:41 +02:00
Kamil Łepek
00c434fb66
Merge pull request #115 from imjfckm/leaks
Fix leaky tests
2019-04-16 10:20:56 +02:00
Katarzyna Łapińska
dbf7f15a76
Merge pull request #118 from KamilLepek/fuzzy
Add randomizers for ctypes and string in pyocf
2019-04-16 10:16:39 +02:00
Jan Musial
003d6505aa Fix leaky tests
Fix some instance tracking to prevent Python-side objects from leaking
buffers.
Also, reduce min size of Cache instance (real minimum should be around ~~19MiB,
but we need to make it more deterministic and 20 MiB seems to be reasonable).

Still some stuff left to do, but it needs more investigation and, for
now, this should suffice just to enable some form of CI.

Signed-off-by: Jan Musial <jan.musial@intel.com>
2019-04-16 09:55:02 +02:00
Kamil Lepek
465e9acd2c Add randomizers for ctypes and string in pyocf 2019-04-16 09:51:02 +02:00
Adam Rutkowski
f0c3e049c6
Merge pull request #117 from mmichal10/null-ptr-on-cache-id-reuse
Fix teardown after failed start cache operation.
2019-04-11 09:03:09 -04:00
Daniel Madej
58106fc929
Merge pull request #110 from KamilLepek/change_mode
Added cache mode change in pyocf and basic test
2019-04-11 13:08:31 +02:00
Michał Wysoczański
10fccdda8a
Merge pull request #114 from arutk/ajrutkow-detach-error-handling
Fix error handling in cache unplug
2019-04-11 10:59:45 +02:00
Kamil Lepek
5ceb69e9c8 Added cache mode change in pyocf and basic test 2019-04-11 10:57:25 +02:00
Michal Mielewczyk
fbfd17ec42 Fix teardown after failed start cache operation.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2019-04-11 04:33:32 -04:00
Adam Rutkowski
a6e7ee528e Fix error handling in cache unplug
Errors not related to cache disk I/O failure should force
cache stop to return with error without deinitializing cache
instance.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2019-04-10 15:15:12 -04:00
Daniel Madej
f9c22262fb
Merge pull request #109 from KamilLepek/add_remove
Pyocf add/remove tests
2019-04-10 15:02:22 +02:00