Commit Graph

769 Commits

Author SHA1 Message Date
Adam Rutkowski
b232f2b633 Service WA write misses in WI engine
WA write must follow follow the same two-pass pattern
as WI does. This change modifies WA engine to default to
WI in case of any miss (either partial or full), not only
partial miss.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-07-20 17:26:36 +02:00
Adam Rutkowski
91b6098fda Two pass write invalidate
Add second pass of write invalidate. It is necessary only
if concurrent I/O had inserted target LBAs to cache after
WI request did traversation. These LBAs might have been
written by WI request behind the concurrent I/O's back,
resulting in making these sectors effectively invalid.
In this case we must update these sectors' metadata to
reflect this. However we won't know about this after we
traverse the request again - hence calling ocf_write_wi
again with req->wi_second_pass set to indicate that this
is the second pass (core write should be skipped).

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-07-20 17:26:35 +02:00
Slawomir Jankowski
13828ab836 Update version
Signed-off-by: Slawomir Jankowski <slawomir.jankowski@intel.com>
2020-04-06 16:30:59 +02:00
Robert Baldyga
72f11d0771
Merge pull request #362 from robertbaldyga/fix-flush-deadlock
Fix deadlock on concurrent flush at the same cache
2020-04-05 10:49:41 +02:00
Robert Baldyga
a9c36477d2 Fix deadlock on concurrent flush at the same cache
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2020-04-03 18:09:35 +02:00
Robert Baldyga
7799b24858 env: posix: Add missing env_mutex_trylock() function
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2020-04-03 18:09:35 +02:00
Robert Baldyga
53dc4020e3
Merge pull request #358 from arutk/req_queue_fix
Do not reference req after adding to queue list
2020-03-27 15:04:51 +01:00
Robert Baldyga
80b410dc2e
Merge pull request #355 from arutk/flush_fixes
Fix stalls and warnings during flush
2020-03-27 14:11:34 +01:00
Adam Rutkowski
e39a76aa5e Do not reference req after adding to queue list
ocf_engine_push_req_(front|back) must not dereference req
pointer after putting the request on queue list and unlocking
the queue. At this point handler interface may asynchronously
pick up the request, handle it and deallocate.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-03-26 01:29:02 +01:00
Adam Rutkowski
b267d5d77d Reduce flush relaxation period by 1 order of magninude
Loop now relaxes every 2^17 (131K) cycles instead of every 1M.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-03-25 23:37:49 +01:00
Adam Rutkowski
fd328bd0a1 Check relaxation condition in each step of flush loop
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-03-25 23:36:43 +01:00
Adam Rutkowski
4d61d56249 Rename flushing functions local variables for readibility
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-03-25 23:29:16 +01:00
Robert Baldyga
cf5e13c4aa
Merge pull request #357 from arutk/parallel_flush_Fix
Queue flush portion requests to the back of IO queue
2020-03-24 23:15:11 +01:00
Adam Rutkowski
64dcae1490 Split global metadata lock critial path
There is no need to constantly hold metadata global lock
during collecting cachelines to clean. Since we are past
freezing dirty request counter, we know for sure that the
number of dirty cache lines will not increase. So the worst
case is when the loop realxes and releases the lock,
a concurent IO to CAS is serviced in WT mode, possibly
inserting and/or evicting cachelines. This does not interfere
with scanning for dirty cachelines. And the lower layer will
handle synchronization with concurrent I/O by acquiring
asynchronous read lock on each cleaned cacheline.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-03-23 19:32:15 -04:00
Adam Rutkowski
3b3a49e8ea Queue flush portion requests to the back of IO queue
In current implementation in case of fast media flushning
container may starve all concurrent containers flushing
due to continous rescheduling of offender requests to the
front of I/O queue. Pushing request to the back of IO
queue ensures FIFO handling and removes possibility of
starvation.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-03-23 19:06:14 -04:00
Adam Rutkowski
c17beec7d4 Do not exclude used cachelines from flushing
Lower layer is prepared to handle used cachelines by
acquiring asynchronus read lock. It is very likely that
by the time the cacheline is actually cleaned its lock
state changes. So checking the lock at the moment of
constructing dirty cachelines list makes little sense.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-03-23 12:59:30 -04:00
Adam Rutkowski
61983c946c Move flush containers sort & submit outside metadata lock
Moving _ocf_mngt_flush_containers outside global metadata
critical section. All this function does is sort core lines
and add queue request.

This fixes stalls reported by Linux scheduler due to
IO threads waiting on global metadata RW semaprhore for
several minutes.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-03-23 12:59:30 -04:00
Robert Baldyga
86d7212217
Merge pull request #356 from robertbaldyga/ut-cache-alloc
ut: Add cache allocation & do little cleanup
2020-03-23 13:21:17 +01:00
Robert Baldyga
f7d191b765 ut: Add cache allocation & do little cleanup
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2020-03-23 12:38:43 +01:00
Robert Baldyga
c295a4f670 ut: Add .gitignore
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2020-03-23 12:33:12 +01:00
Robert Baldyga
ed91895f70
Merge pull request #351 from micrakow/seq_cutoff_pt_fix
Fix seq_cutoff respecting in pt read
2020-03-23 10:46:09 +01:00
Michal Rakowski
6f4d02f251 Fix seq_cutoff respecting in pt read
Signed-off-by: Michal Rakowski <michal.rakowski@intel.com>
2020-03-20 18:58:10 +01:00
Michal Rakowski
2edd05c812 Change get_effective_cache_mode to operate on req instead of io
Signed-off-by: Michal Rakowski <michal.rakowski@intel.com>
2020-03-20 18:58:10 +01:00
Michal Rakowski
d84942daa3 Typo fixes
Signed-off-by: Michal Rakowski <michal.rakowski@intel.com>
2020-03-17 16:36:40 +01:00
Robert Baldyga
22bdb8b004
Merge pull request #352 from robertbaldyga/update-memory-requirement-check
Update memory requirement check
2020-03-17 15:28:56 +01:00
Robert Baldyga
9b9c7e88e7
Merge pull request #350 from imjfckm/check-promotion-memory
Add memory check before engaging promotion policy
2020-03-17 15:20:25 +01:00
Robert Baldyga
94b4bee6de Update memory requirement check
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2020-03-17 14:42:01 +01:00
Jan Musial
d2fe82dc85 Add memory check before engaging promotion policy
Signed-off-by: Jan Musial <jan.musial@intel.com>
2020-03-16 09:09:42 +01:00
Robert Baldyga
b08514c2db
Merge pull request #349 from imjfckm/optimize-nhit-hash
Reorder fields in nhit_hash map to improve memory efficiency
2020-03-09 20:23:32 +01:00
Jan Musial
4eb5612832 Reorder fields in nhit_hash map to improve memory efficiency
Signed-off-by: Jan Musial <jan.musial@intel.com>
2020-03-06 12:36:46 +01:00
Robert Baldyga
2a7a2bd6ca
Merge pull request #348 from robertbaldyga/introduce-core-priv
Introduce core priv
2020-03-03 15:52:59 +01:00
Robert Baldyga
108fe28ad4 Introduce core priv
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2020-03-03 15:37:12 +01:00
Robert Baldyga
b871d32617
Merge pull request #347 from imjfckm/allow-zero-in-alru-params
Allow 0ms activity threshold and wake up time for ALRU
2020-02-25 08:53:45 +01:00
Jan Musial
677e9e07e5 Allow 0ms activity threshold and wake up time for ALRU
Signed-off-by: Jan Musial <jan.musial@intel.com>
2020-02-21 16:05:20 +01:00
Adam Rutkowski
49d8f10953
Merge pull request #346 from robertbaldyga/fix-memory-alloc-in-metadata
Fix memory allocation in metadata
2020-02-14 13:21:05 +01:00
Robert Baldyga
ac7b5aba6b metadata: Allocate memory with ENV_MEM_NOIO flag
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2020-02-14 12:03:21 +01:00
Robert Baldyga
952767905a env: Add extended versions of env_vmalloc functions
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2020-02-14 12:03:21 +01:00
Robert Baldyga
b7e59ee04a metadata: Use proper function for freeing memory
a_req is allocated using env_vmalloc() so we need to free it
using env_vfree(), not env_free().

Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2020-02-14 10:29:15 +01:00
rafalste
1dfae0d166
Merge pull request #345 from arutk/resize_discard_alloc
Fix discard request map allocation
2020-02-10 16:12:17 +01:00
Adam Rutkowski
ee37391e97 Fix discard request map allocation
Discard handling splits large request into several steps.
However the actual size of request map for discard was
determined based on original request size, not step request
size, resulting in waste of memory and allocations > 4K.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-02-10 17:47:11 -05:00
Adam Rutkowski
26fd938ccf Reduce max trim request size to 512K
512K is the maximum request size for which request map
fits into one page (4K) regardless of cacheline size.

Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-02-10 15:57:34 -05:00
Adam Rutkowski
3eadffad0f Add static assert macro ENV_BUILD_BUG_ON to posix env
Signed-off-by: Adam Rutkowski <adam.j.rutkowski@intel.com>
2020-02-10 15:55:52 -05:00
Michał Wysoczański
fabd41250b
Merge pull request #342 from mmichal10/fix-metadata-flush
Fix metadata flush
2020-01-24 17:59:58 +01:00
Michal Mielewczyk
d9c987e068 Flush metadata after changing status of each sector
In case of cleaning metadata used to be flushed only when status of whole cache
line changed to clean.

This patch ensures that metadata flush is triggered after changin status of each
single sector is cache line.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2020-01-24 11:27:56 -05:00
Michal Mielewczyk
2f10365086 Flush metadata after setting dirty status of each sector.
After second dirty write to cache line which was already dirty, metadata flush
was not triggered. In case of dirty shutdown, this led to data corruption.

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
2020-01-24 11:27:56 -05:00
Michał Mielewczyk
30f884ba84
Merge pull request #343 from robertbaldyga/relax-memory-alloc-requirements-metadata-io
metadata: Relax memory allocation requirements
2020-01-24 11:37:31 +01:00
Robert Baldyga
7d82f20614 Remove unused include
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2020-01-24 11:21:04 +01:00
Robert Baldyga
4d25bbe4b3 metadata: Relax memory allocation requirements
Signed-off-by: Robert Baldyga <robert.baldyga@intel.com>
2020-01-24 11:21:04 +01:00
Sławomir Jankowski
89f0e96607
Merge pull request #339 from imjfckm/check-max-cache-size
Detect cache devices that would overflow ocf_cacheline_t
2020-01-22 10:25:16 +01:00
Jan Musial
e5aff609d9 Add test for huge cache
Signed-off-by: Jan Musial <jan.musial@intel.com>
2020-01-21 15:29:24 +01:00