github/nohang
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update readme and remove trash

Browse Source
This commit is contained in:
Alexey Avramov 2019-09-20 00:45:40 +09:00
parent 84e6dd4c20
commit 286ed840e5
4 changed files with 1 additions and 3953 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -66,7 +66,7 @@ Of course, you can also [download more RAM](https://downloadmoreram.com/), tune
For basic usage: For basic usage:
- `Linux` 3.14+ (since `MemAvailable` appeared in `/proc/meminfo`) - `Linux` 3.14+ (since `MemAvailable` appeared in `/proc/meminfo`)
- `Python` 3.3+ (not tested with previous) - `Python` 3.3+
To show GUI notifications: To show GUI notifications:
- [notification server](https://wiki.archlinux.org/index.php/Desktop_notifications#Notification_servers) (most of desktop environments use their own implementations) - [notification server](https://wiki.archlinux.org/index.php/Desktop_notifications#Notification_servers) (most of desktop environments use their own implementations)

3360
old/nohang
View File

File diff suppressed because it is too large Load Diff

359
old/nohang.conf
View File

@ -1,359 +0,0 @@
This is nohang config file.
Lines starting with #, tabs and spaces are comments.
Lines starting with @ contain optional parameters.
All values are case sensitive.
Be careful: nohang doesn't forbid you to shoot yourself in the foot.
The configuration includes the following sections:
0. Common zram settings
1. Memory levels to respond to as an OOM threat
2. Response on PSI memory metrics
3. The frequency of checking the level of available memory
(and CPU usage)
4. The prevention of killing innocent victims
5. Impact on the badness of processes via matching their names, cgroups and
cmdlines with specified regular expressions
6. Customize corrective actions: the execution of a specific command
instead of sending the SIGTERM signal
7. GUI notifications:
- low memory warnings
- OOM prevention results
8. Output verbosity
9. Misc
Just read the description of the parameters and edit the values.
Please restart the program after editing the config.
More docs will be written later.
###############################################################################
0. Common zram settings
See https://www.kernel.org/doc/Documentation/blockdev/zram.txt
You maybe need to set `zram_checking_enabled = True` if you has a big zram disksize.
zram_checking_enabled = False
###############################################################################
1. Thresholds below which a signal should be sent to the victim
Sets the available memory levels at or below which SIGTERM or SIGKILL
signals are sent. The signal will be sent if MemAvailable and
SwapFree (in /proc/meminfo) at the same time will drop below the
corresponding values. Can be specified in % (percent) and M (MiB).
Valid values are floating-point numbers from the range [0; 100] %.
MemAvailable levels.
soft_threshold_min_mem = 8 %
hard_threshold_min_mem = 4 %
SwapFree levels.
soft_threshold_min_swap = 10 %
hard_threshold_min_swap = 5 %
Specifying the total share of zram in memory, if exceeded the
corresponding signals are sent. As the share of zram in memory
increases, it may fall responsiveness of the system. 90 % is a
usual hang level, not recommended to set very high.
Can be specified in % and M. Valid values are floating-point
numbers from the range [0; 90] %.
soft_threshold_max_zram = 60 %
hard_threshold_max_zram = 65 %
###############################################################################
2. Response on PSI memory metrics (it needs Linux 4.20 and up)
About PSI:
https://facebookmicrosites.github.io/psi/
Disabled by default (psi_checking_enabled = False).
psi_checking_enabled = False
Choose a path to PSI file.
By default it monitors system-wide file: /proc/pressure/memory
You also can set file to monitor one cgroup slice.
For example:
psi_path = /sys/fs/cgroup/unified/user.slice/memory.pressure
psi_path = /sys/fs/cgroup/unified/system.slice/memory.pressure
psi_path = /sys/fs/cgroup/unified/system.slice/foo.service/memory.pressure
Execute the command
find /sys/fs/cgroup -name memory.pressure
to find available memory.pressue files (except /proc/pressure/memory).
(actual for cgroup2)
psi_path = /proc/pressure/memory
Valid psi_metrics are:
some_avg10
some_avg60
some_avg300
full_avg10
full_avg60
full_avg300
some_avg10 is most sensitive.
psi_metrics = some_avg10
soft_threshold_max_psi = 60
hard_threshold_max_psi = 90
>= 0, float
psi_excess_duration = 60
psi_post_action_delay = 60
###############################################################################
3. The frequency of checking the amount of available memory
(and CPU usage)
Coefficients that affect the intensity of monitoring. Reducing
the coefficients can reduce CPU usage and increase the periods
between memory checks.
Why three coefficients instead of one? Because the swap fill rate
is usually lower than the RAM fill rate.
It is possible to set a lower intensity of monitoring for swap
without compromising to prevent OOM and thus reduce the CPU load.
Default values are well for desktop. On servers without rapid
fluctuations in memory levels the values can be reduced.
Valid values are positive floating-point numbers.
fill_rate_mem = 4000
fill_rate_swap = 1500
fill_rate_zram = 6000
See also https://github.com/rfjakob/earlyoom/issues/61
max_sleep = 3
min_sleep = 0.1
Sleep time if soft threshold exceeded.
over_sleep = 0.05
###############################################################################
4. The prevention of killing innocent victims
Valid values are integers from the range [0; 1000].
min_badness = 10
Valid values are non-negative floating-point numbers.
Min delay if a victim doesn't respond to SIGTERM in 10 ms.
post_soft_action_delay = 3
post_zombie_delay = 0.1
victim_cache_time = 10
Valid values are True and False.
ignore_positive_oom_score_adj = False
###############################################################################
5. Impact on the badness of processes via matching their names,
cmdlines or UIDs with regular expressions using re.search().
See https://en.wikipedia.org/wiki/Regular_expression and
https://en.wikipedia.org/wiki/Perl_Compatible_Regular_Expressions
Enabling this options slows down the search for the victim
because the names, cmdlines or UIDs of all processes
(except init and kthreads) are compared with the
specified regex patterns (in fact slowing down is caused by
reading all /proc/*/cmdline and /proc/*/status files).
Use script `oom-sort` from nohang package to view
names, cmdlines and UIDs of processes.
5.1. Matching process names with RE patterns
Syntax:
@BADNESS_ADJ_RE_NAME badness_adj /// RE_pattern
New badness value will be += badness_adj
It is possible to compare multiple patterns
with different badness_adj values.
Example:
@BADNESS_ADJ_RE_NAME -500 /// ^sshd$
5.2. Matching CGroup_v1-line with RE patterns
@BADNESS_ADJ_RE_CGROUP_V1 -100 /// ^/system\.slice/
@BADNESS_ADJ_RE_CGROUP_V1 50 /// /foo\.service$
@BADNESS_ADJ_RE_CGROUP_V1 -50 /// ^/user\.slice/
5.3. Matching CGroup_v2-line with RE patterns
@BADNESS_ADJ_RE_CGROUP_V2 100 /// ^/workload
5.4. Matching eUIDs with RE patterns
@BADNESS_ADJ_RE_UID -100 /// ^0$
5.5. Matching realpath with RE patterns
@BADNESS_ADJ_RE_REALPATH 20 /// ^/usr/bin/foo
5.6. Matching cmdlines with RE patterns
A good option that allows fine adjustment.
Prefer chromium tabs and electron-based apps
@BADNESS_ADJ_RE_CMDLINE 200 /// --type=renderer
Prefer firefox tabs (Web Content and WebExtensions)
@BADNESS_ADJ_RE_CMDLINE 300 /// -appomni
@BADNESS_ADJ_RE_CMDLINE -200 /// ^/usr/lib/virtualbox
5.7. Matching environ with RE patterns
@BADNESS_ADJ_RE_ENVIRON 100 /// USER=user
Note that you can control badness also via systemd units via
OOMScoreAdjust, see
www.freedesktop.org/software/systemd/man/systemd.exec.html#OOMScoreAdjust=
###############################################################################
6. Customize corrective actions.
TODO: docs
Syntax:
KEY REGEXP SEPARATOR COMMAND
@SOFT_ACTION_RE_NAME ^foo$ /// kill -SEGV $PID
@SOFT_ACTION_RE_NAME ^bash$ /// kill -9 $PID
@SOFT_ACTION_RE_CGROUP_V1 ^/system\.slice/ /// systemctl restart $SERVICE
@SOFT_ACTION_RE_CGROUP_V1 /foo\.service$ /// systemctl restart $SERVICE
$PID will be replaced by process PID.
$NAME will be replaced by process name.
$SERVICE will be replaced by .service if it exists (overwise it will be
relpaced by empty line)
###############################################################################
7. GUI notifications & low memory warnings
post_action_gui_notifications = False
Enable GUI notifications about the low level of available memory.
Valid values are True and False.
low_memory_warnings_enabled = False
Execute the command instead of sending GUI notifications if the value is
not empty line. For example:
warning_exe = cat /proc/meminfo &
warning_exe =
Can be specified in % (percent) and M (MiB).
Valid values are floating-point numbers from the range [0; 100] %.
warning_threshold_min_mem = 20 %
warning_threshold_min_swap = 25 %
warning_threshold_max_zram = 50 %
warning_threshold_max_psi = 100
Valid values are floating-point numbers from the range [1; 300].
min_post_warning_delay = 20
Ampersands (&) will be replaced with asterisks (*) in process
names and in commands.
###############################################################################
8. Verbosity
Display the configuration when the program starts.
Valid values are True and False.
print_config_at_startup = False
Print memory check results.
Valid values are True and False.
print_mem_check_results = False
min_mem_report_interval = 60
print_proc_table = False
Valid values:
None
cgroup_v1
cgroup_v2
realpath
cmdline
environ
extra_table_info = None
print_victim_status = True
max_victim_ancestry_depth = 3
print_victim_cmdline = False
print_statistics = True
Print sleep periods between memory checks.
Valid values are True and False.
debug_psi = False
debug_gui_notifications = False
debug_sleep = False
separate_log = False
###############################################################################
9. Misc
max_soft_exit_time = 10
post_kill_exe =
forbid_negative_badness = True
###############################################################################
Use cases, feature requests and any questions are welcome:
https://github.com/hakavlad/nohang/issues

233
old/nohang_notify_helper
View File

@ -1,233 +0,0 @@
#!/usr/bin/env python3
# print('Starting nohang_notify_helper')
def decoder(string):
"""
"""
decoded = ''
for i in string.split(':'):
decoded += chr(int(i))
return decoded
def write(path, string):
"""
"""
with open(path, 'w') as f:
f.write(string)
def rline1(path):
"""read 1st line from path."""
try:
with open(path) as f:
for line in f:
return line
except OSError:
exit(1)
def rfile(path):
"""read file."""
with open(path) as f:
return f.read()
def re_pid_environ(pid):
"""
read environ of 1 process
returns tuple with USER, DBUS, DISPLAY like follow:
('user', 'DISPLAY=:0',
'DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus')
returns None if these vars is not in /proc/[pid]/environ
"""
try:
env = str(rline1('/proc/' + pid + '/environ'))
if display_env in env and dbus_env in env and user_env in env:
env_list = env.split('\x00')
# iterating over a list of process environment variables
for i in env_list:
if i.startswith(user_env):
user = i
if user == 'USER=root':
return None
continue
if i.startswith(display_env):
display = i[:10]
continue
if i.startswith(dbus_env):
dbus = i
continue
if i.startswith('HOME='):
# exclude Display Manager's user
if i.startswith('HOME=/var'):
return None
try:
env = user.partition('USER=')[2], display, dbus
except UnboundLocalError:
# print('notify helper: UnboundLocalError')
return None
return env
except FileNotFoundError:
# print('notify helper: FileNotFoundError')
return None
except ProcessLookupError:
# print('notify helper: ProcessLookupError')
return None
def root_notify_env():
"""return set(user, display, dbus)"""
unsorted_envs_list = []
# iterates over processes, find processes with suitable env
for pid in listdir('/proc'):
if path.exists('/proc/' + pid + '/exe') is True:
one_env = re_pid_environ(pid)
unsorted_envs_list.append(one_env)
env = set(unsorted_envs_list)
env.discard(None)
# deduplicate dbus
new_env = []
end = []
for i in env:
key = i[0] + i[1]
if key not in end:
end.append(key)
new_env.append(i)
else:
continue
return new_env
try:
write('/proc/self/oom_score_adj', '0')
except Exception:
pass
try:
from os import listdir, path
from subprocess import Popen, TimeoutExpired
from sys import argv
except OSError:
exit(1)
if len(argv) == 5:
_, uid, debug, title, body = argv
else:
print('{}: invalid input'.format(argv[0]))
exit(1)
uid = uid.partition('--euid=')[2]
debug = debug.partition('--debug=')[2]
if debug == 'True':
debug = True
else:
debug = False
title = title.partition('--title=')[2]
body = decoder(body.partition('--body=')[2])
if len(argv) != 5:
print('nohang_notify_helper: invalid input')
exit(1)
with open('/proc/meminfo') as f:
for line in f:
if line.startswith('SwapTotal'):
swap_total = int(line.split(':')[1][:-4])
if swap_total > 0:
wait_time = 15
else:
wait_time = 3
if debug:
print('nohang_notify_helper: wait_time:', wait_time, 'sec')
if uid != '0':
cmd = ['notify-send', '--icon=dialog-warning', title, body]
if debug:
print('nohang_notify_helper: run cmd:', cmd)
with Popen(cmd) as proc:
try:
proc.wait(timeout=wait_time)
except TimeoutExpired:
proc.kill()
if debug:
print('nohang_notify_helper: TimeoutExpired')
exit()
display_env = 'DISPLAY='
dbus_env = 'DBUS_SESSION_BUS_ADDRESS='
user_env = 'USER='
list_with_envs = root_notify_env()
list_len = len(list_with_envs)
# if somebody logged in with GUI
if list_len > 0:
for i in list_with_envs:
if debug:
print('Send a GUI notification:\n ',
'title: ', [title],
'\n body: ', [body],
'\n user/env:', i
)
# iterating over logged-in users
for i in list_with_envs:
username, display_env, dbus_env = i[0], i[1], i[2]
display_tuple = display_env.partition('=')
dbus_tuple = dbus_env.partition('=')
display_value = display_tuple[2]
dbus_value = dbus_tuple[2]
try:
with Popen([
'sudo', '-u', username,
'env',
'DISPLAY=' + display_value,
'DBUS_SESSION_BUS_ADDRESS=' + dbus_value,
'notify-send',
'--icon=dialog-warning',
title,
body
]) as proc:
try:
proc.wait(timeout=wait_time)
except TimeoutExpired:
proc.kill()
print('TimeoutExpired: notify user: ' + username)
except BlockingIOError:
print('nohang_notify_helper: BlockingIOError')
except OSError:
print('nohang_notify_helper: OSError')
except Exception:
print('nohang_notify_helper: CANNOT SPAWN NOTIFY-SEND PROCESS')
else:
if debug:
print(
'Not send GUI notification: [',
title,
body,
']. Nobody logged-in with GUI. Nothing to do.')