sysk a CLI health monitor in bash and python, how i built it

Design notes on Sysk, a small system health monitor.

Sysk is a command-line system health monitor built in Bash and Python.

Bash collects machine data, handles flags, and drives the user interface. Python runs the inference and decision engines.

This article records the design of v1: what the system does, which decisions shaped it, and where those decisions cost something.

The pipeline is four stages:

collect --> infer --> decide --> print

v1 collects data across five modules:

Each module presented a different reading problem. The rest of the article follows the pipeline above.

The main collection decision was to read kernel-exported files rather than treat programs such as top

or free

as the source of truth.

Advantages

Disadvantages

Files and directories used

/proc/meminfo          -> memory
/proc/cpuinfo          -> cpu identity
/proc/loadavg          -> load average
/proc/stat             -> cpu accounting
/proc/uptime           -> uptime

/sys/block             -> disks
/sys/class/thermal     -> thermal zones and cooling
/sys/class/hwmon       -> hardware monitors (fans, sensors)

This rule is not absolute. Hardware identity and sound did not fit it cleanly. dmidecode

, smartctl

, and pactl

are still dependencies. Sound, in particular, is read through pactl

rather than a kernel file.

Finding the right file for each module was messy. Once the path was known, the output looked cryptic at first glance, but it was stable enough to parse.

Reading files raised the next question: how should the project be structured?

Each major function became its own script, sourced from a single entry point. That was easy to say and immediately produced four portability questions:

Those questions produced the portable layout.

1. Resolve the application path

export SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"

This is the usual “where am I?” idiom. A pure string-manipulation version exists, but the cd

/ pwd -P

form follows symlinks and is the one Sysk uses.

2. Export directory names from that root

export SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
export XDG_CONFIG_HOME=${XDG_CONFIG_HOME:-${SCRIPT_DIR}/.config}
export DUMPS_PATH=${DUMPS_PATH:-${SCRIPT_DIR}/logs}
export CACHE_PATH=${CACHE_PATH:-${SCRIPT_DIR}/.cache}
export CONFIG_DIR="$CACHE_PATH/sysk"
export ENGINE_DIR=${ENGINE_DIR:-${SCRIPT_DIR}/engine}
export RULE_DIR="$ENGINE_DIR/.rules"
export RESULT_DIR="$ENGINE_DIR/results"
export DATE="$(date +%Y_%m_%d)"

Defaults can be overridden by the environment. Paths are derived from SCRIPT_DIR

instead of being pasted in as /home/kit/...

.

3. Source scripts in a fixed order

core -> device scripts -> hardware scripts
for script in "$CORE_DIR"/*.sh; do
    source "$script"
done

for dir in "$DEVICE_DIR"/*; do
    for script in "$dir"/*.sh; do
        source "$script"
    done
done

for script in "$HW_DIR"/*.sh; do
    source "$script"
done

Core files are numbered so the source order is obvious:

00_error.sh
01_privileged.sh
02_check_deps.sh
03_install_deps.sh
04_cleanup.sh
05_parse_flags.sh
tui.sh

00_error.sh

— named error codes01_privileged.sh

— root / sudo handling02_check_deps.sh

— missing binaries03_install_deps.sh

— install those binaries04_cleanup.sh

— cleanup and traps05_parse_flags.sh

— getopts

tui.sh

— text interfaceSome reads (dmidecode

, parts of disk data) need elevated permission. A one-line sudo "$@"

was not enough. The questions were:

sudo

is missing?01_privileged.sh

answers those with small predicates:

is_root() {
    [[ $EUID -eq 0 ]] && return "$ERR_SUCCESS"
    return "$ERR_FAILURE"
}

is_sudo_available() {
    command -v sudo >/dev/null 2>&1 && return "$ERR_SUCCESS"
    return "$ERR_FAILURE"
}

is_sudo_active() {
    sudo -n true >/dev/null 2>&1 && return "$ERR_SUCCESS"
    return "$ERR_FAILURE"
}

kill_sudo() {
    is_sudo_active && {
        sudo -k
        return "$ERR_SUCCESS"
    }
    printf 'sudo not active, aborting...\n' >&2
    return "$ERR_BAD_USAGE"
}

Even with a file-first design, Sysk still depends on a few programs: dmidecode

, smartctl

, pactl

, and jq

.

The flow is:

check dependencies -> get permission -> install missing ones

command -v

on each name; keep the missing ones.

declare -r PACKAGE_MANAGERS=(
    apt dnf pacman zypper emerge
)

os_pkg_manager=""

get_pkg_manager() {
    for pkg in "${PACKAGE_MANAGERS[@]}"; do
        if command -v "$pkg" >/dev/null 2>&1; then
            os_pkg_manager=$pkg
            return 0
        fi
    done
    printf 'unknown package manager, aborting...\n' >&2
    exit 1
}

Each module binds the files it cares about and hides the parsing behind a writer.

CPU

readonly CPU_INFO_FILE="/proc/cpuinfo"
readonly LOAD_INFO_FILE="/proc/loadavg"
readonly CPU_USAGE_FILE="/proc/stat"
readonly UPTIME_INFO_FILE="/proc/uptime"

Memory

readonly MEM_INFO_FILE="/proc/meminfo"

Disks

readonly DRIVE_DIR="/sys/block"

mapfile -t drives < <(
    find "$DRIVE_DIR"/* -maxdepth 1 -printf '%f\n' | grep -Ev '(^loop|ram|zram)'
)

Thermal

readonly SYSTEM_THERMAL_ZONE_PATH="/sys/class/thermal/thermal_zone"
readonly SYSTEM_FAN_INFO_PATH="/sys/class/hwmon"

Sound — collected through pactl

, not /proc

.

Snapshots are JSON, written with jq

and a per-module filter file:

write_memory_json() {
    local MEM_CONFIG="$CONFIG_DIR/memory_$DATE.json"
    mkdir -p "$CONFIG_DIR"

    jq -n \
        --arg total_mem "$(get_total_mem)" \
        --arg available_mem "$(get_available_mem)" \
        -f "$FEATURE_DIR/build_memory.jq" \
        > "$MEM_CONFIG"
}

Example thermal snapshot:

{
  "average_temp": 31.87,
  "thermal_zones": 3,
  "zone_temparatures": {
    "acpitz": "29.80",
    "x86_pkg_temp": "38.00"
  },
  "cooling": {
    "fan_zones": "2",
    "fan_speed": "fan1_input=\"0\"\nfan2_input=\"0\""
  }
}

Problem. The JSON shape is not the same across modules. Some values are scalars, some are objects, some are nested. That leaked into inference and into the rule files.

Fix for v1.

build_*.jq

filtersource

field in the YAML tells the engine where to readresolve

function walks that pathA uniform snapshot schema is still future work.

Inference pipeline:

load data -> load rules -> set status per field -> write result

The engine is a Python script. It uses the standard library plus PyYAML, and it reads the same environment variables Bash exported (CACHE_PATH

, RULE_DIR

, RESULT_DIR

, DATE

).

def load_rules(rule_path: Path) -> dict:
    with rule_path.open(encoding="utf-8") as rule:
        return safe_load(rule)

def load_json(data_path: Path) -> Any:
    try:
        with data_path.open(encoding="utf-8") as data:
            return load(data)
    except (FileNotFoundError, OSError, JSONDecodeError) as err:
        print(f"[ERROR] failed to load {data_path}: {err}", file=sys.stderr)
        sys.exit(1)

Rules live in engine/.rules

as YAML. Each field names the value it wants through source

:

module: cpu
check_interval: 10
system_vars:
  cpu_cores: "auto"

fields:
  - name: cpu_usage_percent
    source: "usage"
    type: "percentage"
    base_value: 100
    warning_multiplier: 0.83
    critical_multiplier: 0.90
    unit: "%"

  - name: load_avg5
    source: "load_avg.1"
    type: "load"
    base_value: "cores"
    warning_multiplier: 1
    critical_multiplier: 1.5
    unit: ""

The collector JSON could not be renamed. The mapping therefore lives in the rules, not in a hard-coded Python dictionary.

source

is a small path:

usage

core_info.cores_usage

load_avg.1

resolve

walks that path:

def resolve(data: dict, source: str) -> Any:
    current = data
    for obj in source.split("."):
        if obj.isdigit():
            current = get_list_value(current, int(obj))
        else:
            current = get_dict_value(current, obj)
    return current

The helpers catch KeyError

, IndexError

, and TypeError

, print on stderr, and exit. resolve

itself stays a straight loop.

Because cpu, memory, and thermal still do not share one value shape, evaluate

dispatches to a small private helper per module (_evaluate_cpu

, and so on) and merges the results into one dictionary. v1 loads the three modules it already has rules for. Disk and sound collection exist; their inference rules are not finished in the same way.

The public main

in inference.py

is only orchestration: check environment variables, check input paths, create RESULT_DIR

if needed, evaluate, write result_{DATE}.json

.

load result -> find warning and critical fields -> write a cause log if needed

The decision script does not re-parse /proc

. It reads today’s result file.

The decision I made for v1: if every field is OK

, exit 0 and write nothing extra. If any field is not OK

, write cause_{DATE}.log

and return a non-zero status (90) so Bash can tell an alert from a clean run.

v1 uses short flags only, parsed with getopts

:

parse_args() {
    while getopts ':hvrm:' opt; do
        case "$opt" in
            h) usage; exit 0 ;;
            v) printf '%s version %s\n' "$PROGNAME" "$VERSION"; exit 0 ;;
        esac
    done
}

Action flags such as -h

and -v

print and exit inside the parser. Setting flags only store values. The TUI runs only when the session is a terminal and quiet mode is off.

All of the pieces meet in one function:

check files and directories
parse flags
install missing dependencies
collect snapshots
run inference
run decision
show the TUI unless -q was passed
exit
main() {
    check_req_files || exit "$ERR_FAILURE"
    check_req_dirs  || exit "$ERR_FAILURE"

    parse_args "$@"
    install_missing_deps

    write_cpu_json &
    write_memory_json
    write_thermal_json
    write_disk_json
    write_sound_json

    mkdir -p "$DUMPS_PATH"
    python3 "$ENGINE_DIR/inference.py" && python3 "$ENGINE_DIR/decision.py"

    display_tui || exit "$ERR_FAILURE"
    exit "$ERR_SUCCESS"
}

main "$@"

The first runs collected modules one after another. CPU sampling waits about one second to compute usage, so the whole collect stage felt slow.

write_cpu_json &

starts that slow job in the background so the other writers can proceed. That made the collect stage feel almost instant.

This only stays correct if the entry point waits for the background job before inference starts. Otherwise Python can open a CPU file that is not finished yet. A wait

after the writers is part of this design, not an optional extra.

The project is small on purpose.

Dated result files are already named result_{DATE}.json

so a later version can compare runs without changing the collector.

sysk/
├── lib/
│   ├── core/          # 00_error.sh ... 05_parse_flags.sh, tui.sh
│   ├── devices/
│   └── hw/
├── engine/
│   ├── .rules/        # cpu.yml, memory.yml, thermal.yml
│   ├── inference.py
│   ├── decision.py
│   └── results/
└── sysk               # entry point

That is the architecture of v1: files in, JSON snapshots, YAML rules, a small path walker, a verdict, and a Bash wrapper that the user actually types. Many features could still be added. The shape above is what is running now.

if you are interested in the project you can check it out at https://github.com/4kit1-glitch/sysk

thanks for being along with me in my journey

source & further reading

dev.to — original article HoneyCrawlPot: Serve Decoy Files to AI-Powered Vulnerability Scanners How to estimate ChatGPT API costs before moving a coding agent to production Fairness Under the Microscope: Why HY3 Beats Nemotron 3 Ultra on lforla's Bias Stereotypes Audit