{"slug": "from-aws-security-hub-to-client-ready-html-a-private-ai-reporting-pipeline", "title": "From AWS Security Hub to Client-Ready HTML: A Private AI Reporting Pipeline", "summary": "A developer described a private two-stage pipeline that pulls consolidated AWS Security Hub CSPM findings from GuardDuty and Inspector, renders a first-stage HTML report inside AWS, then post-processes it on a local EC2 host running a self-hosted model to produce client-ready HTML. The workflow keeps inference within the AWS account boundary using an encrypted EBS volume, a narrowly scoped IAM role, and disabled network egress, with remediation approval remaining with humans. The author argues that sending security reports to external AI services creates a new data-processing boundary and that a Markdown extension is not a data classification boundary.", "body_md": "**Security scope:** This walkthrough is for authorised defensive reporting in AWS accounts you own or operate. It uses only synthetic names and example paths. Do not place real customer findings, account identifiers, IP addresses, or internal hostnames into public examples.\n\nSecurity Hub CSPM provides a useful consolidated security view. It can receive findings from Amazon GuardDuty and Amazon Inspector, and it normalises findings into the AWS Security Finding Format (ASFF). GuardDuty findings and Inspector findings are delivered to Security Hub after the respective integrations are enabled. [AWS documents the supported integrations and their regional behaviour.](https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-internal-providers.html)\n\nThat still leaves a reporting problem. A weekly operational export can contain vulnerability records, exposures, detections, duplicate resource context, workflow state, identifiers, and raw remediation text. It is valuable evidence, but it is not automatically a decision-ready client report.\n\nThis article describes a private two-stage pattern for a hypothetical AWS account in **Singapore (`ap-southeast-1`)**:\n\nThe result is not “AI fixes security.” It is a repeatable, auditable workflow that reduces report-production effort while keeping remediation approval with people.\n\nThe first-stage HTML produced in AWS is useful, but it is usually a poor client-facing experience. It is optimised for traceability back to source findings, not for executive consumption. It tends to contain long ARNs, internal IDs, repeated technical evidence, flattened remediation notes, and no clear distinction between a newly observed finding, a persistent finding, and a verified remediation.\n\nThat is not a limitation of Security Hub. It is a consequence of using one artifact for two different jobs:\n\n| Need | Evidence-oriented AWS artifact | Client-ready post-processed HTML | \n|---|---|---|\n| Preserve source fidelity | Primary purpose | Linked or summarised only | \n| Support triage and audit | Strong | Supporting role | \n| Explain priority and business action | Limited | Primary purpose | \n| Interactive filters and drill-down | Basic or custom-built | Designed in deliberately | \n| Safe external distribution | Requires careful review | Explicitly redacted and classified | \n\nThe post-processing stage retains traceability but gives the report its own visual hierarchy: executive metrics, severity and provider filters, sortable findings, top affected resources, remediation candidates, and downloadable CSV. It also prevents the report renderer from being coupled to the AWS reporting Lambda.\n\nDo not treat a Markdown extension as a data classification boundary. A security report can include account IDs, resource ARNs, IP addresses, application names, software versions, vulnerability information, detection timing, and an unremediated backlog. Sending it to an external AI service creates a new data-processing boundary, may change retention and access assumptions, and reduces control over the output artifact.\n\nThis is not a claim that every managed AI service is unacceptable. An organisation with an approved enterprise agreement, documented retention settings, data-processing assessment, and an approved use case may make a different decision. The point is that this workflow does not require that transfer.\n\nThe local post-processing host keeps inference inside the AWS account boundary. The model runs from an encrypted EBS volume on an EC2 instance. The instance has a narrowly scoped IAM role, no public inbound access, approved administrator access through Systems Manager or a restricted bastion, and network egress disabled after the model download. The residual risks are then ordinary AWS workload risks: IAM misuse, instance compromise, EBS access, log leakage, and insecure report distribution. Treat them as such.\n\n```\nGuardDuty + Inspector + Security Hub CSPM\n                │\n                ▼\n  Weekly EventBridge schedule → Lambda report builder → S3 (KMS encrypted)\n                                         │\n                     deterministic ranking + bounded Bedrock narrative\n                                         │\n                                         ▼\n                 Markdown snapshot + AWS evidence HTML + prior-week state\n\nS3 read-only role → private EC2 local LLM → offline client-ready HTML → S3/client channel\n```\n\nThere are two important controls in this design:\n\nUse `ap-southeast-1` consistently for the hypothetical workload account and the Security Hub aggregation region. Security Hub integrations are regional, and some services or finding types have regional prerequisites. Cross-Region aggregation must be designed deliberately rather than assumed. [AWS documents aggregation behaviour separately.](https://docs.aws.amazon.com/securityhub/latest/userguide/finding-aggregation.html)\n\nIn the AWS console or your reviewed infrastructure-as-code process:\n\n`RecordState=ACTIVE` and workflow statuses requiring attention should normally enter the weekly backlog. Keep resolved and suppressed records in the comparison logic, not in the default active queue.`example-security-reporting-ap-southeast-1`, with Block Public Access, versioning, default SSE-KMS encryption, a restrictive bucket policy, and lifecycle rules.\n\n```\ns3://example-security-reporting-ap-southeast-1/\n  weekly/raw/2026-09-07/weekly-security-summary.md\n  weekly/evidence-html/2026-09-07/security-evidence.html\n  weekly/client-html/2026-09-07/client-security-report.html\n  weekly/state/2026-09-07/findings-manifest.json\n```\n\nThe `findings-manifest.json` is important. It is the durable comparison state, containing a stable finding identifier, provider, workflow status, severity, normalised priority score, and report week. Do not compare findings by title alone.\n\nModel availability, model IDs, and cross-Region inference requirements change. Do not hard-code a model identifier copied from a blog. From an identity that is allowed to query Bedrock, verify what the Singapore Region exposes:\n\n```\naws bedrock list-foundation-models \\\n  --region ap-southeast-1 \\\n  --by-provider Anthropic\n```\n\nUse Claude Sonnet 4.6 only when the account and selected inference mode make it available. If Bedrock returns an inference profile rather than a directly invokable model, configure that profile ID or ARN. The Lambda role needs the appropriate Bedrock inference permission for the approved model or profile. AWS notes that `InvokeModel` requires `bedrock:InvokeModel`; its current SDK guidance recommends the Converse API when the model supports it. [See the Bedrock runtime API documentation.](https://docs.aws.amazon.com/boto3/latest/reference/services/bedrock-runtime/client/invoke_model.html)\n\nThe Lambda should be useful when Bedrock is unavailable. Its deterministic path should:\n\n`GetFindings` with pagination and a constrained filter set.\nUse an explicit scoring policy. The following is illustrative logic, not a copy-and-paste Lambda implementation:\n\n```\nSEVERITY_BASE = {\n    \"CRITICAL\": 100,\n    \"HIGH\": 75,\n    \"MEDIUM\": 45,\n    \"LOW\": 20,\n    \"INFORMATIONAL\": 5,\n}\n\ndef priority_score(finding, asset_criticality=0, exposure_modifier=0):\n    \"\"\"Deterministic, reviewable prioritisation policy.\"\"\"\n    return (\n        SEVERITY_BASE.get(finding[\"Severity\"][\"Label\"], 0)\n        + asset_criticality       # for example, 0–25 from approved resource tags\n        + exposure_modifier      # for example, 0–20 from approved evidence\n    )\n```\n\nKeep the policy versioned. For Inspector, you may add approved vulnerability context such as exploitability or exposure evidence where it is available in your data model. For GuardDuty, do not assume an Inspector-style CVSS value exists. The model must not manufacture either value.\n\nAfter deterministic ranking, pass a compact, redacted structured payload to Claude Sonnet 4.6. Ask it to:\n\nConstrain the response to JSON and validate it before inserting it into Markdown. A safe contract is:\n\n```\n{\n  \"executive_summary\": \"string\",\n  \"top_six_rationale\": [\n    {\"finding_id\": \"string\", \"business_rationale\": \"string\", \"remediation_summary\": \"string\"}\n  ],\n  \"data_quality_notes\": [\"string\"]\n}\n```\n\nThe Lambda should fall back to a deterministic summary if Bedrock fails, times out, exceeds a token budget, or returns invalid JSON. Log the error category and request ID, not the full security-report prompt or response.\n\nA missing finding is not automatically a fixed finding. It may have been archived, filtered out, delayed, or affected by an integration problem. Report these categories separately:\n\nThis distinction is the difference between a credible remediation report and a misleading one.\n\nUse an EventBridge schedule to invoke the Lambda weekly. The execution role should have only the permissions required to read Security Hub findings, invoke the approved Bedrock model or inference profile, write its report prefix in S3, read the prior manifest prefix, use the designated KMS key, and write CloudWatch Logs.\n\nDo not grant `AdministratorAccess`, broad `s3:*`, or broad Bedrock access. Scope S3 permissions to the report bucket and prefixes. If you use a customer-managed KMS key, allow the Lambda role to use it only through S3 for the required bucket. Configure a dead-letter or failure destination and an alarm for failed invocations.\n\nOne correction matters here: **Kali Linux is Debian-derived; it is not an Ubuntu image.** Use an ARM64 Kali AMI if Kali is a requirement. If you start from an Ubuntu ARM64 AMI, keep it as Ubuntu and adapt the hardening baseline accordingly. Do not treat the two images as interchangeable.\n\nFor a CPU-only local model, `t4g.2xlarge` provides 8 vCPUs and 32 GiB RAM. Attach encrypted EBS, use an instance profile rather than access keys, and prefer SSM Session Manager. The instance profile should be read-only to the specific S3 report prefixes and write-only to the final client-report prefix.\n\nInstall the local report transformer package and its dependencies:\n\n```\nsudo apt update && sudo apt -y full-upgrade\nsudo apt install -y python3 python3-venv python3-dev build-essential cmake curl unzip awscli htop sysstat\n\nsudo useradd --system --home /opt/cloud-report-ai --shell /usr/sbin/nologin cloudreport\nsudo mkdir -p /srv/cloud-report-ai/{input,output,checkpoints} /var/log/cloud-report-ai /etc/cloud-report-ai\n\n# Copy the reviewed package to /tmp by an approved internal transfer mechanism.\nsudo unzip -q /tmp/cloud_report_ai_arm64_package.zip -d /opt/cloud-report-ai-release\nsudo mv /opt/cloud-report-ai-release/cloud_report_ai /opt/cloud-report-ai\nsudo rmdir /opt/cloud-report-ai-release\n\nsudo chown -R cloudreport:cloudreport /opt/cloud-report-ai /srv/cloud-report-ai /var/log/cloud-report-ai\n\ncd /opt/cloud-report-ai\nsudo -u cloudreport python3 -m venv .venv\nsudo -u cloudreport .venv/bin/python -m pip install --upgrade pip wheel\nsudo -u cloudreport env CMAKE_ARGS=\"-DGGML_NATIVE=ON\" \\\n  .venv/bin/pip install --no-binary llama-cpp-python -r requirements.txt\n```\n\nThe package uses `llama-cpp-python`, which compiles `llama.cpp` locally for ARM64. That avoids an x86-only binary dependency. Pin and scan the package release in your own software-supply-chain process before production use.\n\nDownload a reviewed quantised model, then remove unrestricted egress when your operating model allows it. For this use case, Qwen2.5 7B Instruct in Q4_K_M GGUF is a practical CPU-only extractor; it is not a replacement for security review.\n\n```\nsudo -u cloudreport mkdir -p /opt/cloud-report-ai/models\nsudo -u cloudreport curl -L --fail --retry 3 \\\n  -o /opt/cloud-report-ai/models/Qwen2.5-7B-Instruct-Q4_K_M.gguf \\\n  https://huggingface.co/bartowski/Qwen2.5-7B-Instruct-GGUF/resolve/main/Qwen2.5-7B-Instruct-Q4_K_M.gguf\n\nsudo cp /opt/cloud-report-ai/config.example.json /etc/cloud-report-ai/config.json\nsudo chown root:cloudreport /etc/cloud-report-ai/config.json\nsudo chmod 640 /etc/cloud-report-ai/config.json\n```\n\nVerify the model checksum from a trusted release record before using it. The external URL above is an operational download location, not a substitute for your supply-chain validation.\n\nThe EC2 role should retrieve only the intended weekly input. Use an explicit input path, not a wide wildcard across every historic report:\n\n```\nexport AWS_REGION=ap-southeast-1\nexport REPORT_BUCKET=example-security-reporting-ap-southeast-1\nexport REPORT_WEEK=2026-09-07\nexport INPUT=/srv/cloud-report-ai/input/weekly-security-summary-${REPORT_WEEK}.md\n\naws s3 cp \\\n  \"s3://${REPORT_BUCKET}/weekly/raw/${REPORT_WEEK}/weekly-security-summary.md\" \\\n  \"$INPUT\" \\\n  --region \"$AWS_REGION\" \\\n  --only-show-errors\n\nsudo chown cloudreport:cloudreport \"$INPUT\"\nsudo chmod 600 \"$INPUT\"\n\ncd /opt/cloud-report-ai\nsudo -u cloudreport .venv/bin/python ai_report_generator.py \\\n  --config /etc/cloud-report-ai/config.json \\\n  --input \"$INPUT\" \\\n  --output \"/srv/cloud-report-ai/output/client-security-report-${REPORT_WEEK}.html\"\n```\n\nThe tool chunk-processes long Markdown, normalises findings into a strict schema, records content-hash checkpoints, deduplicates, and writes a self-contained HTML file with no CDN dependency. Its local JSON-repair path prevents a recoverable malformed model response from wasting a long CPU-bound report run; output still requires schema validation.\n\nUpload the reviewed final report to the designated S3 prefix using the bucket's encryption and retention policy, or transfer it through an approved encrypted channel. Do not make an HTML report public merely because it renders locally in a browser.\n\nThis is the normal weekly operating procedure. The OS, Python environment, local model, package, IAM role, configuration, and output folders already exist. Only the new input Markdown changes.\n\n```\ncd /opt/cloud-report-ai\nsudo -u cloudreport .venv/bin/python --version\nsudo -u cloudreport .venv/bin/python -m pip check\nsudo -u cloudreport test -r /opt/cloud-report-ai/models/Qwen2.5-7B-Instruct-Q4_K_M.gguf\nfree -h\n```\n\nIf `pip check` reports conflicts, stop and resolve them through your change process. Do not upgrade packages automatically in the weekly reporting window.\n\n```\nexport AWS_REGION=ap-southeast-1\nexport REPORT_BUCKET=example-security-reporting-ap-southeast-1\nexport REPORT_WEEK=2026-09-14\nexport INPUT=/srv/cloud-report-ai/input/weekly-security-summary-${REPORT_WEEK}.md\n\naws s3 cp \\\n  \"s3://${REPORT_BUCKET}/weekly/raw/${REPORT_WEEK}/weekly-security-summary.md\" \\\n  \"$INPUT\" --region \"$AWS_REGION\" --only-show-errors\n\nsha256sum \"$INPUT\"\nsudo chown cloudreport:cloudreport \"$INPUT\"\nsudo chmod 600 \"$INPUT\"\n```\n\nCompare the checksum with a manifest value generated by the AWS reporting stage when your process records one. This confirms you transformed the expected artifact, not merely a file with the expected name.\n\n```\ncd /opt/cloud-report-ai\n\nsudo -u cloudreport .venv/bin/python ai_report_generator.py \\\n  --config /etc/cloud-report-ai/config.json \\\n  --input \"$INPUT\" \\\n  --output \"/srv/cloud-report-ai/output/client-security-report-${REPORT_WEEK}.html\"\n```\n\nMonitor the run from a second terminal:\n\n```\ntail -f /var/log/cloud-report-ai/cloud-report-ai.log\n```\n\nOn a CPU-only host, a quiet terminal during a chunk is normal: the model emits its response only after generation completes. Check `htop` or the process CPU usage before treating it as stalled. If CPU is idle for an extended period and the log has not moved, capture the last log lines and investigate the model process, available memory, disk space, and malformed-input handling.\n\nPerform a human review before distribution:\n\nThen apply classification, store the final report in the approved location, and record reviewer approval. Preserve the AWS evidence artifact and manifest for auditability.\n\nThe tangible achievement is a controlled reporting pipeline with deterministic priority, bounded AI assistance, weekly comparison, and a client-readable presentation layer. It reduces repetitive collection, sorting, drafting, and formatting work while preserving a human decision point for remediation.\n\nTime savings should be measured locally, not presented as a universal benchmark. For example, if a manual weekly collection, prioritisation, writing, and formatting cycle takes four analyst-hours, and the automated pipeline leaves 45–60 minutes of review and approval, the expected saving is roughly three analyst-hours per weekly run. Record actual run time, review time, correction rate, and report acceptance rate for several weeks before making a business case.\n\nThe residual limitations remain important:", "url": "https://wpnews.pro/news/from-aws-security-hub-to-client-ready-html-a-private-ai-reporting-pipeline", "canonical_source": "https://dev.to/mike_anderson_d01f52129fb/from-aws-security-hub-to-client-ready-html-a-private-ai-reporting-pipeline-4l0n", "published_at": "2026-09-14 05:01:44+00:00", "updated_at": "2026-09-14 05:26:38.880872+00:00", "lang": "en", "topics": ["ai-tools", "ai-infrastructure", "ai-safety", "mlops", "developer-tools"], "entities": ["AWS Security Hub", "Amazon GuardDuty", "Amazon Inspector", "AWS", "Amazon EC2", "AWS Security Finding Format", "Amazon EBS", "AWS Systems Manager"], "alternates": {"html": "https://wpnews.pro/news/from-aws-security-hub-to-client-ready-html-a-private-ai-reporting-pipeline", "markdown": "https://wpnews.pro/news/from-aws-security-hub-to-client-ready-html-a-private-ai-reporting-pipeline.md", "text": "https://wpnews.pro/news/from-aws-security-hub-to-client-ready-html-a-private-ai-reporting-pipeline.txt", "jsonld": "https://wpnews.pro/news/from-aws-security-hub-to-client-ready-html-a-private-ai-reporting-pipeline.jsonld"}}