{"slug": "security-checks-with-local-llms", "title": "Security Checks with Local LLMs", "summary": "An experiment using local Large Language Models (LLMs) to perform security checks and code quality reviews, motivated by rising costs and new limits for cloud-based LLM APIs. The author selected the `qwen2.5-coder:14b-instruct-q5_K_M` model via Ollama, running on a MacBook Air M5 with 24GB RAM, and created a custom bash script to automate file scanning with configurable prompts and cooldown delays. The article concludes that using a 32k context window provides a good balance between execution speed and hardware temperature, with plans for further experimentation.", "body_md": "Continuing articles [AI-Powered Repository Security Check with Antigravity Workflow](https://dev.to/gdg/ai-powered-repository-security-check-with-antigravity-workflow-5hee) and [https://dev.to/gdg/how-to-build-a-custom-ai-quality-gate-on-cloud-run-from-zero-to-production-1odp](https://dev.to/gdg/how-to-build-a-custom-ai-quality-gate-on-cloud-run-from-zero-to-production-1odp) I've decided to try to outsource some checks to local LLM.\n\nThis article describes my experiment and outcomes. Will be glad to read your questions, proposals, opinions or advices! 🙌\n\n*You can listen a podcast generated based on this publication (thanks NotebookLM):*\n\n# Intro\n\nLast changes in limits management for popular LLM APIs make me thinking about FinOps management. Why should I spend expensive cloud tokens for simple tasks? Also I have a lot of talks at last security and AI events which led me to begin experiments with local LLMs in terms of code generation and code quality checks.\n\n# Hardware\n\nThe hardware for experiments is MacBook Air M5 24GB RAM. I bought it especially for diving into ML topics but it was underloaded since today.\n\n# Pains\n\nThe first pain was an introduction of new limits for the Antigravity IDE. Along with models list changing it led me to think about optimizing my development and security flows which were intended to use cheaper Antigravity tokens prior to more expensive Vertex AI tokens.\n\nThe second pain was the FOMO effect about Machine Learning and MLOps itself.\n\n# Solution Track\n\nAfter some iterations with Ollama and local models I've selected the `qwen2.5-coder:14b-instruct-q5_K_M`\n\nas a base model with optimized context window:\n\n```\n% cat Modelfile-qwen-32k \nFROM qwen2.5-coder:14b-instruct-q5_K_M\nPARAMETER num_ctx 32000\n\n% ollama create qwen-coder-32k -f ./Modelfile-qwen-32k\n\n...\n\n% ollama list\nNAME                                 ID              SIZE      MODIFIED     \nqwen-coder-32k:latest                dc3c4762d967    10 GB     2 hours ago     \nqwen-coder-64k:latest                42f060e717dd    10 GB     2 hours ago     \nqwen2.5-coder:14b-instruct-q5_K_M    05d16c5ac1c1    10 GB     2 hours ago     \ngemma4:e4b                           c6eb396dbd59    9.6 GB    25 hours ago    \ngemma4:e2b                           7fbdbf8f5e45    7.2 GB    25 hours ago\n```\n\nThe 32k window provided me with quite quick execution and a trade-off between the speed and the temperature of my laptop. I think this configuration will be a subject of experiments in near future.\n\nThen I've realized that I have to decompose tasks and give some rest time between requests to my hardware. So the unified script was born:\n\n``` bash\n#!/bin/bash\n\n# Default values\nOUTPUT_DIR=\".\"\nMODEL_NAME=\"qwen-coder-32k\"\nCOEFF=2\nPROMPT_FILE=\"\"\n\nshow_help() {\n    echo \"Usage: $0 -d <directory> -m <file_mask> -p <prompt_file> [OPTIONS]\"\n    echo \"\"\n    echo \"Required parameters:\"\n    echo \"  -d  Directory for searching files\"\n    echo \"  -m  File mask to check\"\n    echo \"  -p  Path to a text file with system prompt (e.g., prompts/strict_table.txt)\"\n    echo \"\"\n    echo \"Optional parameters:\"\n    echo \"  -o  Directory to save the final report (default: current directory)\"\n    echo \"  -e  Exclude directories (comma-separated, e.g., venv,tests,migration)\"\n    echo \"  -f  Exclude file masks (comma-separated, e.g., *test*,__init__.py)\"\n    echo \"  -c  Cooldown delay multiplier (default: 2)\"\n    exit 1\n}\n\n# Argument parsing\nwhile getopts \"d:m:o:e:f:c:p:h\" opt; do\n    case \"$opt\" in\n        d) SRC_DIR=\"$OPTARG\" ;;\n        m) FILE_MASK=\"$OPTARG\" ;;\n        o) OUTPUT_DIR=\"$OPTARG\" ;;\n        e) EXCLUDE_DIRS=\"$OPTARG\" ;;\n        f) EXCLUDE_FILES=\"$OPTARG\" ;;\n        c) COEFF=\"$OPTARG\" ;;\n        p) PROMPT_FILE=\"$OPTARG\" ;;\n        h) show_help ;;\n        *) show_help ;;\n    esac\ndone\n\n# Check required parameters\nif [ -z \"$SRC_DIR\" ] || [ -z \"$FILE_MASK\" ] || [ -z \"$PROMPT_FILE\" ]; then\n    echo \"❌ Error: Required parameters -d, -m, or -p are missing.\"\n    show_help\nfi\n\n# Check if prompt file exists\nif [ ! -f \"$PROMPT_FILE\" ]; then\n    echo \"❌ Error: Prompt file '$PROMPT_FILE' not found!\"\n    exit 1\nfi\n\n# Check Ollama\nif ! pgrep -x \"ollama\" > /dev/null && ! curl -s http://localhost:11434 > /dev/null; then\n    echo \"❌ Error: Ollama is not running!\"\n    exit 1\nfi\n\n# Check jq\nif ! command -v jq &> /dev/null; then\n    echo \"❌ Error: 'jq' utility is not installed. Run: brew install jq\"\n    exit 1\nfi\n\n# Initialize report directory\nmkdir -p \"$OUTPUT_DIR\"\nTIMESTAMP=$(date +%Y%m%d_%H%M%S)\nREPORT_FILE=\"$OUTPUT_DIR/review_report_$TIMESTAMP.md\"\n\n# Write report header\n{\n    echo \"# 🛡️ Review Report\"\n    echo \"Generation date: $(date)\"\n    echo \"Used prompt: \\`$PROMPT_FILE\\`\"\n    echo -e \"\\n---\\n\"\n} > \"$REPORT_FILE\"\n\necho \"==================================================================\"\necho \"🕵️‍♂️ Starting review...\"\necho \"📂 Final report will be saved to: $REPORT_FILE\"\necho \"==================================================================\"\n\n# Build find command\nFIND_CMD=\"find \\\"$SRC_DIR\\\" -type f -name \\\"$FILE_MASK\\\"\"\n\nif [ -n \"$EXCLUDE_DIRS\" ]; then\n    IFS=',' read -ra DIRS <<< \"$EXCLUDE_DIRS\"\n    FOR_FIND=\"\"\n    for dir in \"${DIRS[@]}\"; do\n        if [ -z \"$FOR_FIND\" ]; then\n            FOR_FIND=\"-path '*/$dir/*'\"\n        else\n            FOR_FIND=\"$FOR_FIND -o -path '*/$dir/*'\"\n        fi\n    done\n    FIND_CMD=\"find \\\"$SRC_DIR\\\" \\( $FOR_FIND \\) -prune -o -type f -name \\\"$FILE_MASK\\\" -print\"\nfi\n\n# Start main file processing loop\neval \"$FIND_CMD\" | while read -r file; do\n    if [ ! -f \"$file\" ]; then continue; fi\n\n    # Check file exclusions\n    if [ -n \"$EXCLUDE_FILES\" ]; then\n        IFS=',' read -ra FILE_MASKS <<< \"$EXCLUDE_FILES\"\n        skip_file=false\n        for mask in \"${FILE_MASKS[@]}\"; do\n            if [[ \"$(basename \"$file\")\" == $mask ]]; then\n                skip_file=true\n                break\n            fi\n        done\n        if [ \"$skip_file\" = true ]; then\n            echo \"⏭️ Skipping file (excluded by mask): $file\"\n            continue\n        fi\n    fi\n\n    echo -n \"⏳ Analyzing: $file ... \"\n\n    # Read code and clear comments/empty lines\n    CLEANED_CODE=$(sed -e 's/[[:space:]]*#.*//' -e '/^[[:space:]]*$/d' \"$file\")\n    if [ -z \"$CLEANED_CODE\" ]; then \n        echo \"⚠️ Empty.\"\n        continue\n    fi\n\n    # Write file section to report\n    {\n        echo \"## 📁 File: $file\"\n        echo -e \"\\n### 🔍 Analysis results:\\n\"\n    } >> \"$REPORT_FILE\"\n\n    # Read external prompt and combine with code\n    SYSTEM_PROMPT=$(cat \"$PROMPT_FILE\")\n    FULL_PROMPT=\"$SYSTEM_PROMPT\\n\\n--- TARGET CODE ---\\n$CLEANED_CODE\"\n\n    JSON_PAYLOAD=$(jq -n --arg model \"$MODEL_NAME\" --arg prompt \"$FULL_PROMPT\" '{model: $model, prompt: $prompt, stream: false}')\n\n    # Measure time and send API request\n    START_TIME=$(date +%s)\n    curl -s -X POST http://localhost:11434/api/generate -H \"Content-Type: application/json\" -d \"$JSON_PAYLOAD\" | jq -r '.response' >> \"$REPORT_FILE\"\n    END_TIME=$(date +%s)\n\n    ELAPSED=$((END_TIME - START_TIME))\n    SLEEP_TIME=$((ELAPSED * COEFF))\n\n    echo -e \"\\n\\n---\\n\\n\" >> \"$REPORT_FILE\"\n    echo \"✅ Elapsed: ${ELAPSED}s. Rest: ${SLEEP_TIME}s.\"\n\n    if [ \"$SLEEP_TIME\" -gt 0 ]; then \n        sleep \"$SLEEP_TIME\"\n    fi\ndone\n\necho \"==================================================================\"\necho \"🎉 Review successfully completed!\"\necho \"==================================================================\"\n```\n\nThe logic of the script:\n\n- Get info about which files to check and where they are stored.\n- Get the file with the prompt content.\n- Get some optional parameters about filtering, outputs and delays between requests.\n- For each file:\n- Read the file and clean it from not meaningful things like comments and empty lines.\n- Send the file content into the local LLM along with the prompt.\n- Receive result and save it to the report.\n- Count the processing time for the file and sleep x2 (by default) time to cool down the hardware.\n\n# Outcomes\n\n## Execution Flow\n\n```\n(venv) %n@%m %1~ %# ./scripts/repo-check-1.sh -d scripts -m setup* -p scripts/prompt-infrasec.txt \n==================================================================\n🕵️‍♂️ Starting review...\n📂 Final report will be saved to: ./review_report_20260521_121530.md\n==================================================================\n⏳ Analyzing: scripts/setup-quality-gate-iam.sh ... ✅ Elapsed: 6s. Rest: 12s.\n⏳ Analyzing: scripts/setup-gcp-details.sh ... ✅ Elapsed: 95s. Rest: 190s.\n⏳ Analyzing: scripts/setup-gcp.sh ... ✅ Elapsed: 128s. Rest: 256s.\n==================================================================\n🎉 Review successfully completed!\n==================================================================\n```\n\n## Report\n\n### 🔍 Analysis results:\n\n| Finding / Vulnerability | Recommendation / Fix |\n|---|---|\n| Assigning public access (legacyObjectReader) to GCS bucket | Remove the line `gsutil iam ch allUsers:legacyObjectReader \"gs://${BUCKET_NAME}\"` to prevent making the bucket publicly accessible. Consider using more restrictive permissions based on your security requirements. |\n| Hardcoded service account name in the script | Avoid hardcoding sensitive information like service account names. Instead, retrieve them from a secure source or use environment variables. |\n| Missing encryption settings for GCS bucket | Ensure that the GCS bucket is encrypted by default. Add the `--encryption` flag to the `gsutil mb` command if you want to specify a specific encryption type, such as `--encryption=DEFAULT` . |\n| No logging and monitoring configurations | Implement logging and monitoring for the resources created. Enable Cloud Logging and Monitoring to track access and usage of the secrets and GCS bucket. |\n| Using automatic replication policy for secrets | Consider using a more controlled replication policy for secrets. Automatic replication might not be necessary for all use cases, and you should evaluate whether it aligns with your security and compliance requirements. |\n| Lack of error handling for secret creation | Add proper error handling when creating the secret to ensure that any issues during the creation process are caught and addressed appropriately. |\n| No version control for secrets | Ensure that secrets have a versioning strategy in place. This allows you to manage changes and roll back to previous versions if needed. |\n| Potential for misconfiguration of IAM roles | Double-check the IAM roles being assigned to ensure they align with the principle of least privilege. Avoid assigning broader permissions than necessary for the service account. |\n\n## Conclusion\n\nLooks extremely interesting:\n\n- The time elapsed is quite good for me.\n- The LLM answer is quite similar to cloud LLMs. And it was achieved without prompt tuning or additional context manipulations.\n\nFurther steps planned:\n\n- Experiment with models, context windows, prompts and additional contexts.\n- Check whether it will work on some kind of a local SOHO server for batch tasks.", "url": "https://wpnews.pro/news/security-checks-with-local-llms", "canonical_source": "https://dev.to/alexandertyutin/security-checks-with-local-llms-3d4m", "published_at": "2026-05-21 08:10:57+00:00", "updated_at": "2026-05-21 08:33:24.170417+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "developer-tools", "cybersecurity"], "entities": ["Antigravity", "Vertex AI", "Ollama", "qwen2.5-coder:14b-instruct-q5_K_M", "MacBook Air M5"], "alternates": {"html": "https://wpnews.pro/news/security-checks-with-local-llms", "markdown": "https://wpnews.pro/news/security-checks-with-local-llms.md", "text": "https://wpnews.pro/news/security-checks-with-local-llms.txt", "jsonld": "https://wpnews.pro/news/security-checks-with-local-llms.jsonld"}}