{"slug": "llm-based-hierarchical-topic-modeling-tool", "title": "LLM-Based Hierarchical Topic Modeling Tool", "summary": "A new LLM-based hierarchical topic modeling tool normalizes inconsistent raw strings into a reusable taxonomy for JSONL data, steering with plain-language instructions. The tool maps values like 'bad', 'neg', and 'negative' to 'negative' or 'search' to 'Features|Search', and works with any JSONL file containing strings, nested objects, and arrays of objects. It requires Python 3.9, an API key for a LiteLLM-supported model provider, and internet access while the LLM builds or extends the taxonomy.", "body_md": "Hierarchical topic modeling groups related labels into a taxonomy: broad topics at the top and increasingly specific subtopics below them. This tool is a practical, LLM-assisted version for JSONL data: it turns inconsistent raw strings into a reusable hierarchy that you steer with plain-language instructions.\n\nFor example, it can map `bad`\n\n, `neg`\n\n, and `negative`\n\nto `negative`\n\n, or map `search`\n\nto `Features|Search`\n\n. It is designed for extracted Reddit-study data, but works with any JSONL file containing strings, nested objects, and arrays of objects.\n\nThe input is a JSONL file with raw string values. The output is another JSONL file with the **same records and structure**, except that mapped values have been replaced by their normalized category paths.\n\n`run_normalization.py`\n\n— the interactive normalizer.`normalization.py`\n\n— field discovery, hierarchical mappings, config merging, and JSONL transformation.`ai_helper.py`\n\n— Helper for interacting with LiteLLM`config.py`\n\n— model IDs, API keys, and rate limits.`examples\\reddit_extraction.jsonl`\n\n— a small Reddit-shaped input example.\n\n- Windows, macOS, or Linux.\n- Python 3.9 (probably works with other versions).\n- An API key for a LiteLLM-supported model provider (pretty much all of them).\n- Internet access while the LLM is building or extending the taxonomy.\n\nMost installations use prebuilt Python packages and do not compile anything. If pip reports that `Rust`\n\n, `rustc`\n\n, or `cargo`\n\nis required, install Rust first rather than trying to bypass the error.\n\nOn Windows, download and run the Rust installer from the official [Rust installation page](https://rust-lang.org/install.html). Accept the default stable toolchain. If it offers to install the Visual Studio C++ Build Tools, allow it: Rust’s Windows MSVC toolchain may need them. Close and reopen PowerShell, then verify:\n\n```\nrustc --version\ncargo --version\n```\n\nOn macOS or Linux, install Rust with Rustup:\n\n```\ncurl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\n```\n\nChoose the default installation, then open a new terminal (or run `source \"$HOME/.cargo/env\"`\n\n) and verify:\n\n```\nrustc --version\ncargo --version\n```\n\nOn Linux, a native compiler/linker may also be required when building packages from source. Install your distribution's development tools first if the Rust installation or pip build reports a linker error—for example, `build-essential`\n\non Debian/Ubuntu or `base-devel`\n\non Arch Linux.\n\nRustup is the official recommended installer and installs both `rustc`\n\nand `cargo`\n\n. [Rust installation documentation](https://rust-lang.org/install.html)\n\nOpen PowerShell in this folder:\n\n```\ncd \"C:\\reddit download script\\normalization_tool\"\n```\n\nUse your installed Python directly:\n\n```\npython -m pip install --upgrade pip\npython -m pip install -r requirements.txt\n```\n\nAn isolated environment is recommended if you work on several Python projects, but it is not required. With Miniconda or Anaconda installed:\n\n```\nconda create -n topic-normalizer python=3.11 pip -y\nconda activate topic-normalizer\ncd \"C:\\reddit download script\\normalization_tool\"\npython -m pip install --upgrade pip\npython -m pip install -r requirements.txt\n```\n\nActivate the same environment whenever you use the tool:\n\n```\nconda activate topic-normalizer\n```\n\nOpen `config.py`\n\n. Its three important settings are:\n\n`AI_MODEL_KEYS`\n\n— model ID → one or more API keys.`MODELS`\n\n— the priority order of models to try.`AI_MODEL_LIMITS`\n\n— rate limits for each model/key.\n\nThe model ID must be a valid LiteLLM model identifier for your provider. The included example uses OpenRouter.\n\nIf you downloaded this tool from Git, create your private configuration first:\n\n```\nCopy-Item config.example.py config.py\n```\n\nThis is the quickest setup. Replace the example list with your actual key:\n\n```\nAI_MODEL_KEYS = {\n    \"openrouter/google/gemini-2.5-flash-lite\": [\n        \"sk-or-v1-your-real-key-goes-here\",\n    ],\n}\n\nMODELS = [\"openrouter/google/gemini-2.5-flash-lite\"]\n```\n\nDo **not** commit or share this edited file. Treat it like a password.\n\nThis preserves the same `os.getenv(...)`\n\nlogic in `config.py`\n\n, but supplies those variables automatically whenever the tool starts.\n\n-\n\n`.env.example`\n\nin this folder and rename the copy to exactly`.env`\n\n. -\nOpen\n\n`.env`\n\nin a text editor and fill in the key for the provider you use:\n\n```\nOPENROUTER_API_KEY=sk-or-v1-your-real-key-goes-here\n```\n\n-\nEnsure\n\n`config.py`\n\nreads the same variable name. The existing OpenRouter pattern already does this:\n\n```\nAI_MODEL_KEYS = {\n    \"openrouter/google/gemini-2.5-flash-lite\": [\n        key for key in [os.getenv(\"OPENROUTER_API_KEY\")] if key\n    ],\n}\n```\n\nThe tool loads `.env`\n\nbefore it evaluates `AI_MODEL_KEYS`\n\n. Values already set in PowerShell, Conda, or the operating system take priority over `.env`\n\n. The `.env`\n\nfile is listed in `.gitignore`\n\n, but it is still a plain-text secret file: do not share it, upload it, or commit it manually.\n\nThe direct-paste option remains available. If you paste a key directly into `AI_MODEL_KEYS`\n\n, that direct value is used; `.env`\n\nis simply ignored for that entry.\n\nTo use a different provider, change the model ID, the environment-variable name, and `MODELS`\n\ntogether.\n\nIf you have several keys for the same model, list them all under that model. The helper rotates between them and tracks the limits configured in `AI_MODEL_LIMITS`\n\n.\n\nThe input must be valid **JSON Lines** (`.jsonl`\n\n): one complete JSON object per non-empty line. Values that you want to normalize should still be raw strings.\n\n```\n{\"id\":\"kdnmrkk\",\"overall_platform_sentiment\":\"bad\",\"categories_discussed\":[{\"target_feature\":\"search\",\"sentiment\":\"neg\"}]}\n{\"id\":\"abc123\",\"overall_platform_sentiment\":\"negative\",\"categories_discussed\":[{\"target_feature\":\"Search\",\"sentiment\":\"bad\"}]}\n```\n\nNested arrays work automatically. In this example, `categories_discussed.sentiment`\n\nrefers to the `sentiment`\n\nvalue of every object in the `categories_discussed`\n\narray.\n\n```\npython -m run_normalization\n```\n\nThe script asks for:\n\n**Input JSONL path**— for example`examples\\reddit_extraction.jsonl`\n\n.**Output JSONL path**— press Enter to save beside the input as`<input-name>_normalized.jsonl`\n\n.**Normalization config path**— press Enter for`normalization_config.json`\n\nin this folder. Reuse the same file for future datasets to retain the taxonomy.\n\nThe script then discovers every string path in the JSONL and starts the interactive normalization loop.\n\nEach round shows:\n\n- the number of configured fields;\n- the current\n`ignored_fields`\n\nlist; - every field with unmapped values;\n- the total number of unmapped values for each field and a preview ending in\n`...`\n\n.\n\nGive the LLM one instruction for the current round. Be explicit about both normalization and exclusions. For example:\n\n```\nNormalize overall_platform_sentiment and categories_discussed.sentiment to positive, negative, or neutral. Ignore id, url, created_at, opinion_summary, and user_niche.\n```\n\nThe model returns a complete configuration. The tool merges it into the saved configuration without discarding existing mappings. It then checks the JSONL again and repeats until every discovered string value is either mapped or belongs to an ignored field.\n\nTell the LLM to ignore fields that are identifiers, URLs, dates, long free-text summaries, or any values you do not want changed. The model adds their dot paths to `ignored_fields`\n\nin `normalization_config.json`\n\n.\n\nExample:\n\n```\nIgnore id, url, created_at, and opinion_summary. Normalize user_persona and all sentiment fields.\n```\n\nIgnored fields are not checked for missing mappings and are preserved unchanged in the output. The list is shown at the start of every round so you can see exactly what the script will skip.\n\nChoose how many unmapped values per field are sent to the LLM in a round. Press Enter for 50. A smaller number reduces prompt size and cost; a larger number gives the model more examples.\n\nAnswer `y`\n\nto **Autocomplete until all values are mapped or ignored?** to reuse the same instruction automatically on subsequent rounds. This is useful after giving a broad instruction such as “normalize all values into a concise hierarchy and ignore technical fields.”\n\nAutocomplete stops when all values are mapped or ignored. If an LLM call fails, it retries the loop; stop the program manually if a provider outage persists. Answer `n`\n\nwhen you want to inspect and guide each round yourself.\n\nType `exit`\n\nat the instruction prompt to save the current configuration and stop before creating the normalized JSONL output.\n\nThe configuration supports any depth of hierarchy. A mapping at a nested level is written to the output as a pipe-delimited path.\n\n```\n{\n  \"fields\": [\n    {\n      \"field_path\": \"categories_discussed.target_feature\",\n      \"categories\": [\n        {\n          \"normalized_value\": \"Features\",\n          \"raw_variations\": [],\n          \"subcategories\": [\n            {\n              \"normalized_value\": \"Search\",\n              \"raw_variations\": [\"search\", \"Search function\"],\n              \"subcategories\": []\n            }\n          ]\n        }\n      ]\n    }\n  ],\n  \"ignored_fields\": [\"id\", \"url\"]\n}\n```\n\nHere, `search`\n\nbecomes `Features|Search`\n\nin the output JSONL.\n\nFor complex taxonomies, you can prepare the configuration outside the script before continuing:\n\n- Stop the script or let it save\n`normalization_config.json`\n\n. - Copy the entire JSON configuration.\n- Paste it into a chat UI that can handle large pasted documents, such as DeepSeek.\n- Ask it to review the taxonomy, merge duplicates, improve category names, add hierarchy, or make the exact changes you want.\n- Copy the full corrected JSON object back into\n`normalization_config.json`\n\n. - Restart\n`python -m run_normalization`\n\nso the tool reloads the edited config.\n\nThis gives the normalizer a stronger starting taxonomy for its next interactive round. A future version is intended to make this external review step unnecessary; for now, it is a useful way to fine-tune a large configuration.\n\n- This is instruction-driven taxonomy construction, not a high-coverage or fully automated research pipeline. It only sees the raw string values and samples you provide; it does not have broad context about the original posts unless that context is itself present in the field values.\n- LLMs can hallucinate, misclassify values, choose an overly broad category, or incorrectly ignore a field. Review the configuration and normalized output, especially before analysis or publication.\n- A single raw value can currently map to only\n**one** normalized path. Values that genuinely belong to two separate categories are not represented well by the current configuration format. - Long, highly unique free text is usually a poor fit for this tool. Ignore it or preprocess it into labels first.\n- API cost, model quality, context size, and rate limits affect speed and consistency.\n\nThe tests use mock JSONL records, mocked interactive input, and mocked LLM responses—no API calls are made.\n\n```\npython -m unittest discover -s tests -v\n```\n\n| Problem | What to do |\n|---|---|\n`No configured API keys` |\nSet the environment variable used in `config.py` , or paste a key into `AI_MODEL_KEYS` . Check that the model also appears in `MODELS` . |\n`rustc` or `cargo` is required |\nInstall Rust with Rustup, reopen PowerShell, verify `rustc --version` , then rerun the pip install. |\n| Provider/model error | Check the LiteLLM model identifier, your provider key, and the selected model’s availability. |\n| Invalid JSON line | The error includes a line number. JSONL must have one complete JSON object per non-empty line. |\n| Values remain unmapped | Give a more specific instruction, increase sample size, or tell the LLM to add the field to `ignored_fields` . |", "url": "https://wpnews.pro/news/llm-based-hierarchical-topic-modeling-tool", "canonical_source": "https://github.com/Tryhard-cs/LLM-Hierarchical-Topic-Modeling", "published_at": "2026-07-21 16:26:11+00:00", "updated_at": "2026-07-21 16:43:12.158169+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "natural-language-processing", "developer-tools"], "entities": ["LiteLLM", "OpenRouter", "Rust", "Python", "Windows", "macOS", "Linux"], "alternates": {"html": "https://wpnews.pro/news/llm-based-hierarchical-topic-modeling-tool", "markdown": "https://wpnews.pro/news/llm-based-hierarchical-topic-modeling-tool.md", "text": "https://wpnews.pro/news/llm-based-hierarchical-topic-modeling-tool.txt", "jsonld": "https://wpnews.pro/news/llm-based-hierarchical-topic-modeling-tool.jsonld"}}