cd /news/ai-crawlers/htaccess-mashup · home › topics › ai-crawlers › article
[ARTICLE · art-158534] src=gist.github.com ↗ pub= topic=ai-crawlers verified=true sentiment=· neutral

.htaccess (mashup)

A developer published an .htaccess configuration snippet combining GZIP compression rules with a user-agent blocklist, labeled "Ultimate AI Block List v1.8," that uses mod_rewrite conditions to deny requests from AI crawlers, agents, and training bots. The blocklist matches hundreds of user-agent strings, including GPT, Claude, Anthropic, CCBot, Google-Extended, Perplexity-style research agents, and various RAG and scraping tools.

by read8 min views3 publishedOct 27, 2026

| | Options -Indexes | | | #################### | | | # GZIP COMPRESSION # | | | #################### | | | SetOutputFilter DEFLATE | | | AddOutputFilterByType DEFLATE text/html text/css text/plain text/xml application/x-javascript application/x-httpd-php | | | BrowserMatch ^Mozilla/4 gzip-only-text/html |

|  | BrowserMatch ^Mozilla/4\.0[678] no-gzip | 
|  | BrowserMatch \bMSIE !no-gzip !gzip-only-text/html | 
|  | BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html | 
|  | SetEnvIfNoCase Request_URI \.(?:gif\|jpe?g\|png)$ no-gzip | 

| | # Ultimate AI Block List v1.8 20260105 | | | # https://perishablepress.com/ultimate-ai-block-list/ | | | <IfModule mod_rewrite.c> | | | RewriteEngine On | | | RewriteCond %{HTTP_USER_AGENT} (.ai|-ai|ai|ai.|ai-|ai|ai=|AddSearchBot|Agentic|AgentQL|Agent\ 3|Agent\ API|Agent-|AI\ Agent|AI\ Article\ Writer|AI\ Assistant|AI\ Chat|AI\ Content\ Detector|AI\ Detection|AI\ Dungeon) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (AI\ Journalist|AI\ Legion|AI\ RAG|AI\ Search|AI\ SEO\ Crawler|AI\ Training|AI\ Web|AI\ Writer|AI2|AIBot|aiHitBot|AIMatrix|Airesearch|AISearch|AITraining|Alexa|Alice\ Yandex|AliGenie|AliyunSec|Alpha\ AI) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (AlphaAI|Amazon|Amelia|AndersPinkBot|AndiBot|Anomura|Anonymous\ AI|Anthropic|AnyPicker|Anyword|Applebot-Extended|GoogleAgent-Mariner|Gemini-Deep-Research|Google-NotebookLM|GoogleAgent-URLContext|Google-Firebase|Google-Extended|Aria\ AI|Aria\ Browse|Articoolo|Ask\ AI|Atlassian|AutoGen|Automated\ Writer|AutoML|Autonomous\ RAG) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (AutoRAG|Awario|AWS\ Trainium|Azure|BabyAGI|BabyCatAGI|BardBot|Basic\ RAG|Bedrock|Big\ Sur|Bigsur|Botsonic|Brightbot|Browser\ MCP\ Agent|Browser\ Use|BuddyBot|Bytebot|ByteDance|Bytespider|CarynAI) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (CatBoost|CC-Crawler|CCBot|Chai|Channel3Bot|Character|Charstar\ AI|Chatbot|Chatsonic|ChatUser|Chinchilla|Claude|ClearScope|Clearview|Cognitive\ AI|Cohere|Common\ Crawl|CommonCrawl|Compass|Content\ Harmony) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (Content\ King|Content\ Optimizer|Content\ Samurai|ContentAtScale|ContentBot|Contentedge|ContentShake|Conversion\ AI|Copilot|CopyAI|Copymatic|Copyscape|CoreWeave|Corrective\ RAG|Cotoyogi|CRAB|Crawl4AI|CrawlQ\ AI|Crawlspace|Crew\ AI) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (CrewAI|Crushon\ AI|Cypex|DALL-E|DarkBard|DarkBERT|DataFor|DataProvider|Datenbank\ Crawler|DeepAI|Deep\ AI|DeepL|DeepMind|Deep\ Research|DeepResearch|DeepSeek|Devin|Diffbot|Dolma|Doubao\ AI) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (DuckAssistBot|DuckDuckGo\ Chat|DuckDuckGo-Enhanced|Echobot|Echobox|Elixir|FacebookBot|FacebookExternalHit|Factset|Falcon|FIRE-1|Firebase|Firecrawl|Flyriver|Frase\ AI|FriendlyCrawler|Gato|Gemini|Gemma|Gen\ AI) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (GenAI|Generative|Genspark|Gentoo-chat|Ghostwriter|GigaChat|GLM|GodMode|Goose|GPT|Grammarly|Grendizer|Grok|Groq|GT\ Bot|GTBot|GTP|Hemingway\ Editor|Hetzner|Hugging) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (Hunyuan|Hybrid\ Search\ RAG|Hypotenuse\ AI|iAsk|IbouBot|ICC-Crawler|Ideogram|ImageGen|Imagen|ImagesiftBot|ImageSpider|img2dataset|imgproxy|INK\ Editor|INKforall|Instructor|IntelliSeek|Inferkit|ISSCyberRiskCrawler|Janitor\ AI) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (Jasper|Jenni\ AI|Julius\ AI|Kafkai|Kaggle|Kangaroo|Keyword\ Density\ AI|Kimi|Klaviyo|Knowledge|KomoBot|Kruti|KunatoCrawler|LAIONDown|LangChain|Langfuse|Language\ AI|Le\ Chat|Lensa|Leonardo) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (Lightpanda|LightRAG|LinerBot|LinkupBot|LLaMA|LCC|LLM|Local\ RAG\ Agent|Lovable|Magistral|magpie-crawler|MAI-Image|Manus|MarketMuse|MBZUAI|Meltwater|Meta-AI|Meta-External|Meta-Webindexer|Meta\ AI) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (MetaAI|MetaTagBot|Middleware|Midjourney|Mini\ AGI|MiniMax|Mintlify|Mistral|Mixtral|model-training|Molmo|Monica|Nano|Narrative|NeevaBot|netEstate|Neural\ Text|NeuralSEO|NinjaAI|NodeZero) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (NotebookLM|Nova\ Act|NovaAct|Nytheon|OAI-SearchBot|OAI\ SearchBot|OASIS|Olivia|OLM|Omgili|Onyx|Open\ AI|Open\ Interpreter|OpenAGI|OpenAI|OpenBot|OpenInterpreter|OpenPi|OpenRouter|OpenText\ AI) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (Operator|Outwrite|OVHcloud|Owlin|Page\ Analyzer\ AI|PaLM\ 2|PanguBot|Panscient|Paperlibot|Paraphraser.io|peer39_crawler|Perflexity|Perplexity|Petal|Phind|PiplBot|Pixmo|Playwright|PoeBot|PoeSearchBot) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (Poggio|Poseidon|ProWritingAid|Proximic|PulsarRPA|Puppeteer|Python\ AI|Python\ lxml|PyTorch|Qualified|Quark|QuillBot|Qopywriter|Qwen|RAG\ Agent|RAG\ Azure\ AI|RAG\ Chatbot|RAG\ Database|RAG\ IS|RAG\ Pipeline) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (RAG\ Search|RAG\ with|RAG-|RAG_|Raptor|React\ Agent|Reasoning|Redis\ AI\ RAG|Reka\ Flash|Replicate|RobotSpider|RunPod|Rytr|SaplingAI|SBIntuitionsBot|Scala|Scalenut|Scrap|ScriptBook|Seekr) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (SEObot|SEO\ Content\ Machine|SEO\ Robot|SemrushBot|Sentibot|Serper|ShapBot|Sidetrade|Simplified\ AI|Sitefinity|Skydancer|Skyvern|SlickWrite|SmartBot|Sonic|Sora|Spider/2|SpiderCreator|Spin\ Rewrite|Spinbot) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (Stability|Stable\ Diffusion|StableDiffusionBot|Stagehand|Stealth|Sudowrite|SummalyBot|Super\ Agent|Superagent|SuperAGI|Surfer\ AI|Taiko|TavilyBot|TerraCotta|Text\ Blaze|TextCortex|Thinkbot|Thordata|TikTokSpider|Timpibot) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (Tinybird|Together|Traefik|Tulu|Tülu|TurnitinBot|TwinAgent|uAgents|UI-TARS|VelenPublicWebCrawler|Venus\ Chub\ AI|Vidnami\ AI|Vision\ RAG|WARDBot|WebSurfer|WebText|WebVoyager|Webzio|WeChat|Whisper) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (WordAI|Wordtune|WPBot|Writecream|WriterZen|Writescope|Writesonic|WRTNBot|xAI|Xanthorox|xBot|XLNet|YaK|YaML|YandexAdditional|YouBot|Zanista|Zendesk|Zero|Zhipu|Pandalytics|plainhand-outreach|wpbot) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (Zhuque\ AI|Zimm|BitMoversBot) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (Chrome/(86.0.4240.183|122.0.0.0|123.0.0.0|124.0.0.0|125.0.0.0|126.0.0.0|130.0.0.0|131.0.0.0|133.0.0.0|144.0.0.0)) [NC,OR] |

|  | RewriteCond %{HTTP_USER_AGENT} (Firefox/(124\.0\|126\.0\|12\.0)) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (Edg/124\.0\.0\.0) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (AppleWebKit/537.36\|AppleWebKit/535.45.1\|AppleWebKit/535.1) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (Safari/537.36\|Safari/6531.45.1\|Safari/535.1) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (Opera/8.86\|Opera/9.37) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (Gecko/5512-11-21) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (Trident/4.0) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (MSIE\ 7.0) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (Presto/2.9.169\|Presto/2.9.182) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (FFZBot) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (Cortex-Xpanse\|Palo\ Alto) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (Mastodon) [NC] | 
|  | RewriteRule (.*) - [F,L] | 

| | </IfModule> | | | # 8G FIREWALL v1.5 20250927 | | | # https://perishablepress.com/8g-firewall/ | | | # Addon available: Block AI Bots |

|  | # https://perishablepress.com/ultimate-ai-block-list/ | 
|  | # 8G:[CORE] | 

| | ServerSignature Off | | | Options -Indexes | | | RewriteEngine On | | | RewriteBase / | | | # 8G:[QUERY STRING] | | | <IfModule mod_rewrite.c> |

|  | RewriteCond %{QUERY_STRING} ^(%2d\|-)[^=]+$ [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} ([a-z0-9]{4000,}) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (/\|%2f)(:\|%3a)(/\|%2f) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (etc/(hosts\|motd\|shadow)) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (order(\s\|%20)by(\s\|%20)1--) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (/\|%2f)(\*\|%2a)(\*\|%2a)(/\|%2f) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (`\|<\|>\|\^\|\\|\\\|0x00\|%00\|%0d%0a) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (f?ckfinder\|f?ckeditor\|fullclick) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} ((.*)header:\|(.*)set-cookie:(.*)=) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (localhost\|127(\.\|%2e)0(\.\|%2e)0(\.\|%2e)1) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (cmd\|command)(=\|%3d)(chdir\|mkdir)(.*)(x20) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (globals\|mosconfig([a-z_]{1,22})\|(?<!SAML)request)(=\|\[) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (/\|%2f)((wp-)?config)((\.\|%2e)inc)?((\.\|%2e)php) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (thumbs?(_editor\|open)?\|tim(thumbs?)?)((\.\|%2e)php) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (absolute_\|base\|root_)(dir\|path)(=\|%3d)(ftp\|https?) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (s)?(ftp\|inurl\|php)(s)?(:(/\|%2f\|%u2215)(/\|%2f\|%u2215)) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (\.\|20)(get\|the)(_\|%5f)(permalink\|posts_page_url)(\(\|%28) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} ((boot\|win)((\.\|%2e)ini)\|etc(/\|%2f)passwd\|self(/\|%2f)environ) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (((/\|%2f){3,3})\|((\.\|%2e){3,3})\|((\.\|%2e){2,2})(/\|%2f\|%u2215)) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (benchmark\|exec\|fopen\|function\|html)(.*)($\|%28)(.*)($\|%29) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (php)([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (e\|%65\|%45)(v\|%76\|%56)(a\|%61\|%31)(l\|%6c\|%4c)(.*)($\|%28)(.*)($\|%29) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (/\|%2f)(=\|%3d\|$&\|_mm\|cgi(\.\|-)\|inurl(:\|%3a)(/\|%2f)\|(mod\|path)(=\|%3d)(\.\|%2e)) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (<\|%3c)(.*)(e\|%65\|%45)(m\|%6d\|%4d)(b\|%62\|%42)(e\|%65\|%45)(d\|%64\|%44)(.*)(>\|%3e) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (<\|%3c)(.*)(i\|%69\|%49)(f\|%66\|%46)(r\|%72\|%52)(a\|%61\|%41)(m\|%6d\|%4d)(e\|%65\|%45)(.*)(>\|%3e) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (<\|%3c)(.*)(o\|%4f\|%6f)(b\|%62\|%42)(j\|%4a\|%6a)(e\|%65\|%45)(c\|%63\|%43)(t\|%74\|%54)(.*)(>\|%3e) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (<\|%3c)(.*)(s\|%73\|%53)(c\|%63\|%43)(r\|%72\|%52)(i\|%69\|%49)(p\|%70\|%50)(t\|%74\|%54)(.*)(>\|%3e) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (\+\|%2b\|%20)(d\|%64\|%44)(e\|%65\|%45)(l\|%6c\|%4c)(e\|%65\|%45)(t\|%74\|%54)(e\|%65\|%45)(\+\|%2b\|%20) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (\+\|%2b\|%20)(i\|%69\|%49)(n\|%6e\|%4e)(s\|%73\|%53)(e\|%65\|%45)(r\|%72\|%52)(t\|%74\|%54)(\+\|%2b\|%20) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (\+\|%2b\|%20)(s\|%73\|%53)(e\|%65\|%45)(l\|%6c\|%4c)(e\|%65\|%45)(c\|%63\|%43)(t\|%74\|%54)(\+\|%2b\|%20) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (\+\|%2b\|%20)(u\|%75\|%55)(p\|%70\|%50)(d\|%64\|%44)(a\|%61\|%41)(t\|%74\|%54)(e\|%65\|%45)(\+\|%2b\|%20) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (\\x00\|(\"\|%22\|\'\|%27)?0(\"\|%22\|\'\|%27)?(=\|%3d)(\"\|%22\|\'\|%27)?0\|cast(\(\|%28)0x\|or%201(=\|%3d)1) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (g\|%67\|%47)(l\|%6c\|%4c)(o\|%6f\|%4f)(b\|%62\|%42)(a\|%61\|%41)(l\|%6c\|%4c)(s\|%73\|%53)(=\|\[\|%[0-9A-Z]{0,2}) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (_\|%5f)(r\|%72\|%52)(e\|%65\|%45)(q\|%71\|%51)(u\|%75\|%55)(e\|%65\|%45)(s\|%73\|%53)(t\|%74\|%54)(=\|\[\|%[0-9A-Z]{2,}) [NC,OR] | 

| | RewriteCond %{QUERY_STRING} (j|%6a|%4a)(a|%61|%41)(v|%76|%56)(a|%61|%31)(s|%73|%53)(c|%63|%43)(r|%72|%52)(i|%69|%49)(p|%70|%50)(t|%74|%54)(:|%3a)(.)(;|%3b|)|%29) [NC,OR] | | | RewriteCond %{QUERY_STRING} (b|%62|%42)(a|%61|%41)(s|%73|%53)(e|%65|%45)(6|%36)(4|%34)(_|%5f)(e|%65|%45|d|%64|%44)(e|%65|%45|n|%6e|%4e)(c|%63|%43)(o|%6f|%4f)(d|%64|%44)(e|%65|%45)(.)($)(.)($) [NC,OR] | | | RewriteCond %{QUERY_STRING} (@copy|$(files|get|post)|allow_url(fopen|include)|auto_prepend_file|blexbot|browsersploit|call_user_func_array|(php|web)shell|curl(_exec|test)|disable_functions?|document_root) [NC,OR] | | | RewriteCond %{QUERY_STRING} (elastix|encodeuricom|exploit|fclose|fgets|file_put_contents|fputs|fsbuff|fsockopen|gethostbyname|grablogin|hmei7|hubs_post-cta|input_file|invokefunction|(\b)load_file|open_basedir|outfile|p3dlite) [NC,OR] | | | RewriteCond %{QUERY_STRING} (pass(=|%3d)shell|passthru|phpinfo|phpshells|popen|proc_open|quickbrute|remoteview|root_path|safe_mode|shell_exec|site((.){0,2})copier|sp_executesql|sux0r|trojan|udtudt|user_func_array|wget|wp_insert_user|xertive) [NC,OR] | | | RewriteCond %{QUERY_STRING} (;|<|>|'|"|)|%0a|%0d|%22|%27|%3c|%3e|%00)(.)(/*|alter|base64|benchmark|cast|concat|convert|create|encode|declare|delay|delete|drop|hex|insert|load|md5|null|replace|request|script|select|set|sleep|truncate|unhex|update) [NC,OR] |

|  | RewriteCond %{QUERY_STRING} ((\+\|%2b)(concat\|delete\|get\|select\|union)(\+\|%2b)) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (union)(.*)(select)(.*)(\(\|%28) [NC,OR] | 
|  | RewriteCond %{QUERY_STRING} (concat\|eval)(.*)(\(\|%28) [NC] | 
|  | RewriteRule .* - [F] | 

| | # RewriteRule .* /nG_log.php?log [END,NE,E=nG_QUERY_STRING:%1___%2___%3] | | | </IfModule> | | | # 8G:[REQUEST URI] | | | <IfModule mod_rewrite.c> |

|  | RewriteCond %{REQUEST_URI} (,,,) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (-------) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (\^\|`\|<\|>\|\\\|\\|) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} ([a-z0-9]{2000,}) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (=?\\(\'\|%27)/?)(\.) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(\*\|\"\|\'\|\.\|,\|&\|&?)/?$ [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (\.)(php)($)?([0-9]+)($)?(/)?$ [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} /((.*)header:\|(.*)set-cookie:(.*)=) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (\.(s?ftp-?)config\|(s?ftp-?)config\.) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(f?ckfinder\|fck/\|fckeditor\|fullclick) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)((force-)?download\|framework/main)(\.php) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (\{0\}\|\"?0\"?=\"?0\|\(/\(\|\.\.\.\|\+\+\+\|\\\") [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(vbull(etin)?\|boards\|vbforum\|vbweb\|webvb)(/)? [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (\.\|20)(get\|the)(_)(permalink\|posts_page_url)(\() [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (///\|\?\?\|/&&\|/\*(.*)\*/\|/:/\|\\\\\|0x00\|%00\|%0d%0a) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(cgi_?)?alfa(_?cgiapi\|_?data\|_?v[0-9]+)?(\.php) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (thumbs?(_editor\|open)?\|tim(thumbs?)?)((\.\|%2e)php) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)((boot)?_?admin(er\|istrator\|s)(_events)?)(\.php) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/%7e)(root\|ftp\|bin\|nobody\|named\|guest\|logs\|sshd)(/) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (archive\|backup\|db\|master\|sql\|wp\|www\|wwwroot)\.(gz\|zip) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(\.?mad\|alpha\|c99\|php\|web)?sh(3\|e)ll([0-9]+\|\w)(\.php) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(admin-?\|file-?)(upload)(bg\|_?file\|ify\|svu\|ye)?(\.php) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(etc\|var)(/)(hidden\|secret\|shadow\|ninja\|passwd\|tmp)(/)?$ [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (s)?(ftp\|http\|inurl\|php)(s)?(:(/\|%2f\|%u2215)(/\|%2f\|%u2215)) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(=\|\$&?\|&?(pws\|rk)=0\|_mm\|_vti_\|cgi(\.\|-)\|(=\|/\|;\|,)nt\.) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (\.)(ds_store\|htaccess\|htpasswd\|init?\|mysql-select-db)(/)?$ [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(bin)(/)(cc\|chmod\|chsh\|cpp\|echo\|id\|kill\|mail\|nasm\|perl\|ping\|ps\|python\|tclsh)(/)?$ [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(::[0-9999]\|%3a%3a[0-9999]\|127\.0\.0\.1\|ccx\|localhost\|makefile\|pingserver\|wwwroot)(/)? [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} ^(/)(123\|backup\|bak\|beta\|bkp\|default\|demo\|dev(new\|old)?\|new-?site\|null\|old\|old_files\|old1)(/)?$ [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)?j((\s)+)?a((\s)+)?v((\s)+)?a((\s)+)?s((\s)+)?c((\s)+)?r((\s)+)?i((\s)+)?p((\s)+)?t((\s)+)?(%3a\|:) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} ^(/)(old-?site(back)?\|old(web)?site(here)?\|sites?\|staging\|undefined\|wordpress([0-9]+)\|wordpress-old)(/)?$ [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(filemanager\|htdocs\|httpdocs\|https?\|mailman\|mailto\|msoffice\|undefined\|var\|vhosts\|webmaster\|www)(/) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} ($null$\|\{\$itemURL\}\|cast\(0x\|echo(.*)kae\|etc/passwd\|eval\(\|null(.*)null\|open_basedir\|self/environ\|\+union\+all\+select) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(db-?\|j-?\|my(sql)?-?\|setup-?\|web-?\|wp-?)?(admin-?)?(setup-?)?(conf\b\|conf(ig)?)(uration)?(\.?bak\|\.inc)?(\.inc\|\.old\|\.php\|\.txt) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)((.*)crlf-?injection\|(.*)xss-?protection\|__(inc\|jsc)\|author-panel\|cgi-bin\|database\|down\|(db\|mysql)-?admin)(/) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (/)(haders\|head\|hello\|helpear\|incahe\|includes?\|indo(sec)?\|infos?\|ioptimizes?\|jmail\|js\|king\|kiss\|kodox\|kro\|legion\|libsoft)(\.php) [NC,OR] | 

| | RewriteCond %{REQUEST_URI} (/)(awstats|document_root|dologin.action|error.log|extension/ext|htaccess.|lib/php|listinfo|phpunit/php|remoteview|server/php|www.root.) [NC,OR] |

|  | RewriteCond %{REQUEST_URI} (base64_(en\|de)code\|benchmark\|curl_exec\|echr\|eval\|function\|fwrite\|(f\|p)open\|html\|leak\|passthru\|p?fsockopen\|phpinfo)(.*)($\|%28)(.*)($\|%29) [NC,OR] | 
|  | RewriteCond %{REQUEST_URI} (posix_(kill\|mkfifo\|setpgid\|setsid\|setuid)\|(child\|proc)_(close\|get_status\|nice\|open\|terminate)\|(shell_)?exec\|system)(.*)($\|%28)(.*)($\|%29) [NC,OR] | 

| | RewriteCond %{REQUEST_URI} (/)((c99|php|web)?shell|crossdomain|fileditor|locus7|nstview|php(get|remoteview|writer)|r57|remview|sshphp|storm7|webadmin)(.)(.|%2e|(|%28) [NC,OR] | | | RewriteCond %{REQUEST_URI} /((wp-)((201\d|202\d|[0-9]{2})|ad|admin(fx|rss|setup)|booking|confirm|crons|data|file|mail|one|plugins?|readindex|reset|setups?|story))(.php) [NC,OR] | | | RewriteCond %{REQUEST_URI} (/)(^$|-|!|\w|.(.)|100|123|([^iI])?ndex|index.php/index|3xp|777|7yn|90sec|99|active|aill|ajs.delivery|al277|alexuse?|ali|allwrite)(.php) [NC,OR] | | | RewriteCond %{REQUEST_URI} (/)(analyser|apache|apikey|apismtp|authenticat(e|ing)|autoload_classmap|backup(index)?|bakup|bkht|black|bogel|bookmark|bypass|cachee?)(.php) [NC,OR] | | | RewriteCond %{REQUEST_URI} (/)(clean|cm(d|s)|con|connector.minimal|contexmini|contral|curl(test)?|data(base)?|db|db-cache|db-safe-mode|defau11|defau1t|dompdf|dst)(.php) [NC,OR] | | | RewriteCond %{REQUEST_URI} (/)(elements|emails?|error.log|ecscache|edit-form|eval-stdin|evil|fbrrchive|filemga|filenetworks?|f0x|gank(.php)?|gass|gel|guide)(.php) [NC,OR] | | | RewriteCond %{REQUEST_URI} (/)(logo_img|lufix|mage|marg|mass|mide|moon|mssqli|mybak|myshe|mysql|mytag_js?|nasgor|newfile|news|nf?tracking|nginx|ngoi|ohayo|old-?index)(.php) [NC,OR] | | | RewriteCond %{REQUEST_URI} (/)(olux|owl|pekok|petx|php-?info|phpping|popup-pomo|priv|r3x|radio|rahma|randominit|readindex|readmy|reads|repair-?bak|robot(s.txt)?|root)(.php) [NC,OR] | | | RewriteCond %{REQUEST_URI} (/)(router|savepng|semayan|shell|shootme|sky|socket(c|i|iasrgasf)ontrol|sql(bak|?dump)?|support|sym403|sys|system_log|test|tmp-?(uploads)?)(.php) [NC,OR] | | | RewriteCond %{REQUEST_URI} (/)(traffic-advice|u2p|udd|ukauka|up__uzegp|up14|upa?|upxx?|vega|vip|vu(ln)?(\w)?|webroot|weki|wikindex|wordpress|wp_logns?|wp_wrong_datlib)(.php) [NC,OR] | | | RewriteCond %{REQUEST_URI} (/)(wp-?install|installation|wp(3|4|5|6)|wpfootes|wpzip|ws0|wsdl|wso(\w)?|www|(uploads|wp-admin)?xleet(-shell)?|xmlsrpc|xup|xxu|xxx|zibi|zipy)(.php) [NC,OR] | | | RewriteCond %{REQUEST_URI} (bkv74|cachedsimilar|core-stab|crgrvnkb|ctivrc|deadcode|deathshop|dkiz|e7xue|eqxafaj90zir|exploits|ffmkpcal|filellli7|(fox|sid)wso|gel4y|goog1es|gvqqpinc) [NC,OR] | | | RewriteCond %{REQUEST_URI} (@md5|00.temp00|0byte|0d4y|0day|0xor|wso1337|1h6j5|3xp|40dd1d|4price|70bex?|a57bze893|abbrevsprl|abruzi|adminer|aqbmkwwx|archivarix|backdoor|beez5|bgvzc29) [NC,OR] | | | RewriteCond %{REQUEST_URI} (handler_to_code|hax(0|o)r|hmei7|hnap1|home_url=|ibqyiove|icxbsx|indoxploi|jahat|jijle3|kcrew|keywordspy|laobiao|lock360|longdog|marijuan|mod(aratic|ariimag)) [NC,OR] | | | RewriteCond %{REQUEST_URI} (mobiquo|muiebl|nessus|osbxamip|phpunit|priv8|qcmpecgy|r3vn330|racrew|raiz0|reportserver|r00t|respectmus|rom2823|roseleif|sh3ll|site((.){0,2})copier|sqlpatch|sux0r) [NC,OR] | | | RewriteCond %{REQUEST_URI} (sym403|telerik|uddatasql|utchiha|visualfrontend|w0rm|wangdafa|wpyii2|wsoyanzo|x5cv|xattack|xbaner|xertive|xiaolei|xltavrat|xorz|xsamxad|xsvip|xxxs?s?|zabbix|zebda) [NC,OR] | | | RewriteCond %{REQUEST_URI} (.)(7z|ab4|ace|afm|alfa|as(h|m)x?|aspx?|aws|axd|bash|ba?k?|bat|bz2|cfg|cfml?|cgi|cms|conf\b|config|ctl|dat|db|dist|dll|eml|eng(ine)?|env|et2|exe|fec|fla|git(ignore)?)$ [NC,OR] | | | RewriteCond %{REQUEST_URI} (.)(hg|idea|inc|index|ini|inv|jar|jspa?|lib|local|log|lqd|make|mbf|mdb|mmw|mny|mod(ule)?|msi|old|one|orig|out|passwd|pdb|php.(php|suspect(ed)?)|php([^/])|phtml?|pl|profiles?)$ [NC,OR] | | | RewriteCond %{REQUEST_URI} (.)(psd|pst|ptdb|production|pwd|py|qbb|qdf|rar|rdf|remote|save|sdb|sql|sh|soa|svn|swf|swl|swo|swp|stx|tar|tax|tgz?|theme|tls|tmb|tmd|wok|wow|xsd|xtmpl|xz|ya?ml|za|zlib)$ [NC] |

|  | RewriteRule .* - [F] | 
|  | # RewriteRule .* /nG_log.php?log [END,NE,E=nG_REQUEST_URI:%1___%2___%3] | 

| | </IfModule> | | | # 8G:[USER AGENT] | | | <IfModule mod_rewrite.c> |

|  | RewriteCond %{HTTP_USER_AGENT} ([a-z0-9]{2000,}) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (<\|%0a\|%0d\|%27\|%3c\|%3e\|%00\|0x00\|\\\x22\|\\\") [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (ahrefs\|archiver\|curl\|libwww-perl\|pycurl\|scan) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (oppo\sa33\|(c99\|php\|web)shell\|site((.){0,2})copier) [NC,OR] | 
|  | RewriteCond %{HTTP_USER_AGENT} (base64_decode\|bin/bash\|disconnect\|eval\|unserializ) [NC,OR] | 

| | RewriteCond %{HTTP_USER_AGENT} (acapbot|acoonbot|alexibot|antbot|asterias|attackbot|awario|babbar|backdor|barkrowler|becomebot|binlar|bitsightbot|blackwidow|blekkobot|blex|blowfish|bullseye|bunnys|butterfly|bytespider|careerbot|casper|censysinspect) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (checkpriv|cheesebot|cherrypick|chinaclaw|choppy|claudebot|clshttp|cmsworld|commoncrawl|copernic|copyrightcheck|cosmos|crawlergo|crescent|criteo|datacha|dataforseo|(\b)demon(\b)|diavol|discobot|dittospyder) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (dotbot|dotnetdotcom|dumbot|econtext|emailcollector|emailsiphon|emailwolf|eolasbot|eventures|extract|eyenetie|feedfinder|flaming|flashget|flicky|foobot|fuck) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (g00g1e|getright|gigabot|go-ahead-got|go-http-client|gozilla|grabnet|grafula|harvest|heritrix|hrankbot|httracks?|icarus6j|imagesiftbot|intelx|jetbot|jetcar|jikespider|kmccrew|leechftp|libweb|liebaofast) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (linkscan|linkwalker|lwp-download|majestic|masscan|mauibot|miner|mechanize|mj12bot|morfeus|moveoverbot|mozlila|netmechanic|netspider|nicerspro|nikto|ninja|nominet|nutch) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (octopus|pagegrabber|petalbot|planetwork|postrank|proximic|purebot|queryn|queryseeker|radian6|radiation|realdownload|remoteview|rogerbot|scan|scooter|scrapy|seekerspid) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (semalt|seokicks|seznambot|siclab|sindice|sistrix|sitebot|siteexplorer|sitesnagger|skygrid|smartdownload|snoopy|sosospider|spankbot|spbot|sqlmap|stackrambler|stripper|sucker|surftbot) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (sux0r|suzukacz|suzuran|takeout|teleport|telesoft|true_robots|turingos|turnit|tweetmemebot|vampire|vikspider|voideye|webleacher|webreaper|webstripper|webvac|webviewer|webwhacker|wellknownbot) [NC,OR] | | | RewriteCond %{HTTP_USER_AGENT} (winhttp|wwwoffle|woxbot|xaldon|xxxyy|yamanalab|yioopbot|yisouspider|youda|zeus|zmeu|zoominfobot|zune|zyborg) [NC] |

|  | RewriteRule .* - [F] | 
|  | # RewriteRule .* /nG_log.php?log [END,NE,E=nG_USER_AGENT:%1] | 

| | </IfModule> | | | # 8G:[REMOTE HOST] | | | <IfModule mod_rewrite.c> | | | RewriteCond %{REMOTE_HOST} (163data|amazonaws|colocrossing|crimea|g00g1e|justhost|kanagawa|loopia|masterhost|onlinehome|poneytel|sprintdatacenter|reverse.softlayer|safenet|ttnet|woodpecker|wowrack) [NC] |

|  | RewriteRule .* - [F] | 
|  | # RewriteRule .* /nG_log.php?log [END,NE,E=nG_REMOTE_HOST:%1] | 

| | </IfModule> | | | # 8G:[HTTP REFERRER] | | | <IfModule mod_rewrite.c> |

|  | RewriteCond %{HTTP_REFERER} (order(\s\|%20)by(\s\|%20)1--) [NC,OR] | 
|  | RewriteCond %{HTTP_REFERER} (@unlink\|assert\(\|print_r\(\|\\x00\|%00\|xbshell) [NC,OR] | 

| | RewriteCond %{HTTP_REFERER} (100dollars|best-seo|blue\spill|cocaine|ejaculat|erectile|erections|hoodia|huronriveracres|impotence|levitra|libido|lipitor|mopub.com|phentermin) [NC,OR] | | | RewriteCond %{HTTP_REFERER} (pornhelm|pro[sz]ac|sandyauer|semalt.com|social-buttions|todaperfeita|tramadol|troyhamby|ultram|unicauca|valium|viagra|vicodin|xanax|ypxaieo) [NC] |

|  | RewriteRule .* - [F] | 
|  | # RewriteRule .* /nG_log.php?log [END,NE,E=nG_HTTP_REFERRER:%1] | 

| | </IfModule> | | | # 8G:[HTTP COOKIE] | | | <IfModule mod_rewrite.c> |

|  | RewriteCond %{HTTP_COOKIE} (<\|>\|\'\|%0A\|%0D\|%27\|%00) [NC] | 
|  | RewriteRule .* - [F] | 
|  | # RewriteRule .* /nG_log.php?log [END,NE,E=nG_HTTP_COOKIE:%1] | 

| | </IfModule> | | | # 8G:[REQUEST METHOD] | | | <IfModule mod_rewrite.c> |

|  | RewriteCond %{REQUEST_METHOD} ^(connect\|debug\|move\|trace\|track) [NC] | 
|  | RewriteRule .* - [F] | 
|  | # RewriteRule .* /nG_log.php?log [END,NE,E=nG_REQUEST_METHOD:%1] | 

| | </IfModule> | | | # Block wp2shell REST batch API probing | | | <IfModule mod_rewrite.c> | | | RewriteEngine On | | | # Block the pretty REST endpoint | | | RewriteRule ^wp-json/batch/v1/?$ - [F,L,NC] | | | # Block the query-string endpoint, including URL-encoded slashes |

|  | RewriteCond %{QUERY_STRING} (^\|&)rest_route=(/\|%2F)batch(/\|%2F)v1(&\|$) [NC] | 
|  | RewriteRule ^ - [F,L] | 

| | </IfModule> | | | # PHP2Shell defense in depth: reject direct PHP script requests.*** | | | <IfModule mod_rewrite.c> | | | RewriteEngine On | | | RewriteRule .(php[0-9]?|phtml|phar)(/|$) - [F,END,NC] | | | </IfModule> |

── more in #ai-crawlers 4 stories · sorted by recency
── more on @anthropic 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/htaccess-mashup] indexed:0 read:8min 2026-10-27 · —