Crawl of Fame - October 2026

Reddit will stop accepting new public API access requests on October 31, halt RSS feeds on November 13, 2026, begin removing unregistered apps' API access on January 12, 2027, and close public API access entirely in March 2027, citing RSS as "a common surface for large-scale scraping and automated abuse." Reddit's data-licensing revenue, which includes licensing deals with Google and OpenAI, grew 24% year over year to $43 million in Q2 2026, while the only remaining official paths after March 2027 are a Devvit app, a licensing deal, or nothing. Separately, a user named datasocial published the metadata of 5.6 billion TikTok videos on Hugging Face on October 3, spread across 148 parquet files covering July 2014 to October 2026.

Welcome back to the Crawl of Fame; in this October edition, we’ll talk about social networks but from different angles. Let’s start from Reddit, which announced the end of its RSS feed and its public API, making it more difficult to scrape. A few days later, a small company published the metadata of 5.6 billion TikTok videos on Hugging Face, which is remarkable since TikTok is known for being a hard target to scrape. Then, from the legal perspective, LinkedIn closed another lawsuit against a scraping company, and we’ll dive a bit into it. Give your AI a web data layer – Decodo’s Web Scraping API https://visit.decodo.com/rE0PQB turns any site into clean, structured data your models can actually use. Reddit closes RSS on November 13 and the public API in March 2027 On September 30, Reddit published two posts, one in r/modnews https://www.reddit.com/r/modnews/comments/1wubgvt/continuing our infrastructure updates whats/ for moderators and one in r/redditdev https://www.reddit.com/r/redditdev/comments/1wubcvf/moving data api apps to the developer platform/ for developers. Starting from October 31, Reddit won’t accept new requests for public API access, and from November 13, RSS feeds will stop working. Then, starting January 12, 2027, Reddit begins removing API access for apps and users that haven’t registered, and in March 2027, public access closes to everyone. The reason is obvious and clearly stated by the moderators: “Because RSS is now a common surface for large-scale scraping and automated abuse, we’ll stop supporting RSS feeds on November 13, 2026”. This means that apps and bots should move to the Developer Platform Devvit managed by Reddit itself, rather than being hosted on third-party servers. And there will be no replacement for RSS feeds. In addition, in the past months, Reddit already required logging in to the “old” Reddit endpoint, which was used as a backdoor by scrapers. What’s left for people who collect Reddit data After March 2027, the official paths are three: a Devvit app, a data licensing deal, or nothing. Devvit is made to build apps that run inside Reddit communities; it uses TypeScript, and Reddit told PPC Land https://ppc.land/reddit-to-halt-public-data-api-access-in-march-2027-as-14-000-apps-register/ that “adding additional language support has not been a scalable thing for us,” which is bad news for the many bots written with PRAW. The licensing deals, of course, are where Reddit is pushing. A look at their balance sheet shows that the “other revenue” line, which includes data licensing, grew 24% year over year to $43 million in Q2 2026, according to TechCrunch https://techcrunch.com/2026/09/30/reddit-is-killing-rss-feeds-ending-public-api-access-because-of-ai-bots/ . This is the way Google and OpenAI gather data, and they’re already paying for it. For everyone else, the web version is what remains, and today it’s already behind a JavaScript check. In my opinion, from next spring Reddit scraping becomes a browser job with good residential proxies, and probably with logged-in accounts for anything that goes beyond the public listing. That’s way more expensive per page than an RSS feed, so I expect many small social listening tools and research projects to pay for a vendor or to stop collecting. 5.6 billion TikTok videos on Hugging Face On October 3, a user called datasocial published tiktok-5.6B-videos https://huggingface.co/datasets/datasocial/tiktok-5.6B-videos on Hugging Face. Yes, it’s a dataset about 5.6 billion videos of TikTok, but there are just the metadata, not the videos themselves. This means we have ID, publish date, country, language, caption, hashtags, mentions, on-screen text, sound ID, ad and TikTok Shop flags, the AI-generated flag when TikTok provides it, and the counts of views, likes, comments, shares, saves and downloads, with the date when they were read. The files are 148 parquet files, one per month from July 2014 to October 2026, for 460.5 GB in total. How it was collected The dataset card links to a long technical guide https://datasocial.ai/writing/scraping-tiktoks-mobile-api on datasocial.ai, and for us this is the most interesting part. Instead of scraping the website, they used the private API of TikTok’s Android app, with 24 endpoints. To make it work, they register fake devices built from a catalog of about 250 real Android profiles, they sign every request with the same headers the app uses X-Argus, X-Ladon and the others , and they copy the TLS fingerprint of Android with uTLS. There’s no login anywhere, so everything behind an account is out of reach. Two details from the guide are useful for anyone who works on mobile APIs. The first is that a failure is often a clean HTTP 200 with an empty body, and the guide counts four unrelated causes that produce it. The second is the proxy: since the rate limit is per exit IP, they use a rotating datacenter proxy at about $150 per month, with success rates between 88.2% and 99.3% depending on how you rotate and retry. With this setup, they write they collected 3.23 billion creator profiles, 5.94 billion videos and 2.8 billion comments in three weeks. I didn’t verify these numbers beyond the 5.6 billion rows of the dataset, but if they’re correct, it’s an impressive result for the cost of a datacenter proxy plan. The guide is definitely worth reading, it could be helpful for other apps too. LinkedIn wins against ProAPIs, and the scraper must delete everything On September 16, Judge P. Casey Pitts of the Northern District of California signed a final judgment on consent https://storage.courtlistener.com/recap/gov.uscourts.cand.457309/gov.uscourts.cand.457309.36.0.pdf in the case LinkedIn Corporation v. ProAPIs Inc. I found it through The Record https://therecord.media/linkedin-wins-court-order-blocking-mass-scraping , then I read the documents on CourtListener https://www.courtlistener.com/docket/71527603/linkedin-corporation-v-proapis-inc/ . The defendants are ProAPIs, a Delaware company, Netswift, a Pakistani company founded in 2022, and their CEO Rehmat Alam. LinkedIn filed the complaint https://storage.courtlistener.com/recap/gov.uscourts.cand.457309/gov.uscourts.cand.457309.1.0.pdf on October 2, 2025, so the case lasted less than a year and never went to trial. The defendants never filed an answer and still deny liability, but they agreed to the judgment as part of a settlement. The complaint is the interesting part for us, because it describes the business in detail. ProAPIs sold the iScraper API, which promised to “scrape LinkedIn data efficiently in real-time and at scale” at up to “150 requests / second”, with a plan at $15,000 per month for 5,000,000 API calls. According to LinkedIn, the data came from fake accounts: they created email addresses with automated tools, registered LinkedIn accounts with false names and stock photos, and used them to scrape while logged in. LinkedIn writes that it detects and restricts these accounts “within a few hours”, but in that time each account could scrape hundreds of profiles, and ProAPIs registered “hundreds if not thousands of new accounts per day”. In total, LinkedIn says they used more than a million fake accounts. They also scraped the profiles that members set as visible only to logged-in users, which contradicts the claim of ProAPIs that they only collected “data that’s public on personal profiles”. It’s a clear case of massive scraping behind the login, something that is dangerous to do from a legal perspective, and this case confirms it. The defendants are permanently barred from accessing LinkedIn with fake accounts, scrapers or bots, from circumventing any technical measure, and from selling or giving away LinkedIn data or any information derived from it the text says “including for free” . They also can’t use any third-party service that provides LinkedIn data or accesses LinkedIn in an automated way, so they can’t simply become customers of another vendor. They certified, under penalty of perjury, that they deleted all the data, the reports made with it, the scraping code and the SDKs and APIs used to deliver it, and that they removed from their website every reference to LinkedIn scraping, “regardless of whether LinkedIn is referred to by name”. For me, this case follows a pattern we have already seen. In the hiQ case, the court found a breach of the User Agreement for the same reason: the fake accounts used to scrape behind the login I wrote about it in my article on LinkedIn’s hidden APIs https://www.scraping.club/p/scraping-linkedin-public-data . Proxycurl was sued by LinkedIn in January 2025 and shut down on July 4, 2025 https://nubela.co/blog/goodbye-proxycurl/ , and its founder explained that even winning wouldn’t pay the legal fees, while “LinkedIn, owned by Microsoft, has more or less an unlimited war chest”. ProAPIs settled in less than a year. In all three cases, the weak point is the same: when you create an account, you accept the User Agreement, and from that moment scraping becomes a breach of contract, before any discussion about public data. So if you buy LinkedIn data from a vendor today, ask them where it comes from, because a judgment like this one can close that source in a few months. For this month, that’s it; see you in November with the next edition of the Crawl of Fame.