Is Open Access to Information Still Open?

Legal rules, technical standards and platform policies are converging to make publicly available web data more conditional and harder to access, according to an analysis citing the EU Copyright Directive's text and data mining provisions, the EU General-Purpose AI Code of Practice, the Internet Engineering Task Force's AI Preferences Vocabulary draft, and Cloudflare's default AI crawler blocking. The piece argues the shift from an open web toward a permission-based environment threatens search, journalism, academic research and public-interest monitoring, and that restricting AI training to licensed data risks less representative models. It warns the burden of conditional access will not fall equally across the digital landscape.

Is Open Access to Information Still Open? Legal rules, technical standards and platform policies are making public data harder to access, with potential consequences for trustworthy AI and sustainability monitoring. Access to information depends on something quite simple: the ability to reach what is already there. For much of the internet's history, that principle felt stable. If something was publicly available online, it could be found, indexed, analyzed, archived, and used to create new knowledge, services, and tools. That openness helped make the web one of the most important shared resources modern society has built. That is why the current shift matters so much. Across the internet, new restrictions are emerging that make access to public information more conditional than it used to be. Some come through law and regulation, some through technical standards, and some through the growing power of infrastructure providers and platforms. Viewed separately, these changes can look narrow or highly technical. Taken together, they point to a broader transition: from an open web toward a more permission-based digital environment. That transition matters far beyond the AI sector. Publicly available web data supports search, journalism, academic research, accessibility tools, market transparency, and many forms of public-interest monitoring. It is part of the informational infrastructure through which societies understand themselves. If meaningful access to that layer becomes fragmented or prohibitively expensive, the consequences will extend https://uk.finance.yahoo.com/news/oxycon-2026-experts-warn-open-141700267.html well beyond a dispute between businesses hosting content on the public web and those accessing it. Restrictions are emerging from several directions The current debate is often framed around AI, but the deeper issue is how many different layers of restriction are now converging at once. In Europe, the legal environment around text and data mining has made the question of access more uncertain, especially as policymakers and rights holders debate how opt-outs should be expressed and enforced under the EU Copyright Directive https://eur-lex.europa.eu/eli/dir/2019/790/oj . At the same time, the EU's General-Purpose AI Code of Practice https://digital-strategy.ec.europa.eu/en/policies/contents-code-gpai is adding new compliance expectations around transparency and copyright for AI model providers. Technical governance is moving in the same direction. The Internet Engineering Task Force's AI Preferences Vocabulary https://datatracker.ietf.org/doc/draft-ietf-aipref-vocab/ may appear to be a narrow standards discussion, but such standards often become the practical foundation for future platform rules and even regulation. How content owners express preferences in machine-readable form may soon play a much bigger role in deciding what kinds of access remain possible in practice. Website hosting services are also becoming far more influential in setting these conditions. With its default AI crawler blocking, Cloudflare, previously a reluctant moderator https://informationlabs.org/walling-off-the-web-how-cloudflares-plan-to-save-content-could-break-the-internet/ , decisively made all content more dependent on it. The real risk is unequal access Legitimate concerns regarding attribution, compensation, and responsible use exist among creators, yet the shift toward making access conditional by default creates a burden that will not fall equally across the digital landscape. A big challenge for AI is ensuring it's not biased towards certain cultures, races, or viewpoints. To make AI models, especially large language models, as fair as possible, they need to be trained on really big datasets. The internet could be said to be the biggest one. So, when someone suggests limiting AI training to licensed data or restricting access to web data, we should consider that truly representative and useful AI depends on access to the public web. What makes this even harder is that the public web is shrinking with more and more information being put behind logins. Oxylabs research uncovered https://oxylabs.io/press-area/ai-and-cost-of-information-access that 74% of platforms now require registration even for nominally free content while paid content subscription prices rose 17% since 2020. The more of information is becoming privileged in this way, the more important it becomes what is actually accessible on the shrinking public web. For example, research shows that many reputable websites block automated crawlers, whereas fewer misinformation sites do. A study https://arxiv.org/abs/2510.10315 found that 60% of reputable sites stop at least one AI crawler, but only 9.1% of misinformation sites do. This creates a situation where AI models could be trained on data that leans more towards misinformation than towards trusted sources. Large corporations typically have the resources to negotiate licenses, cover legal costs, and build complex systems to deal with fragmented rules. But researchers, journalists, and startups often don't have these resources. They depend heavily on a web that remains genuinely accessible in practice, not just in theory, to conduct public-interest monitoring and academic inquiry. The future of open access constitutes a broad question about the conditions and beneficiaries of public information work. Ongoing legal disputes in the United States, such as lawsuits against Anthropic https://mashable.com/tech/judge-allows-most-of-reddit-lawsuit-against-anthropic-proceed and other AI developers, demonstrate that the definition of public access remains contested even as what once seemed settled grows increasingly uncertain. Why this matters for sustainability A lot of environmental harm shows up online before official institutions catch it. Protected wildlife sold on a marketplace. An unlicensed waste service operating quietly in the background. A green claim that a company cannot actually support. A patch of valuable land listed for sale before anyone questions what may happen to it. In cases like these, the information is often already public. What is missing is the capacity to find it, verify it, and act on it in time. That is why open access to public web data matters for sustainability. It helps smaller teams monitor risks that would otherwise be easy to miss. Researchers, journalists, nonprofits, and watchdog organizations often lack the resources to access expensive commercial datasets or to run large monitoring operations. They rely on the public web to see patterns early and respond before the damage is harder to undo. Europe is starting to recognize this more clearly, too. When policymakers talk about competitiveness alongside the green and digital transitions https://www.consilium.europa.eu/en/press/press-releases/2025/10/23/european-council-conclusions-on-competitiveness-and-twin-transition/ , they are also acknowledging that environmental goals depend, in part, on digital capacity: the ability to see what is happening, understand it, and respond. This becomes especially tangible in areas where protection is still limited. In Lithuania, forests cover 33.9% of the country, but only 1.2% are protected from cutting, and the average forest age is 54 years. That is exactly why the work of Sengirės fondas https://www.sengiresfondas.lt/en/ Ancient Woods Foundation is so important. When valuable natural areas can still be sold, cleared, or overlooked, access to public information becomes part of environmental protection itself. The European Commission https://digital-strategy.ec.europa.eu/en/policies/green-digital estimates digital technology already accounts for 8 to 10 percent of Europe's electricity and 2 to 4 percent of its emissions, and that's before you count the water used for cooling and the electronic waste https://ewastemonitor.info/the-global-e-waste-monitor-2024/ piling up faster than we recycle it. So the point is not to collect everything possible. The point is to keep access open enough for useful, responsible, and proportionate data collection, especially where it serves public-interest goals. Preserving openness while building better standards Defining responsible data-collection practices remains essential. Frameworks must move beyond simple blanket restrictions that disproportionately harm smaller organizations and public-interest researchers, favoring instead the development of standards that distinguish between wasteful data hoarding and essential, public-good discovery. Ultimately, the objective is to establish a sustainable model that preserves open access and creates significant value. © Copyright IBTimes 2026. All rights reserved.