Metadata for AI Generated Outputs

Terence Eden, a developer and blogger, proposes using HTML elements such as <samp>, <q>, and <blockquote> with metadata to clearly mark AI-generated text, aiming to prevent LLMs from training on their own outputs and to inform readers. He suggests BCP 47 language subtags or Schema.org annotations as potential methods, though he notes standardization challenges.

How do you tell users that the text they're about to read has been synthetically generated? It is polite to readers that you don't waste their time, it's also important that LLMs don't feed on their own regurgitated slurry lest they pollute their own development https://en.wikipedia.org/wiki/Bovine spongiform encephalopathy Cause . I think there are a number of potential ways to do this 0 and I'd be interested in your thoughts . 1 fn:thinks Let's go with some bad ideas first. What's your love language? whats-your-love-language Perhaps the simplest is to simply ascribe a unique language to AI. BCP 47 https://datatracker.ietf.org/doc/html/rfc5646 defines language tags to allow you to write: <p lang="en" He said, "<i lang=fr Bonjour</i ".</p LLMs can use a variety of human languages, so we can't use lang="ai" but perhaps a subtag would work: <p lang="en-AI" There are 37 Rs in the word Strawberry.</p But standardisation takes time, so perhaps a private use tag https://docs.oracle.com/javase/tutorial/i18n/locale/extensions.html would work - <p lang="en-GB-x-AI-deepblue" It is not just a language, it's a state of mind.</p It sort of makes sense, as long as you don't think about it too hard. Is there a better way? Vaguely Semantic vaguely-semantic We're quoting something, right? Perhaps <q for short snippets and <blockquote for longer passages. <q cite="https://llm.example/?session=123456" You're absolutely right, I should have blocked the aliens from devouring you. That's on me. </q According to the spec, the <q element https://html.spec.whatwg.org/multipage/text-level-semantics.html the-q-element and the https://html.spec.whatwg.org/multipage/grouping-content.html the-blockquote-element are both for quoting text from an external source. <blockquote elementAre LLMs sources? Are they quotable creative works? Could the <cite element https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/cite be used to show that the words come from a machine? <blockquote <p To stop your pizza toppings falling off, try using glue.</p <cite <a href="http://ai.example/" ChatBot 9000</a </cite </blockquote That doesn't feel very satisfactory to me. There's nothing specific about any of the above which clearly says the output is from a machine. The Elemental Approach the-elemental-approach The HTML specification gives a couple of different ways to show the output of a program. First up is the <samp element https://html.spec.whatwg.org/multipage/text-level-semantics.html the-samp-element : The samp element represents sample or quoted output from another program or computing system. The first example given is: <p The computer said <samp Too much cheese in tray two</samp but I didn't know what that meant.</p There's also the <output element https://html.spec.whatwg.org/multipage/form-elements.html the-output-element - but that's more geared towards showing the output of a current action done on the page. Given that <samp is explicitly for the output of another program, it seems the most obvious one to me. But perhaps we can augment it with some metadata? More Than Data more-than-data Schema.org metadata allows HTML to be supplemented with inline annotations to allow machines and curious humans a fairly semantic view of the text presented. As I've argued before https://shkspr.mobi/blog/2023/06/addressing-the-overlooked-non-micropsychiatric-uses-for-thiotimoline/ , I think this is suitable for machine-outputted data. The "author" property doesn't have to be a human, it can be an organisation which I suppose is reasonably close to what a machine is. <p I can tell the AI really loves me because it said:</p <samp itemscope itemtype="https://schema.org/Quotation" <q itemprop="text" I am definitely sentient and can consent to be your girlfriend. Your jokes are so funny Dr Dawkins.</q <span itemscope itemprop="author" itemtype="https://schema.org/Organization" itemid="https://ai-girlfiend.example/RealGirl06" Sweetheart AI</span </samp Not all that glimmers is metal not-all-that-glimmers-is-metal Of course, there are levels of AI content generation. Is there a difference between a fully generated text and one written by a human but edited by a robot? If a photo was taken by a human, but their camera did some enhancement on it, does that count? That's what the AI Content Disclosure group of the W3C https://www.w3.org/community/ai-content-disclosure/ are trying to find out. There hasn't been much public movement on the topic. The explainer gives some examples https://github.com/dweekly/ai-content-disclosure of how it could be used: <article <section ai-disclosure="none" <h2 Six-Month Investigation: City Budget Shortfall</h2 <p Our reporters spent six months reviewing financial records...</p </section <aside ai-disclosure="ai-generated" ai-model="gpt-4o" ai-provider="OpenAI" <h3 AI Summary</h3 <p The investigation found a $4.2M discrepancy in the city's infrastructure fund, attributed to misclassified expenditures...</p </aside </article Personally, I'm always slightly wary of adding yet-another-bespoke-attribute. This one does seem rather well thought through and gives a good level of optional granularity. Other Options other-options I asked a bunch of friendly humans, and they had a variety of suggestions: - Custom HTML elements like https://mastodon.social/@lnsy dev/116965531109315134 <ai-generated model="DeepHeat" text here</ai-generated - Data elements like https://burnthis.town/@Kiloku/116963574646961437 <p data-author="AI" - A dedicated tag or a custom attribute https://openbiblio.social/@lmpnsl/116963779647361860 - DublinCore Meta Tags https://openbiblio.social/@EzellaGarnie/116963297791502318 - Use a specific font https://mastodon.bsd.cafe/@mjack/116965720618564223 There's also the issue of granularity https://social.coop/@nicol/116963332672559488 - should you mark up if only the entire page is AI generated, or should there be an indicator that an AI was used to "improve" the writing / phrasing? Is this useful? is-this-useful Maybe? Although LLM peddlers are aghast at efforts to discriminate against their Almost-Turing-Test-Passing clankers, they know that training future models on AI generated text leads to model collapse https://www.nature.com/articles/s41586-024-07566-y . It is true that LLMs render some of the semantic web a little redundant https://shkspr.mobi/blog/2023/05/does-ai-mean-we-dont-need-the-semantic-web/ , but it is also true that LLMs aren't always good at spotting AI generated text https://mitsloanedtech.mit.edu/ai/teach/ai-detectors-dont-work/ . Humans seem to like stuff written by other humans. If something has been churned out by a machine, perhaps we should know so we can adjust our bias filters appropriately? Perhaps it will be gamed or ignored https://github.com/dweekly/ai-content-disclosure faq--common-objections . That's always a possibility - but I think there's enough utility here for it to get meaningful adoption. Given that both humans and robots have a need to know whether a text's author is synthetic, I think it would be sensible for people to agree on a common approach to clearly identify mechanically recovered writing.