GHOSTPRINT — Can You Identify an AI by Its Writing?

An independent experiment called GHOSTPRINT reached about 31% accuracy in cross-validation when trying to identify which of 18 language models produced a given response, using text analysis and machine learning. The project, part of the author's Sandbox Memory Mad Labs, is described as an early-stage research experiment with room for improvement, and the author asked whether others have tried model fingerprinting or identifying LLMs from their writing.

Hey everyone Here’s another experiment I’ve been working on called GHOSTPRINT, part of my Sandbox Memory Mad Labs. The idea came from a question I found interesting. If you took away the model’s name and just looked at its responses, could you figure out which AI wrote them? Different language models have their own patterns, but how much of that can we actually detect? I started experimenting with a dataset of responses from 18 different models, using text analysis and machine learning to see if I could identify the model behind each answer. My early baseline reached about 31% accuracy in cross-validation, so there’s definitely room for improvement. It’s still a research experiment, and I’ve been having fun figuring out what works and what doesn’t. Curious if anyone here has tried model fingerprinting or identifying LLMs from their writing. What techniques worked best for you?.