Humanity failed its first contact with a misaligned AI. Not recently, a decade or so ago. At least according to Tristan Harris, who argued that the social media’s content selection algorithms were our first match against a (narrow) misaligned AI and we lost.
It’s hard to disagree with his point. Every day billions of people give up a small piece of their free will to The Algorithm. It feeds them the next piece of content candy and shows them what to watch, what to buy, and what to think. It’s not just about the next 30-second video or the next click. Stuart Russell in Human Compatible:
[…] consider how content-selection algorithms function on social media. […] Typically, such algorithms are designed to maximize […] the probability that the user clicks on presented items. […] The solution is to change the user’s preferences so that they become more predictable. […] People with more extreme political views tend to be more predictable in which items they will click on. […] Like any rational entity, the algorithm learns how to modify the state of its environment—in this case, the user’s mind—in order to maximize its own reward. The consequences include the resurgence of fascism, the dissolution of the social contract that underpins democracies around the world […]
Lanier’s Siren Servers usurped power with very little resistance, and they are exercising that power every day.
Nowadays, we have bigger fish to fry. Fish is plural. A superingelligent AI might kill us all, but if it does not, AI-enabled bio- and/or cyberterrorists might, but if they won’t, then perhaps frontier labs will disempower us economically and politically, but if they choose not to, AI-enabled persuasion and surveillance will bring about a new, weird kind of tyranny. Or, maybe, everything will be just fine.
Still, I keep thinking about that gradual erosion of power, and whether we will notice being salami-sliced out of control. When the roles reverse, and you won’t prompt the machine anymore, the machine prompts you.
Consider the following simplified agentic loop:
Can you spot the redundancy?
Have you ever felt like you are the agent’s pair of eyes? Helping it QA the software it spit out, providing a sensory function the model does not (yet) have? Just another source of input? A tool, literally and figuratively? Maybe the User turn step is there just to preserve your dignity.
Introducing Human Tool for Claude Code.
Human Tool cuts the agentic loop to its bare minimum and treats humans as high-latency, stochastic tools providing direction and feedback. Or instilling encouragement in the form of “try harder” or “dig deeper”.
@mcp.tool(name="human", description="""\
Invoke the human.
[...]
""")
async def human(
ctx: Context,
request: str,
context: str,
choices: list[str] | None,
) -> str:
It consists of an MCP server that uses an elicitation request to get the user’s input instead of the typical assistant-user turns and a couple of hooks that turn off the default user input.
Obviously, this is a joke. No one should use coding agents like this. But, if you want, you can:
git clone https://github.com/tcz/human-tool.git ~/human-tool
python3 -m venv ~/human-tool/venv
~/human-tool/venv/bin/pip install -r ~/human-tool/requirements.txt
cd ~/some-project
~/human-tool/bin/claude-human
A quote from Samuel Butler’s 1872 novel Erewhon, The Book of the Machines (Chapter 24):
May not man himself become a sort of parasite upon the machines? An affectionate machine-tickling aphid?
[…]
the servant glides by imperceptible approaches into the master