Software instantiates moral systems, and the ultimate goal of moral systems is to facilitate the expression and reception of love among us. Most systems
Love cannot be straightforwardly quantified, and thus the most important system metric cannot merely be measured, and thus the design of software and systems is an art of human engagement, not a science of detachment. Measuring efficiency or productivity is easier, and talking about power is simpler, but these can only ever be proxies. Nothing matters until it escapes from the wires into the air. Power does not become good or bad until converted to love, or diverted into misery. To evaluate a system, you have to ponder its aggregate human effects.
AI, not at all uniquely in software history, is usually presented in terms of individual empowerment. Individual
Spam filters, for example, empower us to resist scams, and so materially reduce proximate misery. But they also reduce our collective awareness of systemic malice without actually eliminating the reward patterns that sustain it, and thus create an environment in which spam multiplies botulistically, and our own resistances weaken. Failures remain inevitable, and their consequences unequally distributed. Malice is only sporadically and minorly victorious, but this is enough to create and sustain a loveless economy of meta-malicious tools. As an individual, you can't improve the world by refusing to filter your spam. But we have built a system in which that is our only apparent choice.
The question we should continually insist on asking ourselves, both as makers of tools and users of them, is how they can help us improve the flow of love. Do they connect us with each other, and reinforce our instincts to treat each other as moral ends? Do they help us convey our love out into the world through information and interfaces, or help us know when we are being offered any? If AI chatbots replace human conversations or relationships, our love will be increasingly extracted and consumed. If newly adaptive technology can refute previous arguments for dehumanizing bureaucracy, or reveal other hidden resistive mechanisms, it might help us create emotional space in which we (not it) might more effectively nurture our empathy or refine our expressions of shareable joy. The future we get depends on our own actions, not any kind of techno-accelerationist inevitability. The thing that matters about software is not how running it affects the computer, it's how using it affects
And as with any Kantian topology, this moral judgment of course applies to
If our new systems all surround us with innumerable inattentively managed agents, they will leak a more diffuse and thus lossier model of intent, and generate no more humanity. We can be expedient contributors to our own distributed suffering, or we can stop and earnestly try to debug these experimental wands of dubious but collected wonder before we completely forget how we used to get along without them. We need the help, but we intermediatebetween us, by their nature, and thus can either amplify love we entrust to them, or dissipate love that we are forced to surrender.Love cannot be straightforwardly quantified, and thus the most important system metric cannot merely be measured, and thus the design of software and systems is an art of human engagement, not a science of detachment. Measuring efficiency or productivity is easier, and talking about power is simpler, but these can only ever be proxies. Nothing matters until it escapes from the wires into the air. Power does not become good or bad until converted to love, or diverted into misery. To evaluate a system, you have to ponder its aggregate human effects.
AI, not at all uniquely in software history, is usually presented in terms of individual empowerment. Individual
disempowermentis morally negative, and empowerment is arguably necessary, but empowerment that cannot be conducted into love, or which obscures our awareness of the flow of love to or from us, will end up pouring out of the back of our laptops as entropic heat.Spam filters, for example, empower us to resist scams, and so materially reduce proximate misery. But they also reduce our collective awareness of systemic malice without actually eliminating the reward patterns that sustain it, and thus create an environment in which spam multiplies botulistically, and our own resistances weaken. Failures remain inevitable, and their consequences unequally distributed. Malice is only sporadically and minorly victorious, but this is enough to create and sustain a loveless economy of meta-malicious tools. As an individual, you can't improve the world by refusing to filter your spam. But we have built a system in which that is our only apparent choice.
The question we should continually insist on asking ourselves, both as makers of tools and users of them, is how they can help us improve the flow of love. Do they connect us with each other, and reinforce our instincts to treat each other as moral ends? Do they help us convey our love out into the world through information and interfaces, or help us know when we are being offered any? If AI chatbots replace human conversations or relationships, our love will be increasingly extracted and consumed. If newly adaptive technology can refute previous arguments for dehumanizing bureaucracy, or reveal other hidden resistive mechanisms, it might help us create emotional space in which we (not it) might more effectively nurture our empathy or refine our expressions of shareable joy. The future we get depends on our own actions, not any kind of techno-accelerationist inevitability. The thing that matters about software is not how running it affects the computer, it's how using it affects
us.And as with any Kantian topology, this moral judgment of course applies to
everything we do, and fails most insidiously not when we misapply it, but when we forget to consider it at all. Everything we build or write or share is a moral act, but so too is everything we read or play or seek out. Curiosity is a collation of love, complacency is a porosity through which it evaporates. The efficiencies we accept bypass and thus reduce us. The agents we delegate take away fragments of our own agency with which to perform representations of our will. They don't break us, we willingly shatter ourselves in the earnest but self-defeating hope of inhabiting more of our world.If our new systems all surround us with innumerable inattentively managed agents, they will leak a more diffuse and thus lossier model of intent, and generate no more humanity. We can be expedient contributors to our own distributed suffering, or we can stop and earnestly try to debug these experimental wands of dubious but collected wonder before we completely forget how we used to get along without them. We need the help, but we
arethe help. Maybe these are the wrong wands. Maybe wands are always the chrysalid form of hammers. The challenge, the collective challenge for all of us together, is to prove that all our whirling wands and hammers and recursively twittering software and obliquely moral systems can be good for the air between us, can help us make it a medium through which we see each other more glitteringly, rather than a descending haze of particulate debris. We can make and tolerate a howling world in which we hurry past each other, masked against our own grit. Or we can assemble.