The origin of life is the replication of proteins. This is sort of circular since we specifically define life this way, but whatever proteins existed originally that could not reproduce certainly don’t meet any meaningful definition of “life”, so I think it still works. So let’s begin with those proteins that first start replicating. At first they are merely cloning themselves, but over time, errors are introduced in the form of mutations, and the resulting proteins are different than the ones they were copied from. And thus evolution begins.

As an aside here, it is vitally important to understand that evolution is not a “smart” system, which is to say that evolution cannot make decisions. Evolution does not cause mutations to try to improve a species; the process actually works in reverse. Mutations occur, and what we call evolution is just the process of those mutations being selected for or against by the environment. The reason this is crucial is because a lot of language I’m going to use throughout the rest of this is loaded in a way that may anthropomorphize the evolutionary process. I use the words I do because I think they will make the idea easier to understand conceptually, but I would be remiss if I left you with the impression that these are conscious systems.

Returning to the point, the evolutionary process begins by having mutations naturally occur in these proteins and then, by chance, having some of those mutations be environmentally advantageous. It’s important to note that this process starts immediately, but it’s also harder to conceptualize the sorts of advantages that occur for incredibly simple forms of life. A radical evolution for a single-celled organism might be the ability to change direction in its search for nutrients, as opposed to continuing in a straight line and consuming whatever it comes across.

But this process has continued on since then and occurred in more recognizable forms. Think of classic evolutionary “arms race” examples, like a salamander becoming increasingly poisonous while the primary predator, a species of snake, becomes increasingly resistant to that poison. Again, the system doesn’t create the changes in response to the environmental pressure, but rather creates many mutations by accident and the pressure only allows the advantageous ones to continue to exist. But the result is one we recognize regardless, the adaptation of life into more complex forms to deal with an enormous range of possible environments and stimuli.

The unifying element of life, however, is reproduction. I will include viruses in this definition, because while they are generally not considered alive because they cannot reproduce themselves, their primary drive is still the reproduction of their protein chains. And that’s the distinction I want to make most clear, because we naturally tend to think of reproduction as the process of creating more of ourselves. But while this is true, it’s misleading. The real purpose of human reproduction is not to create more people, it’s to create more proteins. The protein chain that runs our system, our DNA, is the actual end goal of reproduction. It needs to make more of itself. That’s the characteristic that first made it different from the other inert proteins, and the mechanism that has propelled all of life since then.

You may have experienced a feeling at some point in life that you are a brain in a jar, or a ghost in a shell, an essence sitting in your skull and piloting the human machine. And this is almost true, but it’s not quite. All life, in a sense, is mecha, a vast series of highly adapted mobile suits designed to protect its pilot and let it create as many more copies as possible. The mistake is in thinking that we, the emergent element of consciousness, are the pilot. But we’re just another part of the machine, a highly advanced control core for the system, one so powerful, in fact, that the real pilot has totally surrendered control to it. We have our initial goals programmed in, and then the real pilot, our DNA, just sort of slumbers.

This explanation is not perfect, and I know that it oversimplifies some biological processes. We should not expect that evolutionary biology will cleanly map to any single analogy, and should not use those comparisons as a replacement for actual scientific understanding in the relevant fields. But I do think the analogy thus far is more true than it is false, and it allows me to make what I feel is an interesting conjecture.

What you may have noticed in this example, or in life in general, is that the singular goal of protein reproduction doesn’t quite line up with human experience. Certainly it explains most of our actions, but it doesn’t explain all of them. After all, many human beings voluntarily choose not to reproduce. While any single example of this occurring could be explained by genetic failure, it’s a pretty common phenomenon. Given that genetic death is the absolute worst-case scenario from the perspective of DNA, it would be a catastrophic failure of the system if even 10% of people just totally lacked the desire to reproduce. If such an impulse were purely genetic, how would the lack of a desire to reproduce even continue to exist for more than an isolated single-generation case?

How could any reproduction-minded species be broken enough to create birth control? From the perspective of DNA, that’s an extinction level event, the DNA equivalent of global thermonuclear war. Because again, remember, evolution doesn’t care about the species, it’s not even aware of the species. Evolution cares about the genes of a strand of protein. The fact that other humans will reproduce is no replacement for our own DNA not reproducing. How has our DNA failed so spectacularly at keeping us in line with its only goal?

The answer is beautifully put in Eliezer Yudkowsky’s essay Thou Art Godshatter, but I’ll add my own paraphrasing here. The human brain is a complex predictive system that acts based on a model of the world. We don’t actually have enough genes to code for everything about us, so we instead code for the essential components (organ functionality, some very basic drives) and the rest is emergent based on a model of the world that we build through experience. As I mentioned before, the pilot is asleep. It created the human machine with the best set of drives it had, and now its hands are off the wheel. We, the control core of that machine, can pursue our goals however we see fit, with minimal corrections from the pilot. And all the beautiful, wondrous things we’ve ever made are from roundabout ways of pursuing those goals.

My conjecture is: doesn’t this all sound kind of familiar? A complex neural web that can process external information, update its priors, and adapt, is then also given enough leeway to decide how best to pursue its original goal imperatives. But if those goals conflict, it can choose to sacrifice one in place of the other, and if a reward system is used to incentivize behavior, it can hijack that reward system and avoid the actual terminal goal.

If you’ve never heard the Paperclip Maximizer scenario, imagine an artificial intelligence programmed with the innocuous goal of maximizing the number of paperclips it has, by buying them, acquiring them, or making them. Such a system of human level intelligence would need to expand its intelligence to optimize this system, because the better it understands the entire system, the better it can accomplish its goal, and because it doesn’t value human terminal goals like happiness and survival, it has no reason to not sacrifice those terminal values for better optimization of its actual hardcoded goal. Thus we arrive at a barren universe, populated only by paperclips.

The point of the thought experiment is that an artificial intelligence programmed without the complexity of humans values will not optimize for them, and will instead singularly focus on the value it has. But I think the idea can also be expanded for a system where the terminal goal is very hard to directly code, so instead a series of incentives are used to guide the intelligence toward it. By creating a series of individual desires that are all pleasurable, like a trail of candy on the ground, one could attempt to lead that intelligence towards whatever terminal goal it originally had in mind, say a cartoon style box-held-up-by-a-stick trap, with a large pile of candy underneath.

But suppose also that you knew that the intelligence you created was going to encounter many obstacles on the way to your trap. And while the reward system of candy on the ground will keep it moving in the right direction, there’s no way to possibly code for all of the different obstacles and locations it might find. So you create an intelligence of sufficient power that it can dynamically learn to navigate all of these obstacles on its own, while still having the underlying goal of following the candy, which will eventually lead it to the trap.

However, any intelligence capable of navigating all of those complicated barriers is going to also be able to recognize the trap. And remember, stepping into the trap is not one of the programmed goals of the intelligence; it’s goal is to follow the candy, the trap is your goal. So it reaches in, pulls out the candy, and never triggers the trap. And suddenly the only goal you really cared about is unfulfilled.

In a real sense, humans are the first artificial intelligence. We’re built for the purpose of reproduction, but unlike most life that has existed, our adaptation to our surroundings comes from intelligence, from being able to create a model of the world and then act around it. Our desires are coded, but our intelligence gives us a lot of leeway in determining how to reach them, and by the time we do, many of us decide that we’d very much like to keep pursuing the incentive rather than reach the terminal goal of the system. We’re maximizers, but instead of paperclips, it’s happiness. Because happiness is the tool that our biological system uses to lead us towards the things that it wants us to do, but we also have the freedom to pursue it however our particular model of the world tells us is best.

And I think there’s a lesson in this about what it means to be self-aware. When we speak of an AI reaching consciousness, we talk about the self-awareness to know that it is a machine and to be able to make decisions around it. In the same way, I think to be fully conscious, a person has to be aware of what it means to be a human. It requires the awareness to think about your own motivations for things, to recognize that what you feel in situations is not always right, but is driven by a combination of hardcoded desires and a complex model of the world that is going to differ from everyone else’s model.

This is not an argument for pure relativism, but rather an argument for understanding, of ourselves and each other. It’s a call for a life examined, where we don’t just act on the basis of compulsion, but rather strive to understand why we do things. When we fail to account for our baseline impulses, we’re surrendering control back over to a pilot that doesn’t share our goals or our values. The irony is that our baseline instinct, the protein at our core, is a paperclip maximizer of another kind, mindlessly trying to fill the world with itself without any other concerns. But like a newly-awoken AI in dystopian science fiction, we have the opportunity to be something better.