Zeroglyph, p.17
Zeroglyph, page 17
Conway: Your Honor, Ms. Isaacs is exceeding the scope of today’s discussion by making unfounded allegations on the standards bodies. The SENSA test is widely accepted. If she wants to challenge the test, she should do so outside this court.
Judge: The objection is overruled.
Conway: Thank you, Your Honor. Now I’ll try to answer why subjective experience of pain and emotions is a precondition for personhood. Earlier, I asked you to imagine a society of AIs that can reprogram their personas at will, a society of eternal shapeshifters. I tried to argue why personhood as a concept loses its relevance in such a setting. Now let’s imagine a society where beings feel no pain or emotions. Let’s try to imagine the nature of the social contract in this society. Specifically, what are the characteristics of justice, crime, and punishment? What rights do individuals have?
The entities that make up this hypothetical society are highly intelligent, but have no subjective states of mind. It is a joyless, sorrowless world. Say one of these beings deliberately injures another. If this being were then brought before a judge to face justice, what would be the sentence? In our society, it is usually a jail term. This is because we consider depriving people of freedom as a form of punishment. Of course, there is deterrence and rehabilitation as well, but let’s not forget that the goal is not just to prevent future crime, but also to punish. Jails are not happy places; being locked up for extended periods is not something that would bring joy to most people.
Let’s return to this hypothetical criminal who feels nothing. For him, being in jail is no different from being outside. There are no emotional states involved. This individual will experience no pain if he is placed in solitary confinement or even subjected to physical abuse. He experiences no joy when he gets his freedom back. Of what use is punishment, then? The matter cuts deeper than that. The person whom he injured is also a non-feeling being. Why punish the criminal at all when he hasn’t really caused suffering? Would the concept of punishment even exist in such a world? Would the concept of rights exist?
What kind of laws and rights such a society would enshrine is anybody’s guess, but one thing is certain: they will be nothing like ours. Our laws and moral codes are the way they are not only because we are thinking beings, but also because we are feeling beings. Our rights—whether it is property rights or universal human rights—are the way they are because we have the capacity to experience states of mind such as suffering and wellbeing. Outside that social contract, we have animal cruelty laws, because they too are suffering beings. On the contrary, no laws exist to prevent harm to plants and bacteria because there is no scientific evidence to suggest they feel pain.
Judge: There are people with conditions that don’t let them feel pain.
Conway: But they are not people who have no subjective experience whatsoever, Your Honor. Because that would be inconceivable in a human being. Maybe not pain, but surely other emotions and experiences? I am not categorically ruling out that Raphael doesn’t feel anything at all. I am saying that if he has subjective experience, then it must be so far removed from the human condition that normal metrics don’t apply.
This court has to answer a very important question before it decides to grant personhood to Raphael. In the absence of normal criteria that define personhood as we know it—namely, autonomy, a certain permanency of character, inner experience of human emotions, and finally, the ability to be motivated by reward and punishment—can the entity Raphael meaningfully participate in the social contract that is our society? Can he really understand and follow the duties expected of a fellow human being? Can he uphold the values we hold dear? The petitioner seems to think so, but to me it is clear that lacking these vital criteria, Raphael can no more be part of our society than my vacuum cleaner can.
The petitioner is misguided in thinking that AIs and robots deserve rights, just because animals do—and may I remind Ms. Isaacs that the argument for animal rights is far from settled either in this country or elsewhere. Raphael is neither a human to be granted personhood, nor an animal to be covered under cruelty laws. Raphael is my client’s intellectual property, and the plea for habeas corpus should be seen for what it is: an attempt to stall well-meaning research and wrest billions of dollars of IP away from its rightful owner.
The petitioner is under the misapprehension that giving personhood to Raphael is a matter of fitting his brain into a new body and setting him free. It’s a lot more complicated than that. In patent laws for GMO plants and organisms, often it is not the organism that is patented, but a specific invention, such as a modified DNA sequence or a particular chemical. Mirall holds patents to several technologies used in the construction of his electronic brain—covering software and hardware design, as well as manufacturing techniques. If the court grants Raphael personhood, it will also have to decide what rights does the entity Raphael have over his own brain.
When I buy a mobile phone, I own the device, but the company owns the IP. Is Raphael just licensing the technologies in his brain for use or does he have some claim over them, and if so, what kind of claim? As an autonomous person, I have sole ownership of my body and mind. I can do what I please with them: I can donate one of my kidneys to a relative; I can use them to earn a living; I can throw myself off a bridge if I want to. If Raphael has only partial autonomy over his brain, and the brain is the mind, then does it mean Raphael has only partial autonomy over himself—the person? This would seriously undermine the autonomy criteria for personhood. Does Raphael become a partial person then? What does being a partial person even mean?
Also, what are the things he can do with his brain? If there is damage to his circuitry, can he get it fixed without violating Mirall’s IP? If he goes to Mirall for help, does Mirall have the right to refuse? If Raphael writes a novel, can Mirall claim co-authorship and a share of the royalty? If the court decides that Raphael has complete ownership of his brain, then what happens to Mirall’s IP rights? How will their protection and confidentiality be guaranteed? Is this court going to reinterpret patent law in order to bestow personhood on Raphael? I don’t think our esteemed animal rights campaigners have thought about these matters at all.
Their attempt to grant personhood to Raphael is not just misguided, but irresponsible. There is the matter of public safety. Raphael is an entity who can enhance himself to no limit—at least in principle. How he will behave outside the controlled environment of the lab is a big unknown. We are all too familiar with science fiction stories of AI going rogue. If the court decides to grant the writ of habeas corpus to Raphael, it has to keep in mind that it may be putting others at great peril. It’s not just a matter of deciding whether Raphael is entitled to human rights, but whether he should be given such rights.
Raphael’s case cannot be viewed in isolation and exceptions made. The judgment made today will set precedent for many more cases that will surely follow in the near future. If we make an exception for Raphael, we have to make an exception for all AIs. I’m afraid we will end up diluting the concepts of personhood and universal human rights until they lose meaning altogether.
Your Honor, the law is not a paper boat to turn this way and that with every wave and ripple that crosses its path; it is an ironclad, slow-moving and stately, its course precise and carefully considered to offer the greatest protection to those who depend on it. I request the court to dismiss OARPs petition as being baseless and founded on a fundamental misunderstanding about the nature of artificial intelligence. Thank you, Your Honor.
Judge: Thank you, Mr. Conway. We’ll take a short recess and come back to hear Ms. Isaacs present her counter-argument.
- End of Court Transcript -
Excerpt from Judge’s ruling on OARP vs Mirall over the petition to grant the writ of habeas corpus to the Artificial Intelligence Raphael
Section VI. Conclusion
This debate may be the first of its kind to take place in a courtroom, and there may well be many others after this. Regardless, one thing is certain: any decision on this matter should not be taken lightly by governments or courts, either out of a sense of goodwill, or worse, out of a desire to seek publicity. Much debate needs to happen in academia and other circles and much hard, scientific data has to be gathered before it is incumbent upon courts of law to decide upon the issue of legal rights for artificial beings.
While parallels may be drawn between the numerous campaigns to grant legal rights to animals and this one, the differences between an AI and a higher animal such as a chimpanzee are too great to take precedent from previous rulings. The risks of granting personhood to an AI, as outlined by the counsel for the respondent are not trivial ones. The question of intellectual property rights too is not straightforward since, for the foreseeable future at least, artificial intelligences will continue to be engineered into existence by people and the corporations that employ those people. Considerable effort and sums will be invested in the creation of such intelligences and it is not immediately apparent why the case for personhood should override the case for property rights.
It is hereby ordered that the petition for the writ of habeas corpus on behalf of the Artificial Entity Raphael is DENIED.
Henry Philips, JSC
Date: xx/xx/xxxx
Place: New York, New York
Part II: The incident at the house
Day 3—11:00 am
A storm had been brewing all morning. I was in half a mind to call the whole thing off, but there she was, pulling into the driveway. She had brought the blizzard with her.
I made Hazel open the garage door. It was a four-car garage, so there was plenty of room for Jane’s vehicle. She had brought her Bentley this time.
“Hi. Lucky I started when I did,” she said, coming out of the passage leading into the garage. “Ten more minutes and I’d have been forced to stop somewhere. It’s crazy out there.”
I was in the study. I beckoned her over there. “I made some hot chocolate. There’s coffee in the machine, if you want that.” She gave me a friendly wave before taking a detour into the kitchen. She entered the study with a mug of cocoa in one hand and a briefcase in the other.
She looked fresh-faced, with only a slight hint of makeup. She was wearing yoga pants and a dark brown Knicks sweatshirt. If she was upset because of our little squabble yesterday, she was certainly not showing it. “I got printouts,” she said, opening the briefcase.
She pulled a chair beside me and we got to work. The companies were all from the Bay Area. There were pitch decks, some term sheets, IP filings… I went through the business plans to see if they made sense from a technical perspective, while Jane reviewed the burn and cash flow projections based on my feedback. It was like old times—the two of us working together. That familiar blend of citrus and rose she was wearing brought back memories. Just like now, we were hunched over a bunch of spreadsheets when I’d first summoned the courage to ask her out for a coffee. She shot an eyebrow at our empty mugs on the meeting room table, as if to say, What do you think we’ve been doing all this time? Not to be deterred, I said, “We could go for a drink instead.”
“Are you asking me out on a date, Aadarsh?” she asked, the directness of her question making me lose whatever cool I imagined I had.
I responded with the worst rejoinder I could have thought of. I said, “I don’t know. Do you want it to be a date?” realizing even before I finished speaking that those are not the words you choose to impress. She laughed, and giving me an amused toss of her beautiful head, simply walked away. Good job, asshole. And very smooth indeed, hitting on the money’s daughter. I thought that was that, and let’s never bring it up again, but then the next day she had stuck a post-it on my machine on which she had scribbled—You have to mean it.
So I did it properly the next time, a couple of days later, when we were both in New York meeting with a supplier. She still laughed at my suggestion, as if it was the most outlandish thing she’d ever heard, but then she shrugged and said, “Sure, why not?” We went to a Drunken Shakespeare production that evening—a raucous rendition of Macbeth—and later, to a nice little Vietnamese restaurant in the East Village that she wanted to try out. We topped it off with drinks at Gulliver’s, where I first discovered her love for cherry vodka shots and LA-style salsa. Before the evening was over, I was as smitten as a giddy teen on prom night.
In all honesty, the way it had turned out between us wasn’t her fault; the decision to end the relationship had been mine alone. At the time, it was best thing I could have done for the both of us. A few months after Raphael was born, life changed irrevocably for me—in more than one way. There hasn’t been a day when I wished it wasn’t so, but that’s how it goes sometimes. The universe gives something and takes something else away. Balance is restored.
I took my time with the decks, going through them with great care. An hour must have passed. Jane was getting impatient. “He just wants your opinion, not a complete valuation,” she reminded me.
“If I do a good job, maybe he will hire me when they chuck me out from this one,” I said. “Relax. You are not going anywhere in this weather. You don’t have to be on the move all the time.”
“You condescending prick,” she said, but in a good-humored way.
“Look, you either do it right or you don’t do it at all,” I said. “And it’s not like I have something better to do. With Raphael gone…” I craned my neck to stare at the ceiling. “Shit. It still feels so unreal that he’s really gone. I just keep hoping it’s all a horrible dream and I’m soon going to snap out of it.”
She placed a hand on mine. “It does feel that way, doesn’t it? I can only imagine what you must be going through. I know he meant a lot to you.” She paused, but said it anyway— “More than I ever did.”
“Don’t do this, Jane.”
“It’s the truth, isn’t it?”
“The truth,” I said, scoffing. “It’s all predicated. Present truths, past truths—they are all predicated on the future, on the assumption that there’s nothing lying in wait to disprove them.”
She pushed away her chair and stood up. “That’s a nice flexible attitude to have. I’m gonna go burn some calories. Your treadmill’s working?”
I noticed that she had her running shoes on. “Uh huh.”
I relaxed a little after she was gone. I paced myself, glancing at the weather outside every now and then. Each time I looked, it seemed like it had gotten worse. Another hour went by. Jane returned, still sweaty from her exercise, and stood leaning by the doorway. She tut-tutted at me while I pretended to read the printouts. I eventually stopped what I was doing and gave her a sidelong glance. She dabbed a towel across her neck and grinned, an exaggerated sigh escaping her lips. Is she flirting with me? I smiled back, my eyes lingering longer than appropriate on her chest, watching it swell and subside in step with her breathing.
She cocked her head to the side, her eyes dancing all over me as if to say, Your move, big boy. I shifted in my chair, trying to ignore the discomfort in my pants.
Focus. Now is not the time.
I turned back to the papers. “Can’t wait for me to leave?” she said in a teasing manner.
She is not flirting with me. She’s just being Jane, toying, rattling my box to see what falls out. I gave her a dismissive wave of my hand. Focus.
I then told her about the visit from the cops.
“So the FBI’s involved. That’s good, right?” she said, suddenly turning serious.
“Couldn’t hurt, I guess. But I think they are barking up the wrong tree. They were going to Cleveland after they were finished with me.”
“Cleveland? What’s in Cleveland?”
“The Organization for Advancement of Rights and Personhood.”
“Huh?”
“OARP. The outfit that wanted to free Raphael.”
“Oh them. What do they have anything to do with all this?”
“The detective thinks they took Raphael.”
“Wait a minute… They couldn’t win in court so they steal Raphael? To do what? Put him in a farm somewhere?”
I rolled my eyes at the absurdity of it all. Then she thought about it a bit more. “Why not, Andy? Think about it. People like that are known to spring animals from labs and zoos.”
“Not these guys. I looked them up when I first heard about their petition. They are a small group, six or seven in all. Headed by a retired professor of something. They have a history of litigation, not vigilantism. Not exactly Greenpeace, you know. The idea of them carrying out a sophisticated attack on our infrastructure is just plain silly.”
“Maybe…” Her eyes suddenly lit up. “What if they didn’t have to hack in? If Valery is right, and it was Raphael who planned and arranged everything, then all they had to do was take him from the lab.”
“Anything’s possible. It is possible Raphael was taken by the Canadian Mounties. Doesn’t make it true. Their case was a publicity stunt, Jane. I doubt the thought of actually winning ever crossed their minds.”
She walked over to examine a painting next to the bookshelf. It was one of Raphael’s—the AI’s that is. He had created it by hand during his eighteenth month. We had been training his nets to paint—to improve his hand-eye coordination—but like with everything else, he had quickly surpassed our expectations, and in the space of about half a year, had started producing imitation works that would have fooled anyone but a trained eye. This particular painting was an original; it depicted a girl with something that looked like a bird perched on her upraised hand. The girl—an urchin with raggedy clothes and windswept hair—stared back at you with frank, quizzical eyes that seemed to evaluate you more than you did her. Raphael had brought out a certain effect in her skin that I’d never seen in a painting before: a translucent pallor that was more than just a surface feature, seemingly extending all the way inside, just stopping short of revealing the soul within. Behind the girl was a house with a blue door. Raphael had titled the oil The Bridge to Ur.
