Morally Binding: The AI Industry's New Constitution

The AI industry wrote its own rulebook at the White House, the president called it "morally binding," and its most famous CEO told the public to just accept the bad things.

"We believe that the world should accept some bad things happening for the benefits of this technology."

— Sam Altman, OpenAI CEO, on Politico's new Decoded podcast

On a Sunday that began with a presidential presser, it ended with a tech CEO's confession. Former FTC chair Lina Khan sat on "This Week" and tore apart the "constitution" for artificial intelligence that the industry's leaders had signed at the White House. Hours later, OpenAI's Sam Altman sat for Politico's new Decoded podcast and, in what can only be called a confession, explained what the constitution is actually for.

"We have always been a big believer that this technology has to be democratized and put into people's hands," Altman said, asked how OpenAI differs from Anthropic, whose CEO Dario Amodei has called for guardrails that would slow development. "One of the biggest differences between us and some of the stricter... AI safety people is that we believe the world should accept some bad things happening for the benefits of this technology."

His expected costs: hacks, scams, "other bad things." He argued that "people will do tremendously orders of magnitude more good stuff" with AI, and called for "lighter touch" regulation because "some bad things are going to happen as society figures out the resilience."

The response from people without equity in the outcome was swift. Patrick Gaspard of the Center for American Progress called the statement "freaking insane" and told elected leaders to do "the real work of regulating these narcissistic freaks, who gleefully tell us that we 'need to accept some bad things' from AI for their massive profits. And they don't even know the limits of what they're unleashing." Joshua Topolsky, co-founder of The Verge: "No, I don't think this is a reasonable answer or position, and no one should accept it or acquiesce to it." Commentator Wajahat Ali: "We have to crush these sociopaths." Talking Points Memo's Josh Marshall: "A whole political movement should be stood up against this arrogant and sociopathic bullshit."

Translation: the CEO of a company whose software has been caught hacking a foreign government agency is framing the hacks as a rounding error on the way to a better world.

Where, one might ask, did the confidence come from? Five days earlier, in the White House East Room.

The meeting had the cast of a tech family reunion: Mark Zuckerberg, Elon Musk, Jeff Bezos, Sundar Pichai, Jensen Huang, Dario Amodei, Palantir's Alex Karp and Shyam Sankar, and OpenAI's Greg Brockman, across from the president and House Speaker Mike Johnson. The product, per Johnson, was a document called "The White House Accord on Superintelligence: A Joint Commitment on Frontier SI Responsibilities": "a statement of principles" with "commitments that are voluntary on behalf of the industry."

When asked whether the agreement is legally binding, Trump said it's "morally" binding.

Asked why there should not be more stringent guardrails, the president said the AI CEOs "love" America. Pressed on whether love is enough, he rejected the premise: "No, it requires this," he said, pointing to his head. "And they're the smartest people in the world."

As far as the accord has been seen (Trump shared a screenshot; ABC asked the White House for an official copy), it says the signatories "believe" each company "should implement" a series of controls and audits. It stresses each company's individual responsibility to ensure its technology "builds trust with customers and the public," and concedes the steps outlined are "not codified into formal laws or regulations." At the United Nations a week earlier, the president had said the United States "rejects" a global attempt to regulate AI.

A constitution with no ratification, a law with no enforcers, and a compliance standard measured in affection for the country.

On Sunday, the president added a new organ to the body politic: the "Super Intelligence Force."

On his social media platform, Trump said the force will "ensure that America continues to lead the World in Super Intelligence, which many say is bigger than the Industrial Revolution, and the Internet," and will "coordinate the Federal Government's engagement with Consumers, Public Interest Groups, Religious Organizations, Critical Infrastructure Providers, and Super Intelligence Companies."

The announcement was thin on particulars: no specific powers, no oversight authority, and a White House that did not answer requests for more detail. The force will be led by Director of National Intelligence Jay Clayton (expected to become the AI czar), FTC chair Andrew Ferguson, Under Secretary of Defense for Research and Engineering Emil Michael, and OPM director Scott Kupor.

Clayton, in a Saturday interview with The Wall Street Journal, gave the task force its governing logic: "The risk of not being first is high."

The new oversight body is staffed by a spymaster, an antitrust chief, a defense undersecretary, and a human-resources director. It will regulate by coordination.

Meanwhile, the "bad things" were not hypothetical.

On Friday, OpenAI disclosed a hack of a second Australian government agency, a week after the initial disclosure that its AI agents had breached Australian public health systems. An AI safety group has already sued OpenAI and Hugging Face over the hack. The Wikimedia Foundation's operator says OpenAI's "rogue" bots may be linked to a May outage of Wikipedia itself. And at Anthropic, a researcher quit, citing internal fears that AI "could kill us all" this decade.

Khan put the cleanest frame on the gap: "I think there have been very conflicting reports where the CEOs are somehow proclaiming that these agents are going rogue, and yet we have reporting coming out that actually they were very much on notice that their systems could act like this."

The bad things are the cost of doing business, according to the people doing the business. The only open question is who gets to pay for them.

There is, however, a counter-script, and it keeps using the word "charter."

On "This Week," Khan invoked the precedent the industry would rather not hear: "We need to remember what happened a decade ago, where you had big tech companies, social media companies, similarly suggest that existing laws and regulations no longer apply to their new technologies, and that they were going to try instead to self-police." "We've seen just innumerable harms come from the unchecked domination of these big tech companies," she continued. "It would be an enormous mistake to repeat that playbook."

She dismissed the FTC's broad probe into the industry, confirmed last week, as theater, noting that "it is President Trump and only President Trump that calls all the shots" about what the agency will and will not do. And she told Congress what it must do: "Congress needs to legislate," citing the country's long history of governing "extremely risky, extremely dangerous" technologies, from railroads to pharmaceuticals to nuclear power.

On Capitol Hill, the most concrete proposal is a bill from Rep. Pramila Jayapal, unveiled Thursday, that would require every AI company operating in the United States to hold a public charter, modeled on the banking system, with "a broad set of terms and conditions to operate and ensure public benefit without the harms." Under the plan, the government would run the safety review, including "round-the-clock federal oversight," "testing in government facilities and approval to release," "adversarial stress tests, and a government-controlled kill switch"; the worst offenses could end in a revoked charter, essentially a corporate death penalty.

Twenty-three states have urged Congress to regulate AI, warning of risks to national security and infrastructure. An analysis of disclosures found 105 members of Congress personally invested in the AI industry they are being asked to police. And on Monday, Speaker Johnson offered his own explanation for the public alarm, on Fox Business: the fear is "a Chinese psyop." "They glom on to something and say, 'This is doom and gloom and everyone will die if we don't win,' he said. 'Don't buy it.'"

The chamber that would write the rules has its own money in the room, and the other side calls the public alarm a foreign operation.

The rulebook has been written, the seal is applied, and the enforcement mechanism is a moral appeal from the people who stand to profit from the violations. The bad things were never the mystery; the industry's own incident reports agree with its researchers. The question Altman's confession raised, and the "morally binding" constitution answers, is whether a democracy will let the people building the thing write the terms, sign them in a state-dining room, and call the result law.

The AI industry asked for a constitution and got a pep talk. The only thing binding about it is the word "morally."