California Governor Gavin Newsom has vetoed the controversial "Frontier AI Model Safety Innovation Act" (SB 1047). This bill would have been one of the first to hold AI developers accountable for any significant damage caused by their technology. It faced strong criticism from prominent Democrats and major tech companies like OpenAI and venture capital firm Andreessen Horowitz, who warned that it could stifle innovation in California.
In a statement, Newsom called the legislation "well-intentioned" but argued that it imposed "strict standards on the most fundamental functions." He emphasized that regulation should be based on "proven evidence and science," and highlighted his own executive orders on AI and other laws he had signed that regulate known risks such as deepfakes.
The debate over SB 1047 underscores the challenges lawmakers worldwide face in balancing AI risk control while supporting this emerging technology. Despite the global AI boom sparked by ChatGPT's release two years ago, U.S. policymakers have yet to enact comprehensive legislation on this technology.
California State Senator Scott Wiener, who proposed the bill, expressed disappointment at Newsom's veto, calling it a setback for those who believe in regulating large corporations. Wiener stated that the result leaves everyone less safe.
SB 1047 would have required companies developing advanced AI models to take reasonable measures ensuring their technology does not cause "significant harm," such as mass casualties or property damages exceeding $500 million. Companies would need to implement specific safeguards, including a kill switch for their technology. Additionally, AI models would undergo third-party testing to minimize severe risks.
The bill also aimed to provide whistleblower protections for employees of AI companies wishing to share safety concerns. Companies failing to comply could face lawsuits from the California Attorney General.
Supporters of the legislation argued it would establish common-sense legal standards. However, venture capitalists, startup leaders, and companies like OpenAI warned it would slow innovation and drive AI businesses out of the state.
In a letter opposing the legislation, OpenAI's Chief Strategy Officer, Jason Kwon, stated, "The AI revolution has just begun, and California's unique position as a global AI leader drives the state's economic vitality. SB 1047 threatens this growth, slows innovation, and would cause world-class engineers and entrepreneurs to leave California in search of greater opportunities elsewhere."
Upon announcing his veto, Newsom said he would consult external experts, including AI scholars and entrepreneurs, to develop "workable safeguards" for the technology and continue collaborating with the state legislature on this subject.
On Sunday, Newsom also signed SB 896, which outlines how state government agencies can use AI.
