Governor Newsom announces new initiatives to advance safe and responsible AI, protect Californians

SACRAMENTO – Governor Gavin Newsom announced a series of initiatives to further protect Californians from fast-moving and transformative GenAI technology, while vetoing legislation that falls short of providing a flexible, comprehensive solution to curbing the potential catastrophic risks.

Over the past 30 days, Governor Newsom signed 17 bills covering the deployment and regulation of GenAI technology, the most comprehensive legislative package in the nation on this emerging industry — cracking down on deepfakes, requiring AI watermarking, protecting children and workers, and combating AI-generated misinformation. California has led the world in GenAI innovation while working toward common-sense regulations for the industry and bringing GenAI tools to state workers, students, and educators.

The Governor has asked the world’s leading experts on GenAI to help California develop workable guardrails for deploying GenAI, focusing on developing an empirical, science-based trajectory analysis of frontier models and their capabilities and attendant risks. The Governor will continue to work with the Legislature on this critical matter during its next session.

Building on the partnership created after the Governor’s 2023 executive order, California will work with the “godmother of AI,” Dr. Fei-Fei Li, as well as Tino Cuéllar, member of the National Academy of Sciences Committee on Social and Ethical Implications of Computing Research, and Jennifer Tour Chayes, Dean of the College of Computing, Data Science, and Society at UC Berkeley, on this critical project. Here’s what these leading experts had to say:

• “Frontier AI brings the potential for enormous benefits as well as real risks that require sustained, careful judgment. I look forward to working with California to get the balance right in the days and months ahead.” — Mariano-Florentino (Tino) Cuéllar, president of the Carnegie Endowment for International Peace and member of the National Academy of Sciences Committee on Social and Ethical Implications of Computing Research

• “I am honored to continue the partnership between UC Berkeley, Stanford and Governor Newsom, leveraging the world’s greatest scientists and thinkers on AI, many of whom are located right here in California. It is critical we nurture a robust innovation economy and foster academic research – this is how we’ll ensure AI benefits the most people, in the most ways, while protecting from bad actors and grave harms. The College of Computing, Data Science and Society at UC Berkeley stands at the ready to provide cutting edge science and and policy recommendations to make sure we achieve these goals.”— Jennifer Tour Chayes, Dean of the College of Computing, Data Science, and Society at UC Berkeley

• “Safe and responsible AI is essential for California’s vibrant innovation ecosystem. To effectively govern this powerful technology, we need to depend upon scientific evidence to determine how to best foster innovation and mitigate risk. The Stanford Institute for Human-Centered AI (HAI) was founded with the specific mission of ensuring that AI is developed to benefit society. HAI looks forward to the continued partnership with the State of California through Governor Newsom’s Executive Order on GenAI to ensure California’s leadership on safe, vibrant, and beneficial AI.” — Dr. Fei-Fei Li, Co-Director, Stanford Institute for Human-Centered Artificial Intelligence

The Newsom Administration will also immediately engage academia to convene labor stakeholders and the private sector to explore approaches to use GenAI technology in the workplace. The Administration is committed to continuing partnerships with public sector unions in nation-leading government procurement.

Today, Governor Newsom signed legislation requiring California’s Office of Emergency Services to expand their work assessing the potential threats posed by the use of GenAI to California’s critical infrastructure, including those that could lead to mass casualty events. That bill, SB 896 by Senator Bill Dodd (D-Napa), codifies aspects of the Governor’s recent Executive Order from September 2023. At the Governor’s direction, Cal OES is working with frontier model companies to analyze energy infrastructure risks and convened power sector providers to share threats and security strategies. Building on the work to date and pursuant to SB 896, the Governor has directed Cal OES to undertake the same risk assessment with water infrastructure providers in the coming year and the communications sector shortly after that. Read the signing message here.

Governor Newsom vetoed SB 1047, one of several GenAI bills considered this year by the California Legislature. Read the veto message here.

California’s AI global leadership 

Last year, Governor Newsom signed an executive order laying out how California’s measured approach will focus on shaping the future of ethical, transparent, and trustworthy GenAI, while remaining the world’s GenAI leader.

Within the past two weeks, Governor Newsom signed a series of bills to crack down on sexually explicit deepfakes and require AI watermarking, protect performers’ digital likenesses, and combat deepfake election content. 

AI is already changing the world, and California will play a pivotal role in defining that future. The state is home to 32 of the world’s 50 leading GenAI companies, high-impact research and education institutions, and a quarter of the technology’s patents and conference papers.

California has led the nation in harnessing these transformative technologies while studying the risks they present. The state has undertaken efforts to utilize GenAI to solve challenges, everything from reducing traffic to helping address homelessness. 

Last month, the state partnered with NVIDIA to launch a first-of-its-kind AI collaboration and earlier hosted a GenAI summit with leaders to discuss how the state can best use this transformative technology to better serve the people of California.

Over the past 30 days, Governor Newsom has signed the following bills concerning GenAI technology:

• AB 1008 by Assemblymember Rebecca Bauer-Kahan (D-Orinda) – Clarifies that personal information under the California Consumer Privacy Act (CCPA) can exist in various formats, including information stored by AI systems. (previously signed)
• AB 1831 by Assemblymember Marc Berman (D-Menlo Park) – Expands the scope of existing child pornography statutes to include matter that is digitally altered or generated by the use of AI.
• AB 1836 by Assemblymember Rebecca Bauer-Kahan (D-Orinda) – Prohibits a person from producing, distributing, or making available the digital replica of a deceased personality’s voice or likeness in an expressive audiovisual work or sound recording without prior consent, except as provided. (previously signed)
• AB 2013 by Assemblymember Jacqui Irwin (D-Thousand Oaks) –  Requires AI developers to post information on the data used to train the AI system or service on their websites. (previously signed)
• AB 2355 by Assemblymember Wendy Carrillo (D-Los Angeles) – Requires committees that create, publish, or distribute a political advertisement that contains any image, audio, or video that is generated or substantially altered using AI to include a disclosure in the advertisement disclosing that the content has been so altered. (previously signed)
• AB 2602 by Assemblymember Ash Kalra (D-San Jose) – Provides that an agreement for the performance of personal or professional services which contains a provision allowing for the use of a digital replica of an individual’s voice or likeness is unenforceable if it does not include a reasonably specific description of the intended uses of the replica and the individual is not represented by legal counsel or by a labor union, as specified. (previously signed)
• AB 2655 by Assemblymember Marc Berman (D-Menlo Park) – Requires large online platforms with at least one million California users to remove materially deceptive and digitally modified or created content related to elections, or to label that content, during specified periods before and after an election, if the content is reported to the platform. Provides for injunctive relief. (previously signed)
• AB 2839 by Assemblymember Gail Pellerin (D-Santa Cruz) – Expands the timeframe in which a committee or other entity is prohibited from knowingly distributing an advertisement or other election material containing deceptive AI-generated or manipulated content from 60 days to 120 days, amongst other things. (previously signed)
• AB 2876 by Assemblymember Marc Berman (D-Menlo Park) – Require the Instructional Quality Commission (IQC) to consider AI literacy to be included in the mathematics, science, and history-social science curriculum frameworks and instructional materials.
• AB 2885 by Assemblymember Rebecca Bauer-Kahan (D-Orinda) – Establishes a uniform definition for AI, or artificial intelligence, in California law. (previously signed)
• AB 3030 by Assemblymember Lisa Calderon (D-Whittier) – Requires specified health care providers to disclose the use of GenAI when it is used to generate communications to a patient pertaining to patient clinical information. (previously signed)
• SB 896 by Senator Bill Dodd (D-Napa) – Requires CDT to update report for the Governor as called for in Executive Order N-12-23, related to the procurement and use of GenAI by the state; requires OES to perform a risk analysis of potential threats posed by the use of GenAI to California’s critical infrastructure (w/high-level summary to Legislature); and requires that the use of GenAI for state communications be disclosed.
• SB 926 by Senator Aisha Wahab (D-Silicon Valley) – Creates a new crime for a person to intentionally create and distribute any sexually explicit image of another identifiable person that was created in a manner that would cause a reasonable person to believe the image is an authentic image of the person depicted, under circumstances in which the person distributing the image knows or should know that distribution of the image will cause serious emotional distress, and the person depicted suffers that distress. (previously signed)
• SB 942 by Senator Josh Becker (D-Menlo Park) – Requires the developers of covered GenAI systems to both include provenance disclosures in the original content their systems produce and make tools available to identify GenAI content produced by their systems. (previously signed)
• SB 981 by Senator Aisha Wahab (D-Silicon Valley) – Requires social media platforms to establish a mechanism for reporting and removing “sexually explicit digital identity theft.” (previously signed)
• SB 1120 by Senator Josh Becker (D-Menlo Park) – Establishes requirements on health plans and insurers applicable to their use AI for utilization review and utilization management decisions, including that the use of AI, algorithm, or other software must be based upon a patient’s medical or other clinical history and individual clinical circumstances as presented by the requesting provider and not supplant health care provider decision making. (previously signed)
• SB 1288 by Senator Josh Becker (D-Menlo Park) – Requires the Superintendent of Public Instruction (SPI) to convene a working group for the purpose of exploring how artificial intelligence (AI) and other forms of similarly advanced technology are currently being used in education. (previously signed)
• SB 1381 by Senator Aisha Wahab (D-Silicon Valley) – Expands the scope of existing child pornography statutes to include matter that is digitally altered or generated by the use of AI.