California is testing new generative AI tools. Here’s what to know

By TRÂN NGUYỄN
Associated Press

SACRAMENTO, Calif. (AP) — Generative artificial intelligence tools will soon be used by California’s government.

Democratic Gov. Gavin Newsom’s administration announced Thursday the state will partner with five companies to develop and test generative AI tools that could improve public service.

California is among the first states to roll out guidelines on when and how state agencies can buy AI tools as lawmakers across the country grapple with how to regulate the emerging technology.

Here’s a closer look at the details:

WHAT IS GENERATIVE AI?

Generative AI is a branch of artificial intelligence that can create new content such as text, audio and photos in response to prompts. It’s the technology behind ChatGPT, the controversial writing tool launched by Microsoft-backed OpenAI. The San Francisco-based company Anthropic, with backing from Google and Amazon, is also in the generative AI game.

HOW MIGHT CALIFORNIA USE IT?

California envisions using this type of technology to help cut down on customer call wait times at state agencies, and to improve traffic and road safety, among other things.

Initially, four state departments will test generative AI tools: The Department of Tax and Fee Administration, the California Department of Transportation, the Department of Public Health, and the Health and Human Services Department.

The tax and fee agency administers more than 40 programs and took more than 660,000 calls from businesses last year, director Nick Maduros said. The state hopes to deploy AI to listen in on those calls and pull up key information on state tax codes in real time, allowing the workers to more quickly answer questions because they don’t have to look up the information themselves.

In another example, the state wants to use the technology to provide people with information about health and social service benefits in languages other than English.

WHO WILL USE THESE AI TOOLS?

The public doesn’t have access to these tools quite yet, but possibly will in the future. The state will start a six-month trial, during which the tools will be tested by state workers internally. In the tax example, the state plans to have the technology analyze recordings of calls from businesses and see how the AI handles them afterward — rather than have it run in real-time, Maduros said.

Not all the tools are designed to interact with the public though. For instance, the tools designed to help improve highway congestion and road safety would only be used by state officials to analyze traffic data and brainstorm potential solutions.

State workers will test and evaluate their effectiveness and risks. If the tests go well, the state will consider deploying the technology more broadly.

HOW MUCH DOES IT COST?

The ultimate cost is unclear. For now, the state will pay each of the five companies $1 to start a six-month internal trial. Then, the state can assess whether to sign new contracts for long-term use of the tools.

“If it turns out it doesn’t serve the public better, then we’re out a dollar,” Maduros said. “And I think that’s a pretty good deal for the citizens of California.”

The state currently has a massive budget deficit, which could make it harder for Newsom to make the case that such technology is worth deploying.

Administration officials said they didn’t have an estimate on what such tools would eventually cost the state, and they did not immediately release copies of the agreements with the five companies that will test the technology on a trial basis. Those companies are: Deloitte Consulting, LLP, INRIX, Inc., Accenture, LLP, Ignyte Group, LLC, SymSoft Solutions LLC.

WHAT COULD GO WRONG?

The rapidly growing technology has also raised concerns about job loss, misinformation, privacy and automation bias.

State officials and academic experts say generative AI has significant potential to help government agencies become more efficient but there’s also an urgent need for safeguards and oversight.

Testing the tools on a limited basis is one way to limit potential risks, said Meredith Lee, chief technical adviser for UC Berkeley’s College of Computing, Data Science, and Society.

But, she added, the testing can’t stop after six months. The state must have a consistent process for testing and learning about the tools’ potential risks if it decides to deploy them on a wider scale.