Scoop: San Diego weighs AI tool to help write city policies
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Lindsey Bailey/Axios
The city is considering a contract with a company that sucks up city documents and then uses AI to write new policies for lawmakers to vote on.
Why it matters: This would be a big jump in the city's AI usage and raises questions about the security of public data and whether we want robots drafting policies.
Driving the news: The city is in active talks with Madison AI to create an AI model that would run off the city's documents.
- Madison AI representatives met with city staff last December and followed up with a slide deck and video presentation that outlined their offerings. Their pitch was obtained by Axios in a public records request.
- Its model runs off a city's "every single PDF and archived file that is the institutional knowledge of your organization" and "puts it to work," according to the video.
- The AI writes agendas and staff reports, helps staff prepare for elected officials' questions and drafts policy.
Madison AI didn't return a request for comment.
- It starts at $25,000 a year and is being used in cities across the country, including Culver City and Aspen, Colorado.
Zoom in: San Diego doesn't have a contract with Madison AI and isn't pilot testing it. City staff wouldn't do an interview with Axios, but spokesperson Anthony Santacroce answered questions over email.
- "The City is evaluating solutions from Madison AI that offer potential efficiencies for public records requests and legislative document information," he wrote.
Public records show the city appears very interested.
- At the end of last year, the city's IT assistant deputy director Serena Blanks wrote to Madison AI that she wanted to get the contract moving and said the last step was resolving data security.
- In response, Madison sent security documentation and filled in a technology questionnaire.
Threat level: Uploading public documents to a third-party company raises concerns, says UCSD communication and data science professor Stuart Geiger.
- Madison AI states in its security overview that customer data is never used to train, fine-tune or improve AI models, but it also advertises sharing data in its promotional video.
- "Imagine being able to analyze data across neighboring cities or counties" and "eventually national-level analysis," the video says.
Using AI to write policies also degrades the collaboration that's part of democracy, UCSD communication professor Lilly Irani tells Axios.
- "They're treating creating policy like a manufacturer would treat creating widgets, where they want to cut costs, cut speed, cut labor," Irani says. "But this is democracy, and democracy means we've got to take time to talk to each other."
- Plus, policy wording has to be exactly right, because it's often tested in court, she says.
Mistakes in AI can also multiply, Geiger says.
- "If there's one little error here, one little error there, then because the AI relies on what it's already built, it can cascade," he says. "Especially if people aren't really watching it."