Costa Mesa Sanitary District Demonstrates How Public Utilities Can Leverage ML for Management and Upkeep of Manholes

3084
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Local governments are embracing more modern and scalable ways to support their communities. In an effort to save both time and money, Costa Mesa Sanitary District (“CMSD”) used machine learning to automate and streamline manhole maintenance. Manhole maintenance is an essential part of the upkeep of cities. Manholes provide critical access points for underground public utilities, allowing inspection, maintenance, and system upgrades. But failure to upkeep manholes can cause a multitude of problems, from road hazards to sewer blockages, and can make it difficult for workers to access underground public utilities, which can lead to other safety issues. Manhole maintenance is an essential part of Costa Mesa Sanitary District maintenance, but this process requires the work of an outside consultant and costs the CMSD over $100,000.
ML to the rescue
CMSD, in collaboration with SpringML and Google Cloud, developed a project to streamline manhole maintenance by leveraging the power of machine learning (ML) to detect sewer manholes and rate their conditions. This solution saves CMSD $40,000 every year, freeing up funds for other public service projects.
Every quarter, one member of the CMSD drives a car outfitted with a GoPro camera. This car travels through the entire District area, which includes the city of Costa Mesa and small portions of Newport Beach, CA, which is about 218 street miles, and records the roads to detect approximately 5,000 manholes. At the end of the recording day, CMSD members transfer images and videos from the GoPro SD card into a local server. Then these files are automatically ingested into Google Cloud Storage for processing. From here, the machine learning algorithms detect which manholes need repair.
Google Cloud products are used throughout this project. Once the images and videos are in Google Cloud Storage, a workflow with Cloud Scheduler spins up the VM every night to detect if there are new videos on Cloud Storage. If there are, this triggers cloud machine learning, which reviews the numerous images and videos, rates each manhole, and determines if any require maintenance.
Machine learning to detect and grade manholes
SpringML applied a very systematic approach to detecting and grading the manholes. First, image processing ensures that the region of interest is only the section of the road in front of the vehicle to avoid any privacy concerns. Then SpringML applied a 5-step process using machine learning to detect and grade the manholes.

- SpringML developed two separate custom TensorFlow-based Mask R-CNN models. Mask-R-CNN is a deep neural network that is used for image segmentation tasks, which means it can separate different objects in an image or a video.
The first Mask-R-CNN model was created to accurately detect if an image showed a manhole cover. This was a critical step because sewer manhole covers look similar to water main covers. To make the model accurate, it was important that there be no false positives. SpringML used around 50 images of sewer manholes and other images to train and validate that the algorithm could successfully detect sewer manhole covers. - Once a manhole is accurately detected, the surrounding area is masked using image processing to focus on the manhole and then the cropped images are sent to the second model which is used to detect damage in the area around the manhole, called the apron. To reduce the processing time, the second model only does the inference if a manhole cover is detected.
- The second Mask-R-CNN model uses CMSD Guidelines to decide what types of damage contributed to the rating system. For training purposes, SpringML uses a TensorFlow-Keras inside a Virtual Machine on Google Cloud. The initial model was trained and the Keras weights were saved on Google Cloud Storage. This helped create a versioned system of the models as the model gets refined over time.
- Then, duplicated detections are cleaned up to have a unique detection for each manhole cover.
- Finally, Manholes are graded from a 1-5 rating, with 5 representing high damage and 1 representing low damage.

Once cloud machine learning has analyzed the new videos and images, the final scores are stored in BigQuery. Results are then served to members of CMSD via a simple web application where they can see which manholes need to be maintained. Two staff members review the ML results and determine priorities for repairs. One of the most interesting features of this project is that the model gets continually retrained based on feedback submitted in the web application. For example, if the model inaccurately detects a manhole, a member of CMSD can mark that in the web application and their feedback is immediately used to refine the model.
This solution shows how leveraging machine learning can streamline a necessary local government project and save money and labor while being highly scalable! Not only does this system streamline the manhole maintenance process, but it also allows for more frequent review of manhole conditions and provides a historical view of how the District’s manholes change over time.
Want to learn more about Google Cloud machine learning? Check out this tutorial to learn TensorFlow and Keras and check out more machine learning tools in our AI Platform.
Google Cloud expands availability of enterprise-ready generative AI

1081
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
Generative AI continues to develop at a blistering pace, making it more important than ever that organizations have access to enterprise-ready capabilities to help them leverage this disruptive technology.
Harnessing the power of decades of Google’s research, innovation, and investment in AI, Google Cloud continues to make generative AI available with baked-in security, data governance, and scalability across the board.
To this end, last month, we announced the general availability of Generative AI support on Vertex AI, giving our customers the ability to access powerful foundation models from Google Research and tools for customizing and applying them.
Today we are announcing the general availability (GA) of four important foundation models for Vertex AI. These include Imagen, PaLM 2 for Chat, Codey, and Chirp. For each of these models, organizations can access APIs on Model Garden and do prompt design and tuning on Generative AI Studio.
- Imagen includes four key features:
- Image generation for creating studio-grade images at scale
- Image editing to edit generated or existing images via text prompts
- Image captioning for creating captions of images at scale
- Visual Question & Answering (VQA) for interacting with, analyzing, and explaining images
- PaLM 2 for Chat follows the general availability of PaLM 2 for Text in June
- Codey supports code generation, completion, and code chat
- Chirp supports multilingual Speech AI
We’re also announcing Multimodal Embeddings API in preview, which lets customers combine the power of Vertex AI’s generative AI models with their proprietary data, to generate embeddings, or interchangeable vector representations, of their text and image data. These capabilities can enable data science teams to deliver a variety of downstream tasks such as image classification, content recommendations, and visual search.
In this blog post, we’ll explore what your organization can do with these powerful models and how Vertex AI provides the enterprise-ready capabilities you can use to get up and running with generative AI.
Helping to drive enterprise value from Generative AI models
Powerful models are the foundation of generative AI, but the software, tools, and infrastructure that surround these models are equally important for enterprise adoption. Organizations face challenges not only accessing these models, but also integrating AI while maintaining protection over intellectual property, adhering to regulations around data security and privacy, and ensuring models and applications are safe to use. Many organizations also want to use generative AI without incurring large costs or managing huge clusters.
We help address these challenges head-on with Vertex AI’s platform capabilities for scalable application integration, purpose-built AI infrastructure, secure and private data customization, and responsible use of this technology.
Let’s see how each of these pillars can help your organization.
Access models to build production-ready generative applications
Vertex AI can make it easy to access foundation models, as today’s model announcements attest. While models are an inextricable part of generative AI, the software that helps enterprises use this technology is equally important—which is why Vertex AI also offers a range of tools for tuning, deploying, monitoring, and maintaining models, so you can build differentiated applications using your own data.
Turning to today’s announcements, in May we announced Imagen, our foundation model for image generation. Now, we are excited to announce Imagen is generally available with an allowlist (i.e., approved access via your sales representative), letting onboarded customers start using image generation and editing capabilities. Visual Q&A and Captioning for production workloads are also generally available for all customers. Visual Q&A provides new ways to engage with image-based data like retail products or image libraries. This new capability can give you answers to questions about an image, helping you analyze large amounts of data quickly, and it can even help the visually impaired understand images or graphs that they wouldn’t be able to otherwise. Captioning, meanwhile, can make it easy to generate relevant descriptions for your images. Captions can help with indexing and searching, as well as assigning image descriptions to product listings on eCommerce websites.
“Imagen is beginning to power key capabilities within Omni, Omnicom’s open operating system, that will enable 17,000+ trained and certified users to create audience-driven customized images in minutes. Imagen has been instrumental in offering a scalable platform for image generation and customization. Integrating it into our platform allows us to expand the scope of audience-powered creative inspiration, at a scale that wasn’t previously possible,” said Art Schram, Annalect Chief Product Officer at Omnicom. “We’re starting to adopt the latest features like styles and fine tuning, and engineering data-driven prompts. We look forward to continuing to provide our users relevant visual inspiration in a responsible way.”
“The latest improvements in Imagen’s product preservation capabilities are a perfect match for Typeface’s focus on personalized AI for brands,” explained Vishal Sood, Head of Product at Typeface. “By combining Google Vertex AI’s Imagen with Typeface’s brand-personalized AI, we are able to help enterprises to create 10x personalized content in a fraction of time.”
Google Shopping recently built an application called Product Studio using Imagen on Vertex AI. Product Studio can enable merchants to create rich product images quickly and easily, at a fraction of the time it takes to do professional product photo shoots. “We’re excited about the feedback we’re getting from merchants in our early pilots, who say that Product Studio, which leverages Imagen on Vertex AI, helps them generate and publish lifestyle product photos directly to their product catalogs,” says Jeff Harrell, Google’s Senior Director of Product Management for Merchant Shopping.
Announced in May, PaLM 2 is a family of models that power dozens of Google products, including Bard and Duet AI in Google Cloud. With the PaLM 2 for Chat model, now generally available, you can leverage Google’s PaLM’s variety of abilities for multi-turn chat applications, such as shopping assistants, customer support agents, and more.
ThoughtSpot, provider of a widely-adopted business intelligence platform, is using PaLM 2 to build a new feature in ThoughtSpot for Google Sheets called “AI Explain,” which can instantly generate explanations of charts, visuals, and anomalies, and will launch new conversational AI and ML-enabled predictive forecasting capabilities into its analytics platform.
With Codey, your organization’s developers can accelerate a wide variety of coding tasks, helping to empower them to work efficiently and close skills gaps. The model enables not only code completion and code generation capabilities, but also chat to help with debugging, documentation, learning new concepts, and more. Since launching in preview in May, we’ve added additional programming languages including Go, Google Standard SQL, Java, Javascript, Python, and Typescript. We’ve also improved the quality of code responses and increased serving capacity, enabling your developers with the right tools to enter the era of generative engineering.
“Security and privacy are key to incorporating AI into the software development lifecycle,” said David DeSanto, Chief Product Officer at GitLab. “GitLab leverages Vertex AI to deliver new, AI-powered features with a privacy-first approach, including the ability to run our own models and leverage Codey foundation models built on top of PaLM 2. The GitLab DevSecOps platform empowers organizations to harness the benefits of AI for faster software delivery, while ensuring their data, intellectual property, and source code are protected.”
Originally released in May in preview, Chirp is a version of our 2 billion-parameter speech model, which was trained on millions of hours of audio and supports over 100 languages. Chirp achieves 98% accuracy on English and relative improvement of up to 300% in languages with less than 10 million speakers. Whether the use case involves customer support, transcriptions, or voice control, Chirp can help your organization communicate with customers and constituents inclusively, by engaging audiences in their native languages.
Last but not least, our Multimodal Embeddings API, now in preview, can unlock an array of new applications, such as image and text-based recommendations, by enabling the processing of text and images interchangeably. This capability complements our Text Embeddings API, which became generally available in June, and remains a recommended choice for those with fully text-based use cases. Multimodal Embeddings API makes it possible to categorize images and text together and can be crucial for use cases like retail recommendation systems that can provide relevant outputs from both images of products and text descriptions.
Match generative AI with infrastructure
Beyond access to models and tools for building generative AI apps, you need infrastructure to make sure your apps can scale and reliably perform — ideally without running into daunting compute costs or management overhead that distracts your technical talent from building innovative products. Google Cloud offers the choice and power to run smaller models running finite tasks at the lowest latency levels, as well as to run large models capable of cutting-edge experiments.
As our large language model customers are looking to scale up their projects and applications using our models, they often need assurances that their requests will be serviced with acceptable performance. This is especially critical for delivering real-time applications where customer service is paramount. Starting in August, Vertex AI will support provisioned, dedicated generative AI capacity that can deliver guaranteed throughput. This feature can be especially beneficial to customers who have a high volume of sustained workloads.
Leverage generative AI while protecting data and privacy
One capability enabled by Google Cloud is the ability to customize models using your own data. Vertex AI can help customers keep their data protected, secure, and private. When a company tunes a foundation model in Vertex AI, private data, model outputs, and prompts can be kept private, and they are never used in the foundation model training corpus. We recently published a whitepaper, “Adaptation of Large Foundation Models,” which outlines how we help protect customer data.
Auditability and compliance are essential to helping ensure the security and privacy of customer data. We also engage in comprehensive GDPR privacy efforts, including our transparency commitments for customer data usage and the support for our customer’s Data Protection Impact Assessments (DPIAs). Now, we’re excited to support HIPAA compliance for many of our generally available models on Vertex AI, so that healthcare and life science customers with whom we have a Business Associate Agreement can run workloads with Protected Health Information (PHI) data on Google Cloud.
Innovate responsibly
Our AI Principles put beneficial use, user safety, and avoidance of harms above business outcomes and are embedded in how we develop our AI products. We’ve conducted extensive reviews on our generative AI products to identify potential risks and have developed guardrails to mitigate these impacts. For example, to address concerns around safety, we’ve implemented safety filters for bias, toxicity, and other harmful content. We also equip our customers with the tools they need to help reduce risk within their applications and provide recommendations to help navigate responsible AI.
Bring the power of generative AI to your organization
With both a wide selection of foundation models and extensive, enterprise-grade platform capabilities, Vertex AI continues to unlock ways for your business or organization to access foundation models, tune them on your proprietary data, and leverage them for differentiated apps and digital experiences. To take the next step, visit our product page or reach out to our sales representatives to gain access to our latest capabilities.
5671
Of your peers have already watched this video.
56:20 Minutes
The most insightful time you'll spend today!
How Real Companies Are Innovating with AI Today—and the Benefits They’re Seeing
What do you get when you mix Target, women’s swim wear, and AI? “Joy!” says Mike McNamara, CIO and CDO, Target.
McNamara is just one of the many stories of real businesses conquering old challenges, and new disruptive industry challenges, with artificial intelligence.
McNamara, Nick Rockwell, CTO, The New York Times, Larry Colagiovanni, VP New Product Development, eBay, and Dirk John, CIO, LATAM Airlines, demonstrate how AI solutions allow businesses to find opportunity in chaos, innovation under tough conditions, and revenue in the most unlikely of places.
AI at Target
McNamara shows how AI is being applied to solve some of the most basic, yet critical, challenges that retailers like Target–and any business with large inventories—face. (start at 4:04)
AI at The New York Times
Nick Rockwell, CTO, The New York Times, shares how AI and big data tools are being used to leverage old assets and drive business ideas from them. (start at 12:18)
AI at eBay
Larry Colagiovanni, VP New Product Development, eBay, talks about how AI is helping eBay drive conversational commerce, help buyers sift through 1.1 billion items, and personalize the shopping experience.(start at 28:07)
AI at LATAM Airlines
Dirk John, CIO, LATAM Airlines, discusses how LATAM Airlines adopted advanced analytics tools in just a few weeks to accelerate the company’s understanding of their customers’ needs.(start at 35:58)
Insurer Uses Google Cloud AI to Battle Slow Growth: It Improves Sales by 5% in 8 Weeks

8839
Of your peers have already read this article.
7:30 Minutes
The most insightful time you'll spend today!
For a business to succeed in the long term, it needs to learn not just to adapt to inevitable change, but to harness it. South Africa-based PPS has been an insurance company since 1941 and today is the biggest mutual insurance provider in the country.
As a mutual company, PPS is owned by more than 200,000 members, making them shareholders. In recent years, PPS and other companies like it have been affected by a number of external factors.
“For one thing, technology platforms have brought in a new gig economy that has all kinds of implications for insurance,” says Avsharn Bachoo, CTO at PPS. “What we’ve been seeing is basically a disruption of the South African insurance industry. We chose to see that as an opportunity.”
“Our servers were at the end of their life cycle and we had to decide whether to refresh them or switch completely. To embrace the world of AI and machine learning effectively, we knew we needed a cloud-based infrastructure. We’ve found the answer in Google Cloud Platform.”
—Avsharn Bachoo, CTO, PPS
In early 2018, faced with an uncertain economic environment that was squeezing growth and profitability, PPS decided to transform itself from a traditional broker-based business into a digital insurance provider. A key pillar of this new strategy was to overhaul the company’s technology infrastructure. To turn the strategy into reality, Avsharn and his team chose Google Cloud Platform (GCP).
“Our servers were at the end of their life cycle and we had to decide whether to refresh them or switch completely,” says Avsharn. “To embrace the world of AI and machine learning (ML) effectively, we knew we needed a cloud-based infrastructure. We’ve found the answer in Google Cloud Platform.”
Power, speed, flexibility with Google Cloud Platform
Previously, PPS maintained an on-premises IT infrastructure, which worked for its traditional business but was unsuited for its new way of working. In early 2018, the company started working on new products for its members but this required large amounts of compute power that proved prohibitively expensive with on-premises servers. Even existing products were starting to require more than the infrastructure could deliver. Aging equipment meant that it’s testing and quality assurance environments bore little resemblance to the actual production environment.
“We had no pre-production environments at all,” says Avsharn, resulting in more work for developers after products had been released. Meanwhile, the capital required to buy and configure more servers for new projects meant fewer resources available for innovation, and left the company less able to react to changes in the market. PPS knew it had to find a cloud-based alternative.
Shortly after devising a new digital strategy, PPS engineers attended a training session on cloud infrastructure given by leading South African Google Cloud Partner Siatik. Impressed with the presentation, PPS engaged Siatik to help run a proof of concept for a cloud-based infrastructure, running on GCP. With on-site engineers and constant communication, Siatik formed a very close working relationship with PPS. “The team at Siatik was exemplary,” recalls Avsharn. “They were well-organized, with cutting-edge technical acumen and very creative solutions to our problems. They were real game-changers.”
“We wanted the platform to retrain its models in response to new data and improve its recommendations with more information. Normally this would be a manual process but Google Cloud ML Engine lets the models do this automatically.”
—Kimoon Kim, Lead Solution Architect and Data Engineer, Siatik
The proof of concept was successful, with GCP outperforming the existing infrastructure in terms of how it handled compute demands, databases, and storage.
“It’s the speed of GCP that really impresses us,” says Avsharn. PPS saw that GCP wasn’t just an opportunity to migrate its existing infrastructure to the cloud. With Siatik’s help, it redesigned its monolithic core architecture to one based around microservices using Google Kubernetes Engine (GKE). For data processing and storage, Cloud Dataflow and Cloud Datastore proved invaluable, while Stackdriver helped the IT team stay on top of logging and monitoring the system.
“Google Cloud makes migrations very easy,” says Brett St. Clair, CEO at Siatik. “It takes care of all the hard work with configurations and replications, so when we switch the machines on, everything is ready and working.”
The ease with which PPS migrated to GCP means that it can now tackle strategic goals much more quickly than before. The most ambitious of these is an AI-powered product recommendation platform. Information is collected from customers who opt in at a defined point in their journey, this database is queried using BigQuery, and the information is fed into the platform. The AI model then calculates the most appropriate products for each member, according to their personal history.
“Most of the product recommendation engines out there are based on clustering, where you’re offered products based on your peer groups,” explains Avsharn. “For the first time, we can make recommendations to members based on their individual preferences and historical behavior. That’s really powerful for us.”
Siatik helped PPS use TensorFlow and Cloud Machine Learning Engine to build the AI platform. For the engineers, these easy-to-use tools helped speed up the process considerably, allowing them to host the models locally without any fuss. Previously, it took one to three months to manually build the model and match an offer to a customer. With the AI platform, a match takes just a few minutes. Cloud ML Engine, in particular, helped the platform adapt to new information on the fly and easily make adjustments to its hyperparameters, that is, preset variables which define the model-training process.
“We wanted the platform to retrain its models in response to new data and improve its recommendations with more information,” says Kimoon Kim, Lead Solution Architect and Data Engineer at Siatik. “Normally this would be a manual process but Google Cloud ML Engine lets the models do this automatically.”
“Google Cloud helped us cancel out a lot of the noise around machine learning and AI. We don’t have to build new complicated algorithms or hire huge teams of data scientists to benefit. We just bring our data and use the right tools to focus on what’s really important.”
—Avsharn Bachoo, CTO, PPS
Harnessing artificial intelligence for real-world results
PPS deployed its new AI recommendation platform in December, 2018. Just a couple of months later, its impact was clear. “In around eight weeks, we saw a 5 percent growth in sales,” says Avsharn. “It’s been a direct result of building our recommendation platform with Google Cloud. We can offer the right products to the right members.”
For developers and engineers at PPS, working with Google Cloud gives them access to high performance technology and automation options with GKE. As a result, the infrastructure runs 70 percent faster than before with fewer cores and less memory. Developers can also work in mature testing environments, and for the first time, are able to build pre-production environments, leading to better quality products. More strategically, moving to a serverless, cloud-based infrastructure has helped PPS take control of its budget, moving away from intermittent, large capital spends to more manageable, project-to-project flows of operational expenditure. The company expects to see savings of around 50 percent, or $695,000.
“We have a lot more flexibility with our resources thanks to Google Cloud,” says Avsharn. “When we have a new idea, we don’t have to outlay new capital such as servers before we can even start working on it. We just spin up instances when we want and spin them back down when we’re done.”
With the AI platform deployed and working well, PPS is already looking at ways to improve it, including real-time updates and further automation. Soon, the company will integrate the platform with more sales campaigns for more effective targeting to boost sales even further. Meanwhile, it’s also experimenting with machine learning to spot patterns in data at scale for fraud analytics and risk assessment.
For PPS, working with Google Cloud has helped it transform quickly and effectively from disrupted to disruptor. The company is now looking to gain the same transformative effects by implementing G Suite for increased productivity and collaboration.
“Google Cloud helped us cancel out a lot of the noise around machine learning and AI,” says Avsharn. “We don’t have to build new complicated algorithms or hire huge teams of data scientists to benefit. We just bring our data and use the right tools to focus on what’s really important.”
Custom Voice Feature Can Help Brands Tweak IVR for Better Customer Experiences

5140
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
With the rise of digital assistants and conversational interfaces, people have grown accustomed to hearing and speaking to synthetic voices. But what do those voices sound like? Often, pretty repetitive. We’re all familiar with the Google Assistant voice, for example.
That’s why we are excited to announce the general availability of Custom Voice in our Cloud Text-to-Speech (TTS) API, a new feature that lets you train custom voice models with your own audio recordings to create unique experiences.
For businesses looking to build a strong brand identity, establishing a unique voice can help turn mobile app interactions or customer service based on interactive voice responses (IVR) into differentiated customer experiences. Our TTS API has included a speech synthesis service with a static list of voices for some time, but now, with Custom Voice, moving beyond these predefined options is easier than ever.
Custom Voice lets you simply submit your audio recordings to get access to the new voice directly in the TTS API. Custom Voice TTS includes guidance on the audio requirements to help make sure you generate a high quality custom TTS voice model. Once this new model is trained, all you have to do to start using the newly trained voice is reference the model ID in your calls to the Cloud TTS API.
At Google, we are committed to building safe and accountable AI products, not only because it’s the right thing to do, but because it is a critical step in ensuring successful use in production. As part of Google Cloud’s Responsible AI governance process, we conducted a deep ethical evaluation of Custom Voice TTS, and its relation to synthetic media, in order to surface and mitigate potential harms that it may create. If you are interested in Custom Voice TTS, there is a review process to help ensure each use case is aligned with our AI Principles and adequate voice actor consent is given.
Additionally, to verify that voice actors are actually the ones producing the audio, you will need to submit an audio file producing a sentence that Google Cloud chooses (for example: “I agree that my voice will be used to create a synthetic custom Text-to-Speech voice).
We’re looking forward to seeing this API help businesses solve problems in an easy, fast, and scalable way. TTS Custom Voice is now GA in these languages:
English (US)
English (AU)
English (UK)
Spanish (US)
Spanish (Spain)
French (France)
French (Canada)
Italian (Italy)
German (Germany)
Portugues (Brazil)
Japanese (Japan)
We plan to continue expanding this lineup in order to meet your needs. Ready to try for yourself? Contact your seller to get started on your use case evaluation today!
Supercharging Security with Generative AI

1512
Of your peers have already read this article.
3:30 Minutes
The most insightful time you'll spend today!
At Google Cloud, we continue to invest in key technologies to progress towards our true north star on invisible security: making strong security pervasive and simple for everyone. Our investments are based on insights from our world-class threat intelligence teams and experience helping customers respond to the most sophisticated cyberattacks. Customers can tap into these capabilities to gain perspective and visibility on the most dangerous threat actors that no one else has.
Recent advances in artificial intelligence (AI), particularly large language models (LLMs), accelerate our ability to help the people who are responsible for keeping their organizations safe. These new models not only give people a more natural and creative way to understand and manage security, they give people access to AI-powered expertise to go beyond what they could do alone.
At the RSA Conference 2023, we are excited to announce Google Cloud Security AI Workbench, an industry-first extensible platform powered by a specialized, security LLM, Sec-PaLM. This new security model is fine-tuned for security use cases, incorporating our unsurpassed security intelligence such as Google’s visibility into the threat landscape and Mandiant’s frontline intelligence on vulnerabilities, malware, threat indicators, and behavioral threat actor profiles.
Google Cloud Security AI Workbench powers new offerings that can now uniquely address three top security challenges: threat overload, toilsome tools, and the talent gap. It will also feature partner plug-in integrations to bring threat intelligence, workflow, and other critical security functionality to customers, with Accenture being the first partner to utilize Security AI Workbench.
The platform will also let customers make their private data available to the platform at inference time; ensuring we honor all our data privacy commitments to customers. Because Security AI Workbench is built on Google Cloud’s Vertex AI infrastructure, customers control their data with enterprise-grade capabilities such as data isolation, data protection, sovereignty, and compliance support.

Preventing threats from spreading beyond the first infection
We already provide best-in-class capabilities to help organizations immediately respond to threats. But what if we could not just identify and contain initial infections, but also help prevent them from happening anywhere else? With our AI advances, we can now combine world class threat intelligence with point-in-time incident analysis and novel AI-based detections and analytics to help prevent new infections. These advances are critical to help counter a potential surge in adversarial attacks that use machine learning and generative AI systems. That’s why we’re excited to introduce:
- VirusTotal Code Insight uses Sec-PaLM to help analyze and explain the behavior of potentially malicious scripts, and will be able to better detect which scripts are actually threats.
- Mandiant Breach Analytics for Chronicle leverages Google Cloud and Mandiant Threat Intelligence to automatically alert you to active breaches in your environment. It will use Sec-PaLM to help contextualize and respond instantly to these critical findings.
These new updates build on the existing AI in Google’s industry-leading solutions. For example, Chronicle Security Operations already uses frontline intelligence, integrated reasoning, and machine learning to identify initial infections, prioritize impact, and contain threats. Another example is reCAPTCHA Enterprise, which uses image noising capabilities to help protect your site from adversaries that leverage novel AI advances, greatly enhancing our defenses against bots.
Adding intelligence to reduce toil
At Google Cloud, we help organizations modernize security wherever they are, in part by simplifying their security tools and controls whenever possible. Advances in generative AI can help reduce the number of tools organizations need to secure their vast attack surface areas and ultimately, empower systems to secure themselves. This will minimize the toil it takes to manage multiple environments, to generate security design and capabilities, and to generate security controls. Today, we’re announcing:
- Assured OSS will use LLMs to help us add even more open-source software (OSS) packages to our OSS vulnerability management solution, which offers the same curated and vulnerability-tested packages that we use at Google.
- Mandiant Threat Intelligence AI, built on top of Mandiant’s massive threat graph, will leverage Sec-PaLM to quickly find, summarize, and act on threats relevant to your organization.
These announcements build on existing capabilities that help customers centralize visibility and control, detect targets, and improve security across their platform. For example, Security Command Center (SCC) uses always-on machine learning to detect malicious scripts executing in the customer container environment and immediately alert the customer. In addition, Cloud Data Loss Prevention leverages machine learning to find and classify data, and with Confidential Computing you can collaborate on, train, and deploy sensitive and regulated AI models in the cloud, all while preserving confidentiality.
Evolving how practitioners do security to close the talent gap
At Google, we believe that to truly democratize security, we need to first acknowledge that AI will soon usher in a new era for security expertise that will profoundly impact how practitioners “do” security. Most people who are responsible for security — developers, system administrators, SRE, even junior analysts — are not security specialists by training.
Imagine a world where novices and security experts are paired with AI expertise to free themselves from repetition and burnout, and accomplish tasks that seem impossible to us today. To help power this evolution, we’re embedding Sec-PaLM-based features that can make security more understandable while helping to improve effectiveness with exciting new capabilities in two of our solutions:
- Chronicle AI: Chronicle customers will be able to search billions of security events and interact conversationally with the results, ask follow-up questions, and quickly generate detections, all without learning a new syntax or schema.
- Security Command Center AI: Security Command Center will translate complex attack graphs to human-readable explanations of attack exposure, including impacted assets and recommended mitigations. It will also provide AI-powered risk summaries for security, compliance, and privacy findings for Google Cloud.
These new releases bolster our existing efforts to tackle these issues through capabilities like IAM Recommender, which suggests permissions better suited to actual usage patterns. We will soon be augmenting this capability to cover organizational policies, further enabling the administrator to help improve the security posture of their organization. In addition, Mandiant Automated Defense applies machine learning to help reduce the repetitive Tier 1 alert triage problem and address alert fatigue.
Offering availability
VirusTotal Code Insight, available now in Preview, is our first example of putting Security AI Workbench to work for our customers. We will be rolling out other offerings to trusted testers in coming months, and they will be available in Preview more broadly this summer. Click here for the demo.
Security AI Workbench, including Sec-PaLM and partner integrations, in addition to the product innovations described in our demo, are all building blocks for a larger effort to elevate security across the ecosystem. So far, that effort:
- Provides assistive functions to rapidly develop IT generalist talent to Tier 1 security operator status in a way that wasn’t previously feasible. Security Command Center now can summarize threat intelligence insights and findings for Google Cloud, and Chronicle can quickly generate YARA-L rules or other detections.
- Provides advanced functions such as iterative query and multivariate detection generation, conversational filtering and interaction with results, and smart case awareness to empower advanced Tier 2 and 3 security operators to focus on threat analysis instead of struggling with process and toil. Mandiant Threat Intelligence users now can elevate their core competencies to hunt, investigate, and remediate threats — using the same tools our own Mandiant experts use.
- Fuses threat intelligence and AI-based analytic capabilities, which are unsurpassed in the market. VirusTotal Code Insight enables security teams to help gain insights and identify threats in suspicious code. This can significantly enhance their ability to detect and mitigate potential attacks.
However, this is just an initial step. We’ll continue to iterate and innovate, and we encourage customers and partners to leverage Security AI Workbench in new and exciting ways. Moving forward, we anticipate many new use cases to emerge over time.
Building a safer future
While generative AI has recently captured the imagination, Sec-PaLM is based on years of foundational AI research by Google and DeepMind, and the deep expertise of our security teams. This work includes new efforts to expand our partner ecosystem to provide businesses with security capabilities at every layer of the cybersecurity stack. We have only just begun to realize the power of applying generative AI to security, and we look forward to continuing to leverage this expertise for our customers and drive advancements across the security community.
More Relevant Stories for Your Company

Google Cloud Helped Digitec Galaxus Personalize Over 2 Million Newsletters in a Week
Digitec Galaxus AG is the biggest online retailer in Switzerland, operating two online stores: Digitec, Switzerland's online market leader for consumer electronics and media products, and Galaxus, the largest Swiss online shop with a steadily growing range of consistently low-priced products for almost all daily needs. Known for its efficient, personalized shopping experiences,

How Constellation Brands’ Direct-to-Customer Tech Delivers Economic Impact across Business Portfolio
Editor’s note: Today we’re hearing from Ryan Mason, Director, Head of DTC Growth & Strategy, at alcoholic beverage firm, Constellation Brands on the company’s shift to Direct-to-Consumer (DTC) sales and how Google Cloud’s powerful technology stack helped with this transformation. It’s no secret that consumer businesses have been up-ended in

Streamlining Business Processes with Google’s Document AI: Invoices, Contracts, and Beyond
Editor's note: In this post, I'll be showing some amazing ways Document AI can help you extract meaning from your documents - keep reading, or jump directly into a tutorial using the Cloud Console! Documents are a crucial part of most businesses, used to store and communicate important information. The variety

Case Study: Twitter is Taking Their CX to The Next Level with AutoML
Editor’s note: Since launching its Spaces feature, Twitter has demonstrated that hearing people’s voices can bring conversations on Twitter to life in a completely new way. Next, it aimed to make it easier for customers to join and listen to live conversations they personally care about. In this blog, we






