If you’ve been exploring recently-launched consumer generative AI tools like Bard and thinking about how to build similar experiences for your business, Generative AI App Builder, or Gen App Builder for short, is here to get you started.

Gen App Builder is part of Google Cloud’s recently announced generative AI offerings and lets developers, even those with limited machine learning skills, quickly and easily tap into the power of Google’s foundation models, search expertise, and conversational AI technologies to create enterprise-grade generative AI applications. 

“Google Cloud’s leading AI technology enables STARZ customers to discover more relevant content, increasing engagement with, and the likelihood of completing the content served to them,” says Robin Chacko, EVP Direct-to-Consumer, STARZ. “We’re excited about how generative AI-powered search will help users find the most relevant content even easier and faster.”

Gen App Builder is exciting because unlike most existing generative AI offerings for developers, it offers an orchestration layer that abstracts the complexity of combining various enterprise systems with generative AI tools to create a smooth, helpful user experience. Gen App Builder provides step-by-step orchestration of search and conversational applications with pre-built workflows for common tasks like onboarding, data ingestion, and customization, making it easy for developers to set up and deploy their apps. With Gen App Builder developers can: 

• Build in minutes or hours. With access to Google’s no-code conversational and search tools powered by foundation models, organizations can get started with a few clicks and quickly build high-quality experiences that can be integrated into their applications and websites. 
• Combine the power of foundation models with information retrieval to find relevant, personalized information. Enterprises can build apps that understand user intent via natural language, and surface the right information with associated citations and attributions from a company’s public and private data. They can also fully control what data their applications access and the content or topics they want to address.
• Build multimodal apps that can respond with text, images, and other media. Gen App Builder supports not just text, but also other modalities such as images and videos. It allows developers to build apps using a combination of text and images as inputs to find information across documents, photos, and video content, enabling richer customer interactions. 
• Combine natural conversations with structured flows. Developers can granularly blend the output of foundation models with controls to ground answers in enterprise content, and step-by-step conversation orchestration to guide customers to the right answers.
• Provide the ability to transact and connect to third party apps and services. Gen App Builder makes it simple to create digital assistants and bots that not only serve content, but also connect to purchasing and provisioning systems to enable transactions from the conversational UI, and escalate customer conversations to a human agent when the context demands. 

A new generation of conversational AI experiences and assistants 

Consumers of enterprise applications expect to interact with technology in a seamless, conversational way to quickly find the information they need and act on it. Gen App Builder can help reinvent these customer and employee experiences by ingesting large, complex datasets that are specific to your company–from websites, documents, and transactional systems like billing and inventory, to emails, chat conversations, and more. These AI-powered apps can synthesize information across all of these sources to provide specific, actionable responses, using only the data you have provided. 

Some of the most popular uses are in customer service, where generative apps can contribute to increasing revenue, customer satisfaction, and customer loyalty. For example, if a retail customer reaches out to modify an order, a virtual agent can help them change it to another product. The customer doesn’t even need to provide the new product name—they can just upload an image and let the agent guide them through the rest. Watch this demo to see how a retail chatbot can use multimodal capabilities to help a consumer navigate various options on the website, including giving the customer ideas on how to use the product and even helping them complete the purchase with the ability to transact within the conversational UI. This scenario could apply to multiple industries and use cases, ranging from consumer goods and public services, to finance and internal corporate systems like intranets.

Combining the power of Google-quality search with foundation models

Finding the right information from data across the organization is a critical requirement within any enterprise. Yet it can be challenging to build high-quality enterprise search experiences with existing tools. Current systems struggle to understand user intent, are difficult to implement and customize, and don’t provide a high-quality user experience. 

One of the most exciting features of Gen App Builder is the ability to combine the power of Google-quality search with generative AI to help enterprises find the most relevant and personalized information when they need it. With Gen App Builder, enterprises can build conversational search experiences across their public and private data in minutes or hours with no coding experience. 

Enabling multimodal search across text, images and video within the enterprise is a key aspect of the search experiences in Gen App Builder. In addition to providing high-quality search results, Gen App Builder can conveniently summarize the results and provide corresponding citations in a natural, human-like fashion. Gen App Builder also automatically extracts key information from the data and enables personalized results for users. Watch this demo to see how these capabilities can come together to transform the search experience for employees at a financial services firm. The ability to integrate Google-quality search within the enterprise’s applications means they can enjoy a new level of data utilization, drive increased process efficiencies, and provide delightful experiences to their employees and customers.

“Customers have been shopping at Macy’s for generations. Being able to deliver 360° personalization and contextual recommendations will help ensure that Macy’s is still providing future generations of shoppers with a seamless, exceptional experience,” said Bennett Fox-Glassman, Senior Vice-President, Customer Journey, Macy’s. “We’ve already realized an increase in revenue per visit and conversion rates had great success using Google Cloud’s AI technology and are looking forward to exploring how these latest announcements bring together Natural Language Processing and Generative AI capabilities to deliver next-gen search and conversational experiences for our customers.”

The ability to intuitively interact with complex data across a variety of sources allows organizations to better serve their customers and deliver more relevant offerings. Combined with conversational and fulfillment abilities, the potential for improving customer engagement and employee productivity is immense. We’re excited to see how developers and enterprises use a mix of these capabilities to power new experiences and revenue opportunities.

If you’re interested in a closer look at the Gen App Builder, tune into this session at the Data Cloud & AI Summit. Take a step forward to getting hands-on and join the waitlist for our trusted tester program. And finally, bookmark our generative AI landing page to keep abreast of the latest news, updates and possibilities from this exciting new world of Gen Apps.

Investing in Artificial Intelligence (AI) can bring a competitive advantage to your organization. If you’re in charge of an AI or Data Science team, you’ll want to measure and maximize the value that you’re providing. Here is some advice from our years of experience in the field. 

A checklist to embark on a project: 

As you embark on projects we’ve found it’s good to have the following areas covered: 

• Have a customer. It’s important to have a customer for your work, and that they  agree with what you’re trying to achieve. Be sure to know what value you’re delivering to them. 
• Have a business case.  This will rely on estimates and assumptions, and may take no more than a few minute’s work.  You should revise this, but always know what justifies your team’s effort, and what you (and your customer) expect to get in return. 
• Know what process you will change or create. You’ll want to put your work in production, so you have to be clear about what business operations are changing or created around your work and who needs to be involved to make it happen
• Have a measurement plan. You’ll want to show that ongoing work is impacting some relevant business indicator. Measure and show incremental value. The goal of these measurements is to establish what has changed because of your project that would otherwise not have changed. Be sure to account for other factors like seasonality or other business changes that may affect your measurements.
• Use all the above to get your organization’s support for your team and your work. 

What measures to use?

As you start the work, what measures and indicators can you use to show that your team’s work is useful for your organization?

How many decisions you make. A major function of ML is to automate and optimize decisions: which product to recommend, which route to follow, etc. Use logs to track how many decisions your systems are making. 

Changes to revenue or costs. Better and quicker decisions often lead to increased revenue or savings. If possible, measure it directly, otherwise estimate it (for example fuel costs saved from less distance traveled, or increased purchases from personalized offers). 

As an example, the Illinois Department of Employment Security is using Contact Center AI to rapidly deploy virtual agents to help more than 1 million citizens file unemployment claims. To measure success the team tracked the two outcomes:  (1) the number of web inquiries and voice calls they were able to handle, and (2) the overall cost of the call center after the implementation. Post implementation, they were able to observe more than 140,000 phone and web inquiries per day and over 40,000 after-hours calls per night. They  also anticipate an estimated annual cost savings of $100M based on an initial analysis of IDES’s virtual agent data (see more in the link to case study).

Implementation costs. The other side of increased revenue or savings, is to put your achievements in the context of how much they cost. Show the technology costs that your team incurs and, ideally, how you can deliver more value, more efficiently. 

How much time was saved.  If the team built a routing system then it saved travel time, if it built an email classifier then it saved reading time, etc. Quantify how many hours were given back to the organization thanks to the efficiency of your system. 

In the medical field, quicker diagnostics matter. Johns Hopkins University’s Brain Injury Outcomes (BIOS) Division has focused on studying brain hemorrhage aiming to improve medical outcomes. The team identified the time to insights as a key metric in measuring business success. They experimented with a range of cloud computing solutions like Dataflow, Cloud Healthcare API, Compute Engine, and AI Platform for distributed training to accelerate iterations. As a result, in their recent work they were able to accelerate insights from scans from approximately 500 patients from 2,500 hours to 90 minutes.

How many applications your team supports. Some of your organization’s operations don’t use ML (say reconciling financial ledgers) but others do. Know how many parts of your organization benefit from the optimization and automation your team builds.

User experience. You may be able to measure your customer’s experience: fewer complaints, better reviews, reduced latency, more interactions, etc. This is valid both for internal and external stakeholders. At Google we measure usage and regularly ask for feedback on any internal system or process.

One of our customers, The City of Memphis, is using VisionAI and ML to tackle a common but very challenging issue: identifying and addressing potholes.  The implementation team identified the percentage increase of potholes identified as one of the key metrics along with accuracy and cost savings. The solution captures video footage from it’s public vehicles and leverages Google Cloud capabilities like Compute Engine, AI Platform, and BigQuery to automate the review of videos.  The project increased  pothole detection by 75% with over 90% accuracy. By measuring and demonstrating these outcomes, the team proved the viability of a cost-effective, cloud-based machine learning model and is looking into new applications of AI and ML that will further improve city services and help it build a better future for its 652,000 residents. 

Acknowledgements

Filipe and Payam would like to thank our colleague and co-author Mona Mona (AI/ML Customer Engineer, Healthcare and lifesciences) who contributed equally to the writing.

Natural language understanding (NLU) is getting increasingly better at solving complex problems and these language breakthroughs are creating big waves in Artificial Intelligence. For example, new language models are enabling Everyday Robots to create more helpful robots that can break down user instructions and have even enabled people to generate imaginative visuals from complex text prompts. 

These leaps in NLU are powered by neural networks trained to understand human language. This technology has greatly advanced since the introduction of Google’s Transformer architecture in 2017 with the introduction of large models trained on massive amounts of data like GPT-3 and, even more recently, with GLaM, LaMDA, and PaLM. This latest generation of models are called Large Language Models (LLMs) because of their sheer size and the vast volumes of data on which they are trained, and they can be applied to a range of tasks to create more powerful digital assistants, generate better search results and product recommendations, enforce smarter platform curation and safety features, and much more. 

For these reasons, we’re pleased to announce we’ve updated the Google Cloud Natural Language (NL) API with a new LLM-based model for Content Classification. 

With an expansive pre-trained classification taxonomy, the newest version of Content Classification from the Natural Language API leverages the latest Google research to improve customer use cases spanning actionable insights on user trends, to ad targeting, to content-based filtering. In this article, we’ll explore the NL API’s new capabilities, which are the first of many efforts we’ll be making to bring the power of LLMs to Google Cloud. 

How LLMs help machines understand human language 

As Google Cloud VP and General Manager of AI and Industry Solutions, Andrew Moore has argued, if computer systems become more conversant with natural human languages, they become a foundation for more sophisticated use cases, able to not only understand user intent but also create complex bespoke solutions. Google has been a leading research force in this space, with LLM projects like LaMDA, PaLM and T5 contributing to the Cloud NL API’s improved v2 classification model.

Parsing language is a difficult AI task for machines due in part to the contextual and individual interpretation of words or phrases. The word “server,” for example, could refer to a computer, a restaurant employee, or a tennis player. To understand the word, a model needs to be trained around not only a basic definition but also the context and positioning of the word within a sentence or conversation and its evolving connotations. Because they process voluminous training data via Transformers, LLMs are well-suited to this type of work. 

Thanks to the integration of Google’s latest language modeling technology, and an updated and expanded training data set, the next generation of the Content Classification API not only has over 1,000 labels (up from around 600 previously), but now also supports 11 languages (with Chinese, French, German, Italian, Japanese, Korean, Portuguese, Russia, Spanish, and Dutch joining previously-available English)—and does so with improved accuracy.

​​AI raises questions about the best way to build fairness, interpretability, privacy, and security into these new systems in order to benefit people and society. At Google, we prioritize the responsible development of AI and take steps to offer products where a responsible approach is built in by design. For Content Classification, we limited use of sensitive labels and conducted performance evaluations. See our Responsible AI page for more information about our commitments to responsible innovation. 

Get Started 

Today’s announcement is just the first step in bringing LLM capabilities to Google Cloud AI products, and we’re excited to see how our more powerful Natural Language API helps developers, analysts and data scientists generate insights and offer superior experiences. Our early adopters are implementing the API to improve user recommendations, display ad targeting, and insights about new trends.

If you’re ready to get started with this major leap in Google Cloud language services, visit our NL API documentation, and to learn more about Google Cloud’s AI services, visit our AI and machine learning products page.

Editor’s note: This blog has been updated since it originally published to reflect the total number of generative AI courses that have become available since the publish date.

An AI/ML (artificial intelligence/machine learning) career path can be a great specialty area within the cloud—and one of the most accessible! Because this area is constantly developing, most recently with the rise of generative AI, I want to share some recommendations to help you chart a sustainable career as AI/ML continues to evolve.

Generative AI falls under the overall category of AI and offers a new and exciting way of interacting with information, brands, and other people. Let’s talk about how these disciplines work together and about materials available to help you upskill in these areas.

To this end, we are happy to announce a new set of generative AI training content available at no cost. So, whether you are just getting started or already have a more advanced role, read on to find ways to help reach your desired position. 

ML compared to AI

While the terms ML and AI are often used interchangeably, there are distinct differences. AI is an umbrella concept wherein machines are taught to perform tasks normally associated with human intelligence, such as decision-making and language interaction. ML is a subset of AI dedicated to taking data from the past and training algorithms to create models that can perform highly complex tasks without being explicitly programmed. It’s the basis for most forms of AI that people interact with, like virtual assistants, music recommendations, and chatbots.

Data engineers and ML engineers work with data scientists to get insights from data. They are needed to create software models and get clear results, and develop deployable applications. These skilled roles are needed in every industry!

Change the world for the better with generative AI

I recently wrote about four key pillars of technology trends expected in the next decade, including the role of AI/ML in the cloud environment as one of those pillars, and how you can bridge the skills gap and build your career.

Generative AI is a new type of ML that has made a lot of headlines recently. Research from CIO Dive finds that seven in 10 executives say their companies are investigating or exploring generative AI. Now is a great time to become an expert, while we are at the cusp of this technology becoming more widely adopted. 

To get to generative AI, we need to talk first about deep learning. Deep learning is a subset of ML that uses artificial neural networks to process more complex patterns than traditional ML. Generative AI sits still further down the funnel, as a subset of deep learning that typically involves the Transformer architecture. Essentially, it’s a type of AI that can map long-range dependencies and patterns in large training sets, then use what it learns to produce new content, including text, imagery, audio, and synthetic data.

Generative AI relies on large models, such as large language models (LLMs) that can classify and generate text, answer questions, and summarize documents. For more detail about how this works, check out the video below from Google I/O 2023, which features great information from Dr. Gwendolyn Stripling, Artificial Intelligence Technical Curriculum Developer for Google Cloud.

How Google is offering generative AI

Here at Google, we’ve been heavily invested for decades in research and innovation aimed at helping businesses, governments, and developers maximize the potential of AI. Our vision is to empower builders, innovators, developers, and doers to use AI in unique, responsible, productive ways. 

To deliver on this, we recently announced a variety of solutions to bring generative AI into our offerings, beginning with Generative AI support on Vertex AI and Gen App Builder. 

Vertex AI is Google Cloud’s ML platform for training and deploying ML models and AI applications. With Generative AI support on Vertex AI, data science teams and developers can access foundation models from both Google and other sources, helping them to quickly build, customize, and deploy models for their own use cases. 

As part of generative AI support on Vertex AI, Model Garden and Generative AI Studio are now in preview, including access to PaLM 2 for Text and Chat and Embeddings API for Text, while other features and services are available to select trusted testers. If you’d like early access to Google Cloud’s AI products, join the waitlist here.

We announced our latest PaLM model in production at I/O 2023: PaLM 2. Today, it powers more than 25 products! Additionally, we’ve fine-tuned PaLM 2 for specific use cases, including security and medical domains. You’ve probably also heard of Bard, our experiment for conversational AI, which is fully running on PaLM 2. 

Our Gen App Builder aims to make customer, partner, and employee interactions more effective and helpful. With it, developers — even those with limited data science expertise — can quickly create bots, chat apps, digital assistants, custom search engines, and more. Gen App Builder even makes it possible to create some generative AI apps without any coding skills. 

Now, let’s bring these products to life! Not sure exactly the use case or in need of a little inspiration? Check out this blog from Google Cloud CEO Thomas Kurian to see how customers and partners in the ecosystem are bringing ideas to life with generative AI. And watch this video to see generative AI in action!

Build your AI/ML skills and validate your knowledge to grow your career

There is a lot of excitement around generative AI—it’s truly a brand new path to consider for your AI/ML career! Check out my recommendations below for training options for AI/ML roles. These will help you gain critical skills as generative AI becomes more widely available.

*NEW* training materials, specific to generative AI technologies
Generative AI Learning Path – no cost training 

• [Course] Introduction to Generative AI (1 day)
• [Course] Introduction to Large Language Models (1 day)
• [Course] Attention Mechanism (1 day)
• [Course] Transformer Models and BERT Model (1 day)
• [Course] Introduction to Image Generation (1 day)
• [Course] Create Image Captioning Models (1 day)
• [Course] Encoder-Decoder Architecture (1 day)

Additional AI/ML training – varying learning credits required to complete on Google Cloud Skills Boost 
Introductory level

• [Course with completion badge] How Google Does Machine Learning (1 day)
• [Course with completion badge] MLOps: Getting Started (1 day)
• [Skill badge] Get started with TensorFlow on Google Cloud (8 hours)  
• [Skill badge] Perform foundational ML, AI and data tasks in Google Cloud (7 hours)
• [Course with completion badge] Language, Speech, Text, and Translation with Google Cloud APIs (5 hours)

Intermediate/Multi-level

• [Learning Path] Machine Learning Engineer (Collection of 15 video courses and labs) 
• [Skill badge] Build and Deploy Machine Learning Solutions on Vertex AI (1 day)

Advanced

• [Training + certification exam] Google Cloud Professional ML Engineer Certification (varied time)
• [Skill badge] Machine Learning with TensorFlow in Vertex AI (90 minutes)
• [Course with completion badge] Natural Language Processing in Google Cloud (1 day)

You can also catch up on hands-on demos in our most recent Cloud OnBoard: From Data to AI with BigQuery and Vertex AI. In it, we guide you through all steps of the data-to-AI solution.

Coming up on June 29, we are hosting Getting Started with Vertex Generative AI as part of Innovators Live. Register now!

I also encourage you to consider an Innovators Plus subscription. You get access to over 700 labs, skill badges and courses (including many that are specific to AI/ML), a certification exam voucher, and up to $1,000 in Google Cloud credits. You’ll also get access to live learning events and technical briefings with Google Cloud experts, plus 1:1 consultations (talk with us about your upcoming generative AI project!). It’s a $1,500 package for only $299. Learn more!