Generative AI is a rapidly expanding technology with a wide range of potential applications. Google Cloud Learning is thrilled to offer a new, no-cost Generative AI Fundamentals skill badge. This skill badge is designed for anyone eager to learn about the power of generative AI. No technical skills or prior knowledge required!

For those new to digital credentials, Google Cloud skill badges are digital credentials issued by Google Cloud in recognition of your knowledge of Google Cloud products and services. Individuals can earn skill badges on Google Cloud Skills Boost, and can share their skill badge to their social media profile and resume.

Watch the short videos in the generative AI courses and complete the final quiz to earn the Generative AI Fundamentals skill badge pictured below. In as little as 120 minutes, you will learn the basics of how generative AI works, how Google Cloud AI technology can be used by businesses and individuals, and how the principles of responsible AI lead to ethical decisions about the use of generative AI. 

By earning the skill badge, you will demonstrate your understanding of foundational concepts in generative AI.

The topics covered in the courses include:

1. Introduction to Generative AI 

• Explain how generative AI works
• Describe generative AI model types
• Describe generative AI applications

2. Introduction to Large Language Models

• Define large language models (LLMs)
• Describe LLM use cases
• Explain prompt tuning
• Describe Google’s generative AI development tools

3. Introduction to Responsible AI

• Identify the need for a responsible AI practice within an organization
• Recognize that decisions made at all stages of a project make an impact in Responsible AI
• Recognize that organizations can design an AI infrastructure to fit their own business needs and values

Earn the skill badge and show off your generative AI knowledge today! And for more content to help you stay up to date with generative AI, check out “The Prompt” and our generative AI primer for executives on Transform with Google Cloud.

As Googlers transitioned to working from home during the pandemic, more and more turned to chat-based support to help them fix technical problems. Google’s IT support team looked at many options to help us meet the increased demand for tech support quickly and efficiently. 

More staff? Not easy during a pandemic. 

Let service levels drop? Definitely not. 

Outsource? Not possible with our IT requirements. 

Automation? Maybe, just maybe…

How could we use AI to scale up our support operations, making our team more efficient?

The answer: Smart Reply, a technology developed by a Google Research team with expertise in machine learning, natural language understanding, and conversation modeling. This product provided us with an opportunity to improve our agents’ ability to respond to queries from Googlers by using our corpus of chat data.  Smart Reply trains a model that provides suggestions to techs in real time. This reduces the cognitive load when multi-chatting and helps a tech drive sessions towards resolution.

In the solution detailed below, our hope is that IT teams in a similar situation can find best practices and a few shortcuts to implementing the same kind of time saving solutions. Let’s get into it!

Challenges in preparing our data

Our tech support service for Google employees—Techstop—provides a complex service, offering support for a range of products and technology stacks through chat, email, and other channels. 

Techstop has a lot of data. We receive hundreds of thousands of requests for help per year. As Google has evolved we’ve used a single database for all internal support data, storing it as text, rather than as protocol buffers. Not so good for model training. To protect user privacy, we want to ensure no PII (personal identifiable information – e.g. usernames, real names, addresses, or phone numbers) makes it into the model.

To address these challenges we built a FlumeJava pipeline that takes our text and splits each message sent by agent and requester into individual lines, stored as repeated fields in a protocol buffer. As our pipe is executing this task, it also sends text to the Google Cloud DLP API, removing personal information from the session text, replacing it with a redaction that we can later use on our frontend. 

With the data prepared in the correct format, we are able to begin our model training. The model provides next message suggestions for techs based on the overall context of the conversation. To train the model we implemented tokenization, encoding, and dialogue attributes.

Splitting it up

The messages between the agent and customer are tokenized: broken up into discrete chunks for easier use. This splitting of text into tokens must be carefully considered for several reasons:

• Tokenization determines the size of the vocabulary needed to cover the text.
• Tokens should attempt to split along logical boundaries, aiming to extract the meaning of the text.
• Tradeoffs can be made between the size of each token, with smaller tokens increasing processing requirements but enabling easier correlation between different spans of text.

There are many ways to tokenize text (SAFT, splitting on white spaces, etc.), here we chose sentence piece tokenization, with each token referring to a word segment. 

Prediction with encoders

Training the neural network with tokenized values has gone through several iterations. The team used an Encoder-Decoder architecture that took a given vector along with a token and used a softmax function to predict the probability that the token was likely to be the next token in the sentence/conversation. Below, a diagram represents this method using LSTM-based recurrent networks. The power of this type of encoding comes from the ability of the encoder to effectively predict not just the next token, but the next series of tokens.

This has proven very useful for Smart Reply. In order to find the optimal sequence, an exponential search over each tree of possible future tokens is required. For this we opted to use beam search over a fixed-size list of best candidates, aiming to avoid increasing the overall memory use and run time for returning a list of suggestions. To do this we arranged tokens in a trie, and used a number of post processing techniques, as well as calculating a heuristic max score for a given candidate, to reduce the time it takes to iterate through the entire token list. While this improves the run time, the model tends to prefer shorter sequences. 

In order to help reduce latency and improve control we decided to move to an Encoder-Encoder architecture. Instead of predicting a single next token and decoding a sequence of following predictions with multiple calls to the model, it instead encodes a candidate sequence with the neural network.

In practice, the two vectors – the context encoding and the encoding of a single candidate output – are combined with dot product to arrive at a score for the given candidate. The goal of this network is to maximize the score for true candidates – e.g. candidates that did appear in the training set – and minimize false candidates.

Choosing how to sample negatives affects the model training greatly. Below are some strategies that can be employed:

• Using positive labels from other training examples in the batch.
• Drawing randomly from a set of common messages. This assumes that the empirical probability of each message is sampled correctly. 
• Using messages from context.
• Generating negatives from another model.

As this encoding generates a fixed list of candidates that can be precomputed and stored, each time a prediction is needed, only the context encoding needs to be computed, then multiplied by the matrix of candidate embeddings. This reduces both the time from the beam search method and the inherent bias towards shorter responses.

Dialogue Attributes

Conversations are more than simple text modeling. The overall flow of the conversation between participants provides important information, changing the attributes of each message. The context, such as who said what to whom and when, offers useful bits of input for the model when making a prediction. To that end the model uses the following attributes during its prediction:

• Local User ID’s – we set a finite number of participants for a given conversation to represent the turn taking between messages, assigning values to those participants. In most cases for support sessions there are 2 participants, requiring ID 0, and 1.
• Replies vs continuations – initially modeling focused only on replies. However, in practice conversations also include instances where participants are following up on the previously sent message. Given this, the model is trained for both same-user suggestions and “other” user suggestions.
• Timestamps  – gaps in conversation can indicate a number of different things. From a support perspective, gaps may indicate that the user has disconnected. The model takes this information and focuses on the time elapsed between messages, providing different predictions based on the values. 

Post processing

Suggestions can then be manipulated to get a more desirable final ranking. Such post-processing includes:

1. Preferring longer suggestions by adding a token factor, generated by multiplying the number of tokens in the current candidate.
2. Demoting suggestions with a high level of overlap with previously sent messages.
3. Promoting more diverse suggestions based on embedding distance similarities.

To help us tune and focus on the best responses the team created a priority list. This gives us the opportunity to influence the model’s output, ensuring that responses that are incorrect can be de-prioritized. Abstractly it can be thought of as a filter that can be calibrated to best suit the client’s needs. 

Getting suggestions to agents

With our model ready we now needed to get it in the hands of our techs. We wanted our solution to be as agnostic to our chat platform as possible, allowing us to be agile when facing tooling changes and speeding up our ability to deploy other efficiency features. To this end we wanted an API that we could query either via gRPC or via HTTPs. We designed a Google Cloud API, responsible for logging usage as well as acting as a bridge between our model and a Chrome Extension we would be using as a frontend.

The hidden step, measurement

Once we had our model, infrastructure, and extension in place we were left with the big question for any IT project. What was our impact? One of the great things about working in IT at Google is that it’s never dull. We have constant changes, be it planned or unplanned. However, this does complicate measuring the success of a deployment like this. Did we improve our service or was it just a quiet month?

In order to be satisfied with our results we conducted an A/B experiment, with some of our techs using our extension, and the others not. The groups were chosen at random with a distribution of techs across our global team, including a mix of techs with varying levels of experience ranging from 3 to 26 months. 

Our primary goal was to measure tech support efficiency when using the tool. We looked at two key metrics as proxies for tech efficiency: 

1. The overall length of the chat. 
2. The number of messages sent by the tech.

Evaluating our experiment

To evaluate our data we used a two-sample permutation test. We had a null hypothesis that techs using the extension would not have a lower time-to-resolution, or be able to send more messages, than those without the extension. The alternative hypothesis was that techs using the extension would be able to resolve sessions quicker or send more messages in approximately the same time.

We took the mid mean of our data, using pandas to trim outliers greater than 3 standard deviations away. As the distribution of our chat lengths is not normal, with significant right skew caused by a long tail of longer issues, we opted to measure the difference in means, relying on central limit theorem (CLT) to provide us with our significance values. Any result with a p-value between 1.0 and 9.0 would be rejected. 

Across the entire pool we saw a decrease in chat lengths of 36 seconds.

In reference to the number of chat messages we saw techs on average being able to send 5-6 messages more in less time. 

In short, we saw techs were able to send more messages in a shorter period of time. Our results also showed that these improvements increased with support agent tenure, and our more senior techs were able to save an average of ~4 minutes per support interaction.

Overall we were pleased with the results. While things weren’t perfect, it looked like we were onto a good thing.

So what’s next for us?

Like any ML project, the better the data the better the result. We’ll be spending time looking into how to provide canonical suggestions to our support agents by clustering results coming from our allow list. We also want to investigate ways of making improvements to the support articles provided by the model, as anything that helps our techs, particularly the junior ones, with discoverability will be a huge win for us.

How can you do this?

A successful applied AI project always starts with data. Begin by gathering the information you have, segmenting it up, and then starting to process it. The interaction data you feed in will determine the quality of the suggestions you get, so make sure you select for the patterns you want to reinforce.

Our Contact Center AI allows tokenization, encoding and reporting, without you needing to design or train your own model, or create your own measurements. It handles all the training for you, once your data is formatted properly. 

You’ll still need to determine how best to integrate its suggestions to your support system’s front-end. We also recommend doing statistical modeling to find out if the suggestions are making your support experience better. 

As we gave our technicians ready-made replies to chat interactions, we saved time for our support team. We hope you’ll try using these methods to help your support team scale.

Local governments are embracing more modern and scalable ways to support their communities. In an effort to save both time and money, Costa Mesa Sanitary District  (“CMSD”) used machine learning to automate and streamline manhole maintenance. Manhole maintenance is an essential part of the upkeep of cities. Manholes provide critical access points for underground public utilities, allowing inspection, maintenance, and system upgrades. But failure to upkeep manholes can cause a multitude of problems, from road hazards to sewer blockages, and can make it difficult for workers to access underground public utilities, which can lead to other safety issues. Manhole maintenance is an essential part of Costa Mesa Sanitary District maintenance, but this process requires the work of an outside consultant and costs the CMSD over $100,000.

ML to the rescue 

CMSD, in collaboration with SpringML and Google Cloud, developed a project to streamline manhole maintenance by leveraging the power of machine learning (ML) to detect sewer manholes and rate their conditions. This solution saves CMSD $40,000 every year, freeing up funds for other public service projects.

Every quarter, one member of the CMSD drives a car outfitted with a GoPro camera. This car travels through the entire District area, which includes the city of Costa Mesa and small portions of Newport Beach, CA, which is about 218 street miles, and records the roads to detect approximately 5,000 manholes. At the end of the recording day, CMSD members transfer images and videos from the GoPro SD card into a local server. Then these files are automatically ingested into Google Cloud Storage for processing. From here, the machine learning algorithms detect which manholes need repair.

Google Cloud products are used throughout this project. Once the images and videos are in Google Cloud Storage, a workflow with Cloud Scheduler spins up the VM every night to detect if there are new videos on Cloud Storage. If there are, this triggers cloud machine learning, which reviews the numerous images and videos, rates each manhole, and determines if any require maintenance. 

Machine learning to detect and grade manholes

SpringML applied a very systematic approach to detecting and grading the manholes. First, image processing ensures that the region of interest is only the section of the road in front of the vehicle to avoid any privacy concerns. Then SpringML applied a 5-step process using machine learning to detect and grade the manholes.

1. SpringML developed two separate custom TensorFlow-based Mask R-CNN models. Mask-R-CNN is a deep neural network that is used for image segmentation tasks, which means it can separate different objects in an image or a video. 
   
   The first Mask-R-CNN model was created to accurately detect if an image showed a manhole cover. This was a critical step because sewer manhole covers look similar to water main covers. To make the model accurate, it was important that there be no false positives. SpringML used around 50 images of sewer manholes and other images to train and validate that the algorithm could successfully detect sewer manhole covers. 
2. Once a manhole is accurately detected, the surrounding area is masked using image processing to focus on the manhole and then the cropped images are sent to the second model which is used to detect damage in the area around the manhole, called the apron. To reduce the processing time, the second model only does the inference if a manhole cover is detected.
3. The second Mask-R-CNN model uses CMSD Guidelines to decide what types of damage contributed to the rating system. For training purposes, SpringML uses a TensorFlow-Keras inside a Virtual Machine on Google Cloud. The initial model was trained and the Keras weights were saved on Google Cloud Storage. This helped create a versioned system of the models as the model gets refined over time.
4. Then, duplicated detections are cleaned up to have a unique detection for each manhole cover.
5. Finally, Manholes are graded from a 1-5 rating, with 5 representing high damage and 1 representing low damage. 

Once cloud machine learning has analyzed the new videos and images, the final scores are stored in BigQuery. Results are then served to members of CMSD via a simple web application where they can see which manholes need to be maintained. Two staff members review the ML results and determine priorities for repairs. One of the most interesting features of this project is that the model gets continually retrained based on feedback submitted in the web application. For example, if the model inaccurately detects a manhole, a member of CMSD can mark that in the web application and their feedback is immediately used to refine the model.

This solution shows how leveraging machine learning can streamline a necessary local government project and save money and labor while being highly scalable! Not only does this system streamline the manhole maintenance process, but it also allows for more frequent review of manhole conditions and provides a historical view of how the District’s manholes change over time.

Want to learn more about Google Cloud machine learning? Check out this tutorial to  learn TensorFlow and Keras and check out more machine learning tools in our AI Platform.

San José is one of the most diverse U.S. cities, with residents speaking more than 100 languages. Several years ago, we set out to improve city community interactions through more equitable service management and delivery. This demanded a new approach to automating the intake of requests from a majority population whose first language is not English.

We first created the 311 portal and app, which was an important step as we effectively separated resident service requests from emergencies. The way we describe it to our citizens, you call 311 for a burning question, and 911 for a burning building. In the last fiscal year, San José 311 received nearly 210,000 contacts by phone and an additional 211,000 service requests through the SJ 311 app. 

Through the portal and app, we gave citizens an omnichannel experience enabling them to interact with the city to request improvements, access useful information, and get emergency help when they need it. 

In order to truly serve our diverse communities, we recognized language translation services would be required to offer truly equitable services to everyone. That’s when we started working closely with SpringML, Google Cloud, and other partners with involvement from our Mayor and City CIO.

Building public services with community engagement in mind

When we first rolled out the My San José website and mobile app, we used an out-of-the-box translation service that ended up not working. It had poor accuracy and did not meet our needs to provide all citizens with coherent services. After looking at many other options, we decided to partner with SpringML and Google Cloud to leverage the AutoML Translation with other technologies such as our virtual agent.

SpringML was selected through an open RFP process, and helped us to build and optimize our integrations, interfaces, and more between several systems, making the app and website more intuitive to manage. SpringML delivered the product we needed on time and up to specifications, and additional value came from the training sessions they provided to our team. This enabled us to understand everything we could do with AutoML and opened the door to other enhancements such as simplifying the vernacular used with our residents, making government access easier to navigate regardless of natural language spoken. 

After establishing the My San José app’s translation capabilities using AutoML, SpringML also helped us incorporate Dialogflow virtual agents. Dialogflow also positions us to make modifications with our own staffing practices – something that has become increasingly important amid the frequent changes in service levels from COVID-19 response in the past year.

Responding to community needs

With the app up-and-running, our next step was to bring in community members to help with testing, improvements, and more. We wanted the app and the website to not just be something we provided to the community, but rather something they helped us build so they would readily adopt it.

Thanks to the greater accuracy of translation supported by Google Cloud services, we were able to leverage the expertise of a small pool of community members to evaluate translations. AutoML Translation and Glossary proved to be a powerful combination that pushed us closer to our goals. 

Our primary targets were Spanish and Vietnamese translations. We are now seeing 90 percent accuracy in automated Spanish translations while Vietnamese translations continue to improve. We continue to work to simplify the language used in these services, which makes a big difference in terms of ensuring optimal language accessibility.

This work includes best serving our community members who primarily use phones to get in touch with us through 311 services. Using Google Cloud Contact Center AI, we have been able to effectively manage the calls we receive 24×7 and communicate with residents who speak Spanish as well as English. No matter which channel one of our residents choose to use to reach out, we can serve them efficiently. 

A well-timed release

We’re proud of the work we’ve done. We’ve made many government services available to our community 24 hours a day, 7 days a week — accessible through many channels. Regardless of a person’s native language, the consistency of experiences enjoyed by everyone is improving every day thanks to AI. We’re also actively incorporating more language translation capabilities to better serve more people.

While we began this process several years ago, the recent integration of machine learning language translation with our customer relationship management system in late 2020 was very well-timed because we were able to incorporate this into our COVID-19 pandemic response. 

We’re also beginning to work with other municipalities across the U.S. to share some of the lessons we’ve learned and success we’ve seen in hopes of furthering more equitable citizen services far beyond our City limits. 

We are excited to continue working with SpringML, Google Cloud, and other partners to improve our city and the equity and quality of services that our residents enjoy.

Learn more about how you can work with a Google Cloud Partner here.

May is Asian Pacific American Heritage Month —a time for us to come together to celebrate and remember the important people and history of Asian and Pacific Island heritage. This feature highlights three AAPI founders from the Google For Startups community.

Read on to learn how these three founders built their businesses, leverage Google tech, and suggestion they have for aspiring entrepreneurs.

CultivatePeople 

Founder: Lola Han

Description: CultivatePeople’s compensation software, Kamsa, provides global market pay rate data and helps companies make data-driven salary decisions so they can attract and retain their most valuable asset: employees. 

Why GCP: “CultivatePeople began using GCP when integrating SSO/SAML authentication after  our SaaS product became a high priority. We were able to exceed our clients’ expectations and increase their level of trust in our platform’s capabilities. We’re currently investigating additional machine learning products, such as Cloud AutoML, that will allow us to quickly deliver exciting features.” 

Note from the Founder: “I grew up with immigrant parents who value stability and are risk-averse. My parents discouraged me from starting my own company because they didn’t want to see me struggle financially or see my health suffer (due to stress). I felt strongly about what CultivatePeople could do, so I started the company as a sole founder in 2017 and watched it double in size year over year since. While the ones I love most may not have cheered me on initially, it was important for me to hang on to the encouraging words of former bosses, executives, and founders to keep me focused on my mission. 

My advice for other AAPI founders is to be a “silent assassin” and believe in the mission and values of your organization. Always remember to stop along the way and: 

 1) Enjoy the journey by celebrating wins and giving yourself credit;

2) Follow your intuition—it’s (almost) always right; 

3) Recognize and invest in your people regularly (ie. give increases more than once a year, if warranted);

4) Give regular words of affirmation to employees on even small achievements.”

Swit

Founder: Josh Lee and Max Lim

Description: Swit is a team collaboration platform that seamlessly combines team chat with task management by allowing teams to turn their conversations into trackable tasks and share tasks to chat with simple drag-and-drop functionality, ensuring everyone is on the same page and projects get done faster.

Why GCP: “Swit is a cross-category hybrid work tool for chat and tasks. This functionality requires more complicated and heavier architecture for performance. So, configuring and managing virtual machines was really challenging to scale up our systems, while handling occasional unexpected traffic surges and frequent updates. Eventually we divided our monolithic architecture into 35 microservices when we launched our official product. The migration to GKE took around one month, and it turned out to be well worth the effort—our systems became able to offer high scalability and enough resilience to keep its uptime no matter what happened. Now we’re operating 84 workloads and 252 microservices with high stability with remarkably low downtime – less than 0.00001%/year.”

Note from the Founder: “As an AAPI founder based in Silicon Valley, I feel proud of the work ethics and diligence fellow Korean American entrepreneurs and professionals have long demonstrated here. Especially with K-pop breaking into the mainstream, I feel even more proud of our culture that strives toward an absolute perfection molded through years of training and dedication. The mission-driven culture of Silicon Valley coupled with Google’s edging technology and creativity really helped us build a product that not only encompasses verticals but also transcends cultures. Swit is growing at an unprecedented rate, and we hope  to join the long list of successful AAPI entrepreneurs here. Swit’s close network with the AAPI community wouldn’t have been possible without Google support. We are grateful for this collaborative environment, and we hope to become the next-generation ambassador for collaboration after Google.” 

Check out more from Swit in their founder story.

WISY 

Founder: Min Chen

Description: Wisy develops technology to bring digital efficiency into the physical world, supporting consumer products businesses and making them thrive in the new economy. All of us have a bad experience when we can’t find the product we want to buy. That is a $1.9T problem in the consumer-packaged goods industry that Wisy is solving with AI and analytics to help manufacturers and retailers sell more by reducing out-of-stocks and waste at a global scale.

Why GCP: “GCP has an intuitive, easy to use interface, was lower cost, and offered preemptible instances with flexible compute options. Some of the reasons why Wisy decided to use GCP include instance and payment configurability, privacy and traffic security, cost-efficiency, and Machine Learning.

Wisy has been able to advance quickly with product development, as well as collaborate better and iterate faster in the creation of our AI models, while reducing costs by 40%. At Wisy, we are solving a problem that affects everyone who shops at a store.“

Note from the Founder: “Two years ago, I moved to San Francisco to expand my second startup, Wisy. This is when I learned that my name ‘Min’ stands for ‘minority.’ I was born in China, raised in a Black community in Panama, received scholarships to attend both Carnegie Mellon and UC Berkeley. I worked for 20 years in several countries, but I have never felt so discriminated against due to my race, ethnicity, gender and age than during my time in Silicon Valley. However, this is also the place I learned that my diverse life experience is my competitive advantage. My background enables me to recruit and relate to people in different countries, create scalable and flexible products for multinational customers, and run global operations efficiently.

My recommendation to AAPI founders is to find strength in their multicultural background. Don’t hide what makes you unique, do not limit yourselves, and do not let others limit you. You will lose your edge when trading authenticity for validation. Be proud and own your story.”

If you want to learn more about how Google Cloud can help your startup, visit our Startup Program application page here and sign up for our monthly startup newsletter to get a peek at our community activities, digital events, special offers, and more.To learn more about how you can help #StopAsianHate during Asian Pacific American Heritage Month and beyond, visit their website here.