Over the decades, technology has helped us organize large amounts of physical information in ways that are streamlined, efficient, and easily accessible. Rows upon rows of encyclopedias are no longer needed; simply punch in or speak a query into Google Search and find numerous results at your fingertips. There’s no need to haul around cases of CDs or cassettes either, when you can access hundreds of thousands of songs on music streaming services. 

We wanted to bring that same level of accessibility to one specific type of publication: the printed car manual. Here’s how we shifted the paper manual to become an easy-to-access, voice-activated digital experience for owners of the all-new Sienna. It’s helped this resource become just as modern and useful as Toyotas themselves.

Putting cloud technology in the driver’s seat

Far too often, the printed car manual remains unused, collecting dust in the glove compartment—that is, until it’s needed during a roadside emergency or to solve the meaning of a mysterious light popping up on the dashboard. Even then, thumbing through hundreds of pages during high-stakes moments can be stressful. On top of that, these manuals can be expensive to print and update.

Digital versions, such as a PDF file, are a nice start. But, they’re often little more than reformatted flat documents. In poor visual conditions on the side of a road, the last thing a driver wants to do is squint at a phone or scroll through pages of tiny text. And similar to printed manuals, digital versions don’t facilitate ongoing, real-time conversations between drivers and automakers when it matters the most; they’re often only text and pictures, and at best, schematic drawings.

With that in mind, we set out to elevate and personalize the car manual by creating a voice-activated digital owner’s manual experience, all powered by Google Cloud. A voice-based assistant was the clear choice because, as it felt like the most intuitive, natural option, especially while driving. In fact, 51% of U.S. adults have used a voice assistant while driving, and 95% of all drivers expect to use a voice assistant in the next three years. 

We’re now putting this technology on the road by powering the Toyota Driver’s Companion for the all-new 2021 Toyota Sienna model. Accessible through the existing Toyota app, this companion provides real-time assistance, any time of the day. Drivers can ask questions and the companion will efficiently provide helpful answers in convenient ways, from voice to 3D walkthroughs and explorable environments. 

What a modern car manual should do

The Toyota Driver’s Companion has interactive features to help drivers discover the Sienna’s dashboard, set up the car’s interior and exterior appearance, better understand vehicle maintenance, and explore Dynamic Radar Cruise Control, Lane Departure Alert and other features. 

Here are a few other additional key features to call out within the Toyota Driver’s Companion: 

• An easily accessible virtual voice through the Toyota Driver’s Companion lets app users ask personal questions about their 2021 Sienna such as “what’s the height of my car?” and receive immediate answers either by voice, display or interactive input. 
• The manual automatically connects with the purchased vehicle’s VIN number to create a completely personalized experience, curated specifically for the driver. For example, if an unfamiliar light on the dashboard pops up, the Toyota Driver’s Companion can help identify the light’s meaning.
• Interactive hotspots throughout the vehicle’s interior let drivers explore the cabin virtually. Drivers can discover button functionalities, find specific dials, and learn more about car functions, such as how to slide seats or open doors, to become acclimated with their new vehicle.

To bring this experience to life, we tapped into some of our key Google Cloud solutions:

• APIs powered by Google Cloud artificial intelligence technology make accessing specific vehicle information easy and effortless, by leveraging Google’s natural language processing:
  • Google Cloud DialogFlow API serves as the decision tree that gives intelligence for both finding an answer for a question, i.e., how the Companion responds to the end user’s questions. 
  • One of our Text-to-Speech APIs—called Wavenet—creates the Companion’s realistic voice. 
  • And finally, our Speech-to-Text API “listens” to the user’s voice and finds the correct information to craft responses. That means a driver can ask a question multiple ways, and the Companion will still respond with the right answer. 

Our Firebase mobile app dev platform gleans analytic insights that help improve the overall experience and services for OEMs and drivers alike.https://www.youtube.com/embed/66QxWS-PzIM?enablejsapi=1&

We’re encouraging better consumer experiences by providing faster access to fresh information, in a natural, accessible format—voice. But these new voice-activated experiences aren’t only an opportunity to help out drivers; it’s also about strengthening connections between drivers, their vehicles, and automakers, too. 

Our hope is that through this information exchange, drivers can provide feedback on the most frequently misunderstood features, enabling OEMs to address questions early on. By understanding the most requested features, OEMs can also predict and inform driver questions about features. We’re incredibly excited to help make the driver’s experience more connected and helpful.

In our conversations with customers, few generative AI use cases have driven as much enthusiasm as generative search. Leaders at enterprises know the limits of traditional enterprise search, with queries producing a list of links based on pattern matching, and significant manual investigation required to find the more relevant answers. In generative AI, these leaders see an opportunity to leverage their data more effectively and deeply, including applying it to conversational apps that can answer complex questions, produce accurate summaries that synthesize many sources, and help people get the information they need, faster. 

Enterprise Search on Generative AI App Builder (Gen App Builder) lets organizations create custom chatbots and semantic search applications in as little as a few minutes, with minimal coding needed to get started and enterprise-grade management and security built in. Customers can combine their internal data with the power of Google’s search technologies and generative foundation models, delivering relevant, personalized search experiences for enterprise applications or consumer-facing websites. 

To date, most approaches to combining generative AI and search technology have been inadequate for the scale and reliability needed for enterprise use. For example, building search by breaking long documents into chunks and feeding each segment into an AI assistant typically isn’t scalable and doesn’t effectively provide insights across multiple sources. Likewise, many solutions are limited in the data types they can handle, prone to errors, and susceptible to data leakage. Bespoke, do-it-yourself approaches are generally no easier, with production-grade solutions often requiring complex tasks like integrating embeddings data with foundation models and significant use case data testing. Even when organizations make these efforts, the resulting solutions still tend to lack feature completeness and reliability, with significant investments of time and resources required to achieve high-quality results.

These challenges demonstrate that to effectively implement generative search, organizations typically need more than access to powerful foundation models. They also likely need the ability to ground model outputs in specific data, so that outputs are more relevant and less likely to include mistakes or “hallucinations.” They generally need safeguards that protect their data, how it is accessed, and how it is used. And they generally need the process to be high-performant and scalable out of the box, making the functionality easy to use even if the organization lacks data science and machine learning expertise.

Let’s look at how Enterprise Search on Gen App Builder helps customers bypass these scale and reliability challenges, so they can start leveraging generative search quickly. 

The intersection of generative AI and enterprise data 

Enterprise Search on Gen App Builder lets developers create search engines that help ground outputs in specific data sources for accuracy and relevance, can handle multimodal data such as images, and include controls over how answer summaries are generated. Multi-turn conversations are supported so that users can ask follow up questions as they peruse outputs, and customers have control over their data—including the ability to support HIPAA compliance for healthcare use cases. All of this is available as a fully managed service, so developers can focus on building rather than cloud complexity. 

Gen App Builder’s out-of-box capabilities can remove the need for data chunking, generating embeddings, or managing indexes, hiding complexity behind a straightforward interface that lets developers build apps in minutes with little or no coding and no prior machine learning experience. With the ability to ingest large volumes of documents and support for both unstructured and structured data, apps built with Gen App Builder help customers solve the long-standing headache of finding relevant information across the organization, turning tasks that used to take hours into quick searches or conversational explorations with an app. 

These capabilities are underpinned by both Google’s foundation models and a variety of Google Search technologies, including:

• Semantic search, which helps deliver more relevant results than traditional keyword-based search techniques by using natural language processing and machine learning techniques to infer relationships within the content and intent from the user’s query input. 
• Google’s understanding of how users search for information. 
• Google’s expertise in understanding relevance, which considers factors such as content popularity and user personalization of content when determining the order in which search results are displayed. 

For more advanced use cases, Gen App Builder can be easily integrated with Vertex AI for in-depth foundation model tuning, enabling flexible input and output search formats. 

As always with our AI products, Gen App Builder has been evaluated for alignment with our AI Principles, which is reflected by the product’s many safeguards against bias, toxic content, and unhelpful outputs. Whether for prototypes built in minutes or apps with many custom components, Enterprise Search on Gen App Builder offers a robust suite of user-friendly tools for customers across industries and levels of expertise. 

How customers are innovating with Enterprise Search 

After gaining access to Enterprise Search on Gen App Builder via our trusted tester program, a number of customers are already leveraging the product for novel use cases. 

Priceline is harnessing Gen App Builder and Vertex AI for a range of projects, including internal search engines for employees and a new chatbot to assist customers as they make travel plans. Slated to be available across both desktop and mobile experiences, Priceline’s chatbot will help customers find the right information faster via always-on, personalized experiences, including answering nuanced questions like, “What are the best 4-star hotel options in midtown Manhattan within walking distance to Central Park?” and “Can you help me extend my hotel reservation for an additional night?”

“Priceline is charting a course to transform the novelty of generative AI into lasting value for our customers and our business. We believe it’s not just about having the latest technology; it’s also about practically targeting innovation to the right challenges and opportunities,” said Marty Brodbeck, Chief Technology Officer, Priceline. “With Google Cloud as our AI innovation partner, we’re doubling down on our commitment to delivering the fastest, most seamless and informative booking experience for our customers, from personalized planning and travel inspiration to customer service.”

Vodafone is experimenting with Enterprise Search and foundation models on Gen App Builder to build a tool that can rapidly and securely query documents, search, and understand specific commercial terms and conditions. Vodafone Voice and Roaming Services has over 10,000 contracts with other telecommunications companies worldwide, in a variety of formats such as PDFs, images, and complex tables. Searching this document repository is often a time consuming process for employees. 

“Every day we introduce and manage new services like 5G to a roaming footprint comprising more than 700 operators in 210 countries to ensure both Vodafone and our partners’ business customers, holidaymakers, and Internet of Things devices stay connected when abroad. With Enterprise Search on Gen App Builder, we are building an intelligent assistant to securely and quickly search contracts.” said Sherif Bakir, CEO of Vodafone Voice and Roaming Services. “With generative AI, we’re accelerating otherwise protracted processes, increasing productivity and operational efficiency.”

Software startup Trender.ai, which creates customer monitoring and intelligence solutions for B2B relationships, is using Gen App Builder to build a product that can synthesize information from social media, public sources, and CRM data so that users can form more productive, personal relationships with prospects and customers.

“With Enterprise Search on Gen App Builder, we have been able to do things in the last month that we had projected to take 12-18 months on our prior roadmap,” said Betsy Bilhorn, co-founder at Trender.ai. “We had expected it would take at least 12 months to build and train models from an individual’s public social, web, and other data, and to then be able to ask questions like ‘What things are most important to this person?’ or ‘When and how is the best way to make an introduction to this prospect that they’ll respond to?’ Last year, we thought getting to this vision was a bit of a moonshot for a startup our size — but now we were able to achieve this in under a month.” 

Stop searching for solutions — start discovering insights

We’re excited to see how customers use Enterprise Search on Gen App Builder to leverage data in powerful ways, discover new insights, and create useful, personal, and efficient experiences. 

Today, we’re pleased to help our customers accomplish these goals, with the general availability of Enterprise Search on Gen App Builder for customers on the allowlist (i.e., approved for access). Please contact your Google Cloud sales team for access and pricing details. We are also launching two new features within Enterprise Search on Gen App Builder that are available today in our preview offering: multi-turn search, which supports asking follow-up questions, and content recommendations to find semantically relevant content. Access to the preview offering is available via Google Cloud’s trusted tester program. Read more about Enterprise Search on Gen App Builder and sign up for access on our webpage. 

We’re also bringing generative AI features of Enterprise Search to our existing solutions like Contact Center AI and Document AI. As an example, starting this month, customers can preview generative AI-powered search in Document AI Warehouse. To keep up with our latest generative AI news, don’t miss The Prompt or our generative AI primer for executives on Transform with Google Cloud.

Google Cloud is proud to support our customers with the cleanest cloud in the industry. For the past four years, we’ve matched 100% of our electricity use with renewable energy purchases, and we were the first company of our size to commit going even further by running on carbon-free energy 24/7 by 2030.  As we work to achieve 24/7 carbon-free energy, we help you take immediate action to decarbonize your digital applications and infrastructure. We’re also working with our customers across every industry to develop new solutions for the unique climate change challenges that organizations face. Today, we’re excited to expand our portfolio of carbon-free solutions and announce new partnerships that will help every company build a more sustainable future. 

First, we’re launching Carbon Footprint, a new product that provides customers with the gross carbon emissions associated with their Google Cloud Platform usage. Now available to every GCP user for free in the Cloud Console, this tool helps you measure, track and report on the gross carbon emissions associated with the electricity of your cloud usage. Of course, the net operational emissions associated with your Google Cloud usage is still zero. With growing requirements for Environmental Social and Governance (ESG) reporting, companies are looking for ways to show their employees, boards and customers their progress against climate targets. Using Carbon Footprint, you have access to the gross energy related emissions data you need for internal carbon inventories and external carbon disclosures, with one click. 

Built in collaboration with customers like Atos, Etsy, HSBC, L’Oréal, Salesforce, Thoughtworks and Twitter, our Carbon Footprint reporting introduces a new standard of transparency to support you in meeting your climate goals. You can monitor your gross cloud emissions over time, by project, by product and by region, giving IT teams and developers metrics that can help them reduce their carbon footprint. Our detailed calculation methodology is published so that auditors and reporting teams can verify that their cloud emissions data meets GHG Protocol guidance.

“The power of knowledge combined with the power of technology innovation plays a vital role in proactively responding to the climate crisis we are facing. With Google Carbon Footprint reporting, Atos feeds emissions data in our Decarbonization Data Platform, demonstrating potential emissions reductions from the Google Cloud Platform to our customers. This reporting opens up new levels of emissions transparency, trajectory planning, and data insight to support our customers in meeting, and potentially accelerating towards, their climate goals.”—Nourdine Bihmane, Head of Decarbonization Business Line, Atos

“The capability to measure and understand the environmental footprint of our Public Cloud usage is among the key axis of our sustainable tech roadmap. With Google Cloud Carbon Footprint, we are now able to directly follow the impact of our sustainable infrastructure approach and architecture principles.”—Hervé DUMAS, Sustainability IT Director, L’Oreal

While digital infrastructure emissions are just one part of your environmental footprint, accurately accounting for IT carbon emissions is necessary to measure progress against the carbon reduction targets required to avert the worst consequences of climate change. To help you account for emissions beyond our cloud and across your organization, we’re excited to partner with Salesforce Sustainability Cloud, integrating our Google Cloud Platform emissions data into their carbon accounting platform. 

“As we face unprecedented climate challenges, companies across the globe need to embed sustainability into the core of their business in order to meet growing customer and stakeholder expectations, and reduce their environmental impact. Together, Google Cloud and Salesforce Sustainability Cloud can help our joint customers accelerate their path to Net Zero, leveraging data-driven insights and visualizations to track and reduce their carbon emissions to drive sustainable change.”—Ari Alexander, GM of Salesforce Sustainability Cloud. 

From information to action

With the gross energy-related emissions footprint of data associated with your Google Cloud usage now available, we’re committed to providing tools to not only measure your carbon footprint, but help you reduce it. We recently launched low-carbon region icons to help you choose cleaner regions to locate your Google Cloud resources. New users who see the icons are over 50% more likely to choose clean regions over others, ensuring their applications emit less carbon over time. 

For current Google Cloud users, we’re pleased to announce that Active Assist Recommender will include a new sustainability impact category, extending its original core pillars of cost, performance, security, and manageability. Starting with the Unattended Project Recommender, you’ll soon be able to estimate the gross carbon emissions you’ll save by removing your idle resources. Unattended Project Recommender uses machine learning to identify, with a high degree of confidence, projects that are likely abandoned based on API and networking activity, billing, usage of cloud services, and other signals, and provides actionable recommendations on how to remediate those abandoned projects. By deleting these projects, not only can you reduce costs and mitigate security risks, but you can also reduce your carbon emissions. In August, Active Assist analyzed the aggregate data from all customers across our platform, and over 600,000 gross kgCo2e was associated with projects that it recommended for cleanup or reclamation. If customers deleted these projects they would significantly reduce future gross carbon emissions. Check out this blog to learn more about Active Assist.

Solutions for climate resilience 

Many of our customers face difficult questions about how their business impacts the natural environment today, and how it will be affected by climate change in the future. Answering these questions requires rich datasets about the planet, better analytics tools and smarter models to predict potential outcomes. For over a decade Google Earth Engine has supported scientists and developers with hyperscale computing power and the world’s largest catalog of satellite image data. Today, we are delighted to announce the preview of Earth Engine as part of Google Cloud Platform. Now, you can access Earth Engine and combine it with other geospatial-enabled products like BigQuery. By extending Earth Engine’s powerful platform to enterprises through Google Cloud, we are bringing the best of Google together. 

Over the past year we’ve worked with a number of organizations to use Earth Engine technology with tools like BigQuery and the Cloud AI Platform to develop new solutions for responsible commodity sourcing, sustainable land management and carbon emissions reduction. Earth Engine enables companies to track, monitor and predict changes in the Earth’s surface due to extreme weather events or human-caused activities, thus  helping them save on operational costs, mitigate and better manage risks, and become more resilient to climate change threats. This new offering will wrap the unique data, insights and functionality of Earth Engine with a fully-managed, enterprise-grade experience and reliability.

As we work with our customers to accelerate their sustainability initiatives, earth observation data is proving critical to effectively plan for the long-term impacts of climate change. To extend our geospatial and sustainability use cases we’re also expanding our partnerships with CARTO, Climate Engine, Geotab, NGIS, and Planet to bring their data and core applications to Google Cloud.

These partners will each make their existing platforms and datasets available globally on Google Cloud, giving you low-latency and reliable access to critical data and applications that will inform your sustainability initiatives. By integrating water availability, agricultural data, weather risks, and extensive daily satellite imagery into Earth Engine and BigQuery, you can achieve more ambitious goals for the sustainability of your business and our planet.

Committing to help you meet your climate goals

With each of these tools, we’re working to reduce the barriers you face in adopting more sustainable technology practices. We understand that building more sustainable applications and infrastructure is not easy. You face competing priorities, technical challenges, and the perception that climate action is costly. 

It doesn’t have to be this way. Today, we are making a sustainability pledge to you: teams across Google Cloud are committing to eliminating the barriers you face in building a more sustainable digital future for your organization, and will help you take action today to realize your climate goals. We’ll do this in a number of ways: 

1. In digital transformation projects and workshops, sustainability teams will always have a seat at the planning table, so we can work together on using cloud technology to build a more sustainable future. 
2. We’re putting low-carbon signals natively into our products to help developers choose more sustainable options early in their application development. 
3. We’ll ensure carbon impact is measured consistently with other key performance indicators. Leveraging the social cost of carbon, the ROI models and value assessments you conduct with Google Cloud will project your emissions impact too. 
4. We’ll be transparent about our carbon impact, by publishing third-party reviewed reports and methodologies, so you can trust the data for your own reports and disclosures. 
5. We’ll continue to work with the industry on best practices, including educational resources like Sustainable IT – Decoded, a new masterclass created in partnership with Intel, that shares the expertise of sustainability thought leaders. 

For the next decade we need to work together to avert the worst consequences of climate change. We’ve made tremendous progress in building technology that helps everyone do more for the planet, and we’re excited to see what you do with it. Visit this page to learn more about Google Cloud’s sustainability efforts.