As organizations continue to produce vast quantities of data, they increasingly need platforms that allow them to analyze, store, and extract meaningful insights from that data. And research from analyst firms like Gartner offer an important way for organizations to evaluate and compare cloud data warehouse providers.

Earlier this year, Gartner named Google a Leader in the 2019 Gartner Magic Quadrant for Data Management Solutions for Analytics (DMSA) (report available here). This evaluation covers Google Cloud’s core data analytics offerings, including BigQuery, a serverless, managed data warehouse, Cloud Dataproc, a managed Spark and Hadoop service, and Cloud Dataflow, which enables you to stream and batch-process your data. Here are a few takeaways:

Simplicity and speed

BigQuery’s performance permits complex queries on large-scale data sets to return in seconds, and a substantial number of BigQuery customers maintain data warehouses that store more than 50 terabytes (and a few customers now use more than 100 petabytes). More than half of these customers are loading data either continuously or many times per day. These customers value the ability to extract, transform, load, and analyze their data on a serverless platform, all without maintaining any underlying infrastructure.

A versatile serverless data warehouse

One of BigQuery’s major advantages is its ability to allow customers to address a wide variety of use cases—from a traditional data warehouse to data science. Over the past year, we’ve worked hard to introduce new features in BigQuery like data types for financial and monetary uses, BigQuery GIS for geospatial data, and machine learning capabilities through BigQuery ML. BigQuery’s continuous ingest capabilities make it suitable for an operational—or a real-time—serverless data warehouse.

An expanding ecosystem

With overall increased market adoption, our analytics offerings continue to benefit from a fast-growing partner ecosystem of service providers, and business intelligence (BI) and data integration vendors. In particular, in 2018 we expanded our partnerships with established industry providers, including Confluent, Dell Boomi, Informatica, Looker, Reltio, Tableau, and ThoughtSpot.

More and more organizations are finding value in Google Cloud’s serverless data warehouse and analytics offerings. If you’d like to learn more, you can download a complimentary copy of the Gartner Magic Quadrant for Data Management Solutions for Analytics on the Google site (requires an email address).

_{Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.}

When Google announced our plan to go beyond purchasing renewable power for 100% of our energy usage and operate on 24/7 carbon-free energy by 2030, we noted that achieving this goal will require new transaction structures, advancements in clean energy policy, and innovative new technologies. Today, we’re pleased to announce that one of these new technologies—a first-of-its-kind, next-generation geothermal project—will soon begin adding carbon-free energy to the electric grid that serves our data centers and infrastructure throughout Nevada, including our Cloud region in Las Vegas.  

Google and clean-energy startup Fervo have just signed the world’s first corporate agreement to develop a next-generation geothermal power project, which will provide an “always-on” carbon-free resource that can reduce our hourly reliance on fossil fuels. In 2022, Fervo will begin adding “firm” geothermal energy to the state’s electric grid system, where Google’s commitments already include one of the world’s largest corporate solar-plus-storage power purchase agreements. 

Importantly, this collaboration also sets the stage for next-generation geothermal to play a role as a firm and flexible carbon-free energy source that can increasingly replace carbon-emitting fossil fuels—especially when aided by policies that expand and improve electricity markets; incentivize deployment of innovative technologies; and increase investments in clean energy research, development, and demonstration (RD&D). 

Next-generation geothermal technology

Traditional geothermal already provides carbon-free baseload energy to a number of power grids. But because of cost and location constraints, it accounts for a very small percentage of global clean energy production. 

That’s one reason this new approach is so exciting; by using advanced drilling, fiber-optic sensing, and analytics techniques, next-generation geothermal can unlock an entirely new class of resource. And the US Department of Energy has found that with advancements in policy, technology, and procurement, geothermal energy could provide up to 120 GW of firm, flexible generation capacity in the US by 2050. 

As part of our agreement, Google is partnering with Fervo to develop AI and machine learning that could boost the productivity of next-generation geothermal and make it more effective at responding to demand, while also filling in the gaps left by variable renewable energy sources. Although this project is still in the early stages, it shows promise.

Using fiber-optic cables inside wells, Fervo can gather real-time data on flow, temperature, and performance of the geothermal resource. This data allows Fervo to identify precisely where the best resources exist, making it possible to control flow at various depths. Coupled with the AI and machine learning development outlined above, these capabilities can increase productivity and unlock flexible geothermal power in a range of new places. 

This won’t be the first time that Google is applying software solutions to clean energy applications: we’ve just announced an update to our carbon-intelligent computing program that helps us reduce emissions associated with running applications at Google data centers. And other forms of AI and machine learning are currently being used to increase the value of wind energy. 

Already this year, Google has taken significant strides toward sourcing 24/7 carbon-free energy for all our data centers, office campuses, and Cloud regions. On Earth Day, our CEO Sundar Pichai announced that for the first time, five of our global data center sites operated near or at 90% carbon-free energy in 2020. 

Not only does this Fervo project bring our data centers in Nevada closer to round-the-clock clean energy, but it also acts as a proof-of-concept to show how firm clean energy sources such as next-generation geothermal could eventually help replace carbon-emitting power sources around the world.

Generative AI continues to develop at a blistering pace, making it more important than ever that organizations have access to enterprise-ready capabilities to help them leverage this disruptive technology. 

Harnessing the power of decades of Google’s research, innovation, and investment in AI, Google Cloud continues to make generative AI available with baked-in security, data governance, and scalability across the board. 

To this end, last month, we announced the general availability of Generative AI support on Vertex AI, giving our customers the ability to access powerful foundation models from Google Research and tools for customizing and applying them. 

Today we are announcing the general availability (GA) of four important foundation models for Vertex AI. These include Imagen, PaLM 2 for Chat, Codey, and Chirp. For each of these models, organizations can access APIs on Model Garden and do prompt design and tuning on Generative AI Studio.

• Imagen includes four key features:
  • Image generation for creating studio-grade images at scale
  • Image editing to edit generated or existing images via text prompts 
  • Image captioning for creating captions of images at scale
  • Visual Question & Answering (VQA) for interacting with, analyzing, and explaining images
• PaLM 2 for Chat follows the general availability of PaLM 2 for Text in June 
• Codey supports code generation, completion, and code chat
• Chirp supports multilingual Speech AI 

We’re also announcing Multimodal Embeddings API in preview, which lets customers combine the power of Vertex AI’s generative AI models with their proprietary data, to generate embeddings, or interchangeable vector representations, of their text and image data. These capabilities can enable data science teams to deliver a variety of downstream tasks such as image classification, content recommendations, and visual search. 

In this blog post, we’ll explore what your organization can do with these powerful models and how Vertex AI provides the enterprise-ready capabilities you can use to get up and running with generative AI. 

Helping to drive enterprise value from Generative AI models

Powerful models are the foundation of generative AI, but the software, tools, and infrastructure that surround these models are equally important for enterprise adoption. Organizations face challenges not only accessing these models, but also integrating AI while maintaining protection over intellectual property, adhering to regulations around data security and privacy, and ensuring models and applications are safe to use. Many organizations also want to use generative AI without incurring large costs or managing huge clusters.

We help address these challenges head-on with Vertex AI’s platform capabilities for scalable application integration, purpose-built AI infrastructure, secure and private data customization, and responsible use of this technology. 

Let’s see how each of these pillars can help your organization. 

Access models to build production-ready generative applications 
Vertex AI can make it easy to access foundation models, as today’s model announcements attest. While models are an inextricable part of generative AI, the software that helps enterprises use this technology is equally important—which is why Vertex AI also offers a range of tools for tuning, deploying, monitoring, and maintaining models, so you can build differentiated applications using your own data. 

Turning to today’s announcements, in May we announced Imagen, our foundation model for image generation. Now, we are excited to announce Imagen is generally available with an allowlist (i.e., approved access via your sales representative), letting onboarded customers start using image generation and editing capabilities. Visual Q&A and Captioning for production workloads are also generally available for all customers. Visual Q&A provides new ways to engage with image-based data like retail products or image libraries. This new capability can give you answers to questions about an image, helping you analyze large amounts of data quickly, and it can even help the visually impaired understand images or graphs that they wouldn’t be able to otherwise. Captioning, meanwhile, can make it easy to generate relevant descriptions for your images. Captions can help with indexing and searching, as well as assigning image descriptions to product listings on eCommerce websites. 

“Imagen is beginning to power key capabilities within Omni, Omnicom’s open operating system, that will enable 17,000+ trained and certified users to create audience-driven customized images in minutes. Imagen has been instrumental in offering a scalable platform for image generation and customization. Integrating it into our platform allows us to expand the scope of audience-powered creative inspiration, at a scale that wasn’t previously possible,” said Art Schram, Annalect Chief Product Officer at Omnicom. “We’re starting to adopt the latest features like styles and fine tuning, and engineering data-driven prompts. We look forward to continuing to provide our users relevant visual inspiration in a responsible way.”

“The latest improvements in Imagen’s product preservation capabilities are a perfect match for Typeface’s focus on personalized AI for brands,” explained Vishal Sood, Head of Product at Typeface. “By combining Google Vertex AI’s Imagen with Typeface’s brand-personalized AI, we are able to help enterprises to create 10x personalized content in a fraction of time.”

Google Shopping recently built an application called Product Studio using Imagen on Vertex AI. Product Studio can enable merchants to create rich product images quickly and easily, at a fraction of the time it takes to do professional product photo shoots. “We’re excited about the feedback we’re getting from merchants in our early pilots, who say that Product Studio, which leverages Imagen on Vertex AI, helps them generate and publish lifestyle product photos directly to their product catalogs,” says Jeff Harrell, Google’s Senior Director of Product Management for Merchant Shopping. 

Announced in May, PaLM 2 is a family of models that power dozens of Google products, including Bard and Duet AI in Google Cloud. With the PaLM 2 for Chat model, now generally available, you can leverage Google’s PaLM’s variety of abilities for multi-turn chat applications, such as shopping assistants, customer support agents, and more. 

ThoughtSpot, provider of a widely-adopted business intelligence platform, is using PaLM 2 to build a new feature in ThoughtSpot for Google Sheets called “AI Explain,” which can instantly generate explanations of charts, visuals, and anomalies, and will launch new conversational AI and ML-enabled predictive forecasting capabilities into its analytics platform.

With Codey, your organization’s developers can accelerate a wide variety of coding tasks, helping to empower them to work efficiently and close skills gaps. The model enables not only code completion and code generation capabilities, but also chat to help with debugging, documentation, learning new concepts, and more. Since launching in preview in May, we’ve added additional programming languages including Go, Google Standard SQL, Java, Javascript, Python, and Typescript. We’ve also improved the quality of code responses and increased serving capacity, enabling your developers with the right tools to enter the era of generative engineering.  

“Security and privacy are key to incorporating AI into the software development lifecycle,” said David DeSanto, Chief Product Officer at GitLab. “GitLab leverages Vertex AI to deliver new, AI-powered features with a privacy-first approach, including the ability to run our own models and leverage Codey foundation models built on top of PaLM 2. The GitLab DevSecOps platform empowers organizations to harness the benefits of AI for faster software delivery, while ensuring their data, intellectual property, and source code are protected.”

Originally released in May in preview, Chirp is a version of our 2 billion-parameter speech model, which was trained on millions of hours of audio and supports over 100 languages. Chirp achieves 98% accuracy on English and relative improvement of up to 300% in languages with less than 10 million speakers. Whether the use case involves customer support, transcriptions, or voice control, Chirp can help your organization communicate with customers and constituents inclusively, by engaging audiences in their native languages. 

Last but not least, our Multimodal Embeddings API, now in preview, can unlock an array of new applications, such as image and text-based recommendations, by enabling the processing of text and images interchangeably. This capability complements our Text Embeddings API, which became generally available in June, and remains a recommended choice for those with fully text-based use cases. Multimodal Embeddings API makes it possible to categorize images and text together and can be crucial for use cases like retail recommendation systems that can provide relevant outputs from both images of products and text descriptions.

Match generative AI with infrastructure 
Beyond access to models and tools for building generative AI apps, you need infrastructure to make sure your apps can scale and reliably perform — ideally without running into daunting compute costs or management overhead that distracts your technical talent from building innovative products. Google Cloud offers the choice and power to run smaller models running finite tasks at the lowest latency levels, as well as to run large models capable of cutting-edge experiments. 

As our large language model customers are looking to scale up their projects and applications using our models, they often need assurances that their requests will be serviced with acceptable performance. This is especially critical for delivering real-time applications where customer service is paramount. Starting in August, Vertex AI will support provisioned, dedicated generative AI capacity that can deliver guaranteed throughput. This feature can be especially beneficial to customers who have a high volume of sustained workloads.  

Leverage generative AI while protecting data and privacy 
One capability enabled by Google Cloud is the ability to customize models using your own data. Vertex AI can help customers keep their data protected, secure, and private. When a company tunes a foundation model in Vertex AI, private data, model outputs, and prompts can be kept private, and they are never used in the foundation model training corpus. We recently published a whitepaper, “Adaptation of Large Foundation Models,” which outlines how we help protect customer data. 

Auditability and compliance are essential to helping ensure the security and privacy of customer data. We also engage in comprehensive GDPR privacy efforts, including our transparency commitments for customer data usage and the support for our customer’s Data Protection Impact Assessments (DPIAs). Now, we’re excited to support HIPAA compliance for many of our generally available models on Vertex AI, so that healthcare and life science customers with whom we have a Business Associate Agreement can run workloads with Protected Health Information (PHI) data on Google Cloud. 

Innovate responsibly 
Our AI Principles put beneficial use, user safety, and avoidance of harms above business outcomes and are embedded in how we develop our AI products. We’ve conducted extensive reviews on our generative AI products to identify potential risks and have developed guardrails to mitigate these impacts. For example, to address concerns around safety, we’ve implemented safety filters for bias, toxicity, and other harmful content. We also equip our customers with the tools they need to help reduce risk within their applications and provide recommendations to help navigate responsible AI. 

Bring the power of generative AI to your organization

With both a wide selection of foundation models and extensive, enterprise-grade platform capabilities, Vertex AI continues to unlock ways for your business or organization to access foundation models, tune them on your proprietary data, and leverage them for differentiated apps and digital experiences. To take the next step, visit our product page or reach out to our sales representatives to gain access to our latest capabilities.

As one of the most innate and ubiquitous forms of expression, speech is a fundamental pillar of human interaction. It comes as no surprise, then, that Google Cloud’s Speech API  has become a crucial tool for enterprise customers, launched to general availability (GA) over six years ago and, now, processing over 1 billion voice minutes each month.

With the Speech API, we’ve been pleased to serve thousands of customers and provide industry-leading speech recognition quality and cost-effective products across a range of industries. We want to constantly evolve our offerings and bring new benefits to organizations, which is why today, we’re excited to announce the GA release of our new Speech-to-Text v2 API. 

Speech-to-Text v2 modernizes our API interface and introduces several new features. It also migrates all of our existing functionality, so you can use the same models and features that you were using in STT v1 or v1p1beta1 APIs. This new version of our API also allows us to take advantage of significant cost savings in our serving path, and as such we are reducing our base price, as well as adding pricing incentives for large workloads and those willing to accept longer turnaround times.

This new infrastructure also allows us to serve a wide variety of new types of models, including Chirp , our latest 2B-parameter large speech model. All of these are Generally Available to Google Cloud Platform customers and users starting today.

Let’s have a more thorough look though at the enhanced features of Speech-to-Text API V2 and illustrate how your business can benefit from our new capabilities:

Expanding Speech-to-Text features with V2 API

Since the official launch of Speech-to-Text API back in 2017, we’ve utilized Google’s global infrastructure to host and monitor our production-facing transcription models. This robust, well-connected network has been the backbone of our offering for all of our customers. 

However, a unified view of our Speech-to-Text service has been a crucial request for our enterprise customers  who need to satisfy data residency and compliance requirements, especially in regulated industries like banking and public sector. We listened carefully to this feedback, and starting today, our Speech-to-Text v2 API supports full regionalization, allowing our customers to invoke identical copies of all our transcription models in the Google Cloud Platform region of their choice. 

In addition to giving users the flexibility to deploy in any region, we are adding a number of new features to help developers build on the API: 

• Recognizers: A user-defined named configuration that combines a model identifier, the language-locale of the audio to be transcribed, and the cloud region for the transcription model to run. Once-created, the recognizer can be referenced to every subsequent transcription request, eliminating the need for users to repeatedly define the same configuration parameters. This resourceful implementation of recognizers allows for greater flexibility in authentication and authorization, as users are not longer required to set up dedicated service accounts.
• Cloud Logging: Requests performed using a recognizer object automatically support cloud logging by default. Since the recognizers are defined as named entities, customers can partition traffic based on the recognizer of interest or collectively. 
• Audio Format Auto-Detection: Instead of having our users analyze and manually define the audio configuration settings to pass in a transcription request, the new Speech-to-Text V2 API detects settings like encoding, sampling rate, and channel count, then automatically populates the request configuration parameters. 

Increasing accuracy at Enterprise Scale with Chirp

As part of our continuous investment in foundational speech models, in March 2023 we released research results for our Universal Speech Mode (USM), a family of state-of-the-art speech models with 2B parameters and support for transcriptions of 300+ languages. In May 2023, at Google I/O, we announced Chirp in Private Preview, the latest version of the USM family, fine-tuned for our Cloud-specific use-cases. 

Chirp is now GA through the Speech-to-Text v2 API. Following extensive testing and feedback from our customers, we are making the power of pre-trained large models accessible through a simple enterprise-grade API surface. Our early adopters have seen major strides in customer engagement, thanks to the market-leading accuracy and language coverage of the new model, and we cannot wait to see what opportunities our enterprise customers will unlock.

Introducing new pricing, tiers, and options

We have heard from customers that price can be just as important as quality for many workloads. That’s why the Speech-to-Text API v2 features totally new pricing. We have lowered the cost of real-time and batch transcription from $0.024 per minute to $0.016 per minute. Additionally, we know that pricing can be a concern for those that have very large transcription workloads. For that reason, we are also introducing standard volume tiers, allowing costs as low as $0.004 per minute. As always, additional discounts are available to those with even larger workloads.

Staying true to our commitment on flexibility and choice in pricing, we noticed that even though the majority of our customers are interested in our models for real-time transcription scenarios, many are also interested in non-real time transcriptions for data at rest. With our new API v2 infrastructure, we can take greater advantage of capacity that goes unused at certain times. With our new Dynamic Batch pricing, we are passing this savings on to customers that are less latency sensitive. Aimed at users who can wait up to 24 hours for transcription results, Dynamic Batch is a new discounted pricing tier that offers 75% lower price per minute for transcription audio relative to our Standard tier.

For more information on Dynamic Batch and all our new STT v2 API pricing, check out our pricing page.

Learn more and start you Speech-to-Text journey

For more information to help get you started on your migration journey from V1 to V2, head over to the detailed documentation and try our walkthroughs to make the most of regionalization and recognizers.

If you are curious to learn more about how to use Chirp, you can get  started by following our tutorial. You can also read more about how we built it in our research blog post.

We are very excited to bring all these improvements to our developer ecosystem and provide you with the tools to leverage the power of voice for your businesses, programs, and applications.