While some traditional financial services companies have more slowly transitioned to the cloud, capital markets firms have embraced cloud computing across their entire value chains — front-, middle-, and back-office. We wanted to understand the dynamics behind this rapid adoption, the most common use cases, and the types of technology most in use, particularly as it relates to market data. Google Cloud commissioned Coalition Greenwich to survey 102 institutional capital markets professionals — at exchanges, trading systems, data aggregators, data producers, asset managers, hedge funds, and investment banks — in the United States, Canada, France, Germany, Italy, the Netherlands, Switzerland, and the United Kingdom. 

Our research found that while there are many drivers, demand for easier accessibility is fueling widespread adoption of cloud-based market data services, and associated trading infrastructures, across the buy side and sell side. In fact, 68% of sell-side and buy-side users find it critical for market data providers to offer public cloud-based data services. At the same time, exchanges, market data providers, aggregators, and trading systems are embracing the cloud as a delivery model by offering access to data directly via their own cloud services, APIs or partners.

Here were five noteworthy takeaways from the study: 

1. Cloud services are becoming ubiquitous for data delivery. Today, the cloud is pervasive, with 93% of exchanges, trading systems and data providers offering cloud-based data and services, according to surveyed executives. Moreover, 100% of those surveyed intend to offer new cloud-based services, such as derived data, in the next 12 months.

2. Commercial and investment banks are offering additional connectivity, real-time data feeds, and trading applications delivered via the cloud,demonstrating that it’s not only exchanges, trading systems, and data providers that are moving rapidly to the cloud. Internal use cases abound as well, with 67% of those surveyed consuming cloud-deployed market data, primarily for data analytics. 88% of surveyed sell-side firms intend to consume cloud-based market data services, with digital transformation, data science and quant research as the top use cases.

3. Buy side firms will consume even more cloud-deployed data. Today, 90% of surveyed buy-side firms are consuming cloud-deployed market data, mostly for portfolio management. 70% of buy-side firms intend to consume more public cloud-based market data services in the next 12 months, adding services such as compliance and regulatory reporting.

4. AI/ML, powered by cloud, is moving out of the pilot phase and into mainstream use. Today, 50% of exchanges, trading systems, and data providers are offering data products or services powered by AI/ML, and of those, 42% intend to offer AI-powered trade execution and trading analytics services in the next 12 months. Within commercial and investment banks, 55% said they are currently using AI/ML in the cloud, and while that was true for only 14% of overall buy-side respondents, 44% of large buy-side respondents are using it.

5. Exchanges, trading systems, and data providers are prioritizing public cloud for internal insights. 71% of these firms are using the public cloud, mostly for data transmission, processing, analysis, and long-term data storage. Over the next 12 months, 33% of new public cloud workloads will focus on data mining, data insights and advanced analytics, while 28% of new AI/ML tooling and infrastructure investments will focus on faster analytics and risk reviews, and 27% on data quality maintenance.

“We see new, dramatic shifts on the adoption of cloud across market data,” said David Easthope, Senior Analyst for Coalition Greenwich. “And we expect further proliferation of cloud-based services and greater consumption across the trading and investing lifecycle.”

Conclusions and future predictions

Based on the survey results, Coalition Greenwich predicts five following trends over the next 12 months:

1. Exchanges and trading systems will continue to launch a wide array of new cloud-based and possibly cloud exclusive data services across derived data, end of day data, reference data and pricing data.
2. Data providers will launch new data products such as pre-trade analytics powered by AI/ML in the cloud.
3. Commercial and investment banks will offer additional connectivity, real-time data feeds, and trading applications delivered via the cloud.
4. Buy-side firms will consume even more cloud-deployed data, including real-time market data, portfolio management data, and risk analytics.
5. Exchanges, trading systems and data providers will explore proof-of-concepts around core systems on the cloud. Improvements to AI/ML tooling or infrastructure will ramp up as firms seek more rapid responses to risk initiatives.

To learn more about these findings, download our two full reports, The Future of market data: Distribution and consumption through cloud and AI and Exchanges and data providers: Prioritizing the cloud and AI for internal insights or our short infographic.

Research methodology

The survey was conducted online by Coalition Greenwich on behalf of Google Cloud from March 2021 to April 2021 among 102 executives in North America (n=82), EMEA (n=17) and other (n=3) who are employed full-time and who are participants or influencers in decisions around cloud and/or senior management with a role at a company which is an institutional asset manager, hedge fund, alternative investment manager, exchange and/or trading system, information provider, information aggregator, or other asset manager/asset owner. The survey included wide perspectives from a range of firm size and asset class focus, including equity, fixed income, FX, commodities, multi-asset, and other asset classes.

Foot Notes

1.  We defined market data as direct feeds, consolidated feeds, terminal and desktop products, security and reference data, pricing data, historical data, alternative data, and index data.

“Cloud Wisdom Weekly: for tech companies and startups” is a new blog series we’re running this fall to answer common questions our tech and startup customers ask us about how to build apps faster, smarter, and cheaper. In this installment, we explore how to leverage artificial intelligence (AI) and machine learning (ML) for faster innovation and efficient operational growth.

Whether they’re trying to extract insights from data, create faster and more efficient workflows via intelligent automation, or build innovative customer experiences, leaders at today’s tech companies and startups know that proficiency in AI and ML is more important than ever.

AI and ML technologies are often expensive and time-consuming to develop, and the demand for AI and ML experts still largely outpaces the existing talent pool. These factors put pressure on tech companies and startups to allocate resources carefully when considering bringing AI/ML into their business strategy. In this article, we’ll explore four tips to help tech companies and startups accelerate innovation and reduce costs with AI and ML.

4 tips to accelerate innovation and reduces costs with AI and ML

Many of today’s most innovative companies are creating services or products that couldn’t exist without AI—but that doesn’t mean they’re building their AI and ML infrastructure and pipelines from scratch. Even for startups whose businesses don’t directly revolve around AI, injecting AI into operational processes can help manage costs as the company grows. By relying on a cloud provider for AI services, organizations can unlock opportunities to energize development, automate processes, and reduce costs.

1. Leverage pre-trained ML APIs to jumpstart product development

Tech companies and startups want their technical talent focused on proprietary projects that will make a difference to the business. This often involves the development of new applications for an AI technology, but not necessarily the development of the AI technology itself. In such scenarios, pre-trained APIs help organizations quickly and cost-effectively establish a foundation on which higher-value, more differentiated work can be layered.

For example, many companies building conversational AI into their products and services leverage Google Cloud APIs such as Speech-to-Text and Natural Language. With these APIs, developers can easily integrate capabilities like transcription, sentiment analysis, content classification, profanity filtering, speaker diarization, and more. These powerful technologies help organizations focus on creating products rather than having to build the base technologies.

See this article for examples of why tech companies and startups have chosen Google Cloud’s Speech APIs for use cases that range from deriving customer insights to giving robots empathetic personalities. For an even deeper dive, see

• our AI product page to explore other APIs, including Translation, Vision, and more;
• and the Google Cloud Skills Boost for ML APIs.

2. Use managed services to scale ML development and accelerate deployment of models to production

Pre-trained models are extremely useful, but in many cases, tech companies and startups need to create custom models to either derive insights from their own data or to apply new use cases to public data. Regardless of whether they’re building data-driven products or generating forecasting models from customer data, companies need ways to accelerate the building and deployment of models into their production environments.

A data scientist typically starts a new ML project in a notebook, experimenting with data stored on the local machine. Moving these efforts into a production environment requires additional tooling and resources, including more complicated infrastructure management. This is one reason many organizations struggle to bring models into production and burn through time and resources without moving the revenue needle.

Managed cloud platforms can help organizations transition from projects to automated experimentation at scale or the routine deployment and retraining of production models. Strong platforms offer flexible frameworks, fewer lines of code required for model training, unified environments across tools and datasets, and user-friendly infrastructure management and deployment pipelines.

At Google Cloud, we’ve seen customers with these needs embrace Vertex AI, our platform for accelerating ML development, in increasing numbers since it launched last year. Accelerating time to production by up to 80% compared to competing approaches, Vertex AI provides advanced end-to-end ML Ops capabilities so that data scientists, ML engineers, and developers can contribute to ML acceleration. It includes low-code features, like AutoML, that make it possible to train high performing models without ML expertise.

Over the first half of 2022, our performance tests found that the number of customers utilizing AI Workbench increased by 25x. It’s exciting to see the impact and value customers are gaining with Vertex AI Workbench, including seeing it help companies speed up large model training jobs by 10x and helping data science teams improve modeling precision from the 70-80% range to 98%.

If you are new to Vertex AI, check out this video series to learn how to take models from prototype to production. For deeper dives, see

• this article about Vertex AI’s role in an ambitious project to measure climate change with AI;

• BigQuery has built-in Machine Learning (ML) and Analytics that you can use to create no-code predictions using just SQL queries.
• this blog about how Vertex AI and BigQuery work together to make data analysis easier and more powerful;
• and this blog about Example-based explanations, one of our most recent updates to make model iteration more intuitive and efficient.

3. Harness the cloud to match hardware to use cases while minimizing costs and management overhead

ML infrastructure is generally expensive to build, and depending on the use case, specific hardware requirements and software integrations can make projects costly and complicated at scale. To solve for this, many tech companies and startups look to cloud services for compute and storage needs, attracted by the ability to pay only for resources they use while scaling up and down according to changing business needs.

At Google Cloud, customers share that they need the ability to optimize around a variety of infrastructure approaches for diverse ML workloads. Some use Central Processing Units (CPUs) for flexible prototyping. Others leverage our support for NVIDIA Graphics Processing Units (GPUs) for image-oriented projects and larger models, especially those with custom TensorFlow operations that must run partially on CPUs. Some choose to run on the same custom ML processors that power Google applications—Tensor Processing Units (TPUs). And many use different combinations of all of the preceding.

Beyond matching use cases to the right hardware and benefiting from the scale and operational simplicity of a managed service, tech companies and startups should explore configuration features that help further control costs. For example, Google Cloud features like time-sharing and multi-instance capabilities for GPUs — as well as features like Vertex AI Training Reduction Server — are built to optimize GPU costs and usage.

Vertex AI Workbench also integrates with the NVIDIA NGC catalog for deploying frameworks, software development kits and Jupyter Notebooks with a single click—another feature that, like Reduction Server, speaks to the ways organizations can make AI more efficient and less costly via managed services.

4. Implement AI for operations

Besides using pre-trained APIs and ML model development to develop and deliver products, startup and tech companies can improve operational efficiency, especially as they scale, by leveraging AI solutions built for specific business and operational needs, like contract processing or customer service.

Google Cloud’s DocumentAI products, for instance, apply ML to text for use cases ranging from contract lifecycle management to mortgage processing. For businesses whose customer support needs are growing, there’s Contact Center AI, which helps organizations build intelligent virtual agents, facilitate handoffs as appropriate between virtual agents and human agents, and generate insights from call center interactions. By leveraging AI to help manage operational processes, startups and tech companies can allocate more resources to innovation and growth.

Next steps toward an intelligent future

The tips in this article can help any tech company or startup find ways to save money and boost efficiency with AI and ML. You can learn more about these topics by registering for Google Cloud Next, kicking off October 11, where you’ll hear Google Cloud’s latest AI news, discussions, and perspectives—in the meantime, you can also dive into our Vertex AI quickstarts and BigQuery ML tutorials. And for the latest on our work with tech companies and startups, be sure to visit our Startups page.

Editor’s note: This blog has been updated since it originally published to reflect the total number of generative AI courses that have become available since the publish date.

An AI/ML (artificial intelligence/machine learning) career path can be a great specialty area within the cloud—and one of the most accessible! Because this area is constantly developing, most recently with the rise of generative AI, I want to share some recommendations to help you chart a sustainable career as AI/ML continues to evolve.

Generative AI falls under the overall category of AI and offers a new and exciting way of interacting with information, brands, and other people. Let’s talk about how these disciplines work together and about materials available to help you upskill in these areas.

To this end, we are happy to announce a new set of generative AI training content available at no cost. So, whether you are just getting started or already have a more advanced role, read on to find ways to help reach your desired position. 

ML compared to AI

While the terms ML and AI are often used interchangeably, there are distinct differences. AI is an umbrella concept wherein machines are taught to perform tasks normally associated with human intelligence, such as decision-making and language interaction. ML is a subset of AI dedicated to taking data from the past and training algorithms to create models that can perform highly complex tasks without being explicitly programmed. It’s the basis for most forms of AI that people interact with, like virtual assistants, music recommendations, and chatbots.

Data engineers and ML engineers work with data scientists to get insights from data. They are needed to create software models and get clear results, and develop deployable applications. These skilled roles are needed in every industry!

Change the world for the better with generative AI

I recently wrote about four key pillars of technology trends expected in the next decade, including the role of AI/ML in the cloud environment as one of those pillars, and how you can bridge the skills gap and build your career.

Generative AI is a new type of ML that has made a lot of headlines recently. Research from CIO Dive finds that seven in 10 executives say their companies are investigating or exploring generative AI. Now is a great time to become an expert, while we are at the cusp of this technology becoming more widely adopted. 

To get to generative AI, we need to talk first about deep learning. Deep learning is a subset of ML that uses artificial neural networks to process more complex patterns than traditional ML. Generative AI sits still further down the funnel, as a subset of deep learning that typically involves the Transformer architecture. Essentially, it’s a type of AI that can map long-range dependencies and patterns in large training sets, then use what it learns to produce new content, including text, imagery, audio, and synthetic data.

Generative AI relies on large models, such as large language models (LLMs) that can classify and generate text, answer questions, and summarize documents. For more detail about how this works, check out the video below from Google I/O 2023, which features great information from Dr. Gwendolyn Stripling, Artificial Intelligence Technical Curriculum Developer for Google Cloud.

How Google is offering generative AI

Here at Google, we’ve been heavily invested for decades in research and innovation aimed at helping businesses, governments, and developers maximize the potential of AI. Our vision is to empower builders, innovators, developers, and doers to use AI in unique, responsible, productive ways. 

To deliver on this, we recently announced a variety of solutions to bring generative AI into our offerings, beginning with Generative AI support on Vertex AI and Gen App Builder. 

Vertex AI is Google Cloud’s ML platform for training and deploying ML models and AI applications. With Generative AI support on Vertex AI, data science teams and developers can access foundation models from both Google and other sources, helping them to quickly build, customize, and deploy models for their own use cases. 

As part of generative AI support on Vertex AI, Model Garden and Generative AI Studio are now in preview, including access to PaLM 2 for Text and Chat and Embeddings API for Text, while other features and services are available to select trusted testers. If you’d like early access to Google Cloud’s AI products, join the waitlist here.

We announced our latest PaLM model in production at I/O 2023: PaLM 2. Today, it powers more than 25 products! Additionally, we’ve fine-tuned PaLM 2 for specific use cases, including security and medical domains. You’ve probably also heard of Bard, our experiment for conversational AI, which is fully running on PaLM 2. 

Our Gen App Builder aims to make customer, partner, and employee interactions more effective and helpful. With it, developers — even those with limited data science expertise — can quickly create bots, chat apps, digital assistants, custom search engines, and more. Gen App Builder even makes it possible to create some generative AI apps without any coding skills. 

Now, let’s bring these products to life! Not sure exactly the use case or in need of a little inspiration? Check out this blog from Google Cloud CEO Thomas Kurian to see how customers and partners in the ecosystem are bringing ideas to life with generative AI. And watch this video to see generative AI in action!

Build your AI/ML skills and validate your knowledge to grow your career

There is a lot of excitement around generative AI—it’s truly a brand new path to consider for your AI/ML career! Check out my recommendations below for training options for AI/ML roles. These will help you gain critical skills as generative AI becomes more widely available.

*NEW* training materials, specific to generative AI technologies
Generative AI Learning Path – no cost training 

• [Course] Introduction to Generative AI (1 day)
• [Course] Introduction to Large Language Models (1 day)
• [Course] Attention Mechanism (1 day)
• [Course] Transformer Models and BERT Model (1 day)
• [Course] Introduction to Image Generation (1 day)
• [Course] Create Image Captioning Models (1 day)
• [Course] Encoder-Decoder Architecture (1 day)

Additional AI/ML training – varying learning credits required to complete on Google Cloud Skills Boost 
Introductory level

• [Course with completion badge] How Google Does Machine Learning (1 day)
• [Course with completion badge] MLOps: Getting Started (1 day)
• [Skill badge] Get started with TensorFlow on Google Cloud (8 hours)  
• [Skill badge] Perform foundational ML, AI and data tasks in Google Cloud (7 hours)
• [Course with completion badge] Language, Speech, Text, and Translation with Google Cloud APIs (5 hours)

Intermediate/Multi-level

• [Learning Path] Machine Learning Engineer (Collection of 15 video courses and labs) 
• [Skill badge] Build and Deploy Machine Learning Solutions on Vertex AI (1 day)

Advanced

• [Training + certification exam] Google Cloud Professional ML Engineer Certification (varied time)
• [Skill badge] Machine Learning with TensorFlow in Vertex AI (90 minutes)
• [Course with completion badge] Natural Language Processing in Google Cloud (1 day)

You can also catch up on hands-on demos in our most recent Cloud OnBoard: From Data to AI with BigQuery and Vertex AI. In it, we guide you through all steps of the data-to-AI solution.

Coming up on June 29, we are hosting Getting Started with Vertex Generative AI as part of Innovators Live. Register now!

I also encourage you to consider an Innovators Plus subscription. You get access to over 700 labs, skill badges and courses (including many that are specific to AI/ML), a certification exam voucher, and up to $1,000 in Google Cloud credits. You’ll also get access to live learning events and technical briefings with Google Cloud experts, plus 1:1 consultations (talk with us about your upcoming generative AI project!). It’s a $1,500 package for only $299. Learn more!