While some traditional financial services companies have more slowly transitioned to the cloud, capital markets firms have embraced cloud computing across their entire value chains — front-, middle-, and back-office. We wanted to understand the dynamics behind this rapid adoption, the most common use cases, and the types of technology most in use, particularly as it relates to market data. Google Cloud commissioned Coalition Greenwich to survey 102 institutional capital markets professionals — at exchanges, trading systems, data aggregators, data producers, asset managers, hedge funds, and investment banks — in the United States, Canada, France, Germany, Italy, the Netherlands, Switzerland, and the United Kingdom. 

Our research found that while there are many drivers, demand for easier accessibility is fueling widespread adoption of cloud-based market data services, and associated trading infrastructures, across the buy side and sell side. In fact, 68% of sell-side and buy-side users find it critical for market data providers to offer public cloud-based data services. At the same time, exchanges, market data providers, aggregators, and trading systems are embracing the cloud as a delivery model by offering access to data directly via their own cloud services, APIs or partners.

Here were five noteworthy takeaways from the study: 

1. Cloud services are becoming ubiquitous for data delivery. Today, the cloud is pervasive, with 93% of exchanges, trading systems and data providers offering cloud-based data and services, according to surveyed executives. Moreover, 100% of those surveyed intend to offer new cloud-based services, such as derived data, in the next 12 months.

2. Commercial and investment banks are offering additional connectivity, real-time data feeds, and trading applications delivered via the cloud,demonstrating that it’s not only exchanges, trading systems, and data providers that are moving rapidly to the cloud. Internal use cases abound as well, with 67% of those surveyed consuming cloud-deployed market data, primarily for data analytics. 88% of surveyed sell-side firms intend to consume cloud-based market data services, with digital transformation, data science and quant research as the top use cases.

3. Buy side firms will consume even more cloud-deployed data. Today, 90% of surveyed buy-side firms are consuming cloud-deployed market data, mostly for portfolio management. 70% of buy-side firms intend to consume more public cloud-based market data services in the next 12 months, adding services such as compliance and regulatory reporting.

4. AI/ML, powered by cloud, is moving out of the pilot phase and into mainstream use. Today, 50% of exchanges, trading systems, and data providers are offering data products or services powered by AI/ML, and of those, 42% intend to offer AI-powered trade execution and trading analytics services in the next 12 months. Within commercial and investment banks, 55% said they are currently using AI/ML in the cloud, and while that was true for only 14% of overall buy-side respondents, 44% of large buy-side respondents are using it.

5. Exchanges, trading systems, and data providers are prioritizing public cloud for internal insights. 71% of these firms are using the public cloud, mostly for data transmission, processing, analysis, and long-term data storage. Over the next 12 months, 33% of new public cloud workloads will focus on data mining, data insights and advanced analytics, while 28% of new AI/ML tooling and infrastructure investments will focus on faster analytics and risk reviews, and 27% on data quality maintenance.

“We see new, dramatic shifts on the adoption of cloud across market data,” said David Easthope, Senior Analyst for Coalition Greenwich. “And we expect further proliferation of cloud-based services and greater consumption across the trading and investing lifecycle.”

Conclusions and future predictions

Based on the survey results, Coalition Greenwich predicts five following trends over the next 12 months:

1. Exchanges and trading systems will continue to launch a wide array of new cloud-based and possibly cloud exclusive data services across derived data, end of day data, reference data and pricing data.
2. Data providers will launch new data products such as pre-trade analytics powered by AI/ML in the cloud.
3. Commercial and investment banks will offer additional connectivity, real-time data feeds, and trading applications delivered via the cloud.
4. Buy-side firms will consume even more cloud-deployed data, including real-time market data, portfolio management data, and risk analytics.
5. Exchanges, trading systems and data providers will explore proof-of-concepts around core systems on the cloud. Improvements to AI/ML tooling or infrastructure will ramp up as firms seek more rapid responses to risk initiatives.

To learn more about these findings, download our two full reports, The Future of market data: Distribution and consumption through cloud and AI and Exchanges and data providers: Prioritizing the cloud and AI for internal insights or our short infographic.

Research methodology

The survey was conducted online by Coalition Greenwich on behalf of Google Cloud from March 2021 to April 2021 among 102 executives in North America (n=82), EMEA (n=17) and other (n=3) who are employed full-time and who are participants or influencers in decisions around cloud and/or senior management with a role at a company which is an institutional asset manager, hedge fund, alternative investment manager, exchange and/or trading system, information provider, information aggregator, or other asset manager/asset owner. The survey included wide perspectives from a range of firm size and asset class focus, including equity, fixed income, FX, commodities, multi-asset, and other asset classes.

Foot Notes

1.  We defined market data as direct feeds, consolidated feeds, terminal and desktop products, security and reference data, pricing data, historical data, alternative data, and index data.

Scale-out workloads demand the best combination of performance and price to bring down the cost of delivering applications, all while providing an excellent user experience. We are excited to announce a new virtual machine (VM) family, Tau VMs, coming to Google Cloud. Tau VMs extend Compute Engine’s VM offerings with a new option optimized for cost-effective performance of scale-out workloads. 

T2D, the first instance type in the Tau VM family, is based on 3rd Gen AMD EPYC^TM processors and leapfrogs the VMs for scale-out workloads of any leading public cloud provider available today, both in terms of performance and workload total cost of ownership (TCO). The x86 compatibility provided by these AMD EPYC processor-based VMs gives you market-leading performance improvements and cost savings, without having to port your applications to a new processor architecture. 

As illustrated below, Tau VMs offer 56% higher absolute performance and 42% higher price-performance (est. SPECrate2017_int_base) compared to general-purpose VMs from any of the leading public cloud vendors.

What our customers and partners are saying

Snap
“At Snap, it is critical for our business to continue improving our scale-out compute infrastructure for key Snapchat capabilities like AR, Lenses, Spotlight and Maps,” said Cody Powell, Senior Engineering Manager, Snap Inc. “We were impressed when we tested Google Cloud’s new Tau VMs with Google Kubernetes Engine. While it’s early days, we believe we can gain double digits in infrastructure performance improvements for key workloads—enabling us to do more with less and invest even more in new features for our amazing Snapchat community.”

Twitter
“High performance at the right price point is a critical consideration as we work to serve the global public conversation,” said Nick Tornow, Platform Lead, Twitter. “We are excited by initial tests that show potential for double digit performance improvement. We are collaborating with Google Cloud to more deeply evaluate benefits on price and performance for specific compute workloads that we can realize through use of the new Tau VM family.”

DoiT
“DoiT partners with leading cloud vendors who are focused on growth and cost optimization,” said Yoav Toussia-Cohen, CEO, DoiT International. “In our preliminary testing of Google’s new Tau VMs with the Coremark benchmark, we were thrilled to see the incredible performance at 50% better than a comparable ARM-based offering from another leading public cloud. With Tau VMs, Google Cloud has set a new bar for price-performance, making the cloud even more accessible to digital-native companies. We are excited to bring Google’s Tau VMs to our joint customers.”

Designed for demanding scale-out workloads 

Tau VMs bring the benefit of Google’s long-standing experience engineering platforms for scale-out workloads to our customers. They come in multiple predefined VM shapes, with up to 60vCPUs per VM, and 4GB of memory per vCPU. They offer up to 32 Gbps networking bandwidth and a wide range of network attached storage options, making Tau VMs ideal for scale-out workloads including web servers, containerized microservices, data-logging processing, media transcoding, and large-scale Java applications. 

Google Kubernetes Engine support

Google Kubernetes Engine (GKE) is the de facto standard for organizations looking for advanced container orchestration, delivering the highest levels of reliability, security, and scalability. GKE supports Tau VMs on day 1, helping you optimize price-performance for your containerized workloads. You can add Tau VMs to your GKE clusters by specifying the T2D machine type in your GKE node-pools. 

Pricing

Tau VMs will be priced to support significant TCO and price-performance improvements for your cloud applications. A 32vCPU VM with 128GB RAM will be priced at $1.3520 per hour for on-demand usage in us-central1. 

Coming soon to a Google Cloud region near you

If you are interested in trying out T2D VMs when they become available in Q3 2021 please sign-up here.

Soldiers can now use their CAC cards to log into the service’s newly established Google Workspace account. With it, soldiers who don’t need the full spectrum of information-sharing provided by Army365 can nevertheless find comrades, schedule meetings and coordinate other activities.

“All new entrants to the Army will be provisioned automatically with a Google Workspace account with a new email address … when they receive their CAC card,” Raj Iyer, the service’s chief information officer, said in a social-media post.

“Soldiers will retain their Google account till they fully transition to their first unit after completion of basic training and AIT [advanced infantry training], at which point their commanders will determine if they need an Army365 account.”

The directive applies to all active-duty, reserve, and National Guard soldiers.

The digital media and entertainment industry is experiencing dramatic growth, as audiences migrate to online experiences and content providers seek to deliver new and innovative content. According to The Global Internet Phenomena Report, streaming video accounted for 53.7% of internet bandwidth traffic, up by 4.8% from a year ago. This rapid growth of over-the-top content is straining existing infrastructure, fueling media companies’ shift to the public clouds with their global presence and greater distribution capacities. In addition, other use cases such as gaming, social networks, AR/VR experiences, and education continue to fuel the need for intelligent media services and operations.

Today, at the 2022 NAB Show Streaming Summit, we’re excited to announce the general availability of Media CDN — a modern, extensible platform for delivering immersive experiences with unparalleled scale and intelligence. Media CDN will enable media and entertainment customers to efficiently and intelligently deliver streaming experiences to viewers anywhere in the world. The same infrastructure that Google has built over the last decade to serve YouTube content to over 2 billion users is now being leveraged to deliver media at scale to Google Cloud customers with Media CDN.

Unparalleled planet-scale reach and scale

Media CDN’s foundational advantage is the Google network. We have invested decades of resources to build tremendous capacity and reach in over 200 countries and more than 1,300 cities around the globe. Modern video applications are sensitive to fluctuations in latency, so getting content closer to users enables higher bitrates and reduces rebuffers, resulting in a superior experience for the end user. Media CDN builds on the success of the existing Cloud CDN portfolio for web and API acceleration and complements it by enabling delivery of immersive media experiences.

In addition to running on planet-scale infrastructure, Media CDN tailors delivery protocols to individual users and network conditions. Media CDN includes out-of-the-box support for QUIC (HTTP/3), TLS 1.3, and BBR, optimizing for last-mile delivery . When the Chrome team rolled out widespread support for QUIC, video rebuffer time decreased by more than 9% and mobile throughput increased by over 7%.

Media CDN also achieves industry-leading offload rates. With multiple tiers of caching, we minimize calls to origin — even for infrequently accessed content. This alleviates performance or capacity stress in the content origin and saves costs. These features are built into the product and seamlessly support customer content hosted on Google Cloud, on-premises, or on a third-party cloud.

“We are excited to leverage Media CDN to continue to deliver an exceptional streaming experience for Stan users across Australia. With Google’s massive network, and a deep reach into the ISPs, we are able to deliver the highest quality video for our users, no matter where they are”—John Hogan, Chief Technology Officer, Stan

“Our mission at U-NEXT is to deliver the highest quality and most entertaining content to our users. Google Cloud’s Media CDN helps us efficiently scale our infrastructure, which is challenging with a vast library of content. Media CDN offloaded 98.3% of requests from our origin server while delivering consistent great quality.”—Rutong Li, Chief Technology Officer, U-NEXT

Broader platform for monetization and immersive experiences

While global distribution is critical for a high-quality end-user experience, it’s only one piece of delivering a world-class platform for immersive experiences. Media CDN offers additional capabilities to enable this transformation — ad insertion, ecosystem integrations and platform extensibility, and powerful AI/ML analytics for interactive experiences.

Streaming providers can improve monetization through integrated ad serving via the Video Stitcher API, which allows manipulation of video content to dynamically insert ads.

Through extensible ecosystem integrations, Media CDN connects customers to key capabilities to simplify their operations. For example, the Transcoder API supports custom streaming formats, while the Live Stream API transcodes mezzanine live signals into direct-to-consumer streaming formats, for multiple device platforms.

Media CDN is built with AI/ML that will give viewers more control over how they see, experience, and even interact with content. For example, sports fans watching a game can obtain real-time stats and analytics, viewers can purchase items from virtual billboards, etc.

Cloud-native and developer-friendly operations

Media companies are under pressure to develop and deploy innovative experiences at a furious pace. Media CDN was built by developers, for developers, with automation and observability built in, giving media providers the speed and flexibility they need to integrate delivery provisioning and management into their content release processes.

Media CDN offers comprehensive APIs and automation tools such as Terraform. Detailed, pre-aggregated metrics and playback tracing make it easy to diagnose performance across the entire infrastructure stack. Real-time visibility is provided via Google Cloud’s operations suite, and integrates with tools that developers already use such as Grafana and ElasticSearch.

“Leveraging the same infrastructure as YouTube, Google Cloud’s Media CDN combines geographic reach, API-first architecture and integration with the Cloud operations suite. This is a transformative move that is aligned with the future of the CDN industry.”— Ghassan Abdo, Research Vice President, WW Telecom, Virtualization and CDN, IDC

“Viewers around the world are demanding best-in-class video quality and performance across modes of consumption. A video-first delivery network can be a game changer in this space. We’re excited to partner with Google Cloud and to leverage Media CDN to enable premium video experiences and customer engagements.”—Juan Martin, Founder and CTO, Firstlight Media

Planet-scale advanced security

Media CDN lets streaming media providers take advantage of Google’s decades-long experience delivering video safely, securely, and reliably. The platform includes deep integration with Google Cloud Armor for planet-scale DDoS protection and a rich set of capabilities to detect and mitigate attacks, prevent abuse, manage risk, and comply with regulatory or licensing requirements.

If you want to deliver rich, immersive experiences to global audiences with an extensible, modern delivery platform, we’d love to hear from you. For more information, including technical specifications and platform architecture, please visit cloud.google.com/media-cdn. To get started with Media CDN, contact your sales team.