While some traditional financial services companies have more slowly transitioned to the cloud, capital markets firms have embraced cloud computing across their entire value chains — front-, middle-, and back-office. We wanted to understand the dynamics behind this rapid adoption, the most common use cases, and the types of technology most in use, particularly as it relates to market data. Google Cloud commissioned Coalition Greenwich to survey 102 institutional capital markets professionals — at exchanges, trading systems, data aggregators, data producers, asset managers, hedge funds, and investment banks — in the United States, Canada, France, Germany, Italy, the Netherlands, Switzerland, and the United Kingdom. 

Our research found that while there are many drivers, demand for easier accessibility is fueling widespread adoption of cloud-based market data services, and associated trading infrastructures, across the buy side and sell side. In fact, 68% of sell-side and buy-side users find it critical for market data providers to offer public cloud-based data services. At the same time, exchanges, market data providers, aggregators, and trading systems are embracing the cloud as a delivery model by offering access to data directly via their own cloud services, APIs or partners.

Here were five noteworthy takeaways from the study: 

1. Cloud services are becoming ubiquitous for data delivery. Today, the cloud is pervasive, with 93% of exchanges, trading systems and data providers offering cloud-based data and services, according to surveyed executives. Moreover, 100% of those surveyed intend to offer new cloud-based services, such as derived data, in the next 12 months.

2. Commercial and investment banks are offering additional connectivity, real-time data feeds, and trading applications delivered via the cloud,demonstrating that it’s not only exchanges, trading systems, and data providers that are moving rapidly to the cloud. Internal use cases abound as well, with 67% of those surveyed consuming cloud-deployed market data, primarily for data analytics. 88% of surveyed sell-side firms intend to consume cloud-based market data services, with digital transformation, data science and quant research as the top use cases.

3. Buy side firms will consume even more cloud-deployed data. Today, 90% of surveyed buy-side firms are consuming cloud-deployed market data, mostly for portfolio management. 70% of buy-side firms intend to consume more public cloud-based market data services in the next 12 months, adding services such as compliance and regulatory reporting.

4. AI/ML, powered by cloud, is moving out of the pilot phase and into mainstream use. Today, 50% of exchanges, trading systems, and data providers are offering data products or services powered by AI/ML, and of those, 42% intend to offer AI-powered trade execution and trading analytics services in the next 12 months. Within commercial and investment banks, 55% said they are currently using AI/ML in the cloud, and while that was true for only 14% of overall buy-side respondents, 44% of large buy-side respondents are using it.

5. Exchanges, trading systems, and data providers are prioritizing public cloud for internal insights. 71% of these firms are using the public cloud, mostly for data transmission, processing, analysis, and long-term data storage. Over the next 12 months, 33% of new public cloud workloads will focus on data mining, data insights and advanced analytics, while 28% of new AI/ML tooling and infrastructure investments will focus on faster analytics and risk reviews, and 27% on data quality maintenance.

“We see new, dramatic shifts on the adoption of cloud across market data,” said David Easthope, Senior Analyst for Coalition Greenwich. “And we expect further proliferation of cloud-based services and greater consumption across the trading and investing lifecycle.”

Conclusions and future predictions

Based on the survey results, Coalition Greenwich predicts five following trends over the next 12 months:

1. Exchanges and trading systems will continue to launch a wide array of new cloud-based and possibly cloud exclusive data services across derived data, end of day data, reference data and pricing data.
2. Data providers will launch new data products such as pre-trade analytics powered by AI/ML in the cloud.
3. Commercial and investment banks will offer additional connectivity, real-time data feeds, and trading applications delivered via the cloud.
4. Buy-side firms will consume even more cloud-deployed data, including real-time market data, portfolio management data, and risk analytics.
5. Exchanges, trading systems and data providers will explore proof-of-concepts around core systems on the cloud. Improvements to AI/ML tooling or infrastructure will ramp up as firms seek more rapid responses to risk initiatives.

To learn more about these findings, download our two full reports, The Future of market data: Distribution and consumption through cloud and AI and Exchanges and data providers: Prioritizing the cloud and AI for internal insights or our short infographic.

Research methodology

The survey was conducted online by Coalition Greenwich on behalf of Google Cloud from March 2021 to April 2021 among 102 executives in North America (n=82), EMEA (n=17) and other (n=3) who are employed full-time and who are participants or influencers in decisions around cloud and/or senior management with a role at a company which is an institutional asset manager, hedge fund, alternative investment manager, exchange and/or trading system, information provider, information aggregator, or other asset manager/asset owner. The survey included wide perspectives from a range of firm size and asset class focus, including equity, fixed income, FX, commodities, multi-asset, and other asset classes.

Foot Notes

1.  We defined market data as direct feeds, consolidated feeds, terminal and desktop products, security and reference data, pricing data, historical data, alternative data, and index data.

Getting started with your migration

One of the challenges with cloud migration is that you’re solving a puzzle with multiple pieces. In addition to a number of workloads you could migrate, you’re also solving for challenges you’re facing, the use cases driving you to migrate, and the benefits you’re looking to gain. Each organization’s puzzle will likely get solved in their own unique way, but thankfully there is plenty of guidance on how you can migrate common workloads in successful ways. 

In addition to working directly with our Rapid Assessment and Migration Program (RAMP), we also offer a plethora of self-service guides to help you succeed! Some of these guides, which we’ll cover below, are designed to help you identify the best ways to migrate, which include meeting common organizational goals like minimizing time and risk during your migration, identifying the most enterprise-grade infrastructure for your workloads, picking a cloud that aligns with your organization’s sustainability goals, and more: 

• Get started with this cloud migration guide and checklist
• Key considerations for building a migration factory to Google Cloud
• Understanding the cloud migration strategies at your disposal
• Understanding how to migrate key enterprise workloads to Google Cloud

Which workloads are you moving?

In addition to the guides above which help you through your end-to-end migration, it’s also important to understand the specific workloads you’ve got on-premises (or in other clouds). Each workload has their own unique nuances — what works for one might not work perfectly for another. This is where leveraging the expertise of our RAMP team is crucial, and why we have lots of migration guides for specific workloads and end-states. For each workload, we’re highlighting a few guides that we think could be most helpful, but you can also click each topic to learn more or find more guides. 

Microsoft

• Modernization path for .NET applications on Google Cloud
• Managed Service for Microsoft Active Directory is GA
• Migrating Microsoft workloads to Google Cloud yields results

VMware

• Migrate, scale, and innovate at speed
• Boost Retail Intelligence and Agility in the Cloud
• Retire your tech debt: Move vSphere 5.5+ to Google Cloud VMware Engine
• Financial Services Spotlight:  Elevating agility and security in the cloud

Oracle

• Curious about Google Cloud Bare Metal Solution? Start here
• Getting Started with Google Cloud’s Bare Metal Solution
• Guide – Bring new life to your databases with an Oracle migration

SAP

• SAP on Google Cloud: Migration strategies
• Run your SAP workloads on Google Cloud Bare Metal Solution
• eBook – Google Cloud for SAP: The enterprise cloud for business agility, insights, and innovation

Storage

• How to transfer your data to Google Cloud
• An Overview of the Transfer Appliance
• Migration to Google Cloud: Transferring your large datasets
• Transferring data from Amazon S3 to Cloud Storage using VPC Service Controls and Storage Transfer Service

Databases

• Overview of Database Migration Service
• Database migration: Concepts and principles (Part 1)
• Database migration: Concepts and principles (Part 2)
• Database cloud migrations made easy for PostgreSQL, MySQL

Data Warehouses

• Migrating data warehouses to BigQuery: Introduction and overview
• Oracle to BigQuery migration
• Amazon Redshift to BigQuery migration guide
• IBM Netezza to BigQuery migration guide
• Teradata to BigQuery migration guide

Take the next step

When it comes to migration, we’re committed to meet every organization where they are. We fully understand the nuances and challenges of cloud migration, and at Google Cloud we have one singular goal: to help you realize true business value through your cloud migrations. 

Visit our Migration Architecture Center to find even more guides to use during your migration, or if you’re looking to dive a little deeper into planning, sign up for a free discovery and assessment of your existing IT landscape.

Small and medium-sized businesses and enterprises are the backbone of the US economy generating more than 44% of GDP¹. Yet these organizations are still underserved when it comes to online financial tools—and dealing with payments is no exception. Complaints include hidden fees, limited capabilities, long-term lease agreements, and poor customer service.

These are the issues that we wanted to solve when we launched Helcim in 2020 in Calgary (Alberta, Canada). We provide a payment service that offers low rates through our interchange plus pricing model, no monthly fee for the core payments offering, and numerous payment options, as well as simple, affordable hardware such as card readers. 

Our digital-first approach makes it easy for owners of small and medium-sized businesses to get started. Online sign-up means no paperwork and near instant access to Helcim’s software and all-in-one platform experience. Merchants can choose from a range of payment solutions from the Helcim app including in-person payments, SMS payment requests, online invoices with pay now buttons, and more. 

To achieve our goals, we built most of the business systems and processes in-house including our technology stack, financial partnerships, marketing, and everything in between. This is something that few startups would dare to do, but it enabled us to build a payments platform offering the rich capabilities and performance SMBs really need.

Taking control with the cloud

Before migrating our infrastructure to Google Cloud, it was hosted at two colocation data centers in Calgary. This model served us well, but as we grew, most of our hardware needed to be replaced to maintain service security and performance.

A successful round of Series A funding also impacted our trajectory. Giving us the fuel we needed to scale and innovate faster. When we considered the choice between making a large capital investment in our existing environment, or to transition to the cloud, the decision was clear: The cloud was the way to go.

We looked at other big names in cloud hosting and tested another platform. But Google Cloud is by far the best environment for us. It’s much easier for a lean technology team to manage, as we embark on our first cloud strategy. It also offers all the tools and advanced machine learning capabilities we need.

Google Cloud also comes with the backing of Alphabet, a business that in the past five years has spent more on research and development than any other organization in the S&P 500². The ability to engage the Google Workspace account team for guidance to improve our everyday processes was another bonus. 

A variety of investment and training programs from Google Cloud also influenced our decision. We participated in Google for Startups Accelerator Canada which gave us access to Google Cloud experts across all our technology domains. This helped accelerate our infrastructure migration while ensuring we optimized every service from day one. We were also eligible for $100,000 USD of Google Cloud credits, covering our Google Cloud costs which helped us get everything up and running cost-effectively. The partnership with the wider Google team has been extremely impactful as we stand everything up for our business.

But ultimately, it’s the sheer depth and breadth of the Google Cloud environment that makes the difference. Here are the Google Cloud tools we currently have at Helcim.

BigQuery

We chose BigQuery because it is easy to aggregate new data and apply the right access controls. It also delivers outstanding performance running our analytics workloads.

Vertex AI

Vertex AI makes the deployment of new models exponentially faster, and with Google Cloud, we can do most of the work through containers instead of proprietary tooling.

GKE

GKE has enabled us to migrate to a fully managed cloud service and avoid a lift-and-shift exercise that would have prevented us from getting the full benefits of a containerized infrastructure. Running a new GKE environment also enabled us to significantly reduce platform latency by 20%-50%. We can now deploy projects in hours instead of days. 

Cloud Storage

We updated our centralized file system to use Cloud Storage, which is faster and more scalable.

Cloud SQL

All of our software today is built on top of MySQL, so CloudSQL was the natural choice for cloud database management.

Cloud Run

Cloud Run is more flexible than other serverless tools and was an easy way to reduce the burden of managing some of our services.

Boosting performance across the business

By migrating to Google Cloud, we transformed our application performance. We were able to decouple the infrastructure between systems and use modern server hardware for our compute and database instances. With very little change to our code, we saw a 50%+ increase in the speed of our entire platform.

Giving developers more control of the technology running their systems via containers means that they can enhance systems through more frequent language updates and by deploying new technologies to optimize workloads.

We can more closely monitor our systems to diagnose issues and fix their root cause faster. Being able to quickly add resources gives us further options if we experience platform latency or increased traffic.

Maintaining momentum with machine learning

Leveraging Vertex AI has significantly reduced the time it takes the team to build and deploy new machine learning models. Greater agility in our data stack has also freed up time for exploratory work in the data team. For instance, by creating low fidelity proxy data for human behavior in the application process, we created a new model that will reduce the number of manually reviewed batches by more than 10%.

We’ve also been able to improve our deployment process and the time to rollback. As a result, breaking changes in production have been reduced from more than five minutes to less than 30 seconds.

Thanks to BigQuery, we can make better use of data to support key business decisions. Previously it was hard to aggregate data from different sources and while maintaining our strict requirements for customer data confidentiality. We also needed specialist SQL knowledge to consume it. By investing in a more modern data stack, the availability of trusted data across the organization has increased exponentially. 

We’ve also overcome the constraints imposed by static hardware environments especially when maintaining a high-availability configuration between two locations. With Google Cloud, we’re no longer constrained by such a rigid arrangement and the deployment velocity of new infrastructure tooling has been reduced from months to days.

Security is another area where Google Cloud excels. From hackers and fraudsters to bots and web attacks, it protects our users, applications, and data, while facilitating compliance with local and regional authorities. We can also integrate more easily with our security partners ensuring that we can empower our team to stay ahead of cyber criminals and other external threats.

Building the payments platform for the future, today

When we look to the future, Google Cloud opens the door to dozens of opportunities to widen our appeal to SMBs while remaining competitive. Its advanced infrastructure for cloud computing, data analytics and ML supports our roadmap to profitability and will help us attract future rounds of funding.

Above all it provides a foundation for growth. We grew 400% in 2021 and raised more capital in the spring of 2022 to grow even faster. In 2022 we were also listed as one of the top payments processors by industry publications such as Nerdwallet and Merchant Maverick. With Google Cloud, we can build on this success, continue to innovate, and help our SMB customers take their payments and e-commerce strategies to the next level. 

If you want to learn more about how Google Cloud can help your startup, visit our page here to get more information about our program, and sign up for our communications to get a look at our community activities, digital events, special offers, and more. 

To learn more about Google for Startups Accelerators and to apply to a program in your region, visit the website here. 

^{1. Small Businesses Generate 44 Percent Of U.S. Economic Activity
2. Alphabet: Big Value In Big Tech}