How Cloud Networks Enable CSPs to Deliver 5G
5126
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
Communication services providers (CSPs) are experiencing a period of disruption. Overall revenue growth is decelerating and is projected to remain below 1 percent per year, following a trend that started even before the pandemic.1 At the same time, driven by the pandemic, data consumption in 2020 increased by 30 percent relative to 2019, with some operators even reporting increases of 60 percent.2
The combination of pressure on revenues with rising data traffic costs is forcing operators to innovate in three fundamental ways. First, operators are looking to establish new sources of revenue. Second, increased network utilization must be met with a reduction in network cost. And third, there is an opportunity to gain new customers by improving the customer experience.
Fortunately, 5G offers a path forward across each of these three areas. Concepts such as network slicing and private networks allow CSPs to offer differentiated network services to public sector and enterprise customers. The disaggregation of hardware and software allows new vendors with unique strengths to enter the market and to enable CSPs to build, deploy, and operate networks in fundamentally new ways. And the ability to place workloads at the edge permits CSPs to offer compelling experiences to consumers and businesses alike. In this blog, we will discuss how CSPs can create a solid foundation for their cloud networks.
Understanding telecommunications networks
First, it is useful to consider the way telecommunications networks were traditionally built. Initially, networks were built using physical network functions (PNFs) — appliances that used a tight combination of hardware and software to perform a specific function. PNFs offered the benefit of being purpose-built for a specific application, but they were inflexible and difficult to upgrade. As an example, deploying new features frequently required replacing the entire PNF, i.e., deploying a new hardware appliance.
The first step in improving deployment agility came with the concept of virtualized network functions (VNFs), software workloads designed to operate on commercial off-the-shelf (COTS) hardware. Rather than utilizing an integrated hardware and software appliance, VNFs disaggregated the hardware from the software. As such, it became possible to procure the hardware from one vendor and the software from another. It also became possible to separate the hardware and software upgrade cycles.
However, while VNFs offered advantages over PNFs, VNFs were still an intermediate step. First, they typically needed to be run within a virtual machine (VM), and as such required a hypervisor to interface between the host operating system (OS) and the guest OS inside the VM. The hypervisor consumed CPU cycles and added inefficiency. Second, the VNF itself was frequently designed as a monolithic function. This meant that while it was possible to upgrade the VNF separately from the hardware, such an upgrade, even for a feature that affected only a portion of the VNF, required deployment of the entire large VNF. This created risk and operational complexity, which in turn meant that upgrades were delayed just as they were with PNFs.
Creating the foundation for cloud networks
The trick to establishing your cloud based network resides in the challenge of moving from VNFs to containerized network functions (CNFs) — network functions organized as containers as a collection of small programs, each of which can be independently operated.
The concept of containers is not new. In fact, Google has been using containerized workloads for over 15 years. Kubernetes, which Google developed and open-sourced, is the world’s most popular container orchestration system, and is based on Borg, Google’s internal container management system.3 There are lots of benefits to using containers, but fundamentally, it frees developers from worrying about resource scheduling, interprocess communication, security, self-healing, load balancing, and many other tedious (but important!) tasks.
Consider just a couple examples of benefits that containerization brings to network functions. First, when upgrading the network function to implement new features, you no longer need to re-deploy the entire network function. Instead, you only need to re-deploy the containers that are affected by the upgrade. This improves developer velocity and reduces the risk of the upgrade because, rather than infrequent upgrades that each introduce substantial changes, you can now have frequent upgrades that each deploy small changes. Small changes are less risky because they are easier to understand and to roll back in case of anomaly. Incidentally, this also improves your security posture because it reduces the time between when a security vulnerability is discovered and when a patch is deployed.
Speaking of security, another example of the benefits that containerization brings to network functions is an automatic zero-trust security posture. In Kubernetes, the communication among microservices can be handled by a service mesh, which manages mundane aspects of inter-services communication such as retries in case of failure and providing observability into communication. It can also manage other essential aspects such as security. For example, Anthos Service Mesh, which is a fully-managed implementation of the open-source Istio service mesh (also co-developed by Google), includes the ability to authenticate and encrypt all communications using mutual TLS (mTLS) and to deploy fine-grained access control for each individual microservice.
Automation and orchestration for cloud networks
CNFs bring tremendous benefits, but they also bring challenges. In place of a relatively small number of network appliances, we now have a large number of containers, each of which requires configuration, management, and maintenance. In the past, many of these processes were accomplished using manual techniques, but this is impossible to accomplish economically and reliably at the scale required by CNFs.
Fortunately, there are cloud-native approaches to solving these challenges. First, consider the problem of autonomously deploying and maintaining CNFs. The ideal way is to use the concept of Configuration as Data. Unlike imperative techniques such as Infrastructure as Code, which provide a detailed description of a sequence of steps that need to be executed to achieve an objective, Configuration as Data is a declarative method whereby the user specifies the desired end state (i.e., the actual desired configuration) and relies on automated controllers to continuously drive the infrastructure to achieve that state. Kubernetes includes such automated controllers, and the great news is that this method can be used not just for infrastructure but also for the applications residing on top of it, including CNFs. This cloud-native technique frees you from the toil and associated risk of writing detailed configuration procedures, so you can focus on the business logic of your applications.
As another example, consider the problem of understanding your network performance, including anomaly detection, root cause analysis, and resolution. The cloud-native approach starts with creating a data platform where both infrastructure and CNF monitoring data can be ingested, regularized, processed, and stored. You can then correlate data sets against each other to detect anomalies, and with AI/ML techniques, you can even anticipate anomalies before they happen. AI/ML is likewise indispensable in gaining an understanding of why the anomaly is happening, i.e. performing root cause analysis, and automated closed-loop controllers can be developed to correct the problem, ideally before it even happens.
Architecting for the edge
The transition from VNFs to CNFs is a critical piece in addressing the challenge that CSPs face today, but it alone is not enough. CNFs need infrastructure to run on, and not all infrastructure is created equal.
Consider a typical 5G network. There are some functions, such as those associated with an access network, that need to be deployed at the edge. These functions require low latency, high throughput, or even a combination of the two. In 5G networks, examples of such functions include the radio unit (RU), distributed unit (DU), centralized unit (CU), and the user plane function (UPF). The first three are components of the radio access network (RAN), while the last is a component of the 5G core. At the same time, there are some other control plane functions such as the session management function (SMF) or the authentication and mobility management function (AMF) that do not have such tight latency and high throughput requirements and can thus be placed in a more centralized data center. Furthermore, consider an AI/ML use case where a particular model (perhaps for radio traffic steering) needs to run at the network edge because of its latency requirements. While the model itself needs to run at the edge, model training (i.e., generating the model coefficients) is frequently a compute-intensive exercise that is latency-insensitive and is thus more optimal to run in a public cloud region.
All of these use cases have one thing in common: they call for a hybrid deployment environment. Some applications must be deployed at the edge as close to the user as possible. Others can be deployed in a more centralized environment. Still others can be deployed in a public cloud region to take advantage of the large amount of compute and economies of scale available therein. Wouldn’t it be convenient — if not transformational — if you could use a single environment for deploying at the edge, in a private datacenter, and in public cloud, with a consistent set of security, lifecycle management, policy, and orchestration resources across all such locations? This is indeed what Google Distributed Cloud, enabled by Anthos, brings to the table.
With Google Distributed Cloud, you can architect a 5G network deployment such as the one shown below.
Business benefits of cloud networks
Beyond the technical benefits, consider the business benefits of such an architecture. First, by following the best practices of hardware and software disaggregation, it permits the CSP to procure the infrastructure and the network functions from different vendors, spurring competition among vendors. Second, each workload is placed in precisely the right location, enabling efficient utilization of hardware resources and offering compelling low-latency, high-throughput services to users. Third, because the architecture utilizes a common hybrid platform (Anthos), it makes it easy to move workloads across infrastructure locations. Fourth, the separation of workloads into microservices accelerates time-to-market when developing new features or applications, such as those enabling enterprise use cases. And finally, the container management platform supports the simultaneous deployment of both network functions and edge applications on the same infrastructure, allowing the operator to deploy new experiences such as AR/VR directly on bare metal as close to the user as possible.
The next generation cloud network is now
There is a lot more we could say, but perhaps the most important takeaway is that this architecture is not a future dream. It exists today, and Google is working with leading CSPs and network vendor partners to deploy it, helping them realize the promise of 5G to deliver new revenues, reduce operating costs, and enable new customer experiences.
To learn more, watch the video series on the cloudification of CSP networks.
Discover what’s happening at the edge: How CSPs Can Innovate at the Edge.
1.Statista, Forecast growth worldwide telecom services spending from 2019 to 2024
2 PricewaterhouseCoopers, Global entertainment and media outlook 2021-2025
3. Borg: The Predecessor to Kubernetes
Modernize your Windows Server Workloads using Google Cloud Platform
DOWNLOAD RESEARCH REPORTS6252
Of your peers have already downloaded this article
1:30 Minutes
The most insightful time you'll spend today!
Application Modernization is an important enabler of Digital Transformations (DX), which fuel competitive advantage through increased productivity and business agility. Public cloud infrastructure proves to be a solid foundation for application modernization by providing Self-Service Provisioning capabilities, cloud-based & cloud-native technologies, and easier access to technology innovations such as AI/ML.
Windows Server-based enterprise applications rely on the underlying infrastructure for platform performance, security, and availability. A better performing cloud platform enables them to perform better and hence prove to be more resource-optimized and cost-effective.
Download this IDC report to understand why you should move your Windows Server workloads to Google Cloud and the benefits you can derive.
Explore Google Cloud’s Bi-monthly Technical Learning Series for Innovators in Public Sector
3348
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
Cloud engineers face a constant barrage of new cloud services, products, and innovations. By late 2021, Google Cloud alone had released thousands of new features across hundreds of services. Couple this with other technologies and service releases, and it quickly becomes a herculean task for engineers to navigate, consume, and stay current on the ever changing technology landscape. We have heard from engineers this often leads to anxiety and frustration as engineers struggle to keep up. They are faced with a plethora of training options but often lack the time and funding.
Google Cloud has reinvigorated technical training to make it more informative and applicable to public sector customers and partners. We aim to maximize your training experience so you can get targeted training when you need it. The Google Cloud Public Sector Technical Learning Series addresses customer feedback and provides fun and practical training. Sessions are currently running every two weeks.
“Short and sweet” technical topics geared to subjects you care about
Generic training doesn’t always resonate with public sector technologists. Our new curriculum targets specific public sector use cases, is delivered by customer engineers, and can be accomplished in less than two hours. This means participants can apply the learnings directly to real-life challenges quickly.
Easy to find, easy to enroll
Training opportunities should always be at your fingertips. Our automated training platform will ensure that you only need to enroll once. The system will automatically notify you of upcoming sessions so you can plan in advance and at your convenience. Sessions will be offered on a recurring basis to meet the needs of your organization.
Fun and engaging
Typical training sessions often include a sea of glazed eyes, unresponsive to basic prompts, falling asleep at our desks, we have all been there. But it doesn’t have to be this way. Our goal is to infuse Google culture into our training through interactive exchanges and tangible rewards to keep participants inspired and engaged.
Traditional technology training doesn’t always help you navigate the nuts and bolts of how to effectively introduce a product into an organization. But we know that technology doesn’t operate in isolation; it supports and becomes part of a living organism, managed by humans and confined by other components of an organization’s structure (e.g. existing systems or decentralized business units).
Part of a larger community of like-minded engineers
Learning with – and from – a community of peers is one way to overcome the challenges and complexities of applying new technology within a complex organization. We created the Public Sector Connect community for this very reason. It is one example of how we surface best practices for public sector innovators. During weekly “Coffee Hours” and working sessions, our community members share their journey and lessons learned with each other. We know that innovation evolves through iteration and diverse perspectives, and Public Sector Connect is committed to helping surface critical challenges and solutions, and connecting those who are solving similar problems. Join the community today.
6357
Of your peers have already watched this video.
2:15 Minutes
The most insightful time you'll spend today!
What’s Google Cloud Firestore Database and What are its Benefits for Business and Developers?
Cloud Firestore is a NoSQL document database that simplifies storing, syncing, and querying data for your mobile and web apps at global scale.
Cloud Firestore is a fast, fully managed, serverless, cloud-native NoSQL document database that simplifies storing, syncing, and querying data for your mobile, web, and IoT apps at global scale.
Its client libraries provide live synchronization and offline support, while its security features and integrations with Firebase and Google Cloud Platform (GCP) accelerate building truly serverless apps.
Here’s other stuff it’s good at:
Sync data across devices, on or offline
With Cloud Firestore, your applications can be updated in near real time when data on the back end changes. This is not only great for building collaborative multi-user mobile applications, but also means you can keep your data in sync with individual users who might want to use your app from multiple devices.
With Firebase Realtime Database, we felt we had built the best force-plate testing software on the market. Thanks to Cloud Firestore, in only two weeks, we built a system that’s significantly better and includes features we never thought possible to ship on Day 1.
Chris Wales, CTO, Hawkin Dynamics
Cloud Firestore has full offline support, so you can access and make changes to your data, and those changes will be synced to the cloud when the client comes back online. Built-in offline support leverages local cache to serve and store data, so your app remains responsive regardless of network latency or internet connectivity.
Simple and effortless
Cloud Firestore’s robust client libraries make it easy for you to update and receive new data while worrying less about establishing network connections or unforeseen race conditions. It can scale effortlessly as your app grows. Cloud Firestore allows you to run sophisticated queries against your data. This gives you more flexibility in the way you structure your data and can often mean that you have to do less filtering on the client, which keeps your network calls and data usage more efficient.
Enterprise-grade, scalable NoSQL
Cloud Firestore is a fast and fully managed NoSQL cloud database. It is built to scale and takes advantage of GCP’s powerful infrastructure, with automatic horizontal scaling in and out, in response to your application’s load. Security access controls for data are built in and enable you to handle data validation via a configuration language.
Google Cloud’s Data Analytics May Recap
6905
Of your peers have already read this article.
5:00 Minutes
The most insightful time you'll spend today!
May was a very busy month for data analytics product innovation. If you didn’t have the chance to attend our inaugural Data Cloud Summit, video replays of all our sessions are now available so feel free to watch them at your own pace.
In this blog, I’d like to share some background behind the innovations we released in May, why we built them the way we did, and the type of value they can bring your company and your team.
But first, a huge thank you!
This week, we had the honor to announce that Google has been named a Leader in The Forrester Wave™: Streaming Analytics, Q2 2021 report. Forrester gave Dataflow a score of 5 out of 5 across 12 different criteria, stating: “Google Cloud Dataflow has strengths in data sequencing, advanced analytics, performance, and high-availability”.
Google has more than a decade of experience in building real-time and internet-scale systems for its own needs, and we are excited to see that our ability to provide customers with a reliable, scalable, and performant platform is bearing fruit.
This announcement comes on the back of the release of The Forrester Wave™: Cloud Data Warehouse, Q1 2021 report, which also named Google Cloud as a Leader.
We couldn’t be more excited about the recognition and appreciate all your feedback and trust in the work that we do to support your goal in accelerating data-powered innovation.
Innovation galore
Your feedback and your passion is the fuel that drives our ambition to deliver more and better services to you. That’s why, this year, we didn’t want to wait until Google Cloud Next to share some great products we have been working on. On May 26, our team announced a slew of new products, services and programs. Watch a quick summary below:
https://youtube.com/watch?v=DG1mOPMXJvw%3Fenablejsapi%3D1%26
Meeting you where you are
An important design principle behind all of our services is “meeting you where you are”. This means we aim to provide you with the tools and software you need to innovate on your own terms. Here are three new services that will help you do just that:
Datastream
Datastream, our new serverless change data capture (CDC) and replication service, allows your company to synchronize data across heterogeneous databases, storage systems, and applications reliably and with minimal latency to support real-time analytics, database replication, and event-driven architectures. Datastream delivers change streams from Oracle and MySQL databases into Google Cloud services such as BigQuery, Cloud SQL, Cloud Storage, and Cloud Spanner, saving time and resources while ensuring your data is accurate and up-to-date.
- Under the hood, Datastream reads CDC events (inserts, updates, and deletes) from source databases, and writes those events with minimal latency to a data destination. It leverages the fact that each database source has its own CDC log—binlog for MySQL and LogMiner for Oracle—which it uses for its own internal replication and consistency purposes.
- Datastream integrates with purpose-built and extensible Dataflow templates to pull the change streams written to Cloud Storage, and create up-to-date replicated tables in BigQuery for analytics. It also leverages Dataflow templates to replicate and synchronize databases into Cloud SQL or Cloud Spanner for database migrations and hybrid cloud configurations.
- Datastream also powers a Google-native Oracle connector in Cloud Data Fusion’s new replication feature for easy ETL/ELT pipelining. By delivering change streams directly into Cloud Storage, customers can leverage Datastream to implement modern, event-driven architectures.
Looker and BigQuery Omni on Microsoft Azure
Research on multi cloud adoption is unequivocal — 92% of businesses in 2021 report having a multi cloud strategy. We want to continue supporting your choice by providing the flexibility you need to see your strategy through.
- This past month, we introduced Looker, hosted on Microsoft Azure. For the first time, you can now choose Azure, Google Cloud, or AWS for your Looker instance. You can also self-host your Looker instance on-premises.
- We also introduced BigQuery Omni for Azure, which along with last year’s introduction of BigQuery Omni for AWS, will help you access and securely analyze data across Google Cloud, AWS, and Azure.
The cost of moving data between cloud providers isn’t sustainable for many, and it’s still difficult to seamlessly work across clouds. BigQuery Omni represents a new way of analyzing data stored in multiple public clouds, which is made possible by BigQuery’s separation of compute and storage. By decoupling these two, BigQuery provides scalable storage that can reside in Google Cloud or other public clouds, and stateless resilient compute that executes standard SQL queries.
- Unlike competitors, BigQuery Omni doesn’t require you to move or copy your data from one public cloud to another, where you might incur egress costs. You also benefit from the same BigQuery interface on Google Cloud, enabling you to query data stored in Google Cloud, AWS, and Azure without any cross-cloud movement or copies of data.
- BigQuery Omni’s query engine runs the necessary compute on clusters in the same region where your data resides. For example, you can query Google Analytics 360 Ads data stored in Google Cloud and query logs data from your ecommerce platform and applications that are stored in AWS S3 and/or Microsoft Azure.
Then, using Looker, you can build a dashboard that allows you to visualize your audience behavior and purchases alongside your advertising spend.
Dataplex
We understand that most organizations still struggle to make high-quality data easily discoverable and accessible for analytics, across multiple silos, to a growing number of people and tools within their organization.
They are often forced to make tradeoffs. For instance, moving and duplicating data across silos to enable diverse analytics use cases or leaving their data distributed but limiting the agility of decisions.
- Dataplex provides an intelligent data fabric that enables you to centrally manage, monitor, and govern your data across data lakes, data warehouses, and data marts, while also ensuring data is securely accessible to a variety of analytics and data science tools.
- One of the core tenets of Dataplex is letting you organize and manage your data in a way that makes sense for your business, without data movement or duplication. For that, we provide logical constructs like lakes, data zones, and assets. These constructs enable you to abstract away the underlying storage systems and become the foundation for setting policies around data access, security, lifecycle management, and so on.
- For example, you can create a lake per department within your organization (e.g. Retail, Sales, Finance, etc.) and create data zones that map to data readiness and usage (e.g. landing, raw, curated_data_analytics, curated_data_science, etc.).
Once you have your lakes and zones setup, you can attach data to these zones as assets. You can add data from different types of storage (e.g. GCS Bucket and BigQuery dataset) under the same zone. You can also attach data across multiple projects under the same zone. You can ingest data into your lakes and zones using the tools of your choice, including services such as Dataflow, Data Fusion, Dataproc, Pub/Sub, or choose from one of our partner products. Dataplex comes with built-in 1-click templates for common data management tasks.
To find out more about Dataplex, head to cloud.google.com/dataplex or watch the video below:
https://youtube.com/watch?v=bbFeAt7cw1g%3Fenablejsapi%3D1%26
Helping you innovate everyday
Sharing data is hard. Traditional data sharing techniques use batch data pipelines that are expensive to run, create late arriving data, and can break with any changes to the source data. These techniques also create multiple copies of data, which brings unnecessary costs and can bypass data governance processes. They also fail to offer features for data monetization, such as managing subscriptions and entitlements. Altogether, these challenges mean that organizations are unable to realize the full potential of transforming their business with shared data.
Analytics Hub
To address these limitations, we are introducing Analytics Hub, a new fully managed service that helps organizations unlock the value of data sharing, leading to new insights and increased business value.
This new service is built on the tremendous experience and feedback we have received over the years. For example, BigQuery has had cross-organizational, in-place data sharing capabilities since its inception in 2010—and the functionality is very popular. Over a 7-day period in April, we had over 3,000 different organizations sharing over 200 petabytes of data. These numbers don’t include data sharing between departments within the same organization.
Analytics Hub takes sharing to the next level, making it easy for you to publish, discover, and subscribe to valuable datasets that you can combine with your own data to derive unique insights.
This includes:
- Shared datasets: As a data publisher, you create shared datasets that contain the views of data that you want to deliver to your subscribers. Data subscribers can search through the datasets that are available across all exchanges for which they have access and subscribe to relevant datasets. In addition, the publisher can track subscribers, disable subscriptions, and see aggregated usage information for the shared data.
- Curated, self-service data exchanges: Exchanges are collections used to organize and secure shared datasets. By default, exchanges are completely private, but granular roles and permissions make it easy to deliver data to the right audience—whether internal or public.
This is just the beginning for Analytics Hub. Please sign up for the preview, which is scheduled to be available in the third quarter of 2021.
Dataflow Prime
At Google Cloud, we have the great privilege of working with some of the most innovative organizations in the world. And this work provides us with a unique perspective into the future of big data processing. Dataflow Prime is a new platform based on a serverless, no-ops, and auto-tuning architecture that brings unparalleled resource utilization and radical operational simplicity to big data processing. This new service introduces a large number of exciting capabilities but I’d like to highlight three key aspects of the product:
- Vertical Autoscaling: Dataflow Prime dynamically adjusts the compute capacity allocated to each worker based on utilization, detecting when jobs are limited by worker resources and automatically adding more resources. Vertical Autoscaling works hand in hand with Horizontal Autoscaling to seamlessly scale workers to best fit the needs of the pipeline. As a result, it no longer takes hours or days to determine the perfect worker configuration to maximize utilization.
- Right Fitting: Each stage of a pipeline typically has a different resource requirement than the others. Until now, either all workers in the pipeline would have had the higher memory and GPU, or none of them would. Pipelines either had to waste resources or suffer slower workloads. Right Fitting solves this problem by creating stage-specific pools of resources, optimized for each stage.
- Smart Recommendations: Smart Recommendations automatically detects problems in your pipeline and shows potential fixes. For example, if your pipeline is running into permissions issues, a Smart Recommendation will detect which IAM permissions you need to enable to unblock your job. If you are using an inefficient coder in your job, Smart Recommendations will surface more performant coder implementations that can help you save on costs.
What’s next
We’re excited to hear your thoughts and feedback about all these exciting new services. I would also highly recommend that you connect with members of the community to learn more about their story and journey. A good example to start with is the Data To Value customer panel we produced at our inaugural Data Cloud Summit with the Chief Data Officers of Keybank and Rackspace. You can watch it for free below:
https://youtube.com/watch?v=ITI2Q3MkxuA%3Fenablejsapi%3D1%26
The Fantastic Story of How BMG Enables a Micropayments Strategy So Music Artists Get Paid
7432
Of your peers have already read this article.
3:30 Minutes
The most insightful time you'll spend today!
The music industry is rapidly changing. Only 20 years ago, the availability of music and the infrastructure that was required to make an album a sales success were incredibly complex and expensive. With the decline of physical sales and a fundamental shift to digital, music streaming now accounts for more than half of all sales globally.
At the same time, technology has democratized music-making; in many ways, it has made the musical landscape more diverse. Artists can upload their music with the click of a button. But while it’s easier for creators to share their songs with audiences, getting paid has become more fragmented.
Although music is booming, people no longer buy it outright. Instead, listeners download music digitally or subscribe to streaming services to have their libraries with them at all times. To monetize digital content effectively, artists need to know when, where, and how often their songs are played on each service. To help them navigate this complicated royalties landscape and maximize their profits, Berlin-based international music company BMG provides customized, transparent, and fair services to songwriters and artists.
With publishing and recording divisions under one roof, the subsidiary of international media giant Bertelsmann works with both emerging artists and established stars, including John Legend, Kylie Minogue, Mick Jagger, and Keith Richards. With the MyBMG web and mobile application, clients can view and analyze their royalty details in real time and collect payment. When a new record is released, BMG uses data to maximize its impact and revenue for its creators.
“We make sure that everyone who uses our clients’ music knows who needs to be paid the associated royalties, then we collect these royalties and share them out quickly and transparently,” explains Sebastian Hentzschel, Chief Information Officer at BMG. “When our artists release new music, we make sure that it’s marketed and promoted effectively around the world.”
“We needed a scalable solution for our royalty workloads that was intuitive for our developers. We also wanted a partner, not a client-vendor relationship. With autoscaling via BigQuery, excellent customer support, and a clean and simple user interface, Google Cloud ticks every box for us.”
—Gaurav Mittal, Vice President Group Technology, BMG
Getting up to speed with a new way of paying artists
In this digital world, artists aren’t just paid every time a fan buys an album—they’re paid a small amount, or royalty, for each song downloaded or streamed by a listener. So, when the industry shifted to digital, the volume of data that BMG needed to handle grew exponentially. “One CD sale is equivalent to about 1,500 streamed songs or plays,” says Gaurav Mittal, Vice President Group Technology at BMG. “That means IT departments have to process 1,500 times the amount of data to calculate payments for artists, and this makes scalable micropayment processing very important.”
Until 2019, BMG’s infrastructure was entirely hosted on-premises. Hardware limitations made it challenging to scale on-demand, making it harder to handle the data peaks that royalty processing can bring. “With our on-premises infrastructure, we were going to hit a ceiling in a few years,” says Gaurav. “We still managed to process royalty payments for our clients, but it was increasingly time consuming and expensive. To keep focusing on our clients, rather than our infrastructure, we decided to migrate to Google Cloud.”
From the outset, Gaurav and his team had a clear vision for the partnership: “Most importantly, we needed a scalable solution for our royalty workloads that was intuitive for our developers. We also wanted a partner, not a client-vendor relationship,” he says. “With autoscaling via BigQuery, excellent customer support, and a clean and simple user interface, Google Cloud ticks every box for us.”
Keeping artists happy with business-as-usual payouts during migration
To move applications to the cloud while keeping payment cycles on track for its artists, BMG teamed up with Google Cloud partner Rackspace Technology. “We selected Rackspace Technology because it combines strong technical muscle and a global footprint, with the customer service of a local boutique firm,“ shares Gaurav.
BMG’s own technology team put together the outline for the Google Cloud architecture, which they passed on to Rackspace Technology for optimizations and the ultimate stamp of approval. Whenever Gaurav and his developers needed support, Rackspace Technology was ready to go. “So far, we’ve migrated 17 applications successfully, and Rackspace Technology has been 100% spot-on with each suggestion,” says Gaurav. “I can’t recall a single flaw in a Rackspace Technology-approved architecture, and that really says something.”
When BMG began its migration in August 2019, the team developed an ambitious two-year plan. Only 14 months later, however, the project is more than 75% complete. Among the solutions that BMG is using today are Cloud Storage to securely store 130 TB of data, and Cloud SQL as its standard database technology. The web applications run on Compute Engine, App Engine, and Google Kubernetes Engine.
“After successfully moving a few applications, it was clear that with the strong teamwork of BMG and Rackspace Technology, together with the ease of use of Google Cloud, we could speed up the project without sacrificing quality,” says Gaurav. “We’re set to complete our migration six months before schedule, helping us to quickly move out of our hybrid environment.”
“Our income-tracking teams are very savvy on the data, and the simplicity of Google Cloud empowers them to self-serve analytics, rather than wait for IT. Our teams are much more productive.”
—Gaurav Mittal, Vice President Group Technology at BMG
Royalty reporting and processing with BigQuery and Dataproc
So far, all of BMG’s critical workloads are up and running on Google Cloud. Royalty calculations, for example, which require incredible processing power and the collection of many micropayments to ensure full and timely payout, run entirely on Dataproc with output stored on BigQuery for downstream integration and reporting.
Enabling more harmonious workflows through self-serve analytics
As the new beating heart of BMG’s royalty reporting, BigQuery changed the rhythm of collaboration company-wide. In the past, income tracking teams had to contact IT departments if they needed deeper data insights for their work. By integrating Data Catalog with BigQuery, BMG has made the data more accessible to all teams. This helps them detect missing income and new revenue streams independently, maximizing profits for artists.
“Our income-tracking teams are very savvy on the data, and the simplicity of Google Cloud empowers them to self-serve analytics, rather than wait for IT,” says Gaurav. “Our teams are much more productive.”
“Google Cloud enables us to be more client focused and deliver better features faster. We believe it’s just the beginning. We offer rights and royalty services for music publishing, recorded music, neighboring rights, and books. Without scalability limitations, it’s absolutely conceivable to offer our platform as a service to other companies or industries, such as gaming. Google Cloud has opened a world of possibilities.”
—Sebastian Hentzschel, Chief Information Officer, BMG
With a leaner IT environment, BMG can focus its effort on the needs of its clients. Beyond improvements in royalty processing, it can concentrate on app development, releasing new features and enhancements more frequently. By hosting applications on Google Kubernetes Engine, App Engine, and Compute Engine, BMG has built a CI/CD pipeline with automated deployments and testing to significantly speed up workflows.
“In our old system, it could take several weeks to set up an environment,” says Gaurav. “With Google Kubernetes Engine, any of our developers can complete the process in a few clicks. Having that autonomy makes our developers more motivated and self-driven.”
“Google Cloud enables us to be more client focused and deliver better features faster,” adds Sebastian. “We believe it’s just the beginning. We offer rights and royalty services for music publishing, recorded music, neighboring rights, and books. Without scalability limitations, it’s absolutely conceivable to offer our platform as a service to other companies or industries, such as gaming. Google Cloud has opened a world of possibilities.”
With the migration almost complete, BMG is looking forward to its next technology project. It plans to leverage AutoML to further scale and automate royalty tracking with machine learning. On the marketing side, advanced analytics will help BMG determine the effectiveness of promotional campaigns around the world, further increasing profits for artists. By connecting Google Data Studio to BigQuery, BMG will increase the quality of its analyses, helping musicians better understand the reach of their music around the world.
In the end, Gaurav shares, helping musicians is what it all comes down to. “We’re a new kind of music company because we build our services around our artist, songwriter, and publisher clients, not the other way around,” he says. “Google Cloud is helping us maintain strong relationships with our clients, and that’s music to our ears.”
More Relevant Stories for Your Company
The Future of Cloud Computing
Increasingly, technology, data, and human interactions will follow a new model of real-time inputs, enabling change without interruption and positive iteration. Cloud computing powers an important shift for information technology, and it will usher in new and better ways to serve customers, make discoveries, and build great products and services.
A Pro’s Tip on Choosing the Right Google Cloud Compute Options
Where should you run your workload? It depends...Choosing the right infrastructure options to run your application is critical, both for the success of your application and for the team that is managing and developing it. This post breaks down some of the most important factors that you need to consider
Google and Fervo Agreement to Shape-up Plans for 24/7 Carbon-free Energy by 2030
When Google announced our plan to go beyond purchasing renewable power for 100% of our energy usage and operate on 24/7 carbon-free energy by 2030, we noted that achieving this goal will require new transaction structures, advancements in clean energy policy, and innovative new technologies. Today, we’re pleased to announce that one of
Autonom8: Achieving growth and profits for businesses with Google Cloud
With Google Cloud, Autonom8 can run a platform that accelerates and streamlines customer journeys in a scalable, reliable, cost-effective infrastructure, while using advanced optical character recognition to enable intelligent document processing. Google Cloud results: Increased margins by up to 30% by switching from a home-grown OCR system to Cloud Vision
