When moving software applications from on-premise hardware to the cloud, it often “just works,” but it’s never guaranteed. This is especially the case for applications that are hardware intensive. This blog post examines what happened when a media company took software for real-time video broadcasts into the cloud. We’ll share how we, Google Cloud, collaborated with media software provider Vizrt to meet the demanding requirements of an eSports broadcaster. Together, we delivered a solution that not only met but exceeded the expected performance from a cloud-based deployment.

Video broadcasting in the cloud

Video broadcasts are a very hardware-intensive workflow. By needing to process and store data streams in near-real-time, broadcasts stress GPU, memory, disk, and CPU. In addition, the performance requirements quickly increase as producers add additional video streams into the mix, such as in this case, an eSports broadcast.

Vizrt’s customer wanted to increase their broadcast production by doubling the amount of camera feeds from 8 to 16, to have a more compelling and elaborate production.

At the heart of the eSports broadcasters’ production was Viz Vectar Plus, Vizrt’s software-based 4K switcher. While the client wanted to move more of its production into the cloud, Viz Vectar Plus was designed initially for on-premise hardware. So, when they tried a straightforward “lift and shift ” to the cloud, it surprised no one that the software didn’t run as well. They turned to Google Cloud and Vizrt to make it run the way they needed it to. 

Troubleshooting lift and shift

Initially, we suspected that the issue could be in the design of the cloud deployment, i.e., the configuration of the VM hosting the software. So the focus of our troubleshooting was to find a cloud configuration that 1) made sure the software worked to spec and 2) did so optimally considering costs, robustness, and performance. Furthermore, we wanted to ensure that all components met performance specifications, particularly throughput, IOPS, and network bandwidth, as this was a video media application. Only after we validated the cloud deployment would we ask Vizrt to investigate the code itself. We would:

1. Set up a test environment.
2. Benchmark the environment.
3. Test various configurations.
4. Validate that the vendor software was optimally using the configuration.  

This high-level methodology is straightforward. However, we approached the details in a particular order, considering we were optimizing for broadcast video. We honed in on the optimal cloud configuration by testing the following elements, prioritized in order of expected impact:  

1. VM type: The VM type largely dictates the available memory and CPU configurations. However, because this was a VM workflow, we had to pick N1’s. Today, they are the only VM type that can be attached to GPUs, which are practically a requirement for broadcast video.  
2. Disk type: Video broadcasts require high I/O speeds to handle high-quality video streams. We went from an HDD to a much-faster SSD. 
3. CPU size: We increased the VM CPU cores from 16 in increments up to 32. Increasing CPU size indeed increased performance, but did not return the level of performance we needed.
4. SSD size: We increased the SSD size (and the accompanying higher IOPS and throughput that comes with increasing the size) enabling more simultaneous recordings. Again, this only partially worked. 
5. Disk count. We noticed read/write problems when the application was reading/writing with a single drive. There are two typical ways to approach this: 1) Separate read/writes tasks among two discs and 2) striping the data streams across discs. Implementing these had improved but marginal improvements in performances.  

After our testing, we arrived at the following optimal configuration:

• VM: n1-standard-32 instance w/ 500Gb boot drive
• GPU: 1 T4 GPU 
• SSD: 1 persistent disk with 1TB*

* We would later determine that two SSDs for separate read and write operations would be more optimal

This configuration was able to produce between 6 – 12 streams. Compared to the on-premise target of 8 streams, this was about as good but was not the customer’s target of 16. So we would need Vizrt to take the ball from here to optimize the software itself.

Optimizing media applications for cloud

We provided Vizrt our recommended configuration, performance notes, and the following best practices that are generally applicable to cloud-based video workloads:

• Separate read and write operations to two different disks to enable higher performance for both operations. 
• A second 1 TB SSD persistent disk can be attached to the VM instance to increase performance.

With this information, Vizrt engineers worked their magic, providing daily patches to test; with each daily iteration the overall solution was found quickly. Not only were they able to meet the broadcaster’s request of 16 feeds, but they were also able to go even further to 44. Over a 5x improvement by optimizing for the cloud! 

Teamwork in troubleshooting 

Because of the specialized nature of media and entertainment, workflow situations across multiple companies are common as specialized applications hand their work from one to another. 

“By working in partnership with Google Cloud we managed to build a system that can scale in ways that probably none of us thought would be possible. This allowed Viz Vectar Plus to run fully in the cloud using NDI and opened up amazing possibilities for making shows,” Dr. Andrew Cross, President R&D, Vizrt Group. “We ended up with great feedback from the customer, who were appreciative of how Google Cloud and Vizrt collaborated on a solution.”

The results speak for themselves: A satisfied customer with over a 5x improvement in results. That’s what we call a good game.

There are multiple reasons a growing number of database administrators, enterprise architects, application developers and other technology practitioners are moving to Google Cloud’s various database services.

Some are being driven by missing features in offering from other providers such as AWS. In Gartner’s Magic Quadrant for Operational Database Management Systems, the research and advisory firm points out that, “AWS’s surveyed reference customers scored its overall product capabilities one standard deviation (STD) below the mean. Their responses identified missing features such as multiregion writes and autosharding.”

Others are moving to database services on Google Cloud Platform driven by a few benefits. According to Gartner, “Reference customers repeatedly commented on Google’s ease of use and implementation, reliability and integration (with other services and other systems). Reference customers scored Google a full STD above the mean for satisfaction with GCP’s pricing; it received the second-highest satisfaction score of any vendor in this Magic Quadrant.

If you are looking to leverage the power of Google Cloud database offerings—but were unsure of which database services comes closest to the service you are currently using, here’s a handy map to find your way.

There’s a new subsea cable in town: Topaz, the first-ever fiber cable to connect Canada and Asia.

Once complete, Topaz will run from Vancouver to the small town of Port Alberni on the west coast of Vancouver Island in British Columbia, and across the Pacific Ocean to the prefectures of Mie and Ibaraki in Japan. We expect the cable to be ready for service in 2023, not only delivering low-latency access to Search, Gmail and YouTube, Google Cloud, and other Google services, but also increasing capacity to the region for a variety of network operators in both Japan and Canada.

Google is spearheading construction of the project, joined by a number of local partners in Japan and Canada to deliver the full Topaz subsea cable system. Other networks and internet service providers will be able to benefit from the cable’s additional capacity, whether for their own use or to provide to third parties. And, similar to other cables we’ve built, with Topaz we will exchange fiber pairs with partners who have systems along similar routes. This is a longstanding practice in the industry that strengthens the intercontinental network lattice for network operators, for Google, and for users around the world.

Network infrastructure investments like Topaz bring significant economic activity to the regions where they land. For example, according to a recent Analysys Mason study, Google’s historical and future network infrastructure investments in Japan are forecasted to enable an additional $303 billion (USD) in GDP cumulatively between 2022 and 2026.

The width of a garden hose, the Topaz cable will house 16 fiber pairs, for a total capacity of 240 Terabits per second (not to be confused with TSPs). It includes support for Wavelength Selective Switch (WSS), an efficient and software-defined way to carve up the spectrum on an optical fiber pair for flexibility in routing and advanced resilience. We’re proud to bring WSS to Topaz and to see the technology is being implemented widely across the submarine cable industry.

While Topaz is the first trans-Pacific fiber cable to land on the West Coast of Canada, it’s not the first communication cable to connect to Vancouver Island. In the 1960s, the Commonwealth Pacific Cable System (COMPAC) was a copper undersea cable linking Vancouver with Honolulu (United States), Sydney (Australia), and Auckland (New Zealand), expanding high-quality international phone connectivity. Today, COMPAC is no longer in service but its legacy lives on. The original cable landing station in Vancouver — the facility where COMPAC made landfall on Canadian soil — has been upgraded to fit the needs of modern fiber optics and will house the eastern end of the Topaz cable.

Traditional and treaty rights, and local communities, are deeply important to our infrastructure projects. The Topaz cable is built alongside the traditional territories of the Hupacasath, Maa-nulth, and Tseshaht, and we have consulted with and partnered with these First Nations every step of the way.

“Tseshaht is very proud of this collaboration and our partnership with Google, who has been very respectful and thoughtful in its engagement with our Nation. That’s how we carry ourselves and that’s how we want business to carry themselves in our territory.“ — Tseshaht First Nation – Elected Chief Councillor-Ken Watts

“The five First Nations of the Maa-nulth Treaty Society are pleased that we have concluded an agreement with Google Canada and have consented to the installation of a new, high-speed fiber optic cable through our traditional territories. This agreement, in which both Google Canada and our Nations benefit, is based on respect for our constitutionally protected treaty and aboriginal rights and enhances the process of reconciliation. We would also like to acknowledge the sensitivity that Google Canada expressed during our talks in regard to the pain and trauma experienced by our people as a result of residential school experience. We look forward to a long and mutually beneficial relationship with Google Canada.” —Chief Charlie Cootes, President of the Maa-nulth Treaty Society

“Google’s respect towards our Nation is appreciated and has good energy behind it.” —Hupacasath First Nation – Elected Chief Councilor – Brandy Lauder

With the addition of Topaz today, we have announced investments in 20 subsea cable projects. This includes Curie, Dunant, Equiano, Firmina and Grace Hopper, and consortium cables like Blue, Echo, Havfrue and Raman — all connecting 29 cloud regions, 88 zones, 146 network edge locations across more than 200 countries and territories. Learn about Google Cloud’s network and infrastructure on our website and in the below video.

Enterprises are increasingly seeking out technologies that help them create unique experiences for customers with speed and at scale. At the same time, customers want flexibility when deciding where to manage their enterprise data, particularly when it comes to business-critical applications.

That’s why I’m thrilled that Sprinklr, the unified customer experience management (Unified-CXM) platform for modern enterprises, has partnered with Google Cloud to accelerate its go-to-market strategy and grow awareness among our joint customers. Sprinklr will work closely with our global salesforce, benefitting from our deep relationships with enterprises that have chosen to build on Google Cloud. 

Akin to Google Cloud’s mission to accelerate every organization’s ability to digitally transform their business through data-powered innovation, Sprinklr’s primary objective is to empower the world’s largest and most loved brands to make their customers happier by listening, learning, and taking action through insights. With this strategic partnership now in place, Sprinklr and Google Cloud will go-to-market together with the end-customer as our sole focus.

Traditionally, brands have adopted point solutions to manage segments of the customer journey. In isolation, these may work — but they rarely work collaboratively, even when vendors build “Frankenstacks” of disconnected products. These solutions can’t deliver a 360° view of the customer, and often reinforce departmental silos. All of which creates point-solution chaos.

Sprinklr’s approach is fundamentally different and is the way out of the aforementioned point-solution chaos. As the first platform purpose-built for unified customer experience management (Unified-CXM) and trusted by the enterprise, Sprinklr’s industry-leading AI and powerful Care, Marketing, Research, and Engagement solutions enable the world’s top brands to learn about their customers, understand the marketplace, and reach, engage, and serve customers on all channels to drive business growth. 

Sprinklr was built from the ground up as a platform-first solution, designed to evolve and grow with the rapid expansion of digital channels and applications. The results? Faster innovation. Stronger performance. And a future-proof strategy for customer engagement on an enterprise scale.

“Sprinklr works with large, global companies that want flexibility when deciding where to manage their enterprise data and consider our platform a business-critical application,” said Doug Balut, Senior Vice President of Global Alliances, Sprinklr. “Giving our customers the opportunity to manage Sprinklr on Google Cloud empowers them to create engaging customer experiences while maintaining the high security, scalability, and performance they need to run their business.”

To learn more about this exciting partnership and the challenges we jointly solve for customers, check out the recent conversation between Google Cloud’s VP of Marketing, Sarah Kennedy, and Sprinklr’s Chief Experience Officer, Grad Conn. Or read the press release on the partnership.

New locations in Eventarc

Back in August, we announced more Eventarc locations (17 new regions, as well as 6 new dual-region and multi-region locations to be precise). This takes the total number of locations in Eventarc to more than 30. You can see the full list in the Eventarc locations page or by running gcloud eventarc locations list . 

What does location mean in Eventarc?

An Eventarc location usually refers to the single region that the Eventarc trigger gets created in. However, depending on the trigger type, the location can be more than a single region:

• Pub/Sub triggers only support single-region locations.
• Cloud Storage triggers support single-region, dual-region, and multi-region locations.
• Cloud Audit Logs triggers support single-region locations and the special global region.

Before looking into trigger location in more detail, let’s look at other locations relevant in Eventarc.

What other locations are relevant in Eventarc?

Triggers connect event sources to event targets:

Each event source, event target, and trigger has its own location. Sometimes, these locations have to match and sometimes they can be different.  

Here’s an example of a trigger connecting Cloud Storage events from a bucket in the europe-west1 region to a Cloud Run service in the us-central1 region with a trigger located in the europe-west1 region:

  gcloud eventarc triggers create trigger-storage \
  --destination-run-service=hello \
  --destination-run-region=us-central1 \
  --location=europe-west1 \
  --event-filters="type=google.cloud.storage.object.v1.finalized" \
  --event-filters="bucket=my-bucket-in-europe-west1-region" \
  --service-account=$PROJECT_NUMBER-compute@developer.gserviceaccount.com

In many cases, you don’t have control over the location of the event source. In the example above, the Cloud Storage bucket is in the europe-west1 region. That’s the location that you need to work with and it has implications for the trigger location (which I’ll get to later). 

The location of the event target is the region of the service where you want the events to go. You get to choose this from one of the supported regions when you deploy your Cloud Run service. You typically want this to be in the same region as your event source for latency and data locality reasons (but this is not strictly a requirement). In the example above, the event source (bucket) is in europe-west1 but the event target (Cloud Run service) is in  us-central1 as specified by the --destination-run-region flag. 

The location of the trigger is dictated by the event source location, but the trigger type also comes into play. It is specified by the –location flag. Let’s take a look at the trigger location for each trigger type in more detail.

Location in Pub/Sub triggers

In a Pub/Sub trigger, you connect a Pub/Sub topic to an event target. Pub/Sub topics are global and not tied to a single region. However, when you create a Pub/Sub trigger, you need to specify a region for it (because Eventarc triggers need to live in a region) with the --location flag as follows:

  gcloud eventarc triggers create trigger-pubsub \
  --destination-run-service=hello \
  --destination-run-region=us-central1 \
  --location=us-central1 \
  --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
  --transport-topic=projects/your-projectid/topics/your-topic

By specifying a location, Eventarc automatically configures the geofencing feature in Pub/Sub such that events only persist in the specified location. As I noted above, you typically want to (but are not required to) choose the same region for the trigger and the Cloud Run service for lower latency and data locality. You can also use regional Pub/Sub service endpoints to publish to the topic to ensure that all of the data stays in a single region. 

Location in Cloud Storage triggers

In a Cloud Storage trigger, you connect a Cloud Storage bucket to an event target. A Cloud Storage bucket can be in a single-region (e.g. europe-west1), dual-region (e.g. eur4), or multi-region (e.g. eu) location. The location of the bucket dictates the location of the trigger and they have to match. The earlier trigger example was for a bucket in the  europe-west1 single-region location. Here’s another trigger connecting Cloud Storage events from a bucket in the eu multi-region location. Notice how the location flag matches the bucket region:

  gcloud eventarc triggers create trigger-storage \
  --destination-run-service=hello \
  --destination-run-region=us-central1 \
  --location=eu \
  --event-filters="type=google.cloud.storage.object.v1.finalized" \
  --event-filters="bucket=my-bucket-in-eu-multi-region" \
  --service-account=$PROJECT_NUMBER-compute@developer.gserviceaccount.com

If the bucket region and the trigger region do not match, you’ll see an error:

  ERROR: (gcloud.eventarc.triggers.create) INVALID_ARGUMENT: The request was invalid: Bucket "my-bucket-in-eu-multi-region" location "eu" does not match trigger location "europe-west1". Try again by creating the trigger in "eu".

Location in Cloud Audit Logs triggers

In a Cloud Audit Logs trigger, you connect any event source that emits Audit Logs to an event target. The location of the event source will dictate the trigger location. This is typically a single region but there is a special global region that’s necessary in some cases. 

For example, if you want to read Cloud Storage events from a bucket in the europe-west1 region with an Audit Logs trigger, you will create the trigger with the same location. Note that this will match all buckets in the europe-west1 region as there’s no filter by bucket in Audit Logs:

  gcloud eventarc triggers create trigger-auditlog \
  --destination-run-service=hello \
  --destination-run-region=us-central1 \
  --location=europe-west1 \
  --event-filters="type=google.cloud.audit.log.v1.written" \
  --event-filters="serviceName=storage.googleapis.com" \
  --event-filters="methodName=storage.objects.create" \
  --service-account=$PROJECT_NUMBER-compute@developer.gserviceaccount.com

On the other hand, if you want to match a dual-region or a multi-region bucket such as eu, you will create the trigger with the global location as Audit Logs triggers only support a single or global region. Note that this will match all buckets in all regions globally:

  gcloud eventarc triggers create trigger-storage \
  --destination-run-service=hello \
  --destination-run-region=us-central1 \
  --location=global \
  --event-filters="type=google.cloud.storage.object.v1.finalized" \
  --event-filters="bucket=my-bucket-in-europe-west1-region" \
  --service-account=$PROJECT_NUMBER-compute@developer.gserviceaccount.com

As you can see from this example, if you want to read Cloud Storage events, the native Cloud Storage trigger is a much better option, but this example illustrates a typical case in which a global Audit Log trigger is necessary. 

That wraps up this closer look at locations in Eventarc. Feel free to reach out to me on Twitter @meteatamel for any questions or feedback.