New locations in Eventarc

Back in August, we announced more Eventarc locations (17 new regions, as well as 6 new dual-region and multi-region locations to be precise). This takes the total number of locations in Eventarc to more than 30. You can see the full list in the Eventarc locations page or by running gcloud eventarc locations list . 

What does location mean in Eventarc?

An Eventarc location usually refers to the single region that the Eventarc trigger gets created in. However, depending on the trigger type, the location can be more than a single region:

• Pub/Sub triggers only support single-region locations.
• Cloud Storage triggers support single-region, dual-region, and multi-region locations.
• Cloud Audit Logs triggers support single-region locations and the special global region.

Before looking into trigger location in more detail, let’s look at other locations relevant in Eventarc.

What other locations are relevant in Eventarc?

Triggers connect event sources to event targets:

Each event source, event target, and trigger has its own location. Sometimes, these locations have to match and sometimes they can be different.  

Here’s an example of a trigger connecting Cloud Storage events from a bucket in the europe-west1 region to a Cloud Run service in the us-central1 region with a trigger located in the europe-west1 region:

  gcloud eventarc triggers create trigger-storage \
  --destination-run-service=hello \
  --destination-run-region=us-central1 \
  --location=europe-west1 \
  --event-filters="type=google.cloud.storage.object.v1.finalized" \
  --event-filters="bucket=my-bucket-in-europe-west1-region" \
  --service-account=$PROJECT_NUMBER-compute@developer.gserviceaccount.com

In many cases, you don’t have control over the location of the event source. In the example above, the Cloud Storage bucket is in the europe-west1 region. That’s the location that you need to work with and it has implications for the trigger location (which I’ll get to later). 

The location of the event target is the region of the service where you want the events to go. You get to choose this from one of the supported regions when you deploy your Cloud Run service. You typically want this to be in the same region as your event source for latency and data locality reasons (but this is not strictly a requirement). In the example above, the event source (bucket) is in europe-west1 but the event target (Cloud Run service) is in  us-central1 as specified by the --destination-run-region flag. 

The location of the trigger is dictated by the event source location, but the trigger type also comes into play. It is specified by the –location flag. Let’s take a look at the trigger location for each trigger type in more detail.

Location in Pub/Sub triggers

In a Pub/Sub trigger, you connect a Pub/Sub topic to an event target. Pub/Sub topics are global and not tied to a single region. However, when you create a Pub/Sub trigger, you need to specify a region for it (because Eventarc triggers need to live in a region) with the --location flag as follows:

  gcloud eventarc triggers create trigger-pubsub \
  --destination-run-service=hello \
  --destination-run-region=us-central1 \
  --location=us-central1 \
  --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
  --transport-topic=projects/your-projectid/topics/your-topic

By specifying a location, Eventarc automatically configures the geofencing feature in Pub/Sub such that events only persist in the specified location. As I noted above, you typically want to (but are not required to) choose the same region for the trigger and the Cloud Run service for lower latency and data locality. You can also use regional Pub/Sub service endpoints to publish to the topic to ensure that all of the data stays in a single region. 

Location in Cloud Storage triggers

In a Cloud Storage trigger, you connect a Cloud Storage bucket to an event target. A Cloud Storage bucket can be in a single-region (e.g. europe-west1), dual-region (e.g. eur4), or multi-region (e.g. eu) location. The location of the bucket dictates the location of the trigger and they have to match. The earlier trigger example was for a bucket in the  europe-west1 single-region location. Here’s another trigger connecting Cloud Storage events from a bucket in the eu multi-region location. Notice how the location flag matches the bucket region:

  gcloud eventarc triggers create trigger-storage \
  --destination-run-service=hello \
  --destination-run-region=us-central1 \
  --location=eu \
  --event-filters="type=google.cloud.storage.object.v1.finalized" \
  --event-filters="bucket=my-bucket-in-eu-multi-region" \
  --service-account=$PROJECT_NUMBER-compute@developer.gserviceaccount.com

If the bucket region and the trigger region do not match, you’ll see an error:

  ERROR: (gcloud.eventarc.triggers.create) INVALID_ARGUMENT: The request was invalid: Bucket "my-bucket-in-eu-multi-region" location "eu" does not match trigger location "europe-west1". Try again by creating the trigger in "eu".

Location in Cloud Audit Logs triggers

In a Cloud Audit Logs trigger, you connect any event source that emits Audit Logs to an event target. The location of the event source will dictate the trigger location. This is typically a single region but there is a special global region that’s necessary in some cases. 

For example, if you want to read Cloud Storage events from a bucket in the europe-west1 region with an Audit Logs trigger, you will create the trigger with the same location. Note that this will match all buckets in the europe-west1 region as there’s no filter by bucket in Audit Logs:

  gcloud eventarc triggers create trigger-auditlog \
  --destination-run-service=hello \
  --destination-run-region=us-central1 \
  --location=europe-west1 \
  --event-filters="type=google.cloud.audit.log.v1.written" \
  --event-filters="serviceName=storage.googleapis.com" \
  --event-filters="methodName=storage.objects.create" \
  --service-account=$PROJECT_NUMBER-compute@developer.gserviceaccount.com

On the other hand, if you want to match a dual-region or a multi-region bucket such as eu, you will create the trigger with the global location as Audit Logs triggers only support a single or global region. Note that this will match all buckets in all regions globally:

  gcloud eventarc triggers create trigger-storage \
  --destination-run-service=hello \
  --destination-run-region=us-central1 \
  --location=global \
  --event-filters="type=google.cloud.storage.object.v1.finalized" \
  --event-filters="bucket=my-bucket-in-europe-west1-region" \
  --service-account=$PROJECT_NUMBER-compute@developer.gserviceaccount.com

As you can see from this example, if you want to read Cloud Storage events, the native Cloud Storage trigger is a much better option, but this example illustrates a typical case in which a global Audit Log trigger is necessary. 

That wraps up this closer look at locations in Eventarc. Feel free to reach out to me on Twitter @meteatamel for any questions or feedback.

When our founders began Mambu in 2011, their goal was to bring the latest digital technologies to the banking and finance world. Banking, in particular, is an industry built on decades of deep legacy technology. So initially, Mambu was embraced by microfinance — 100 organizations in 26 countries in just the first two years. 

Since then, acceptance of modernizing core banking services by using composable, cloud technologies has grown across financial services institutions (FSIs). We now service top-tier banks, fintech startups, and other finance organizations across six continents, helping them deliver flexible, personalized, customer-centric banking products and services that their customers can depend upon.

One of the main reasons we’ve been able to scale the Mambu composable banking platform across the globe and at our current pace is our partnership with Google Cloud. The decision to move forward with Google Cloud happened for several reasons.

1. Flexibility and openness. Many FSIs are on hybrid and multicloud technology stacks, as they may still be transitioning from legacy systems, or have data residency requirements that have led them to use different clouds in different regions. Mambu meets customers wherever they are in their cloud journeys. We support interoperability without vendor lock-in. This need for openness, as well as the scalability benefits, led Mambu to evolve our platform on Google Kubernetes Engine (GKE). Many customers use open-source Kubernetes because, this common foundation can help streamline integration, speed up time to market, and reduce development. Just as important, the Google Cloud open cloud approach matches our company values. 

2. Security and data residency. For customers in highly regulated finance industries, security isn’t just top of mind,  it’s the No. 1 requirement. In addition to Google Cloud’s secure infrastructure, external audit certifications, and encryption, its wide array of regions has allowed us to expand into more countries, where we serve banks that must meet local data residency requirements. For example, Google Cloud’s Jakarta Cloud Region, has allowed us to support Bank Jago in Indonesia as it brings more financial inclusion to the unbanked in that country.

3. Availability. It’s critical for banks to maintain basic financial functionality, like accepting deposits and serving cash, even amid a service disruption. We needed a cloud partner with impeccable redundancy, failover, and disaster recovery capabilities.

In addition to GKE, the Mambu platform uses several other Google Cloud services for specific functions, including Cloud Armor, Cloud Load Balancing, Cloud VPN, Cloud Memorystore, and Google Cloud Operations.

Another important reason we chose to partner with Google Cloud was its expansive ecosystem and commitment to innovation. We are midway into a three-year journey to modernize our own technology stack to meet customer needs.

Building a roadmap with Google Cloud

Our customers need a core banking technology platform that will grow with them as they bring to market innovative services built on the latest technology advances. For Mambu to be that platform, we need a cloud partner that supports and scales with our growth. While we originally built our cloud architecture on GKE and Compute Engine (among a few other Google Cloud services), we’re now looking to a serverless future where we can scale more easily and leverage managed services within Google Cloud and its partner ecosystem to focus on our core offerings. 

These are just some of the modernization and customer-led innovations that we’re cooking up: 

• More workloads in GKE: Like many companies, our cloud transformation is a work in progress. While much of our codebase is in GKE, we’re continuing to break up some larger pieces of code into microservices to increase agility and velocity, enabling us to make consistent updates to discrete areas of our platform without affecting the whole. GKE is the leader for orchestrating microservices at scale and continues to be a natural fit.
• Native BigQuery integration: Mambu customers collect a tremendous amount of data within the platform that can be used for analytics, personalization, and other use cases. We’re planning to create a seamless integration for feeding core banking data from Mambu into BigQuery so that customers can better leverage their valuable data.
• CloudSQL vs. self-managed MySQL: We have almost completed migrating from our own MySQL instances to managed Cloud SQL databases, which will open up new opportunities to implement customer-centric solutions such as BigQuery integration.
• A serverless future with Cloud Run: Compute Engine is working well for Mambu, providing the flexibility to choose the virtual machines that best balance performance and cost needs. As we seek even more time and cost efficiencies, we believe the elastic scalability of a serverless architecture built on Cloud Run will get us there, and we’re considering going serverless in the future. Doing so would abstract infrastructure for simpler management, while allowing us to fire up containers to meet our customers’ high transactions-per-second needs, spin them down when not needed, and pay only when they run. It would also boost security: Without long-running compute, there are no patches or fixes, and each new instance is isolated and fresh by default.

These are just a handful of examples of the ways we want to best leverage Google Cloud services to simplify how we manage our tech stack, as well as continue to bolster security, scalability, and performance. There are many other ideas we’re exploring: using Dataproc and Datastream to support the specific data needs of Islamic banking, Cloud Functions so that customers can run their own queries against Mambu, and AI-enabled features. 

At Mambu, our mission is to empower our customers to deliver great modern financial experiences easily to everyone around the world. Every time Google Cloud opens a new data center, we can enter a new market. Every time we move to a new-to-us managed service via the Google Cloud Marketplace, we free up time to build new ways to deliver customer-centric banking solutions. And so, we look forward to continuing this partnership with Google Cloud well into the future.

Enterprise agility and the ability to innovate, adapt and respond quickly to the ever-changing risk and regulatory landscape is no longer a choice, but the cornerstone of successful digital transformation and commercial growth. Traditional access to and ways of managing data invariably create challenges in dealing with multiple data repositories, reconciliations, fire-drills, etc.

In response, the move to cloud is increasing significantly. It enables risk analytics and regulatory reporting at scale in a secure environment with data storage, management and encryption capabilities as a standard. In addition, as regulatory reporting requirements become more granular, machine learning can help facilitate new insights and allow for risk management to become more embedded into operational processes.

This webinar will address the day-to-day challenges in risk management and regulatory compliance, while also exploring how technological innovations can provide massive improvements and potential.

Key themes

• Real-life data challenges in the eyes of risk managers: can compliance, fraud detection and identifying liquidity positions be improved through the use of AI?
• Innovative approaches to streamline regulatory reporting to derive deeper customer insights from data at the moment of truth.
• Reimagining operations: how to modernise the data infrastructure to accommodate data explosion, drive flexibility and deliver a more cost effective outcome.