Editor’s note: This blog was originally published by Siemplify on Feb. 19, 2021.

The number of unfilled cybersecurity jobs stretches into the millions, and a critical part of the problem is the length of time it takes to backfill a position.

Industry group ISACA has found that the average cybersecurity position lies vacant for up to six months. Some positions, like security analyst, are difficult to find suitable candidates for thanks to workplace challenges such as lack of management support and burnout.

As the old phrase goes, time is money. So when organizations are fortunate enough to fill a position with the appropriate talent, they want to be able to make up for lost time as quickly as possible. This is especially true for roles in the security operations center, a setting notorious for needing staff to field never-ending alerts generated by an often-disparate collection of security tools.

Training new analysts can be a daunting task. They need time to get acquainted with the SOC’s technology stack and processes. Without documentation, they often ask senior analysts for guidance. This can create distractions and consume time.

A reliance on community knowledge—undocumented, not widely-known information within an organization—creates inconsistency within the SOC that contributes to longer ramp-up times for new analysts. Undocumented processes, combined with security tools that don’t talk to each other, typically mean a SOC will need to spend nearly 100 hours—the equivalent of 2 1/2 weeks—getting a single new analyst up to speed.

Enter automation. Throughout an analyst’s career in the SOC, a security orchestration, automation, and response (SOAR) solution can be their best friend, helping expedite routine tasks and liberating them to perform more exciting work. But the technology can also allow even the most junior analysts to have an auspicious onboarding experience—hitting the ground running on day one, acclimated to their new environment, and feeling comfortable about and confident in their future.

Here are five ways a SOAR solution can, among many other activities, aid in analyst onboarding

1) The SOAR solution deploys automated playbooks

The average SOC receives large numbers of alerts per day, and many will be false positives. That amounts to a lot of dead-ends for analysts to chase and leaves little time to investigate legitimate anomalous network activity. The sheer volume of alerts has even prompted some analysts to turn off high-alert features on detection tools, potentially causing teams to miss something important.

SOAR helps analysts hurdle these roadblocks by allowing teams to create custom, automated playbooks, workflows that equalize resources and knowledge across the SOC, and help maintain consistency in the face of new hires and staff turnover. And if analysts should need to create or edit any of the steps in these playbooks, the optimal SOAR solution will enable them to do this without knowledge of specific coding or query languages, acumen that a novice analyst may lack.

2) The SOAR solution groups related alerts

As multiple alerts from different security tools are generated, some SOAR solutions allow you to automatically consolidate and group these alerts into one cohesive interface. This is what is known as taking a threat-centric approach to investigations, with the SOAR looking for contextual relationships in the alerts and, if identified, grouping these alerts into a single case. Having the ability to work more manageable and focused cases right off the bat will help ensure a smoother transition for new analysts.

3) The SOAR solution pieces together the security stack

https://www.siemplify.co/getstartedFrom next-generation firewalls to SIEM to endpoint detection and response, the security stack in any given organization can be vast and complex. No incoming analyst has reasonable time to familiarize themselves with every tool living within the stack—or to manually tap into these different tools to obtain the appropriate context to apply to alerts. A SOAR solution alleviates this challenge by delivering context-rich data that can be analyzed in one central platform, eliminating the need for multiple consoles for alert triage, investigation and remediation. Plus, with a SOAR solution, there is no need for the SOC to directly touch a detection tool that another group may manage.

4) The SOAR solution streamlines collaboration to enable easy escalation and information sharing

Often the SOC is not capable of responding to every threat, meaning other departments, such as networking, critical ops, or change management need to be involved. In addition, executive personnel are likely interested in security trends happening within the organization. Because not every group communicates in the same way—or consumes information in the same way—breakdowns can occur, and frustrations can mount, especially for a new analyst. A SOAR solution can even the playing field by automatically generating instructions, updates, or reports from the SOC to other teams, and vice versa. SOAR is also a useful solution for collaborating within the SOC team as well, especially in the age of remote and hybrid work.

5) The SOAR solution prevents analysts from quickly burning out.

There is a reason why the SOC has obtained the dubious acronym of “sleeping on chair.” Life in this environment can be a tedious, mental grind, prompting certain inhabitants to literally fall asleep from boredom.

SOAR solutions can counter this tedium in two notable ways. They can prevent analysts from having to stare at a multitude of monitors while working long shifts. They can also free analysts to work on more strategic and thought-provoking assignments, which can help improve the company’s overall security posture—and ensure a new entrant to the SOC doesn’t lose steam immediately.

To learn more about SOAR from Siemplify, now part of Google Cloud SecOps suite, including how to download the free community edition, visit siemplify.co/GetStarted.

One of the fundamental questions that retailers constantly ask themselves is how well they know their customers and how they can build deeper relationships. Knowing who their customers are can have a direct impact on improving customer engagement, which can lead to increased conversion rates, customer loyalty, lifetime value, and higher return on marketing spend. 

Retailers have started to use innovative new methods like identity graphs, which connect information from multiple sources such as internal databases, marketing systems, online interactions and social media platforms. While these new methods have shown promise, the best and most effective approach remains customer self-identification through an authentication process.

As a result, retailers require a sophisticated user identity platform that protects customer information with proper authentication and authorization, one which incorporates advanced mechanisms like multi-factor authentication and intelligent account protection, to ensure customer information is protected throughout its lifecycle. 

Google Cloud Identity Platform (GCIP) is a customer identity and access management (CIAM) solution that can enable retailers (and any other organization) to add identity and access management capabilities to online applications. GCIP is easy to administer and develop with, and is able to grow with an organization’s online user base.

With a customer identity and access management (CIAM) solution like GCIP, a retailer can quickly and easily implement powerful identity management and security capabilities such as:

• Advanced authentication through email or SMS/text messaging.
• Quicker onboarding with social logins from Google, Facebook and others.
• Flexibility to support external identity providers through identity federation.
• Support for multi-tenant B2B use cases with the ability to create unique silos of users and configurations.
• Easy to use interfaces and libraries to help web/mobile application developers quickly integrate and launch. 
• Broad authentication protocol support, including SAML 2.0, OAuth/OIDC or simple email- and password-based authentication.
• Built on Google Cloud’s planet-scale infrastructure to support a growing customer base.

How does it work?

GCIP allows end users to authenticate to web and mobile applications. For retailers, this could be their online shopping app for mobile devices or an ecommerce website accessible over the internet through a web browser.

Users can sign-in with their email/passwords, phone numbers or using their social logins . If the organization uses an existing OpenID Connect (OIDC) provider, a Security Assertion Markup Language (SAML 2.0) provider or even a custom authentication system which issues signed tokens, Google Cloud Identity platform can be used as the authentication orchestrator.

Using Google Cloud Identity Platform as the primary authentication system

GCIP can serve as the primary authentication system, meaning users are provisioned into GCIP. In this case, the end user identities are securely persisted in the “account store” database. The database is a part of fully managed GCIP services and needs no provisioning or scaling from the retailer as their end user base grows. A high level architectural overview of how this flow may look like is shown below:

Using Google Cloud Identity Platform with social logins

Google Cloud Identity Platform can also be used with popular social logins such as Google, Facebook and others. This gives users a familiar option and eliminates the need to provision users in the GCIP identity store. Below is a high-level architecture diagram showing how GCIP integrates with the social login providers.

Customizing user flows

Many organizations want to be able to customize their login flows. Sometimes, an organization wants to make data related to new user creation and user logins (barring the passwords) available to other systems. These systems can include analytical systems, recommendations or promotions engines (such as sending welcome emails and promotions upon new sign-ups,) security operations (for example, to run analysis on sign-in requests to help detect account takeover attempts,) customer master data management systems, and customer data platforms. GCIP supports blocking functions and asynchronous functions to help customize the login flows and enable syndication of the login or sign-up activity related data to external systems.

Next steps

Google Cloud Identity Platform can provide a robust, scalable, secure-by-design mechanism for managing and authenticating users to online apps. To learn more about GCIP and how it can benefit your organization, please check out the documentation.

When you choose to run your business on Google Cloud you benefit from the same planet-scale infrastructure that powers Google’s products such as Maps, YouTube, and Workspace. 

We have picked 10 ways in which Google Cloud Infrastructure services outshine alternatives in the market in how they simplify your operations, save money, and secure your data. 

1. Custom Machine Types means no wasted resources

Compute Engine offers predefined machine types that you can use when you create a VM instance. A predefined machine type has a preset number of vCPUs and a preset amount of memory; each type is billed at a set price as described on the Compute Engine pricing page. 

If predefined machine types don’t meet your needs, you can create a VM instance with a custom number of vCPUs and custom amount of memory, effectively building a custom machine type. Custom machine types are available only for general-purpose machine families. When you create a custom machine type, you are deploying a custom machine type from the E2, N2, N2D, or N1 machine family on GCP.  No other leading cloud vendor offers custom machine types so extensively.

Custom machine types are a good idea for workloads that aren’t a good fit for the predefined machine types and for workloads that require more processing power or memory but don’t need all of the upgrades provided by the next machine type level. This translates into lower operating costs.   They are also useful for controlling software licensing costs that are based on the number of underlying compute cores. 

Jeremy Lloyd, Infrastructure and Application Modernization Lead at Appsbroker, a Google partner: 

“Custom machine types coupled with Google’s StratoZone data center discovery tool provides Appsbroker with the flexibility we need to provide cost efficient virtual machines matched to a virtual machine’s actual utilization. As a result, we are able to keep our customers’ operating costs low while still providing the ability to scale as needed.”

2. Compute Engine Virtual Machines are optimized for scale-out workloads 

For scale-out workloads, T2D, the first instance type in the Tau VM family, is based on 3rd Gen AMD EPYC processors and leapfrogs VMs for scale-out workloads of any leading public cloud provider today, both in terms of performance and price-performance. Tau VMs offer 56% higher absolute performance and 42% higher price-performance compared to general-purpose VMs from any leading public cloud vendor (source). The x86 compatibility provided by these AMD EPYC processor-based VMs gives you market-leading performance improvements and cost savings, without having to port your applications to a new processor architecture. Sign up here  if you are interested in trying out T2D instances in Preview. 

For SAP HANA, Google Cloud has demonstrated with SAP how we can run the world’s largest scale-out HANA system in the public cloud (96TB).   With such innovation, you are covered as your business grows exponentially.

3. Largest single node GPU-enabled VM

Google is the only public cloud provider to offer up to 16 NVIDIA A100 GPUs in a single VM, making it possible to train very large AI models. Users can start with one NVIDIA A100 GPU and scale to 16 GPUs without configuring multiple VMs for single-node ML training, without crossing the VM layer. 

Additionally, customers can choose smaller GPU configurations—1, 2, 4 and 8 GPUs per VM—providing the flexibility to scale their workload as needed. 

The A2 VM family was designed to meet today’s most demanding applications—workloads like CUDA-enabled machine learning (ML) training and inference, for example. This family is built on the A100 GPU which offers up to 20x the compute performance compared to the previous generation GPU and comes with 40 GB of high-performance HBM2 GPU memory. To speed up multi-GPU workloads, the A2 VMs use NVIDIA’s HGX A100 systems to offer high-speed NVLink GPU-to-GPU bandwidth that delivers up to 600 GB/s. A2 VMs come with up to 96 Intel Cascade Lake vCPUs, optional Local SSD for workloads requiring faster data feeds into the GPUs and up to 100 Gbps of networking. A2 VMs provide full vNUMA transparency into the architecture of underlying GPU server platforms, enabling advanced performance tuning. Google Cloud offers these GPUs globally. 

4. ​​Non-disruptive maintenance means you worry less about planned downtime

Compute Engine offers live migration (non-disruptive maintenance) to keep your virtual machine instances running even when a host system event, such as a software or hardware update, occurs. Google’s Compute Engine live migrates your running instances to another host in the same zone without requiring your VMs to be rebooted. Live migration enables Google to perform maintenance that is integral to keeping infrastructure protected and reliable without interrupting any of your VMs. When a VM is scheduled to be live-migrated, Google provides a notification to the guest that a migration is imminent. 

Live migration keeps your instances running during:

• Regular infrastructure maintenance and upgrades
• Network and power grid maintenance in the data centers
• Failed hardware such as memory, CPU, network interface cards, disks, power, and so on. This is done on a best-effort basis; if a hardware component fails completely or otherwise prevents live migration, the VM crashes and restarts automatically and a hostError is logged.
• Host OS and BIOS upgrades
• Security-related updates
• System configuration changes, including changing the size of the host root partition, for storage of the host image and packages

Live migration does not change any attributes or properties of the VM itself. The live migration process transfers a running VM from one host machine to another host machine within the same zone. All VM properties and attributes remain unchanged, including internal and external IP addresses, instance metadata, block storage data and volumes, OS and application state, network settings, network connections, and so on. This has the benefit of reducing operational and maintenance overhead, helps you build a more robust security posture where infrastructure can be consciously revamped from a known good state and minimizes risks for advanced persistent threats. 

Refer to Lessons learned from a year of using live migration in production on Google Cloud from the Google engineering team.

5. Trusted Computing: Shielded VMs guard you against advanced, persistent attacks

Establishing trust in your environment is multifaceted, involving hardware and firmware, as well as host and guest operating systems. Unfortunately, threats like boot malware or firmware rootkits can stay undetected for a long time, and an infected virtual machine can continue to boot in a compromised state even after you’ve installed legitimate software. 

Shielded VMs can help you protect your system from attack vectors like:

• Malicious guest OS firmware, including malicious UEFI extensions
• Boot and kernel vulnerabilities in the guest OS
• Malicious insiders within your organization

To guard against these kinds of advanced persistent attacks, Shielded VMs use:

• Unified Extensible Firmware Interface (UEFI) BIOS: Helps ensure that firmware is signed and verified
• Secure and Measured Boot: Helps ensure that a VM boots an expected, healthy kernel
• Virtual Trusted Platform Module (vTPM): Establishes root-of-trust, underpins Measured Boot, and prevents exfiltration of vTPM-sealed secrets
• Integrity Monitoring: Provides tamper-evident logging, integrated with Stackdriver, to help you quickly identify and remediate changes to a known integrity state

The Google approach allows customers to deploy Shielded VMs with only a simple click, thereby easing implementation. 

6. Confidential Computing encrypts data while in use

Google Cloud was a founding member of the Confidential Computing Consortium. Along with encryption of data in transit and at rest using customer-managed encryption keys (CMEK) and customer-supplied encryption keys (CSEK), Confidential VM adds a “third pillar” to the end-to-end encryption story by encrypting data while in use. Confidential Computing uses processor-based technology that allows data to be encrypted in use while it is being processed in the public cloud. Confidential VM allows you to to encrypt memory in use on a Google Compute Engine VM by checking a single checkbox. 

All Confidential VMs support the previously mentioned Shielded VM features under the covers—you can think of Shielded VM as helping to address VM integrity, while Confidential VM addresses the memory encryption aspect which relies on CPU features. With the confidential execution environments provided by Confidential VM and AMD Secure Encrypted Virtualization (SEV), Google Cloud keeps customers’ sensitive code and other data encrypted in memory during processing. Google does not have access to the encryption keys. In addition, Confidential VM can help alleviate concerns about risk related to either dependency on Google infrastructure or Google insiders’ access to customer data in the clear. 

See what Google Cloud partners say about Confidential Computing here. 

7. Advanced networking delivers full-stack networking and security services with fast, consistent, and scalable performance

Google Cloud’s network delivers low latency, reduces operational costs and ensures business continuity, enabling organizations to seamlessly scale up or down in any region to meet business needs. Our planet-scale network uses advanced software-defined networking and security with edge caching services to deliver fast, consistent, and scalable performance. With 28 regions, 85 zones, and 146 PoPs connected by 16 subsea fiber cables around the world, Google Cloud’s network offers a full stack of layer 1 to layer 7 services for enterprises to run their workloads anywhere. Enterprises can be assured that they have best-in-class networking and security services connecting their VMs, containers, and bare metal resources in hybrid and multi-cloud environments with simplicity, visibility, and control. 

Google Cloud’s network has protected customers from one of the world’s largest DDoS attacks at 2.54 Tbps. With our multi-layer security architecture and products such as Cloud Armor, our customers ran their business with no disruptions. Furthermore, our recent integration of Cloud Armor with reCAPTCHA Enterprise adds best-in-class bot and fraud management to prevent volumetric attacks. Cloud Armor is deployed with our Cloud Load Balancer and Cloud CDN, extending the secure benefits at the network edge for traffic coming into Google Cloud so customers have security, performance, and reliability all built in. Furthermore, we are excited to offer Cloud IDS in preview, which was co-developed with security industry leader, Palo Alto Networks, to run natively in Google Cloud. 

Our advanced networking capabilities also extends to GKE and Anthos networking. With the GKE Gateway controller, customers can manage internal and external HTTPS load balancing for a GKE cluster or a fleet of GKE clusters with multi-tenancy while maintaining centralized admin policy and control. Unlike other Kubernetes offerings, we offer eBPF dataplane which brings powerful tooling such as Kubernetes network policy and logging to GKE. eBPF is known to kernel engineers as a “superpower” for its unique architecture to load and unload modules in kernel space, and now this capability is built in with Google Cloud networking. 

For observability and monitoring, our customers deploy Network Intelligence Center, Google Cloud’s comprehensive network monitoring, verification and optimization platform. With four key modules in Network Intelligence Center, and several more to come, we are working towards realizing our vision of proactive network operations that can predict and heal network failures, driven by AI/ML recommendations and remediation. Network Intelligence Center provides unmatched visibility into your network in the cloud along with proactive network verification. Centralized monitoring cuts down troubleshooting time and effort, increases network security and improves the overall user experience.  

8. Regional Persistent Disk for High Availability

Regional Persistent Disk is a storage option that provides synchronous replication of data between two zones in a region. Regional Persistent Disks can be a great building block if you need to ensure high availability of your critical applications as they offer cost-effective durable storage and replication of data between two zones in the same region. 

Regional Persistent Disks are also easy to set up within the Google Cloud Console. If you are designing robust systems or high availability services on Compute Engine, Regional Persistent Disks combined with other best practices such as backing up your data using snapshots enable you to build an infrastructure that is highly available and recoverable in a disaster. Regional Persistent Disks are also designed to work with regional managed instance groups. In the unlikely event of a zonal outage, Regional Persistent Disks allow continued I/O through failover of your workloads to another zone. Regional Persistent Disks can help meet zero RPO and near-zero RTO requirements and other stringent SLAs that your critical applications might require by maximizing application availability and protection of data during events such as host/VM failures and zonal outages. 

9. Cloud Storage’s single namespace for dual-region and multi-region means managing regional replication is incredibly simple

Similar to how Persistent Disk makes data more available by replicating data across zones, Cloud Storage provides similar benefits for object storage. Cloud Storage within a region is cross-zone by definition, reducing the risk that a zonal outage would take down your application. Cloud Storage adds to this by also providing a cross-region option that can protect against a regional outage and gets your data closer to distributed users. This comes in the form of Dual-region or Multi-region settings for a bucket. These are the simplest to implement cross-region replication offerings in the industry—just a simple button or API call to enable them. In addition to being simple to implement, they offer an added advantage of using a single bucket name that spans regions. 

This is unique in the industry. Competitive offerings currently require setting up and managing two distinct buckets, one in each region and they don’t offer the strong consistency properties Cloud Storage offers across regions. Operations and app development are burdened by this design. Google’s single namespace approach dramatically simplifies application development (the app runs on single region or dual/multi-region without any changes), and provides simpler application restarts and testing for DR.

10. Predictive autoscaling 

Customers use predictive autoscaling to improve response times for applications with long initialization times or for applications with workloads that vary predictably with daily or weekly cycles. When you enable predictive autoscaling, Compute Engine forecasts future load based on your Managed Instance Group’s history and scales out the MIG’s in advance of predicted load, so that new instances are ready to serve when the load arrives. Without predictive autoscaling, an autoscaler can only scale a group reactively, based on observed changes in load in real time. 

With predictive autoscaling enabled, the autoscaler works with real-time data as well as with historical data to cover both the current and forecasted load. Forecasts are refreshed every few minutes (faster than competing clouds) and consider daily and weekly seasonality, leading to more accurate forecasts of load patterns.

For more information, see How predictive autoscaling works and Checking if predictive autoscaling is suitable for your workload.

These are just a few examples of customer-centric innovation that set Google Cloud infrastructure apart.  Bring your applications and let the platform work for you.   

Get started by learning about your options for migration, or talk to our sales team to join the thousands of customers who have embarked upon this journey.

Acknowledgement

Special thanks to Dheeraj Konidena (Google) for contributing to this article.

Part of being a security leader whose organization is taking on a digital transformation is preparing for hard questions – and complex answers – on how to implement a transformation strategy.

In our previous CISO Survival Guide blog, we discussed how financial services organizations can more securely move to the cloud. We examined how to organize and think about the digital transformation challenges facing the highly-regulated financial services industry, including the benefits of the Organization, Operation, and Technology (OOT) approach, as well as embracing new processes like continuous delivery and required cultural shifts.

As part of Google Cloud’s commitment to shared fate, today we offer tips on how to ask the right questions that can help create the conversations that lead to better transformation outcomes for your organization. While there often is more than one right answer, a thoughtful, methodical approach to asking targeted questions and maintaining an open mind about the answers you hear back can help achieve your desired result. These questions are designed to help you figure out where to start and where to end your organization’s security transformation. By asking the following questions, CISOs and business leaders can develop a constructive, focused dialogue which can help determine the proper balance between implementing security controls and fine-tuning the risk tolerance set by the executive management and the board of directors.

To start the conversation, begin by asking:

• What defines our organization’s culture?
• How can we best integrate the culture with our security goals?

CISOs should ask business leaders:

• What makes a successful transformation?
• What are the key goals of the transformation?
• What data is (most) valuable?
• What data can be retired, reclassified, or migrated?
• What losses can we afford to take and still function?
• What is the real risk that the organization is willing to accept?

Business leaders should ask CISOs and the security team:

• What are the best practices for protecting our valuable data?
• What is the business impact of implementing those controls?
• What are the top threats that we need to address?

CISOs and business leaders should ask:

• Which threats are no longer as important?
• Where could we potentially use spending for more cost-effective controls such as firewalls and antivirus software?
• What benefits do we get from refactoring our applications?
• Are we really transforming, or lifting and shifting?
• How should we perform identity and access management to meet our business objectives?
• What are the core controls needed to ensure enterprise-level performance for the first workloads?

CISOs and risk teams should ask:

• How can we use the restructuring of an existing body of code to streamline security functions?
• How should we monitor our security posture to ensure we are aligned with our risk appetite?

Business and technical teams should ask:

• What’s our backup plan?
• What do we do if that fails?

Practical advice and the realities of operational transformation

Some organizations have been working in the cloud for more than a decade and have already addressed many operational procedures, sometimes with painful lessons learned along the way. If you’ve been operating in the cloud securely for that long, we recognize that there’s a lot to be gained from understanding your approaches to culture, operational expertise, and technology.

However, there are still many organizations that have not thought through how they will operate in a cloud environment until it’s almost ready – and at that point, it might be too late. If you can’t detail how a cloud environment will operate before its launch, how will you know who should be responsible for maintaining it?

Who are the critical stakeholders, along with those responsible for engineering and maintaining specific systems, who should be identified at the start of the transformation? There are likely several groups of stakeholders, such as those aligned with operations for transformation, and those focused on control design for cloud aligned with operations.

If you don’t have the operators involved in the design phase, you’re destined to create clever security controls with very little practical value because those tasked with day-to-day maintenance most likely won’t have the expertise or training to effectively operate these controls.

This is complicated by the fact that many organizations are struggling to recruit and retain resources with the right skills to operate in the cloud. We believe that training current employees to learn new cloud skills, and giving them the time away from other responsibilities, can help build skilled, diverse cloud security teams.

If your organization continually experiences high turnover in security leadership and skilled staff, it’s up to you to navigate your culture to ensure greater consistency. You can, of course, choose to supplement internal knowledge with trusted partners – however, that’s an expensive strategy for ongoing operational cost.

We met recently with a security organization that turns over skilled staff and leadership every two to three years. This rate of churn results in a continual resetting of security goals. This particular team joked that it’s like “Groundhog Day” as they constantly re-evaluate their best security approaches yet make no meaningful progress. This is not a model to emulate.

Many security controls fail not because they are improperly engineered, but because the people who use them – your security team – are improperly trained and insufficiently motivated. This is especially true for teams with high turnover rates and other organizational misalignments. A security control that blocks 100% of attacks might be engineered correctly, but if you can’t efficiently operate it, the effectiveness of the control will plummet to zero over time. Worse, it then becomes a liability because you incorrectly assume you have a functioning control.

In our next blog, we will highlight several proven approaches that we believe can help guide your security team through your organization’s digital transformation.

To learn more now, check out:

• Previous blog
• Podcast: CISO walks into the cloud: Frustrations, successes, lessons… and does the risk change?
• Report: CISO’s Guide to Cloud Security Transformation

Backup and DR data is a valuable business asset, and ensuring that it’s safe and accessible is essential. In particular, you want to be able to monitor your backups to ensure that the data is indeed protected and that you can quickly recover it in the event of a disaster, user error, or failed upgrade.

Users of Google Cloud’s Backup and DR service often ask:

1. What are the best practices for monitoring my backup and recovery jobs?
2. How can I proactively identify issues and troubleshoot? 
3. How can I create alerts and be notified about important events? 

The good news is that Google Cloud Backup and DR now integrates with Cloud Logging and Cloud Monitoring tools. Now, you can monitor backup events, jobs, appliance health, resource consumption and user actions in the same way you monitor other Google Cloud workloads and services. 

With this integration, you can now:

• Monitor events related to backup and restore – With detailed Google Cloud Backup and DR service event logs in Cloud Logging, you can now use log based alerts to create a customisable notification, in near-real time. This lets you monitor important events such as  backup job failures, jobs that did not run, local backup storage saturation, and network failures.
• Configure fine grained alerting – You can write custom event queries on a wide variety of dimensions such as event severity, application type, application name, job type, job name, error message and many more. This gives a lot of flexibility to define alerts for specific events or conditions within a system, thus reducing noise and helping you identify and fix issues easily. 
• Get notified on your preferred notification channel – Google Cloud offers seven pre-built notification channels that let you receive customisable notifications directly over email, SMS, Slack or the Google Cloud mobile app. Alternatively, you can integrate with your own monitoring and event management tools using webhooks or Pub/Sub. 
• Derive useful insights to troubleshoot issues – With Log Analytics, you can use BigQuery to query your data using SQL queries and generate operational insights, which can help you reduce time spent troubleshooting. For example, you can analyze the key reasons that backup and restore jobs fail for a given application or application type. 

The integration also lets you proactively discover deeper issues related to backup and recovery, such as capacity-related issues or certain job failures that happen periodically, by monitoring recurring events in the logs over time. With log-based metrics, you can: 

• Create trend charts to track important metrics such as the number of backup or restore jobs that failed during a day/week/month 
• Receive a notification when the number of occurrences crosses a threshold, for example receive a notification if a snapshot pool saturates more than five times a week 
• Monitor trends in data, such as latency values in logs, and receive a notification if the values change in an unacceptable way.

Getting started

The time to learn that your backup failed should never be when you go to restore. By integrating our Backup and DR service with Cloud Monitoring and Cloud Logging, you can get valuable assurances about the health of your backup and business continuity processes with the same tools that you use to manage other Google Cloud workloads. To get started, check out the Backup and DR event logs page. You can also watch a video that shows you how to set up and use the service and configure some custom alerts.