Editor’s note: Today we are hearing from Jono MacDougall , Principal Software Engineer at Ravelin. Ravelin delivers market-leading online fraud detection and payment acceptance solutions for online retailers. To help us meet the scaling, throughput, and latency demands of our growing roster of large-scale clients, we migrated to Google Cloud and its suite of managed services, including Cloud Bigtable, the scalable NoSQL database for large workloads.

As a fraud detection company for online retailers, each new client brings new data that must be kept in a secure manner and new financial transactions to analyze. This means our data infrastructure must be highly scalable and constantly maintain low latency. Our goal is to bring these new organizations on quickly without interrupting their business. We help our clients with checkout flows, so we need latencies that won’t interrupt that process—a critical concern in the booming online retail sector. 

We like Cloud Bigtable because it can quickly and securely ingest and process a high volume of data. Our software accesses data in Bigtable every time it makes a fraud decision. When a client’s customer places an order, we need to process their full history and as much data as possible about that customer in order to detect fraud, all while keeping their data secure. Bigtable excels at accessing and processing that data in a short time window. With a customer key, we can quickly access data, bring it into our feature extraction process, and generate features for our models and rules. The data stays encrypted at rest in Bigtable, which keeps us and our customers safe.

Bigtable also lets us present customer profiles in our dashboard to our client, so that if we make a fraud decision, our clients can confirm the fraud using the same data source we use.

We have configured our bigtable clusters to only be accessible within our private network and have restricted our pods access to it using targeted service accounts. This way the majority of our code does not have access to bigtable and only the bits that do the reading and writing have those privileges.

We also use Bigtable for debugging, logging, and tracing, because we have spare capacity and it’s a fast, convenient location. 

We conduct load testings against Bigtable.  We started at a low rate of ~10 Bigtable requests per second and we peaked at ~167000 mixed read and write requests per second  at absolute peak. The only intervention that was done to achieve this was pressing a single button to increase the number of nodes in the database. No other changes were made.

In terms of real traffic to our production system, we have seen ~22,000 req/s (combined read/write) on Bigtable in our live environment as a peak within the last 6 weeks.

Migrating seamlessly to Google Cloud 

Like many startups, we started with Postgres, since it was easy and it was what we knew, but we quickly realized that scaling would be a challenge, and we didn’t want to manage enormous Postgres instances. We looked for a kind of key value store, because we weren’t doing crazy JOINS or complex WHERE clauses. We wanted to provide a customer ID and get everything we knew about it, and that’s where key value really shines.  

I used Cassandra at a previous company, but we had to hire several people just for that chore. At Ravelin we wanted to move to managed services and save ourselves that headache. We were already heavy users and fans of BigQuery, Google Cloud’s serverless, scalable data warehouse, and we also wanted to start using Kubernetes. This was five years ago, and though quite a few providers offer Kubernetes services now, we still see Google Cloud at the top of that stack with Google Kubernetes Engine (GKE). We also like Bigtable’s versioning capability that helped with a use case involving upserts. All of these features helped us choose Bigtable.

Migrations can be intimidating, especially in retail where downtime isn’t an option. We were migrating not just from Postgres to Bigtable, but also from AWS to Google Cloud. To prepare, we ran in AWS like always, but at the same time we set up a queue at our API level to mirror every request over to Google Cloud. We looked at those requests to see if any were failing, and confirmed if the results and response times were the same as in AWS. We did that for a month, fine tuning along the way. 

Then we took the big step and flipped a config flag and it was 100% over to Google Cloud. At the exact same time, we flipped the queue over to AWS so that we could still send traffic into our legacy environment. That way, if anything went wrong, we could fail back without missing data. We ran like that for about a month, and we never had to fail back. In the end, we pulled off a seamless, issue-free online migration to Google Cloud.

Flexing Bigtable’s features

For our database structure, we originally had everything spread across rows, and we’d use a hash of a customer ID as a prefix. Then we could scan each record of history, such as orders or transactions. But eventually we got customers that were too big, where the scanning wasn’t fast enough. So we switched and put all of the customer data into one row and the history into columns. Then each cell was a different record, order, payment method, or transaction. Now, we can quickly look up the one row and get all the necessary details of that customer. Some of our clients send us test customers who place an order, say, every minute, and that quickly becomes problematic if you want to pull out enormous amounts of data without any limits on your row size. The garbage collection feature makes it easy to clean up big customers.  

We also use Bigtable replication to increase reliability, atomicity, and consistency. We need strong consistency guarantees within the context of a single request to our API since we make multiple bigtable requests within that scope. So within a request we always hit the same replica of Bigtable and if we have a failure, we retry the whole request. That allows us to make use of the replica and some of the consistency guarantees, a nice little trade-off where we can choose where we want our consistency to live.https://www.youtube.com/embed/0-eH5u7rrQQ?enablejsapi=1&

We also use BigQuery with Bigtable for training on customer records or queries with complicated WHERE clauses. We put the data in Bigtable, and also asynchronously in BigQuery using streaming inserts, which allows our data scientists to query it in every way you can imagine, build models, and investigate patterns and not worry about query engine limitations. Since our Bigtable production cluster is completely separate, doing a query on BigQuery has no impact on our response times. When we were on Postgres many years ago, it was used for both analysis and real time traffic and it was not the optimal solution for us. We also use Elasticsearch for powering text searches for our dashboard.

If you’re using Bigtable, we recommend three features:

• Key visualizer. If we get latency or errors coming back from Bigtable, we look at the key visualizer first. We may have a hotkey or a wide row, and the visualizer will alert us and provide the exact key range where the key lives, or the row in question. Then we can go in and fix it at that level. It’s useful to know how your data is hitting Bigtable and if you’re using any anti-patterns or if your clients have changed their traffic pattern that exacerbated some issue.
• Garbage collection. We can prevent big row issues by putting size limits in place with the garbage collection policies.  
• Cell versioning. Bigtable has a 3d array, with rows, columns, and cells, which are all the different versions. You can make use of the versioning to get history of a particular value or to build a time series within one row. Getting a single row is very fast in Bigtable so as long as you can keep the data volume in check for that row, making use of cell versions is a very powerful and fast option. There are patterns in the docs that are quite useful and not immediately obvious. For example, one trick is to reverse your timestamps (MAXINT64 – now) so instead of the latest version, you can get the oldest version effectively reversing the cell version sorting if you need it.

Google Cloud and Bigtable help us meet the low-latency demands of the growing online retail sector, with speed and easy integration with other Google Cloud services like BigQuery. With their managed services, we freed up time to focus on innovations and meet the needs of bigger and bigger customers. 

Learn more about Ravelin and Bigtable, and check out our recent blog, How BIG is Cloud Bigtable?

What’s the most difficult question a security operations team can face? For some, is it, “Who is trying to attacks us?” Or perhaps, “Which cyberattacks can we detect?” How do teams know when they have enough information to make the “right” decision? Metrics can help inform our responses to those questions and more, but how can we tell which metrics are the best ones to rely on during mission-critical or business-critical crises?

As we discussed in our blogs, “Achieving Autonomic Security Operations: Reducing toil” and “Achieving Autonomic Security Operations: Automation as a Force Multiplier,” your Security Operations Center (SOC) can learn a lot from what IT operations discovered during the Site Reliability Engineering (SRE) revolution. In this post, we discuss how those lessons apply to your SOC, and center them on another SRE principle—Service Level Objectives (SLOs).

Even though industry definitions can vary for these terms, SLI, SLO, and SLA have specific meanings, wrote the authors of the Service Level Objectives chapter in our e-book, “Site Reliability Engineering: How Google runs production systems.” (All subsequent quotes come from the SLO chapter of the book, which we’ll refer to as the “SRE book.”)

• SLI: “An SLI is a service level indicator—a carefully defined quantitative measure of some aspect of the level of service that is provided.”
• SLO: “An SLO is a service level objective: a target value or range of values for a service level that is measured by an SLI.”
• SLA: An SLA is a Service Level Agreement about the above: “an explicit or implicit contract with your users that includes consequences of meeting (or missing) the SLOs they contain.”

In practice, we measure something (SLI) and we set the target value (SLO); we may also have an agreement about it (SLA).

This is not about cliches like “what gets measured gets done” here, but metrics and SLIs/SLOs will to a large extent determine the fate of your SOC. For example, SOCs (including at some Managed Security Service Providers) that obsessively focus on “time to address the alert” end up reducing their security effectiveness while making things go “whoosh” fast. If you equate mean time to detect or discover (MTTD) with “time to address the alert” and then push the analyst to shorten this time, attackers gain an advantage while defenders miss things and lose.

How to choose which metrics to track

One view of metrics would be that “whatever sounds bad” (such as attacks per second or incidents per employee) needs to be minimized, while “whatever sounds good” (such as successes, reliability, or uptime) needs to be maximized.

But the SRE experience is that sometimes good metrics have an optimum level, and yes, even reliability (and maybe even security). The book’s authors, Chris Jones, John Wilkes, and Niall Murphy with Cody Smith, cite an example of a service that defied common wisdom and was too reliable.

“Its high reliability provided a false sense of security because the services could not function appropriately when the service was unavailable, however rarely that occurred… SRE makes sure that global service meets, but does not significantly exceed, its service level objective,” they wrote.

The SOC lesson here is that some security metrics have optimum value. The above-mentioned time to detect has an optimum for your organization. Another example is the number of phishing incidents, which may in fact have an optimum value. If nobody phishes you, it’s probably because they already have credentialed access to many of your systems – so in your SOC, think of SLI optimums, and don’t automatically assume zero or infinite targets for metrics.

Three specific quotes from the SRE book remind us that “good metrics” may need to be balanced with other metrics, rather than blindly pushed up:

• “User-facing serving systems generally care about availability, latency, and throughput.”
• “Storage systems often emphasize latency, availability, and durability.”
• “Big data systems, such as data processing pipelines, tend to care about throughput and end-to-end latency.”

In a SOC, this may mean that you can detect threats quickly, review all context related to an incident, and perform deep threat research—but the results may differ for various threats. A fourth guidepost explains why your SOC should care even about this: “Whether or not a particular service has an SLA, it’s valuable to define SLIs and SLOs and use them to manage the service.” Indeed, we agree that SLIs and SLOs matter more for your SOC than any SLAs or other agreements.

Metrics matter, but so does flexibility

When considering the list of most difficult questions a security operations team can face, it’s vital to understand how to evaluate metrics to reach accurate answers. Consider another insight from the book: “Most metrics are better thought of as distributions rather than averages.”

If the average alert response is 20 minutes, does that mean that “all alerts are addressed in 18 to 22 minutes,” or that “all alerts are addressed in five minutes, while one alert is addressed in six hours?” Those different answers point to very different operational environments.

What we’ve seen before in SOCs is that a single outlier event is probably the one that matters most. As the authors put it, “The higher the variance in response times, the more the typical user experience is affected by long-tail behavior.” So, in security land, that one alert that took six hours to respond to was likely related to the most dangerous activity detected.

To address this, the book advises, “Using percentiles for indicators allows you to consider the shape of the distribution.” Google detection teams track the 5% and 95% values, not just averages.

Another useful concept from SRE is the “error budget,” a rate at which the SLOs can be missed, and tracked on a daily or weekly basis. It’s a SLO for meeting other SLOs.

The SOC value here may not be immediately obvious, but it’s vital to understanding the unique role security occupies in technology. In security, metrics can be a distraction because the real game is about preventing the threat actor from achieving their objectives. Based on our own experiences, most blue teams would rather miss the SLO and catch the threat in their environment. The defenders win when the attacker loses, not when the defenders “comply with a SLA.” The concept of the error budget might be your best friend here.

The SRE book takes that line of thinking even further. “It’s both unrealistic and undesirable to insist that SLOs will be met 100% of the time: doing so can reduce the rate of innovation and deployment.”

More broadly, and as we said in our recent paper with Deloitte on SOCs, rigid obeisance is its own vulnerability to exploit. “This adherence to process and lack of ability for the SOC to think critically and creativity provides potential attackers with another opportunity to successfully exploit a vulnerability within the environment, no matter how well planned the supporting processes are.”

To be successful at defending their organizations, SOCs must be less like the unbending oak and more like the pliant but resilient willow.

Track metrics but stay focused on threats

A third interesting puzzle from our SRE brethren: “Don’t pick a target based on current performance.”

We all want to get better at what we do, so choosing a target goal for improvement based on our existing performance can’t be bad, right? It turns out, however, that choosing a goal that sets up unrealistic or otherwise unhelpful, or woefully insufficient, expectations can do more harm than good.

Here is an example: An analyst handles 30 alerts a day (per their SLI), and their manager wants to improve by 15% so they set the SLO to 35 alerts a day. But how many alerts are there? Leaving aside the question of whether it is the right SLI for your SOC, what if you have 5,000 alerts, and you drop 4,970 of them on the floor. When you “improve,” you still drop 4,965 on the floor. Is this a good SLO? No, you need to hire, automate, filter, tune, or change other things in your SOC, not set better SLO targets that seemingly improve upon today’s numbers.

To this, our SRE peers say: “As a result, we’ve sometimes found that working from desired objectives backward to specific indicators works better than choosing indicators and then coming up with targets… Start by thinking about (or finding out!) what your users care about, not what you can measure.”

In the SOC, this probably means start with threat models and use cases, not the current alert pipeline performance.

SOC guidance can sometimes be more cryptic than we’ve let on. One challenging question is determining how many metrics we really need in a typical SOC. SREs wax philosophical here: “Choose just enough SLOs to provide good coverage of your system’s attributes.”

In our experience, we haven’t seen teams succeed with more than 10 metrics, and we haven’t seen people describe and optimize SOC performance with fewer than 3. However, SREs offer a helpful, succinct test: “If you can’t ever win a conversation about priorities by quoting a particular SLO, it’s probably not worth having that SLO.”

SLOs will get to define your SOC, so define them the way you want your SOC to be, the book advises. “It’s better to start with a loose target that you tighten than to choose an overly strict target that has to be relaxed when you discover it’s unattainable. SLOs can—and should—be a major driver in prioritizing work for SREs and product developers, because they reflect what users care about.”

Importantly, make SLOs for your SOC transparent within the company. As the SREs say, “Publishing SLOs sets expectations for system behavior.” The benefit is that nobody can blame you for non-performance if you perform to those agreed upon SLOs.

Finally, here are some examples of metrics from our teams at Google. In addition to reviewing all escalated alerts, they collect and review weekly:

• event volume
• event source counts
• pipeline latency
• triage time median
• triage time at 95%

Analyzing these metrics can reveal useful guidance for applying SRE principles and ideas with their detection and response teams.

Event volume: What we need to know here is what is driving the volume. Is the event volume normal, high, or low—and why? Was there a flood of messages? New data source causing high volume? What caused it? Any bad signals? Or is there a problematic area of the business that needs strategic follow-up to implement additional controls?

Event source count: Are there signals or automation that’s behaving abnormally? Is there new automation that’s misbehaving? Counting events for each source call makes for a decent SLI.

Pipeline latency: Here at Google, we aim for a confirmed detection within an hour of an event being generated. The aspirational time is 5 minutes. This means that the event pipeline latency is something that must be tracked very diligently. This also means that we must scrutinize automation latency. To achieve this, we try to remove self-caused latency so that we’re not hiding the pain of bad signals or bad automation.

We triage median and 95p time: We track the response time to events. As the SRE book points out, tracking only a single average number can get you in trouble very quickly. Note that triage time is not the same as time to resolution, but more of a dwell time for an attacker before they are discovered.

Incident resolution times: When you have a SLI but not a SLO, this can be the proverbial elephant in the room and create all sorts of bad incentives to “go fast” instead of “go good.” Specifically, SLO without SLI causes harm from encouraging the analysis to resolve quickly and potentially increase the risk of missing serious security incidents, especially when subtle signals are involved.

When reviewing alert escalations, we look to determine if the analysis is deep enough, if handoffs contain the right information for our response teams, and to get a sense of analyst fatigue. If analysts are phoning in their notes, it’s a sign that they’re over a particular signal or that there are a ton of duplicate incidents and we need to drive the business in some way.

By measuring these and other factors, metrics allow us to drive down the cost of each detection. Ultimately, this can help our detection and response operation scale faster than the threats.

Related posts:

• “Achieving Autonomic Security Operations: Automation as a Force Multiplier”
• “Achieving Autonomic Security Operations: Reducing toil”
• “Taking an autonomic approach to security operations” video
• “New Paper: “Future Of The SOC: Process Consistency and Creativity: a Delicate Balance” (Paper 3 of 4)”
• “New Paper: “Autonomic Security Operations — 10X Transformation of the Security Operations Center””
• “EP75 How We Scale Detection and Response at Google: Automation, Metrics, Toil” podcast episode

Over the past decade, cybersecurity has posed an increasing risk for organizations. In fact, cyber incidents topped the recent Allianz Risk Barometer for only the second time in the survey’s history. The challenges in combating these risks only continue to grow. Adversaries tend to be agile and are consistently looking for new ways to land within your digital environments. They also drive attack vectors that work, which means enterprise risk leaders are now forced to look for new ways of securing infrastructure and data.

Cloud comes of age in the modern day cybersecurity threat landscape

When cloud, in its various delivery models, was first introduced, it didn’t fit neatly into the security frameworks that had seemingly protected networks for many decades. Public cloud was the answer to ongoing IT challenges: scale, resources, security capabilities, and budget cycle limitations. Now, public cloud is meeting the increasing challenge of implementing cybersecurity controls and frameworks that are capable of protecting today’s global enterprise.

Cloud adoption – with all its scale and redistribution of longstanding security paradigms – is the optimal choice for infrastructure and security, particularly as organizations grapple with the need to engage in digital transformation. We assert that successful digital transformation is impossible without incorporating the use of the scale, security architecture, and resiliency of the cloud.

Consequently, cloud adoption becomes a necessary component of roadmap discussions and planning as your organization looks to reduce overall risk. Risk leaders and enterprise cybersecurity leaders must consider that moving data, digital processes, and priority workloads to the public cloud is a crucial step for meeting the current and future digital needs of the enterprise. Going forward, this digital transformation increasingly will include hybrid infrastructure environments composed of a combination of on-premises and cloud solutions.

Pinpointing where threats thrive

As digital environments become more complex within a given organization, proactively countering adversaries becomes all the more difficult. It’s harder to implement, scale, and adhere to existing security and control frameworks. It’s also increasingly challenging to apply framework guidance to new applications, build and support infrastructure within a secure foundation, and maintain good cyber hygiene through the digital lifecycle.

As reported by TechTarget, the 2020 hack of the SolarWinds Orion IT performance monitoring system is a prime example. It grabbed headlines “not because a single company was breached, but because it triggered a much larger software supply chain incident.” This vulnerability in popular, commercially available, and widely utilized software compromised the data, networks, and systems of thousands of companies when a routine software update turned out to be backdoor malware.

A close look at the root problems behind high-profile security breaches reveals that it’s a lack of agility and an inability to scale resources that prohibit the modern security organization’s ability to respond quickly enough to counter new challenges. Look even closer and you’ll often find an insufficient implementation of best practices and ineffective solutions, leaving an organization continually chasing the next tool or solution and scrambling to stay ahead of emerging threats.

While the cost to individual businesses is high, most organizations struggle with the needed skills and resources to rigorously maintain data security basics and ensure readiness for inevitable attacks. The previous sentence is especially true when you consider that maintaining an effective state of cybersecurity readiness is a costly practice that requires the continual development of expertise, the evaluation of new tools, and an ongoing element of vigilance.

Threat visibility is a big part of the problem. You can’t protect your company from what you can’t see. For individual enterprises – with critical data workloads housed in a combination of on-premises servers, a variety of endpoints, and both private and public cloud instances – staying ahead in the ongoing battle requires a new approach.

The identification of actionable alerts and other data contributes to a better overall state of readiness. Thought leadership and discussions related to Autonomic Security Operations provide a promising outlook for security organizations willing to lean into the changing technology landscape – a landscape that now benefits from leveraging automation and machine learning currently used in security stacks. Reducing the chance of introducing vulnerabilities or missing-critical alerts starts with ensuring full visibility into an increasingly expanding and complex environment.

The evolution of a shared responsibility to a shared fate

Industry megatrends are driving cloud adoption and with it a path to improved cybersecurity. Among these trends is the concept of shared fate as an evolution of the historical shared-responsibility model. Shared fate drives a flywheel of increasing trust which develops as more enterprises transition to the cloud. This compels an even higher security investment and a more vested interest from cloud service providers.

At Google Cloud, shared fate means we take an active stake in our customers’ security posture, offering capabilities and defaults that help ensure secure deployments and configurations in the public cloud. We also offer experience-based guidance on how to configure cloud workloads for security, and can assist with risk management, reduction, and transfer.

The Google Cloud Risk Protection Program represents the continuing evolution of the shared-fate model. The program offers a practical solution that provides the modern enterprise a snapshot comparison of its current security state against well-adopted cloud-security frameworks. It also give you an opportunity to explore cyber insurance designed to meet your needs from our partners Allianz and Munich Re.

When performed with diligence, cloud adoption can help increase your overall cybersecurity effectiveness. Using a hybrid approach – and steadily reducing which data assets remain on premises – can strengthen your overall security posture and reduce risks to the organization.

Cloud security and the ability to reduce risk

In comparison to the enterprise-by-enterprise security scramble to protect data and workloads in individual private clouds, global public cloud solutions like Google Cloud can be a force multiplier when adhering to established best practices. By that, we mean, quite literally, that you get more security at every touchpoint – from infrastructure and software to access and data security.

Strong security in the public cloud starts with the foundational pieces: the hardware and design elements. At Google, for example, we take a security-by-design approach within both the data center and purpose-built components themselves. Within Google Cloud, data is encrypted by default – both at rest and in transit. Google’s baseline security architecture adheres to the zero trust principles, meaning that every network, device, person, or service initially cannot be trusted.

Embarking on a zero trust architecture journey gives modern security practitioners the ability to methodically shut down traditional attack vectors. Zero trust also provides more granular visibility and control of rapidly expanding environments. The recent emphasis of its benefits, as the U.S. White House set forth through an executive order on increasing cybersecurity resilience, is an example of the wide-scale recognition by both government and industry on the benefits of this approach.

Since adopting a zero trust approach more than a decade ago, Google has achieved a recognizable level of maturity, reflected by our internal infrastructure and multiple enterprise offerings, enabling different aspects of the zero trust security journey.

Compliance and privacy drive critical elements of the cloud adoption cycle

Privacy frameworks, regulatory compliance, and data sovereignty are driving critical elements of the cloud adoption cycle. Cloud providers must ensure they have the necessary controls, attestations, and abilities to audit in order to provide organizations with the tools to preemptively satisfy regulatory and compliance mandates across the globe.

Now consistently expected to be part of a design feature that’s built into the cloud journey, it cannot simply be an add-on capability. The direction of this evolution promises to play more of a role in the future of cloud adoption, not less. Because this is an ongoing component of enterprise risk evaluation, your business must consider cloud providers that can partner on this critical aspect of the journey – and not leave you without the resources to respond to this growing critical need.

Building trust into your digital transformation journey

Digital transformation is difficult because the modern enterprise must build and design for both today and tomorrow. From a security perspective, the challenge has often been that security industry practitioners cannot always predict what the future will look like. That said, there are clear steps you can take to mitigate all-around risk throughout the process.

How you approach the cloud is, of course, integral to your journey, but it doesn’t need to be an all-or-nothing proposition. And although technology debt continues to persist with legacy systems, that doesn’t mean you shouldn’t begin to move forward.

Google Cloud enables you to modernize at your own pace and understand what’s realistic. We recommend you move what data you can to a more secure public cloud today, followed by a phased approach to move more in the months and years that follow. The key tenets of our approach to security in the public cloud include:

• The security-by-design posture of Google Cloud can help modern-day enterprises scale security capabilities and reduce risk with an architecture built on zero trust principles.
• The Google Cloud approach to security and resiliency includes a framework to help you protect against adverse cyber events by using our comprehensive suite of solutions.
• Google Cloud can help ensure your organization adheres to the requirements of a growing and increasingly complex regulatory and compliance environment.
• The ideal model of a future organization is one where cloud plays a major role in infrastructure design and architecture. Your organization should begin to view public cloud as an enabler of the business and a core component of digital transformation.

As you transition more data to the public cloud, it’s paramount that trust is ingrained in every step you take with your cloud service provider. Many service providers readily take on a shared responsibility with your organization when it comes to security. At Google, we take it several steps further with our shared fate model to help ensure data security in the public cloud. Your future and our’s are part of the same data security journey.

All over the world, businesses, schools and users depend on G Suite to help them stay connected and get work done. Google designs, builds, and operates our products on a secure foundation, aimed at thwarting attacks and providing the protections needed to keep you safe. G Suite and Google Meet are no exception.

Google Meet’s security controls are turned on by default, so that in most cases, organizations and users won’t have to do a thing to ensure the right protections are in place. Here, we’ll summarize the key capabilities of Google Meet that help protect you.

Proactive protections to combat abuse and block hijacking attempts

Google Meet employs an array of counter-abuse protections to keep your meetings safe. These include anti-hijacking measures for both web meetings and dial-ins. 

Google Meet makes it difficult to programatically brute force meeting IDs (this is when a malicious individual attempts to guess the ID of a meeting and make an unauthorized attempt to join it) by using codes that are 10 characters long, with 25 characters in the set. We limit the ability of external participants to join a meeting more than 15 minutes in advance, reducing the window in which a brute force attack can even be attempted. External participants cannot join meetings unless they’re on the calendar invite or have been invited by in-domain participants. Otherwise, they must request to join the meeting, and their request must be accepted by a member of the host organization. 

In addition, we’re rolling out several features to help schools keep meetings safe and improve the remote learning experiences for teachers and students, including:

• Only meeting creators and calendar owners can mute or remove other participants. This ensures that instructors can’t be removed or muted by student participants.
• Only meeting creators and calendar owners can approve requests to join made by external participants. This means that students can’t allow external participants to join via video, and that external participants can’t join before the instructor.
• Meeting participants can’t rejoin nicknamed meetings once the final participant has left. This means if the instructor is the last person to leave a nicknamed meeting, students can’t join later without the instructor present.

Secure deployment and access controls for admins and end-users

To limit the attack surface and eliminate the need to push out frequent security patches, Google Meet works entirely in your browser. This means we do not require or ask for any plugins or software to be installed if you use Chrome, Firefox, Safari, or Microsoft Edge. On mobile, we recommend that you install the Google Meet app. 

To help ensure that only authorized users administer and access Meet services, we support multiple 2-Step Verification options for accounts that are secure and convenient. These include hardware and phone-based security keys and Google prompt. Additionally, Google Meet users can enroll their account in our Advanced Protection Program (APP), which provides our strongest protections available against phishing and account hijacking and is specifically designed for the highest-risk accounts. 

For G Suite Enterprise and G Suite for Education customers, we offer Access Transparency, which logs any Google access to Google Meet recordings stored in Drive, along with the reason for the access (support team actions that you might have requested, for example). Customers can also use data regions functionality to store select/covered data of Google Meet recordings in specific regions (i.e. US or Europe). 

Secure, compliant, and reliable meeting infrastructure 

In Google Meet, all data is encrypted in transit by default between the client and Google for video meetings on a web browser, on the Android and iOS apps, and in meeting rooms with Google meeting room hardware. Meet adheres to IETF security standards for Datagram Transport Layer Security (DTLS) and Secure Real-time Transport Protocol (SRTP). For every person and for every meeting, Meet generates a unique encryption key, which only lives as long as the meeting, is never stored to disk, and is transmitted in an encrypted and secured RPC (remote procedure call) during the meeting setup.

Security is an integral part of all our operations at Google. Our team of full-time security and privacy professionals supports our software engineering and operations to ensure that security is always a part of how we build and run our services. All of our Google Cloud and G Suite customers benefit from these capabilities, including: 

• Secure-by-design infrastructure: Google Meet benefits from Google Cloud’s  defense-in-depth approach to security, which utilizes the built-in protections and global-private network that Google uses to secure your information and safeguard your privacy.
• Compliance certifications: Our Google Cloud products, including Google Meet, regularly undergo independent verification of their security, privacy, and compliance controls, including validation against standards such as SOC, ISO/IEC 27001/17/18, HITRUST, and FedRAMP. We support your compliance requirements around regulations such as GDPR and HIPAA, as well as COPPA and FERPA for education. 
• Incident management: We have a rigorous process for managing data and security incidents that specifies actions, escalations, mitigation, resolution, and notification of any potential incidents impacting customer data.
• Reliability: Google’s network is engineered to accommodate peak demand and handle future growth. Our network is resilient and engineered to accommodate the increased activity we’ve seen on Google Meet.
• Transparency: At Google Cloud, we’re clear about our commitments regarding customer data: we process customer data according to your instructions; we never use customer data for advertising purposes; and we publish the locations of our Google data centers, which are highly available, resilient, and secure.  

During COVID-19 and beyond, we will continue to protect Google Meet users and their data, and keep innovating with new features to make our tools helpful, secure, and safe.

We created Traffic Director to bring to you a fully managed service mesh product that includes load balancing, traffic management and service discovery. And now, we’re happy to announce the availability of a fully-managed zero-trust security solution using Traffic Director with Google Kubernetes Engine (GKE) and Certificate Authority (CA) Service.

When platform administrators and security professionals think about modernizing their applications with a forward-looking security posture, they look for “zero-trust” security. This security posture is based on few fundamental blocks:

1. A means of allocating and asserting service identity (for example, using X.509 certificates)
2. Mutual authentication (mTLS) or server authentication (TLS)
3. Encryption for all traffic flows (TLS encryption)
4. Authorization checks and minimal privileges
5. Infrastructure to make all of the above manageable and reliable

Traffic Director does this by integrating with CA Service, a highly available private CA which issues private certificates expressing service identities, and provides a managed mTLS certificate infrastructure with full certificate lifecycle management. Together, these solve both certificate issuance and CA rotation complexities. 

With Traffic Director managing your service-to-service security, you can now enjoy end-to-end encryption, service-level authentication and granular authorization policies for your service mesh.

With this new capability, you can now:

• Implement mutual TLS (mTLS) and TLS between your services, including certificate lifecycle management. Communications within your mesh are authenticated and encrypted.
• Enable identity-based authorization, as well as authorization based on other parameters (such as the request method). These concepts underpin role-based access controls (RBAC) and enable you to take a “least privileges” stance where only authorized services can communicate with each other based on ALLOW/DENY rules.

mTLS is supported whether you’re using Envoy or proxyless gRPC for your service mesh. Authorization support for proxyless gRPC is coming later this year. Check out our documentation to learn more and get started with Envoy or proxyless gRPC.