Compute Engine is a customizable compute service that lets you create and run virtual machines on Google’s infrastructure. You can create a Virtual Machine (VM) that fits your needs. Predefined machine types are pre-built and ready-to-go configurations of VMs with specific amounts of vCPU and memory to start running apps quickly. With Custom Machine Types, you can create virtual machines with the optimal amount of CPU and memory for your workloads. This allows you to tailor your infrastructure to your workload. If requirements change, using the stop/start feature you can move your workload to a smaller or larger Custom Machine Type instance, or to a predefined configuration.

Machine types

In Compute Engine, machine types are grouped and curated by families for different workloads. You can choose from general-purpose, memory-optimized, compute-optimized and accelerator-optimized families. 

• General-purpose machines are used for Day-to-day computing at a lower cost and for balanced price/performance across a wide range of VM shapes. The use cases that best fit here are web serving, app serving, back office applications, databases, cache, media-streaming, microservices, virtual desktops, development environments.
• Memory-Optimized machine are recommended for ultra high-memory workloads such as in-memory analytics and large in-memory databases such as SAP HANA 
• Compute-Optimized machines are recommended for ultra high performance workloads such as High Performance Computing (HPC), Electronic Design Automation (EDA), gaming, video transcoding, single-threaded applications.
• Accelerator-Optimized machines are optimized for high performance computing workloads such as Machine learning (ML), Massive parallelized computations and High Performance Computing (HPC)

How does it work?

You can create a VM instance using a boot disk image, a boot disk snapshot, or a container image. The image can be a public operating system (OS) image or a custom one. Depending on where your users are you can define the zone you want the virtual machine to be created in. By default all traffic from the internet is blocked by the firewall and you can enable the HTTP(s) traffic if needed. 

Use snapshot schedules (hourly, daily, or weekly) as a best practice to back up your Compute Engine workloads. Compute Engine offers live migration by default to keep your virtual machine instances running even when software or hardware update occurs. Your running instances are migrated to another host in the same zone instead of requiring your VMs to be rebooted. 

Availability

For High Availability (HA) Compute Engine offers automatic failover to other regions or zones in event of a failure. Managed instance groups (MIGs) help keep the instances running by automatically replicating instances from a predefined image. They also provide application based autohealing health checks. If an application is not responding on a VM, the auto healer automatically recreates that VM for you. Regional MIGs let you spread app load across multiple zones. This replication protects against zonal failures. MIGs work with load balancing services to distribute traffic across all of the instances in the group. 

Compute Engine offers autoscaling to automatically add or remove VM instances from a managed instance group based on increases or decreases in load. Autoscaling lets your apps gracefully handle increases in traffic, and it reduces cost when the need for resources is lower. You define the autoscaling policy for automatic scaling based on the measured load, CPU utilization, requests per second or other metrics.

Active Assist’s new feature, predictive autoscaling, helps improve response times for your applications–When you enable predictive autoscaling, Compute Engine forecasts future load based on your Managed Instance Group’s (MIG) history and scales it out in advance of predicted load, so that new instances are ready to serve when the load arrives. Without predictive autoscaling, an autoscaler can only scale a group reactively, based on observed changes in load in real time. With predictive autoscaling enabled, the autoscaler works with real-time data as well as with historical data to cover both the current and forecasted load. That makes predictive autoscaling ideal for those apps with long initialization times and whose workloads vary predictably with daily or weekly cycles. For more information, see How predictive autoscaling works or check if predictive autoscaling is suitable for your workload, and to learn more about other intelligent features, check out Active Assist.

Pricing

You pay for what you use. But you can save cost by taking advantage of some discounts! Sustained use saving are automatic discounts applied for running instances for a significant portion of the month. If you know your usage upfront, you can take advantage of committed use discounts which can lead up to significant savings without any upfront cost. And by using short lived preemptive instances you can save up to 80%, they are great for batch jobs and fault tolerant workloads. You can also optimize resource utilization with automatic recommendations. For example if you are using a bigger instance for a workload that can run on a smaller instance you can save costs applying these recommendations.

Security

Compute Engine provides you default hardware security. Using Identity and Access Management (IAM) you just have to ensure that proper permissions are given to control access to your VM resources. All the other basic security principles apply, if the resources are not related and don’t require network communication amongst themselves, consider hosting them on different VPC networks. By default, users in a project can create persistent disks or copy images using any of the public images or any images that project members can access through IAM roles. You may want to restrict your project members so that they can create boot disks only from images that contain approved software that meet your policy or security requirements. You can define an organization policy that only allows Compute Engine VMs to be created from approved images. This can be done by using the Trusted Images Policy to enforce images that can be used in your organization. 

By default all VM families are Shielded VMs. Shielded VMs are virtual machine instances that are hardened with a set of easily configurable security features to ensure that when your VM boots, it’s running a verified bootloader and kernel — is the default for everyone using Compute Engine, at no additional charge. For more details on Shielded VMs refer to the documentation here.

For additional security, you also have the option to use Confidential VM to encrypt your data in use, while it’s being processed in Compute Engine. For more details on Confidential VM refer to the documentation here.

Use cases

There are many use cases Compute Engine can serve in addition to running websites and databases. You can also migrate your existing systems onto Google Cloud, with Migrate for Compute Engine, enabling you to run stateful workloads in the cloud within minutes rather than days or weeks. Windows, Oracle or VMware applications have solution sets enabling a smooth transition to Google Cloud. To run windows applications either bring your own license leveraging Sole-tenant nodes or using the included licenced images. 

Conclusion

Whatever your application use case may be, from legacy enterprise applications to digital native applications, Compute Engine’s families will fit it. For a more in-depth look into Compute Engine check out the documentation. 

For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev

Security operations teams are facing a “perfect storm” of challenges from nation-state actors turning their attention to financial crime, to rising uncertainty and potential complexities because of rapidly advancing cloud migration and IoT adoption, to the long-lamented skills shortage. 

Now imagine having to face this trifecta without visibility into your IT and security infrastructure because scale and cost concerns have impeded your ability to ingest and monitor critical telemetry. To help organizations rethink threat detection and broader security team efforts, we showcased at this month’s Google Cloud Security Talks a keynote on why DEI is a cybersecurity imperative, Mandiant’s review of lessons for organizations two years after SolarWinds, how Google Chrome already helps protect your business (and what more you can make do with it,) how best to encourage your organization to pursue Zero Trust, and three key ways in which Chronicle Security Operations can help your organization scale threat detection and actionable outcomes. These include:

1. Operationalizing threat intelligence

Harnessing the power of threat intelligence to acquire context, prioritize risk, and act on detections requires more than just ingesting a set of indicator of compromise (IOC) feeds. Threat intelligence must have breadth and depth of coverage to provide custom details on the tools, tactics, and procedures of threat actors. Explore why part of the “magic” behind Google Cloud’s security is the sheer scale of threat intelligence we can share with our customers. 

2. Driving detection as code

We want to enable organizations that are more advanced in their cybersecurity maturity to build their own custom detections and rules, which means offering our customers a powerful detection-authoring platform. Chronicle Security Operations can help analysts build specific detections around, for example, known bad files or a specific registry key change. We also demonstrated how to create more complex detections, such as flagging process execution patterns or crafting a suspicious-behavior trigger from a specific activity sequence.

3. Leveraging curated detections

With curated detections, which we announced in August, we are putting the power of Google Cloud’s intelligence in the hands of security operations teams everywhere. Our detections offer actionable, ready-to-use threat detection content curated, built, and maintained by Google Cloud Threat Intelligence (GCTI) researchers. These detection sets cover a wide variety of threats for your network and beyond, including attacks such as ransomware, remote-access tools (RAT), infostealers, data exfiltration, and suspicious activity.

You can watch all nine of our Security Talks recorded sessions here, and keep an eye out for our next Security Talks, coming in March 2023.

There is no one way to recover. The past year has seen unprecedented challenges for business across the world, with social distancing and quarantine measures forcing many organizations to quickly adapt to remote working. A new report from BCG Platinion, titled “Finding a New Normal in the Cloud”, points out that while companies have done well to survive so far, the overall business environment is still a tough one with contracted economies and lowered revenues. CIOs have to find the right balance between technological innovation and lowering their cash burn rate. 

The report identifies five key ways in which companies can use cloud computing to optimize their operations and reduce overall IT costs by as much as 10%. 

1. Go beyond VPN

The BCG Platinion report states that companies need “rapid, efficient, highly scalable, and device agnostic solutions”. Traditionally, when employees had to work offsite, companies provided them with VPNs. But the sheer scale of the COVID-19 pandemic has shown these solutions to be expensive, slow, inconvenient, and hard to manage when entire workforces are working remotely.

Instead, BCG Platinion emphasizes the need for cloud-based solutions such as BeyondCorp Enterprise for more secure, more effective, lower cost remote access at scale. BeyondCorp Enterprise offers customers a zero trust platform for simple and secure access with continuous end-to-end protection that can be used on any device at any time. Deliveroo, a global food delivery company headquartered in the UK, uses BeyondCorp Enterprise to bring the zero trust model to its distributed workforce. 

“Having secure access to applications and associated data is critical for our business,” says Vaughn Washington, VP of Engineering at Deliveroo. “With BeyondCorp Enterprise, we manage security at the app level, which removes the need for traditional VPNs and associated risks.”¹

With a low cost cloud subscription model, BeyondCorp Enterprise eliminates the need for hardware, operating, and maintenance costs that come with VPN solutions and can also enable organizations to offer protection to the extended workforce at a fraction of the cost. BCG Platinion estimates that solutions like BeyondCorp Enterprise can save companies as much as 50% versus the cost of a traditional VPN. 

2. Use SaaS to keep productivity up 

With so much of the workforce fragmented due to social distancing measures, the need for seamless, efficient collaboration is greater than ever. According to BCG Platinion, a fully functional Software-as-a-Service productivity solution such as Google Workspace helps to “alleviate the traditional costs and burdens around availability, backup, and maintenance of on-premise collaboration infrastructure.” BCG Platinion analysts estimate that adopting a SaaS solution can lower computing costs for end users by up to 35%. 

But working in the cloud does more than lower costs. The BCG Platinion report cites a 2020 study from Forrester that found adopting Google Workspace boosted revenue growth by 1.5%, reduced the need for on-demand tech support by 20%, and cut the risk of data breaches by more than 95%. Over the last year, companies of all sizes have looked on the conditions of the pandemic as an opportunity to change the way they work. 

“Airbus has spent the past year thinking about what it actually means to return to work and we’re looking to support greater flexibility with Google Workspace in a leading role,” says Andrew Plunkett, Airbus Vice President Digital Workplace. “In 2020, we held 5.6M Google Meet sessions and we now have more than 70,000 shared Drives where people collaborate. Google Workspace has changed the way people work at Airbus and that will continue as the solution empowers the hybrid work reality.”²

3. Reduce IT overhead and management costs with cloud-first devices

Working in the cloud can be made even more effective with devices specifically designed for the cloud, says the BCG Platinion report. “Cloud-native devices such as Google’s Chromebooks and Chromeboxes are cost-effective, easy to deploy, simple to use, and highly secure,” says BCG Platinion. Additionally, with “thin client” devices, companies can save on hardware costs compared with traditional laptops and desktops.The report suggests that a thin client approach can produce savings of up to 25% in end-user technology and support. 

Organizations and businesses of all sizes have found that Chrome OS and devices have greatly enhanced their capacity to work together in even the most challenging circumstances. Chrome OS provides employees with a modern experience and devices that stay fast, have built-in security, deploy quickly, and reduce the total cost of ownership. “Chromebooks are simple-to-use and cost-effective devices that do everything that our staff need them to do, which is mainly accessing Google Workspace online,” says Henry Lewis, Head of Platform for the London Borough of Hackney. “As soon as the Grab and Go Chromebooks were available, they were well used every day.”³

4. Lift and shift for easier transitions

While migrating to the cloud is a priority for many organizations to succeed, it does not have to happen all at once and in the same way. A total cloud migration can be a huge undertaking, involving redesigns of architecture and refactoring of applications. For many organizations, this process can take several months, even years. 

But in times like these, CIOs need to make decisions quickly and reduce cash burn as much as possible. When resources are stretched and time is tight, BCG Platinion recommends a “lift-and-shift” approach for minimal disruption. With Google Compute Engine, for example, organizations can simply rehost their existing workloads on virtual machines without transformation. BCG Platinion reports that moving non-critical workloads to the cloud with lift-and-shift approaches can reduce IT spend by as much as 4% in just three months. Additionally, a quick, effective migration paves the way for more advanced IT infrastructure changes, creating effects that ripple long into the future. 

5. Make data work for you with the cloud

Data is central to any industry and making the most out of it is more important now than ever before. With business as usual upended by the pandemic, organizations have turned to data for urgent tasks like demand forecasting or predicting supply-chain disruptions. A McKinsey report from last year points out that while many organisations had already started to engage with analytics and AI technologies, their progress was dramatically accelerated by the urgency of the pandemic: “Analytics capabilities that once might have taken these organizations months or years to build came to life in a matter of weeks.”

With the right setup a company can use data to drive efficiencies, respond quickly to its customers and open new markets. But big wins require huge amounts of information and powerful analytics, which means that effective data handling can be very difficult and expensive to do with on-premises architecture. 

Moving to a cloud-based infrastructure minimizes infrastructure costs while opening up cutting edge techniques like machine learning for greater insight. The BCG Platinion report found that using a cloud-based data platform was not only cheaper and more efficient for businesses, but could result in a 70% increase in “effectiveness” which it defined as increased sales, lower costs of procured goods, and reduced inventory holding costs. “Moving to cloud provides competitive advantage as you scale innovation, accelerating the creation of new services to keep ahead of the competition,” explains Norbert Faure, Managing Director, Platinion Western Europe at Boston Consulting Group (BCG).

The key mission of any data platform is to make sure that the right people have access to the right information at the right time. Google Cloud helps to unify data across the entire business, increase agility, and innovate faster with a range of products. BigQuery runs complex analytics at infinite scale while helping organizations save up to 34% on the total cost of ownership compared with alternative cloud-based data warehouse solutions.⁴ Cloud Spanner provides a fully managed relational database that operates at 99.999% availability. Meanwhile, with Vertex AI, businesses can access the same groundbreaking machine learning tools that Google uses itself for unprecedented insight on a unified AI platform. “The whole Google Cloud suite of products accelerates the process of getting established and up and running, so we can perform the ‘bread-and-butter’ work of data science,” says David Herberich, VP, Head of Data at fintech startup Branch.⁵

Minimize costs today, innovate for tomorrow

As the world recovers from the pandemic, continued success depends on staying nimble and adaptive, argues the BCG Platinion report. Cloud computing can make companies more resilient by helping them to keep costs low, reducing overall IT spend by as much as 10% overall according to BCG Platinion. “For CIOs, cloud migrations can offer important near-term savings and benefits, while also reinvigorating progress toward the long-term goal of digital transformation.“

To learn more, read the full report from BCG Platinion.

^{1. BeyondCorp Enterprise: Introducing a safer era of computing
2. Building the future of work with Google Workspace
3. Hackney Council: Empowering 4,000 staff to keep serving their community from home
4. The Economic Advantages of Google BigQuery versus Alternative Cloud-based EDW Solutions
5. Fintech startup, Branch makes data analytics easy with BigQuery}

In fast-moving organizations, it’s not uncommon for cloud resources, including entire projects, to occasionally be forgotten about. Not only such unattended resources can be difficult to identify, but they also tend to create a lot of headaches for product teams down the road, including unnecessary waste and security risks. 

To help you prune your idle cloud resources, we’re excited to introduce Unattended Project Recommender. It’s a new feature of Active Assist that provides you with a one-stop shop for discovering, reclaiming, and shutting down unattended projects. With actionable and automatic recommendations, you no longer have to worry about wasting money or mitigating security risks presented by your idle resources. Unattended Project Recommender uses machine learning to identify, with a high degree of confidence, projects that are likely abandoned based on API and networking activity, billing, usage of cloud services, and other signals. This feature is available via the Recommender API today, making it easy for you to integrate with your company’s existing workflow management and communication tools, or export results to a BigQuery table for custom analysis.

Thousands of projects can be unattended in large organizations, presenting major security risks

Your cloud projects can go abandoned or unattended for a number of reasons — ranging from a test environment that’s no longer needed, to project cancellation, to project owner switching jobs, and more. Not only can such projects contribute to your cloud bill (waste) but they may contain security issues such as open firewalls or privileged service account keys that attackers can exploit to get a hold of your cloud resources for cryptocurrency mining or, worse, compromise your company’s sensitive data. These security risks tend to grow over time because the latest best practices and patches are usually not applied to unattended projects. 

We experience this issue here at Google, too… In fact, it has been on Google’s internal security team’s radar for some time now, so we joined forces and looked into this problem together, starting with our very own “google.com” organization cloud projects. We quickly found some projects that were unattended, but remediating this issue was easier said than done due to challenges in several areas:

• Detection: With lots of signals available to you via sources like Cloud Monitoring, what are the right ones you should look at (e.g. API, networking, user activity)? How can you tell the difference between an unattended project and a project that has a low level of activity by design (e.g. a “shell” project that holds an auth token)?
• Remediation: Once you have identified a project that seems abandoned, how do you go about ensuring that it’s indeed an unattended project? How do you reduce the risk of deleting something that might be essential to a production workload, causing irreversible data loss? How do you solve this at the scale of your entire organization, beyond a one-time cleanup? 

Over the course of 2021 we built and tested a Google-internal prototype first, cleaning up many of our internal unattended projects, and then worked with a number of Google Cloud customers to build and tune this feature based on real-life data (thank you to all of our early adopters for working with us and your generous feedback that helped us shape this feature!) It was not uncommon for us to come across organizations with thousands of unattended projects, and we’re very excited to bring Unattended Project Recommender to all customers, in public preview.

Discovering and acting on unattended project recommendations

Unattended Project Recommender analyzes usage activity across all projects under your organization, including the following data:

• API activity (e.g. service accounts with authentication activity, API calls consumed)
• Networking activity (ingress and egress)
• Billing activity (e.g. services with billable usage)
• User activity (e.g. active project owners)
• Cloud services usage (e.g. number of active VMs, BigQuery jobs, storage requests)

Based on these signals, it can generate recommendations to clean up projects that have low usage activity (where “low usage” is defined using a machine learning model that ranks projects in your organization by level of usage), or recommendations to reclaim projects that have high usage activity but no active project owners. Here’s what an example post-processed summary list of recommendations can look like for the “foobar” organization that has 3 projects:

  Project ID: demo-project-307815
Recommendation: CLEANUP_PROJECT

Project ID: new-project
Recommendation: N/A

Project ID: bobs-playground-project
Recommendation: RECLAIM_PROJECT

In addition to the recommendations, you can also examine the underlying project activity insights that the recommendations are based upon. The insights provide additional information that can be useful for integration with your organization’s existing workflows and automation (e.g. send an auto-generated email or chat message to project owners based on the list provided by the owners field). Here’s an example insight payload:

  content:
  activeAppengineInstanceDailyCount: 0
  activeCloudsqlInstanceDailyCount: 0
  activeGceInstanceDailyCount: 3
  activeServiceAccountDailyCount: 1
  apiClientDailyCount: 18922           // Daily average API calls produced
  bigqueryInflightJobDailyCount: 0
  bigqueryInflightQueryDailyCount: 0
  bigqueryStorageDailyBytes: 0
  bigqueryTableDailyCount: 0
  consumedApiDailyCount: 0             // Daily average API calls consumed
  datastoreApiDailyCount: 0
  gcsObjectDailyCount: 11
  gcsRequestDailyCount: 0
  gcsStorageDailyBytes: 2663548
  hasActiveOauthTokens: false          // OAuth tokens used in the last 180 days
  hasBillingAccount: true
  numActiveUserOwners: 1
  owners:                              // List of project owners
  - activeOnProject: false
    member: user:user1@example.com
  - activeOnProject: true
    member: user:user2@example.com
  serviceWithBillableUsage:
  – Cloud Storage
  - Compute Engine
  vpcEgressDailyBytes: 264456938       // Daily average VPC egress bytes
  vpcIngressDailyBytes: 392435047      // Daily average VPC ingress bytes
  usagePercentile: 20                  // Level of usage relative to other projects

GCP projects are used in many different ways and for many different purposes. In case you get a recommendation to delete a project that’s being used in a way that’s out of the scope for this feature, you can dismiss the recommendation and it will stop showing up for the given project. 

Restoring deleted projects

When you choose to shut down a project using the projects.delete() method, it gets marked for deletion. After a project is marked for deletion, it becomes unusable, all resources within that project are shut down, and a 30-day wait period for the project and all of its data to get fully deleted begins.

In case a useful project is accidentally shut down, you have the option to restore the project within that 30-day wait period. Since restoring allows you to recover most but not necessarily all of your project data and resources, we recommend carefully examining the utilization insights associated with a project and considering any additional utilization signals that may not be captured by the Unattended Project Recommender before taking the cleanup action.

Early customer success stories

A number of enterprise customers are already using Unattended Project Recommender to keep their organizations clean of unattended projects and resources.

Decathlon, a French sporting goods retailer, is excited for the insight Unattended Project Recommender will bring to their environment, and are already deploying it as a part of their latest cloud security initiatives.

“After a thorough test of this feature and the validation of our CISO, we ended up deleting our first 775 projects, and no one complained! A great help to improve our security. The next step for us will be to operationalize it at scale, and implement a company wide policy for unattended resource management.” —Adeline Villette, Cloud Security Officer

For Veolia, one of the world’s largest water, waste and energy management companies, not only does this feature reduce security risks and waste, but also helps drive cultural shift and alignment with its ecological transformation strategy.

“This feature allows us to reduce our costs and security debt on assets that are no longer in use, and is also fully in line with Veolia’s philosophy of limiting its carbon footprint. After having tested Unattended Project Recommender on more than 3,000 projects throughout our organization, we are looking to bring it as proactive alerts to our project owners at scale.”—Thomas Meriadec, Product Manager

Box, a secure cloud content management provider, views it as a foundation for building a repeatable process to remediate unused resources.

“Unattended Project Recommender is a great fit for us. It gives us a unified view of project usage across our entire organization and enables us to address security risks of legacy projects in a systematic and organized manner, ensuring an even safer environment.” —Matt Bowes, Staff Security Engineer

Getting started with the Unattended Project Recommender

To help you get started, we’ve prepared a Cloud Shell tutorial (source code) that you can use to find unattended project recommendations within your own Projects/Folders/Organization. Click this button to clone the tutorial from GitHub and run in your Cloud Shell environment:

As you can see, listing recommendations for your projects only takes a few clicks with the tutorial (special thanks to Lanre Ogunmola, Security & Compliance Specialist, for making this look so easy)! For additional detail on using the gcloud CLI or API to discover unattended project recommendations, please refer to the documentation page.

You can also automatically export all recommendations from your Organization to BigQuery and then investigate the recommendations with DataStudio or Looker, or use Connected Sheets that let you use Google Workspace Sheets to interact with the data stored in BigQuery without having to write SQL queries.

As with any other Recommender, you can choose to opt out of data processing at any time by disabling the appropriate data groups in the Transparency & control tab under Privacy & Security settings.

We hope that you can leverage Unattended Project Recommender to improve your cloud security posture and reduce cost, and can’t wait to hear your feedback and thoughts about this feature! Please feel free to reach us at active-assist-feedback@google.com and we also invite you to sign up for our Active Assist Trusted Tester Group if you would like to get early access to the newest features as they are developed.