Indian Retailer Figures Optimizes Hyperlocal Delivery to Increase Customer Experience - Build What's Next

11223

Of your peers have already watched this video.

1:30 Minutes

The most insightful time you'll spend today!

Case Study

Indian Retailer Figures Optimizes Hyperlocal Delivery to Increase Customer Experience

Anyone who follows the Indian e-commerce scene knows that one of the largest challenges these companies face is hyperlocal delivery.

That was a problem facing Wellness Forever, a retail chain of pharmacies with 150-plus stores across India.

“Exactly a year ago, we started our journey of hyperlocal deliveries. This optimization was a big time challenge for us to understand how to optimize this,” Palani Subbiah, CTO, Wellness Forever.

The problem in front of Wellness Forever was to identify which customer could can be sold from which store, so that a delivery could be made within 90 minutes.

“We handle a large amount of customer data and we wanted to use insights to help and improve the customer satisfaction index,” says Subbiah.

To do that Wellness Forever leveraged Google  Big Query to run massive amount of data to come up with the operational insights. They also used Firebase and Google Maps.

“By 2021, we are going to have about 450 stores. Those stores are going to be not only a physical store, which is a digital store.

Explainer

FAQs: Everything Your Need to Know About Cloud Computing

6609

Of your peers have already read this article.

1:30 Minutes

The most insightful time you'll spend today!

Cloud computing is an ever-expanding subject as experts introduce and adopt newer approaches and technologies that broaden its scope. From containers, Kubernetes, microservice architecture, to app modernization enrich your know-how on Google Cloud Platform.

There are a number of terms and concepts in cloud computing, and not everyone is familiar with all of them. To help, we’ve put together a list of common questions, and the meanings of a few of those acronyms. You can find all these, and many more, in our learning resources.

What are containers?

Containers are packages of software that contain all of the necessary elements to run in any environment. In this way, containers virtualize the operating system and run anywhere, from a private data center to the public cloud or even on a developer’s personal laptop. Containerization allows development teams to move fast, deploy software efficiently, and operate at an unprecedented scale. Read more.

Containers vs. VMs: What’s the difference?

You might already be familiar with VMs: a guest operating system such as Linux or Windows runs on top of a host operating system with access to the underlying hardware. Containers are often compared to virtual machines (VMs). Like virtual machines, containers allow you to package your application together with libraries and other dependencies, providing isolated environments for running your software services. However, the similarities end here as containers offer a far more lightweight unit for developers and IT Ops teams to work with, carrying a myriad of benefits. Containers are much more lightweight than VMs, virtualize at the OS level while VMs virtualize at the hardware level, and share the OS kernel and use a fraction of the memory VMs require. Read more.

What is Kubernetes?

With the widespread adoption of containers among organizations, Kubernetes, the container-centric management software, has become the de facto standard to deploy and operate containerized applications. Google Cloud is the birthplace of Kubernetes—originally developed at Google and released as open source in 2014. Kubernetes builds on 15 years of running Google’s containerized workloads and the valuable contributions from the open source community. Inspired by Google’s internal cluster management system, Borg, Kubernetes makes everything associated with deploying and managing your application easier. Providing automated container orchestration, Kubernetes improves your reliability and reduces the time and resources attributed to daily operations. Read more.

What is microservices architecture?

Microservices architecture (often shortened to microservices) refers to an architectural style for developing applications. Microservices allow a large application to be separated into smaller independent parts, with each part having its own realm of responsibility. To serve a single user request, a microservices-based application can call on many internal microservices to compose its response. Containers are a well-suited microservices architecture example, since they let you focus on developing the services without worrying about the dependencies. Modern cloud-native applications are usually built as microservices using containers. Read more.

What is ETL?

ETL stands for extract, transform, and load and is a traditionally accepted way for organizations to combine data from multiple systems into a single database, data store, data warehouse, or data lake. ETL can be used to store legacy data, or—as is more typical today—aggregate data to analyze and drive business decisions. Organizations have been using ETL for decades. But what’s new is that both the sources of data, as well as the target databases, are now moving to the cloud. Additionally, we’re seeing the emergence of streaming ETL pipelines, which are now unified alongside batch pipelines—that is, pipelines handling continuous streams of data in real time versus data handled in aggregate batches. Some enterprises run continuous streaming processes with batch backfill or reprocessing pipelines woven into the mix. Read more.

What is a data lake?

A data lake is a centralized repository designed to store, process, and secure large amounts of structured, semistructured, and unstructured data. It can store data in its native format and process any variety of it, ignoring size limits. Read more.

What is a data warehouse?

Data-driven companies require robust solutions for managing and analyzing large quantities of data across their organizations. These systems must be scalable, reliable, and secure enough for regulated industries, as well as flexible enough to support a wide variety of data types and use cases. The requirements go way beyond the capabilities of any traditional database. That’s where the data warehouse comes in. A data warehouse is an enterprise system used for the analysis and reporting of structured and semi-structured data from multiple sources, such as point-of-sale transactions, marketing automation, customer relationship management, and more. A data warehouse is suited for ad hoc analysis as well custom reporting and can store both current and historical data in one place. It is designed to give a long-range view of data over time, making it a primary component of business intelligence. Read more.

What is streaming analytics?

Streaming analytics is the processing and analyzing of data records continuously rather than in batches. Generally, streaming analytics is useful for the types of data sources that send data in small sizes (often in kilobytes) in a continuous flow as the data is generated. Read more.

What is machine learning (ML)?

Today’s enterprises are bombarded with data. To drive better business decisions, they have to make sense of it. But the sheer volume coupled with complexity makes data difficult to analyze using traditional tools. Building, testing, iterating, and deploying analytical models for identifying patterns and insights in data eats up employees’ time. Then after being deployed, such models also have to be monitored and continually adjusted as the market situation or the data itself changes. Machine learning is the solution. Machine learning allows businesses to enable the data to teach the system how to solve the problem at hand with machine learning algorithms—and how to get better over time. Read more.

What is natural language processing (NLP)?

Natural language processing (NLP) uses machine learning to reveal the structure and meaning of text. With natural language processing applications, organizations can analyze text and extract information about people, places, and events to better understand social media sentiment and customer conversations. Read more.

Learn more

This is just a sampling of frequently asked questions about cloud computing. To learn more, visit our resources page at cloud.google.com/learn.

Blog

Google’s Latest ‘Carbon Footprint’ can Flag Users about Carbon Emission Levels from their Cloud Usage

7078

Of your peers have already read this article.

4:00 Minutes

The most insightful time you'll spend today!

Google's commitment towards sustainability intensifies with the launch of the latest product, Carbon Emission that helps measure, report and track on the gross carbon emission associated with electricity for cloud usage. Learn more!

Google Cloud is proud to support our customers with the cleanest cloud in the industry. For the past four years, we’ve matched 100% of our electricity use with renewable energy purchases, and we were the first company of our size to commit going even further by running on carbon-free energy 24/7 by 2030.  As we work to achieve 24/7 carbon-free energy, we help you take immediate action to decarbonize your digital applications and infrastructure. We’re also working with our customers across every industry to develop new solutions for the unique climate change challenges that organizations face. Today, we’re excited to expand our portfolio of carbon-free solutions and announce new partnerships that will help every company build a more sustainable future. 

First, we’re launching Carbon Footprint, a new product that provides customers with the gross carbon emissions associated with their Google Cloud Platform usage. Now available to every GCP user for free in the Cloud Console, this tool helps you measure, track and report on the gross carbon emissions associated with the electricity of your cloud usage. Of course, the net operational emissions associated with your Google Cloud usage is still zero. With growing requirements for Environmental Social and Governance (ESG) reporting, companies are looking for ways to show their employees, boards and customers their progress against climate targets. Using Carbon Footprint, you have access to the gross energy related emissions data you need for internal carbon inventories and external carbon disclosures, with one click. 

Built in collaboration with customers like AtosEtsyHSBCL’OréalSalesforceThoughtworks and Twitter, our Carbon Footprint reporting introduces a new standard of transparency to support you in meeting your climate goals. You can monitor your gross cloud emissions over time, by project, by product and by region, giving IT teams and developers metrics that can help them reduce their carbon footprint. Our detailed calculation methodology is published so that auditors and reporting teams can verify that their cloud emissions data meets GHG Protocol guidance.

google cloud carbon footprint.gif

“The power of knowledge combined with the power of technology innovation plays a vital role in proactively responding to the climate crisis we are facing. With Google Carbon Footprint reporting, Atos feeds emissions data in our Decarbonization Data Platform, demonstrating potential emissions reductions from the Google Cloud Platform to our customers. This reporting opens up new levels of emissions transparency, trajectory planning, and data insight to support our customers in meeting, and potentially accelerating towards, their climate goals.”—Nourdine Bihmane, Head of Decarbonization Business Line, Atos

“The capability to measure and understand the environmental footprint of our Public Cloud usage is among the key axis of our sustainable tech roadmap. With Google Cloud Carbon Footprint, we are now able to directly follow the impact of our sustainable infrastructure approach and architecture principles.”—Hervé DUMAS, Sustainability IT Director, L’Oreal

While digital infrastructure emissions are just one part of your environmental footprint, accurately accounting for IT carbon emissions is necessary to measure progress against the carbon reduction targets required to avert the worst consequences of climate change. To help you account for emissions beyond our cloud and across your organization, we’re excited to partner with Salesforce Sustainability Cloud, integrating our Google Cloud Platform emissions data into their carbon accounting platform. 

“As we face unprecedented climate challenges, companies across the globe need to embed sustainability into the core of their business in order to meet growing customer and stakeholder expectations, and reduce their environmental impact. Together, Google Cloud and Salesforce Sustainability Cloud can help our joint customers accelerate their path to Net Zero, leveraging data-driven insights and visualizations to track and reduce their carbon emissions to drive sustainable change.”—Ari Alexander, GM of Salesforce Sustainability Cloud. 

From information to action

With the gross energy-related emissions footprint of data associated with your Google Cloud usage now available, we’re committed to providing tools to not only measure your carbon footprint, but help you reduce it. We recently launched low-carbon region icons to help you choose cleaner regions to locate your Google Cloud resources. New users who see the icons are over 50% more likely to choose clean regions over others, ensuring their applications emit less carbon over time. 

For current Google Cloud users, we’re pleased to announce that Active Assist Recommender will include a new sustainability impact category, extending its original core pillars of cost, performance, security, and manageability. Starting with the Unattended Project Recommender, you’ll soon be able to estimate the gross carbon emissions you’ll save by removing your idle resources. Unattended Project Recommender uses machine learning to identify, with a high degree of confidence, projects that are likely abandoned based on API and networking activity, billing, usage of cloud services, and other signals, and provides actionable recommendations on how to remediate those abandoned projects. By deleting these projects, not only can you reduce costs and mitigate security risks, but you can also reduce your carbon emissions. In August, Active Assist analyzed the aggregate data from all customers across our platform, and over 600,000 gross kgCo2e was associated with projects that it recommended for cleanup or reclamation. If customers deleted these projects they would significantly reduce future gross carbon emissions. Check out this blog to learn more about Active Assist.

co2 est.gif
As we roll-out this feature, users will see a recommendation card in the Carbon Footprint dashboard to reduce emissions. They can investigate the associated projects and choose to delete them to reduce emissions

Solutions for climate resilience 

Many of our customers face difficult questions about how their business impacts the natural environment today, and how it will be affected by climate change in the future. Answering these questions requires rich datasets about the planet, better analytics tools and smarter models to predict potential outcomes. For over a decade Google Earth Engine has supported scientists and developers with hyperscale computing power and the world’s largest catalog of satellite image data. Today, we are delighted to announce the preview of Earth Engine as part of Google Cloud Platform. Now, you can access Earth Engine and combine it with other geospatial-enabled products like BigQuery. By extending Earth Engine’s powerful platform to enterprises through Google Cloud, we are bringing the best of Google together

Over the past year we’ve worked with a number of organizations to use Earth Engine technology with tools like BigQuery and the Cloud AI Platform to develop new solutions for responsible commodity sourcing, sustainable land management and carbon emissions reduction. Earth Engine enables companies to track, monitor and predict changes in the Earth’s surface due to extreme weather events or human-caused activities, thus  helping them save on operational costs, mitigate and better manage risks, and become more resilient to climate change threats. This new offering will wrap the unique data, insights and functionality of Earth Engine with a fully-managed, enterprise-grade experience and reliability.

Earth Engine.gif

As we work with our customers to accelerate their sustainability initiatives, earth observation data is proving critical to effectively plan for the long-term impacts of climate change. To extend our geospatial and sustainability use cases we’re also expanding our partnerships with CARTOClimate EngineGeotabNGIS, and Planet to bring their data and core applications to Google Cloud.

These partners will each make their existing platforms and datasets available globally on Google Cloud, giving you low-latency and reliable access to critical data and applications that will inform your sustainability initiatives. By integrating water availability, agricultural data, weather risks, and extensive daily satellite imagery into Earth Engine and BigQuery, you can achieve more ambitious goals for the sustainability of your business and our planet.

Committing to help you meet your climate goals

With each of these tools, we’re working to reduce the barriers you face in adopting more sustainable technology practices. We understand that building more sustainable applications and infrastructure is not easy. You face competing priorities, technical challenges, and the perception that climate action is costly. 

It doesn’t have to be this way. Today, we are making a sustainability pledge to you: teams across Google Cloud are committing to eliminating the barriers you face in building a more sustainable digital future for your organization, and will help you take action today to realize your climate goals. We’ll do this in a number of ways: 

  1. In digital transformation projects and workshops, sustainability teams will always have a seat at the planning table, so we can work together on using cloud technology to build a more sustainable future. 
  2. We’re putting low-carbon signals natively into our products to help developers choose more sustainable options early in their application development. 
  3. We’ll ensure carbon impact is measured consistently with other key performance indicators. Leveraging the social cost of carbon, the ROI models and value assessments you conduct with Google Cloud will project your emissions impact too. 
  4. We’ll be transparent about our carbon impact, by publishing third-party reviewed reports and methodologies, so you can trust the data for your own reports and disclosures. 
  5. We’ll continue to work with the industry on best practices, including educational resources like Sustainable IT – Decoded, a new masterclass created in partnership with Intel, that shares the expertise of sustainability thought leaders. 

For the next decade we need to work together to avert the worst consequences of climate change. We’ve made tremendous progress in building technology that helps everyone do more for the planet, and we’re excited to see what you do with it. Visit this page to learn more about Google Cloud’s sustainability efforts.

3285

Of your peers have already watched this video.

5:30 Minutes

The most insightful time you'll spend today!

How-to

The 5-Min Demo: How to Develop, Train and Deploy Training Models on Kubernetes

Machine learning has taken businesses by storm. A growing number of organizations, both large and small, and spread across a swathe of industries, are figuring out how to quickly adopt ML.

But in the midst of that accelerated push, infrastructure and data science teams have to come to terms with high operational overheads especially considering all the time and effort it takes to develop a specific model and have it run in different places.

If you’re a data scientist or part of the data team, you’ve probably been here: You develop a model on your laptop, train it on the cloud, and then serve it in production, which could be on-prem or in a different cloud. The challenge is that in these various environments, hardware and software configurations could be totally different—and that leads to things breaking down.

That’s where Kubernetes comes in. It provides a way to easily develop training and deployment learning models in the very scalable and flexible way.

In quick, five-minute video, watch how an open-source framework called Kubeflow is used to run machine learning models on Kubernetes. Improve efficiency now!

Case Study

Architecting Data Pipelines Directly Improves Customer Experience at Universe.com

6102

Of your peers have already read this article.

3:30 Minutes

The most insightful time you'll spend today!

To make sure Universe.com customers were getting a top-notch online experience, the company had to make the right technology choices. It's first step? To centralize multiple data sources and create a single data warehouse on Google BigQuery that could serve as the foundation for all of its reporting requirements, both internal and external. The result? Overall performance increased by 10x!

At Universe, we serve customers day and night and are always working to make sure they have a great experience, whether online or at one of our live events.

Our technology has to make that possible, and our legacy systems weren’t cutting it anymore. What we needed was a consistent, reliable infrastructure that would help our internal teams provide a fast and innovative ticket-buying experience to customers.

With our data well-managed, we could free up time for our developers to bring new web features to customers, like tailored add-ons at checkout.

Our team of about 20 software engineers needed more flexibility and agility in our infrastructure; we were using various data processing tools, and it wasn’t easy to share data across teams so that everyone saw the same information.

We also needed to incorporate streaming data into the data warehouse to ensure the consistency and integrity of data that’s read in a particular window of time from multiple sources. Our developer teams needed to be able to ship new features faster, and the data back ends were getting in the way.   

In addition, when GDPR regulations went into effect, we needed to make sure all our data was anonymized, and we couldn’t do that with our legacy tools.

Finding the right data tools for the job
To make sure our customers were getting a top-notch online experience, we had to make the right technology choices.

Our first step was to centralize multiple data sources and create a single data warehouse that could serve as the foundation for all of our reporting requirements, both internal and external.

The new technology infrastructure we built had to let us move and analyze data easily, so our teams could focus on using that data and insights to better serve our customers.

Previously, we had lots of siloed systems and applications running in AWS. We did a trial using Redshift, but we needed more flexibility than it offered in how we loaded historical data into our cloud data warehouse. Though we were using MongoDB Atlas for our transactional database, it was important to continue using SQL for querying data.  

The trial task that really sold us on BigQuery was when we wanted to alter a small table that had about 20 million rows, used for internal reporting.

We needed to add a value, but our PostgreSQL system wouldn’t allow it. Using Apache Beam, we set up a simple pipeline that moved data from the original source into BigQuery to see if we could add the column there.

BigQuery ingested the data and let us add the new value in seconds. That was a significant moment that led us to start looking at how we could build end-to-end solutions on Google Cloud. BigQuery gave us multiple options to load our historical data in batches and build powerful pipelines. 

We also explored Google Cloud’s migration tools and data pipeline options. Once we saw how Cloud Dataflow worked, with its Apache Beam back-end, we never looked back. Google Cloud provided us with the data tools we needed to build our data infrastructure. 

Cloud for data, and for users
Introducing new technologies isn’t always simple—companies sometimes avoid it altogether because it’s so hard. But our Google Cloud onboarding process has been easy. 

It took us less than two months to fully deploy our BigQuery data warehouse using the Cloud Dataflow-Apache Beam combination.

Moving to Google Cloud brought us a lot of technology advantages, but it’s also been hugely helpful for our internal users and customers. The data analytics capabilities that we’re now able to offer users has really impressed our internal teams, including developers and DevOps, even those who haven’t used this type of technology before.

Some internal clients are already entirely self-service. We’ve hosted frequent demos, and also hosted some “hack days,” where we share knowledge with our internal teams to show them what’s possible. 

We quickly found that BigQuery helped us solve scale and speed problems.

One of our main pain points had been adding upsell opportunities for customers during the checkout process. The legacy technology hadn’t allowed us to quickly reflect those changes in the data warehouse. With BigQuery, we’re able to do that, and devote fewer resources to making it happen. We’ve also eliminated the time we were spending tuning memory and availability, since BigQuery handles that. Database administration and tuning required specialized knowledge and experience and took up time. With BigQuery, we don’t have to worry about configuring that hardware and software. It just works.

We’ve also eliminated the time we were spending tuning memory and availability, since BigQuery handles that. Database administration and tuning required specialized knowledge and experience and took up time. With BigQuery, we don’t have to worry about configuring that hardware and software. It just works.

Two features in particular that we implemented using BigQuery have helped us improve the performance of our core transactional database. First, using Cloud Dataflow to convert raw MongoDB logs to structured rows under a BigQuery table, which we can then query using SQL to identify slow or underperforming queries. Second, we can now query multiple logging tables using wildcards, since we load Fastly logs to BigQuery. 

Along with MongoDB Atlas as our main transactional database, much of our infrastructure now runs as Google Cloud microservices using Google Kubernetes Engine (GKE), including the home page and our payment system. Kubernetes cron jobs power background scheduled jobs, and we also use Cloud Pub/Sub. Cloud Storage handles any data storage if any space constraints emerge. 

Our overall performance has increased by about 10x with BigQuery. Both our customers and internal clients, like our sales and finance teams, benefit from the new low-latency reporting. Reports that used to be weekly or monthly are now available in near-real time. It’s not only faster to read records, but faster to move the data, too.

We have Cloud Dataflow pipelines that write to multiple places, and the speed of moving and processing data is incredibly helpful. We stream in financial data using Cloud Dataflow in streaming mode, and plan to have different streaming pipelines as we grow. We have several batch pipelines that run every day. We can move terabytes of data without performance issues, and process more than 100,000 rows of data per second from the underlying database. It used to take us a month to move that volume of data into our data warehouse. With BigQuery, it takes two days.

We’re also enjoying how easy and productive these tools are. They make our life as software engineers easier, so we can focus on the problem at hand, not fighting with our tools. 

What’s next for Universe
Our team will continue to push even more into Google Cloud’s data platform. We have plans to explore Cloud Datastore next. We’re also moving our databases to PostgreSQL on GCP, using Cloud Dataflow and Beam. BigQuery’s machine learning tools may also come into play as Universe’s cloud journey evolves, so we can start doing predictive analytics based on our data. We’re looking forward to gaining even more speed and agility to meet our business goals and customer needs.    

Blog

End Security Risks with the Unattended Projects Recommender Feature

6124

Of your peers have already read this article.

3:00 Minutes

The most insightful time you'll spend today!

Google Cloud's Unattended Project Recommender in the Active Assist helps organizations identify abandoned projects based on API and networking activity, billing, usage of cloud services, and other signals. Learn how!

In fast-moving organizations, it’s not uncommon for cloud resources, including entire projects, to occasionally be forgotten about. Not only such unattended resources can be difficult to identify, but they also tend to create a lot of headaches for product teams down the road, including unnecessary waste and security risks. 

To help you prune your idle cloud resources, we’re excited to introduce Unattended Project Recommender. It’s a new feature of Active Assist that provides you with a one-stop shop for discovering, reclaiming, and shutting down unattended projects. With actionable and automatic recommendations, you no longer have to worry about wasting money or mitigating security risks presented by your idle resources. Unattended Project Recommender uses machine learning to identify, with a high degree of confidence, projects that are likely abandoned based on API and networking activity, billing, usage of cloud services, and other signals. This feature is available via the Recommender API today, making it easy for you to integrate with your company’s existing workflow management and communication tools, or export results to a BigQuery table for custom analysis.

Thousands of projects can be unattended in large organizations, presenting major security risks

Your cloud projects can go abandoned or unattended for a number of reasons — ranging from a test environment that’s no longer needed, to project cancellation, to project owner switching jobs, and more. Not only can such projects contribute to your cloud bill (waste) but they may contain security issues such as open firewalls or privileged service account keys that attackers can exploit to get a hold of your cloud resources for cryptocurrency mining or, worse, compromise your company’s sensitive data. These security risks tend to grow over time because the latest best practices and patches are usually not applied to unattended projects. 

We experience this issue here at Google, too… In fact, it has been on Google’s internal security team’s radar for some time now, so we joined forces and looked into this problem together, starting with our very own “google.com” organization cloud projects. We quickly found some projects that were unattended, but remediating this issue was easier said than done due to challenges in several areas:

  • Detection: With lots of signals available to you via sources like Cloud Monitoring, what are the right ones you should look at (e.g. API, networking, user activity)? How can you tell the difference between an unattended project and a project that has a low level of activity by design (e.g. a “shell” project that holds an auth token)?
  • Remediation: Once you have identified a project that seems abandoned, how do you go about ensuring that it’s indeed an unattended project? How do you reduce the risk of deleting something that might be essential to a production workload, causing irreversible data loss? How do you solve this at the scale of your entire organization, beyond a one-time cleanup? 

Over the course of 2021 we built and tested a Google-internal prototype first, cleaning up many of our internal unattended projects, and then worked with a number of Google Cloud customers to build and tune this feature based on real-life data (thank you to all of our early adopters for working with us and your generous feedback that helped us shape this feature!) It was not uncommon for us to come across organizations with thousands of unattended projects, and we’re very excited to bring Unattended Project Recommender to all customers, in public preview.

Discovering and acting on unattended project recommendations

Unattended Project Recommender analyzes usage activity across all projects under your organization, including the following data:

  • API activity (e.g. service accounts with authentication activity, API calls consumed)
  • Networking activity (ingress and egress)
  • Billing activity (e.g. services with billable usage)
  • User activity (e.g. active project owners)
  • Cloud services usage (e.g. number of active VMs, BigQuery jobs, storage requests)

Based on these signals, it can generate recommendations to clean up projects that have low usage activity (where “low usage” is defined using a machine learning model that ranks projects in your organization by level of usage), or recommendations to reclaim projects that have high usage activity but no active project owners. Here’s what an example post-processed summary list of recommendations can look like for the “foobar” organization that has 3 projects:

  Project ID: demo-project-307815
Recommendation: CLEANUP_PROJECT

Project ID: new-project
Recommendation: N/A

Project ID: bobs-playground-project
Recommendation: RECLAIM_PROJECT

In addition to the recommendations, you can also examine the underlying project activity insights that the recommendations are based upon. The insights provide additional information that can be useful for integration with your organization’s existing workflows and automation (e.g. send an auto-generated email or chat message to project owners based on the list provided by the owners field). Here’s an example insight payload:

  content:
  activeAppengineInstanceDailyCount: 0
  activeCloudsqlInstanceDailyCount: 0
  activeGceInstanceDailyCount: 3
  activeServiceAccountDailyCount: 1
  apiClientDailyCount: 18922           // Daily average API calls produced
  bigqueryInflightJobDailyCount: 0
  bigqueryInflightQueryDailyCount: 0
  bigqueryStorageDailyBytes: 0
  bigqueryTableDailyCount: 0
  consumedApiDailyCount: 0             // Daily average API calls consumed
  datastoreApiDailyCount: 0
  gcsObjectDailyCount: 11
  gcsRequestDailyCount: 0
  gcsStorageDailyBytes: 2663548
  hasActiveOauthTokens: false          // OAuth tokens used in the last 180 days
  hasBillingAccount: true
  numActiveUserOwners: 1
  owners:                              // List of project owners
  - activeOnProject: false
    member: user:user1@example.com
  - activeOnProject: true
    member: user:user2@example.com
  serviceWithBillableUsage:
  – Cloud Storage
  - Compute Engine
  vpcEgressDailyBytes: 264456938       // Daily average VPC egress bytes
  vpcIngressDailyBytes: 392435047      // Daily average VPC ingress bytes
  usagePercentile: 20                  // Level of usage relative to other projects

GCP projects are used in many different ways and for many different purposes. In case you get a recommendation to delete a project that’s being used in a way that’s out of the scope for this feature, you can dismiss the recommendation and it will stop showing up for the given project. 

Restoring deleted projects

When you choose to shut down a project using the projects.delete() method, it gets marked for deletion. After a project is marked for deletion, it becomes unusable, all resources within that project are shut down, and a 30-day wait period for the project and all of its data to get fully deleted begins.

In case a useful project is accidentally shut down, you have the option to restore the project within that 30-day wait period. Since restoring allows you to recover most but not necessarily all of your project data and resources, we recommend carefully examining the utilization insights associated with a project and considering any additional utilization signals that may not be captured by the Unattended Project Recommender before taking the cleanup action.

Early customer success stories

A number of enterprise customers are already using Unattended Project Recommender to keep their organizations clean of unattended projects and resources.

Decathlon, a French sporting goods retailer, is excited for the insight Unattended Project Recommender will bring to their environment, and are already deploying it as a part of their latest cloud security initiatives.

“After a thorough test of this feature and the validation of our CISO, we ended up deleting our first 775 projects, and no one complained! A great help to improve our security. The next step for us will be to operationalize it at scale, and implement a company wide policy for unattended resource management.” —Adeline Villette, Cloud Security Officer

For Veolia, one of the world’s largest water, waste and energy management companies, not only does this feature reduce security risks and waste, but also helps drive cultural shift and alignment with its ecological transformation strategy.

“This feature allows us to reduce our costs and security debt on assets that are no longer in use, and is also fully in line with Veolia’s philosophy of limiting its carbon footprint. After having tested Unattended Project Recommender on more than 3,000 projects throughout our organization, we are looking to bring it as proactive alerts to our project owners at scale.”Thomas Meriadec, Product Manager

Box, a secure cloud content management provider, views it as a foundation for building a repeatable process to remediate unused resources.

“Unattended Project Recommender is a great fit for us. It gives us a unified view of project usage across our entire organization and enables us to address security risks of legacy projects in a systematic and organized manner, ensuring an even safer environment.” —Matt Bowes, Staff Security Engineer

Getting started with the Unattended Project Recommender

To help you get started, we’ve prepared a Cloud Shell tutorial (source code) that you can use to find unattended project recommendations within your own Projects/Folders/Organization. Click this button to clone the tutorial from GitHub and run in your Cloud Shell environment:

google cloud shell.jpg

As you can see, listing recommendations for your projects only takes a few clicks with the tutorial (special thanks to Lanre Ogunmola, Security & Compliance Specialist, for making this look so easy)! For additional detail on using the gcloud CLI or API to discover unattended project recommendations, please refer to the documentation page.

You can also automatically export all recommendations from your Organization to BigQuery and then investigate the recommendations with DataStudio or Looker, or use Connected Sheets that let you use Google Workspace Sheets to interact with the data stored in BigQuery without having to write SQL queries.

As with any other Recommender, you can choose to opt out of data processing at any time by disabling the appropriate data groups in the Transparency & control tab under Privacy & Security settings.

We hope that you can leverage Unattended Project Recommender to improve your cloud security posture and reduce cost, and can’t wait to hear your feedback and thoughts about this feature! Please feel free to reach us at active-assist-feedback@google.com and we also invite you to sign up for our Active Assist Trusted Tester Group if you would like to get early access to the newest features as they are developed.

More Relevant Stories for Your Company

Blog

Partnering with Google Cloud is the Key Behind Recent Healthcare Innovations

It’s simply amazing to witness how some of our systems integrators employ Google Cloud solutions to drive innovation in ways we at Google may never have considered—especially in healthcare.  According to analyst firm MarketsandMarkets, the market for the Cloud in healthcare is projected to grow 43% between 2020 and 2025

How-to

Prototyping Language Applications Made Easy with Generative AI

Did you know generative AI allows developers to prototype applications quickly? With Generative AI Studio on Google Cloud, developers can quickly explore and customize AI models that can be leveraged in Google Cloud applications. Watch along and see how developers, with the right tools, can experiment with new ideas in

Blog

Start-up Paves Way for More Inclusive Clinical Research: Honoring Black Founders of Acclinate with Google Cloud

Editor’s note: February is Black History Month—a time for us to come together to celebrate the diverse set of experiences, perspectives and identities that make up the Black experience. Over the next few weeks, we will highlight Black-led startups and how they use Google Cloud to grow their businesses. Today’s

Trend Analysis

2022 Healthcare Trends: Healthcare Data, M&As, Better Patient Care, AI in Drug Development & Strategic Partnerships

The COVID-19 pandemic continues to push the healthcare and life sciences industry in entirely new ways. In record time, we’ve witnessed public health officials, vaccine developers, equipment manufacturers, and essential workers take life saving actions—regularly putting their own lives at risk—to respond to the exceptional challenges of our time.  Yet

SHOW MORE STORIES