5 Ways You Need to Know to Reduce Costs with Containers

3182
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
“Cloud Wisdom Weekly: for tech companies and startups” is a new blog series we’re running this fall to answer common questions our tech and startup customers ask us about how to build apps faster, smarter, and cheaper. In this installment, Google Cloud Product Manager Rachel Tsao explores how to save on compute costs with modern container platforms.
Many tech companies and startups are built to operate under a certain degree of pressure and to efficiently manage costs and resources. These pressures have only increased with inflation, geopolitical shifts, and supply chain concerns, however, creating urgency for companies to find ways to preserve capital while increasing flexibility. The right approach to containers can be crucial to navigating these challenges.
In the last few years, development teams have shifted from virtual machines (VMs) to containers, drawn to the latter because they are faster, more lightweight, and easier to manage and automate. Containers also consume fewer resources than VMs, by leveraging shared operating systems. Perhaps most importantly, containers enable portability, letting developers put an application and all its dependencies into a single package that can run almost anywhere.
Containers are central to an organization’s agility, and in our conversations with customers about why they choose Google Cloud, we hear frequently that services like Google Kubernetes Engine (GKE) and Cloud Run help tech companies and startups to not only go to market quickly, but also save money. In this article, we’ll explore five ways to help your business quickly and easily reduce compute costs with containers.
5 ways to control compute costs with containers
Whether your company is an established player that is modernizing its business or a startup building its first product, managed containerized products can help you reduce costs, optimize development, and innovate. The following tips will help you to evaluate core features you should expect of container services and include specific advice for GKE and Cloud Run.
- Identify opportunities to reduce cluster administration
Most companies want to dedicate resources to innovation, not infrastructure curation. If your team has existing Kubernetes knowledge or runs workloads that need to leverage machine types or graphics processing units (GPUs), you may be able to simplify provisioning with GKE Autopilot. GKE Autopilot provisions and manages the cluster’s underlying infrastructure, all while you pay for only the workload, not 24/7 access to the underlying node-pool compute VMs. In this way, it can reduce cluster administration while saving you money and giving you hardened security best practices by default.
- Consider serverless to maximize developer productivity
Serverless platforms continue the theme of empowering your technical talent to focus on the most impactful work. Such platforms can promote productivity by abstracting away aspects of infrastructure creation, letting developers work on projects that drive the business while the platform provider oversees hardware and scalability, aspects of security, and more.
For a broad range of workloads that don’t need machine types or GPUs, going serverless with Cloud Run is a great option for building applications, APIs, internal services, and even real-time data pipelines. Analyst research supports that Cloud Run customers achieve faster deployments with less time spent monitoring services, resulting in reinvested productivity that lets these customers do more with fewer resources.
Designed with high scalability in mind, and an emphasis on the portability of containers, Cloud Run also supports a wide range of stateless workloads, including jobs that run to completion. Moreover, it lets you maximize the skills of your existing team, as it does not require cluster management, a Kubernetes skillset or prior infrastructure experience. Additionally, Cloud Run leverages the Knative spec and a container image as a deployment artifact, enabling an easy migration to GKE if your workload needs change.
With Cloud Run, gone are the days of infrastructure overprovisioning! The platform scales down to zero automatically, meaning your services always have the capacity to meet demand, but do not incur costs if there is no traffic.
- Save with committed use discounts
Committed use discounts provide discounted pricing in exchange for committing to a minimal level of usage in a region for a specified term. If you are able to reliably predict your resource needs, for instance, you can get a 17% discount for Cloud Run (for either one year or three years), and either a 20% discount (for one year) or a 45% discount (for three years) on GKE Autopilot.
- Leverage cost management features
Minimum and maximum instances are useful for ensuring your services are ready to receive requests but do not cause cost overages. For Google Cloud customers, best practices for cost management include building your container with Cloud Build, which offers pay-for-use pricing and can be more cost efficient than steady-state build farms.
Relatedly, if you choose to leverage serverless containers with Cloud Run, you can set minimum instances to avoid the lag (i.e., the cold start) when a new container instance is starting up from zero. Minimum instances are billed at one-tenth of the general Cloud Run cost. Likewise, if you are testing and want to avoid costs spiraling, you can set a maximum number of instances to ensure your containers do not scale beyond a certain threshold. These settings can be turned off anytime, resulting in no costs when your service is not processing traffic. To have better oversight of costs, you can also view built-in billing reports and set budget alerts on Cloud Billing.
- Match workload needs to pricing models
GKE Autopilot is great for running highly reliable workloads thanks to its Pod-level SLA. But if you have workloads that do not need a high level of reliability (e.g., fault tolerant batch workloads, dev/test clusters), you can leverage spot pricing to receive a discount of 60% to 91% compared to regularly-priced pods. Spot Pods run on spare Google Cloud compute capacity as long as resources are available. GKE will evict your Spot Pod with a grace period of 25 seconds during times of high resource demand, but you can automatically redeploy as soon as there is available capability. This can result in significant savings for workloads that are a fit.
Innovation requires balance
Put into practice, these tips can help you and your business to get the most out of containers while controlling management and resource costs. That said, it is worth noting that while managing cloud costs is important, the relationship between “cloud” and “cost” is often complex. If you are adopting cloud computing with only the primary goal of saving money, you may soon run into other challenges. Cloud services can save your business money in many ways, but they can also help you get the most value for your money. This balance between cost efficiency and absolute cost is important to keep in mind so that even in challenging economic landscapes, your tech company or startup can continue growing and innovating.
Beyond cost savings, many tech and startup companies are seeking improved business agility, which is the ability to deploy new products and features frequently and with high quality. With deployment best practices built into GKE Autopilot and Cloud Run, you can transform the way your team operates while maximizing productivity with every new deployment.
You can learn if your existing workloads are appropriate for containers with this fit assessment and these guides for migrating to containers. For new workloads, you can leverage these guides for GKE Autopilot and Cloud Run. And for more tips on cost optimization, check out our Architecture Framework for compute, containers, and serverless.
If you want to learn more about how Google Cloud can help your startup, visit our page here to get more information about our program and apply for our Google for Startups Cloud Program, and sign up for our communications to get a look at our community activities, digital events, special offers, and more.
Google Cloud is Every Retailer’s Most Trusted Cloud

6621
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
Whether they were ready for it or not, the COVID-19 pandemic transformed many retailers into digital businesses. Retailers made huge investments into commerce technologies, customer experience tools, sales and fulfillment technology, and improving digital experiences to continue providing their goods and services to their customers. Now, more than a year into the COVID-19 pandemic, digital retail is the new normal. In fact, many retailers are planning on expanding their digital investments. However, as their digital footprint expands, so do their threats and security concerns.
As a digital-focused retailer, your website is the most visible part of your attack surface. Your website is where your customers search for goods or services, make payments, or learn more about your brand. However, your website does not operate in isolation. There is an underlying infrastructure as well as the services that run on top of it that need protection from a wide array of attacks that may seek to compromise your data, internal employees, business, and customers. During this week’s Google Cloud Retail Summit, we’ve shared why Google Cloud is built to be the most trusted cloud for retailers. From providing you with control over your data as you move from your own data centers to the public cloud to giving you built-in technology to protect your applications all the way to your end users, Google Cloud helps you safely migrate to and operate in our Trusted Cloud.
Trusted Cloud Gives You Control, Transparency, and Sovereignty
- Access Transparency: We offer the ability to monitor and approve access to your data or configurations by Google Cloud support and engineering based on specific justifications and context, so you have visibility and control over insider access.
- Certificate Authority Service (CAS): CAS is a highly scalable and available service that simplifies and automates the management and deployment of private CAs while meeting the needs of modern developers and applications. With CAS, you can offload to the cloud time-consuming tasks associated with operating a private CA, like hardware provisioning, infrastructure security, software deployment, high-availability configuration, disaster recovery, backups, and more, allowing you to stand up a private CA in minutes, rather than the months it might normally take to deploy.
- Confidential Computing: We already encrypt data at-rest and in-transit, but customer data needs to be decrypted for processing. Confidential Computing is a breakthrough technology which encrypts data in-use—while it’s being processed. Confidential VMs take this technology to the next level by offering memory encryption so that you can further isolate your workloads in the cloud. With the beta launch of Confidential VMs, we’re the first major cloud provider to offer this level of security and isolation while giving you a simple, easy-to-use option for your newly built and “lift and shift” applications.
- Cloud Key Management: We allow you to configure the locations where your data is stored, where your encryption keys are stored, and where your data can be accessed from. We give you the ability to manage your own encryption keys, even storing them outside Google’s infrastructure. Using our External Key Management service, you have the ability to deny any request by Google to access encryption keys necessary to decrypt customer data at rest for any reason.
Trusted cloud Helps You Prevent, Detect, and Respond to Threats
- BeyondCorp Enterprise is Google’s comprehensive zero trust product offering. Google has over a decade of experience managing and securing cloud applications at a global scale, and this offering was developed based on learnings from our experience managing our own enterprise, feedback from customers and partners, as well as informed by leading engineering and security research. We understand that most customers host resources across different cloud providers. With this in mind, BeyondCorp Enterprise was purpose-built as a multicloud solution, enabling customers to securely access resources hosted not only on Google Cloud or on-premises, but also across other clouds such as Azure and Amazon Web Services (AWS).
- Cloud Armor: We’re simplifying how you can use Cloud Armor to help protect your websites and applications from exploit attempts, as well as Distributed Denial of Service (DDoS) attacks. With Cloud Armor Managed Protection Plus, you will get access to DDoS and WAF services, curated rule sets, and other services for a predictable monthly price.
- Chronicle: Chronicle is a threat detection solution that identifies threats, including ransomware, at unparalleled speed and scale. Google Cloud Threat Intelligence for Chronicle surfaces highly actionable threats based on Google’s collective insight and research into Internet-based threats. Threat Intel for Chronicle allows you to focus on real threats in the environment and accelerate your response time.
- Google Workspace Security: Used by more than five million organizations worldwide, from large banks and retailers with hundreds of thousands of people to fast-growing startups, Google Workspace and Google Workspace for Education include the collaboration and productivity tools found here. Google Workspace and Google Workspace for Education are designed to help teams work together securely in new, more efficient ways, no matter where members are located or what device they use. For instance, Gmail scans over 300 billion attachments for malware every week and prevents more than 99.9% of spam, phishing, and malware from reaching users. We’re committed to protecting against security 1 threats of all kinds, innovating new security tools for users and admins, and providing our customers with a secure cloud service.
- Identity & Access Management IAM: Identity and Access Management (IAM) lets administrators authorize who can take action on specific resources, giving you full control and visibility to manage Google Cloud resources centrally. For enterprises with complex organizational structures, hundreds of workgroups, and many projects, IAM provides a unified view into security policy across your entire organization, with built-in auditing to ease compliance processes.
- reCAPTCHA Enterprise: reCAPTCHA has over a decade of experience defending the internet and data for its network of more than 5 million sites. reCAPTCHA Enterprise builds on this technology with capabilities, such as two-factor authentication and mobile application support, designed specifically for enterprise security concerns. With reCAPTCHA Enterprise, you can defend your website against common web-based attacks like credential stuffing, account takeovers, and scraping and help prevent costly exploits from malicious human and automated actors. And, just like reCAPTCHA v3, reCAPTCHA Enterprise will never interrupt your users with a challenge, so you can run it on all webpages where your customers interact with your services.
- Security Command Center: With Security Command Center (SCC), our native posture management platform, you can prevent and detect abuse of your cloud resources, centralize security findings from Google Cloud services and partner products, and detect common misconfigurations, all in one easy-to-use platform. We have Premium tier for Security Command Center to provide even more tools to protect your cloud resources. It adds new capabilities that let you spot threats using Google intelligence for events in Google Cloud Platform (GCP) logs and containers, surface large sets of misconfigurations, perform automated compliance scanning and reporting. These features help you understand your risks on Google Cloud, verify that you’ve configured your resources properly and safely, and document it for anyone who asks.
- VirusTotal: VirusTotal inspects items with over 70 antivirus scanners and URL/domain blocklisting services, in addition to a myriad of tools to extract signals from the studied content. Any user can select a file from their computer using their browser and send it to VirusTotal.
- Web Risk API: With Web Risk, you can quickly identify known bad sites, warn users before they click infected links, and prevent users from posting links to known infected pages from your site. Web Risk includes data on more than a million unsafe URLs and stays up to date by examining billions of URLs each day.
Trusted cloud Plays an Active Role in Our Shared Fate
Our trusted cloud provides a shared-fate model for risk management. We stand with retailers from day one, helping them implement best practices for safely migrating to and operating in our trusted cloud.
We hope you enjoy the sessions we’ve created for you with Google Cloud Retail Summit and that they help you understand the ways our trusted cloud can help secure retailers all over the world.
2022’s First Cloud CISO Perspectives: Recap of the Megatrends, Releases and News

9372
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
I’m excited to share our first Cloud CISO Perspectives post of 2022. It’s already shaping up to be an eventful year for our industry and we’re only in month one. There’s a lot to recap in this post, including the U.S. government’s recent efforts to address critical security issues, like open source software security and zero trust architectures. We’ve also released new resources from our Google Cybersecurity Action Team like the Cloud Security Megatrends and the Boards of Directors whitepaper on cloud risk governance.
Cloud Security Megatrends
We’re often asked if the cloud is more secure than on-prem (and why) so we shared our answer in a recent blog post. At Google Cloud, security by design is our priority. We’ve long adopted zero-trust principles for our baseline security architectures and built a global network that relies on defense in depth layers to protect against configuration errors and attacks. But security is always evolving and that is why we also take advantage of the following megatrends:
- Economy of scale: Decreasing the marginal cost of security raises the baseline level of security.
- Shared fate: A flywheel of increasing trust drives more transition to the cloud, which compels even higher security and even more skin-in-the-game from the cloud provider.
- Healthy competition: The race by deep-pocketed cloud providers to create and implement leading security technologies is the tip of the spear of innovation.
- Cloud as the digital immune system: Every security update the cloud gives the customer is informed by some threat, vulnerability, or new attack technique often identified by someone else’s experience. Enterprise IT leaders use this accelerating feedback loop to get better protection.
- Software-defined infrastructure: Cloud is software defined, so it can be dynamically configured without customers having to manage hardware placement or cope with administrative toil. From a security standpoint, that means specifying security policies as code, and continuously monitoring their effectiveness.
- Increasing deployment velocity: Because of cloud’s vast scale, providers have had to automate software deployments and updates, usually with automated continuous integration/continuous deployment (CI/CD) systems. That same automation delivers security enhancements, resulting in more frequent security updates.
- Simplicity: Cloud becomes an abstraction-generating machine for identifying, creating and deploying simpler default modes of operating securely and autonomically.
- Sovereignty meets sustainability: The cloud’s global scale and ability to operate in localized and distributed ways creates three pillars of sovereignty. This global scale can also be leveraged to improve energy efficiency.
If you’re an IT decision maker, pay attention to these megatrends that will continue to drive and reinforce cloud security and will outpace the security of on-prem infrastructure well into the future.
U.S. Federal government cybersecurity momentum
- Open source software security: Earlier this month, Google participated in the White House Summit on open source software security. The meeting came at a critical time for the industry following December’s Log4j vulnerabilities and was both a recognition of the challenge and an important first step towards addressing it. The open source software ecosystem is not homogenous, despite the fact that the industry often thinks of or treats it this way. Some of it, like Linux, is highly curated, while other critical software is supported through diffuse communities including technology companies and other stakeholders. There is also a long tail of many other critical projects driven by a dedicated community of maintainers around the world, including Googlers. In light of this reality, we welcomed the chance to share our recommendations to advance the future of open source software security. Some work we’ve done includes founding the Open Source Security Foundation, which has been instrumental already in making security improvements. We’ve also helped drive a number of key security initiatives within the open source community including security scorecards, the SLSA framework to improve the security and integrity of open source packages, and Secure Open Source Rewards to financially incentivize improvements to critical open source security projects.
- OMB’s Federal zero trust strategy: The publication of the Office of Management and Budget’s zero trust architecture strategy marks an important step for the U.S. federal government’s efforts to modernize under Executive Order 14028. Google Cloud supports this approach, which recognizes the immense security benefits offered by modern computing architectures. For the past decade, Google has successfully applied zero trust principles through our BeyondCorp and BeyondProd frameworks for providing end-user access and securing our cloud workloads. And we’ve brought these best practices from our own journey to global governments and businesses of any size through solutions like BeyondCorp Enterprise and capabilities like Binary Authorization and Anthos Service Mesh, which are embedded in Anthos, our managed application platform. For Federal agencies embarking on this zero trust journey, the Google Cybersecurity Action Team will offer our expertise by conducting Zero Trust Foundations strategy workshops, which can help organizations in the public and private sectors develop actionable and achievable strategies and plans for zero trust implementation.
Google Cybersecurity Action Team Highlights
Here are the latest updates, products, services and resources across our security teams this month:
Security
- Democratizing security operations: We recently announced that Siemplify, a leading security orchestration, automation and response (SOAR) provider, is joining Google Cloud to help companies better manage their threat response. Providing a proven SOAR capability with Chronicle’s approach to security analytics is an important step forward in our vision to advance invisible security and democratize security operations for every organization.
- Security by design: The Highmark Health security team is using “secure-by-design” techniques to address the security, privacy, and compliance aspects of its Living Health solution with Google Cloud’s Professional Services Organization (PSO). Google has long advocated for and followed security by design principles, which is why we’re continuously building enhanced security, controls, resiliency and more into our cloud products and services.
- Secure collaboration for hybrid work environments: The Google Workspace team shared its recommendations for businesses as they prepare for the future of work, where the hybrid/flexible work model is becoming standard practice and a new approach to security is essential.
- Anthos Policy Controller CIS Benchmark enforcement: A big part of our shared fate philosophy is to build secure products and not just security products. A recent example of this in action is embedding CIS benchmark policy conformance in the Anthos Policy Controller. We believe the more we embed approaches like this into our products, the more application and infrastructure teams can intrinsically embed security at the start and reduce toil for the security team.
- DevOps for technology-driven organizations and startups: A key success factor for many security programs is the partnership and integration with development teams, and there are some great resources and lessons in our DORA research.
- Security by design with Chrome OS: ABN AMRO’s Asia-Pacific region team recently shared how they are using Chrome OS and CloudReady to work securely in the cloud, reduce total cost of ownership, and add flexibility for employees. This is a great example of secure by design principles in the use of Chromium.
Risk & Compliance
- Boards of Directors summary guide to cloud risk governance: The latest whitepaper from the Google Cybersecurity Action Team outlines how boards of directors can prioritize safe, secure, and compliant adoption processes for cloud technologies within their organizations.
- TruSight Risk Assessment of Google Cloud: TruSight recently released a comprehensive
risk assessment report on Google Cloud. Our Enterprise Trust team collaborated on this robust assessment of Google Cloud services to validate the design and implementation of controls. TruSight’s risk assessment of our security controls will help customers accelerate and complete their risk management due diligence. - Data governance: Check out this new blog series on data governance where our teams explain the role of data governance, its importance, and the necessary processes to run an effective data governance program. Implementing data governance will help maximize value derived from business data, build user trust, and ensure compliance with required security measures.
Controls and Products
- Encrypting Data Fusion: To help meet the security, privacy and compliance requirements of customers in regulated industries like finance or public sector, we announced the general availability of Customer Managed Encryption Keys (CMEK) integration for Cloud Data Fusion, which enables encryption of both user data and metadata at rest with a key that customers can control through our Cloud Key Management Service (KMS).
Don’t forget to sign-up for our newsletter if you’d like to have our Cloud CISO Perspectives post delivered every month to your inbox. We’ll be back next month with more updates and security-related news.
10173
Of your peers have already watched this video.
1:00 Minutes
The most insightful time you'll spend today!
Predict User Churn on Gaming Apps with Google Analytics Data using BigQuery ML
User retention can be a major challenge for mobile game developers. According to the Mobile Gaming Industry Analysis in 2019, most mobile games only see a 25% retention rate for users after the first day. To retain a larger percentage of users after their first use of an app, developers can take steps to motivate and incentivize certain users to return. But to do so, developers need to identify the propensity of any specific user returning after the first 24 hours.
In this blog post, we will discuss how you can use BigQuery ML to run propensity models on Google Analytics 4 data from your gaming app to determine the likelihood of specific users returning to your app.
You can also use the same end-to-end solution approach in other types of apps using Google Analytics for Firebase as well as apps and websites using Google Analytics 4. To try out the steps in this blogpost or to implement the solution for your own data, you can use this Jupyter Notebook.
Using this blog post and the accompanying Jupyter Notebook, you’ll learn how to:
- Explore the BigQuery export dataset for Google Analytics 4
- Prepare the training data using demographic and behavioural attributes
- Train propensity models using BigQuery ML
- Evaluate BigQuery ML models
- Make predictions using the BigQuery ML models
- Implement model insights in practical implementations
Google Analytics 4 (GA4) properties unify app and website measurement on a single platform and are now default in Google Analytics. Any business that wants to measure their website, app, or both, can use GA4 for a more complete view of how customers engage with their business. With the launch of Google Analytics 4, BigQuery export of Google Analytics data is now available to all users. If you are already using a Google Analytics 4 property, you can follow this guide to set up exporting your GA data to BigQuery.
Once you have set up the BigQuery export, you can explore the data in BigQuery. Google Analytics 4 uses an event-based measurement model. Each row in the data is an event with additional parameters and properties. The Schema for BigQuery Export can help you to understand the structure of the data.
In this blogpost, we use the public sample export data from an actual mobile game app called “Flood It!” (Android, iOS) to build a churn prediction model. But you can use data from your own app or website.
Here’s what the data looks like. Each row in the dataset is a unique event, which can contain nested fields for event parameters.
SELECT *FROM `firebase-public-project.analytics_153293282.events_*`TABLESAMPLE SYSTEM (1 PERCENT)

This dataset contains 5.7M events from over 15k users.
SELECTCOUNT(DISTINCT user_pseudo_id) as count_distinct_users,COUNT(event_timestamp) as count_eventsFROM`firebase-public-project.analytics_153293282.events_*

Our goal is to use BigQuery ML on the sample app dataset to predict propensity to user churn or not churn based on users’ demographics and activities within the first 24 hours of app installation.

In the following sections, we’ll cover how to:
- Pre-process the raw event data from GA4
- Identify users & the label feature
- Process demographic features
- Process behavioral features
- Train classification model using BigQuery ML
- Evaluate the model using BigQueryML
- Make predictions using BigQuery ML
- Utilize predictions for activation
Pre-process the raw event data
You cannot simply use raw event data to train a machine learning model as it would not be in the right shape and format to use as training data. So in this section, we’ll go through how to pre-process the raw data into an appropriate format to use as training data for classification models.
This is what the training data should look like for our use case at the end of this section:

Notice that in this training data, each row represents a unique user with a distinct user ID (user_pseudo_id).
Identify users & the label feature
We first filtered the dataset to remove users who were unlikely to return the app anyway. We defined these ‘bounced’ users as ones who spent less than 10 mins with the app. Then we labeled all remaining users:
- churned: No event data for the user after 24 hours of first engaging with the app.
- returned: The user has at least one event record after 24 hours of first engaging with the app.
For your use case, you can have a different definition of bounce and churning. Also you can even try to predict something else other than churning, e.g.:
- whether a user is likely to spend money on in-game currency
- likelihood of completing n-number of game levels
- likelihood of spending n amount of time in-game etc.
In such cases, label each record accordingly so that whatever you are trying to predict can be identified from the label column.
From our dataset, we found that ~41% users (5,557) bounced. However, from the remaining users (8,031), ~23% (1,883) churned after 24 hours:
SELECTbounced,churned,COUNT(churned) as count_usersFROMbqmlga4.returningusersGROUP BY 1,2ORDER BY bounced

To create these bounced and churned columns, we used the following snippet of SQL code.
...#churned = 1 if last_touch within 24 hr of app installation, else 0IF (user_last_engagement < TIMESTAMP_ADD(user_first_engagement,INTERVAL 24 HOUR),1,0 ) AS churned,#bounced = 1 if last_touch within 10 min, else 0IF (user_last_engagement <= TIMESTAMP_ADD(user_first_engagement,INTERVAL 10 MINUTE),1,0 ) AS bounced,...
You can view the Jupyter Notebook for the full query used for materializing the bounced and churned labels.
Process demographic features
Next, we added features both for demographic data and for behavioral data spanning across multiple columns. Having a combination of both demographic data and behavioral data helps to create a more predictive model.
We used the following fields for each user as demographic features:
geo.countrydevice.operating_systemdevice.language
A user might have multiple unique values in these fields — for example if a user uses the app from two different devices. To simplify, we used the values from the very first user engagement event.
CREATE OR REPLACE VIEW bqmlga4.user_demographics AS (WITH first_values AS (SELECTuser_pseudo_id,geo.country as country,device.operating_system as operating_system,device.language as language,ROW_NUMBER() OVER (PARTITION BY user_pseudo_id ORDER BY event_timestamp DESC) AS row_numFROM `firebase-public-project.analytics_153293282.events_*`WHERE event_name="user_engagement")SELECT * EXCEPT (row_num)FROM first_valuesWHERE row_num = 1 #first engagement);
Process behavioral features
There is additional demographic information present in the GA4 export dataset, e.g. app_info, device, event_params, geo etc. You may also send demographic information to Google Analytics through each hit via user_properties. Furthermore, if you have first-party data on your own system, you can join that with the GA4 export data based on user_ids.
To extract user behavior from the data, we looked into the user’s activities within the first 24 hours of first user engagement. In addition to the events automatically collected by Google Analytics, there are also the recommended events for games that can be explored to analyze user behavior. For our use case, to predict user churn, we counted the number of times the follow events were collected for a user within 24 hours of first user engagement:
user_engagementlevel_start_quickplaylevel_end_quickplaylevel_complete_quickplaylevel_reset_quickplaypost_scorespend_virtual_currencyad_rewardchallenge_a_friendcompleted_5_levelsuse_extra_steps
The following query shows how these features were calculated:
WITHevents_first24hr AS (SELECTe.*FROM`firebase-public-project.analytics_153293282.events_*` eJOINbqmlga4.returningusers rONe.user_pseudo_id = r.user_pseudo_idWHERETIMESTAMP_MICROS(e.event_timestamp) <= r.ts_24hr_after_first_engagement)SELECTuser_pseudo_id,SUM(IF(event_name = 'user_engagement', 1, 0)) AS cnt_user_engagement,# ... repeated for all behavior data ...SUM(IF(event_name = 'use_extra_steps', 1, 0)) AS cnt_use_extra_steps,FROMevents_first24hrGROUP BY1
View the notebook for the query used to aggregate and extract the behavioral data. You can use different sets of events for your use case. To view the complete list of events, use the following query:
SELECTevent_name,COUNT(event_name) as event_countFROM`firebase-public-project.analytics_153293282.events_*`GROUP BY 1ORDER BYevent_count DESC
After this we combined the features to ensure our training dataset reflects the intended structure. We had the following columns in our table:
- User ID:
user_pseudo_id
- Label:
churned
- Demographic features
countrydevice_osdevice_language
- Behavioral features
cnt_user_engagementcnt_level_start_quickplaycnt_level_end_quickplaycnt_level_complete_quickplaycnt_level_reset_quickplaycnt_post_scorecnt_spend_virtual_currencycnt_ad_rewardcnt_challenge_a_friendcnt_completed_5_levelscnt_use_extra_stepsuser_first_engagement
At this point, the dataset was ready to train the classification machine learning model in BigQuery ML. Once trained, the model will output a propensity score between churn (churned=1) or return (churned=0) indicating the probability of a user churning based on the training data.
Train classification model
When using the CREATE MODEL statement, BigQuery ML automatically splits the data between training and test. Thus the model can be evaluated immediately after training (see the documentation for more information).
For the ML model, we can choose among the following classification algorithms where each type has its own pros and cons:

Often logistic regression is used as a starting point because it is the fastest to train. The query below shows how we trained the logistic regression classification models in BigQuery ML.
CREATE OR REPLACE MODEL bqmlga4.churn_logregTRANSFORM(EXTRACT(MONTH from user_first_engagement) as month,EXTRACT(DAYOFYEAR from user_first_engagement) as julianday,EXTRACT(DAYOFWEEK from user_first_engagement) as dayofweek,EXTRACT(HOUR from user_first_engagement) as hour,* EXCEPT(user_first_engagement, user_pseudo_id))OPTIONS(MODEL_TYPE="LOGISTIC_REG",INPUT_LABEL_COLS=["churned"]) ASSELECT*FROMbqmlga4.train
We extracted month, julianday, and dayofweek from datetimes/timestamps as one simple example of additional feature preprocessing before training. Using TRANSFORM() in your CREATE MODEL query allows the model to remember the extracted values. Thus, when making predictions using the model later on, these values won’t have to be extracted again. View the notebook for the example queries to train other types of models (XGBoost, deep neural network, AutoML Tables).
Evaluate model
Once the model finished training, we ran ML.EVALUATE to generate precision, recall, accuracy and f1_score for the model:
SELECT*FROMML.EVALUATE(MODEL bqmlga4.churn_logreg)

The optional THRESHOLD parameter can be used to modify the default classification threshold of 0.5. For more information on these metrics, you can read through the definitions on precision and recall, accuracy, f1-score, log_loss and roc_auc. Comparing the resulting evaluation metrics can help to decide among multiple models.Furthermore, we used a confusion matrix to inspect how well the model predicted the labels, compared to the actual labels. The confusion matrix is created using the default threshold of 0.5, which you may want to adjust to optimize for recall, precision, or a balance (more information here).
SELECTexpected_label,_0 AS predicted_0,_1 AS predicted_1FROMML.CONFUSION_MATRIX(MODEL bqmlga4.churn_logreg)

This table can be interpreted in the following way:

Make predictions using BigQuery ML
Once the ideal model was available, we ran ML.PREDICT to make predictions. For propensity modeling, the most important output is the probability of a behavior occurring. The following query returns the probability that the user will return after 24 hrs. The higher the probability and closer it is to 1, the more likely the user is predicted to return, and the closer it is to 0, the more likely the user is predicted to churn.
SELECTuser_pseudo_id,returned,predicted_returned,predicted_returned_probs[OFFSET(0)].prob as probability_returnedFROMML.PREDICT(MODEL bqmlga4.churn_logreg,(SELECT * FROM bqmlga4.train)) #can be replaced with a proper test dataset
Utilize predictions for activation
Once the model predictions are available for your users, you can activate this insight in different ways. In our analysis, we used user_pseudo_id as the user identifier. However, ideally, your app should send back the user_id from your app to Google Analytics. In addition to using first-party data for model predictions, this will also let you join back the predictions from the model into your own data.
- You can import the model predictions back into Google Analytics as a user attribute. This can be done using the Data Import feature for Google Analytics 4. Based on the prediction values you can Create and edit audiences and also do Audience targeting. For example, an audience can be users with prediction probability between 0.4 and 0.7, to represent users who are predicted to be “on the fence” between churning and returning.
- For Firebase Apps, you can use the Import segments feature. You can tailor user experience by targeting your identified users through Firebase services such as Remote Config, Cloud Messaging, and In-App Messaging. This will involve importing the segment data from BigQuery into Firebase. After that you can send notifications to the users, configure the app for them, or follow the user journeys across devices.
- Run targeted marketing campaigns via CRMs like Salesforce, e.g. send out reminder emails.
You can find all of the code used in this blogpost in the Github repository:
What’s next?
Continuous model evaluation and re-training
As you collect more data from your users, you may want to regularly evaluate your model on fresh data and re-train the model if you notice that the model quality is decaying.
Continuous evaluation—the process of ensuring a production machine learning model is still performing well on new data—is an essential part in any ML workflow. Performing continuous evaluation can help you catch model drift, a phenomenon that occurs when the data used to train your model no longer reflects the current environment.
To learn more about how to do continuous model evaluation and re-train models, you can read the blogpost: Continuous model evaluation with BigQuery ML, Stored Procedures, and Cloud Scheduler
More resources
If you’d like to learn more about any of the topics covered in this post, check out these resources:
- BigQuery export of Google Analytics data
- BigQuery ML quickstart
- Events automatically collected by Google Analytics 4
- Qwiklabs: Create ML models with BigQuery ML
Or learn more about how you can use BigQuery ML to easily build other machine learning solutions:
- How to build demand forecasting models with BigQuery ML
- How to build a recommendation system on e-commerce data using BigQuery ML
Let us know what you thought of this post, and if you have topics you’d like to see covered in the future! You can find us on Twitter at @polonglin and @_mkazi_.Thanks to reviewers: Abhishek Kashyap, Breen Baker, David Sabater Dinter.
3443
Of your peers have already watched this video.
51:50 Minutes
The most insightful time you'll spend today!
API Design Best Practices and Common Pitfalls
The job of an API is to make the application developer as successful as possible. When crafting APIs, the primary design principle should be to maximize application developer productivity and promote adoption.
API designers and developers generally understand the importance of adhering to design principles while implementing an interface. No one wants to design or implement a bad API! Even so, it’s sometimes tempting to look for shortcuts to reach those aggressive sprint timelines, get to the finish line, and deploy an API. The result? Not so desirable!
So what are the API design principles that help optimize developer productivity?
Watch Martin Nally explore the problems and solutions of quality API design. In this webcast , Nally shares insights on topics like designing quality APIs within the limitations of legacy software and recommendations on the technology choices that best satisfy the varying goals.
Nally is a Senior Software Engineer, Apigee for Google Cloud and has almost 40 years of experience as a software developer and architect and is an expert in web APIs. He currently designs and implements systems using APIs at Google.
11243
Of your peers have already watched this video.
1:30 Minutes
The most insightful time you'll spend today!
Indian Retailer Figures Optimizes Hyperlocal Delivery to Increase Customer Experience
Anyone who follows the Indian e-commerce scene knows that one of the largest challenges these companies face is hyperlocal delivery.
That was a problem facing Wellness Forever, a retail chain of pharmacies with 150-plus stores across India.
“Exactly a year ago, we started our journey of hyperlocal deliveries. This optimization was a big time challenge for us to understand how to optimize this,” Palani Subbiah, CTO, Wellness Forever.
The problem in front of Wellness Forever was to identify which customer could can be sold from which store, so that a delivery could be made within 90 minutes.
“We handle a large amount of customer data and we wanted to use insights to help and improve the customer satisfaction index,” says Subbiah.
To do that Wellness Forever leveraged Google Big Query to run massive amount of data to come up with the operational insights. They also used Firebase and Google Maps.
“By 2021, we are going to have about 450 stores. Those stores are going to be not only a physical store, which is a digital store.
More Relevant Stories for Your Company

Amadeus: Shaping the Future of Travel with Apigee
If you’ve taken a trip in the past 30 years, then you’ve probably used Amadeus technology. Our solutions connect over 1.5 billion travellers every year to the journeys they want, linking them via travel agents, search engines, and tour operators to over 700 airlines, 110 airports, 580,000 hotel properties, 40

Are Open Banking Regulations an Effective Entry into the API Economy?
With stated goals of increasing competition, innovation, and financial inclusion, bank regulators across the world are mandating that banks open their systems, enabling consumers to share their financial data with third parties. One touted benefit of this sharing is that it may create digital banking ecosystems that offer consumers more services

Speeding up migrations to Google Cloud with migVisor by EPAM
Application modernization is quickly becoming one of the pillars of successful digital transformation and cloud migration initiatives. Many organizations are becoming aware of the dramatic benefits that can be achieved by moving legacy, on-premises apps and databases into cloud native infrastructure and services, such as reduced Total Cost of Ownership

On-Demand Webinar: How APIs Help Walgreens Merge Physical and Digital Retail
APIs are how modern businesses rapidly expand into new contexts—making it possible for companies like Walgreens to transform from brick-and-mortar businesses into omnichannel organizations that serve customers in innovative ways. Headquartered in Deerfield, Illinois, Walgreens is the second-largest pharmacy store chain in the United States. It specializes in filling prescriptions,






