Google Cloud Offering a Natural Migration Path for Developers Accustomed to Heroku
2781
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
Modern developers worldwide have grown accustomed to the comfort of writing code, pushing to a remote Git repository and having that code be deployed at an accessible URL without having to worry about how it is deployed. This was a workflow popularized by Heroku years ago which brought joy and productivity to developers even if it did impose some loss of flexibility for operation teams.
To address that loss of flexibility when meeting security and integration requirements, Heroku introduced Private Spaces. Private Spaces provide network isolation from the internet since any application or datastore provisioned by Heroku is accessible to the internet by default.
Cloud Run is quickly becoming the “swiss army knife” of serverless here at Google Cloud and it’s a natural migration path for developers accustomed to Heroku. The fundamentals are all there:
- Continuously Deploy via Git push using open source Buildpacks or Dockerfiles
- Set CPU and Memory requirements for each instance
- Horizontally scalable apps that scale from zero to thousands of instances to meet traffic demands automatically
So while devs are kept happy, can Cloud Run do something for the Ops folks? Yes. Here are some things available right in the Cloud Run UI:
- Proper Secret Management with IAM based access control. No more setting secrets as environment variables.
- Traffic management between different revisions for blue-green or canary deployments.
- Define SLIs and SLOs with ease. Eg: 90% of requests have to be served under 200ms in a calendar month.
- Secure your service with tools such as Software Delivery Shield, Binary Authorization, and Cloud Armor. Definitely deserves its own blog post.
Recreating Private Spaces on Cloud Run
Let’s focus on network isolation now, let’s say you have an internet-facing app and a private backend API that talks to a private database. Simplest architecture ever, it conceptually looks a bit like this:
Let’s address the database first. If you want to use Postgres then Cloud SQL is most likely what you want, but do keep in mind that we have other datastores that speak Postgres such as AlloyDB and Spanner.
Cloud SQL allows you to create a Postgres instance that’s isolated from the internet by simply unchecking the Public IP checkbox and checking the Private IP checkbox. This will assign an IP address to your Postgres instance on your project’s network.
Once the DB is provisioned you’ll see the IP clearly listed, such as:
Of course there’s so much more to say about CloudSQL, to learn more please take a look at our documentation.
Ok now that you’ve dealt with Postgres, let’s address the private backend API on Cloud Run.
When creating a new Cloud Run service via the Google Cloud Console, Ingress can be limited to “Internal traffic only” so only traffic from internal sources, including your VPC, can access the service. In other words, the internet can not touch it.
As an additional level of security, it’s also possible to enforce that only requests from authorized users be served, In this case a “user” is most likely another service using its associated service account which will need the “roles/run.invoker” in order to call this service.
Now let’s make sure that our Backend API Service can reach the Postgres instance by configuring a VPC Connector. This will allow Cloud Run services to reach into the VPC and therefore, the internal IP for the Postgres instance.
Once the VPC Connector is created, you can associate it with a Cloud Run service.
Then it’s just a matter of configuring your code to use the Postgres instance’s private IP address. A good 12-Factor app friendly spot to do that is with a connection string in an environment variable as part of the Cloud Run service configuration. Better yet, as this may contain a DB password, you can use Secret Manager to mount this environment variable from an encrypted and protected secret.
Finally, let’s now set up that Front End Cloud Run service which will respond to requests from the internet, and securely communicate with the backend API service.
For the frontend service choose to “Allow all traffic” and also “Allow unauthenticated invocations” so anyone on the web can access our URL. We could of course choose the middle option and use Cloud Load Balancing in conjunction with Cloud Armor which provides defenses against DDoS and application attacks, and offers a rich set of WAF rules. However, let’s keep it simple for now.
Keep in mind that our Backend service will only accept requests from within our VPC network, and that we don’t have a private IP address for Cloud Run.
So let’s ensure that all egresses from our Frontend actually get routed to the VPC Connector, this way when our Frontend calls a Backend API via it’s URL endpoint, the Backend will receive the request from within the VPC and allow it in.
PS: If your Backend requires authentication don’t forget to create a Service Account for your Frontend Service and then give it the necessary role following a service-to-service auth pattern.
And that’s it. You now have an operationally acceptable private space like environment with an app composed of two Cloud Run services where the Backend service and Postgres instance are network isolated from the Internet. If after reading this blog you would like to get hands-on experience with the technologies mentioned above, then take a look at Google Cloud Skills Boost. There you will find learning paths, quests, and labs curated to boost your cloud skills in a particular area.
For example here’s a great lab that takes you through developing a REST API on Cloud Run using Go.
Google Products Helps HMH’s Healthcare Staff Work from Anywhere Efficiently and Securely!
10183
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
Hackensack Meridian Health (HMH) executive Mark Eimer explains how an ambitiously-timed rollout of a comprehensive suite of Google products helped the entire organization—from doctors to IT staff—achieve better security, cultivate a more equitable work environment, and ultimately, improve patient outcomes.
How does a recently merged, 17-hospital healthcare system fast-track a platform migration and hardware rollout securely and in a way that improves work for everyone, regardless of location or role? These are the questions that kept me up at night in early 2020, when the pandemic demanded a “big bang”—something our legacy laptops and operating systems couldn’t handle.
We began our work with Google in 2020 with the adoption of Chrome as our default browser. As we migrated platforms, keeping patient data safe was of the utmost importance to us, along with providing every staff member with the tools they needed to work virtually. Our staff often experienced issues accessing our web-based applications using Internet Explorer or Edge Browser, a problem that went away when we switched to Chrome. Chrome’s versatile compatibility also made it easier for my team to migrate all of our web-based operations, and Chrome’s security and manageability were key components to making this switch a huge win for the organization.
The success of this migration led us to extend our Google partnership to patient care applications—where Google’s expertise in AI and ML helps scale the use of diagnostics tools and improve other aspects of the patient journey.
Achieving security at every step
Like so many other healthcare organizations, we’ve been concerned about ransomware attacks. This is part of why we moved to Google Workspace and distributed over 3,000 Chrome OS devices in kiosk mode in March of 2020, when many of us went remote due to the pandemic. We were very concerned about team members accessing corporate applications through home devices that were running EOL operating systems (WIN7), as well as a general lack of antivirus and encryption measures.
We were protected by the fact that Google’s software and hardware both had built-in security features that we needed to stave off sophisticated attackers. For example, Chrome OS automatically updates to the latest security update and encrypts data living outside the cloud on the hardware. These features protected us from security-related disruptions, letting us securely move a huge library of file shares and emails across thousands of accounts to Google Chrome OS in just four months.
A year later, in March 2021, we migrated the enterprise over to Google Workspace and saw an immediate reduction in spam by 30% from the inherent built-in AI/ML. This meant staff were less likely to receive (and click through) phishing attempts. My team could connect, create, and collaborate easily and securely—even as more of us were working from home and needed to access sensitive data remotely.
Leveling the playing field
As an organization, we were surprised by how many team members didn’t have personal computers at home. We quickly decided that if we needed team members to work from home, the health network would have to supply hardware. Chromebooks’ lower price tag compared to PCs—on top of their built-in security controls—allowed us to purchase, deploy, and support that initial distribution of 3,000 Chromebooks to team members in less than three weeks, providing devices to every eligible remote employee instead of just a select few. This was vital to reaching our equitable technology goal as part of our diversity and inclusion initiative: everybody has the same tools to do good work.
When all employees have what they need to do their jobs well, we get better patient outcomes. Before we began this cloud adoption journey, patient and staff experiences were different within the hospitals and outside of them.
Now it’s the same wherever our staff is, and we’ve seen efficiency and accessibility benefits extend to the patient side. For example, we built a web-based contact center that supports 80 locations that use Workspace and Chrome OS devices. Since customer service, admin, and providers are all on the same system, it has become a one-stop shop for patients.
Furthermore, through the Grow with Google program, we were able to provide another benefit to employees that drove our equity goals. Google trained 50 non-IT staff members—from environmental services, food and nutrition, and other non-tech areas who were interested in making a career change to IT—on the Google products we were using. They may not have thought about switching to a career in IT before the Grow with Google program came to our organization, but through this partnership, they now have that opportunity.
A strategic, long-term partner
With any large-scale rollout, the work doesn’t end once laptops are in employee hands. Google has shown their commitment to long-term collaboration as they continuously optimize their products for the unique needs of healthcare providers and go the extra mile in tailoring tools to our staff’s workflows.
For example, on the Chrome OS side, the Google team has helped our registration desks and document centers with device integration for hardware like credit card readers and e-signature pads. They’ve also helped us meet security and privacy requirements mandated by state and federal governments around HIPAA, Medicaid, and Medicare reimbursements. Over this next year, we’ll look at a feature roadmap with Google Cloud to deliver further enhancements, iterating on the product itself to meet our needs for the present and the future.
Delivering the future of healthcare
The benefits we’ve seen around security and usability—and the ability to provide all staff with equal access to Google’s technology—are why we’re expanding our partnership with Google to both the administrative and clinical sides of HMH. In addition to further Google rollouts with corporate, next year we’re distributing Chromebooks to all 350 of our ambulatory clinics.
We’re also working with the Google professional services team to create a custom AI model that analyzes 3D mammogram images. This AI model will enable two providers to read mammograms—which adheres to international best practices but is currently rare in the US—without requiring additional time. Conducting double readings of mammograms will yield better health outcomes for our patients, such as a lower patient recall rate and an increased accuracy in detecting breast cancer.
We’re currently building the model using a variety of Google Cloud products, including Cloud Healthcare API. Once complete, this model is expected to be trained, deployed, and maintained in Google’s Vertex AI, allowing our providers to be more productive as they make clinical decisions with AI support. As the model is proven over time, we plan to make the predictive services accessible to other healthcare organizations.
With Google, we’re able to achieve a unified architecture for storing data as well as training and deploying AI models, which enable our staff to work more efficiently and securely from anywhere. While I may not be able to predict the future as accurately as AI can, I foresee our continued partnership with Google as a key part of HMH’s improved provider and patient outcomes.
Customer Voices: How Firms from Across Industries Leverage Google Cloud
DOWNLOAD CASE STUDY14071
Of your peers have already downloaded this article
1:30 Minutes
The most insightful time you'll spend today!
From powering everyday operations and accelerating application innovation, to providing tools for specific business needs and executing on big ideas, to advancing the security of technology solutions, companies from across industries have leveraged Google Cloud for business benefits.
Companies from across industries have turned to Google Cloud for transforming their business, modernizing their infrastructure, and gleaning intelligence from data. For instance:
- Johnson & Johnson achieved a 41% increase in search results from high-quality job applicants, significantly improving the company’s ability to quickly hire top talent.
- Sony Network Communications now processes 10 billion monthly queries faster, which advances data analysis.
- University College Dublin saw significant 6-figure savings by eliminating legacy hardware, software, and maintenance.
And there are many such examples. Read the collection of case studies to find out how companies from across industries and geographies leveraged Google Cloud for measurable business benefits and for solving complex problems.
BigQuery Omni: Your Solution for Multi-Cloud Geospatial Analytics
1485
Of your peers have already read this article.
6:30 Minutes
The most insightful time you'll spend today!
As we become increasingly reliant on technology to make decisions, geospatial data is becoming more critical than ever. It is a powerful resource that can be used to solve a variety of problems, from tracking the movement of goods, identifying interest areas and potential areas of disaster.
Geospatial data incorporates data with a geographic component, such as latitude and longitude coordinates, addresses, postal codes, or place names, and can be obtained from a variety of sources, including satellites, sensors, and surveys, making it an incredibly powerful tool with a broad range of applications.
One of the key features of BigQuery, Google Cloud’s serverless enterprise data warehouse, is its ability to analyze geospatial data. However, oftentimes geospatial data sits in a variety of public clouds, not just Google Cloud. To access it effectively, you need a multi-cloud analytics solution that lets you capitalize on the distinct capabilities of each cloud platform, while extracting insights and value from data sitting across multiple cloud platforms.
BigQuery Omni is a multi-cloud analytics solution that enables the analysis of data stored across public cloud environments, including Google Cloud, Amazon Web Services (AWS) and Microsoft Azure, without the need to transfer the data. With BigQuery Omni, users can employ the same SQL queries and tools used to analyze data in Google Cloud to analyze data in other clouds, making it easier to gain insights from all data, regardless of the storage location. For businesses using multiple clouds, BigQuery Omni is an excellent tool to unify analytics and optimize the value of data.
With BigQuery Omni, organizations can analyze location-based information or geographic components, such as latitude and longitude coordinates, addresses, postal codes, or place names without even copying their data to Google Cloud. For example, you could use BigQuery Omni to analyze data from a fleet of delivery vehicles to track their location and identify potential problems.
BigQuery Omni and geospatial data analysis
If you are working with geospatial data, BigQuery Omni is a powerful tool that can help you to get insights from your data. It is scalable, reliable, and secure, making it a great option for unifying your analytics and getting the most out of your data.
Here are a few examples of ways organizations might use BigQuery Omni for geospatial data:
- A transportation company could use BigQuery Omni to analyze data from GPS sensors in its vehicles to track the movement of its fleet and identify potential problems.
- A retail company could use BigQuery Omni to analyze data from its point-of-sale systems to track customer behavior and identify trends.
- A government agency could use BigQuery Omni to analyze data from its weather sensors to track the movement of storms and identify areas at risk of flooding.
BigQuery Omni and geospatial data can be used together to gain insights into a variety of business problems. Specifically, some of the advantages of using BigQuery Omni and geospatial data include:
- Access to quality geospatial data: BigQuery supports loading of newline-delimited GeoJSON files and provides built-in support for loading and querying geospatial data. Data from public data sources like BigQuery public datasets, the Earth Engine catalog, and the United States Geological Survey (USGS) can be easily integrated into your BigQuery environment. Earth Engine has an integrated data catalog with a comprehensive collection of analysis-ready datasets, including satellite imagery and climate data. This data can be combined with proprietary data sources such as SAP, Oracle, Esri ArcGIS Server, Carto, and QGIS.
- Loading and preprocessing of geospatial data: BigQuery has built-in support for loading and querying geospatial data types, and you can use partner solutions such as FME Spatial ETL to load data.
- Working with different geospatial data types and formats: BigQuery supports a variety of file types and formats including WKT, WKB, CSV and GeoJSON.
- Coordinate reference systems: BigQuery’s geography data type is globally consistent. That means that your data is registered to the WGS84 reference system and your analyses can span a city block or multiple continents.
Overall, geospatial analytics with BigQuery Omni provides a wide range of technical capabilities for processing and analyzing geospatial data, making it a powerful tool for businesses that need to work with location-based data.
Analyzing geospatial data with BigQuery Omni
Imagine a retailer who has a large chain of department stores with locations all over the country. They are looking to expand their business and want to identify areas with high sales potential. They want a way to get a better understanding of their sales volume within specific geographic boundaries. To achieve this goal, the retailer turns to the GIS (Geographic Information System) functions built into BigQuery. Here are the steps what the retailer takes to analyze this dataset:
Step 1 : An initial orders dataset on AWS S3 contains 5.54 million rows, and with separate locations (300 rows) and zipcode (33144 rows) metadata files on AWS S3.
Orders Parquet files:
Location and Zipcode files:
Step 2 : The retailer uses BigQuery Omni to establish a connection between the data stored in AWS and BigQuery, enabling them to access the S3 datasets externally.
External Connection for AWS
External Table for orders
External Table for locations
External Table for zipcode
Step 3 : They combine the orders and locations datasets using BigQuery Omni, and remotely aggregate the data on AWS. Joining this dataset with geospatial datasets helps them derive geospatial coordinates.
The final aggregated dataset is reduced to 23 rows. Subsequently, they bring back the result dataset, which contains just 23 rows. This helps them reduce the extraction of millions of rows to just 23 rows for their geospatial analytics.
Select sales.store_city store_city,
sales.number_of_sales_last_10_mins number_of_sales_last_10_mins,
sales.store_zip store_zip,
ST_GeogPoint(zip_lat_lng.longitude ,
zip_lat_lng.latitude ) geo
FROM (
SELECT
FORMAT_DATETIME("%X",
CURRENT_DATETIME("America/Los_Angeles")) current_time,
MAX(DATETIME(time_of_sale,
"America/Los_Angeles")) time_of_last_sale,
COUNT(1) number_of_sales_last_10_mins,
locations.city store_city,
locations.zip store_zip
FROM
`bqomni-blog.aws_locations.orders_small` sales
JOIN
`bqomni-blog.aws_locations.locations` locations
ON
sales.store_id = locations.id
GROUP BY
locations.city,
locations.zip ) sales
JOIN
`bqomni-blog.aws_locations.zipcode` zip_lat_lng
ON
cast(sales.store_zip as INT) = cast(zip_lat_lng.zipcode as INT)
WHERE ST_WITHIN( ST_GeogPoint(zip_lat_lng.longitude , zip_lat_lng.latitude ) ,ST_GeogFromText(zip_lat_lng.zipcode_geom ) )
AND zip_lat_lng.state_name = "New York"
ORDER BY number_of_sales_last_10_minsAggregated Sales data by region
To build richer views of their sales volume data, the retailer uses BigQuery GeoViz integration, a powerful tool that allows for visualizing geographic data on maps.BigQuery Geo Viz is a web tool for visualization of geospatial data in BigQuery using Google Maps APIs. You can run a SQL query and display the results on an interactive map
BigQuery Geo view for sales data analysis
With the geo-tagged data in place, the retailer can now see regional sales volume, sales density, distribution by department by time, and distribution within department, all powered through BigQuery.
Satellite view
Benefits of using BigQuery Omni
Beyond geospatial analysis, BigQuery Omni offers a number of benefits, including:
Reduced costs: BigQuery Omni’s ability to eliminate data transfers between clouds can help organizations reduce costs and simplify data management, making it a valuable tool for multi-cloud analytics. Also, the ability to access and analyze data across multiple clouds can reduce the need for data replication and synchronization, which can further simplify the ETL process and improve data consistency.
Unified governance: BigQuery Omni uses the same security controls as BigQuery, which include features such as encryption, access controls, and audit logs, to help protect data from unauthorized access.
Single pane for analytics: BigQuery Omni provides a single interface for querying data across all three clouds, which can simplify the process of analyzing data and reduce the need for organizations to use multiple analytics tools.
Flexibility: Analyze data stored in any of the supported cloud storage services, giving organizations the flexibility to work with the data they have regardless of where it’s located.
BigQuery Omni is a valuable tool for geospatial analysis because it allows you to analyze data from multiple sources without having to move the data. This can save you time and money, and it can also help you to get more accurate insights from your data.If you are looking for a way to improve the accuracy, efficiency, and decision-making of your business, using BigQuery Omni to analyze geospatial data can be a powerful tool.
References
- Omni-introduction
- Omni-benefits
- Omni-aws-create-connection
- Omni-azure-create-connection
- BigQuery Omni Retail Demo on Looker
- Omni-pricing
- Working with Geospatial Data
- Geospatial Analytics – Intro
Learn more about how BigQuery Omni can help your organization.
Assuring Compliance in the Cloud: Paper by Google Cloud’s Office of the CISO
4901
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
Cloud transformation and the adoption of modern DevOps technology presents both opportunities and challenges for IT compliance functions. With DevOps style application development, the feedback loop for developers and engineers is much tighter than with traditional application development pipelines, enabling speed and agility of application release cycles. While speedy CI/CD is a critical advantage of DevOps, it also shifts compliance left in the development timeline, and therefore puts pressure on the IT risk & compliance organization to modernize their approach to regulatory compliance as well. With the ongoing shift towards cloud technologies and DevOps, modernization of regulatory compliance is no longer optional for an IT compliance function
Compliance modernization is a broad mandate that spans the way the function is governed; the tools, technology, and analytics it uses; the number and nature of its connections to other parts of the business; verifiability and auditability of the controls’ evidence, the expectations assigned to it; and more.
Public cloud technology is becoming a core part of many industries today, and with this comes some potential risks such as cloud misconfigurations exposing intellectual property, loss of physical control of assets, skillset scarcity around cloud based security and compliance.
Given the constantly changing risk landscape, it is critical that regulations more closely align to address these risks. As regulations and risks evolve, the aim of a modern compliance function is to help an organization stay compliant as it goes through a digital transformation. As organizations go through digital transformation, IT compliance also needs to transform — via upgrading the technology stack, modifying the business processes and most importantly re-skilling people to become cloud aware.
Today we are releasing the new paper by Google Cloud’s Office of the CISO. In the paper we reveal a new approach for modernizing your compliance approach using modern approaches and Google Cloud toolsets. Your team can leverage the paper to add value to enterprises, both by charting a course to the safe use of cloud technology and by reducing risk through the use of the public cloud.
Read the paper “Assuring Compliance in the Cloud.”
Also, review these related resources:
- “Risk Governance of Digital Transformation in the Cloud“ paper
- “Making Compliance Cloud-native” (episode 14) with Zeal Somani
- Google Cloud Compliance resource center
- Our compliance blueprints: PCI DSS on GKE and GCP FedRAMP Blueprint
- Google Cloud security best practices center
Serverless for Startups, the Best Way to Succeed: Expert Says
7024
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
As Google Cloud has become a choice for more startups, I’ve experienced an increase in founders asking how they should think about cloud services. Though each startup is different and requirements may vary across industries and regions, I’ve seen a few core best practices that help startups to succeed—as well as several traps to avoid.
For example, If you go with a public cloud provider, you’re ideally not starting from ground zero like you would running your own data center, but it’s important not to introduce similar complexity in a virtualized environment. Just because you’re using public cloud infrastructure doesn’t mean you want to manage it.
Instead, you want to leverage platforms that abstract away complexity so your team can focus on delivering value to customers. That’s where serverless comes in. Serverless platforms are fully managed by the provider, offering automatic scaling for workloads, as well as provisioning, configuring, patching, and management of servers and clusters. Freed from these resource-intensive tasks, your technical talent can focus on the things that differentiate your business, not on IT curation.
This applies to not only running stateless applications, but also managing and analyzing data. You should be collecting data points about which features are most popular on your platform, what people are buying, the types of activities that help your customers get their jobs done, and so on. This information is essential to building and executing on a product roadmap that will serve your customers. However, it’s not enough to collect this data, you need to make your data accessible, secure, and easy for your team to analyze—so where do you run and host it?
On Google Cloud, this is where options like Spanner and Cloud SQL can play a large role, as can BigQuery for analysis. You won’t have to worry about standing up infrastructure or patching servers—you can just stream your data to our data management platforms, where it’s available whenever someone needs to run a query. By leveraging a serverless architecture, your startup can be data-driven without having to invest in the traditional complexity of database administration and management—and that can significantly change your playing field.
Serverless is just one of the factors you should consider as you build out your tech stack. To hear my thoughts on a range of other topics relevant to startups — such as security, cloud credits, and the differences among managed services — check out the below video or visit our Build and Grow page.
More Relevant Stories for Your Company
Google Maps Platform Can Elevate FinTech Experience with Less Risks and Higher Security
The financial services industry is changing—an estimated $68 trillion in wealth transferring from baby boomers to millennials.1 This means financial service providers will have to deliver the speed, ease-of-use, technological sophistication, and tailored services that millennials have come to expect. In fact, half of all millennials are willing to switch to
Enhancing Collaboration with Sheets, Python, and Google Cloud
See how you can enhance collaboration within your organization using Google Sheets. Watch to learn about a new set of tools to create custom functions that tap into the power of Python and to expose functions in a standardized fashion throughout your organization.
5 Ways You Need to Know to Reduce Costs with Containers
“Cloud Wisdom Weekly: for tech companies and startups” is a new blog series we’re running this fall to answer common questions our tech and startup customers ask us about how to build apps faster, smarter, and cheaper. In this installment, Google Cloud Product Manager Rachel Tsao explores how to save
App Engine Basics to Help You Build and Deploy Low-latency, Scalable Apps
App Engine is a fully managed serverless compute option in Google Cloud that you can use to build and deploy low-latency, highly scalable applications. App Engine makes it easy to host and run your applications. It scales them from zero to planet scale without you having to manage infrastructure. App
