Data continues to grow in volume and is increasingly distributed across lakes, warehouses, clouds, and file formats. As more users demand more use cases, the traditional approach to build data movement infrastructure is proving difficult to scale. Unlocking the full potential of data requires breaking down these silos, and is increasingly a top priority for enterprises.

Earlier this year, we previewed BigLake, a storage engine that extends innovations in BigQuery storage to open file formats running on public cloud object stores. This allows customers to build secure multi-cloud data lakes over open file formats. BigLake provides consistent, fine-grained security controls for Google Cloud and open-source query engines to interact with data. Today, we are excited to announce General Availability for BigLake, and a set of new capabilities to help you build a differentiated data platform.

“We are using GCP to build and extend one of the street’s largest risk systems. During several tests we have seen the great potential and scale of BigLake. It is one of the products that could support our cloud journey and drive application’s future efficiency” – Scott Condit, Director, Risk CTO Deutsche Bank.

Build a distributed data lake that spans across warehouses, object stores & clouds with BigLake

Customers can create BigLake tables on Google Cloud Storage (GCS), Amazon S3 and ADLS Gen 2 over supported open file formats, such as Parquet, ORC and Avro. BigLake tables are a new type of external table that can be managed similar to data warehouse tables. Administrators do not need to grant end users access to files in object stores, but instead manage access at a table, row or a column level. These tables can be created from a query engine of your choice, such as BigQuery or open-source engines using the BigLake connector. Once these tables are created, BigLake and BigQuery tables can be centrally discovered in the data catalog and managed at scale using Dataplex.

BigLake extends the BigQuery storage API to object stores to help you build a multi-compute architecture. BigLake connectors are built on the BigQuery storage API and enable Google Cloud DataFlow and open-source query engines (such as Spark, Trino, Presto, Hive) to query BigLake tables by enforcing security. This eliminates the need to move the data to a query engine specific use case and security only needs to be configured at one place and is enforced everywhere.

“We are using GCP to design datalake solutions for our customers and transform their digital strategy to create a data-driven enterprise. Biglake has been critical for our customers to quickly realize the value of analytical solutions by reducing the need to build ETL pipelines and cutting-down time-to-market. The performance & governance features of BigLake enabled a variety of data lake use cases for our customers.” – Sureet Bhurat, Founding Board member – Synapse LLC

BigLake unlocks new use cases using Google Cloud and OSS Query engines

During the preview, we saw a large number of customers use BigLake in various ways. Some of the top use cases include:

Building secure and governed data lakes for open-source workloads – Workloads migrating from Hadoop, Spark first customers, or those using Presto/Trino, can now use BigLake to build secure, governed and performant data lakes on GCS. BigLake tables on GCS provide fine-grained security, table management (vs giving access to files), better query performance and integrated governance with Dataplex. These characteristics are accessible across multiple OSS query engines when using the BigLake connectors.

“To support our data driven organization, Wizard needs a data lake solution that leverages open file formats and can grow to meet our needs. BigLake allows us to build and query on open file formats, scales to meet our needs, and accelerates our insight discovery. We look forward to expanding our use cases with future BigLake features” – Rich Archer, Senior Data Engineer – Wizard

Eliminate or reduce data duplication across data warehouses and lakes – Customers who use GCS, and BigQuery managed storage had to previously create two copies of data to support users using BigQuery and OSS engines. BigLake makes the GCS tables more consistent with BigQuery tables, reducing the need to duplicate data. Instead, customers can now keep a single copy of data split across BigQuery storage and GCS, and data can be accessed by BigQuery or OSS engines in either places in a consistent, secure manner.

Fine-grained security for multi-cloud use cases – BigQuery Omni customers can now use BigLake tables on Amazon S3, and ADLS Gen 2 to configure fine grained security access control, and take advantage of localized data processing, and cross cloud transfer capabilities to do multi-cloud analytics. Tables created on other clouds are centrally discoverable on Data catalog for ease of management & governance

Interoperability between analytics and data science workloads – Data science workloads, using either Spark or Vertex AI notebooks can now directly access data in BigQuery or GCS through the API connector, enforcing security & eliminating the need to import data for training models. For BigQuery customers, these models can be imported back into BigQuery ML to produce inferences.

Build a differentiated data platform with new BigLake capabilities

We are also excited to announce new capabilities as part of this General Availability launch. These include:

• Analytics Hub support: Customers can now share BigLake tables on GCS with partners, vendors or suppliers as linked data sets. Consumers can access this data in place through the preferred query engine of their choice (BigQuery, Spark, Presto, Trino, Tensorflow).
• BigLake tables is now the default table type BigQuery Omni, and has been upgraded from the previous default of external tables.
• BigQuery ML support: BigQuery customers can now train their models on GCS BigLake tables using BigQuery ML, without needing to import data, and accessing the data in accordance to the access policies on the table.
• Performance acceleration (preview): Queries for GCS BigLake tables can now be accelerated using the underlying BigQuery infrastructure. If you would like to use this feature please get in touch with your account team or fill out this form.
• Cloud Data Loss Prevention (DLP) profiling support (coming soon): Cloud DLP can soon scan BigLake tables to identify and protect sensitive data at scale. If you would like to use this feature please get in touch with your account team.
• Data masking and audit logging (Coming soon): BigLake tables now support dynamic data masking, enabling you to mask sensitive data elements to meet compliance needs. End user query requests to GCS for BigLake tables are now audit logged and are available to query via logs.

Next steps

Refer to BigLake documentation to learn more, or get started with this quick start tutorial. If you are already using external tables today, consider upgrading them to BigLake tables to take advantage of above mentioned new features. For more information, reach out to the Google cloud account team to see how BigLake can add value to your data platform.

We are pleased to announce the availability of Google Cloud Monitoring uptime checks in the Google Cloud mobile app. 

The uptime check capability offered by Cloud Monitoring is a simple yet powerful observability tool to monitor an application’s availability and performance on URLs provided by customers. The primary purpose of an uptime check is to track resource availability on an on-going basis. It’s essentially a pulse check for a system and is often the first to detect impending issues. Using uptime checks can proactively detect availability and latency issues with an application and reduce the time and severity your users are impacted by an outage. Uptime check can validate if the response code from your application is as expected, and also the latency of your service from different regions. Uptime checks can further inform your SLAs and SLOs, and track your service’s health over time. SREs at Google use uptime checks as an important part of their daily routine.

Given the critical role served by uptime checks, we heard your feedback on having the ability to track these checks in the Google Cloud mobile app. With this launch SRE’s, operators or administrators can do exactly that, in addition to easily accessing their cloud resources (App Engine, Compute, Databases, Storage), managing permissions using IAM, viewing logs, incidents, errors, and billing.

To access uptime checks, simply tap the “Operations” tab in the Google Cloud mobile app on your iOS or Android device.

After navigating to “Uptime Checks”, you will see a list of all the services you have previously added using the Create public uptime check documentation.

The list is sorted by status, so all failing checks are at the top. You can easily filter by status using the “Filters” option in the top-right corner of the screen.

Tapping any of the uptime checks will take you to the “details”-screen.

Here you’ll see your application’s uptime, latency, status per region (if you’re monitoring your application from more than one cloud region) and graphs to give you a sneak preview. The “passed check”-graph is showing how many checks passed in the last hour up to 6 weeks. You can easily change the interval using the scale above the graphs.

Scroll down the page to see more data. At the bottom of the screen, we’ve included some configuration details for this uptime check, so you can check the above data against this specific setting.

We hope you find this new uptime check feature useful in your daily work. To create new uptime checks, start here: Create public uptime check.

We’re working hard to make the Google Cloud app your companion for monitoring and managing your Google Cloud resources on the go.

For pricing information on uptime checks, please see the Cloud Monitoring pricing summary.

This new feature is available for you to use today. If you have any feedback, we would love to hear from you — just click the “Send feedback” button in the app. And if you don’t have the app – go ahead and download the app today from Google Play or the Apple App Store.

Expectations for both consumer and commercial products have changed. Consumers want products that evolve with their needs, adapt to their preferences, and stay up-to-date over time. Manufacturers, in turn, need to create products that provide engaging customer experiences not only to better compete in the marketplace, but also to provide new monetization opportunities. 

However, embedding intelligence into new and existing products is challenging. Updating hardware is costly, and existing connected products do not have the capability to add new features. Furthermore, manufacturers do not have sufficient customer insights due to product telemetry and customer data silos, and may lack the AI expertise to quickly develop and deploy these features. 

That’s why today we’re launching Intelligent Products Essentials, a solution that allows manufacturers to rapidly deliver products that adapt to their owners, update features over-the-air using AI at the edge, and provide customer insights using analytics in the cloud. The solution is designed to assist manufacturers in their product development journeys—whether developing a new product or enhancing existing ones. 

With Intelligent Products Essentials, manufacturers can:

• Personalize customer experiences: Provide a compelling ownership experience that evolves over the lifetime of the product. For example, a chatbot that contextualizes responses based on product status and customer profile.
• Manage and update products over-the-air: Deploy updates to products in the field,  gather performance insights and evolve capabilities over time with monetization opportunities.
• Predict parts and service issues: Detect operating thresholds, anomalies and predict failures to proactively recommend service using AI, reducing warranty claims, decreasing parts shortages and increasing customer satisfaction.

In order to help manufacturers quickly deploy these use cases and many more, Intelligent Products Essentials provides the following:

• Edge connections: Connect and ingest raw or time-series product telemetry from various device platforms utilizing IoT Core or Pub/Sub and enable deployment and management of firmware over-the-air and machine learning models with Vertex AI at the edge.
• Ownership App Template: Easily build connected product companion apps that work on smartphones, tablets, and computers. Use a pre-built API and accompanying sample app that can incorporate product or device registration, identity management, and provide application behavior analytics using Firebase.
• Product fleet management: Manage, update and analyze fleets of connected products via APIs, Google Kubernetes Engine, and Looker.
• AI services: Create new features or capabilities for your products using AI and machine learning products such as DialogFlow, Vision AI, AutoML, all from Vertex AI.

Enterprise data integration: Integrate data sources such as Enterprise Asset Management (EAM), Enterprise Resource Planning (ERP), Customer Relationship Management (CRM) systems and others using Dataflow and BigQuery.

Intelligent Products Essentials helps manufacturers build new features across consumer, industrial, enterprise, and transportation products. Manufacturers can implement the solution in-house, or work with one of our certified solution integration partners like Quantifi and Softserve. 

“The focus on intelligent products that Google Cloud is deploying provides a digital option for manufacturers and users. At its heart, systems like Intelligent Product Essentials are all about decision making. IDC sees faster and more effective decision-making as the fundamental reason for the drive to digitize products and processes. It’s how you can make faster and more effective decisions to meet heightened customer expectations, generate faster cash flow, and better revenue realization,” said Kevin Prouty, Group Vice President at IDC. “Digital offerings like Google’s Intelligent Product Essentials potentially go the last mile with the ability to connect the digital thread all the way through to the final user.”

Customers adopting Intelligent Products Essentials

GE Appliances, a Haier company, are enhancing their appliances using new AI-powered intelligent features to enable:

• Intelligent cooking: Help cook the perfect meal to personal preferences, regardless of your expertise and abilities in the kitchen.
• Frictionless service: Build smart appliances that know when they need maintenance and make it simple to take action or schedule services.
• Integrated digital lifestyle: Make appliances useful at every step of the way by integrating them with digital lifestyle services – for example, automating appliance behaviors according to customer calendars, such as oven preheating or scheduling the dishwasher to run in the late evening.

“Intelligent Products Essentials enhances our smart appliances ecosystem, offering richer consumer habit insights. This enables us to develop and offer new features and experiences to integrate with their digital lifestyle.“ —Shawn Stover, Vice-president Smart Home Solutions at GE Appliances.https://www.youtube.com/embed/zaWAJN8aKOw?enablejsapi=1&

Serial 1, Powered by Harley-Davidson, is using Intelligent Product Essentials to manage and update its next generation eBicycles, and personalize its customers’ digital ownership experiences. 

“At Serial 1, we are dedicated to creating the easiest and most intuitive way to experience the fun, freedom, and adventure of riding a pedal-assist electric bicycle. Connectivity is a key component of delivering that mission, and working together to integrate Intelligent Product Essentials into our eBicycles will ensure that our customers enjoy the best possible user experience.”— Jason Huntsman, President, Serial 1. 

Magic Leap, an augmented reality pioneer with industry-leading hardware and software, is building field service solutions with Intelligent Products Essentials with the goal of connecting manufacturers, dealers, and customers to more proactive and intelligent service.

“We look forward to using Intelligent Products Essentials to enable us to rapidly integrate manufacturers’ product data with dealer service partners into our field service solution. We’re excited to partner with Google Cloud as we continue to push the boundaries of physical interaction with the digital world.” — Walter Delph, Chief Business Officer, Magic Leap

Intelligent Product Essentials is available today. To learn more, visit our website.

More than 3 billion users stay in touch, share ideas, and get more done together in Google Workspace, across apps like Meet, Chat, Drive, Docs, Sheets, Slides, and more. And starting today, we’re rolling out a new version designed to help people bring the apps they know and love to use in their personal lives to their work life. The new Google Workspace Essentials Starter Edition is a no-cost solution for business users looking to enhance teamwork and unlock innovation with secure-by-design collaboration. With Essentials Starter, we’re making it easy for employees to choose their own productivity tools and bring modern collaboration to work.

Work easily and securely within your current environment

As the world of work continues to evolve in unprecedented ways, secure collaboration—at speed, at scale, across platforms, on multiple devices—has never been more important. Essentials Starter helps employees and their teams break down silos and work together in new ways, even if their organization still relies on legacy productivity tools that weren’t built for the hybrid era of work. There’s no need for a new email address, file conversions, new plug-ins, or desktop software. All of the tools in Essentials Starter will work quickly and easily within your existing environment. And because we designed Google Workspace to operate on our industry-leading cloud foundation, Essentials Starter provides encrypted and secure access to files, helping keep users safe and their information private.

How Essentials Starter works

1. Sign up with your work email for a no-cost Essentials Starter account. No credit card required and there’s no limited trial period.
2. Invite teammates to collaborate using Google Docs, Slides, Sheets, Chat, Drive, and Meet.
3. Unlock innovation with new ways of working with immersive, virtual meetings, and by storing, sharing, and collaborating on 100+ file types, including Microsoft Office, without the need for file conversions.

Collapse the boundaries between people, places, and timezones

In our recently commissioned hybrid work global survey with Economist Impact, when workers were asked what was most important to realize the benefits of hybrid work, their number one choice was “new technologies that allow for time and location flexibility.”*

Google Workspace was born in the cloud and designed for maximum flexibility; to be used from the location and device of your choice. Google Workspace brings chat, files, meetings, and your favorite apps into an intelligent hub so you—and your team—can connect, create, and collaborate from your PCs, tablets, and smartphones, across locations and time zones.

Modern collaboration capabilities

Using the cloud-first approach of Google Workspace, teams can:

• Experience modern collaboration without the burden of version control or email attachments. Brainstorm, build presentations, and glean insights from data using Google Sheets, Slides, and Docs.
• Host secure video meetings to bring everyone together in one place. Conduct one-to-one video meetings (unlimited) and immersive team meetings (3-100 people; up to 60 minutes in duration each).
• Real-time messaging with Google Chat. Share quick updates, files, and high-fives across the team with Google Chat, from one-to-one messages to larger group conversations.
• A dedicated place for team collaboration in Spaces. Spaces are tightly integrated with Google Workspace tools like Chat, Drive, Docs, Sheets, Slides, and Meet, providing a better way for people to engage in topic-based discussions, share knowledge and ideas, move projects forward, and build communities and team culture.
• Access work content securely from your devices. With 15 GB of storage in Google Drive, you can store, share, and access work content from your mobile device, tablet, or computer, and Drive for desktop can enable you to sync files and folders to your PC or Mac. Your Drive storage limits remain separate across your personal and Google Workspace Essentials Starter user accounts.
• Work with existing tools without file conversions. Store, share, or co-edit 100+ file types, including Microsoft Office documents and PDFs without the need for file conversions.
• Easily manage your team of collaborators. Add and remove team members effortlessly using a simple dashboard.

Over the past two years, organizations around the world have relied on Google Workspace to reimagine how work happens and enable modern collaboration across their teams to unlock innovation. During a time of enormous uncertainty, our customers have launched game-changing products, reached new audiences, accelerated lifesaving research, and digitally transformed their own customer experiences. Along the way, they’ve used Google Workspace to empower their employees—across homes, offices, and on the frontlines—to connect, create, and collaborate in powerful new ways. And starting today, with Essentials Starter, employees can make the switch from legacy productivity tools and experience the difference with Google Workspace.

Get started today.

^{* Economist Impact survey commissioned by Google in October 2021. The most popular answer when workers were asked about the most important condition needed to achieve the long-term success of hybrid work models was based on 42% of 1,244 respondents.}

Did you know API traffic for Apigee customers increased 46% year-over-year, to 2.21 trillion calls, between 2019 and 2020? If you have more questions on APIs trends and their role in disrupting enterprises digitally, you can catch up on key take aways from the Google Cloud’s State of API Economy 2021 report and insights from Google’s experts on API-best practices.