Analytics Hub for Secure Data Sharing and Analytics Unlocks True Data Value and Insights

6736
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
Customers tell us that sharing and exchanging data with other organizations is a critical element of their analytics strategy, but it’s hamstrung by unreliable data and processes, and only getting harder with security threats and privacy regulations on the rise.
Furthermore, traditional data sharing techniques use batch data pipelines that are expensive to run, create late arriving data, and can break with any changes to the source data. They also create multiple copies of data, which brings unnecessary costs and can bypass data governance processes. These techniques do not offer features for data monetization, such as managing subscriptions and entitlements. Altogether, these challenges mean that organizations are unable to realize the full potential of transforming their business with shared data.
To address these limitations, we are introducing Analytics Hub, a new fully managed service, available in Q3, in preview, that helps you unlock the value of data sharing, leading to new insights and increased business value. With Analytics Hub you get:
- A rich data ecosystem by publishing and subscribing to analytics-ready datasets.
- Control and monitoring over how your data is being used, because data is shared in one place.
- A self-service way to access valuable and trusted data assets, including data provided by Google. For example, a unique dataset from Google Search Trends will be available, that you can query and combine with your own data.
- An easy way to monetize your data assets without the overhead of building and managing the infrastructure.
Built on a decade of cross-organizational sharing
While Analytics Hub is a new service, it builds on BigQuery, Google’s petabyte-scale, serverless cloud data warehouse. BigQuery’s unique architecture provides separation between compute and storage, enabling data publishers to share data with as many subscribers as you want without having to make multiple copies of your data. With BigQuery, there are no servers to deploy or manage, which means that data consumers get immediate value from shared data. Data can be provided and consumed in real-time using the streaming capabilities of BigQuery and you can leverage the built in machine learning, geospatial, and natural language capabilities of BigQuery or take advantage of the native business intelligence support with tools like Looker, Google Sheets, and Data Studio.
BigQuery has had cross-organizational, in-place data sharing capabilities since it was introduced in 2010. We took a look at usage metrics in BigQuery and found that over a 7 day period in April, we had over 3,000 different organizations sharing over 200 petabytes of data. These numbers don’t include data sharing between departments within the same organization.

As you can see, data sharing in BigQuery is already popular. But we want to make it easier and even more scalable.
Raising the bar on data sharing
To make data sharing easier and more scalable in BigQuery, Analytics Hub introduces the concepts of shared datasets and exchanges. As a data publisher, you create shared datasets that contain the views of data that you want to deliver to your subscribers. Next, you create exchanges, which are used to organize and secure shared datasets. By default, exchanges are completely private, which means that only the users and groups that you give access to can view or subscribe to the data. You can also create internal exchanges or leverage public exchanges provided by Google. Finally, you publish shared datasets into an exchange to make them available to subscribers.
Data subscribers search through the datasets that are available across all exchanges for which they have access and subscribe to relevant datasets. This creates a linked dataset in their project that they can query and join with their own data. Subscribers pay for the queries that they run against the data while the publisher pays for the storage of the data. Data providers can add new data, new tables, or new columns to the shared dataset and these will be immediately available to subscribers. In addition, the publisher can track subscribers, disable subscriptions, and see aggregated usage information for the shared data.
Analytics Hub makes it easy for you to publish, discover, and subscribe to valuable datasets that you can combine with your own data to derive unique insights. Here are some types of data that will be available through Analytics Hub:
- Public datasets: Easy access to the existing repository of over 200 public datasets, including data about weather and climate, cryptocurrency, healthcare and life sciences, and transportation.
- Google datasets: Unique, freely-available datasets from Google. One example of this is the COVID-19 community mobility dataset. Another example is the forthcoming Google Trends dataset, which will provide the top 25 search terms and top 25 rising search terms over a 5 year window in 210 distinct locations in the US. Trends data can be used by everyone in the organization to gain insights into what customers care about.
- Commercial (paid for) datasets: We are working with leading commercial data providers to bring their data products to Analytics Hub. If you are interested in delivering your data via Analytics Hub, we’re also introducing Data Gravity, an initiative that provides storage benefits and new distribution paths for data published through Analytics Hub.
- Internal datasets: We know that data sharing can be challenging in larger organizations. Analytics Hub can be used for internal data, for example, to share standardized customer demographics with your sales engineering and data science teams.
Customers and partners using Analytics Hub

“Google Search Trends data has always been an important tool for our WPP agency data teams. At WPP we believe that data variety is a superpower which is why we are excited to use the new Trends dataset availability within BigQuery, plus the launch of Analytics Hub. The best creativity in the world is informed by data insights, and influenced by what people search for, so the operational efficiencies we’ll gain via the Analytics Hub and the insights we can drive with Trends data are just phenomenal.”
—Di Mayze Global Head of Data and AI, WPP

“Equifax Ignite is our shared data analytics environment within our Equifax data fabric. We are excited to partner with Google to leverage Analytics Hub and BigQuery to deliver data to over 400 statisticians and data modelers as well as securely sharing data with our partner financial institutions.”
—Kumar Menon, SVP Data Fabric and Decision Science, Equifax

“The flow of data and insights between our teams at Deloitte and our clients is paramount for building truly transformational data cultures. With its purpose-built architecture for secure data exchanges and sharing analytics resources, Google Cloud’s Analytics Hub can help provide significant operational efficiencies for how Deloitte teams support our clients’ data-driven initiatives within their industry ecosystems. It will also help minimize the worries about scale, privacy and security, or the administrative burden associated with each.”
—Navin Warerkar, Managing Director, Deloitte Consulting LLP, and US Google Cloud Data & Analytics GTM Lead

“Crux Informatics is proud to partner with Google to support the launch of Analytics Hub, removing friction for those who need access to analytics-ready data. With thousands of datasets from over 140 sources, Crux Informatics will accelerate access to data on Analytics Hub and together provide a more efficient and cost effective solution to deliver datasets in Google Cloud’s ecosystem.”
—Will Freiberg, CEO, Crux Informatics
Next steps for Analytics Hub
This is just the beginning for Analytics Hub. As we get to preview and general availability, we will be adding additional capabilities, including workflows for publishing and subscribing, publishing analytics assets (Looker Blocks, Data Studio reports, Connected Google Sheets) along with the shared data, the ability for data publishers to specify query restrictions on the usage of their data, and making it easy for data publishers to create sandbox environments for subscribers to work with their data, even if they are not yet on Google Cloud. We will provide features in Analytics Hub for monetization of data, including managing subscriptions, data entitlements, and billing.
Please sign up for the preview, which is scheduled to be available in the third quarter of 2021. In the meantime, you can learn more about BigQuery and how to leverage its built-in data sharing capabilities. Please go to g.co/cloud/analytics-hub to register your interest in Analytics Hub.
How Google Cloud & NGIS’ Partnership Powers Sustainability & Responsible Sourcing for Consumer Brands

6196
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
In the competitive world of consumer goods, sustainability matters more than ever. In recent Google survey, 82% of consumers said sustainability is more top of mind now than it was before COVID-191 and 78% said that big businesses have a role to play in helping to fight climate change.2 As well as delivering on customers’ heightened expectations, sustainable business practices can help organizations reduce waste and lower operational costs and even help attract top talent.
Not only is sustainability good for business performance and the bottom line, it’s great for the planet, too. And it’s something we care deeply about at Google. Since our earliest days, we have focused on developing services that significantly improve the lives of billions of people while operating our business in an environmentally sustainable way. In 2007, we were the first major company to become carbon neutral, and in 2017 we were the first to achieve 100% renewable energy. Today, we proudly operate the cleanest cloud in the industry. We’ve also been powering humanitarian, scientific, and environmental initiatives studying geospatial information and making it available for analysis via Google Earth Engine.
Powering the future of responsible sourcing
How does all this relate to CPG supply chains? The world relies on raw materials like palm oil, soy, and cocoa to produce the items we consume every day—such as coffee, chocolate, frozen foods, shampoo, toothpaste, cosmetics, and even household cleaning products. Yet as demand for these materials continues to grow, forests are under threat. Change is needed.
As sustainability moves further into the spotlight, in-demand crops face increased supplier scrutiny. Environmentally conscious companies want to know what percentage of their raw materials is sourced from deforestation-free suppliers—and how they can improve that number. Until recently, many CPG brands have found it hard to get real-time, reliable visibility into operations at a local supplier level, globally.
Google Cloud, in partnership with NGIS, is helping brands gain a deeper understanding of raw material sourcing practices across supplier networks, so they can improve supplier performance and compliance in the fight against deforestation. The TraceMark solution, developed by NGIS, uses Google Earth Engine and BigQuery to analyze and visualize how suppliers behave over time.
Google Earth Engine is the world’s largest archive of open Earth data. It combines a multi-petabyte catalog of satellite imagery and geospatial datasets with planetary-scale analysis capabilities to help people detect changes, map trends, and quantify differences on the Earth’s surface.With Earth Engine, organizations can analyze the potential social, and environmental impacts of the decision they make.
Together, TraceMark, Google Earth Engine and BigQuery help companies visualize, monitor, and measure the impact of suppliers’ farming practices. The process starts with Earth Engine, which aggregates and harmonizes planetary data into images that it has been collecting for decades. Then, boundary maps for different suppliers are created and NGIS applies climate data science and machine learning to turn pixels into insights. Finally, BigQuery and Vertex AI -Google Cloud’s unified artificial intelligence platform are used to produce supplier scoring, which is integrated into downstream ERP and procurement systems to support more informed decision-making.
These efforts support the responsible sourcing of raw materials by making supply chains more agile, transparent and traceable.

“NGIS is thrilled to work with Google Cloud and its partners to enable business accountability for sustainable practices at all levels of their supply chain,” said Nathan Eaton, Executive Director, NGIS. “Google Earth Engine provides unique geospatial capability that [gives] leaders visibility and control over their environmental footprint and that of their suppliers in a way that was not previously possible.”
Unilever commits to working towards a deforestation-free supply chain
Unilever is a great example of a company committed to using technology in the quest to become more sustainable. Since 2020, Unilever has partnered with Google Cloud to use data for eco-friendly decision-making, particularly when it comes to sustainable commodity sourcing.
By combining the power of cloud computing with satellite imagery and AI, we’re helping Unilever build a more holistic view of the forests, water cycles, and biodiversity that intersect its supply chain. By gaining a complete picture of these ecosystems, Unilever can detect deforestation while simultaneously prioritizing critical areas of forest and habitats in need of protection.
The cleanest cloud for your sustainable transformation
We are excited to bring Google Earth Engine and NGIS to our CPG customers to help them meet their sustainability goals. Looking ahead, we’re committed to furthering our own ambitious sustainability goals and empowering CPG brands with the technology to do more for our environment and our shared future.
Contact your cloud seller to learn more about Tracemark sustainable sourcing solutions and how Google Cloud can help you advance your sustainability initiatives.
Explore TraceMark on the Google Cloud Marketplace
To know more about how we are helping CPGs transform digitally read this ebook
1. Google/C Space, BR, FR, DE, IN, MX, U.K., U.S., qualitative survey activity, n = 528, Nov. 24–Nov. 26, 2020.
2. Google/Ipsos, Google Sustainability, BR, FR, DE, IN, JP, U.K., U.S., n=16,959 online population 18–70, July 2021.
An Expert’s Opinion on What Early-stage Startups Must Know

6448
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
As lead for analytics and AI solutions at Google Cloud, my team works with startups building on Google Cloud. This puts us in the fortunate position to learn from founders and engineers about how early-stage startups’ investments can either constrain them or position them for success, even at the seed level. In this post, I want to share a few of the best practices to keep in mind as you’re building.
Understand your value proposition before diving into a technology stack
If you’re launching a startup in the cloud, you’re no doubt thinking about a technology stack, but it’s important to step back a bit and think carefully about the major value proposition that your startup offers to your customers. That value proposition is going to fundamentally drive the kind of technology that you should pick.
For example, does your system need processing in real time, or can it be done in a batch mode? Can you rely on once-a-day insights or do the insights have to come in as events happen?
Additionally, what kind of latency will your customers face? That latency makes your value proposition either usable or unusable. Early on in Google’s development, leaders realized that no one was going to wait more than a few hundred milliseconds for a web page to show them their results, and that realization drove the technology decisions that have allowed Google to scale from being a startup in a garage to being a trillion dollar company. Your startup needs to define its value to customers with this level of specificity before it can build a technology stack suited to its needs.
Focus on customer interactions
A few companies have gracefully pulled off big IT pivots that reshaped their value proposition. Netflix, for example, moved from mostly sending DVDs through the mail to becoming a streaming service and major content producer. That’s a huge shift in the user experience and the technology stack necessary to support it, even if the underlying value proposition (i.e., get content to customers) was broadly the same. But it’s also an outlier. If you’re planning for potential changes of this magnitude, rather than focused on getting your value proposition to users, you probably need to sharpen what that value proposition is.
Specifically, you need a clear vision of how customers will access and interact with your business. Typically, they’ll do so over a website or a mobile app, but there are still so many variables.
Are customers going to transmit documents? If so, in what format? Is handwriting supported or is input limited to typing? Can they use images for optical character recognition? Will it mostly be forms? Will the data be structured or unstructured? If all that sounds a little overwhelming, don’t worry, it’ll seem simpler by the end of this article—but also be aware: we’re just getting warmed up.
Imagine that most of your customers will access your business via voice, so you know you’ll want to prioritize conversational workflows. That’s a start—but dig deeper. Even if we suppose you’re usingDialogflow, a Google Cloud conversational AI platform that lets you build and deploy virtual agents, we’re still not really seeing the value proposition. How will all this work, from the beginning of a typical full customer interaction to the resolution? How many interactions will have to be facilitated over low-bandwidth connections, for example? When it comes to user interactions, make sure you can see an end-to-end use case.
Another example: you’re building a retail website, and one of your end-to-end use cases involves the customer asking if a certain amount of a given product is in stock, whether it’s one unit of the product, ten or hundreds. If the product is not sufficiently stocked, you want your app to offer similar items that are. Will your technology stack support this end-to-end use case?
These considerations are not an argument for premature optimization. There’s value in moving fast, getting minimum viable products to users, and then iterating. But in the early stages, you only get one chance to start on the right foot—and how you navigate that chance will influence a lot of dollars and effort down the road. You need to make sure you have business use cases, not just an idea, before you can start designing a technology stack.
Here’s how to get in the right frame of mind. Pick three use cases: two that are “bread and butter” and one that is technologically complex. Make sure your proposed technology stack can support all three, end to end.
Default toward higher levels of abstraction
Now that we’re in the right frame of mind, we’re ready to think about the technology stack more directly.
As a startup, you’ll need to conserve resources, and to do that, you’ll want to build at the highest level of abstraction possible for your value proposition. For example, you probably don’t want your people setting up clusters. You don’t want them configuring things if they can use a fully managed service. You want them focused on building your prototype, not managing infrastructure.

This focus has definitely informed how we create products at Google Cloud, as our canonical data stack—Pub/Sub, Dataflow, BigQuery, and Vertex AI—consists of auto-scaling and serverless products.
But management of infrastructure is not the only place where you should err toward a less-is-more philosophy.
When it comes to architecture, choose no-code over low-code and low-code over writing custom code. For example, rather than writing ETL pipelines to transform the data you need before you land it into BigQuery, you could use pre-built connectors to directly land the raw data into BigQuery. That’s no code right there. Then, transform the data into the form you need using SQL views directly in the data warehouse. This is called ELT, and it is low code. You will be a lot more agile if you choose an ELT approach over an ETL approach.
Another place is when you choose your ML modeling framework. Don’t start with custom TensorFlow models. Start with AutoML. That’s no-code. You can invoke AutoML directly from BigQuery, avoiding the need to build complex data and ML pipelines. If necessary, move on to pre-built models from TensorFlow Hub, HuggingFace, etc. That’s low-code. Build your own custom ML models only as a last resort.

Focus on getting your vision to market, not chasing technology hype
The goal is to pick the right technology stack for bringing your vision to market, generating value for customers, conserving resources, and maintaining flexibility for growth. Early IT investments should usually gravitate toward things that preserve flexibility, such as managed services built on standard protocols or open APIs, but they needn’t always rush to the flashiest technologies. The answer isn’t always ML, for example. The answer might be heuristics to start, with a path to ML once you have collected enough data. You want to make sure that your intelligence layer has enough abstraction so you can mark it up with simple rules at first, but then replace it with a more robust system as you go along.
Launch and iterate fast with these principles
The preceding discussion is a reminder that your most expensive resource is your people—and that you really want them to be focused on building your prototype, minimum viable product or production app You want to launch fast and iterate fast, and the only way you can do that is by focusing on the things that differentiate you.
But regardless of the technologies you use, the bottom line is the same: follow these four principles.
- Figure out your major value proposition and design your tech stack around it.
- Be very careful about user interactions. User experience is super important; you need to make sure you deliver the kind of experience that your customers have grown to expect.
- When you’re building, pick the highest possible level of abstraction possible—the most fully managed tools and no-code/low-code frameworks that give you the functionality that you need.
- Instead of choosing new or flashy technologies, consider if you can build a “good enough” minimum viable product quickly and come back to a better implementation later.
To learn more about why startups are choosing Google Cloud, click here.
Latest Features and Updates to Globally Bolster Translation Services

5288
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Let’s face it: in the globalized world, which is now more than ever a digital demand world, you need to scale and reach your customers right where they’re at. Translation is a critical piece of that, whether you’re translating a website in multiple languages or releasing a document, a piece of software, or training materials.
Manual translation does not scale, which is why machine translation, powered by machine learning (ML), is becoming more important to our customers. Machine translation has historically been challenging because of the sheer volume and breadth of content that can add value when translated into multiple languages. Companies acquire and share content in many languages and formats, and scaling translation to meet needs is a tall order due to multiple document formats, integrations with optical character recognition (OCR), and the need to correct for domain-specific terminology.
Our goal is to simplify translation services, while enabling flexibility and control for our customers’ unique needs across industries. Read on to learn more about recent features and updates.
Formatting matters: Document Translation is now GA
In many cases, the layout of a document dictates how it should be interpreted—e.g., readers navigate text and discern meaning based on formatting, like bold or italicized text, or markups for headers, paragraphs, and columns. Previously, to automate translation of documents, text needed to be separated from these layout attributes, meaning the document’s structure was either lost or needed to be recreated later in the developer pipeline, after the text had been translated. This required translation teams to do a lot of extra work and maintain a lot of additional code. But now, those steps are unnecessary. Formatting can be retained throughout the translation process, handled directly by the Translation API Advanced.
This feature lets customers translate documents in 100+ languages and supports document types such as Docx, PPTx, XLSx, and PDF while preserving document formatting.
And if your needs go beyond Document Translation, we can help you translate audio as well. For real-time streaming translation, check out the Media Translation API, and for offline transcription translation, combine the Translation API with the Video Intelligence API.
Real-Time translation when you need it, Batch when you don’t
One of the biggest differentiators for Translation API Advanced’s document translation capabilities is the ability to do real-time, synchronous processing for a single file.
For example, if you are translating a business document such as HR documentation, online translation provides flexibility for smaller files and provides faster results. You can easily integrate with our APIs via REST or gRPC with mobile or browser applications, with instant access to 100+ language pairs so that content can be understandable in any supported language.
Meanwhile, batch translation allows customers to translate multiple files into multiple languages in a single request. For each request, customers can send up to 100 files with a total content size of up to 1 GB or 100 million Unicode codepoints, whichever limit is hit first.
State of the Art (SOTA) accuracy, with flexibility for customization
In order to achieve the highest level of accuracy for your translation, we now support multiple options:
- Use Google’s SOTA translation models: Each year, Google heavily invests to improve the quality of our translations across Apps, Cloud APIs, and Chrome, as well to enable multilanguage answers in Search. A popular metric for automatic quality evaluation of Machine translation systems is the BLEU score, which is based on the similarity between machine translation and the reference translations that were generated by people. While we push out incremental improvements for individual models on a monthly cadence, there are also times where we make significant leaps. In the releases since 2019, we have improved our average BLEU score by 5pts on average across 100+ languages and 7pts on low resource languages.
- Leverage glossaries for specific terms and phrases: Glossary is our terminology control feature. It allows you to import source content to define preferred translations, such as product names or department names. Then, when calling the glossary in the API request, your preferred translations will be enforced. This will work for words as well as phrase translation.
- Pick a pre-trained model with model selection: If you create custom models for machine translation, we don’t think you should have multiple client libraries and multiple APIs to maintain in order for you to use the best model for your needs. Translation API Advanced now supports Model Selection. Pick your pretrained model or pick your custom ML model built on AutoML for any language pair you’ve created and use the same API and the same client library.
- Build custom translation models with AutoML: AutoML Translation is a suite of ML products that enable you to build high quality models for your own use case or data, with limited-to-no ML expertise or coding required. Bring your past human-validated translations to improve translation specificity for your domain.
Keep localization local with Regional Endpoints
If you are a customer operating in the EU, we recently launched an endpoint specifically for EU regionalization. This is a configurable endpoint for customers to store and perform machine translation processing of customer data only in the EU multi region. For now, this only supports our pretrained translation models and glossary, but batch translations will be coming soon.
How Eli Lilly uses Cloud Translation to translate content globally
Historically, translations at Eli Lilly have been complicated: numerous translation vendors have been needed for different languages and organizations, all with their own processes and expectations. On top of that, translations have been costly and slow.
To solve this, Eli Lilly took a codified approach to enable users and systems to spend less time and resources to safely generate quality translations.
Learn more, and even catch a demo, from Thomas Griffin, Translation Tech Lead & Global Regulatory Architect for Eli Lilly.

Learn more
- To get started using Cloud Translation – Advanced, complete the setup and then try the Translate text (Advanced edition) quickstart.
- Document Translation is priced per page. For more information, see pricing.

TPUs Can Cut Deep Learning Costs by upto 80%—and Other Things You Didn’t Know About TPUs
DOWNLOAD E-BOOK3518
Of your peers have already downloaded this article
6:15 Minutes
The most insightful time you'll spend today!
The Tensor Processing Unit (TPU) is a custom ASIC chip—designed from the ground up by Google for machine learning workloads—that powers several of Google’s major products including Translate, Photos, Search Assistant and Gmail.
Cloud TPU provides the benefit of the TPU as a scalable and easy-to-use cloud computing resource to all developers and data scientists running cutting-edge ML models on Google Cloud.
But what is a TPU, how is it different from CPUs and GPUs, and how much does it lower cost by?
Download the whitepaper, What Makes TPUs Fine-tuned for Deep Learning?, to find out:
- Back to the basics: How CPUs and GPUs work
- The difference between CPUs, GPUs and TPUs
- Why TPUs are best-suited for deep-learning workloads
- Cost-benefit analysis: How much you can save by leveraging TPU-architecture
3959
Of your peers have already watched this video.
1:30 Minutes
The most insightful time you'll spend today!
90-Sec Demos of Truly Incredible Customer Care Calls Powered By AI
How real, fast, and easy do you think an AI-powered call center experience is for customers today?
It’s likely you have seen AI-powered chatbots and are deeply unimpressed with the static nature of the interaction between customers and service representatives.
But that’s not how Google Cloud’s AI-powered contact centers are.
Here have a look yourself. Here’s what a customer from a bank or financial organisation would experience.
And here’s what a customer from a telecom company would experience.
More Relevant Stories for Your Company

Building Unique Customer Experiences with Speed & Scale: Sprinklr & Google Cloud
Enterprises are increasingly seeking out technologies that help them create unique experiences for customers with speed and at scale. At the same time, customers want flexibility when deciding where to manage their enterprise data, particularly when it comes to business-critical applications. That’s why I’m thrilled that Sprinklr, the unified customer

Supercharge Marketing with Ready-Made, Centralized Smart Analytics
Your company has lots of valuable marketing data, but it’s spread across many systems and teams. That's a problem. In fact, only 13% of organisations feel like they're making the most out of their available customer data today. And the reason is that data--marketing and customer data--is spread across many

Smart analytics: Deep dive on roadmap
Data across organizations is growing and that organizations need a very strong analytics platform to leverage this data create insights and make real-time decisions on top of this data. That’s driving the advent of three large trends. First is the convergence of data lakes and data warehouses, that’s enable organizations

An AI-Powered Cost Cutting Guide: 8 Strategies for Maximizing Profits
We are increasingly seeing one question arise in virtually every customer conversation: How can the organization save costs and drive new revenue streams? Everyone would love a crystal ball, but what you may not realize is that you already have one. It's in your data. By leveraging Data Cloud and






