AI researchers and engineers need better data to enable better AI solutions. The quality of an AI solution is determined by both the learning algorithm (such as a deep-neural network model) and the datasets used to train and evaluate that algorithm. Historically, AI research has focused much more on algorithms than datasets, despite their vital importance. As a result, many algorithms are freely available as starting points, but many important problems lack large, high-quality open datasets. Further, creating new datasets is expensive and error-prone.

Recently, the data-centric AI movement has emerged, which aims to develop new methodologies and tools for constructing better datasets to fix this problem. Conferences, workshops,  challenges, and platforms are being launched to support improving data quality and to foster data excellence. Thought leaders such as Andrew Ng at Landing.AI and Chris Re at Stanford University are encouraging AI developers to focus more on iterative data engineering than they do tuning their learning algorithms. Our CHI-best-paper-award-winning paper, “Everyone wants to do the model work, not the data work” highlighted the significance of data quality in the practice of ML. 

At Google, we are excited to contribute to data-centric AI. Today, Google Cloud is adding a new high value dataset to the Public Dataset Program, and Google researchers are announcing DataPerf, a new multi-organizational effort to develop benchmarks for data quality and data centric algorithms.

Google Cloud is committed to helping users improve their data quality, starting with supporting better public data. The Public Datasets program provides high quality datasets pre-configured on GCP for easy access. Google Cloud is adding a new high-value dataset developed by the MLCommons™ Association (which Google co-founded) to the Public Datasets program: The Multilingual Spoken Words Corpus: a rich audio speech dataset with more than 340,000 keywords in 50 languages with upwards of 23.4 million examples.

This new public dataset is aligned with the MLCommons Association vision for “open” datasets – accessible by all – that are “living” – continually being improved to raise quality and increase representation and diversity.

Google researchers, in collaboration with multiple organizations, are announcing the DataPerf effort at the NeurIPS Data-Centric AI workshop today, to develop benchmarks to improve data quality. Much like the the MLPerf™ benchmarking effort which is now the industry standard for machine learning hardware/software speed, DataPerf brings together the originators of prior efforts including: CATS4ML, Data-Centric AI Competition, DCBench, Dynabench, and the MLPerf benchmarks to define clear metrics that catalyze rapid innovation. DataPerf will measure the utility of training and test data for common problems, and algorithms for working with datasets such as: selecting core sets, correcting errors, identifying under-optimized data slices, and valuing datasets prior to labeling.

Together, supporting open, living datasets for core ML tasks, and the development of benchmarks to direct the rapid evolution of those datasets will empower the researchers and engineers who use Google Cloud to do even more amazing things – and we can’t wait to see what they create!

Acknowledgements: In collaboration with Lora Aroyo and Praveen Paritosh.

Introduction

In my experience, I have observed that it’s common in machine learning to surrender to the process of experimenting with many different algorithms in a trial and error fashion, until you get the desired result. My peers and I at Google have a People and Planet AI YouTube series where we talk about how to train and host a model for environmental purposes using Google Cloud and Google Earth Engine. Our focus is inspiring people to use deep learning, and if we could rename the series, we would call it AI for Minimalists since we would recommend artificial neural networks for most of our use cases. And so in this episode we give an overview of what deep learning is and how you can use it for tracking deforestation in supply chains. I also included a summary of the architecture and products you can use in this blog that I presented at the 2022 Geo For Good Summit. For those of you interested in diving even deeper into code, please visit our end-to-end sample (click “open in colab” at the bottom of the screen to view this tutorial in a notebook format).

What’s included in this article

• What is Deep Learning?
• Measuring deforestation in extractive supply chains with ML
• When to build a custom model outside of Earth Engine?
• How to build a model with Google Cloud & Earth Engine?
• Try it out!

What is Deep Learning?

Out of the many ML algorithms out there, I’m happy to share that deep learning or artificial neural networks is a technique that can be used for almost any supervised learning job.

In supervised learning, you tell a computer the right answers to look for, through examples. Deep learning is very flexible, and is a great go-to algorithm. Especially for images, audio, or video files which are types of multidimensional data. This is because each of these data types have one or more dimensions with specific values for each point.

And training a model to classify tree species using satellite images is kind of like an image segmentation problem, where every pixel in the image is classified.

“Deep learning approaches problems differently”

David Cavazos, Developer Programs Engineer

There’s no writing a function with explicit & sequential steps that reviews every single pixel one by one for every image, as traditional software development does. Let’s say you wish to build a model that classifies tree species. You don’t spend time coding all the instructions, but instead give a computer examples of images with tree species labels, and let it learn from these examples. And when you want to add more species, it’s as simple as adding new images of that species to retrain the model.

Measuring deforestation in extractive supply chains with ML

So let’s say we would like to measure deforestation using deep learning; to get started with building a model we first need a dataset that includes satellite images with an even amount of labels marking where there are trees and where there aren’t. Next, we choose a goal, here are a few common ones. In our case, we simply want to know if there are trees or not for every pixel, and so this would be a binary semantic segmentation problem.

And based on this goal, we expect the outputs to be the percentage of trees for every pixel; as a number between 0 and 1. Zero represents no trees, and one represents a high confidence there are trees.

But how do we go from input images into probabilities of trees? Well think about it this way…there are many ways to approach this problem, here are 3 common ways of doing so. My peers and I prefer using Fully convolutional networks when building a map with ML predictions

And since a model is a collection of interconnected layers, we must come up with an arrangement of layers that transforms our data inputs based on our desired outputs. Each layer by the way has something called an activation function, which performs the transformations of each layer before it passes them to the next layer.

FYI Below is a handy dandy table, with our recommended activation and loss functions to choose from based on your goal. We hope this saves you time.

We then reach the fourth and last layer. Depending on our goals at the beginning, we also choose an appropriate loss function that helps us score how well the model did during training.

After choosing layers and functions you will split your data into training and validation datasets. Just remember that all of this work is about experimenting repeatedly until you reach desired results. Our 8min episode gives this overview more in detail.

When to build a custom model outside of Earth Engine

So now that we covered what is deep learning, the next step is understanding which tools to use to build our deforestation model. For starters it’s important to call out that Google Earth Engine is a wonderful tool that helps organizations of all sizes find insights about changes on the planet, in order to make a climate positive impact. It has built-in machine learning algorithms (classifiers) that let users quickly spin them up, with just a basic machine learning background. This is fantastic place to start when using ML on geospatial data, however there are multiple situations where you will want to opt to build a custom model such as:

• You want to use a popular ML library such as TensorFlow Keras.
• You wish to build a state of the art model to build a global and accurate land cover map product such as Google’s Dynamic World.
• Or because you generally have too much data to process that you can’t execute it in just one task in Earth Engine (and are trying to figure out hacky ways to export your data).

Whenever you identify with any of these options, you will want to roll up your sleeves and dive into building a custom model, which does require expertise and of course working with multiple products. But I have good news, using deep learning is a great go-to algorithm.

How to build a model with Google Cloud & Earth Engine?

To get started, you will need an account with Google Earth Engine which is free for non-commercial entities and Google Cloud account which has a free tier if you are just getting started for all users. I have broken up the products you would use by function.

If you are interested in looking deeper into this overview, visit our slides here starting from slide 53 and read the speaker notes. Our code sample also walks through how to integrate with all of these projects end to end (just scroll down and click “open in colab”). But here is a quick visual summary. The main place to start is to identify which are the inputs and which are the outputs.

In our latest episodes for our People and Planet AI YouTube series, we walk through how to train a model and then host it in a relatively inexpensive web hosting platform called Cloud Run in episodes of less than 10mins.

There are a few options presented in the slides, however the current best practice is to train a model using Vertex AI. Do note though that Google Earth Engine is currently not integrated with Vertex AI (we are working on this), but it is with the older (ML predecessor) called Cloud AI Platform (which is the recommended ML platform to use moving forward). As such, if you would like to import your model for detecting deforestation back into Earth Engine after training it in Vertex AI for example, you can host the model in Cloud AI Platform and get predictions. Just note that it’s a 24 hour paid service and so it can cost upwards of $100 or more a month to host your model to stream predictions. It also currently supports the following model building platforms if you don’t wish to use TensorFlow.

A cheaper alternative but without the convenience AI Platform offers is to manually translate the model’s output, which is NumPy Arrays into Cloud Optimized GeoTIFFs in order to load it back into Earth Engine using Cloud Run. Within this web service you would store the NumPy arrays into a Cloud Storage bucket, then spin up a container image with GDAL, an open source geospatial library in order to convert them into Cloud Optimized GeoTIFF files into Cloud Storage. This way you can view predictions from your browser or Earth Engine.

Try it out

This was a quick overview of deep learning and what Cloud products you can use to solve meaningful environmental challenges like detecting deforestation in extractive supply chains. If you would like to try it out, check out our code sample here (click “open in colab” at the bottom of the screen to view the tutorial in our notebook format or click this shortcut here).

Vertex AI launched with the premise “one AI platform, every ML tool you need.” Let’s talk about how Vertex AI streamlines modeling universally for a broad range of use cases. 

The overall purpose of Vertex AI is to simplify modeling so that enterprises can fast track their innovation, accelerate time to market, and ultimately increase return on ML investments. Vertex AI facilitates this in several ways. Features like Vertex AI Workbench, for example, speed up training and deployment of models by five times compared to traditional notebooks. Vertex AI Workbench’s native integration with BigQuery and Spark means that users without data science expertise can more easily perform machine learning work. Tools integrated into the unified Vertex AI platform, such as state of the art pre-trained APIs and AutoML, make it easier for data scientists to build models in less time. And for modeling work that lends itself best to custom modeling, Vertex AI’s custom model tooling supports advanced ML coding, with nearly 80% fewer lines of code required (compared to competitive platforms) to train a model with custom libraries. Vertex AI delivers all this while maintaining a strong focus on Explainable AI. 

Yet organizations with the largest investments in AI and machine learning, with teams of ML experts, require extremely advanced toolsets to deliver on their most complex problems. Simplified ML modeling isn’t relegated to simple use cases only.

Let’s look at Vertex AI Neural Architecture Search (NAS), for instance. 

Vertex AI NAS enables ML experts at the highest level to perform their most complex tasks with higher accuracy, lower latency, and low power requirements. Vertex AI NAS originates from the deep experience Alphabet has with building advanced AI at scale. In 2017, the Google Brain team recognized we need a better way to scale AI modeling, so they developed Neural Architecture Search technology to create an AI that generates other neural networks, trained to optimize their performance in a specific task the user provides. To the astonishment of many in the field, these AI-optimized models were able to beat a number of state of the art benchmarks, such as ImageNet and SOTA mobilenets, setting a new standard for many of the applications we see in use today, including many Google-internal products. Google Cloud saw the potential of such a technology and shipped in less than a year a productized version of the technique (under the brand AutoML). Vertex AI NAS is the newest and most powerful version of this idea, using the most sophisticated innovation that has emerged since the initial research.

Customer organizations are already implementing Vertex AI NAS for their most advanced workloads. Autonomous vehicle company Nuro is using Vertex AI NAS, and Jack Guo, Head of Autonomy Platform at the company, states, “Nuro’s perception team has accelerated their AI model development with Vertex AI NAS. Vertex AI NAS have enabled us to innovate AI models to achieve good accuracy and optimize memory and latency for the target hardware. Overall, this has increased our team’s productivity for developing and deploying perception AI models.” 

And our partner ecosystem is growing for Vertex AI NAS. Google Cloud and Qualcomm Technologies have collaborated to bring Vertex AI NAS to the Qualcomm Technologies Neural Processing SDK, optimized for Snapdragon 8. This will bring AI to different device types and use cases, such as those involving IoT, mixed reality, automobiles, and mobile.

Google Cloud’s commitments to making machine learning more accessible and useful for data users, from the novice to the expert, and to increasing the efficacy of machine learning for enterprises are at the core of everything we do. With the suite of unified machine learning tools within Vertex AI, organizations can take advantage of every ML tool they need on one AI platform. 

Ready to start ML modeling with Vertex AI? Start building for free. Want to know how Vertex AI Platform can help your enterprise increase return on ML investments? Contact us.

At Google, we believe anyone should be able to quickly and easily turn their artificial intelligence (AI) idea into reality. Open source software (OSS) has become increasingly important to this goal, heavily influencing the pace of innovation in AI and machine learning (ML) ecosystems. Over the last two decades, ML has transformed Google services including Search, YouTube, Assistant, and Maps, and the basis for this transformation has always been our “open first” approach through investments in projects and ecosystems like TensorFlow, Jax, and PyTorch.

These OSS efforts are important because many AI technologies rely on closed or exclusive approaches. This wall-garden approach creates high barriers to entry for developers; limits efforts to make AI explainable, ethical, equitable; and stunts innovation. We’re committed to open ecosystems, as we firmly believe no one company should own AI/ML innovation. In this blog post, we’ll explore some of Google’s most significant OSS AI and ML contributions from recent years, as well as how our commitment to open technologies can help organizations innovate faster and more flexibly.

Openness is the way to operate as an ecosystem, not a single project

Google’s OSS initiatives extend and enable AI initiatives according to three pillars:

• Access — OSS allows developers, researchers and organizations of all sizes to leverage the latest ML technology. It is a key part of democratizing innovation in ML, fostering software diversity and choice for customers, and lowering operating cost while accelerating scale for everyone.
• Transparency — Open source datasets, ML algorithms, training models, frameworks, and compilers ensure due diligence and validation by the larger community. This is paramount when it comes to ML as it bolsters reproducibility, interpretability, ensures equity, and boosts security.
• Innovation — With more access and transparency, more innovation comes naturally. Our customers and partners take advantage of open source ML toolsets and frameworks to create more innovation in the field by contributing their own OSS.

Google’s ongoing commitment to open source AI

Google’s commitment to open standards spans over two decades of OSS contributions like TensorFlow, JAX, TFX, MLIR, KubeFlow, and Kubernetes, as well as sponsorship for critical OSS data science initiatives like Project Jupyter and NumFOCUS. Initiatives like these have helped Google become the leading Cloud Native Computing Foundation (CNCF) contributor—and by building on these efforts, Google Cloud seeks to be the best platform for the OSS AI community and ecosystem.

The perils of closed technologies can emerge at many points across ML pipelines, which is why Google’s OSS strategy encompasses the entire “idea-to-production” lifecycle, from acquiring data, to training models, to managing infrastructure, to facilitating experimentation and model refinement:

Data acquisition: starting the journey from idea to production-ready ML model

The journey from an idea to a production ML model starts with data. TensorFlow Datasets not only help users acquire ready-to-use, customizable, and highly-optimized datasets (including image, audio, and text), but also provides a set of helpful APIs that make it easy for users to organize their own datasets, regardless of whether they build with TensorFlow, Jax, or other ML frameworks.

Model development and training: shortening the path from data to useful ML

OSS libraries help developers and researchers design, implement, train, test, and debug ML algorithms. Our contributors on this front include:

• The TensorFlow core framework, which offers APIs to help data scientists and developers build and train production-grade ML models on distributed and accelerated infrastructure powered by GPUs or TPUs;
• Google’s founding membership of the PyTorch Foundation, which positions us to increase adoption of ML by building an ecosystem of open-source projects with PyTorch;
• Keras, a simple and powerful ML framework, well integrated with TensorFLow, that makes it easy for developers to quickly build and train ML models, or to leverage pre-trained AI applications;
• Model Garden, which provides implementations of many state-of-the-art computer vision and natural language processing models, maintained by Google and accessible to all, alongside APIs to accelerate training and experiments;
• Jax, a lean, intuitive, and composable system that brings together automatic differentiation (Autograd) and the Accelerated Linear Algebra (XLA) optimizing compiler to offer high-performance ML for fast research and production;
• TensorFlow Hub, a repository of trained ML models ready for fine-tuning and deployment; and,
• MediaPipe open source cross-platform, which lets users leverage customizable ML solutions for live and streaming media, including text and video.

ML infrastructure management: scaling valuable models with powerful backends

Accessing and managing infrastructure for ML, especially at scale, can be a blocker for many organizations, which is why Google has invested in initiatives including:

• The TFX (or TensorFlow Extended) platform, which offers software frameworks and tooling for full MLOps deployments, helping developers with data automation, model tracking, performance monitoring, and model retraining;
• Kubeflow, which makes deployments of ML workflows on Kubernetes simple, portable and scalable; and,
• TRC (TPU Research Cloud), which gives access to a cluster of more than 1,000 Cloud TPU devices at no charge to selected researchers who publish peer-reviewed papers and/or open source code.

Experimentation and model optimization: encouraging discovery and iteration

Data, tools for model training, and infrastructure can achieve only so much without strong processes for experimentation and optimization—which is why we’ve contribution to projects like xManager, which enables anyone to run and keep track of ML experiments locally or on Vertex AI and Tensorboard, which simplifies tracking and visualizing of model performance metrics.

These areas of focus will help not only our customers but the open-source AI community as a whole, and we’re excited to share more OSS news in coming days and months. To start exploring why many organizations choose Google Cloud for their open-source AI needs, visit our “open cloud” page and be sure to register for Google Cloud Next ‘22 for all our latest news.

Thanks to all the contributors to this blog post: Matt Vasey, George Elissaios, Warren Barkley, Manvinder Singh, James Rubin, Abhishek Ratna, Thea Lamkin, Amin Vahdat, Andrew Moore, Max Sapozhnikov, Gandhi, Vikram Kasivajhula