At Google I/O this year, we introduced Vertex AI to bring together all our ML offerings into a single environment that lets you build and manage the lifecycle of ML projects. In a previous post, we gave you an overview of Vertex AI, sharing how it supports your entire ML workflow—from data management all the way to predictions. Today, we’ll talk a little about how to manage ML datasets with Vertex AI.

Many enterprises want to use data to make meaningful predictions that can bolster their business or help them venture into new markets. This often requires using custom machine learning models—something not every business knows how to create or use. This is where Vertex AI can help. Vertex AI provides tools for every step of the machine learning workflow—from managing data sets to different ways of training the model, evaluating, deploying, and making predictions. It also supports varying levels of ML expertise, so you don’t need to be an ML expert to use Vertex AI.https://www.youtube.com/embed/CN2X6oIlnmI?enablejsapi=1&

Types of data you can use in Vertex AI

Datasets are the first step of the machine learning lifecycle—to get started you need data, and lots of it. Vertex AI currently supports managed datasets for four data types—image, tabular, text, and videos. 

Image

Image datasets let you do:

• Image classification—Identifying items within an image.
• Object detection—Identifying the location of an item in an image
• Image segmentation—Assigning labels to pixel level regions in an image.

To ensure your model performs well in production, use training images similar to what your users will send. For example, if users are likely to send low quality images, be sure to have blurry and low resolution images in your data set. Don’t forget to include different angles, backgrounds, and resolutions. We recommend you include at least 1,000 images per label (item you want to identify), but you can always get started with 10 per label. The more examples you provide, the better your model will be.

Tabular

Tabular datasets enable you to do:

• Regression—Predicting a numerical value.
• Classification—Predicting a category associated with a particular example.
• Forecasting—Predicting the likelihood of sudden events or demands.

Tabular data sets support hundreds of columns and millions of rows. 

Text

With text datasets, you can do:

• Classification—Assigning one or more labels to an entire document.
• Entity extraction—Identifying custom text entities within a document, like “too expensive” or “great value”.
• Sentiment analysis—Identifying the overall sentiment expressed in a block of text, for example, if a customer was happy or upset or frustrated.

Video

Video datasets enable:

• Classification—Labeling entire videos, shots, or frames.
• Action recognition—Identifying clips video clips where specific actions occur.
• Object tracking—Tracking specific objects in a video.

Creating and managing datasets in Vertex AI

Now that we’ve covered the different types of data you can use, let’s shift to creating and managing those datasets. In the Cloud Console, go to Vertex AI dashboard page and click Datasets, then click Create Project.

Say you want to classify items within a set of photos. Create an image dataset and select image classification. You can import files directly from your computer, which will be stored in Cloud Storage. Then, you’ll need to add the corresponding labels (items you want to identify) for your images. If you already have labels, you can use the Import File option to import a CSV with your image URLs and their labels. If your data is not labeled and you would like human help to label it, you can use the Vertex AI data labeling service. Once the files are uploaded, you can create labels and assign them to the images. You can also analyze the images in the data set, the number of images per label, and a few other properties. 

Depending on the type of data you use, your options might vary slightly. For example, if you want to use tabular data, you could upload a CSV file from your computer, use one from Cloud Storage, or select a table from BigQuery directly. Once you select the table, the data is available for analysis.

More to come

This concludes our overview of creating and managing datasets in Vertex AI. In a future installment, we’ll go over the next phase of the machine learning workflow: building and training ML models. 

If you enjoyed this post, keep an eye out for more AI Simplified episodes on YouTube. In the meantime, here’s where you can learn more about Vertex AI.

At one point or another, many of us have used a local computing environment for machine learning (ML). That may have been a notebook computer or a desktop with a GPU. For some problems, a local environment is more than enough. Plus, there’s a lot of flexibility. Install Python, install JupyterLab, and go!

What often happens next is that model training just takes too long. Add a new layer, change some parameters, and wait nine hours to see if the accuracy improved? No thanks. By moving to a Cloud computing environment, a wide variety of powerful machine types are available. That same code might run orders of magnitude faster in the Cloud.

Customers can use Deep Learning VM images (DLVMs) that ensure that ML frameworks, drivers, accelerators, and hardware are all working smoothly together with no extra configuration. Notebook instances are also available that are based on DLVMs, and enable easy access to JupyterLab. 

Benefits of using the Vertex AI custom training service

Using VMs in the cloud can make a huge difference in productivity for ML teams. There are some great reasons to go one step further, and leverage our new Vertex AI custom training service. Instead of training your model directly within your notebook instance, you can submit a training job from your notebook.

The training job will automatically provision computing resources, and de-provision those resources when the job is complete. There is no worrying about leaving a high-performance virtual machine configuration running.

The training service can help to modularize your architecture. As we’ll discuss further in this post, you can put your training code into a container to operate as a portable unit. The training code can have parameters passed into it, such as input data location and hyperparameters, to adapt to different scenarios without redeployment. Also, the training code can export the trained model file, enabling working with other AI services in a decoupled manner.

The training service also supports reproducibility. Each training job is tracked with inputs, outputs, and the container image used. Log messages are available in Cloud Logging, and jobs can be monitored while running.

The training service also supports distributed training, which means that you can train models across multiple nodes in parallel. That translates into faster training times than would be possible within a single VM instance.

Example Notebook

In this blog post, we are going to explain how to use the custom training service, using code snippets from a Vertex AI example. The notebook we’re going to use covers the end-to-end process of custom training and online prediction. The notebook is part of the ai-platform-samples repo, which has many useful examples of how to use Vertex AI.

Custom model training concepts

The custom model training service provides pre-built container images supporting popular frameworks such as TensorFlow, PyTorch, scikit-learn, and XGBoost. Using these containers, you can simply provide your training code and the appropriate container image to a training job. 

You are also able to provide a custom container image. A custom container image can be a good choice if you’re using a language other than Python, or are using an ML framework that is not supported by a pre-built container image. In this blog post, we’ll use a pre-built TensorFlow 2 image with GPU support.

There are multiple ways to manage custom training jobs: via the Console, gcloud CLI, REST API, and Node.js / Python SDKs. After jobs are created, their current status can be queried, and the logs can be streamed.

The training service also supports hyperparameter tuning to find optimal parameters for training your model. A hyperparameter tuning job is similar to a custom training job, in that a training image is provided to the job interface. The training service will run multiple trials, or training jobs with different sets of hyperparameters, to find what results in the best model. You will need to specify the hyperparameters to test; the range of values to explore for those hyperparameters; and details about the number of trials.

Both custom training and hyperparameter tuning jobs can be wrapped into a training pipeline. A training pipeline will execute the job, and can also perform an optional step to upload the model to Vertex AI after training.

How to package your code for a training job

In general, it’s a good practice to develop your model training code that is self-contained when especially executing them inside containers. This means the training codebase would operate in a standalone manner when executed. 

Below is a template of such a self-contained, heavily-commented Python script that you can follow for your own projects too.

  # Imports go here
import tensorflow_datasets as tfds
import tensorflow as tf
…
# Define the hyperparameters and constants like epochs, batch size, number of GPUs, etc
parser = argparse.ArgumentParser()
parser.add_argument('--lr', dest='lr',
                   default=0.01, type=float,
                   help='Learning rate.')
parser.add_argument('--epochs', dest='epochs',
                   default=10, type=int,
                   help='Number of epochs.')
...
args = parser.parse_args()
...
# Prepare data loaders
def make_datasets_unbatched():
 # Scaling CIFAR10 data from (0, 255] to (0., 1.]
 def scale(image, label):
   image = tf.cast(image, tf.float32)
   image /= 255.0
   return image, label
 datasets, info = tfds.load(name='cifar10',
                           with_info=True,
                           as_supervised=True)
 return datasets['train'].map(scale).cache().shuffle(BUFFER_SIZE).repeat()
# Build our model, compile, and train it
model = [define your model]
model.compile(loss=..., optimizer=..., metrics=...)
model.fit(...)
# Serialize our model
model.save(MODEL_DIR)

Note that the MODEL_DIR needs to be a location inside a Google Cloud Storage (GCS) bucket. This is because the training service can only communicate with that and not with our local system. Here is a sample location inside a GCS Bucket to save a model: gs://caip-training/cifar10-model where caip-training is the name of the GCS bucket.

Although we are not using any custom modules in the above code listing, one can easily incorporate them as we would normally inside a Python script. Refer to this document if you want to know more. Next up, we will review how to configure the training infrastructure, including the type and number of GPUs to use, and submit a training script to run inside the infrastructure. 

How to submit a training job, including configuring which machines to use

To train a deep learning model efficiently on large datasets, we need hardware accelerators that are suited to run matrix multiplication in a highly parallelized manner. Distributed training is also common when it comes to training a large model on a large dataset. For this example, we will be using a single Tesla K80 GPU. Vertex AI supports a range of different GPUs (find out more here). 

Here is how we initialize our training job with the Vertex AI SDK:

  job = aiplatform.CustomTrainingJob(
   display_name=JOB_NAME,
   script_path="task.py",
   container_uri=TRAIN_IMAGE,
   requirements=["tensorflow_datasets==1.3.0"],
   model_serving_container_image_uri=DEPLOY_IMAGE,
)

(aiplatform is aliased as from google.cloud import aiplatform)

Let’s review the arguments:

• display_name refers to a unique identifier to the training job used for easily locating it. 
• script_path refers to the path of the training script to run. This is the script we discussed in the section above.
• container_uri refers to the URI of the container that will be used to run our training script. For this, we have several options to choose from. For this example, we will use gcr.io/cloud-aiplatform/training/tf-gpu.2-1:latest. We will use this same container for deployment as well but with a slightly changed container URI. You can find the containers available for model training here and the containers available for deployment purposes can be found here. 
• requirements let us specify any external packages that might be required to run the training script. 
• model_serving_container_image_uri specifies the container URI that would be used during deployment. 

Note: Using separate containers for distinct purposes like training and deployment is often a good practice, since it isolates the relevant dependencies for each purpose.

We are now all set up to submit a custom training job:

  model = job.run(
       model_display_name=MODEL_DISPLAY_NAME,
       args=CMDARGS,
       replica_count=1,
       machine_type=TRAIN_COMPUTE,
       accelerator_type=TRAIN_GPU.name,
       accelerator_count=TRAIN_NGPU
   )

Here, we have:model_display_name that provides a unique name to identify our trained model. This comes in handy later down the pipeline when we would deploy it using the prediction service. args are our command-line arguments typically used to specify things like hyperparameter values.replica_count denotes the number of worker replicas to be used during training. machine_type specifies the type of base machine to be used during training. accelerator_type denotes the type of accelerator to be used during training. If we are interested in using a Tesla K80, then TRAIN_GPU should be specified as aip.AcceleratorType.NVIDIA_TESLA_K80. (aip is aliased as from google.cloud.aiplatform import gapic as aip.)accelerator_count specifies the number of accelerators to use. For a single host multi-GPU configuration, we would set the replica_count to 1 and then specify the accelerator_count as per our choice depending on the resource available under the corresponding compute zone. 

Note that model here is a google.cloud.aiplatform.models.Model object. It is returned by the training service after the job is completed. 

With this setup, we can actually start a custom training job that we can monitor. After we submit the above training pipeline, we should see some initial logs resembling this:

The link highlighted  in Figure 2 will redirect to the dashboard of the training pipeline which looks like so:

As seen in Figure 3, the dashboard provides a comprehensive summary of all the necessary artifacts related to our training pipeline. Monitoring your model training is also very important especially to catch any early training bugs. To view the training logs, we need to click the link beside the “Custom job” tab (refer to Figure 3). There also we are presented with roughly similar information as shown in Figure 3 but this time it includes the logs as well: 

Note: Once we submit the custom training job, a training pipeline is first created to provision the training. Then inside the pipeline, the actual training job is started. This is why we see two very similar dashboards above but they have different purposes. Let’s check out the logs (which is maintained using Cloud Logging automatically):

With Cloud Logging, it is also possible to set alerts on the basis of different criteria. For example, alerting the users when the training job fails or completes so that some immediate action could be taken. You can refer to this post for more details.  

After the training pipeline is completed, on your end, you will notice the success status:

Accessing the trained model

Recall that we had to serialize our model inside a GCS Bucket in order to make it compatible with the training service. So, after the model is trained, we can access it from that location. We can even directly load it using the following line of code:

  model = tf.keras.models.load_model('gs://[your-bucket-name]/[model-name]')

Note that we are referring to the TensorFlow model that resulted from training. The training service also maintains a similar “model” namespace to help us manage these models. Recall that the training service returns a  google.cloud.aiplatform.models.Model object as mentioned earlier. It comes with a deploy() method that allows us to deploy our model programmatically within minutes with several different options. Check out this link if you are interested in deploying your models using this option. 

Vertex AI also provides a dashboard for all the models that have been trained successfully and it can be accessed with this link. It resembles this:

If we click the model as listed in Figure 7, we should be able to directly deploy from the interface:

In this post, we will not be covering deployment, but you are encouraged to try it out yourself. After the model is deployed to an endpoint, you will be able to use it to make online predictions.

Wrapping Up

In this blog post, we discussed the benefits of using the Vertex AI custom training service, including better reproducibility and management of experiments. We also walked through the steps to convert your Jupyter Notebook codebase to a standard containerized codebase, which will be useful not only for the training service, but for other container-based environments. The example notebook provides a great starting point to understand each step, and to use as a template for your own projects.

If you’ve been exploring recently-launched consumer generative AI tools like Bard and thinking about how to build similar experiences for your business, Generative AI App Builder, or Gen App Builder for short, is here to get you started.

Gen App Builder is part of Google Cloud’s recently announced generative AI offerings and lets developers, even those with limited machine learning skills, quickly and easily tap into the power of Google’s foundation models, search expertise, and conversational AI technologies to create enterprise-grade generative AI applications. 

“Google Cloud’s leading AI technology enables STARZ customers to discover more relevant content, increasing engagement with, and the likelihood of completing the content served to them,” says Robin Chacko, EVP Direct-to-Consumer, STARZ. “We’re excited about how generative AI-powered search will help users find the most relevant content even easier and faster.”

Gen App Builder is exciting because unlike most existing generative AI offerings for developers, it offers an orchestration layer that abstracts the complexity of combining various enterprise systems with generative AI tools to create a smooth, helpful user experience. Gen App Builder provides step-by-step orchestration of search and conversational applications with pre-built workflows for common tasks like onboarding, data ingestion, and customization, making it easy for developers to set up and deploy their apps. With Gen App Builder developers can: 

• Build in minutes or hours. With access to Google’s no-code conversational and search tools powered by foundation models, organizations can get started with a few clicks and quickly build high-quality experiences that can be integrated into their applications and websites. 
• Combine the power of foundation models with information retrieval to find relevant, personalized information. Enterprises can build apps that understand user intent via natural language, and surface the right information with associated citations and attributions from a company’s public and private data. They can also fully control what data their applications access and the content or topics they want to address.
• Build multimodal apps that can respond with text, images, and other media. Gen App Builder supports not just text, but also other modalities such as images and videos. It allows developers to build apps using a combination of text and images as inputs to find information across documents, photos, and video content, enabling richer customer interactions. 
• Combine natural conversations with structured flows. Developers can granularly blend the output of foundation models with controls to ground answers in enterprise content, and step-by-step conversation orchestration to guide customers to the right answers.
• Provide the ability to transact and connect to third party apps and services. Gen App Builder makes it simple to create digital assistants and bots that not only serve content, but also connect to purchasing and provisioning systems to enable transactions from the conversational UI, and escalate customer conversations to a human agent when the context demands.

A new generation of conversational AI experiences and assistants 

Consumers of enterprise applications expect to interact with technology in a seamless, conversational way to quickly find the information they need and act on it. Gen App Builder can help reinvent these customer and employee experiences by ingesting large, complex datasets that are specific to your company–from websites, documents, and transactional systems like billing and inventory, to emails, chat conversations, and more. These AI-powered apps can synthesize information across all of these sources to provide specific, actionable responses, using only the data you have provided. 

Some of the most popular uses are in customer service, where generative apps can contribute to increasing revenue, customer satisfaction, and customer loyalty. For example, if a retail customer reaches out to modify an order, a virtual agent can help them change it to another product. The customer doesn’t even need to provide the new product name—they can just upload an image and let the agent guide them through the rest. Watch this demo to see how a retail chatbot can use multimodal capabilities to help a consumer navigate various options on the website, including giving the customer ideas on how to use the product and even helping them complete the purchase with the ability to transact within the conversational UI. This scenario could apply to multiple industries and use cases, ranging from consumer goods and public services, to finance and internal corporate systems like intranets.

Combining the power of Google-quality search with foundation models

Finding the right information from data across the organization is a critical requirement within any enterprise. Yet it can be challenging to build high-quality enterprise search experiences with existing tools. Current systems struggle to understand user intent, are difficult to implement and customize, and don’t provide a high-quality user experience. 

One of the most exciting features of Gen App Builder is the ability to combine the power of Google-quality search with generative AI to help enterprises find the most relevant and personalized information when they need it. With Gen App Builder, enterprises can build conversational search experiences across their public and private data in minutes or hours with no coding experience. 

Enabling multimodal search across text, images and video within the enterprise is a key aspect of the search experiences in Gen App Builder. In addition to providing high-quality search results, Gen App Builder can conveniently summarize the results and provide corresponding citations in a natural, human-like fashion. Gen App Builder also automatically extracts key information from the data and enables personalized results for users. Watch this demo to see how these capabilities can come together to transform the search experience for employees at a financial services firm. The ability to integrate Google-quality search within the enterprise’s applications means they can enjoy a new level of data utilization, drive increased process efficiencies, and provide delightful experiences to their employees and customers.

“Customers have been shopping at Macy’s for generations. Being able to deliver 360° personalization and contextual recommendations will help ensure that Macy’s is still providing future generations of shoppers with a seamless, exceptional experience,” said Bennett Fox-Glassman, Senior Vice-President, Customer Journey, Macy’s. “We’ve already realized an increase in revenue per visit and conversion rates had great success using Google Cloud’s AI technology and are looking forward to exploring how these latest announcements bring together Natural Language Processing and Generative AI capabilities to deliver next-gen search and conversational experiences for our customers.”

The ability to intuitively interact with complex data across a variety of sources allows organizations to better serve their customers and deliver more relevant offerings. Combined with conversational and fulfillment abilities, the potential for improving customer engagement and employee productivity is immense. We’re excited to see how developers and enterprises use a mix of these capabilities to power new experiences and revenue opportunities.

If you’re interested in a closer look at the Gen App Builder, tune into this session at the Data Cloud & AI Summit. Take a step forward to getting hands-on and join the waitlist for our trusted tester program. And finally, bookmark our generative AI landing page to keep abreast of the latest news, updates and possibilities from this exciting new world of Gen Apps.

The combination of a vital need for IT experts among businesses and a digital skills gap is making lifelong learning increasingly critical. Beyond professional development, learning new skills offers additional rewards from building peer connections to boosting your creativity. That’s why in 2020 Krishna Deepak Nallamilli and I launched KIMO.ai to reimagine how people approach learning, especially in developing markets. Our team is building the artificial intelligence needed to generate individual learning paths through a wide range of quality digital learning content.

Google Cloud and its Startup Program have been instrumental in connecting our team with the tools, people, processes, and best practices to grow our business. 

Existing learning platforms lack engagement

Outside of traditional education settings, massive open online learning courses (MOOCs)—often modeled after university courses—can provide a flexible and affordable way to upskill or reskill. But the vast majority of people who participate in MOOC programs fail to complete courses. Based on our research, the challenge with existing learning platforms is a lack of engagement, primarily caused by limited direction on which skills to learn, whether AI, fintech, blockchain, or other in-demand disciplines.   

We’ve also received feedback that many corporate learning management systems–developed as online training systems to upskill employees–tend to be poorly designed and time-consuming to use. 

Overall, a significant challenge with most existing learning platforms is that they’re generic. For example, suppose you’re interested in learning about AI. In that case, you need AI-related coursework that applies to your industry and the job you want because AI in medicine is vastly different from AI in financial services. Today’s online learning options typically take a one-size-fits-all approach and fail to capture the nuances of what learners really need to get ahead. 

Building a future-proof learning platform 

The commitment to highly personalized, accessible learning inspired KIMO.ai, a platform that we believe is the future of education. Depending on your goals, current skills, location, and other factors, our AI-based platform will identify which coursework (and where to find those classes) to build the skills you need. The more personalized, relevant learning recommendations even take into account people’s preferences for podcasts, MOOCs, books, articles, videos, courses, publications, and more.  

In a mix of cooperation and competition we call “coopetition,” KIMO.ai will regularly recommend courses from other established online learning systems if, based on our automated assessment, it’s the best option for a learner. There’s also the option to access free content only. 

Google cultural alignment fosters trust

Our platform started with one developer exploring NLP models and Google APIs. As we’ve grown our team and launched our beta to 110,000 users in developing markets, we discovered there is a lot of interest in our platform, and we believe we can make a significant impact. In feedback forums, we also learned that we need to focus our efforts on the mobile experience to improve engagement since 99% of the beta testers use mobile devices. 

Beyond our team’s high level of trust in Google Cloud solutions, our team also appreciates the cultural alignment with Google. We value Google’s developer-centric approach and rely on tools like Dataflow for batch data processing and Cloud TPU to reliably run machine learning models with AI services on Google Cloud. We also build all of our deployments on Google Kubernetes Engine (GKE), which makes it easy to manage all our containerized workloads 

On the front end, Google App Engine makes it easy to deploy apps and experiment, and it integrates seamlessly with Firebase for authentication and more. BigQuery is our serverless data warehouse that efficiently scales to support the millions of articles, videos, and other learning resources we need to analyze to provide the targeted coursework recommendations our learners require.

As we grow our business having a network of trusted advisors is also extremely valuable. By working closely with DoIT International, the 2020 Google Cloud Global Reseller Partner of the Year, our team has access to their cloud, Kubernetes, and machine learning expertise. DoIT has already helped us quickly resolve IT issues and create analytics dashboards that give us insights to continually enhance our services. 

Building for a growing industry

The dynamic edtech market is growing rapidly and estimated to become an $11B industry by 2025. We’re proud to be part of the next wave of personalized education that has the potential to empower people in developing markets and beyond to grow their skills with coursework tailored to their exact needs and how they like to learn. This year, we will deliver our platform to at least 400,000 more people. We’re excited to see how they use it and where it takes them. 

If you want to learn more about how Google Cloud can help your startup, visit our Startup Program application page here to get more information about our program, and sign up for our communications to get a look at our community activities, digital events, special offers, and more.