Enterprise agility and the ability to innovate, adapt and respond quickly to the ever-changing risk and regulatory landscape is no longer a choice, but the cornerstone of successful digital transformation and commercial growth. Traditional access to and ways of managing data invariably create challenges in dealing with multiple data repositories, reconciliations, fire-drills, etc.

In response, the move to cloud is increasing significantly. It enables risk analytics and regulatory reporting at scale in a secure environment with data storage, management and encryption capabilities as a standard. In addition, as regulatory reporting requirements become more granular, machine learning can help facilitate new insights and allow for risk management to become more embedded into operational processes.

This webinar will address the day-to-day challenges in risk management and regulatory compliance, while also exploring how technological innovations can provide massive improvements and potential.

Key themes

• Real-life data challenges in the eyes of risk managers: can compliance, fraud detection and identifying liquidity positions be improved through the use of AI?
• Innovative approaches to streamline regulatory reporting to derive deeper customer insights from data at the moment of truth.
• Reimagining operations: how to modernise the data infrastructure to accommodate data explosion, drive flexibility and deliver a more cost effective outcome.

Geospatial data is a critical component for a comprehensive analytics strategy. Whether you are trying to visualize data using geospatial parameters or do deeper analysis or modeling on customer distribution or proximity, most organizations have some type of geospatial data they would like to use – whether it be customer zipcodes, store locations, or shipping addresses. However, converting geographic data into the correct format for analysis and aggregation at different levels can be difficult. In this post, we’ll walk through some examples of how you can leverage the Google Cloud platform alongside Google Cloud Public Datasets to perform robust analytics on geographic data. The full queries can be accessed from this notebook here. 

Public US Geo Boundaries dataset

BigQuery hosts a slew of public datasets for you to access and integrate into your analytics. Google pays for the storage of these datasets and provides public access to the data via the bigquery-public-data project. You only pay for queries against the data. Plus, the first 1 TB per month is free! These public datasets are valuable on their own, but when joined against your own data they can unlock new analytics use cases and save the team a lot of time. 

Within the Google Cloud Public Datasets Program there are several geographic datasets. Here, we’ll work with the geo_us_boundaries dataset, which contains a set of tables that have the boundaries of different geospatial areas as polygons and coordinates based on the center point (GEOGRAPHY column type in BigQuery), published by the US Census Bureau.

Mapping geospatial points to hierarchical areas

Many times you will find yourself in situations where you have a string representing an address. However, most tools require lat/long coordinates to actually plot points. Using the Google Maps Geocoding API we can convert an address into a lat/long and then store the results in the BigQuery table. 

With a lat/long representation of our point, we can join our initial dataset back onto any of the tables here using the ST_WITHIN function. This allows us to check and see if a point is within the specified polygon. 

ST_WITHIN(geography_1, geography_2)

This can be helpful for ensuring standard nomenclature; for example, metropolitan areas that might be named differently. The query below maps each customers’ address to a given metropolitan area name.

  SELECT 
   cust.id as customer_id, 
   metro.name as metro_name 
FROM `looker-private-demo.retail.customers` as cust
,`bigquery-public-data.geo_us_boundaries.metropolitan_divisions` as metro
WHERE ST_WITHIN(ST_GEOGPOINT(cust.longitude, cust.latitude),metro.metdiv_geom)

It can also be useful for converting to designated market area (DMA), which is often used in creating targeted digital marketing campaigns.

  SELECT 
   cust.id as customer_id, 
   dma.dma_name 
FROM `looker-private-demo.retail.customers` as cust
,`bigquery-public-data.geo_us_boundaries.designated_market_area` as dma
WHERE ST_WITHIN(ST_GEOGPOINT(cust.longitude, cust.latitude),dma.dma_geom)

Or for filling in missing information; for example, some addresses may be missing zip code which results in incorrect calculations when aggregating up to the zipcode level. By joining onto the zip_codes table we can ensure all coordinates are mapped appropriately and aggregate up from there.

  SELECT 
   zip.zip_code, 
   count(distinct cust.id) as unique_customers
FROM `looker-private-demo.retail.customers` as cust
,`bigquery-public-data.geo_us_boundaries.zip_codes` as zip
WHERE ST_WITHIN(ST_GEOGPOINT(cust.longitude, cust.latitude),zip.zip_code_geom)
GROUP BY 1

Note that the zip code table isn’t a comprehensive list of all US zip codes, they are zip code tabulation areas (ZCTAs). Details about the differences can be found here. Additionally, the zip code table gives us hierarchical information, which allows us to perform more meaningful analytics. One example is leveraging hierarchical drilling in Looker. I can aggregate my total sales up to the country level, and then drill down to state, city and zipcode to identify where sales are highest. You can also use the BigQuery GeoViz tool to visualize geospatial data!

Aside from simply checking if a point is within an area, we can also use ST_DISTANCE to do something like find the closest city using the centerpoint for the metropolitan area table. 

  SELECT 
cust.id as customer_id, 
ARRAY_AGG(
  metro.name order by ST_DISTANCE(
   ST_GEOGPOINT(cust.longitude, cust.latitude),
  metro.internal_point_geom) asc limit 1)[offset(0)] as metro_name
FROM
`looker-private-demo.retail.customers` as cust
,`bigquery-public-data.geo_us_boundaries.metropolitan_divisions` as metro
GROUP BY cust.id

This concept doesn’t just hold true for points, we can also leverage other GIS functions to see if a geospatial area is contained within areas that are listed in the boundaries datasets. If your data comes into BigQuery as a GeoJSON string, we can convert it to a GEOGRAPHY type using the ST_GEOGFROMGEOJSON function. Once our data is in a GEOGRAPHY type we can do things like check to see what urban area the geo is within – using either ST_WITHIN or ST_INTERSECTS to account for partial coverage. Here, I am using the customer’s zip code to find all metropolitan divisions where the zip code polygon and the metropolitan polygon intersect. I am then selecting the metropolitan area that has the most overlap (or the intersection has the largest area) to be the customer’s metro that we use for reporting.

  SELECT 
   cust.id as customer_id, 
   ARRAY_AGG(
      metro.name order by ST_AREA(
        ST_INTERSECTION(zip.zip_code_geom,metro.metdiv_geom)
      ) desc limit 1)[offset(0)] as metro_name  
FROM
`looker-private-demo.retail.customers` as cust
JOIN `bigquery-public-data.geo_us_boundaries.zip_codes` as zip on      cust.zip=zip.zip_code
,`bigquery-public-data.geo_us_boundaries.metropolitan_divisions` as metro
WHERE ST_INTERSECTS(zip.zip_code_geom,metro.metdiv_geom)
GROUP BY cust.id

The same ideas can be applied to the other tables in the dataset including the county, urban areas and National Weather Service forecast regions (which can also be useful if you want to join your datasets onto weather data).

Correcting for data discrepancy

One problem that we may run into when working with geospatial data is that different data sources may have different representations of the same information. For example, you might have one system that records state as a two letter abbreviation and another using the full name. Here, we can use the state table to join the different datasets.

  SELECT 
   st.state_name, 
   sum(ab.sales+fn.sales) as total_sales 
FROM `bigquery-public-data.geo_us_boundaries.states` as st
LEFT JOIN abbreviated_table as ab on ab.state = st.state
LEFT JOIN fullname_table as fn on fn.state = st.state_name
WHERE COALESCE(ab.state, fn.state) IS NOT NULL
GROUP BY 1

Another example might be using the tables as a source of truth for fuzzy matching. If the address is a manually entered field somewhere in your application, there is a good chance that things will be misspelled. Different representations of the same name may prevent tables from joining with each other or lead to duplicate entries when performing aggregations. Here, I use a simple Soundex algorithm to generate a code for each county name, using helper functions from this blog post. We can see that even though some are misspelled they have the same Soundex code.

Next, we can join back onto our counties table so we make sure to use the correct spelling of the county name. Then, we can simply aggregate our data for more accurate reporting. 

  SELECT
 c.county_name,
 sum(sales) as total_sales
FROM
 table
 JOIN `bigquery-public-data.geo_us_boundaries.counties` as c
 on testing.dq_fm_Soundex(table.county) = testing.dq_fm_Soundex(c.county_name)
WHERE c.state_fips_code = cast(36 as string)
GROUP BY 1

Note that fuzzy matching definitely isn’t perfect and you might need to try different methods or apply certain filters for it to work best depending on the specifics of your data.

The US Geo Boundary datasets allow you to perform meaningful geographic analysis without needing to worry about extracting, transforming or loading additional datasets into BigQuery. These datasets, along with all the other Google Cloud Public Datasets, will be available in the Analytics Hub. Please sign up for the Analytics Hub preview, which is scheduled to be available in the third quarter of 2021, by going to g.co/cloud/analytics-hub.

Investment management is a heavily data-driven industry—portfolio managers and investment researchers require a large number of data sources to guide them in shaping their investment strategies. 

New cloud capabilities and technologies enable investment managers to process data faster than ever before and iterate on ideas quickly to fuel innovation in the signal generation process and gain a competitive edge. 

Using the cloud for investment research workflows makes it easier to onboard data from data providers, spin up large compute workloads in the midst of market volatility or during heavy research cycles, and manage complex machine learning or natural language workflows to gain market insights.

We hear from industry leaders that they’re exploring new ways to run investment research. “Differentiated investment strategies require new types of information sources, and new ways to process that information,” David Easthope, senior analyst, Market Structure and Technology, Greenwich Associates. “And that, of course, relies heavily on having access to reliable and scalable storage, computational, and AI / ML resources. More specifically, quantitative strategies can benefit from the computational platforms and embedded AI/ML capabilities the cloud can offer.” 

Google Cloud gives investment managers essential components to work and operate faster as they bring their investment research workflows to the cloud. Here are the key highlights:

1. Simplify, speed up your data acquisition, discovery, and analytics

The foundation of any investment strategy starts with data—acquiring it, detecting patterns, and analyzing it for insights. Enabling data providers to easily share large datasets such as tick history within a high-performance analytics engine can greatly reduce the data engineering overhead when possible.

Once data is onboarded, you can tag business and technical metadata related to your datasets and provide portfolio managers the ability to discover these datasets via a search interface.

We further review analytics options for various scenarios, including aggregating massive datasets, creating dashboards, and incorporating streaming analytics workloads.

2. Take advantage of burst compute workloads

Data engineers and researchers require ready access to burst compute capabilities to perform backtesting, portfolio simulations and run risk calculations. Cloud works well for these workloads due to its elasticity, consumption-based models, and hardware evolution.

Many investment managers are shifting to a container-based strategy along with a Kubernetes-based scheduler for greater consistency, scaling and efficiency in environments with a large number of researchers. Cloud managed services and a rich suite of CI/CD tools can make this vision a reality while improving security and developer productivity.

3. Tackle machine learning (ML) and model deployment with the help from cloud

Quantitative researchers scour vast amounts of market and alternative data sources searching for signals and correlations, while ML engineers have the challenge of taking these signals and moving them to production.

Google Cloud empowers users to create and operationalize their models without wasting valuable time with a comprehensive set of MLOps tools. 

In this paper, we explore multiple solutions for ML and model deployment. Those capabilities reduce the amount of time operationalizing ML models, so quants and data scientists have more time to devote to differentiating activities. 

4. Get the data you need in less time with Natural Language and Document AI

Thousands of financial filings, news articles, and sell-side research reports are generated every day, and it’s difficult for humans alone to process this volume of information. These documents are often generated in many languages and the ability to do entity recognition, sentiment or syntactical analysis in those languages, or perhaps translate them into the language of the portfolio manager is of critical importance. Google Cloud provides these capabilities through pre-trained models, or allows you to train high-quality models with your own datasets.

Getting started

There are plenty of emerging technologies, tools, and approaches available to help investment managers today. At Google Cloud, we can help you access, organize, and utilize these essential components to make your research faster, reliable, and more valuable.

To learn more about these four keys to better investment research, check out our whitepaper for more.

Reinforcement learning (RL) is a form of machine learning whereby an agent takes actions in an environment to maximize a given objective (a reward) over this sequence of steps. Applications of RL include learning-based robotics, autonomous vehicles and content serving. The fundamental RL system includes many states, corresponding actions, and rewards to those actions. Translate that into a movie recommender system: The ‘state’ is the user, the ‘action’ is the movie to recommend to the user, and the ‘reward’ is the user rating of the movie. RL is a great framework for optimizing ML models, as mentioned by Spotify in the keynote in the Applied ML Summit 2021.

In this article, we’ll demonstrate an RL-based movie recommender system executed in Vertex AI and built with TF-Agents, a library for RL in TensorFlow. This demo has two parts: (1) a step-by-step guide leveraging Vertex Training, Hyperparameter Tuning, and Prediction services; (2) a MLOps guide to build end-to-end pipelines using Vertex Pipelines and other Vertex services.

TF-Agents meets Vertex AI

In reinforcement learning (RL), an agent takes a sequence of actions in a given environment according to some policy, with the goal of maximizing a given reward over this sequence of actions. TF-Agents is a powerful and flexible library enabling you to easily design, implement and test RL applications. It provides you with a comprehensive set of logical modules that support easy customization:

• Policy: A mapping from an environment observation to an action or a distribution over actions. It is the artifact produced from training, and the equivalent of a “Model” in a supervised learning setup.
• Action: A move or behavior that is outputted by some policy, and chosen and taken by an agent.
• Agent: An entity that encapsulates an algorithm to use one or more policies to choose and take actions, and trains the policy.
• Observations: A characterization of the environment state.
• Environment: Definition of the RL problem to solve. At each time step, the environment generates an observation, bears the effect of the agent action, and then given the action taken and the observation, the environment responds with a reward as feedback.

A typical RL training loop looks like the following:

A typical process to build, evaluate, and deploy RL applications would be:

1. Frame the problem: While this blog post introduces a movie recommendation system, you can use RL to solve a wide range of problems. For instance, you can easily solve a typical classification problem with RL, where you can frame predicted classes as actions. One example would be digit classification: observations are digit images, actions are 0-9 predictions and rewards indicate whether the predictions match the ground truth digits.
2. Design and implement RL simulated experiments: We will go into detail on simulated training data and prediction requests in the end-to-end pipeline demo.
3. Evaluate performance of the offline experiments.
4. Launch end-to-end production pipeline by replacing the simulation constituents with real-world interactions.

Now that you know how we’ll build a movie recommendation system with RL, let’s look at how we can use Vertex AI to run our RL application in the cloud. We’ll use the following Vertex AI products:

• Vertex AI training to train a RL policy (the counterpart of a model in supervised learning) at scale
• Vertex AI hyperparameter tuning to find the best hyperparameters
• Vertex AI prediction to serve trained policies at endpoints
• Vertex Pipelines to automate, monitor, and govern your RL systems by orchestrating your workflow in a serverless manner, and storing your workflow’s artifacts using Vertex ML Metadata.

Step-by-step RL demo

This step-by-step demo showcases how to build the MovieLens recommendation system using TF-Agents and Vertex AI services, primarily custom training and hyperparameter tuning, custom prediction and endpoint deployment. This demo is available on Github, including a step-by-step notebook and Python modules.

The demo first walks through the TF-Agents on-policy (which is covered in detail in the demo) training code of the RL system locally in the notebook environment. It then shows how to integrate the TF-Agents implementation with Vertex AI services: It packages the training (and hyperparameter tuning) logic in a custom training/hyperparameter tuning container and builds the container with Cloud Build. With this container, it executes remote training and hyperparameter tuning jobs using Vertex AI. It also illustrates how to utilize the best hyperparameters learned from the hyperparameter tuning job during training, as an optimization.

The demo also defines the prediction logic, which takes in observations (user vectors) from prediction requests and outputs predicted actions (movie items to recommend), in a custom prediction container and builds the container with Cloud Build. It deploys the trained policy to a Vertex AI endpoint, and uses the prediction container as the serving container for the policy at the Vertex AI endpoint.

End-to-end workflow with a closed feedback loop: Pipeline demo

Pipeline architecture

Building upon our RL demo, we’ll now show you how to scale this workflow using Vertex Pipelines. This pipeline demo showcases how to build an end-to-end MLOps pipeline for the MovieLens recommendation system, using Kubeflow Pipelines (KFP) for authoring and Vertex Pipelines for orchestration.

Highlights of this end-to-end demo include:

• RL-specific implementation that handles RL modules, training logic and trained policies as opposed to models
• Simulated training data, simulated environment for predictions and re-training
• Closing of the feedback loop from prediction results back to training
• Customizable and reproducible KFP components

An illustration of the pipeline structure is shown in the figure below.

The pipeline consists of the following components:

• Generator: to generate MovieLens simulation data as the initial training dataset using a random data-collecting policy, and store in BigQuery [executed only once]
• Ingester: to ingest training data in BigQuery and output TFRecord files
• Trainer: to perform off-policy (which is covered in detail in the demo) training using the training dataset and output a trained RL policy
• Deployer: to upload the trained policy, create a Vertex AI endpoint and deploy the trained policy to the endpoint

In addition to the above pipeline, there are three components which utilize other GCP services (Cloud Functions, Cloud Scheduler, Pub/Sub):

• Simulator: to send recurring simulated MovieLens prediction requests to the endpoint
• Logger: to asynchronously log prediction inputs and results as new training data back to BigQuery, per prediction requests
• Trigger: to recurrently execute re-training on new training data

Pipeline construction with Kubeflow Pipelines (KFP)

You can author the pipeline using the individual components mentioned above:

  @dsl.pipeline(
   pipeline_root=PIPELINE_ROOT,
   name=f"{PIPELINE_NAME}-startup")
def pipeline(
   # Pipeline configs
   project_id: str,
   raw_data_path: str,
   training_artifacts_dir: str,
 
   # BigQuery configs
   bigquery_dataset_id: str,
   bigquery_location: str,
   bigquery_table_id: str,
   bigquery_max_rows: int = 10000,
 
   # TF-Agents RL configs
   batch_size: int = 8,
   rank_k: int = 20,
   num_actions: int = 20,
   driver_steps: int = 3,
   num_epochs: int = 5,
   tikhonov_weight: float = 0.01,
   agent_alpha: float = 10) -> None:
 """Authors a RL pipeline for MovieLens movie recommendation system.
 
 Integrates the Generator, Ingester, Trainer and Deployer components. This
 pipeline generates initial training data with a random policy and runs once
 as the initiation of the system.
 
 Args:
   project_id: GCP project ID. This is required because otherwise the BigQuery
     client will use the ID of the tenant GCP project created as a result of
     KFP, which doesn't have proper access to BigQuery.
   raw_data_path: Path to MovieLens 100K's "u.data" file.
   training_artifacts_dir: Path to store the Trainer artifacts (trained policy).
 
   bigquery_dataset: A string of the BigQuery dataset ID in the format of
     "project.dataset".
   bigquery_location: A string of the BigQuery dataset location.
   bigquery_table_id: A string of the BigQuery table ID in the format of
     "project.dataset.table".
   bigquery_max_rows: Optional; maximum number of rows to ingest.
 
   batch_size: Optional; batch size of environment generated quantities eg.
     rewards.
   rank_k: Optional; rank for matrix factorization in the MovieLens environment;
     also the observation dimension.
   num_actions: Optional; number of actions (movie items) to choose from.
   driver_steps: Optional; number of steps to run per batch.
   num_epochs: Optional; number of training epochs.
   tikhonov_weight: Optional; LinUCB Tikhonov regularization weight of the
     Trainer.
   agent_alpha: Optional; LinUCB exploration parameter that multiplies the
     confidence intervals of the Trainer.
 """
 # Run the Generator component.
 generate_op = create_component_from_func(
     func=generator_component.generate_movielens_dataset_for_bigquery,
     output_component_file=f"generator-{OUTPUT_COMPONENT_SPEC}",
     packages_to_install=[
         "google-cloud-bigquery==2.20.0",
         "tensorflow==2.5.0",
         "tf-agents==0.8.0",
     ])
 generate_task = generate_op(
     project_id=project_id,
     raw_data_path=raw_data_path,
     batch_size=batch_size,
     rank_k=rank_k,
     num_actions=num_actions,
     driver_steps=driver_steps,
     bigquery_tmp_file=BIGQUERY_TMP_FILE,
     bigquery_dataset_id=bigquery_dataset_id,
     bigquery_location=bigquery_location,
     bigquery_table_id=bigquery_table_id)
 
 # Run the Ingester component.
 ingest_op = create_component_from_func(
     func=ingester_component.ingest_bigquery_dataset_into_tfrecord,
     output_component_file=f"ingester-{OUTPUT_COMPONENT_SPEC}",
     packages_to_install=[
         "google-cloud-bigquery==2.20.0",
         "tensorflow==2.5.0",
     ])
 ingest_task = ingest_op(
     project_id=project_id,
     bigquery_table_id=generate_task.outputs["bigquery_table_id"],
 
     bigquery_max_rows=bigquery_max_rows,
     tfrecord_file=TFRECORD_FILE)
 
 # Run the Trainer component and submit custom job to Vertex AI.
 train_op = create_component_from_func(
     func=trainer_component.training_op,
     output_component_file=f"trainer-{OUTPUT_COMPONENT_SPEC}",
     packages_to_install=[
         "tensorflow==2.5.0",
         "tf-agents==0.8.0",
     ])
 train_task = train_op(
     training_artifacts_dir=training_artifacts_dir,
     tfrecord_file=ingest_task.outputs["tfrecord_file"],
     num_epochs=num_epochs,
     rank_k=rank_k,
     num_actions=num_actions,
     tikhonov_weight=tikhonov_weight,
     agent_alpha=agent_alpha)
 
 worker_pool_specs = [
     {
         "containerSpec": {
             "imageUri":train_task.container.image,
         },
         "replicaCount": TRAINING_REPLICA_COUNT,
         "machineSpec": {
             "machineType": TRAINING_MACHINE_TYPE,
             "acceleratorType": TRAINING_ACCELERATOR_TYPE,
             "acceleratorCount": TRAINING_ACCELERATOR_COUNT,
         },
     },
 ]
 train_task.custom_job_spec = {
     "displayName": train_task.name,
     "jobSpec": {
         "workerPoolSpecs": worker_pool_specs,
     }
 }
 
 # Run the Deployer components.
 # Upload the trained policy as a model.
 model_upload_op = gcc_aip.ModelUploadOp(
     project=project_id,
     display_name=TRAINED_POLICY_DISPLAY_NAME,
     artifact_uri=training_artifacts_dir,
     serving_container_image_uri=f"gcr.io/{PROJECT_ID}/{PREDICTION_CONTAINER}:latest",
 )
 # Model uploading has to occur after training completes.
 model_upload_op.after(train_task)
 # Create a Vertex AI endpoint. (This operation can occur in parallel with
 # the Generator, Ingester, Trainer components.)
 endpoint_create_op = gcc_aip.EndpointCreateOp(
     project=project_id,
     display_name=ENDPOINT_DISPLAY_NAME)
 # Deploy the uploaded, trained policy to the created endpoint. (This operation
 # has to occur after both model uploading and endpoint creation complete.)
 model_deploy_op = gcc_aip.ModelDeployOp(
     project=project_id,
     endpoint=endpoint_create_op.outputs["endpoint"],
     model=model_upload_op.outputs["model"],
     deployed_model_display_name=TRAINED_POLICY_DISPLAY_NAME,
     machine_type=ENDPOINT_MACHINE_TYPE)

The execution graph of the pipeline looks like the following:

Refer to the GitHub repo for detailed instructions on how to implement and test KFP components, and how to run the pipeline with Vertex Pipelines.

Applying this demo to your own RL projects and production

You can replace the MovieLens simulation environment with a real-world environment where RL quantities like observations, actions and rewards capture relevant aspects of said real-world environment. Based on whether you can interact with the real world in real-time, you may choose either on-policy (showcased by the step-by-step demo) or off-policy (showcased by the pipeline demo) training and evaluation.

If you were to implement a real-world recommendation system, here’s what you’d do:

You would represent users as some user vectors. The individual entries in the user vectors may have actual meanings like age. Alternatively, they may be generated through a neural network as user embeddings. Similarly, you would define what an action is and what actions are possible, likely all items available on your platform; you would also define what the reward is, such as whether the user has tried the item, how long/much the user has spent on the item, user rating of the item, and so on. Again, you have the flexibility to decide on representations for framing the problem that maximize performance. During training or data pre-collection, you may randomly sample users (and build the corresponding user vectors) from the real world, use those vectors as observations to query some policy for items to recommend, and then apply that recommendation to users and obtain their feedback as rewards.

This RL demo can also be extended to ML applications other than recommendation system. For instance, if your use case is to build an image classification system, then you can frame an environment, where observations are the image pixels or embeddings, actions are the predicted classes, and rewards are feedback on the predictions’ correctness.

Conclusion

Congratulations! You have learned how to build reinforcement learning solutions using Vertex AI in a fully managed, modularized and reproducible way. There is so much you can achieve with RL, and you now have many Vertex AI as well as Google Cloud services in your toolbox to support you in your RL endeavors, be it production systems, research or cool personal projects.

Additional resources

• [Recap] step-by-step demo link: GitHub link
• [Recap] end-to-end pipeline demo: GitHub link
• TF-Agents tutorial on bandits: Introduction to Multi-Armed Bandits
• Vertex Pipelines tutorial: Intro to Vertex Pipelines