If you’re managing a crucial application that has to be fully fault-tolerant, you need your system to be able to handle every fault, no matter the type and scope of failure, with minimal downtime and data loss. Protecting against these faults means juggling numerous variables that can impact performance as well as recovery time and cost.

Today’s managed database services take over the operational complexity that used to exist for database administrators. Growing your organization’s tolerance required adding machines, compute, and storage, plus the operational costs of IT management: performing backups, writing scripts, creating dashboards, and carrying out testing to make sure your platform is ready when problems arise–all in a secure way.

At Google, our Cloud SQL managed database service offers three fault tolerance mechanisms —backup, high availability, and replication—and there are three major factors to consider for each of them:

• RTO (recovery time objective): When a failure happens, how much time can be lost before significant harm occurs?
• RPO (recovery point objective): When a failure happens, how much data can be lost before significant harm occurs?
• Cost: How cost-effective is this solution?  

We’ve heard from customers like Major League Baseball, HSBC, and Equifax that they have strict data-protection needs and require highly fault-tolerant multi-region applications—and they’ve all chosen Cloud SQL to meet those needs. 

Let’s take a closer look at how the decision-making process plays out for each recovery solution.  

High availability (HA)

If your application is business critical, you require minimum RTO and zero RPO— a high availability configuration ensures that you and your customers are protected. If the primary instance fails, there’s another standby instance ready to take over with no data loss. There’s an additional cost here, but doing this manually brings a great operational cost, since you have to detect and verify the fault, do the failover, and make sure it’s correct—you can’t have two primary instances or you risk data corruption—then finally connect the application to the new database.

Cloud SQL removes all that complexity. Choose high availability for a given instance and we’ll replicate the data across multiple zones, synchronously, to each zone’s persistent disk. If an HA instance has a failure, you don’t have to think about when to fail over because Cloud SQL detects the failure and automatically initiates failover, for a full recovery and no data loss within minutes. Cloud SQL also moves the IP address during failover so your application can easily reconnect. MLB, for example, uses Cloud SQL high availability to serve prediction data to live games with minimal downtime. Dev/test instances don’t need those same guarantees, but can use local backups to recover from any potential failure.

Cross-region replica

If a whole Google Cloud region goes down you still need your business to continue to run. That’s where cross-region replication comes in, a hot standby replica in another Google Cloud region provides RTO of minutes and RPO typically less than a minute . If you create a read replica in a region separate from your primary instance and you get hit with a regional outage, your application and database can start serving customers from another region within minutes. But this solution can be complex and enabling it yourself can be difficult and time-consuming. Securing cross-geography traffic demands end-to-end encryption and can bring connectivity issues too. 

This is where the fully managed Cloud SQL solution shines. We offer MySQL, PostgreSQL and SQL Server database engines as a cross-region replication solution that’s easily configured and bolstered by Google’s interconnected global network. Just say, “I’m in U.S. East, I want to create a replica in U.S. West,” and it’s done, reliably and securely.

Backup

When you suffer data loss because of an operations error (for example, a bug in a script dropped your tables) or human error (for example, someone dropped the wrong table by accident), backups help you restore lost data to your Cloud SQL instance. Our low cost backup mechanism features point-in-time, granular recovery, meaning that if you accidentally delete data or something else goes wrong, you can ask for recovery of, for example, the state of that database down to the millisecond, such as Monday at 12:53pm. Your valuable data is replicated multiple times in multiple geographic locations automatically. This enables the automatic handling of failover in cases of major failure. You can always rest assured that your database is available and data is secure, even in the times of major failure crises. 

Cloud SQL provides automated and on-demand backups. With automated backups, Google manages the backups so that you can easily restore them when required. Also, the scheduled backing is automatically taken by default. With on-demand backup, you can create a backup at any time. This could be useful if you are about to perform a risky operation on your database, as Cloud SQL lets you select a custom location for your backup data. When the backup is stored in multiple regions, and there’s an outage in the region that contains the source instance, you can restore a backup to a new or existing instance in a different region. This is also useful if your organization needs to comply with data residency regulations that require you to keep your backups within a specific geographic boundary.

Putting it all together

For critical workloads, MLB configures their Cloud SQL instances with backups, high availability, and cross-region replication. Doing so ensures they can recover from many failure types.

• To recover from human error (“Oops, I didn’t mean to delete that”), MLB uses backups and point-in-time recovery to recovery to a millisecond or specific database transaction
• To automatically recover from primary instance failures and zonal outages, MLB uses Cloud SQL’s high availability configuration
• To protect against regional outages, MLB uses cross-region replication

Creating a robust configuration, like MLB did, takes just a few minutes. Get started in our Console or review documentation.

At Quantum Metric, we’re in the business of bringing our customers business insights that are based on customer experience data and analytics for mid-market and Fortune 500 companies.

Our software, powered by big data, machine intelligence, and Google Cloud, helps our customers identify, quantify, prioritize and measure opportunities to improve digital experiences.

As companies move to a more agile product lifecycle, including continuous deployment and continuous integration, they’re finding that it’s critical to receive perpetual quantified feedback and insights from their data in real time to understand where the largest opportunities exist. 

Each year, billions of customer interactions are captured through browsers or mobile apps on PCs, tablets, and mobile devices. This data, fed into the Quantum Metric platform, can show if a customer had a password problem they couldn’t solve or struggled when trying to purchase something and abandoned their cart.

It also can show if the customer tried in vain to complete an online change to their service provider’s subscription, to reach tech support, or couldn’t find the size or color they were looking for while shopping online.

Most importantly, the Quantum Metric platform quantifies the business value of the issue, helping organizations prioritize where they can make the largest impact to their business.

Success overwhelms our initial architecture
Initially, the Quantum Metric experience analytics software ran on a MySQL open source relational database management system (RDBMS). The MySQL RDBMS worked great for simple queries, when there was a specific question to ask of the data.

Soon, though, we knew we needed to offer more advanced data science capabilities. Our bigger customers wanted to ask questions across very large data sets—days, weeks, months, and years worth of data. They wanted to pose iterative questions using complex filters to answer their most challenging business questions. 

With more complex queries across more data, response times from our RDBMS went from 100 to 500 milliseconds to as long as 20 minutes.

That delay was slowing down our ability and time to insights, which also reduced the value we could provide to our customers, since iterative exploration and analysis requires real-time query responses. Because of the need for real-time responses, there were certain questions that we just weren’t able to ask of the data. It became clear that we needed a much more robust data warehouse solution. 

There were also operational challenges with MySQL and massive-scale data ingestion. We spent a lot of time into the wee hours of the night and morning handling errors and recovering databases.

We tried to address these challenges by sharding, partitioning, and indexing the data to optimize for the types of questions customers were asking. But the problems were escalating and happening more often, from once a month across the customer base to monthly for at least 20 different customers.

We could tune the platform for today and tomorrow’s workload, with good guesses at where indexes could be used, but we simply couldn’t continue to horizontally scale MySQL in a cost-efficient and operationally efficient manner. 

Speed breeds innovation
Once we started exploring options that could better scale with our business, we looked at NoSQL technologies like Cassandra (a partitioned row-store database), MySQL’s Column Store (a columnar store database), and Vertica (a columnar store database)—each with unique ways of handling data storage and accessibility.

But with high volumes of complex queries across large data stores, all of these solutions began to fail, bogged down with multiple, simultaneous users. We could have solved the problems with more raw compute and storage, but it would have been prohibitively expensive to run and require a large team to operate. 

We then decided to try BigQuery, and it was transformative.

We connected our front end to BigQuery via APIs. Once data is 15 minutes old, it is automatically extracted, loaded, and transformed (ETL) to BigQuery.

We continuously update the legacy MySQL RDBMS so its data is integrated with BigQuery data when queries require real-time data. Most query response times are within 100-200 milliseconds, matching what we initially experienced with MySQL.

When traffic from our customers scales up, we can now scale on-demand to accommodate it, thanks to BigQuery’s hundreds of thousands of CPUs. Our customers no longer run into slow response times, and we’ve gained confidence that we can offer them—and their users—advanced insights and better experiences without delay.

More importantly, with this scale of query power, we were able to build data science algorithms into the platform, which iteratively query BigQuery based on the results, and help quantify the impact of a specific issue to a specific segment of users. Adding these capabilities was possible because of the massive scale of BigQuery. 

In addition to new insights and fast response times, we wanted our customers to be able to ask complex questions using very simple language.

For example: “Show me high-loyalty customers, located in specific geographic areas, who visited the web site at least five times, based on specific campaigns, and never booked a seat on a flight.”

This was exactly the kind of query that was used by a major U.S. airline to understand the multi-million dollar impact of a failure affecting their most valuable customers: their high-loyalty members.

And this was all done while maintaining the highest standard of care of customer data and privacy by default, using multiple layers of encryption of data in transit, at rest, and a unique military-grade encryption approach. This approach encrypts PII, including even session cookies, with a RSA-2048 key available only to a select few and used for use cases such as fraud analysis.   

It’s no exaggeration to say that BigQuery has totally transformed our business. It provides the petabyte scale and speed we were missing, in addition to taking care of operational maintenance, a task that was burying our team with MySQL.

We’re now able to support some of the largest companies in the world that require real-time, petabyte-scale analytics. That lets them serve more customers faster with higher quality, and take advantage of BigQuery’s power and scale to innovate.

There are other cloud solutions that can address petabyte analytics, but the most unique value proposition of BigQuery was its on-demand scaling and operational management, with extremely cost-effective pay-as-you-go billing. While today we are at a scale where we have round-the-clock querying needs, our early days had very sporadic query loads where we needed instant scale, then a long lull of nothing. The unique business model of BigQuery’s pay-by-bytes-scanned allowed us to have access to a massive-scale querying platform without breaking the bank. 

Using BigQuery powers better customer experience and reduces purchasing friction
Among the many features of Quantum Metric is the ability to replay online customer sessions. In the example below from a mobile e-commerce site, each action is displayed chronologically. Why did this customer’s transaction fail?

Diving deeper, the session replay shows that the user tried to change the item quantity in the checkout cart, which resulted in a failed API call. Powered by BigQuery, Quantum Metric can then show how many other end users had this issue, with a simple click of “Show More Errors Like This.”

With BigQuery’s massive scale, Quantum Metric will then quantify the impact of that issue, so companies can prioritize which issues need attention immediately. If this is the issue that’s impacting the business the most, our customer can use a single click to open a Jira ticket, forwarding the discovery to their product and engineering teams. Those teams can then re-engineer the experience in near-real time, addressing the failed API call and cutting out the frustrating time it takes for engineers to reproduce the issue.

Once we had a powerful back end in BigQuery, we realized that Quantum Metric’s platform could be used to ask complex questions from vast datasets. We built some of the processes that data scientists use to formulate those queries right into our product.  

For example, we added one series of processes to our platform to help customers understand whether a suspect issue is really impacting end-user experience. Is it something that should be prioritized and fixed? Does this really affect the user experience? Does it have financial impact? These and other questions can be pre-defined as a complex query in Quantum Metric to let our customer quickly gain insight on how an issue is impacting the business. Customers were blown away when they heard this was possible. It was the holy grail for what they were looking for in data science. It really sets us apart from our competition.

Today, with every company heavily dependent on data, those companies that can uncover and act on insights fastest are the ones that will succeed. BigQuery gives us the data warehouse platform we need to provide our customers with fast, reliable technology tools. It frees us from having to deal with the minutiae of technology infrastructure operations, so we can focus on finding and extracting the magic in customer data. With the power and scale of BigQuery, combined with the real-time capture of every user experience with 100% fidelity, we’re able to offer a self-service analytics platform that provides insights into digital journey friction points and acts as the indisputable arbitrator of truth. 

Founded in 2006, PaGaLGuY started as a forum that enabled students, typically aged between 20 and 30 to discuss and seek advice on academic issues. By 2011, PaGaLGuY had increased its traffic to about 250,000 page views per month. The business is now one of India’s largest education networks and provides an app that users can download to their Android and iOS devices.

Over the past six years, PaGaLGuY has extended its service to include video advice from experts on education topics and grown the number of views of its pages to 1.5 million per day. As Head of Technology for the business, Sandeep Kalidindi has played a key role in ensuring PaGaLGuY is as engaging as possible to users. “Because the product is advertising based, the greater the user engagement, the greater the advertising revenue,” Kalidindi explains.

Google Cloud Platform Results

• Supported growth to 1.5 million page views per day and demand spikes that see requests increase from about 90 per second to about 1,200 per second
• Reduced API latency from about 1 second to about 40 milliseconds
• Reduced system administration time from three to four days per week to 30 minutes every two weeks

In 2015, PaGaLGuY’s senior management team decided to deliver an even more relevant experience for users of the education network. “The core thing we had to do was personalise the experience for each and every student that visited PaGaLGuY,” Kalidindi says. “So we had to capture each student’s data to customise what they see when they open the site.”

The business also found traffic to the network was straining its infrastructure. During demand peaks, created by exams involving as many as 5 million students, PaGaLGuY would be inaccessible for periods of 30 minutes to one hour. Furthermore, average API latency had climbed to an unacceptable 1 second, compromising performance.

PaGaLGuY needed to access extensive compute resources to undertake its planned change. Had the business relied on a physical technology architecture to undertake the transformation, it would have had to purchase capacity equivalent to 16 new servers. “There was no way with a small team we could grow to that extent in a short time,” Kalidindi says. “This was the right moment for us to explore cloud services.”

The business established two primary requirements the selected cloud service needed to meet. First, PaGaLGuY had to be able to scale the platform with costs rising only in proportion to the increase in resources consumed. Accordingly, the business would have to minimise the number of employees required to manage the cloud environment. Second, the platform had to give PaGaLGuY easy access to student data and the ability to undertake prompt, granular analysis.

PaGaLGuY reviewed available public cloud services and determined that Google Cloud Platform (GCP) was the best fit for its business. “Google Cloud Platform was considerably more mature than the alternatives, with a high degree of automation and a suite of managed services,” Kalidindi says. PaGaLGuY management then discussed with Google how to optimise cost, performance and availability of its personalised education network on GCP.

With assistance from Google and business transformation specialists Searce, PaGaLGuY was able to deliver the platform into production on GCP in 10 months. “Searce was very proactive in ensuring the environment met our needs and allowing us to gain priority access to Google services in development,” Kalidindi says. “Their team was integral to the success of the migration.”

PaGaLGuY has been running in production in GCP for two years. The education network’s GCP architecture comprises a scalable back-end built on Google App Engine; a managed environment for its containerised applications in Google Kubernetes Engine; messaging-oriented middleware through Google Cloud Pub/Sub; a relational database in Google Cloud SQL; a managed data analytics warehouse running in Google BigQuery; stream and batch data processing through Google Cloud Dataflow; and object storage in Google Cloud Storage.

PaGaLGuY has leveraged GCP services to break down its platform application from a monolithic build to a series of microservices running in Google App Engine that enable independent deployment cycles, minimise test and quality assurance overheads and provide clearer monitoring and logging.

Running on GCP has enabled PaGaLGuY to add new personalisation features and grow fourfold without having to add any new engineers or administrators to accommodate the increased traffic. The business has also used the platform to seamlessly collect and aggregate students’ data for analysis, reporting and delivering a more targeted user experience. Furthermore, PaGaLGuY has been able to provide its management team with direct access to Google BigQuery to scrutinise data rather than require them to wait at least a day to view reports created by the product or technology teams.

Support demand peaks of 1,200 requests per second

“Thanks to Google Cloud Platform, we can easily support demand peaks that see requests per second rise from an average 90 per second to about 1,200 per second for as long as 45 minutes,” Kalidindi says. Due to GCP’s scalability, PaGaLGuY can ensure its education network remains available and performance remains consistent during those periods.

Latency cut to 40 milliseconds

The business has also reduced average API latency from 1 second to about 40 milliseconds. Furthermore, using GCP has enabled PaGaLGuY to automate most of its processes and reduce system administration requirements from three to four days a week across its team members to about half an hour per week.

The performance of GCP has transformed PaGaLGuY’s culture and processes. “Once our team was exposed to Google Cloud Platform and understood the superiority of the platform, our mindset changed from ‘let us do everything on our own’ to ‘let us do what we do best’ and delegate the remainder,” Kalidindi says. The quality of the service provided by GCP means PaGaLGuY effectively considers the cloud provider as part of its team. “We are always eager to see what new services are being launched and are extremely excited about what Google Cloud Platform can provide as part of its roadmap.” he concludes.

Cloud Spanner is a fully managed relational database with unlimited scale, strong consistency, and up to 99.999% availability. It is designed for highly concurrent applications that read and update data, for example, to process payments or for online game play. To ensure the consistency across multiple concurrent transactions, Cloud Spanner uses a combination of shared locks and exclusive locks to control access to the data. In this blog, we will explore the different types of locks present in Cloud Spanner. We will also discuss some common cases of transactional locking in Cloud Spanner, and what to look out for to detect when these cases might be occurring.

What is a lock?

Before we get into the details, let us first quickly recap and define a lock in the context of database systems.

“Locks” in databases are a mechanism for concurrency control. Locks are typically held on a resource, which may mean rows, columns, tables or even entire databases. When a resource is locked by a transaction, it cannot be accessed by another transaction until the lock is released.

Timeline view of transactions

Before we proceed to discuss transaction locking in the context of read and read-write transactions, it is important to recap the timeline view of the transactions in Spanner.

Please also keep in mind the concept of write buffering in transactions, since we refer to it in the Common Transaction Patterns section below. Write buffering refers to the Cloud Spanner server(s) accepting the writes. Note that these writes are not durable until a commit has been performed.

Types of locks in Cloud Spanner

Cloud Spanner operations acquire locks when the operations are part of a read-write transaction. Read-only transactions do not acquire locks. Unlike other approaches that lock entire tables or rows, the granularity of transactional locks in Spanner is a cell, or the intersection of a row and a column. This means that two transactions can read and modify different columns of the same row at the same time. To maximize the number of transactions that have access to a particular data cell at a given time, Cloud Spanner uses different lock modes.

Here is a brief description of the different lock types. Learn more about each lock type in the Cloud Spanner documentation.

1. ReaderShared Lock – Acquired when a read-write transaction reads data.
2. WriterShared Lock – Acquired when a read-write transaction writes data without reading it.
3. Exclusive Lock – Acquired when a read-write transaction which has already acquired a ReaderShared lock tries to write data after the completion of read. It is a special case for a transaction to hold both the ReaderShared lock and WriterShared lock at the same time.
4. WriterSharedTimestamp Lock – Special type of lock acquired when inserting new rows with the transaction’s commit timestamp as part of the primary key.

Handling Lock Conflicts

Since read-write transactions use locks to execute atomically, they run the risk of deadlocking. For example, consider the following scenario: transaction Txn1 holds a lock on record A and is waiting for a lock on record B, and Txn2 holds a lock on record B and is waiting for a lock on record A. The only way to make progress in this situation is to abort one of the transactions so it releases its lock, allowing the other transaction to proceed.

Cloud Spanner uses the standard “wound-wait” algorithm to handle deadlock detection. Under the hood, Spanner keeps track of the age of each transaction that requests conflicting locks. It also allows older transactions to abort younger transactions, where “older” means that the transaction’s earliest read, query, or commit happened sooner.

Common Transaction Patterns

Armed with the basics of transactions and locks in Cloud Spanner, we will now walk through a few practical use-cases. We take the example of an application which queries and updates users’ balance in a table named accounts. The accounts table has the following columns –

Case 1: Transaction waiting to get exclusive lock because of higher priority shared-lock

Sequence

1. Txn1 begins.
2. Txn2 begins.
3. Txn2 reads the table and acquires a ReadShared Lock.
4. Txn1 buffers its write.
5. Txn1 tries to commit, but since Txn2 has higher priority (because it has executed its first operation first), Txn1 will have to wait until Txn2 releases the ReadShared Lock.

After committing, Txn2 releases its ReadShared Lock. Txn1 is now able to upgrade to an Exclusive Lock. It acquires the Exclusive Lock and commits.

Note 1: UPDATE WHERE is always transformed into SELECT WHERE, so an UPDATE statement is not a blind write, but a read-write operation.

Note 2: While the above example considers the use-case of querying and updating a single row, the same transaction pattern is also applicable across a key-range.

What to watch out for

• Timeouts due to Transactions waiting for acquiring locks. Refer to transaction statistics and lock statistics to understand how to detect this.

When does this typically happen

• Concurrent updates to a single key/key-range in a table.

Case 2: Transaction aborted because of concurrent execution succeeding

Sequence

1. Txn1 begins.
2. Txn2 begins.
3. Txn1 reads a row from the table and acquires a ReadShared Lock.
4. Txn2 reads the same row as Txn1 and acquires a ReadShared Lock.
5. Txn1 buffers its write.
6. Txn2 buffers its write.
7. Txn1 tries to commit, has higher priority (because it has executed its first operation first), it will get priority in upgrading to an Exclusive Lock. It acquires the Exclusive Lock and commits.

Since Txn1 has committed and was the higher priority transaction, Txn1 will abort Txn2.. The abort will likely happen before Txn2 tries to commit.

What to watch out for

• High number of transaction aborts due to wounding of transactions. Refer to transaction statistics and lock statistics to understand how to detect this.

When does this typically happen

• Concurrent updates to a single key/key-range. Typically this happens in the case of hot keys being present in the table.

Case 3: Transaction waiting to get exclusive lock because of higher priority shared-lock & getting aborted because of prior succeeding concurrent execution

Sequence

1. Txn1 begins.
2. Txn2 begins.
3. Txn2 reads the table and acquires a ReadShared Lock.
4. Txn1 reads the table and acquires a ReadShared Lock.
5. Txn1 buffers its write.
6. Txn2 buffers its write.
7. Txn1 tries to commit, but since Txn2 holds a ReadShared Lock, it has to wait until it gets cleared.
8. Since Txn2 has higher priority (because it has executed its first operation first), it will acquire the Exclusive Lock first. Txn2 upgrades its ReaderShared Lock to an Exclusive Lock and commits.

Finally, after Step 8, when Txn1 tries to commit (since ReaderShared Lock from Txn2 is now cleared) it gets aborted. This is done to prevent deadlock with the higher priority transaction (Txn2).

Note: In Case 1, even though Txn2 had acquired a ReaderShared Lock earlier as in Case 3, it never upgraded to an Exclusive Lock (since there were no writes). Hence there was no need to abort Txn1, it could simply wait for Txn2 to release its ReaderShared lock and then commit.

What to watch out for

• High number of transaction aborts due to high contention between transactions. Refer to transaction statistics and lock statistics to understand how to detect this.

When does this typically happen

• Concurrent updates to a single key/key-range. Typically this happens in the case of hot keys being present in the table.

Recommendations

In order to mitigate these issues and reduce their occurrence. We recommend adopting the following best practices:

• If you need to perform more than one read at the same timestamp, and know in advance that you only need to read, consider using a read-only transaction. Because read-only transactions don’t write, they don’t hold locks and they don’t block other transactions. Further, read-only transactions never abort, so you don’t need to wrap them in retry loops.
• Always acquire ReadShared Locks on the smallest subset of keys or key ranges. This reduces the chances of lock contention.
• Analyze your code to only include critical path code within a transaction. Avoiding unneeded remote calls or complex, long-running business logic is a good way to ensure a transaction process quickly.
• Analyze your needs for multi-split transactions. Since transactions that update more than one split use a 2-phase commit protocol, they hold locks for a longer duration, thereby increasing chances of lock contention.

Get started today

Spanner’s unique architecture allows it to scale horizontally without compromising on the consistency guarantees that developers rely on in modern relational databases. Try out Spanner today for free for 90 days or for as low as $65 USD per month.

Over the past two decades, consumers have become accustomed to receiving personalized recommendations in all facets of their online life. Whether that be recommended products while shopping on Amazon, a curated list of apps in the Google Play store, or relevant videos to watch next on YouTube. In fact, in a Verge article “​​How YouTube perfected the feed: Google Brain gave YouTube new life,” the Google Brain team reveals how their recommendation engine has impacted the platform with “more than 70 percent of the time people spend watching videos on the site being driven by YouTube’s algorithmic recommendations” thereby increasing time spent on the platform by 20X in three years. 

It’s become clear that personalized recommendations are no longer a differentiator for an organization but rather something consumers have come to expect in their day-to-day experiences online. So what should you do if you are behind the curve and want to get started or simply want to improve upon what you already have? While there are all sorts of techniques, from content-based systems to deep learning methods, our goal in this recommender-focused blog series is to demystify three available approaches to building recommendation systems on Google Cloud: Matrix Factorization in BigQuery Machine Learning (BQML),  Recommendations AI, and deep retrieval techniques available via the Two-Tower built-in algorithm.

One of these approaches can be used to meet you where you are in your personalization journey, no matter if you are just starting or if you are well into it. This first blog post will introduce our three approaches and when to use them. 

What is Matrix Factorization and how does it work?

Collaborative filtering is a foundational model for building a recommendation system as the input dataset is simple and the embeddings are learned for you. How does Matrix factorization fit into the mix you might be wondering?  Matrix factorization is simply the model that applies collaborative filtering. BQML enables users to create and execute a matrix factorization model by using standard SQL directly in the data warehouse. 

Collaborative filtering begins by creating an interaction matrix. The interaction matrix represents users as a row and items as columns in your dataset. This interaction matrix often is sparse in nature as not all users will have interacted with many items in your catalog. This is where embeddings come into play. Generating embeddings for users and items not only allows you to collapse many sparse features into a lower dimensional space but they also allow you to derive a similarity measure so that similar users/items fall nearby in the embedding space. These similarity measures are key as collaborative filtering uses similarities between users and items to make the end recommendations. The underlying assumption being that similar users will like similar items whether that be movies or handbags. 

What’s required to get started?

To train a matrix factorization model you need a table that includes three input columns: user(s), item(s), and an implicit or explicit feedback variable (e.g., ratings is an example of explicit feedback). With the base input dataset in place, you can then easily run your model in BigQuery after specifying several hyperparameters in your CREATE MODEL SQL statement. Hyperparameters are available to specify the number of embeddings, the feedback type, the amount of L2 regularization applied and so on.    

Why use this approach and who is it a good fit for?  

As mentioned earlier, Matrix Factorization in BQML is a great way for those new to recommendation systems to get started. Matrix factorization has many benefits: 

• Little ML Expertise: Leveraging SQL to build the model lowers the level of ML expertise needed
• Few Input Features: Data inputs are straightforward, requiring a simple interaction matrix
• Additional Insight: Collaborative filtering is adept at discovering new interests or products for users 

While Matrix Factorization is a great tool for deriving recommendations it does come with additional considerations and potential drawbacks depending upon the use case. 

• Not Amenable to Large Feature Sets: The input table can only contain two feature columns (e.g., user(s), item(s)). If there is a need to include additional features such as contextual signals, Matrix factorization may not be the right method for you.  
• New Items: If an item is not available in the training data, the system can’t create an embedding for it and will have difficulty recommending similar items. While there are some workarounds available to address this cold-start issue, if your item catalog often includes new items, Matrix factorization may not be a good fit.  
• Input Data Limitations: While the input matrix is expected to be sparse, training examples without feedback can cause problems. Filtering for items and users that have at least a handful of feedback (e.g., ratings) examples can improve the model. More information on limitations can be found here. 

In summary, for users with a simplified dataset looking to iterate quickly and develop a baseline recommendation system, Matrix Factorization is a great approach to begin your personalization AI journey. 

What is Recommendations AI and how does it work?

Recommendations AI is a fully managed service which helps organizations deploy scalable recommendation systems that use state-of-the-art deep learning techniques, including cutting-edge architectures such as two-tower encoders, to serve personalized and contextually relevant recommendations throughout the customer journey.

Deep learning models are able to improve the context and relevance of recommendations in part because they can easily address the previously mentioned limitations of Matrix Factorization. They incorporate a wide set of user and item features, and by definition they emphasize learning successive layers of increasingly meaningful representations from these features. This flexibility and expressivity allows them to capture complex relationships like short-lived fashion trends and niche user behaviors. However, this increased relevance comes at a cost, as deep learning recommenders can be difficult to train and expensive to serve at scale. 

Recommendations AI helps organizations take advantage of serving these deep learning models and handles the MLOps required to serve these models globally with low latency. Models are automatically retrained daily and tuned quarterly to capture changes in customer behavior, product assortment, pricing, and promotions. Newly trained models follow a resilient CI/CD routine which validates they are fit to serve and promotes them to production without service interruption. The models achieve low serving latency by using a scalable approximate nearest neighbors (ANN) service for efficient item retrieval at inference time. And, to maintain consistency between online and offline tasks, a scalable feature store is used, preventing common production challenges such as data leakage and training-serving skew.  

What’s required to get started?

To get started with Recommendations AI we first need to ingest product and user data into the API:

• Import product catalog: For large product catalog updates, ingest catalog items in bulk using the catalogItems.import method. Frequent catalog updates can be schedule with Google Merchant Center or BigQuery
• Record user events: User events track actions such as clicking on a product, adding items to cart, or even purchasing an item. These events need to be ingested in real time to reflect the latest user behavior and then joined to items imported in the product catalog 
• Import historical user events: The models need sufficient training data before they can provide accurate predictions. The recommended user event data requirements are different across model types (learn more here)

Once the data requirements are met, we are able to create one or multiple models to serve recommendations: 

• Determine your recommendation types and placements:  The location of the recommendation panel and the objective for that panel impact model training and tuning. Review the available recommendations types, optimization objectives, and other model tuning options to determine the best options for your business objectives.
• Create model(s): Initial model training and tuning can take 2-5 days depending on the number of user events and size of the product catalog 
• Create serving configurations and preview recommendations: After the model is activated, create serving configurations and preview the recommendations to ensure your setup is functioning as expected before serving to production traffic

Once models are ready to serve, consider setting up A/B experiments to understand how newly trained models impact your customer experience before serving them to 100% of your traffic. In the Recommendations AI console, see the Monitoring & Analytics  page for summary and placement-specific metrics (e.g., recommender-engaged revenue, click-through-rate, conversion rate, and more).

Why use this approach and who is it a good fit for?  

Recommendations AI is a great way to engage customers and grow your online presence through personalization. It’s used by teams who lack technical experience with production recommendation systems, as well as customers who have this technical depth but want to allocate their team’s effort towards other priorities and challenges. No matter your team’s technical experience or bandwidth, you can expect several benefits with Recommendations AI: 

• Fully managed service: no need to preprocess data, train or hypertune machine learning models, load balance or manually provision you infrastructure – this is all taken care of for you. The recommendation API also provides a user-friendly console to monitor performance over time. 
• State-of-the-art AI: take advantage of the same modeling techniques used to serve recommendations across Google Ads, Google Search, and YouTube. These models excel in scenarios with long-tail products and cold-starts users and items
• Deliver at any touchpoint: serve high-quality recommendations to both first-time users and loyal customers anywhere in their journey via web, mobile, email, and more
• Deliver globally: serve recommendations in any language anywhere in the world at low-latency with a fully automated global serving infrastructure
• Your data, your models: Your data and models are yours. They’ll never be used for any other Google product nor shown to any other Google customer

For users looking to leverage state of the art AI to fuel their recommendation systems but need an existing solution to get up and running more quickly, Recommendations AI is the right solution for you. 

What are Two Tower encoders and how do they work?

As a reminder, in recommendation system design, our objective is to surface the most relevant set of items for a given user or set of users. The items are usually referred to as the candidate(s) where we might include information about the items such as the title or description of the item, other metadata about the item like language, number of views, or even clicks on the item over time. User(s) are often represented in the form of a query to a recommendation system where we might provide details about the user such as the location of the user, preferred languages, and what they have searched for in the past.   

Let’s start with a common example. Imagine that you are creating a movie recommendation system. The input candidates for such a system would be thousands of movies and the query set can consist of millions of viewers. The goal of the retrieval stage is to select a smaller subset of movies(candidates) for each user and then score and rank order them before presenting the final recommended list to the query/user.

The retrieval stage is able to refine our list of candidates by encoding both the candidate and the query data so they share the same embedding space. A good embedding space will place candidates which are similar to one another closer together and dissimilar items/queries farther apart in the embedding space.

Once we have a database of query and candidate embeddings we can then use an approximate nearest neighbor search method to then generate a list of final “like” candidates, i.e. find a certain number of nearest neighbors for a given query/user and surface final recommendations.

What’s required to get started?

At the most basic level, in order to train a two-tower model you need the following inputs:

• Training Data: Training data is created by combining your query/user data with data about the candidates/items. The data must include matched pairs, cases where both user and item information is available. Data in the training set can include many formats from text, numeric data, or even images. 
• Input Schema: The input schema describes the schema of the combined training data along with any specific feature configurations.

Several services within Vertex AI have come available that complement the existing Two-Tower built-in algorithm and can be leveraged in your execution: 

• Nearest Neighbor (ANN) Service: Vertex AI Matching Engine and  ScANN provide a high-scale and low-latency Approximate Nearest Neighbor (ANN) service so you can more easily identify similar embeddings.
• Hyperparameter Tuning Service: A hyperparameter tuning service such as Vizier can help you identify the optimal hyperparameters such as the number of hidden layers, the size of the hidden layers, and the learning rate in fewer trials. 
• Hardware Accelerators: Specialized hardware, such as GPUs or TPUs, can be valuable in your recommendation system to help accelerate experiments and improve the speed of training cycles. 

Why use this approach and who is it a good fit for?  

The Two-Tower built-in algorithm can be considered the “custom sports car” of recommendation systems and comes with several benefits: 

• Greater Control: While Recommendations AI uses the two-tower architecture as one of the available architectures it doesn’t provide granular control or visibility into model training, example generation, and model validation details. In comparison, the Two-Tower built in algorithm provides a more customizable approach as you are training a model directly in a notebook environment. 

• More Feature Options: The Two Tower approach can handle additional contextual signals ranging from text to images. 
• Cold Start Cases: Leveraging a rich set of features not only enhances performance but also allows the candidate generation to work for new users or new candidates.

While the Two-Tower built in algorithm is an excellent and best-in class solution for deriving   recommendations, it does come with additional considerations and potential drawbacks depending upon the use case.

• Technical ML Expertise Required: Two tower encoders are not a “plug and play” solution like the other approaches mentioned above. In order to effectively leverage this approach, appropriate coding and ML expertise is required. 
• Speed to Insight: Building out a custom solution via two-tower encoders may require additional time as the solution is not pre-built for the user.   

For users looking for greater control, increased flexibility, and have the technical chops to easily work within a managed notebook environment – the two-tower built in algorithm is the right solution for them. 

What’s next?

In this article, we explored three common methods for building recommendation systems on Google Cloud Platform. As you can see thus far, there are alot of considerations to take into account before choosing a final approach. In an effort to help you align more quickly we have distilled the decision criteria down to a few simple steps (see below for more details).

In the next installments of this series, we will dive more deeply into each method, explore how hardware accelerators can play a key role in recommendation system design, and discuss how recommendation systems may be leveraged in key verticals. Stay tuned for future posts in our recommendation systems series. Thank you for reading! Have a question or want to chat? Find authors here – R.E. [Twitter | LinkedIn], Jordan [LinkedIn], and Vaibhav [LinkedIn].

Acknowledgements

Special thanks to Pallav Mehta, Henry Tappen,Abhinav Khushraj, and Nicholas Edelman for helping to review this post. 

References

• Announcing ScaNN: Efficient Vector Similarity Search
• Factorization Machines for Item Recommendation with Implicit Feedback Data