Today we’re announcing the HarbourBridge Schema Assistant, which provides a guided schema-design workflow for migrating from MySQL or PostgreSQL to Spanner. HarbourBridge imports dump files (from mysqldump or pg_dump) or directly connects to your source database, and converts the source database schema to an equivalent Spanner schema. The new Schema Assistant capability displays the source schema and Spanner schema side-by-side, highlights errors and walks you through a series of steps to validate and optimize your Spanner schema. It also produces a browsable assessment report with an overall migration-fitness score for Spanner, a table-by-table detailed analysis of type mappings and a list of features used in the source database that aren’t supported by Spanner. It supports editing of table and column names, column types, primary keys and constraints, as well as dropping of tables, columns, foreign keys and secondary indexes.

The new Schema Assistant complements HarbourBridge’s existing data and schema migration capabilities and is a critical step towards our goal of building a complete open-source migration toolkit. HarbourBridge continues to support command-line schema and data migration and turn-key Spanner evaluation.

Complementing the bulk data migration capabilities of HarbourBridge, we are also announcing the ability to migrate change events from MySQL to Cloud Spanner.

Supported Features in Schema Assistant

1. Global type mapping. Users can customize the global mapping for how types should be mapped to Spanner consistently across the schema. For example, mapping large integers in source schema to Spanner’s NUMERIC.
2. Local type mapping. Users can override the custom type mapping for a given table/column.
3. Session management. A session keeps track of all the changes made to the schema mapping.
4. Customization of secondary indexes. Users can add, edit and delete secondary indexes to optimize their Spanner performance.
5. Customization of foreign keys and interleaved tables. Table interleaving is an important design consideration when migrating to Cloud Spanner as explained in more detail in this blog post.

Features in the pipeline

We are already working to further expand the supported set of schema editing features and welcome your feedback. We are particularly excited to expand the Schema Assistant’s design recommendations for optimizing Spanner schemas e.g. in-depth recommendations for primary key design.

HarbourBridge is open source and we gladly accept contributions from the wider community.

Over the last decade, financial service organizations have been adopting a cloud-first mindset. According to InformationWeek, lower costs and enhanced scalability were the biggest drivers for cloud adoption in financial services, and cloud-native applications allow access to the latest technology and talent, enabling adopters to rebuild transaction processing systems capable of supporting very high volumes and low latency.

Both Neo4j and Google Cloud have been using relationship-based data representations since the beginning, and we’re dedicated to using this technology to help financial services customers drive business transformation. We are excited about the prospects of financial services (FinServ) cloud systems and believe that graph data in the cloud can help solve significant challenges in the industry.

Data Challenge #1: Risk Management and Compliance

First among the top concerns for any CIO moving to the cloud is risk management and compliance. Disconnected, uncontextualized, or stale data create opportunities for fraud and financial crimes to occur. The fact is when it comes to FinServ, the question is not “if” but rather how often an attack will occur.  Unfortunately, incidents have been trending upward over the last decade, and COVID has only exacerbated this reality. Financial crimes affect the bottom line both in the remediation of these crimes and in intangibles like brand value.  

Add to this the complexity of international banking, which makes “compliance” a moving target. Penalties due to noncompliance are a constant concern to any FinServ organization.

The tabular representation of information with a fixed number of columns that never change prevents a description of an ever changing world with changing characteristics. Relational databases are great if the world you describe does not move fast but have limitations when data structures are highly interlinked and not homogeneous.

Neo4j Aura on Google Cloud provides a foundation for creating dynamic, futureproof, scalable applications that adhere to the security standards and protocols today’s financial services organizations require to meet the challenges of finding and preventing bad actors. This also includes enterprise scalability; reaching over 1 Billion nodes and relationships to streamline queries and provide solutions that meet regulatory and privacy compliance across geographies. Neo4j has helped some organizations save billions of USD in fraud in the first year of deployment alone.  

What makes graph technology the best choice for fraud detection use cases is that the relationships between the data-points are as important as the data-points themselves. Let’s take as an example, one John Smith approaches a multi-national banking institution to manage the primary account for his new holding corporation.  

While no one has any record of John R Smith Holdings LLC, the bank’s application built on graph technology understands that there are several well-known entities owned by John Smith Holdings. The application also identifies several well-known board members who bank with this institution. Due to this relationship-driven approach, the bank now understands John R Smith is not “John Smith,” who previously attempted to open an account for his holding corporation, which had no information associated with it prior to two months ago.

Data Challenge #2 Manual Processes and Inefficiencies 

The ubiquity of the cloud offers an opportunity to deploy automation at unprecedented levels to tackle the errors and inefficiencies that manual processing allows to creep into processes. When data comes from disparate, perhaps legacy systems – which may have become siloed and “untouchable” over the years – further complexity arises. As an example, if someone in sales types “John Smith” into a CRM system not knowing that John R Smith is the spelling in the customer data master, it may result in two separate and potentially conflicting records. Being able to join those records together in a mastered view helps to solve this problem. In addition, low data quality equates to an increase in risk, costs, and implementation times for new systems. 

Neo4j Aura on Google Cloud provides automation and artificial intelligence (AI) that reduces manual processes and the errors that accompany them. In this graph architecture each node, which can represent a person, will have labels, relationships, and properties associated with it. This allows for the use of AI which can easily understand that John Smith in the CRM is the same John R Smith in the customer master. The information contained in Neo4j can be connected bi-directionally to ensure consistency across applications and data sources. 

One of the benefits of this approach is that linking information allows organizations to keep the full value of the data, rather than forcing the data into predetermined tabular representations, with the risk of losing valuable information and insights.

Data Challenge #3: Customer Engagement and Insight

Another significant concern is the high expectations today’s customers have for every interaction. End users are accustomed to predictable experiences on their digital devices, and FinServ apps are no exception. Added to this, the “Covid economy” has driven digital adoption significantly across demographics; even among customers who might traditionally have used in-person services. This also equates to increased expectations for personalized, predictable experiences with every digital interaction. We know that latency has always been a key consideration for financial trading, but a recent ComputerWeekly study showed that every financial organization should ensure their visible latency is at 10 milliseconds or less. Customers no longer accept their broadband is at fault.

Finally, blind spots in the customer journey often result in dissatisfaction, which ultimately leads to increased churn. Without gaining actionable insights from your customers, there is no room to innovate and iterate on what they are looking for in your products and services. And this translates to losing market share and competitive advantage.

The NoSQL architecture, specifically the dynamic schema and structure of Neo4j Aura gives you the ability to take charge of your data and make changes according to your development cycles or newer data models. This equates to faster builds, more comprehensive releases and a wider, richer data-set that can be contextualized and understood instantly. Graph technology is the logical choice for building a Customer 360 application. Under this approach organizations not only get valuable insight into the individual client’s behavior and patterns, but also those of their family, friends and colleagues. This allows for stronger personalization, targeted campaigns and successful execution, resulting in increased customer satisfaction and retention levels.

Graph Technology on Google Cloud

Neo4j is a recognized leader in graph database technology and the only fully integrated graph solution on Google Cloud, helping to fill a common need for Google Cloud customers. Both Neo4j and Google Cloud are invested in continuing to grow our partnership and mutual product direction.  

You can find and deploy the Neo4j graph database straight from the Google Cloud marketplace, whether you want to download the software for an on-premises deployment, use the virtual machine image, or use the hosted solution, Aura on Google Cloud, the graph database-as-a-service. In any deployment, you get the same enterprise-grade scalability, reliability, and connectivity along with successful, repeatable use cases you can rely on to resolve your particular challenges and integrated billing. 

For a real-world example of how graph technology can optimize financial services, you can read our Case Study with fintech Current. Current, a leading U.S. financial technology platform with over three million members, used Neo4j Aura on Google Cloud to create a personalization engine based on client relationships. 

To learn more about Neo4j Aura on Google Cloud for FinServ organizations, register for our webinar on Thursday, December 16 with Jim Webber, Chief Scientist, CTO Field Ops at Neo4j and Antoine Larmanjat, Technical Director, Office of the CTO, Google Cloud. 

Click here to Register

Cloud Retail Search, part of Discovery Solutions For Retail portfolio, helps retailers significantly improve the shopping experience on their digital platform with ‘Google-quality’ search. Cloud Retail Search offers advanced search capabilities such as better understanding user intent and self-learning ranking models that help retailers unlock the full potential of their online experience.

Google Cloud’s Discovery Solutions For Retail are a set of services that can help retailers improve their digital engagement and are offered as part of our industry solutions.

Executive Summary

Retailers are always working on trying to keep up with the ever changing consumer expectations and trying to forecast the next trend that can impact sales and revenue.

The pandemic brought its own (and largely new) set of challenges which further complicated the issue over the last two years. The retailers were forced to adapt to the new consumer (low physical touch) behavior in which the browsing and product research was largely digital (endless aisle) and accelerated other trends such as buy online and pick up in stores (BOPIS), curbside pick up and pick up lockers. According to a McKinsey Global Survey from early last year, the pandemic has accelerated the pace of digital transformation by several years.

The National Retail Federation (NRF) estimates that retail sales are expected to grow between 6% and 8% in 2022 (slower growth rate than in 2021), as consumers spend more on services instead of goods, deal with inflation and higher food & gas prices due to geopolitical disruptions in the world.

And the competition continues to be fierce as ever. Amazon continues its dominance in the U.S. retail world and new PYMNTS data shows that Amazon’s share of US Ecommerce sales hit an all-time high of 56.7% in 2021.

Customers now have more choices than ever on how they want to engage with the retailers, where they want to spend the money and make their purchase. They also have increased expectations from the retailers around providing a high quality product discovery experience, which is forcing the retailers to invest heavily on improving customer engagement on their digital platforms to boost conversion rate and overall customer loyalty.

This is where Retail Search can help by providing an enhanced search experience that uses Google-quality search models to understand the customer intent and takes into account the retailer’s first party data (such as promotions, available inventory and price) for ranking results.

How is Google Cloud Retail Search Different

The Ecommerce platform on-site search use case is not new and retailers have been trying to solve it effectively for the last two decades. Most retailers recognize that search is a critical service on the platform and have spent countless resources to improve and fine tune it over the years. Yet the challenge remains. According to a Baymard Institute study in late as 2019, 61% of sites still required their users to search by the exact product type jargon the site uses.

However, users now expect the same robust and intuitive search features as is offered by Google.com and other popular web platforms, who seem to have the uncanny ability to intelligently interpret and yield relevant results to complex search queries.

Google’s decades of experience and research in search technology benefits Cloud Retail Search solution and that is what differentiates it from the competition.

• Advanced Query Understanding: Retail Search can provide more relevant results for the same query due to better query understanding features and knowing when to broaden or narrow the query results. While most search engines still rely largely on keyword based or matching tokens results, Retail Search has the advantage of being able to leverage Google search algorithms to return highly relevant results for product listings and category pages.
• Semantic Search: Intent recognition is a key requirement for semantic search and identifying what the customers mean when they enter the query is a key strength of Retail Search. This is critical for retailers since this has a direct impact on Clickthrough rate, Conversion rate and the Bounce Rate.
• Personalized Search Results: Another key differentiator for Retail Search is its ability to leverage user interaction data and ranking models to provide hyper personalized search results. Retailers are able to optimize search performance to deliver desired outcomes: better engagement, revenue, or conversions.
• Self-Learning and Self-Managed Solution: Retail Search models get better over time because of the self-learning capabilities built into the solution. In addition, the service is fully managed, which saves precious resources needed to keep it running and managing its set up.
• Strong Security Controls: The service runs on Google Cloud and follows security best practices to keep our customers’ data secure. Google never shares model weights or customer data across customers using the Retail API or other Discovery Solution products. For more details about this data use, see a description of Retail API data use.

High Level Conceptual View

Here is a simplified high-level view of Retail Search API. Retailers can call the API for the given search query and get back the results which can then be displayed on their digital properties.

The returned results contains two types of information:

• Search results: Query search results including product listings and category pages based on advanced query understanding and semantic search.
• Dynamic faceted search attributes: Faceted Search is a feature that allows further refinement of the search by providing ways to apply additional filters while returning results.

Retail Search needs the following datasets as input to train its machine learning models for search:

• Product Catalog: Information about the available products including product categories, product description, in-stock availability, and pricing.
• User Events: This is the clickstream data that contains user interaction information such as clicks and purchases.
• Inventory / Pricing Updates: Incremental updates to in-stock availability and pricing as that information is updated.

(Keeping the product catalog up to date and recording user events successfully is crucial for getting high-quality results. Set up Cloud Monitoring alerts to take prompt action in case any issues arise).

Retailers also have the ability to set up business/config rules to customize their search results and optimize for business revenue goals such as Clickthrough rate, Conversion rate, Average size order etc.

How to get started

Retail Search is generally available now and anyone with a Google cloud account can access it. If you don’t already have an account, you can start with a trial account for free here.

Establish a Success Criteria: It’s important to establish a success criteria for measuring the effectiveness of Retail search. Get a consensus on which factor(s) you want to include in scope for measuring the effectiveness of Retail Search. This could include one or two from the following: Search Conversion Rate, Search Average Order Value, Search Revenue Per Visit and Null Search Rate (No Results Found).

• Initial Set Up: Create a Google Cloud Project and set up the Retail API. When you set up the Retail API for a new project, the Google Cloud Console displays the following three panels to help you configure your Retail API project:
  • Catalog: displays product catalog and a link to import catalog.
  • Event: displays user events and a link to import historical user events.
  • Serving configs: contains details on serving configuration and a link to create a new serving configuration.

• Measuring Performance: Retail dashboards provide metrics to help you determine how incorporating the Retail API is affecting the results. You can view summary metrics for your project on the Analytics tab of the Monitoring & Analytics page in Cloud Console.

• Set up A/B Experiments: To measure the performance of Retail Search with another search solution, you can set up A/B tests using a third-party experiment platform such as Google Optimize.

Summary:

As retailers try to navigate through the post-pandemic world where supply chain failures and digital transformation acceleration are major focus areas, they now also have to keep a close eye on the recent geopolitical challenges resulting in rising inflation and costs.

While we can all agree that in-store shopping will continue to be a major source of revenue, it is also important for retailers to tweak the in-store experience for the digital world. Trends such as buy online and pick up in stores (BOPIS), curbside pick up and pick up lockers are here to stay.

Given all the above, consumer engagement and digital experience is more important now than ever before. The cost of search abandonment is way too high and has both short and longer term impact. Retail Search is a great solution to help reduce churn, improve conversion and retention. It provides Google-quality search models to help understand customer intent and the retailers have the ability to set up business/config rules to optimize search results for business revenue goals such as Clickthrough rate, Conversion rate and Average size order.