Enhancing Romi’s Conversations: The Role of BigQuery and LLMs

898
Of your peers have already read this article.
3:30 Minutes
The most insightful time you'll spend today!
MIXI, Inc. (MIXI) is a social networking organization that provides a diverse range of services for friends and family to enjoy together, such as the social-media platform mixi, a mobile game called Monster Strike, and a family photo and video sharing service known as FamilyAlbum. One of our current projects is Romi, a social robot launched in April 2021 that uses Speech-to-Text by Google Cloud as its speech recognition engine.
Since the late 2010s, the social robot market has been booming, with some models becoming increasingly affordable for consumers, from robotic tutors that promote social and cognitive development for children, to companion robots for elderly care. But with Romi, there is a marked difference in the quality of dialogue that makes Romi distinct from most social robots.
The biggest feature of Romi is that the AI developed internally by MIXI can generate natural exchange of communication. The size of a hand-held device, Romi can be placed anywhere in a room and has a screen to demonstrate different facial expressions. It responds to conversation within context. Until now, AI has been used to interpret the intentions behind user speech, but Romi is an AI-powered robot that takes it a step further, generating spoken conversations. After all, Romi was created to offer heartwarming communication to those who are looking for it. This form of speech recognition did not exist before Romi was released. We hope users will enjoy conversing with it, including the occasional unexpected response.

The speech recognition part was one of the most critical aspects of Romi. Most of the infrastructure that makes up Romi uses a main public cloud, which was used for other services then. As for speech recognition, we decided to try out the Speech-to-Text tool by Google Cloud, which was praised for its overwhelmingly high accuracy, and the prototype’s results were very positive. Even though we tried other companies’ services before making the final decision, our conclusion about Speech-to-Text remains the same.
The accuracy and responsiveness of Speech-to-Text made the tool an effective one for a social robot like Romi. Google Cloud also provided a sense of security with its high reliability that has been demonstrated in enabling Romi’s workloads, and will be able to support continuous development of Romi’s services for the long run.
With the rapid development of speech recognition technology, MIXI decided to re-examine the speech recognition engine for Romi in June 2022, about a year after its release. We eventually decided to continue its use of Speech-to-Text. We reviewed about 10 companies’ Japanese-compatible speech recognition engines, and found that Speech-to-Text offered the best results. In addition, Speech-to-Text has several speech recognition transcription models, but we found that the latest short model, which specializes in short utterances, is more suitable for Romi than the default model.
The cost-savings that Speech-to-Text delivers is also impressive. The billing unit was changed from 15 seconds increments rounded up, to one second in November, and huge cost reductions could be expected with Romi. This is important to us because Romi does not have trigger phrases, such as “OK Google,” so as to achieve more natural conversations. As a result, it can recognize and process more speech as compared to other social robots. While this results in a more user-friendly experience, it also requires greater workloads and can incur a higher cost compared to most speech recognition engines. But with the updated billing system that Speech-to-Text delivers, we are able to continue refining Romi’s speech recognition accuracy while keeping costs low.
Improving data analysis with BigQuery
Google Cloud was only used for speech recognition initially, but as Romi’s range of service expanded, more aspects of Romi were hosted on Google Cloud. Among these features, the machine learning platform for AI was moved to Google Cloud at an early stage. To be able to make use of a cloud platform at an affordable cost makes Google Cloud very appealing. Premium Support and technical account management helped us with our cost considerations.
Furthermore, MIXI started migrating the data analysis platform for Romi to BigQuery last year. BigQuery was chosen because it excels at bringing together and analyzing big data in various formats, as in-depth data analysis becomes necessary to improve Romi’s services. What also makes BigQuery an attractive choice was the ability to introduce structured query language (SQL) to BigQuery, a language that the development team from MIXI is familiar with.
In particular, we are grateful for the use of software like Looker. It takes a lot of work, even for engineers, to write complex queries, but with Looker, even non-engineers can intuitively perform fairly complex analysis. About half a year ago, we held regular briefings mainly for employees interested in data analysis, and now they voluntarily conduct analysis, conduct discussions based on the results, and create new projects and ideas. This has become a regular workflow for us.
Currently, what is popular in AI-based communication is the emergence of large-scale language models (LLMs) that learn from huge amounts of data, and generate natural responses on a different level than before.
To improve the conversational experience with Romi, we have been looking into relevant LLM technologies for a while now. It is important to be able to use high performance GPUs as inexpensively as possible in order to run PoC at high speed. We will continue to focus on Google Cloud services, including Compute Engine and VertexAI.
Arab Bank Accelerates its App Development and Testing Using Apigee and Anthos

6157
Of your peers have already read this article.
3:30 Minutes
The most insightful time you'll spend today!
Founded in 1930 and headquartered in Jordan, Arab Bank is one of the oldest banks in the Middle East. Operating out of 28 countries, we’ve earned our customers’ trust with a prudent approach to operations and respect for the cultures and customs in the region.
With a few exceptions where cloud providers have hosted their datacenter in a Middle Eastern or North African country, the banking sector, in general, in the region has been slow to adopt cloud technology for a number of reasons, including concern about data security, maturity and security controls of cloud services (PaaS and SaaS), and regulations in place. But on the other hand, we saw the opportunity to accelerate our development and testing using the cloud, as well as to partner with the fintech community and digital service providers to integrate their solution in the banking ecosystem. We needed more flexibility to connect with the outside world, and a more open architecture to help us drive our internal innovation at a faster rate with the help of the fintech industry. By collaborating with Google Cloud, we reached those goals and accelerated app development and testing through products like Apigee and Anthos. We’re now offering innovative apps and services to our customers and employees that leverage new technological capabilities to give more agility and flexibility, and to optimize our workloads.
Embracing the cloud in a regulated industry
To get started with the cloud, we needed to create internal awareness about cloud technology, the API layer, containers and their benefits amongst our leaders and staff. Google helped us educate and get buy-in from key functions by organizing open technology demonstration sessions and discussion panels. When considering potential cloud providers we had four decision criteria: maturity of security controls, ease of use, cost, and scalability / agility for new deployments and continuous innovation. This last factor was critical, and we were impressed by Google Cloud’s innovation roadmap, both via direct conversations and at Google’s Next conference, where we met a lot of people passionate about technology, innovation and building something new.
Going back to our journey, given the above-mentioned regional limitations, we started to develop a hybrid cloud approach. This helped us continue to operate on-premises for a number of services in production, particularly those that have personally identifiable information (PII) or other sensitive data attached, and to leverage the cloud for development, testing and production workloads that don’t contain customer data.
In the short term, we didn’t anticipate that our many jurisdictions would allow data to be transported to other countries. But cloud tools will allow us to tokenize or anonymize customer data while maintaining customer data on-premises. This applies to many digital journeys such as customer onboarding, credit facility online applications, or marketplace navigation. In the coming years, we predict our API integration with partners will accelerate and enrich the overall digital value proposition of our business segments, namely consumer banking, small and medium-size businesses and large corporate and institutional clients.
Building connections and cornerstones in the cloud
The first move in our digital transformation was to implement Apigee, Google Cloud’s API management platform, to connect to the world’s digital banking ecosystem. Apigee provides the security, sharing, mediation policies, and developer portal capabilities for us to successfully meet Open Banking standards while focusing on innovation.
On the back of the Apigee implementation, we created an accelerator program to incubate Fintech ideas that can, in turn, integrate into our digital platforms and be offered to our customers. We also developed various banking APIs, all designed and documented in accordance with PSD2 and Open Banking regulations, and made them available to our partners. These APIs exposed on our API development portal offer the needed code structure for fintech companies to design creative solutions around them.
Next, we adopted Anthos, Google Cloud’s managed application platform. Anthos has become a cornerstone of our operations because it works across hybrid cloud, offering integration of microservice containers and fueling collaborative opportunities with external parties. Our current Anthos infrastructure includes several hundreds of microservices now running on containers in Google Kubernetes Engine (GKE) and on-premises. We now use the cloud for collaboration, development and testing, but not for production, which is done on-premises.
Along the way, Google Cloud’s Professional Services Organization (PSO) helped us through the entire cloud setup process, and with the adoption of Anthos. We originally built on the cloud tools through an iterative process, learning from our successes and errors along the way. Now that we have a better sense of how Anthos operates, we’re building a fresh infrastructure atop a sound, stable, and resilient foundation that will let us scale easily as we work to transform Arab Bank into a digital-first enterprise, that is our ambition.
Currently products running on Anthos include customer acquisition and onboarding via mobile apps, and our Arabi-Pay app, which allows customers to instantly pay each other via WhatsApp or other messaging platforms. Leveraging Anthos, our instant loan service for Arab Bank salaried employees can grant and disburse loans up to $7,000 in less than seven minutes.
In addition, we’ve built a number of digital journeys for our Small and Medium Enterprise (SME) customers, such as our SME client digital onboarding process and paperless SME lending platform.
While some may think that digital adoption in this part of the world can be slow, as customer contact remains anchored in our customs, the recent COVID-19 pandemic has accelerated the adoption of digital banking services and electronic payments, inspiring more confidence to buy and pay online. Thanks to our rich and user-friendly banking app that relies on Apigee and Anthos for critical customer journeys, over 90% of new-to-bank customers are using our mobile apps. Within the next 18 months, we predict that number will be closer to 100%.
Of course, with higher customer adoption comes the challenge of potential service interruptions. A single moment of downtime can be highly visible to many digital customers. But Google Cloud’s Anthos and Apigee give us the flexibility to resume processes at a fast rate, so any interruptions are almost invisible to our customers. In fact, when the COVID-19 pandemic hit, though our branches could be open only for limited hours each day, our consumer clients in particular were able to take advantage of our digital services in a very self-sufficient manner. Being well positioned with Google Cloud, we could also keep our internal teams and external partners connected and productive. Without Google Cloud, continuing the digital transformation of the bank at the pace we wanted would have been a big challenge.
Collaborating across borders and time zones
With Google Cloud, our ability to collaborate and partner has transformed significantly. We operate 24/7 now because our developers are scattered across multiple geographies and different time zones. Because testing and deployment can run around the clock, including on weekends, we currently deploy a new digital journey in a few weeks, faster than ever before. This has given our organization a spirited mindset that prioritizes innovation, and raised the bar in terms of our operating model.
Another consideration about Google Cloud tools is the elimination of inefficient processes typically seen in a software development lifecycle. We now build in a completely agile manner, from design squads until deployment in production. Compared to where we were two years ago, when we had an annual maximum of two systems releases in production, we now have close to monthly releases of our digital packages. In addition, we also supplement those monthly releases with additional ad-hoc releases and fixes in between. As a result, we have removed internal silos and improved tremendously the collaboration between the product and sales teams, operations, IT Dev Factory and Infrastructure, as well as our supporting functions.
Through the agile process facilitated by APIs, we introduced Design Thinking workshops involving external customers and prospects early on to understand their true pain points better and emotions during the existing journeys and how to make new digital journeys frictionless. As a result, the relevance of our products for various customer personas has improved tremendously.
Transcending banking
With Google Cloud, we can offer our customers so much more than just banking. We’ve become more digitally relevant to their lives. For example, we recently launched a mortgage app that helps customers all the way from home selection through mortgage negotiations and closing, and even to getting the home decorated. It’s an end-to-end journey in which API integration with key regional players was a cornerstone to our success.
For other digital products, we have an extensive roadmap of lifestyle-based solutions relevant to each segment and age group. We’ve only scratched the surface of the services we can provide, and we see the cloud as the future for everything we want to do.
Read more about Google Cloud’s Open Banking solution to learn how you can simplify and accelerate the process of delivering open banking as required by PSD2. You can also view our video on Open Banking, powered by Apigee API Management.

4116
Of your peers have already downloaded this article
10:32 Minutes
The most insightful time you'll spend today!
While microservices are lauded as catalysts for speed and scale, the conversation is incomplete if it does not include APIs. Without APIs, microservices cannot be securely, reliably, and adaptably scaled across an organization or to outside partners. Moreover, because these APIs and their microservices are meant to be widely consumed and reused, it’s not enough for companies to merely create the APIs and move on. Rather, the APIs should be continually managed so the business can control how its microservices are accessed, generate insight into how they are used, and encourage reliability of its services.
As microservices grow beyond their original use cases and are being leveraged to share important functionality throughout the enterprises and even with external partners, the problem that arises is how this sharing creates challenges for teams trying to secure, monitor, understand usage of, and fully take advantage of microservices.
In this eBook we explore how leading enterprises are facing these challenges head-on and scaling their microservices strategies. Learn how APIs make it possible for microservices to be securely, reliably, and adaptably shared, and how an API management platform enables enterprises to ensure that they maintain control over their microservices as consumption grows.
3357
Of your peers have already watched this video.
24:40 Minutes
The most insightful time you'll spend today!
A Guide to Anthos: the App Modernization Platform for Hybrid and Multi-cloud Deployments
Anthos gives you a consistent platform for all your application deployments, both legacy as well as cloud native, while offering a service-centric view of all your environments.
You can build enterprise-grade containerized applications faster with managed Kubernetes on cloud and on-premises environments. Create a fast, scalable software delivery pipeline with cloud-native tooling and guidance.
It also helps leverage a programmatic, outcome-focused approach to managing policies for apps across environments, and enable greater awareness and control with a unified view of your services’ health and performance.
Tune in to this brief explainer video for an overview of Anthos, deployment examples, and explore options to get started illustrated through a demo with Bradley Wong, Product Manager at Google Cloud. Learn to make the best first step in your journey of application modernization and gain a clear understanding of the value of Anthos for your hybrid and multi-cloud workloads.
Swiggy: Delivering Local Food Within 40 Minutes

8805
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
Founded in 2014, Swiggy started small, delivering food to a few neighborhoods in Bengaluru, India. As the company grew, the team wanted a mapping technology that could help expand the service throughout India.
Swiggy needed a scalable mapping platform that covered a wide geographic area and offered tools to help the company to build an efficient mobile app and website for customers and delivery staff.
Customers find restaurants and order from them using the Android app, iOS app, or the website. Swiggy worked with Google Maps Partner Media Agility and used a variety of Google Maps Platform APIs to develop web and mobile apps that incorporate relevant local restaurant details.
Google Maps Platform Results
- Built a hyper-local delivery service that is growing throughout India at a rate of 25 percent per month
- Deliveries are made quickly, resulting in higher customer satisfaction and retention—users have been so satisfied that nearly 80 percent of its orders are from repeat customers
- Drivers seamlessly handle tens of thousands of orders per day
In order to guarantee fast food delivery, Swiggy returns only restaurants within four to five kilometers of the customer’s location. The Directions API is used by drivers to easily route to restaurants and customers. The customer can track the progress of the delivery and estimated arrival time using a mobile app or the website.
“Google Maps provides the most accurate and reliable data, which is crucial for us because maps and location are central to our business. We also knew Google’s intuitive interface would provide a great customer experience with little to no learning curve… Google Maps’ ability to provide customer location and the distances of nearby restaurants is the backbone of our success, because it ensures a reliable, consistent customer experience,” said Aman Jain, Senior Product Manager, Swiggy.
Neo4J & Google Cloud: Graph Data in Cloud to Address Challenges in FinServ Industry

6892
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
Over the last decade, financial service organizations have been adopting a cloud-first mindset. According to InformationWeek, lower costs and enhanced scalability were the biggest drivers for cloud adoption in financial services, and cloud-native applications allow access to the latest technology and talent, enabling adopters to rebuild transaction processing systems capable of supporting very high volumes and low latency.
Both Neo4j and Google Cloud have been using relationship-based data representations since the beginning, and we’re dedicated to using this technology to help financial services customers drive business transformation. We are excited about the prospects of financial services (FinServ) cloud systems and believe that graph data in the cloud can help solve significant challenges in the industry.
Data Challenge #1: Risk Management and Compliance
First among the top concerns for any CIO moving to the cloud is risk management and compliance. Disconnected, uncontextualized, or stale data create opportunities for fraud and financial crimes to occur. The fact is when it comes to FinServ, the question is not “if” but rather how often an attack will occur. Unfortunately, incidents have been trending upward over the last decade, and COVID has only exacerbated this reality. Financial crimes affect the bottom line both in the remediation of these crimes and in intangibles like brand value.
Add to this the complexity of international banking, which makes “compliance” a moving target. Penalties due to noncompliance are a constant concern to any FinServ organization.
The tabular representation of information with a fixed number of columns that never change prevents a description of an ever changing world with changing characteristics. Relational databases are great if the world you describe does not move fast but have limitations when data structures are highly interlinked and not homogeneous.
Neo4j Aura on Google Cloud provides a foundation for creating dynamic, futureproof, scalable applications that adhere to the security standards and protocols today’s financial services organizations require to meet the challenges of finding and preventing bad actors. This also includes enterprise scalability; reaching over 1 Billion nodes and relationships to streamline queries and provide solutions that meet regulatory and privacy compliance across geographies. Neo4j has helped some organizations save billions of USD in fraud in the first year of deployment alone.
What makes graph technology the best choice for fraud detection use cases is that the relationships between the data-points are as important as the data-points themselves. Let’s take as an example, one John Smith approaches a multi-national banking institution to manage the primary account for his new holding corporation.
While no one has any record of John R Smith Holdings LLC, the bank’s application built on graph technology understands that there are several well-known entities owned by John Smith Holdings. The application also identifies several well-known board members who bank with this institution. Due to this relationship-driven approach, the bank now understands John R Smith is not “John Smith,” who previously attempted to open an account for his holding corporation, which had no information associated with it prior to two months ago.
Data Challenge #2 Manual Processes and Inefficiencies
The ubiquity of the cloud offers an opportunity to deploy automation at unprecedented levels to tackle the errors and inefficiencies that manual processing allows to creep into processes. When data comes from disparate, perhaps legacy systems – which may have become siloed and “untouchable” over the years – further complexity arises. As an example, if someone in sales types “John Smith” into a CRM system not knowing that John R Smith is the spelling in the customer data master, it may result in two separate and potentially conflicting records. Being able to join those records together in a mastered view helps to solve this problem. In addition, low data quality equates to an increase in risk, costs, and implementation times for new systems.
Neo4j Aura on Google Cloud provides automation and artificial intelligence (AI) that reduces manual processes and the errors that accompany them. In this graph architecture each node, which can represent a person, will have labels, relationships, and properties associated with it. This allows for the use of AI which can easily understand that John Smith in the CRM is the same John R Smith in the customer master. The information contained in Neo4j can be connected bi-directionally to ensure consistency across applications and data sources.
One of the benefits of this approach is that linking information allows organizations to keep the full value of the data, rather than forcing the data into predetermined tabular representations, with the risk of losing valuable information and insights.
Data Challenge #3: Customer Engagement and Insight
Another significant concern is the high expectations today’s customers have for every interaction. End users are accustomed to predictable experiences on their digital devices, and FinServ apps are no exception. Added to this, the “Covid economy” has driven digital adoption significantly across demographics; even among customers who might traditionally have used in-person services. This also equates to increased expectations for personalized, predictable experiences with every digital interaction. We know that latency has always been a key consideration for financial trading, but a recent ComputerWeekly study showed that every financial organization should ensure their visible latency is at 10 milliseconds or less. Customers no longer accept their broadband is at fault.
Finally, blind spots in the customer journey often result in dissatisfaction, which ultimately leads to increased churn. Without gaining actionable insights from your customers, there is no room to innovate and iterate on what they are looking for in your products and services. And this translates to losing market share and competitive advantage.
The NoSQL architecture, specifically the dynamic schema and structure of Neo4j Aura gives you the ability to take charge of your data and make changes according to your development cycles or newer data models. This equates to faster builds, more comprehensive releases and a wider, richer data-set that can be contextualized and understood instantly. Graph technology is the logical choice for building a Customer 360 application. Under this approach organizations not only get valuable insight into the individual client’s behavior and patterns, but also those of their family, friends and colleagues. This allows for stronger personalization, targeted campaigns and successful execution, resulting in increased customer satisfaction and retention levels.
Graph Technology on Google Cloud

Neo4j is a recognized leader in graph database technology and the only fully integrated graph solution on Google Cloud, helping to fill a common need for Google Cloud customers. Both Neo4j and Google Cloud are invested in continuing to grow our partnership and mutual product direction.
You can find and deploy the Neo4j graph database straight from the Google Cloud marketplace, whether you want to download the software for an on-premises deployment, use the virtual machine image, or use the hosted solution, Aura on Google Cloud, the graph database-as-a-service. In any deployment, you get the same enterprise-grade scalability, reliability, and connectivity along with successful, repeatable use cases you can rely on to resolve your particular challenges and integrated billing.
For a real-world example of how graph technology can optimize financial services, you can read our Case Study with fintech Current. Current, a leading U.S. financial technology platform with over three million members, used Neo4j Aura on Google Cloud to create a personalization engine based on client relationships.
To learn more about Neo4j Aura on Google Cloud for FinServ organizations, register for our webinar on Thursday, December 16 with Jim Webber, Chief Scientist, CTO Field Ops at Neo4j and Antoine Larmanjat, Technical Director, Office of the CTO, Google Cloud.
More Relevant Stories for Your Company

3 Ways To Reduce App Downtime With Google Cloud’s API Monitoring Tools
How many times have you closed an application when you encounter the “spinning wheel of death” (a melodramatic way of saying an application that is taking too long to load)? In today’s digital economy where many organizations rely on applications as a primary source of revenue, that spinning wheel of

Learn Modern App Development Practices to Ship Software Faster
Cloud-native, Kubernetes, Serverless have been the hottest and most widely discussed topics given the velocity and agility benefits. Learn more about how you can leverage these modern app development practices to ship software faster, while reducing costs and improving security and compliance. Learn how Google Cloud lets you modernize existing
State of DevOps 2019
The 2019 Accelerate State of DevOps Report confirms analyst reports that DevOps is crossing the chasm with the proportion of elite performers almost tripling compared to last year. The report finds that delivering software quickly, reliably, and safely is at the heart of technology transformation and organizational performance. The report

Building Globally Scalable Services with Istio and ASM
Building distributed applications is hard! Building globally scalable distributed applications is harder. Maintaining and growing these services as your business grows is even harder. Learn how to create a globally scalable platform for your business on Google Cloud using service meshes. See how to build a platform on Google Cloud







