Enhancing Romi’s Conversations: The Role of BigQuery and LLMs
903
Of your peers have already read this article.
3:30 Minutes
The most insightful time you'll spend today!
MIXI, Inc. (MIXI) is a social networking organization that provides a diverse range of services for friends and family to enjoy together, such as the social-media platform mixi, a mobile game called Monster Strike, and a family photo and video sharing service known as FamilyAlbum. One of our current projects is Romi, a social robot launched in April 2021 that uses Speech-to-Text by Google Cloud as its speech recognition engine.
Since the late 2010s, the social robot market has been booming, with some models becoming increasingly affordable for consumers, from robotic tutors that promote social and cognitive development for children, to companion robots for elderly care. But with Romi, there is a marked difference in the quality of dialogue that makes Romi distinct from most social robots.
The biggest feature of Romi is that the AI developed internally by MIXI can generate natural exchange of communication. The size of a hand-held device, Romi can be placed anywhere in a room and has a screen to demonstrate different facial expressions. It responds to conversation within context. Until now, AI has been used to interpret the intentions behind user speech, but Romi is an AI-powered robot that takes it a step further, generating spoken conversations. After all, Romi was created to offer heartwarming communication to those who are looking for it. This form of speech recognition did not exist before Romi was released. We hope users will enjoy conversing with it, including the occasional unexpected response.
The speech recognition part was one of the most critical aspects of Romi. Most of the infrastructure that makes up Romi uses a main public cloud, which was used for other services then. As for speech recognition, we decided to try out the Speech-to-Text tool by Google Cloud, which was praised for its overwhelmingly high accuracy, and the prototype’s results were very positive. Even though we tried other companies’ services before making the final decision, our conclusion about Speech-to-Text remains the same.
The accuracy and responsiveness of Speech-to-Text made the tool an effective one for a social robot like Romi. Google Cloud also provided a sense of security with its high reliability that has been demonstrated in enabling Romi’s workloads, and will be able to support continuous development of Romi’s services for the long run.
With the rapid development of speech recognition technology, MIXI decided to re-examine the speech recognition engine for Romi in June 2022, about a year after its release. We eventually decided to continue its use of Speech-to-Text. We reviewed about 10 companies’ Japanese-compatible speech recognition engines, and found that Speech-to-Text offered the best results. In addition, Speech-to-Text has several speech recognition transcription models, but we found that the latest short model, which specializes in short utterances, is more suitable for Romi than the default model.
The cost-savings that Speech-to-Text delivers is also impressive. The billing unit was changed from 15 seconds increments rounded up, to one second in November, and huge cost reductions could be expected with Romi. This is important to us because Romi does not have trigger phrases, such as “OK Google,” so as to achieve more natural conversations. As a result, it can recognize and process more speech as compared to other social robots. While this results in a more user-friendly experience, it also requires greater workloads and can incur a higher cost compared to most speech recognition engines. But with the updated billing system that Speech-to-Text delivers, we are able to continue refining Romi’s speech recognition accuracy while keeping costs low.
Improving data analysis with BigQuery
Google Cloud was only used for speech recognition initially, but as Romi’s range of service expanded, more aspects of Romi were hosted on Google Cloud. Among these features, the machine learning platform for AI was moved to Google Cloud at an early stage. To be able to make use of a cloud platform at an affordable cost makes Google Cloud very appealing. Premium Support and technical account management helped us with our cost considerations.
Furthermore, MIXI started migrating the data analysis platform for Romi to BigQuery last year. BigQuery was chosen because it excels at bringing together and analyzing big data in various formats, as in-depth data analysis becomes necessary to improve Romi’s services. What also makes BigQuery an attractive choice was the ability to introduce structured query language (SQL) to BigQuery, a language that the development team from MIXI is familiar with.
In particular, we are grateful for the use of software like Looker. It takes a lot of work, even for engineers, to write complex queries, but with Looker, even non-engineers can intuitively perform fairly complex analysis. About half a year ago, we held regular briefings mainly for employees interested in data analysis, and now they voluntarily conduct analysis, conduct discussions based on the results, and create new projects and ideas. This has become a regular workflow for us.
Currently, what is popular in AI-based communication is the emergence of large-scale language models (LLMs) that learn from huge amounts of data, and generate natural responses on a different level than before.
To improve the conversational experience with Romi, we have been looking into relevant LLM technologies for a while now. It is important to be able to use high performance GPUs as inexpensively as possible in order to run PoC at high speed. We will continue to focus on Google Cloud services, including Compute Engine and VertexAI.
3944
Of your peers have already downloaded this article
2:20 Minutes
The most insightful time you'll spend today!
Web APIs use HTTP, by definition. In the early days of web APIs, people spent a lot of time and effort figuring out how to implement the features of previous-generation distributed technologies like CORBA and DCOM on top of HTTP. This led to technologies like SOAP and WSDL. Experience showed that these technologies were more complex, heavyweight, and brittle than was useful for most web APIs. The idea that replaced SOAP and WSDL was that you could use HTTP more directly with much less technology layered on top.
Most modern web APIs are much simpler than SOAP or WSDL APIs, but preserve some of the basic ideas of remote procedure call—which is not native to HTTP—implemented much more lightly on top of HTTP. These APIs have come to be known as RESTful APIs. Apigee allows developers to only use HTTP without additional concepts.
Here’s Why: When you design any interface, you should try to put yourself in the shoes of the user. As an API provider, you may work on a single API or a small group of APIs, but it is likely that your users deal with many more APIs than yours. This means that they probably come to your API with significant knowledge of basic HTTP technologies and standards, as well as other APIs. Because of this, there is a lot of value in adhering to standards and established conventions, rather than inventing your own.
Download the E-book to get deeper insights into how you can make your developers’ life easier.
4017
Of your peers have already downloaded this article
4:20 Minutes
The most insightful time you'll spend today!
Modern web APIs allow businesses to build compelling experiences that generate, consume, and combine digital assets. They’ve become the foundation upon which digital business is built.
But the path to success can be unclear, rough, and challenging. The game has changed, after all, and this requires tossing out many of the rules by which you run your business.
You’re not alone. Other businesses have struggled down the path toward increased competitiveness, agility, and opportunity—and we, at Google Cloud, are privileged to partner with hundreds of customers across myriad industries as they gear up to go digital.
We’ve seen the successful decisions and strategies that have enabled companies to adapt to new digital realities. But we’ve also seen businesses make decisions that slowed, stymied, or, in some cases, reversed progress toward becoming an agile digital business.
There’s no one path to travel. There are many ways to make your business digital, but there are also many pitfalls. This is the survival guide to help navigate a smarter digital journey.
There’s one bit of advice we can offer above all else: GET STARTED. If you don’t take steps to adapt to the new competitive landscape and to your customers’ expectations, the very survival of your business is at risk.
Download the E-book to know more about the best and worst practices of how a business leads, measures, invests in, and executes an API initiative.
Swiggy: Delivering Local Food Within 40 Minutes
8817
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
Founded in 2014, Swiggy started small, delivering food to a few neighborhoods in Bengaluru, India. As the company grew, the team wanted a mapping technology that could help expand the service throughout India.
Swiggy needed a scalable mapping platform that covered a wide geographic area and offered tools to help the company to build an efficient mobile app and website for customers and delivery staff.
Customers find restaurants and order from them using the Android app, iOS app, or the website. Swiggy worked with Google Maps Partner Media Agility and used a variety of Google Maps Platform APIs to develop web and mobile apps that incorporate relevant local restaurant details.
Google Maps Platform Results
- Built a hyper-local delivery service that is growing throughout India at a rate of 25 percent per month
- Deliveries are made quickly, resulting in higher customer satisfaction and retention—users have been so satisfied that nearly 80 percent of its orders are from repeat customers
- Drivers seamlessly handle tens of thousands of orders per day
In order to guarantee fast food delivery, Swiggy returns only restaurants within four to five kilometers of the customer’s location. The Directions API is used by drivers to easily route to restaurants and customers. The customer can track the progress of the delivery and estimated arrival time using a mobile app or the website.
“Google Maps provides the most accurate and reliable data, which is crucial for us because maps and location are central to our business. We also knew Google’s intuitive interface would provide a great customer experience with little to no learning curve… Google Maps’ ability to provide customer location and the distances of nearby restaurants is the backbone of our success, because it ensures a reliable, consistent customer experience,” said Aman Jain, Senior Product Manager, Swiggy.
Kaluza & Google Cloud: Committed to Powering Up 73 Million EVs by 2040
1228
Of your peers have already read this article.
4:00 Minutes
The most insightful time you'll spend today!
Electric vehicles already account for one in seven car sales globally, and with new gas and diesel cars being phased out across the world, global sales are forecast to reach 73 million units in 2040. But with power grids becoming increasingly dependent on variable energy sources such as wind and solar, rising demand from electric vehicles risks overstraining grids at peak times, potentially leading to power outages.
At Kaluza, we believe that our platform has a vital role to play in helping power grids and utility companies to stabilize their networks, while at the same time delivering more affordable, cleaner energy to the consumer. Powered by Google Cloud, the advanced algorithms behind our Kaluza Flex solution automatically charge electric vehicles when the power supply is at its cheapest and greenest, helping to accelerate the global transition towards a zero-carbon future.
Decarbonizing the grid with low-cost smart charging programs
Launched by OVO Energy in 2019, Kaluza has taken its deep understanding of the energy market to partner with some of the world’s major energy suppliers and vehicle manufacturers, including AGL in Australia, Fiat and Nissan in the UK, and Mitsubishi Corporation and Chubu in Japan, to launch smart charging programs that help customers save money while reducing their carbon footprint.
A good example of this is Charge Anytime, which we recently launched with OVO Energy in the UK. With this tariff, customers use Kaluza to smart-charge their electric vehicle, and pay just 10p per kWh — a third of their household electricity rate — to do so. This means that if the customer plugs in their vehicle to charge when they get home from work at, say, 6:00 p.m. — a time when both demand and the carbon intensity on the grid are at their highest — their vehicle will then be smartly charged at the lowest cost and greenest periods throughout the night, ready for when they need it in the morning.
This smart charging reduces the energy company’s costs by enabling them to take advantage of lower wholesale electricity prices. These savings are then passed on to the end customer through tariffs such as Charge Anytime, saving customers hundreds of pounds a year and reducing their carbon footprint. Meanwhile, the National Grid is able to reduce the strain on the network during peak hours, while simultaneously using up the excess renewable energy that might otherwise have gone to waste.
Optimized charging schedules, fueled by Google Cloud
Behind Kaluza’s smart charging solution lies some sophisticated technology, all of which is built on Google Cloud. Our core optimization engine gathers real-time data from a wide range of sources, including battery and charging data from the electric vehicles, and data from the energy suppliers and grid operators, such as the carbon intensity, and price forecasts.
After passing through our real-time data backbone, that data is stored in BigQuery where it’s used to train and validate our smart charging optimization models. These models are then deployed with Google Kubernetes Engine so that whenever a customer plugs in an electric vehicle, data from that vehicle passes in real-time through our optimization engine to calculate the ideal charging schedule for that vehicle, ensuring it uses the cheapest, least carbon-intensive energy available.
Customer interface: easy to use, simple to build
Of course, the customer isn’t aware of any of this complexity. All they need to do is open their charging app and use Kaluza’s intuitive interface to set what time they want their car to be ready and how much charge they want in their battery. Then they simply plug in their car, and our algorithms take care of the rest.
Customers can also use Kaluza to view breakdowns of how much carbon and money they’ve saved, along with insights around things like billing and battery life, all of which is backed by Cloud SQL.
With Google Cloud, we were able to roll out this end-user app very quickly. Instead of having to build a different version of the app for each operating system, we were able to build an OS-agnostic version using Flutter, which then builds the app for each platform, enabling us to get to market faster.
This has been a benefit of our architecture in general. With Google Cloud taking the complexity out of otherwise time-consuming development processes, we’ve been able to experiment with and validate propositions rapidly, and roll out new products and features at speed, to ensure that we remain at the vanguard of a rapidly evolving sector.
Giving energy companies full visibility with BigQuery and Looker
As for the grid operators and energy companies, the Kaluza platform allows them to visualize how many participating electric vehicles are plugged into the network at any one time. BigQuery and Looker Studio dashboards provide granular insights, such as how many vehicles are idle, how many are charging, and how well our optimization engine is working.
The platform also allows companies to view those vehicles on an aggregate level, and identify any issues. Google Cloud machine learning capabilities can even allow grid operators to use this aggregate view to forecast how much energy will be required at any one time, and dial the power generation up or down accordingly.
Ultimately, these insights help network operators and utility companies to optimize energy usage, and balance out the peaks and troughs of supply and demand, ensuring that excess renewable energy is captured, while carbon-intensive fuel use is reduced.
Feeding energy back into the grid with bidirectional charging
Vehicle-to-Grid (V2G), or bidirectional, charging is something that we are very excited about, and have begun rolling out in the UK, as we prepare to launch in other global markets. Built on the same Google Cloud architecture as the rest of Kaluza Flex, V2G not only enables smart charging, but allows electric vehicles to feed stored energy back into the grid.
Imagine an electric vehicle battery that can store 40 kWh of energy, but only uses 5 kWh a day. That leaves 35 kWh a day that can be charged to the battery during times of low demand, when energy is cheapest and greenest, then fed back into the network during peak periods, removing the need for more fossil fuels to be burned to meet demand.
Not only does V2G go further than conventional smart charging to make use of renewable energy when it’s abundant and support the balancing of the energy system, it also results in even lower energy prices for the customer, as they are effectively selling energy back to the grid. For example, as part of a large domestic V2G trial made in partnership with OVO and Nissan, Kaluza saved customers an average of £450 a year, with some customers saving up to £800/year by selling surplus energy back to the grid — transforming their homes into mini power stations.
With Kaluza Flex, we have a platform that offers benefits for all parties, from the grid operators, through to energy retailers and vehicle manufacturers, and all the way to the customer. Now, our aim is to bring this exciting offering to as many markets as possible, a goal which is made easier thanks to the scalable infrastructure and time-saving solutions of Google Cloud.
As more people make the switch to electric vehicles, our goal is to ensure that smart charging becomes standard practice, as we help to deliver on the potential of electric vehicles to contribute to a greener, decarbonized future.
A CIO’s Guide to the Cloud: Hybrid and Human Solutions to Avoid Trade-offs
3863
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
What do CIOs and CTOs deliver for the company? If you said “technology,” that’s just the beginning. According to their research, McKinsey found that 85% of CIOs and CTOs interviewed in the spring of 2019 said they were essential for at least two of the three most common CEO priorities—revenue acceleration, improved agility and time to market, and cost reduction.
IT modernization – including migrating to the cloud – is key to business growth and agility. Yet, according to a recent McKinsey study, 80% of CIOs report that regardless of their level of cloud migration, they still haven’t reached their projected agility and business benefits. Sometimes, this is because of issues like training and skills gaps in the IT workforce. Surprisingly often though, the barrier to reaching the goals is based on trade-offs that CIOs themselves feel they must make to strike a balance between the perfect and the possible.
But what if you could have it all without the trade-offs? As Will Grannis, Managing Director of the CTO Office at Google, and Arul Elumalai, Partner at McKinsey & Company discussed in our recent digital conference, many of the compromises CIOs make can be avoided with new technology, modern architectures and by encouraging a transformation mindset across the business. In interviews, CIOs explained how they’ve leveraged the best of the cloud without compromising on security, agility, and flexibility. Here’s how these leaders avoid three of the top perceived trade-offs—both with technology and by transforming their operating model.
Trade-off #1: Developer agility vs. control and governance
Moving to the cloud offers new opportunities for speed, but 69% of organizations indicate that stringent security guidelines and code review processes can slow developers significantly. One CISO of a multinational company mentioned that cloud development was so fast that they had to institute manual checks on their developers’ code. So much for agility.
To overcome this trade-off and maintain both speed and security, some respondents found success in DevOps, hiring security-experienced talent and introducing automation for security and quality. Building in security into the CI/CD pipeline and increasing automation don’t just eliminate the tradeoff, they result in higher quality and faster innovation.
At Google Cloud, we’ve also observed that customers with strong DevOps practices have increased speed-to-market and product/service quality. From our own journey, we’ve learned seven critical lessons essential to adopting a DevOps model, ranging from taking up small projects and embracing open source to building an overall DevOps culture.
Trade-off #2: Single-vendor benefits vs. freedom from lock-in
CIOs perceive benefits to using the fewest number of clouds, specifically avoiding introducing multiple systems that require their teams to develop and maintain multiple skillsets. Unfortunately, 83% of the CIOs interviewed said that while they would prefer fewer clouds, the potential financial and technical lock-in drives them to multiple providers.
Successful CIOs said that they can avoid lock-in pitfalls not just with contractual guardrails and executive and board education, but with evolving hybrid cloud technologies that provide additional choices. Hybrid cloud platforms based on containers can further mitigate the risk of using a single cloud vendor. The key to successful hybrid architectures is the infrastructure abstraction and portability that containers create for them, enabling disparate environments to work together.
This notion has been at the heart of our strategy at Google Cloud with Anthos, which provides an abstraction layer and an application modernization platform for hybrid and multi-cloud environments. Enterprises can use Anthos to modernize how they develop, secure, and operate hybrid-cloud environments and enable consistency across cloud environments.
Trade-off #3: Best-of-breed tools vs. standardization and familiarity
Optimizing tool chains for different environments can improve productivity, but many CIOs believe that this means reduced functionality and tools. While 77% of CIOs said they had to standardize to the lowest common denominator, some have found a better solution. Rather than giving up the languages, libraries, and frameworks that their teams prefer, effective leaders said that they found success by investing in training programs to upscale talent and adopting new open and vendor-agnostic solutions. Architectures that are based on open-source components have been the keys that helped remove this tradeoff, and eliminate the notion of a lowest common denominator.
This is why we have built Anthos on open-source components like Kubernetes, Istio and Knative. Anthos gives your business the choice you need. With the ability to create code that works in most environments using the tools, languages, and systems you prefer, you can do more without major changes to how you work.
Regardless of your current cloud adoption level, check out “Unlock business acceleration in a hybrid cloud world” to discover more about McKinsey’s findings, including how CIOs drive agility, methods to make trade-offs unnecessary, and how to prepare your team for the cloud. Then, stay tuned for subsequent posts that take a closer look at how hybrid solutions and strategies can help CIOs drive a transformation mindset across the business—without compromising on security, agility, and flexibility.
More Relevant Stories for Your Company
Custom Voice Feature Can Help Brands Tweak IVR for Better Customer Experiences
With the rise of digital assistants and conversational interfaces, people have grown accustomed to hearing and speaking to synthetic voices. But what do those voices sound like? Often, pretty repetitive. We’re all familiar with the Google Assistant voice, for example. That’s why we are excited to announce the general availability
Your Guide to Kubernetes Best Practices
Kubernetes made a splash when it brought containerized app management to the world a few years back. Now, many of us are using it in production to deploy and manage apps at scale. Along the way, we’ve gathered tips and best practices on using Kubernetes and Google Kubernetes Engine (GKE) to your
Google Cloud Helps LiveRamp Capture, Manage, Process and Visualize Data at Scale
Editor’s note: Today we’re hearing from Sagar Batchu, Director of Engineering at LiveRamp. He shares how Google Cloud helped LiveRamp modernize its data analytics infrastructure to simplify its operations, lower support and infrastructure costs and enable its customers to connect, control, and activate customer data safely and securely. LiveRamp is a data
