Enhanced AI-based Photo Editing: Let’s Enhance Fuels The Next Wave of Innovation with Google Cloud and NVIDIA - Build What's Next
Blog

Enhanced AI-based Photo Editing: Let’s Enhance Fuels The Next Wave of Innovation with Google Cloud and NVIDIA

2712

Of your peers have already read this article.

3:30 Minutes

The most insightful time you'll spend today!

By using Google Cloud and NVIDIA, Let’s Enhance addressed the real-time inference serving and infrastructure management challenges of taking their AI-enabled service to production at scale in a secure Google Cloud infrastructure. Read to know more.

There’s an explosion in the number of digital images generated and used for both personal and business needs. On e-commerce platforms and online marketplaces for example, product images and visuals heavily influence the consumer’s perception, decision making and ultimately conversion rates. In addition, there’s been a rapid shift towards user-generated visual content for ecommerce — think seller-generated product imagery, host-generated rental property photos and influencer-generated social media content. The challenge? These user-generated images are often captured using mobile cameras and vary greatly in terms of their size, quality, compression ratios and resolution, making it difficult for companies to provide consistent high-quality product images on their platforms.

That’s exactly the problem that Let’s Enhance, a computer vision startup with teams across the US and Ukraine, set out to solve with AI. The Let’s Enhance platform improves the quality of any user-generated photo automatically using AI-based features to enhance images through automatic upscaling, pixelation and blur fixes, color and low-light correction, and removing compression artifacts — all with a single click and no professional equipment or photo-editing chops.

“Let’s Enhance.io is designed to be a simple platform that brings AI-powered visual technologies to everyone — from marketers and entrepreneurs to photographers and designers,” said Sofi Shvets, CEO and Co-founder of Let’s Enhance.

To date, Let’s Enhance has processed more than 100M photos for millions of customers worldwide for use-cases ranging from digital art galleries, real estate agencies, digital printing, ecommerce and online marketplaces. With the introduction of Claid.ai, their new API to automatically enhance and optimize user-generated content at scale for digital marketplaces, they needed to process millions of images every month and manage sudden peaks in user demand.

However, building and deploying an AI-enabled service at scale for global use is a huge technical challenge that spans model building, training, inference serving and resource scaling. It demands an infrastructure that’s easy to manage and monitor, can deliver real-time performance to end customers wherever they are and can scale as user-demand peaks, all while optimizing costs.

To support their growing user-base, Let’s Enhance chose to deploy their AI-powered platform to production on Google Cloud and NVIDIA. But before diving into their solution, let’s take a close look at the technical challenges they faced in meeting their business goals.

Architecting a solution to fuel the next wave of growth and innovation

To deliver the high-quality enhanced images that end-customers see, Let’s Enhance products are powered by cutting-edge deep neural networks (DNNs) that are both compute- and memory-intensive. While building and training these DNN models in itself is a complex, iterative process, application performance — when processing new user-requests, for instance — is crucial to delivering a quality end user experience and reducing total deployment costs.

A single inference or processing request i.e., from user-generated image, which can vary widely in size, at the input to the AI-enhanced output image, requires combining multiple DNN models within an end-to-end pipeline. The key inference performance metrics to optimize for included latency (the time it takes from providing an input image to the enhanced image being available) and throughput ( the number of images that can be processed per second)..

Together, Google Cloud and NVIDIA technologies provided all the elements that the Let’s Enhance team needed to set themselves up for growth and scale. There were three main components in the solution stack:

  • Compute resources: A2 VMs, powered by NVIDIA A100 Tensor Core GPUs
  • Infrastructure management: Google Kubernetes Engine (GKE)
  • Inference serving: NVIDIA Triton Inference Server

Improved throughput and lower costs with NVIDIA A100 Multi-Instance GPUs (MIG) on Google Cloud

To meet the computational requirements of the DNN models and deliver real-time inference performance to their end-users, Let’s Enhance chose Google Cloud A2 VMs powered by NVIDIA A100 Tensor Core GPUs as their compute infrastructure. A100 GPU’s Multi-Instance GPU (MIG) capability offered them the unique ability to partition a single GPU into two independent instances and simultaneously process two user-requests at a time, making it possible to service a higher volume of user requests while reducing the total costs of deployment.

The A100 MIG instances delivered a 40% average throughput improvement compared to the NVIDIA V100 GPUs, with an increase of up to 80% for certain image enhancement pipelines using the same number of nodes for deployment. With improved performance from the same sized node pools, Let’s Enhance observed a 34% cost savings using MIG-enabled A100 GPUs.

Simplified infrastructure management with GKE

To offer a guaranteed quality-of-service (QoS) to their customers and manage user demand, Let’s Enhance needed to provision, manage and scale underlying compute resources while keeping utilization high and costs low.

GKE offers industry-leading capabilities for training and inference such as support for 15,000 nodes per cluster, auto-provisioning, auto-scaling and various machine types (e.g. CPU, GPU and on-demand, spot), making it the perfect choice for Let’s Enhance.

With support for NVIDIA GPUs and NVIDIA GPU sharing capabilities, GKE can provision multiple A100 MIG instances to process user requests in parallel and maximize utilization. As the compute required for the deployed ML pipelines increases (e.g., a sudden surge in inference requests to service), GKE can automatically scale to additional node-pools with MIG partitions — offering finer granularity to provision the right-sized GPU acceleration for workloads of all sizes.

“Our total averaged throughput varied between 10 and 80 images / sec. Thanks to support for NVIDIA A100 MIG and auto scaling mechanisms in GKE we can now scale that up to 150 images/sec and more, based on user-demand and GPU availability,” said Vlad Pranskevičius, Co-founder and CTO, Let’s Enhance.

In addition, GKE’s support for dynamic scheduling, automated maintenance and upgrades, high availability, job API, customizability and fault tolerance simplified managing a production deployment environment, allowing the Let’s Enhance team to focus on building advanced ML pipelines.

High-performance inference serving with NVIDIA Triton Inference Server

To optimize performance and simplify the deployment of their DNN models onto the GKE-managed node pools of NVIDIA A100 MIG instances, Let’s Enhance chose the open-source NVIDIA Triton Inference Server, which deploys, runs and scales AI models from any framework onto any GPU- or CPU-based infrastructure.

NVIDIA Triton’s support for multiple frameworks enabled the Let’s Enhance team to serve models trained in both TensorFlow and PyTorch, eliminating the need to set up and maintain multiple serving solutions for different framework backends. In addition, Triton’s ensemble model and shared memory features helped maximize performance and minimize data transfer overhead for their end-to-end image processing pipeline, which consists of multiple models and large amounts of raw image data transferring between them.

“We saw over 20% performance improvement with Triton vs. custom inference serving code, and were also able to fix several errors during deployment due to Triton’s self-healing features”, said Vlad. “Triton is packed with cool features, and as I was watching its progress from the very beginning, the pace of product development is just amazing.”

To further enhance end-to-end inference performance, the team is adopting NVIDIA TensorRT, an SDK to optimize trained models for deployment with the highest throughput and lowest latency while preserving the accuracy of predictions.

“With the subset of our models which we converted from Tensorflow to NVIDIA TensorRT we observed a speedup of anywhere between 10% and 42%, depending on the model, which is impressive,” said Vlad. “For memory-intensive models like ours, we were also able to achieve predictable memory consumption, which is another great benefit of using TensorRT, helping prevent any unexpected memory-related issues during production deployment.”

Teamwork makes the dream work

By using Google Cloud and NVIDIA, Let’s Enhance addressed the real-time inference serving and infrastructure management challenges of taking their AI-enabled service to production at scale in a secure Google Cloud infrastructure.

GKE, NVIDIA AI software and A2 VMs powered by NVIDIA A100 GPUs brought together all the elements they needed to deliver the desired user experience and build a solution that can dynamically scale their end-to-end pipelines based on user demand.

The close collaboration with the Google Cloud and NVIDIA team, every step of the way, also helped Let’s Enhance get their solution to market fast. “The Google Cloud and NVIDIA teams are incredible to work with. They are highly professional, always responsive and genuinely care about our success,” said Vlad. “We were able to get technical advice and recommendations directly from Google Cloud product and engineering teams, and also have a channel to share our feedback about Google Cloud products, which was really important to us.”

With a solid foundation and solution architecture that can meet their growing business needs, Let’s Enhance is marching towards their goal of bringing AI-enhanced digital images to everyone. “Our vision is to help businesses effectively manage user-generated content and increase conversion rates with next-gen AI tools,” said Sofi. ”Our next step towards that is to expand our offerings to completely replace manual work for photo preparation, including as well as cover pre-stages as image quality assessment and moderation.”

To learn more about the Let’s Enhance products and services, check out their blog here.

Blog

Discover Gen App Builder: Transform Search and Conversational Experiences Through Generative AI

1250

Of your peers have already read this article.

3:30 Minutes

The most insightful time you'll spend today!

Unlock the power of AI-driven search and conversations using Gen App Builder. Learn how generative AI is reshaping interactions and driving more engaging, personalized user experiences in the digital landscape. Read more!

If you’ve been exploring recently-launched consumer generative AI tools like Bard and thinking about how to build similar experiences for your business, Generative AI App Builder, or Gen App Builder for short, is here to get you started.

Gen App Builder is part of Google Cloud’s recently announced generative AI offerings and lets developers, even those with limited machine learning skills, quickly and easily tap into the power of Google’s foundation models, search expertise, and conversational AI technologies to create enterprise-grade generative AI applications. 

“Google Cloud’s leading AI technology enables STARZ customers to discover more relevant content, increasing engagement with, and the likelihood of completing the content served to them,” says Robin Chacko, EVP Direct-to-Consumer, STARZ. “We’re excited about how generative AI-powered search will help users find the most relevant content even easier and faster.”

Gen App Builder is exciting because unlike most existing generative AI offerings for developers, it offers an orchestration layer that abstracts the complexity of combining various enterprise systems with generative AI tools to create a smooth, helpful user experience. Gen App Builder provides step-by-step orchestration of search and conversational applications with pre-built workflows for common tasks like onboarding, data ingestion, and customization, making it easy for developers to set up and deploy their apps. With Gen App Builder developers can: 

  • Build in minutes or hours. With access to Google’s no-code conversational and search tools powered by foundation models, organizations can get started with a few clicks and quickly build high-quality experiences that can be integrated into their applications and websites. 
  • Combine the power of foundation models with information retrieval to find relevant, personalized information. Enterprises can build apps that understand user intent via natural language, and surface the right information with associated citations and attributions from a company’s public and private data. They can also fully control what data their applications access and the content or topics they want to address.
  • Build multimodal apps that can respond with text, images, and other media. Gen App Builder supports not just text, but also other modalities such as images and videos. It allows developers to build apps using a combination of text and images as inputs to find information across documents, photos, and video content, enabling richer customer interactions. 
  • Combine natural conversations with structured flows. Developers can granularly blend the output of foundation models with controls to ground answers in enterprise content, and step-by-step conversation orchestration to guide customers to the right answers.
  • Provide the ability to transact and connect to third party apps and services. Gen App Builder makes it simple to create digital assistants and bots that not only serve content, but also connect to purchasing and provisioning systems to enable transactions from the conversational UI, and escalate customer conversations to a human agent when the context demands.

A new generation of conversational AI experiences and assistants 

Consumers of enterprise applications expect to interact with technology in a seamless, conversational way to quickly find the information they need and act on it. Gen App Builder can help reinvent these customer and employee experiences by ingesting large, complex datasets that are specific to your company–from websites, documents, and transactional systems like billing and inventory, to emails, chat conversations, and more. These AI-powered apps can synthesize information across all of these sources to provide specific, actionable responses, using only the data you have provided. 

Some of the most popular uses are in customer service, where generative apps can contribute to increasing revenue, customer satisfaction, and customer loyalty. For example, if a retail customer reaches out to modify an order, a virtual agent can help them change it to another product. The customer doesn’t even need to provide the new product name—they can just upload an image and let the agent guide them through the rest. Watch this demo to see how a retail chatbot can use multimodal capabilities to help a consumer navigate various options on the website, including giving the customer ideas on how to use the product and even helping them complete the purchase with the ability to transact within the conversational UI. This scenario could apply to multiple industries and use cases, ranging from consumer goods and public services, to finance and internal corporate systems like intranets.

Combining the power of Google-quality search with foundation models

Finding the right information from data across the organization is a critical requirement within any enterprise. Yet it can be challenging to build high-quality enterprise search experiences with existing tools. Current systems struggle to understand user intent, are difficult to implement and customize, and don’t provide a high-quality user experience. 

One of the most exciting features of Gen App Builder is the ability to combine the power of Google-quality search with generative AI to help enterprises find the most relevant and personalized information when they need it. With Gen App Builder, enterprises can build conversational search experiences across their public and private data in minutes or hours with no coding experience. 

Enabling multimodal search across text, images and video within the enterprise is a key aspect of the search experiences in Gen App Builder. In addition to providing high-quality search results, Gen App Builder can conveniently summarize the results and provide corresponding citations in a natural, human-like fashion. Gen App Builder also automatically extracts key information from the data and enables personalized results for users. Watch this demo to see how these capabilities can come together to transform the search experience for employees at a financial services firm. The ability to integrate Google-quality search within the enterprise’s applications means they can enjoy a new level of data utilization, drive increased process efficiencies, and provide delightful experiences to their employees and customers.

“Customers have been shopping at Macy’s for generations. Being able to deliver 360° personalization and contextual recommendations will help ensure that Macy’s is still providing future generations of shoppers with a seamless, exceptional experience,” said Bennett Fox-Glassman, Senior Vice-President, Customer Journey, Macy’s. “We’ve already realized an increase in revenue per visit and conversion rates had great success using Google Cloud’s AI technology and are looking forward to exploring how these latest announcements bring together Natural Language Processing and Generative AI capabilities to deliver next-gen search and conversational experiences for our customers.”

The ability to intuitively interact with complex data across a variety of sources allows organizations to better serve their customers and deliver more relevant offerings. Combined with conversational and fulfillment abilities, the potential for improving customer engagement and employee productivity is immense. We’re excited to see how developers and enterprises use a mix of these capabilities to power new experiences and revenue opportunities.

If you’re interested in a closer look at the Gen App Builder, tune into this session at the Data Cloud & AI Summit. Take a step forward to getting hands-on and join the waitlist for our trusted tester program. And finally, bookmark our generative AI landing page to keep abreast of the latest news, updates and possibilities from this exciting new world of Gen Apps.

3073

Of your peers have already watched this video.

14:30 Minutes

The most insightful time you'll spend today!

Explainer

Overview of AI Notebooks on Google Cloud

Artificial intelligence and machine learning are one of the most disruptive technologies that enterprises have encountered in the last four or five years. And AI and ML will continue to disrupt enterprises going forward for the next 10 years.

The key to unlocking value with artificial intelligence starts with a model. And the simplest way to develop a model is to use notebooks,  and within notebooks to use open source frameworks to develop AI models.

Notebooks are the go-to tool for data scientists to develop and deploy AI/ML models. In this overview video, Suds Narasimhan, Product Manager, Google Cloud, walks you through an overview of cloud AI notebooks, Google Cloud’s managed Jupyter lab notebook service for enterprises on the Google Cloud.

He explores how cloud AI notebooks can help enterprise data scientists explore data quickly and develop an AI and ML model and deploy it into production.

Research Reports

How Data Efficiency with Google Cloud Empower Governments to Make Data-first Decisions

5058

Of your peers have already read this article.

1:30 Minutes

The most insightful time you'll spend today!

Respond to changes with confidence by aligning data strategy with infrastructure. Read this blog on insights from the Built to Last: A Survey on Organizational Data Efficiency in Times of Crisis on how governments can leverage integrated data systems

Presently, every government agency has to take a hard look at their data capabilities and decide whether their current infrastructure supports their workflow. For many, it doesn’t. Most data systems are developed with a strict set of parameters in mind before implementation, which can limit flexibility and long-term use. Particularly during a crisis, flexible “living systems” offer tremendous advantages as they’re able to change capacity rapidly. Building living data systems with the cloud in mind allows organizations to respond to a changing world with confidence.

Last summer, the Government Business Council conducted a survey of government employees to understand the impacts of data efficiency on government operations. The report Built to Last: A Survey on Organizational Data Efficiency in Times of Crisis offers key insights into organizational efficacy and whether organizations can adapt to a crisis at speed. It also highlights differences between traditional data systems and living data systems. 

Data needs to be readily available

When the pandemic first hit, many agencies needed to create or transition their systems to allow employees to work remotely. This change tested the limits of existing data systems. Even after finding a cloud service provider, agencies encountered the challenges of migrating their data to the cloud. 

Government organizations had decades of data stored in paper records. Most have been working to transfer these records to a digital format, but the process has been slow. They are also faced with collecting sizable amounts of data in real time from their ongoing services,  which involves interfacing with the public, external vendors, or third-party institutions. 

Building the cloud into a flexible data system can solve both issues. Old records can be digitized and given an easy-to-access home for those who need them. Incoming data, both internal and external, can be made accessible as well. Migrating data to the cloud also doubles as a way to create backups of raw data, adding an extra layer of security. Most importantly, building in the cloud unlocked the capacity to scale when demand rises. 

Data should be updated in real-time

One of the key takeaways from the Government Business Council report is the fact that agencies are better able to adapt at speed when data efficiencies are higher. 74% of organizations with pandemic related functions reported a moderate to severe impact to their jobs at the onset of the pandemic. Of those organizations, the ones reporting their data efficiency as “very good” have largely already recovered. That adaptability directly affects an agency’s ability to make informed decisions during a time of a crisis.

Having a real-time data solution in place lets agencies make near real-time decisions. A great example of this from early in the pandemic is vaccine distribution. Google Cloud supported multiple states, such as the State of Wyoming, in distributing vaccines efficiently while handling challenges such as reaching rural populations. Data systems that gathered real-time patient data made a difference in the number of vaccines distributed. Knowing population data and patient risk factors enabled quick and effective decision-making.

A global pandemic is far from the only crisis that needs effective data analytics. Natural disasters, food deserts, public health issues, and more can all be handled more efficiently by having real-time data at hand. Effective data analytics systems are the digital equal of “having your ear to the ground” in each community. They provide valuable insights into what people  need.

Data needs to be accessible and easy to use

Making data easy to work with and understand sets phenomenal data systems apart from functional ones. Having data in the cloud is a great first step, but agencies need to be able to easily access and quickly use the data to accomplish their goals. This is where traditional data systems fail most often. Traditional IT systems and data strategies are designed for a specific purpose, usually identified before development and implementation begin. That means that when the data living in those systems needs to be used differently, adapting to new requirements can be difficult. 

Data can often feel “locked” in traditional systems; the data is there, but there’s no way to get to it or work with it in a way that meets the needs of a crisis. Flexible data systems address this by allowing for greater accessibility. Google Cloud, for example, has customizable tools, such as Contact Center AI and Document AI, which let agencies work with data in ever-changing ways. This also produces greater data transparency since data sets can be worked with and accessed more easily.

Governments need to respond to the changing needs of their constituents in emergencies. While traditional data systems can handle slowly shifting demands on the system, they do not serve agencies well in a crisis. When urgency, accuracy, and accessibility all matter, flexible systems rise to the challenge. The pandemic has pushed agencies to adapt in real time, and many have realized they need a system that adapts with them.

Google Cloud has a suite of tools to create integrated data ecosystems. These ecosystems can scale with increasing demand, meet dynamic development needs, and adapt to a changing landscape. Data-first decision-making is a core tenet of “living data systems.” Google Cloud data systems have handled everything from administering vaccines to detecting fraud. In each of these applications, a core tenet of data-first decision making was implemented at scale. 

For more insights on how flexible data systems help the public sector, download the full report “Built to Last: A Survey on Organizational Data Efficiency in Times of Crisis.

E-book

Speech Analytics and AI: Leverage Every Customer Interaction as a Data Point

DOWNLOAD E-BOOK

4462

Of your peers have already downloaded this article

3:00 Minutes

The most insightful time you'll spend today!

Contact centers can transform customer experience using AI-driven speech analytics to evaluate every customer interaction and use it as a ‘data point’ to identify key patterns, enquiries, pain points, reviews and feedback to enhance customer experience with real-time, personalized recommendations. Knowlarity’s AI-based cloud telephony solutions for businesses built with Google Cloud takes speech analytics to another level!

Download the article to empower your contact centers with AI-powered speech analytics to make predictive analysis, reduce call handling time and volume as well as train contact center agents to provide customers with quick and real-time feedback.

3148

Of your peers have already watched this video.

11:30 Minutes

The most insightful time you'll spend today!

Explainer

Transitioning Kagglers to TPU with TF 2.x

Kaggle has, historically, become synonymous with machine learning competitions but it’s much more than that. Kaggle is a data science platform. Over 5 million data scientists from all over the world come to Kaggle to not only not only participate in machine learning competitions but to learn data science build their skills, polish their portfolios and share data sets and code.

Earlier on Kaggle introduced TPU support through its competition platform. In this video, Addison Howard, Program Manager, Google Cloud and Phil Culliton, Kaggle Data Scientist, Google Cloud talk about how Kaggler competitors transition from GPU to TPU use – first in Colab, and then in Kaggle notebooks.

More Relevant Stories for Your Company

Webinar

Chronicle Security Analytics: Key to Address Security Data Overload

Google Cloud's Chronicle is a security analytics platform built for modern use cases to combat modern threats. In today's world, enterprises have undergone significant changes in all aspects, and must adapt to their security needs to counter threats and attacks. Watch the video to learn Google Cloud's initiative to help

Case Study

City of San Jose Ensures Critical Services Reach Community Using AI Translation

San José is one of the most diverse U.S. cities, with residents speaking more than 100 languages. Several years ago, we set out to improve city community interactions through more equitable service management and delivery. This demanded a new approach to automating the intake of requests from a majority population

Case Study

Google Cloud’s Firebase Realtime Database and BigQuery AllowsCastbox to Ramp Up Customer Experience

Demand for spoken audio content such as podcasts remains robust despite the proliferation of video services and other entertainment options for consumers. Shibin Li, Co-founder of Castbox, credits growth of the global podcast platform to the following: speed and availability, market-leading features, the proliferation of smart devices to deliver audio content,

Blog

Analytics Hub for Secure Data Sharing and Analytics Unlocks True Data Value and Insights

Customers tell us that sharing and exchanging data with other organizations is a critical element of their analytics strategy, but it’s hamstrung by unreliable data and processes, and only getting harder with security threats and privacy regulations on the rise.  Furthermore, traditional data sharing techniques use batch data pipelines that are

SHOW MORE STORIES