As the volume of data that people and businesses produce continues to grow exponentially, it goes without saying that data-driven approaches are critical for tech companies and startups across all industries. But our conversations with customers, as well as numerous industry commentaries, reiterate that managing data and extracting value from it remains difficult, especially with scale.

Numerous factors underpin the challenges, including access to and storage of data, inconsistent tools, new and evolving data sources and formats, compliance concerns, and security considerations. To help you identify and solve these challenges, we’ve created a new whitepaper, “The future of data will be unified, flexible, and accessible,” which explores many of the most common reasons our customers tell us they’re choosing Google Cloud to get the most out of their data.

For example, you might need to combine data in legacy systems with new technologies. Does this mean moving all your data to the cloud? Should it be in one cloud or distributed across several? How do you extract real value from all of this data without creating more silos?

You might also be limited to analyzing your data in batch instead of processing it in real-time, adding complexity to your architecture and necessitating expensive maintenance to combat latency. Or you might be struggling with unstructured data, with no scalable way to analyze and manage it. Again, the factors are numerous—but many of them accrue to inadequate access to data, often exacerbated by silos, and insufficient ability to process and understand it.

The modern tech stack should be a streaming stack that scales with your data, provides real-time analytics, incorporates and understands different types of data, and lets you use AI/ML to predictively derive insights and operationalize processes. These requirements mean that to effectively leverage your data assets:

• Data should be unified across your entire company, even across suppliers, partners, and platforms., eliminating organizational and technology silos.
• Unstructured data should be unlocked and leveraged in your analytics strategy.
• The technology stack should be unified and flexible enough to support use cases ranging from analysis of offline data to real-time streaming and application of ML without maintaining multiple bespoke tech stacks.
• The technology stack should be accessible on-demand, with support for different platforms, programming languages, tools, and open standards compatible with your employees’ existing skill sets.

With these requirements met, you’ll be equipped to maximize your data, whether that means discerning and adapting to changing customer expectations or understanding and optimizing how your data engineers and data scientists spend their time. In coming weeks, we’ll explore aspects of the whitepaper in additional blog posts—but if you’re ready to dive in now, and to steer your tech company or startup towards success by making your data better work for you, click here to download your copy, free of charge.

Google Cloud results

• Helps enable instant document approval with optical character recognition by Vision API
• Processes 100,000 documents in 20 minutes with automated document processing product Recognic, powered by Vision API and Apigee
• Helps increase the number of applications processed by 30% within the same timeframe

The insurance landscape in India has seen significant changes in recent years with the adoption of new technology. As one of the major insurance providers in the country, ICICI Prudential Life Insurance has aimed to lead in this transformation journey. “There has been a data explosion across India over the past few years, together with a high mobile penetration rate. Today, about 60% of our customers approach us via mobile, for example, which was certainly not the case before,” says Alpesh Karnik, SVP, IT, at ICICI Life Insurance.

Consumer expectations have also evolved, with easier access to information and online services. “Consumers today are more informed on the importance of investing in insurance products, so there’s much more of a pull factor when it comes to sales, but they also want to be able to get these products quickly and easily,” adds Alpesh. To meet the demands of these consumers, ICICI Prudential Life Insurance realized it needed to make its processes even faster and more efficient. Looking to upgrade its infrastructure, the company turned to Google Cloud.

“The biggest benefit of using Recognic and Vision API is that it eliminates the initial waiting time, which can result in drop-offs. Now customers can know immediately whether their documents are sufficient, or if they need to revise or submit any others.”—Alpesh Karnik, SVP, IT, ICICI Prudential Life Insurance

Serving customers better by speeding up processes with Google Cloud

ICICI Prudential Life Insurance’s distributors were already using tablets to input customer data faster and more efficiently, but many of the company’s solutions still required a team at the back end to manually sift through documents for approval. This meant that customers needed to wait five or six hours, or sometimes until the next working day, to know if their documents were approved or needed revision.

That all changed after partnering with Google Cloud Premier Partner Searce to take advantage of its AI/ML powered automated document processing product Recognic, which is built on Google Cloud. Developed using the optical character recognition (OCR) capabilities of Cloud Vision, Recognic reads, understands, and validates documents at scale, enabling organizations that handle massive amounts of paperwork to digitize these documents and then accurately store and index them.

“Google Cloud has cut down the middle- and back-office work, leading to a 30% increase in the number of applications we can process in the same time span without the need for additional resources.”—Alpesh Karnik, SVP, IT, ICICI Prudential Life Insurance

“In the case of ICICI Prudential, the biggest benefit of using Recognic and Vision API is that it eliminates the initial waiting time, which can result in drop-offs. Now customers can know immediately whether their documents are sufficient, or if they need to revise or submit any others,” Alpesh adds.

Alpesh explains that if the details on the application form match the documents provided, the case doesn’t need to go to the underwriter for further checks and can go directly to policy issuance. “Google Cloud has cut down the middle- and back-office work, leading to a 30% increase in the number of applications we can process in the same time span without the need for additional resources.”

ICICI Prudential Life Insurance is also working with Searce to build deep learning models into Recognic so that it can overcome template barriers and input data from a variety of forms. This is particularly helpful for financial and medical documents underwriting because unlike a passport or driving license, financial documents have a higher structural complexity.

As customer data becomes more important in the work of ICICI Prudential Life Insurance, so does protecting it, and the company is taking every measure to safeguard the security and privacy of its customers’ information. “Details of customers’ contactability are automatically removed by Google Cloud after processing is complete. This step in the workflow gives us the confidence that data is not stored at any level of the optical character recognition process,” says Alpesh.

Partnering with the right teams for dedicated support

In achieving the best solution for its business goals, ICICI Prudential Life Insurance recognizes the importance of its decision to work with partners that truly understand the insurance business. “There are many intricacies involved in this business, and it’s clear that both Google Cloud and Searce really took the time to understand our underwriting processes before coming up with a solution,” says Alpesh. He adds that during the implementation process, all findings were well documented and queries were responded to quickly.

“We didn’t want to take any shortcuts deploying Recognic, but at the same time, we didn’t want to draw out the implementation process. The excellent support from both Google Cloud and Searce throughout the journey was reassuring for us as they were always thinking ahead.”

Future-proofing the organization through machine learning and AI

In the coming years, Alpesh foresees the insurance industry to be even more agile than it is today. “I doubt elaborate processes such as underwriting or operations checks will need to be done manually in the future. Everything will be done through machine learning and AI.” In light of this, ICICI Prudential Life Insurance is doing everything it can to prepare, as customers’ expectations are set to keep evolving. “We have to be prepared for the future, and I believe that with Google Cloud, we can do it.”

About ICICI Prudential Life Insurance

ICICI Prudential Life Insurance aims to lead the Indian insurance field through quality products and a hassle-free claim settlement experience. A customer-centric company, it offers long-term savings and protection plans to meet customers’ needs at every stage of life.Industries: Financial Services & InsuranceLocation: India

About Searce

Searce is a niche cloud consulting business with futuristic tech in its DNA, focused on “realizing the Next in the Now” for its clients. Specializing in cloud data engineering, AI/ML, and ad

At 340 square miles, the City of Memphis is among the largest in the United States in terms of land area. Memphis has over 6,800 lane-miles of city streets, enough to drive back and forth to Los Angeles four times. Keeping these streets well maintained and safe for citizens and visitors is a major priority for the city.

Lots of traffic, lots of roads, and a four-season climate prone to wintertime freeze-thaw-refreeze cycles means the opportunity for potholes. Although the city aims to fill potholes within five business days of notification, it can take longer, especially during winter and early spring. Last year, the city’s Public Works crews repaired some 63,000 potholes, only 20% of which were reported by residents. Approximately 32,000-man-hours each year are spent repairing potholes, with seasonal fluctuations requiring ten to twelve Street Maintenance crews working steadily during the winter months. Still, many went unreported, leading the city to flag pothole request resolution under “needs improvement” on its open data portal website.

Like many large cities, Memphis also struggles with vacant and blighted properties. Nearly 15,000 properties in Memphis are likely vacant, and city officials contend that many are owned by out-of-town investors who live elsewhere and do not take necessary restoration or maintenance steps. These properties can decrease the value of surrounding real estate and discourage new businesses and other residents from moving to an area. Citizen frustration and concerns over the number of blighted properties has made blight eradication a major focus of the City of Memphis.

Historically, residents reported potholes and blighted properties by calling 311, or more recently by using the Memphis 311 app. However, these reports only covered about 20 percent of the problems — often the worst cases. And by the time residents took the initiative to submit a 311 report, they usually weren’t feeling good about the situation.

Recognizing that potholes and vacant properties are often the most visible indicators of whether a city government is doing its job efficiently, Memphis Mayor Jim Strickland and CIO Mike Rodriguez began looking for ways they could apply technology to fix the problems. Mike approached Google for ideas, and Google recommended conducting a machine learning proof-of-concept (POC) with SpringML, a Google Cloud Partner.

“Memphis is focused on easy living, and we want to do everything we can to keep our citizens happy,” says Mike Rodriguez. “Working with Google and SpringML to reduce potholes and urban blight using machine learning and artificial intelligence was an easy decision.”

Bringing machine learning to city operations and budgets

The city’s goal is to detect potholes and abandoned properties by analyzing video footage of roads and residential properties. It wanted to classify potholes by width and depth, and share the information with workers who can repair them. For abandoned properties, it wanted to enable more strategic deployment of resources for homeowners citywide and take action to hold neglectful property owners accountable.

The POC began by training TensorFlow models for ML object detection using preconfigured AI Platform Deep Learning VM Images on Compute Engine. SpringML helped set up cameras and developed a user interface to collect pothole data and automate the 311 ticketing process.

Together, the teams analyzed 30 days of video from a moving city bus and high-resolution video from 360-degree cameras mounted to a code enforcement vehicle, overlaid with data from 311 reports. As the models were refined, accuracy quickly climbed from 50 percent to over 90 percent as models were taught to differentiate a pothole from a manhole cover or other object.

The city also imported routes, potholes, and paving data along with geolocation data from ArcGIS and Google Maps into BigQuery to better understand street conditions and the proximity of potholes to one another. BigQuery also analyzes city property records, tax records, 311 reports, and third-party survey data on-demand to predict where homes are starting to become run down and where neighborhood decay is most likely to occur. The SpringML team created a pilot analysis to begin vacant property protections and developed a user interface tool to interact with the model’s results.

“Google Cloud Platform made it possible for us to experiment with machine learning and artificial intelligence to help solve our city’s problems while working within the budget constraints of a municipal IT organization,” says Mike. “Google turned a ‘nice to have’ into a ‘let’s do this!'”

Identifying 75 percent more potholes

Memphis expects to substantially reduce the number of potholes on its streets, creating a better driving experience for residents and visitors alike. Because drivers won’t be as likely to swerve to miss a pothole, streets will be safer and friendlier to bicycles and scooters. Fewer potholes will also save the city between $10,000 and $20,000 annually in city claims that it pays out in cases where vehicle damage results from a pothole that was not addressed in a timely manner.

“Historically, Public Works has relied primarily upon Street Maintenance crews to proactively locate and fill potholes. As Memphis has over 6,800 lane-miles of public streets, it is a daunting task to reliably survey the entire system in an efficient and systematic way,” says Robert Knecht, Public Works Director for the City of Memphis. “The outcome of the data collected will be invaluable to Public Works so that it can ensure it is managing the city’s street system in a more proactive manner.”

Memphis will be able to better prioritize road maintenance based on condition and impact, increasing the efficiency of its Public Works road crews. Analyzing video of streets also gave the city visibility into issues it wasn’t previously aware of, such as curbs, gutters, and manhole covers that had been mistakenly paved over and need to be excavated. The ML process is easily transferrable to other concerns as well, helping the city identify illegal signs or spools of cable hanging on light posts that could be potentially unsafe.

Helping communities recover and thrive

Memphis is also having success in analyzing predictive trends to combat high rates of abandoned and blighted properties, surpassing 97.5 percent accuracy. “In the past, Public Works experimented with comprehensive, city-wide blight identification by using approximately 200 volunteers to survey and photograph over 237,000 city parcels. This effort was costly, took a long time to complete, and resulted in inconsistent data collection,” says Robert. “Blighted property conditions can change quickly in a city the size of Memphis. Now, with this new technology, Memphis will be able to make a significant difference in the efforts to proactively and comprehensively identify and manage blighted and substandard properties.”

Code Enforcement with better data-driven detection mechanisms enables the city to also identify cases where homeowners are not physically or financially able to keep up with the challenges of homeownership and make them aware of resources that are available to assist them. Memphis Code Enforcement can do a better job of finding people living in derelict properties that pose hazards to inhabitants’ health and safety, and help them fix those problems or find a new place to live.

“Using SpringML and Google Cloud Platform to detect indicators of vacant or blighted properties will help Memphis create safer neighborhoods that will be more attractive to businesses and home buyers,” says Mike. “Property values and employment will go up, crime will go down, and social services can be more focused and effective.”

Revolutionizing service delivery for citizens

Memphis is proving the viability of a cost-effective, cloud-based machine learning model that other cities can follow. The city is already looking into new applications of AI and ML that will further improve city services and help it build a better future for its 652,000 residents.

As part of his commitment to a transparent government, Memphis Mayor Jim Strickland created an open data policy that commits to releasing raw data and sharing it with citizens in a variety of downloadable formats. Going forward, this transparency will help citizens understand how their needs are being served and uncover new, innovative use cases for AI and ML.

“Our goal is to become a smart city, and technologies such as Google Cloud Platform and SpringML put us ahead of the game,” says Mayor Strickland. “Google understands data, and there isn’t a better company to help us analyze our data resources for actionable insights.”

The home lending journey entails processing an immense number of documents daily from hundreds of thousands of borrowers. Currently, home lending document processing relies on some outdated digital models and a high dependency on manual labor, resulting in slow processing times and higher origination costs. Scaling a business that sorts through millions of documents daily, while increasing efficacy and accuracy, is no small feat. When it comes to applying for a mortgage loan, consumers expect a digital experience that’s as good as the in-person one. Roostify simplifies the home lending journey for lenders and their customers.

No time to spare: Overcoming document processing challenges with AI

Roostify provides enterprise cloud applications for mortgage and home lenders. In order to empower its customers to deliver a better, more personalized lending experience, they needed to automate and scale their in-house document parsing functionality.

As a key component of its document intelligence service, Roostify is leveraging Google Cloud’s Lending DocAI machine learning platform to automate processing documents required during a home loan application process, such as tax returns or bank statements with multi-language support. This partnership delivers data capture at scale, enabling Roostify customers to automatically identify document types from the uploaded file and to extract relevant entities such as wages, tax liabilities, names, and ID numbers for further processing, and make things move faster in the cumbersome lending process. 

Roostify’s solutions leverage Google Cloud’s Lending DocAI, which is built on the recently announced Document AI platform, a unified console for document processing. Customers can easily create and customize all the specialized parsers (e.g., mortgage lending documents and tax returns parsers) on the platform without the need to perform additional data mapping or training. All Google Cloud’s specialized parsers are fine-tuned to achieve industry-leading accuracy, helping customers and partners confidently unlock insights from documents with machine learning. Learn more about the solution from the GA launch blog and the overview video.

Integrating Lending DocAI’s intelligent document processing capabilities into the Roostify platform means more innovation for their customers and tangible results: faster loan processing times, fewer document intake errors, and lower origination costs. Additional support in Google Lending DAI for other languages and more documents like global Know Your Customer (KYC) documents or payroll reports is in the near future.

Full integration of AI solutions

Working together with Roostify’s platform team, we were able to help them solve their document processing challenge through integration of various GCP products such as Lending DocAI (LDAI), Data Loss Prevention (DLP) for redacting sensitive data, BigQuery for data warehousing and analytics, and Firestore for API status. To make it very safe and secure, all data was encrypted end-to-end at Rest and in Transit. LDAI won’t require any training data to process. It is an easy plug and play API.

Here is a sneak peek in the high level deployment architecture for LDAI in Roostify environment:

Here are the steps for processing data:

1. Receives document processing request from the client.
2. API Function directs requests to the pre-processing service. For Async requests a processing ID is generated and returned to the caller.
3. Pre-processing service sends the request for further processing (Long/short PDF conversion), calling other microservices and receives back the responses. Any error in the response received is then sent to the response processing service. 
4. If the response is synchronous, the pre-processing service directs it to the LDAI Invoker service. 
   1. If the response is asynchronous, the pre-processing service feeds it into the Cloud Pub/Sub service.
5. Cloud Pub/Sub service feeds the response back to the LDAI Invoker service.
6. LDAI Invoker service routes the request to the Google LDAI API for classification if there are multiple pages in the document.
7. Document will be split based on LDAI response and then saved in a GCS bucket for temporary storage.
8. LDAI entity interface for single page processing and then LDAI Invoker sends LDAI results to LDAI Response Processing
9. If a request is a synchronous request the LDAI Response Processor sends results to the API Function so that it can complete the synchronous call and respond to the rConnect caller.
   1. If the request is an asynchronous request the LDAI Response Processor will respond to the caller’s webhook and complete the transaction.
10. Finally, Data stored in the GCP bucket will be deleted.

All the responses that come from the LDAI API can optionally feed into BigQuery via the Response Processor, after parsing it through Data Loss Prevention (DLP) API to redact the PII/sensitive information.  Throughout the processing of both asynchronous and synchronous requests all transactions are logged using Cloud Logging.  For asynchronous transactions, the state is maintained throughout the process using Cloud Firestore.

Roostify currently uses this technology to power two different solutions: Roostify Document Intelligence and Roostify Beyond™. Roostify Document Intelligence is a real-time document capture, classification, and data extraction solution built for home lenders. It ingests documents uploaded by borrowers and loan officers, identifies the relevant documents, and extracts and classifies key information. Roostify Document Intelligence is available as a standalone API service to any home lender with any digital lending infrastructure already in place. 

Roostify Beyond™ is a robust suite of AI-powered solutions that enables home lenders to create intelligent experiences from start to close. It combines powerful data, insightful analytics, and meaningful visualization to streamline the underwriting process. Roostify Beyond™ is currently available only to Roostify customers as part of an Early Adopter program and will be rolled out to the market later this year.

Through this partnership, Roostify has enabled its customers to adopt a data-first approach to their home lending processes, which will lead to improved user experiences and significantly reduced loan processing times.

Fast track end-to-end deployment with Google Cloud AI Services (AIS)

Google AIS (Professional Services Organization), in collaboration with our partner Quantiphi, helped Roostify deploy this system into production and fast-tracked the development multifold to generate the final business value.

The partnership between Google Cloud and Roostify is just one of the latest examples of how we’re providing AI-powered solutions to solve business problems.

Cloud developers — you’ve got it all. You can code in your choice of languages, enjoy portability with containers, minimize complexity with serverless, and manage the entire software lifecycle by following DevOps principles. But let’s face it, building and onboarding new cloud applications still requires a lot of manual planning, synthesis, and, yes, hard work. You need to research and plan your deployment, create a workable, secure architecture, and of course, you need to write the actual code,

For the last several decades, the cloud has been primarily a “do it yourself” model with volumes of options that have made development more complicated. The cloud went from overwhelmingly exciting to…. a bit overwhelming.

What if we all could bring that excitement back? What if you had some help that was available whenever and wherever you needed it? 

Say hello to Duet AI for Google Cloud

Powered by Google’s state-of-the-art generative-AI foundation models, Duet AI for Google Cloud is an always-on AI collaborator that provides help to users of all skill levels where they need it. With Duet AI, we’re on a mission to deliver a new cloud experience that’s personalized and intent-driven, and can deeply understand your environment to assist you in building secure, scalable applications, while providing expert guidance. 

As we evolve Google Cloud with Duet AI, we are looking to build a cloud platform that is more human-centric, holistic, and helpful, with responsible AI at the center of the experience:

• Human-centric: With Duet AI, we are making Google Cloud more accessible and personal to any type of user at any skill level by providing them with support whenever they need it, from code recommendations for developers, to prompt-based data insights for data engineers, to chat-based app creation for business users.
• Holistic: With generative AI at the center of the cloud experience, cloud development can be more cohesive, with fewer silos across functions, services, and tech stacks, providing a holistic picture in the format you want, wherever you are in Google Cloud.
• Helpful: To deliver smarter, contextual recommendations for building and operating apps with Google Cloud, we pre-trained Codey, one of the foundation models that powers Duet AI, with Google Cloud-specific content like documentation and sample code, and fine-tuned it based on Google Cloud  user behaviors and patterns. 
• Responsible: Our AI Principles set out our commitment to developing technology responsibly. Your code and recommendations will not be reused for any model learning and development. This helps ensure the privacy of your data and code, and also the integrity of the knowledge space from which our AI models are trained.

New capabilities available in Duet AI for Google Cloud

Here are some of the new capabilities available to get us started on our mission to deliver a new personalized and intent-driven cloud experience:

• Code assistance provides AI-driven code assistance for cloud users such as application developers and data engineers. It gives code recommendations as they type in real time, generates full functions and code blocks, and identifies vulnerabilities and errors in the code, while suggesting fixes.

Code assistance auto-generates code for creating a Google Cloud Storage bucket

Code assistance will be available through multiple products and services across Google Cloud, such as in Cloud Workstations, our fully-managed secure development environment, and other code-editing experiences in the Google Cloud Console. Developers will also find code assistance in Cloud Shell Editor or via our Cloud Code IDE extensions for VSCode and JetBrains IDEs. It supports multiple languages including Go, Java, Javascript, Python, and SQL.

• Chat assistance allows people to use simple natural language to get answers on specific development or cloud-related questions. Users can engage with chat assistance to get real-time guidance on various topics, such as how to use certain cloud services or functions, or get detailed implementation plans for their cloud projects. It can also provide architectural or coding best practices, helping to reduce the need to go searching for relevant documents.

Use chat assistance to get the detailed steps for deploying an app on Cloud Run

Chat assistance will also be available across multiple Google Cloud surface areas, for example IDEs, the Cloud Console, and through products and services. Whether you’re a developer, operator, data engineer, or security professional, you’ll be able to leverage chat assistance to help get more work done faster. 

Looking to optimize these features further for developers specialized in one particular area? With Generative AI support in Vertex AI, enterprises can fine-tune Codey using their own code base. They can consume these customized Codey models directly from Vertex AI today, and later this year, they will be able to connect it to the built-in Duet AI experience. And don’t worry, if you choose to train Codey with your code, your private data is kept private, and not used in the broader foundation model training corpus. You will have transparency and control over where data is stored and how or if it is used. 

• Duet AI for AppSheet will let users create intelligent business applications,  connect their data, and build workflows into Google Workspace via natural language.  With no coding required, users will be able to build apps by describing their needs in a chat guided by AI-powered prompts. This makes app creation accessible to more users, which can allow developer teams to focus their time on other high-impact work.

Create business applications with Duet AI for AppSheet using natural language

Experiment with Duet AI for Google Cloud today

We believe that having an assistant who is constantly evolving by your side will not only reduce an already overwhelmed developer’s workload, but also bring back the excitement of cloud development. With Duet AI, you can navigate the cloud with more confidence, ease, and — dare we say it — fun. 

And this is just the beginning. The future of the cloud experience that we are shaping with Duet AI is full of possibilities. We believe the future of developer productivity is more targeted personalized assistance. Check here to see our vision for Duet AI for Google Cloud – the redefinition of productivity in the workplace through unique end-to-end AI assisted technologies. 

These early features of Duet AI for Google Cloud are available today for limited users and we will be expanding access very soon. Sign up here to join Google Cloud’s AI Trusted Tester Program.