Cloud developers — you’ve got it all. You can code in your choice of languages, enjoy portability with containers, minimize complexity with serverless, and manage the entire software lifecycle by following DevOps principles. But let’s face it, building and onboarding new cloud applications still requires a lot of manual planning, synthesis, and, yes, hard work. You need to research and plan your deployment, create a workable, secure architecture, and of course, you need to write the actual code,

For the last several decades, the cloud has been primarily a “do it yourself” model with volumes of options that have made development more complicated. The cloud went from overwhelmingly exciting to…. a bit overwhelming.

What if we all could bring that excitement back? What if you had some help that was available whenever and wherever you needed it? 

Say hello to Duet AI for Google Cloud

Powered by Google’s state-of-the-art generative-AI foundation models, Duet AI for Google Cloud is an always-on AI collaborator that provides help to users of all skill levels where they need it. With Duet AI, we’re on a mission to deliver a new cloud experience that’s personalized and intent-driven, and can deeply understand your environment to assist you in building secure, scalable applications, while providing expert guidance. 

As we evolve Google Cloud with Duet AI, we are looking to build a cloud platform that is more human-centric, holistic, and helpful, with responsible AI at the center of the experience:

• Human-centric: With Duet AI, we are making Google Cloud more accessible and personal to any type of user at any skill level by providing them with support whenever they need it, from code recommendations for developers, to prompt-based data insights for data engineers, to chat-based app creation for business users.
• Holistic: With generative AI at the center of the cloud experience, cloud development can be more cohesive, with fewer silos across functions, services, and tech stacks, providing a holistic picture in the format you want, wherever you are in Google Cloud.
• Helpful: To deliver smarter, contextual recommendations for building and operating apps with Google Cloud, we pre-trained Codey, one of the foundation models that powers Duet AI, with Google Cloud-specific content like documentation and sample code, and fine-tuned it based on Google Cloud  user behaviors and patterns. 
• Responsible: Our AI Principles set out our commitment to developing technology responsibly. Your code and recommendations will not be reused for any model learning and development. This helps ensure the privacy of your data and code, and also the integrity of the knowledge space from which our AI models are trained.

New capabilities available in Duet AI for Google Cloud

Here are some of the new capabilities available to get us started on our mission to deliver a new personalized and intent-driven cloud experience:

• Code assistance provides AI-driven code assistance for cloud users such as application developers and data engineers. It gives code recommendations as they type in real time, generates full functions and code blocks, and identifies vulnerabilities and errors in the code, while suggesting fixes.

Code assistance auto-generates code for creating a Google Cloud Storage bucket

Code assistance will be available through multiple products and services across Google Cloud, such as in Cloud Workstations, our fully-managed secure development environment, and other code-editing experiences in the Google Cloud Console. Developers will also find code assistance in Cloud Shell Editor or via our Cloud Code IDE extensions for VSCode and JetBrains IDEs. It supports multiple languages including Go, Java, Javascript, Python, and SQL.

• Chat assistance allows people to use simple natural language to get answers on specific development or cloud-related questions. Users can engage with chat assistance to get real-time guidance on various topics, such as how to use certain cloud services or functions, or get detailed implementation plans for their cloud projects. It can also provide architectural or coding best practices, helping to reduce the need to go searching for relevant documents.

Use chat assistance to get the detailed steps for deploying an app on Cloud Run

Chat assistance will also be available across multiple Google Cloud surface areas, for example IDEs, the Cloud Console, and through products and services. Whether you’re a developer, operator, data engineer, or security professional, you’ll be able to leverage chat assistance to help get more work done faster. 

Looking to optimize these features further for developers specialized in one particular area? With Generative AI support in Vertex AI, enterprises can fine-tune Codey using their own code base. They can consume these customized Codey models directly from Vertex AI today, and later this year, they will be able to connect it to the built-in Duet AI experience. And don’t worry, if you choose to train Codey with your code, your private data is kept private, and not used in the broader foundation model training corpus. You will have transparency and control over where data is stored and how or if it is used. 

• Duet AI for AppSheet will let users create intelligent business applications,  connect their data, and build workflows into Google Workspace via natural language.  With no coding required, users will be able to build apps by describing their needs in a chat guided by AI-powered prompts. This makes app creation accessible to more users, which can allow developer teams to focus their time on other high-impact work.

Create business applications with Duet AI for AppSheet using natural language

Experiment with Duet AI for Google Cloud today

We believe that having an assistant who is constantly evolving by your side will not only reduce an already overwhelmed developer’s workload, but also bring back the excitement of cloud development. With Duet AI, you can navigate the cloud with more confidence, ease, and — dare we say it — fun. 

And this is just the beginning. The future of the cloud experience that we are shaping with Duet AI is full of possibilities. We believe the future of developer productivity is more targeted personalized assistance. Check here to see our vision for Duet AI for Google Cloud – the redefinition of productivity in the workplace through unique end-to-end AI assisted technologies. 

These early features of Duet AI for Google Cloud are available today for limited users and we will be expanding access very soon. Sign up here to join Google Cloud’s AI Trusted Tester Program.

In the United Kingdom, the popularity of online grocery shopping is expected to surge from about 6% of the market today to 9% by 2021, according to market research firm Mintel. One of the pioneers of online-only grocery retailing is Ocado, based in Hatfield, Hertfordshire in the U.K. Since starting commercial deliveries in 2002, the company has grown to 600,000 active customers, 260,000 weekly orders, and £1.39 billion in annual revenue.

Ocado takes supermarket trips out of the equation by enabling shoppers to purchase items online through its convenient web and mobile applications. Items are then picked and packed in automated warehouses and shipped directly to customers in a one-hour time slot of their choosing. Ocado’s delivery punctuality is 95%, order accuracy is 99%, and its service footprint now reaches more than 70% of the U.K. population.

“Google Cloud Platform gives us the flexibility and performance to tackle the large and complex data challenges unique to our business.”

—Paul Clarke, Chief Technology Officer, Ocado

The company achieved its success by building in-house almost all the technology and automation that powers its end-to-end e-commerce, fulfillment, and logistics platform. Ocado also developed a new platform, the Ocado Smart Platform (OSP), which offers large brick-and-mortar grocery retailers around the world access to a best-in-class solution for online grocery.

Democratizing machine learning

The shopping journey for online grocery retailing differs significantly from other e-businesses. Customers often buy dozens of products at once, a single household may have multiple buyers using multiple devices, and product shelf life may only be a couple of days.

“We often say that having built an end-to-end platform that can do online grocery scalably and profitably, we can do other forms of online retail; but the reverse does not necessarily follow,” says Paul Clarke, Chief Technology Officer at Ocado. “Google Cloud Platform gives us the flexibility and performance to tackle the large and complex data challenges unique to our business.”

The Ocado business model takes advantage of consumers’ shifting preferences and the links between digital technology and shopping experiences.

“Google Cloud Machine Learning Engine gives us the agility we need. Our developers were able to try out TensorFlow and see firsthand the benefits of machine learning in the cloud.”

—Paul Clarke, Chief Technology Officer, Ocado

The company has been building machine learning into its systems for over five years. Until recently, Ocado machine learning applications required specialist data scientists, typically with PhDs in machine learning, who would build these solutions from the ground up. It also required the specialist who set up the system and costly on-premises infrastructure to train and run these systems.

However, working with Google as a private alpha testing site for Google Cloud Machine Learning Engine accelerated its adoption of artificial intelligence (AI).

“We’ve been talking about how the cloud could democratize AI for some time,” says Paul. “Google Cloud Machine Learning Engine gives us the agility we need. Our developers were able to try out TensorFlow and see firsthand the benefits of machine learning in the cloud.”

TensorFlow is an open source software library for machine learning developed by the Google Brain team. Ocado developers, engineers, and data scientists now use TensorFlow for many of their machine learning projects. They deploy the models they build on Google Cloud Machine Learning Engine, which lets them train models faster across servers, desktop computers, and mobile devices through a single application program interface (API). Additionally, Google Cloud Machine Learning Engine integrates easily with the other Google Cloud Platform products used widely at Ocado.

What do customers really want?

One of the first TensorFlow models Ocado created was a machine learning algorithm that tags and categorizes customer emails and then prioritizes them for response.

The contact center receives thousands of emails each day and Ocado wanted to automate determining which ones needed to be answered immediately and which ones could wait.

For example, a first-time customer expressing their delight in using Ocado doesn’t need to be responded to with the same urgency as a customer who is missing an item from their order or who won’t be home to receive the delivery.

“Enabling agents to respond without having to sort through less-urgent emails improves Ocado’s responsiveness and customer service.”

—James Donkin, General Manager, Ocado

“We get a lot of emails from customers saying, ‘Our service was great,’ or ‘The driver was very courteous,'” says James Donkin, General Manager, Ocado. “But when issues like weather or road conditions potentially affect delivery, we often get surges of urgent questions. Enabling agents to respond without having to sort through less-urgent emails improves Ocado’s responsiveness and customer service.”

Using Google Cloud Machine Learning Engine, TensorFlow, and a large data set culled from several years’ worth of manually categorized customer emails, Ocado experimented on which kind of neural network architecture would best prioritize emails. After testing its models, Ocado implemented the highest-performing one and has been able to respond to urgent messages four times faster. The company also discovered that 7% of its emails don’t require a response at all, which means call center representatives now have more time to devote to higher priority messages.

“Without Google Cloud Machine Learning Engine, it would have been a lot harder to succeed on a project like email classification,” says Roland Plaszowski, who has recently managed several big data projects and initiatives at Ocado.

“Even if we invested significantly in infrastructure, it would be difficult to manage because of the computational intensity. It’s challenging and expensive to run machine learning projects at the same time without infrastructure that you can scale easily.”

Ocado also uses machine learning to predict customer behavior and improve experiences. By analyzing order data, Ocado makes shopping as frictionless as possible. For example, the ordering system can pre-populate customers’ shopping carts with items they are most likely to purchase, remind customers about items they may have forgotten, and notify them of multi-buy offers they haven’t completed, for example, only buying one of a buy one, get one free offer. Based on machine learning from previous purchase data, the Ocado system can also offer new products that are likely to delight customers.

“You will regularly see items that are more personally relevant to you instead of items that are being promoted more generally,” says James. “I’m a vegetarian, so I’m offered specials for vegetarian products that I normally buy and new ones that I’ve never bought. I’m also less likely to see things that I’m not interested in.”

Machines and machine learning

Within the Internet of Things (IoT), Ocado is looking to enhance its warehouse robots with machine learning. An integral part of the OSP, thousands of robots continually stream data into Google Cloud Storage and Google BigQuery.

Ocado data scientists apply machine learning to create a type of swarm intelligence that enables warehouse robots to work cooperatively to achieve a common goal. Projects include modules to search robot telemetry data, such as whether a battery pack is operating within standard tolerances or whether firmware has been successfully loaded, and use it to optimize maintenance schedules or detect patterns in wear and tear.

“Another challenge we’re looking at is how to embed machine learning directly into robots so they become smarter in terms of self-testing, exception handling, and error recovery,” says Paul. “This is a challenging combination of IoT, data analytics, and machine learning that we believe Google BigQuery and Google Cloud Machine Learning are particularly well suited to helping Ocado achieve.”

Scaling for new business

Scalability is also a major reason behind some of Ocado’s cloud initiatives, including the migration of all its on-premises data to the cloud. Ocado wanted to improve customer experiences, empower business teams with greater insight, and reduce IT overhead, so it consolidated onto Google Cloud Platform.

“The old databases just weren’t fast enough,” says Paul. “We needed a solution that could scale with the amount of data we generate and how we use it. Google Cloud Storage and Google BigQuery now provide the backbone, from a data point of view, for the Ocado Smart Platform.”

Ocado estimates its business, product, and transaction data is approaching two petabytes. Combining customer and supply chain data helps both internal Ocado operations and the company’s ambitions to commercialize OSP.

“When compared with other options for expansion internationally, selling OSP as a managed service lets us turn companies that could have been competitors into customers,” says Paul. “We want to build OSP once and then turn it on for multiple business-to-business customers.”

Each time Ocado adds a new hosting customer to OSP, it will launch a customized instance to fit that customer’s requirements. The capacity and performance of each new OSP instance must be able to scale quickly as the backend platform for established retailers with large numbers of products, customers, and transactions.

Ocado’s first OSP customer, Morrisons, is already benefiting from this first-of-a kind solution. Morrisons is one of the UK’s four largest supermarkets and uses OSP to power its online retail business. Using Google Cloud Platform, Ocado has stored, processed, and analyzed terabytes of Morrisons’ data using a dedicated data lake and Google BigQuery.

In addition to using Google Cloud Platform for OSP, Ocado also adopted it for its own online grocery retail business operation. Ocado originally used the Apache Spark and Apache Hadoop open-source frameworks on Google Compute Engine for its data platform. Moving to Google BigQuery frees Ocado business analysts from the complex query setup and workflows associated with Spark and Hadoop. Plus, it lets Ocado share data analytics with suppliers and partners.

Google BigQuery is well integrated with TensorFlow on Google Cloud Machine Learning Engine and Google Cloud Dataproc, the Apache Spark and Apache Hadoop service that lets Ocado use open source data tools for batch processing, querying, streaming, and machine learning. Google Cloud Dataflow and Google Cloud Dataproc handle cluster management, and provide an easy-to-use framework so developers can spend less time and money on administration and more time on delivering valuable business features.

Switching from Hadoop to Google BigQuery revealed a series of cost and performance improvements. For example, Ocado no longer needed to decide how many instances to bring up in a cluster or wait for the instances to spin up. Google handled everything.

“We simply ran our queries and paid for the resources that we use,” adds Roland. “One big win with Google BigQuery is we don’t have to do maintenance. Best of all, we saw Google BigQuery outperform our Hadoop cluster by over 80 times on our largest dataset, and for only two-thirds the cost.”

At Google Cloud, our customers are at the forefront of digital transformation—launching entirely new businesses and products built in the cloud, redefining entire industries with data and artificial intelligence, delivering innovative new consumer experiences, or committing to sustainable new ways of doing business.

We’re committed to our customers’ success, and over the past two years we’ve invested significantly in providing ongoing and integrated support through our customer care portfolio, including launching new Premium and Mission Critical Support offerings that allow us to monitor, prevent, and mitigate impacts quickly, while delivering the fastest response times in the industry.

Today, I’m proud to unveil several new resources and offerings for our customers, including a new Custom AI Solutions practice, new Global Delivery Centers, expanded Executive Briefing Centers, and new leaders to help continue to drive our organization forward.

Helping businesses innovate with a new Custom AI Solutions practice

We are investing in services and offerings to help our customers innovate and drive innovative change to business processes, culture, and customer experiences with Google Cloud products and services. Artificial intelligence (AI) and machine learning (ML) technologies are foundational for many such digital transformations, and to help customers create real business value with these technologies, we’re excited to launch a Custom AI offering to address our customers’ most critical innovation needs. 

AI and ML technologies are foundational to digital transformations, yet they are not “one size fits all.” Each customers’ problems and opportunities are unique; a rideshare company will use AI differently than a brick and mortar retailer, or a healthcare company, or an insurance firm. To help organizations deploy AI and ML more effectively, we’re launching a new Custom AI Solutions practice, offering customers custom-built AI and ML solutions built into Vertex AI; access to Google’s engineering expertise; and predictable, subscription-based pricing. 

Our teams are already partnering closely with early customers to build and deploy custom AI solutions. For instance, USAA, the large North American insurer, is using Google Cloud ML to process near-real-time damage estimates based on digital images to create a streamlined operations experience.

Learn more about our Custom AI Solutions practice offering here, and how we work together with our customers here. To get in touch, please contact our sales team.

Launching new Global Delivery Centers

Our drive to digitally transform our customers’ businesses is often manifested through our Global Delivery Centers, which expand our professional consulting and emerging practices capabilities available to customers and partners around the world.

The teams at our Global Delivery Centers help customers get up-and-running on Google Cloud quickly and cost effectively, and consult with customers to rapidly build capacity in areas like data analytics, hybrid and multi-cloud, artificial intelligence, and machine learning. More importantly, they help customers successfully execute projects in support of their most mission-critical business objectives. Critically, these centers also help our global partner ecosystem quickly ramp their Google Cloud practices, and get fast, expert consultation for their customers too.

In 2022, we aim to triple the size of our Global Delivery Center teams in Argentina, Poland, and India. In addition, we’ll invest in building deep Google Cloud talent in Mexico and Portugal, furthering our commitment to the industry’s best expert consultation for customers and partners around the world.

Expanding our Executive Briefing Centers footprint

In addition to our Global Delivery Centers, we’re also pleased to expand our resources and facilities that enable digital and face-to-face meetings and working sessions with our customers. This year, we will launch four new Executive Briefing Centers, located on Google Cloud campuses in London, Paris, Singapore, and Munich. 

These centers provide an opportunity to listen to our customers, share the best of Google Cloud’s solutions, and inspire digital transformation, in conversations facilitated by Google Cloud leadership, engineers, and industry experts. By bringing this experience to our customers in-region we can foster deeper partnerships and develop cloud solutions that meet their requirements for security, privacy, and digital sovereignty without compromising on functionality or innovation. 

Adding new leadership to enable customer success

Finally, I’m excited to welcome two new leaders to Google Cloud on the Customer Experience team, who will help scale our Delivery Centers and deliver exceptional experiences for our customers.

Heading our Global Delivery Center experience is Sunil Rao. Sunil comes to Google Cloud from Accenture, where he spent 18-plus years working with large technology customers across many industry verticals and managed large global teams in Accenture Advanced Technology Center. Sunil will also lead our Technical Onboarding Center, which helps businesses around the world get up to speed with technologies that are critical to understanding customer and business process contexts, and delivering great experiences.

Additionally, Lee Moore is joining Google Cloud to lead Customer Experience in North America. Lee spent nearly 30 years at Accenture in various leadership positions, including services integration for complex problem-solving, product development across a number of industry verticals, and building long-term customer relationships.

You’ll be hearing much more from us in the coming months, as we build out even more powerful and effective cloud-based services and offerings, work with customers to deliver new analytics- and AI-based tools and services, and work with our growing list of partners to help ensure customer success, across the globe.

Large Machine Learning (ML) models – such as large language models, generative AI, and vision models – are dramatically increasing the number of trainable parameters and are achieving state-of-the-art results. Increasing the number of parameters results in the model being too large to fit on a single VM instance thus demands distributed compute to spread the model across multiple nodes. Google Kubernetes Engine (GKE) has built-in support for NCCL Fast Socket, to help improve the time to train large ML models with distributed, multi-node clusters.

Enterprises are looking for faster and cheaper performance to train their ML models. With distributed training, communicating gradients across nodes is a performance bottleneck. Optimizing inter-node latency is critical to reduce training time and costs. Distributed training uses collective communication as a transport layer over the network between the multiple hosts. Collective communication primitives such as all-gather, all-reduce, broadcast, reduce, reduce-scatter, and point-to-point send and receive are used in distributed training in Machine Learning. 

The NVIDIA Collective Communication Library (NCCL) is commonly used by popular ML frameworks such as TensorFlow and PyTorch. It is a highly optimized implementation for high bandwidth and low latency between NVIDIA GPUs. Google developed a proprietary version of NCCL called NCCL Fast Socket to optimize performance for deep learning on Google Cloud.

NCCL Fast Socket uses a number of techniques to achieve better and more consistent NCCL performance.

• Use of multiple network flows to attain maximum throughput. NCCL Fast Socket introduces additional optimizations over NCCL’s built-in multi-stream support, including better overlapping of multiple communication requests.
• Dynamic load balancing of multiple network flows. NCCL can adapt to changing network and host conditions. With this optimization, straggler network flows will not significantly slow down the entire NCCL collective operation.
• Integration with Google Cloud’s Andromeda virtual network stack.This increases overall network throughput by avoiding contentions in virtual machines (VMs).

We tested (NVIDIA NCCL tests) the performance of NCCL Fast Socket vs NCCL on various machine shapes with 2 node GKE clusters.

The following chart shows the results. For each machine shape, the NCCL performance without Fast Socket is normalized to 1. In each case, using NCCL Fast Socket demonstrated increased performance in a range of 1.3 to 2.6 times faster internetwork communication speed.

As a built-in feature, GKE users can take advantage of NCCL Fast Socket without changing or recompiling their applications, ML frameworks (such as TensorFlow or PyTorch), or even the NCCL library itself. To start using NCCL Fast Socket, create a node pool that uses the plugin with the --enable-fast-socket and --enable-gvnic flags. You can also update an existing node pool using gcloud container node-pools update.

gcloud container node-pools create NODEPOOL_NAME \
    --accelerator type=ACCELERATOR_TYPE, count=ACCELERATOR_COUNT \
    --machine-type=MACHINE_TYPE \
    --cluster=CLUSTER_NAME \
    --enable-fast-socket \
    --enable-gvnic

To achieve better network throughput with NCCL, Google Virtual NICs (gVNICs) must be enabled when creating VM instances. For detailed instructions on how to use gVNICs, please refer to the gVNIC guide. 

To verify that NCCL Fast Socket has been enabled, view the kube-system pods:

kubectl get pods -n kube-system

And the output should b similar to:

NAME                         READY   STATUS    RESTARTS   AGE
fast-socket-installer-qvfdw  2/2     Running   0          10m
fast-socket-installer-rtjs4  2/2     Running   0          10m
fast-socket-installer-tm294  2/2     Running   0          10m

To learn more visit GKE NCCL Fast Socket documentation. We look forward to hearing how NCCL Fast Socket improves your ML Training experience on GKE.