Large Machine Learning (ML) models – such as large language models, generative AI, and vision models – are dramatically increasing the number of trainable parameters and are achieving state-of-the-art results. Increasing the number of parameters results in the model being too large to fit on a single VM instance thus demands distributed compute to spread the model across multiple nodes. Google Kubernetes Engine (GKE) has built-in support for NCCL Fast Socket, to help improve the time to train large ML models with distributed, multi-node clusters.

Enterprises are looking for faster and cheaper performance to train their ML models. With distributed training, communicating gradients across nodes is a performance bottleneck. Optimizing inter-node latency is critical to reduce training time and costs. Distributed training uses collective communication as a transport layer over the network between the multiple hosts. Collective communication primitives such as all-gather, all-reduce, broadcast, reduce, reduce-scatter, and point-to-point send and receive are used in distributed training in Machine Learning. 

The NVIDIA Collective Communication Library (NCCL) is commonly used by popular ML frameworks such as TensorFlow and PyTorch. It is a highly optimized implementation for high bandwidth and low latency between NVIDIA GPUs. Google developed a proprietary version of NCCL called NCCL Fast Socket to optimize performance for deep learning on Google Cloud.

NCCL Fast Socket uses a number of techniques to achieve better and more consistent NCCL performance.

• Use of multiple network flows to attain maximum throughput. NCCL Fast Socket introduces additional optimizations over NCCL’s built-in multi-stream support, including better overlapping of multiple communication requests.
• Dynamic load balancing of multiple network flows. NCCL can adapt to changing network and host conditions. With this optimization, straggler network flows will not significantly slow down the entire NCCL collective operation.
• Integration with Google Cloud’s Andromeda virtual network stack.This increases overall network throughput by avoiding contentions in virtual machines (VMs).

We tested (NVIDIA NCCL tests) the performance of NCCL Fast Socket vs NCCL on various machine shapes with 2 node GKE clusters.

The following chart shows the results. For each machine shape, the NCCL performance without Fast Socket is normalized to 1. In each case, using NCCL Fast Socket demonstrated increased performance in a range of 1.3 to 2.6 times faster internetwork communication speed.

As a built-in feature, GKE users can take advantage of NCCL Fast Socket without changing or recompiling their applications, ML frameworks (such as TensorFlow or PyTorch), or even the NCCL library itself. To start using NCCL Fast Socket, create a node pool that uses the plugin with the --enable-fast-socket and --enable-gvnic flags. You can also update an existing node pool using gcloud container node-pools update.

gcloud container node-pools create NODEPOOL_NAME \
    --accelerator type=ACCELERATOR_TYPE, count=ACCELERATOR_COUNT \
    --machine-type=MACHINE_TYPE \
    --cluster=CLUSTER_NAME \
    --enable-fast-socket \
    --enable-gvnic

To achieve better network throughput with NCCL, Google Virtual NICs (gVNICs) must be enabled when creating VM instances. For detailed instructions on how to use gVNICs, please refer to the gVNIC guide. 

To verify that NCCL Fast Socket has been enabled, view the kube-system pods:

kubectl get pods -n kube-system

And the output should b similar to:

NAME                         READY   STATUS    RESTARTS   AGE
fast-socket-installer-qvfdw  2/2     Running   0          10m
fast-socket-installer-rtjs4  2/2     Running   0          10m
fast-socket-installer-tm294  2/2     Running   0          10m

To learn more visit GKE NCCL Fast Socket documentation. We look forward to hearing how NCCL Fast Socket improves your ML Training experience on GKE.

From the newsroom to the boardroom, everywhere we turn these days the topic of conversation is artificial intelligence (AI). From the smallest startups to the largest enterprises, every business is looking for ways to incorporate generative AI technology into their products or services. 

Generative AI is a new breed, able to not only discern patterns in data but generalize from them and create new data—and it’s moving fast. With new technologies rapidly emerging, there’s a lot to consider when choosing a tech stack that’s right for your startup. 

Our mission is to empower startups to build generative AI applications quickly, efficiently, and responsibly. In addition to our technology, we offer a variety of fresh educational and consulting initiatives, as well as comprehensive plans tailored to specific industry applications. 

At the recent Google Cloud Startup Summit, leaders from One AI and MongoDB weighed in on various decisions startups face when choosing a development platform to integrate AI into their products. (You can watch the full conversation here.) They shared key insights into the challenges startups face when applying AI technology to real-world business and product use cases. 

In this blog, we’ll explore why One AI – a platform that empowers businesses to deploy tailored AI solutions – relies on MongoDB on Google Cloud for performance, scale, functionality, and TCO. We will also cover the things that startups should keep in mind when building their generative AI stack.

Fine-tuning generative AI for startups

First, let’s start with a little background. 

AI-based capabilities have been around for decades now, but generative AI is distinct. It’s a more mature version of AI, powered by models pre-trained on very large datasets composed of massive quantities of images, text, and data. These models are known as foundation models, and they include large language models (LLMs), text-to-image models, multimodal models, and more. Foundation models let intelligent applications  generate new images, text, and data based on queries or prompts. This in turn allows companies to deploy those capabilities in their products and services faster since they don’t have to retrain the whole AI from scratch. 

Over the course of their diverse startup careers, Amit Ben, CEO at One AI and his team have built AI-based capabilities from the ground up for various products in various fields.

“And each time, we had to rebuild the tech stack over again,” Amit explains. “With the advent of generative AI, startups now have the ability to deploy much faster — with a lower TCO and higher confidence — and deliver the capabilities they need into their products and services. It finally makes sense for every company to have AI in its product portfolio.”

“For us to be able to focus on that,” Amit adds, “we need to make sure we have a rock-solid foundation that we can build on.”

That foundation is MongoDB on Google Cloud. 

Helping startups build fast and with flexibility

Startups can scale from ideation to growth with Google Cloud’s global availability, market-leading sustainability, and the same zero-trust security model that Google itself depends on.

With MongoDB, startups can take advantage of iteration cycles that are 3-5X faster, reduce sprawl and complexity, and benefit from the scalable infrastructure and advanced analytics tools on Google Cloud. 

With MongoDB on Google Cloud, Amit and his team are confident they can adapt to new schemas, to new data, and to the scale they need for both writing and reading, all while operating on a scalable platform and infrastructure they can rely on for the long haul. 

A common mistake for startups is turning to niche, single-point solutions. But this can backfire when they realize their solution doesn’t provide the security, scalability, and performance they need to grow. 

Additionally, startups tend to have tight iteration cycles as they find their ideal product market fit. The ability to build fast and with improved flexibility is a key differentiator in a startup environment. 

From cutting-edge automation to rock-solid redundancy and performance, there are many reasons why startups choose MongoDB on Google Cloud. And now, a dedicated partnership helps startups like One AI scale more quickly, more securely, and more successfully. 

Google Cloud and MongoDB for startups

Choosing the right technology to accelerate time to market is critical to a startup’s success. Not only is it easy to get started, but Google Cloud and MongoDB also provide the foundation for users to scale without limits — so startups can focus on innovating and growing their businesses. 

Learn more about the powerful startup programs available from Google Cloud and MongoDB.