There are a number of terms and concepts in cloud computing, and not everyone is familiar with all of them. To help, we’ve put together a list of common questions, and the meanings of a few of those acronyms. You can find all these, and many more, in our learning resources.

What are containers?

Containers are packages of software that contain all of the necessary elements to run in any environment. In this way, containers virtualize the operating system and run anywhere, from a private data center to the public cloud or even on a developer’s personal laptop. Containerization allows development teams to move fast, deploy software efficiently, and operate at an unprecedented scale. Read more.

Containers vs. VMs: What’s the difference?

You might already be familiar with VMs: a guest operating system such as Linux or Windows runs on top of a host operating system with access to the underlying hardware. Containers are often compared to virtual machines (VMs). Like virtual machines, containers allow you to package your application together with libraries and other dependencies, providing isolated environments for running your software services. However, the similarities end here as containers offer a far more lightweight unit for developers and IT Ops teams to work with, carrying a myriad of benefits. Containers are much more lightweight than VMs, virtualize at the OS level while VMs virtualize at the hardware level, and share the OS kernel and use a fraction of the memory VMs require. Read more.

What is Kubernetes?

With the widespread adoption of containers among organizations, Kubernetes, the container-centric management software, has become the de facto standard to deploy and operate containerized applications. Google Cloud is the birthplace of Kubernetes—originally developed at Google and released as open source in 2014. Kubernetes builds on 15 years of running Google’s containerized workloads and the valuable contributions from the open source community. Inspired by Google’s internal cluster management system, Borg, Kubernetes makes everything associated with deploying and managing your application easier. Providing automated container orchestration, Kubernetes improves your reliability and reduces the time and resources attributed to daily operations. Read more.

What is microservices architecture?

Microservices architecture (often shortened to microservices) refers to an architectural style for developing applications. Microservices allow a large application to be separated into smaller independent parts, with each part having its own realm of responsibility. To serve a single user request, a microservices-based application can call on many internal microservices to compose its response. Containers are a well-suited microservices architecture example, since they let you focus on developing the services without worrying about the dependencies. Modern cloud-native applications are usually built as microservices using containers. Read more.

What is ETL?

ETL stands for extract, transform, and load and is a traditionally accepted way for organizations to combine data from multiple systems into a single database, data store, data warehouse, or data lake. ETL can be used to store legacy data, or—as is more typical today—aggregate data to analyze and drive business decisions. Organizations have been using ETL for decades. But what’s new is that both the sources of data, as well as the target databases, are now moving to the cloud. Additionally, we’re seeing the emergence of streaming ETL pipelines, which are now unified alongside batch pipelines—that is, pipelines handling continuous streams of data in real time versus data handled in aggregate batches. Some enterprises run continuous streaming processes with batch backfill or reprocessing pipelines woven into the mix. Read more.

What is a data lake?

A data lake is a centralized repository designed to store, process, and secure large amounts of structured, semistructured, and unstructured data. It can store data in its native format and process any variety of it, ignoring size limits. Read more.

What is a data warehouse?

Data-driven companies require robust solutions for managing and analyzing large quantities of data across their organizations. These systems must be scalable, reliable, and secure enough for regulated industries, as well as flexible enough to support a wide variety of data types and use cases. The requirements go way beyond the capabilities of any traditional database. That’s where the data warehouse comes in. A data warehouse is an enterprise system used for the analysis and reporting of structured and semi-structured data from multiple sources, such as point-of-sale transactions, marketing automation, customer relationship management, and more. A data warehouse is suited for ad hoc analysis as well custom reporting and can store both current and historical data in one place. It is designed to give a long-range view of data over time, making it a primary component of business intelligence. Read more.

What is streaming analytics?

Streaming analytics is the processing and analyzing of data records continuously rather than in batches. Generally, streaming analytics is useful for the types of data sources that send data in small sizes (often in kilobytes) in a continuous flow as the data is generated. Read more.

What is machine learning (ML)?

Today’s enterprises are bombarded with data. To drive better business decisions, they have to make sense of it. But the sheer volume coupled with complexity makes data difficult to analyze using traditional tools. Building, testing, iterating, and deploying analytical models for identifying patterns and insights in data eats up employees’ time. Then after being deployed, such models also have to be monitored and continually adjusted as the market situation or the data itself changes. Machine learning is the solution. Machine learning allows businesses to enable the data to teach the system how to solve the problem at hand with machine learning algorithms—and how to get better over time. Read more.

What is natural language processing (NLP)?

Natural language processing (NLP) uses machine learning to reveal the structure and meaning of text. With natural language processing applications, organizations can analyze text and extract information about people, places, and events to better understand social media sentiment and customer conversations. Read more.

Learn more

This is just a sampling of frequently asked questions about cloud computing. To learn more, visit our resources page at cloud.google.com/learn.

While the shift to digital business and the cloud has been well under way for some years now, organizations today have a new sense of urgency due to COVID-19. Delivering digital transformation is no longer a ‘nice to have’ option, rather, it is an operational imperative. Taking advantage of the infrastructure, platform and solution gains that cloud and microservices architecture provide is a must for brands today. 

At Google Cloud, we understand the pressures and challenges organizations of all sizes, across all industries are facing. The pandemic has dramatically impacted global commerce at-large, exposing (for many organizations across multiple sectors) gaps in omnichannel capabilities, business continuity and forecasting plans, not to mention spots in supply chain agility, resilience and responsiveness. 

A rapidly evolving consumer-driven commerce landscape has put innovation squarely in the spotlight for supply chain teams all over the world, with the effects of the global pandemic making it increasingly difficult for manufacturers, wholesalers, third party logistics providers and retailers (in particular) to weather the perfect storm of fast-moving consumer trends and a need for ‘always on’ digital innovation. 

These same effects have driven increasing interest and uptake of technology like the Manhattan Active® suite of solutions, as well as our own cloud platform; both of which afford organizations the levels of agility, flexibility and scalability needed to insulate their people, processes and long-term business strategies against unforeseen future obstacles such as global pandemics or international trade disputes.

An excellent example of this agility, flexibility and scalability in action is PVH’s response to the global pandemic. One of the most admired fashion and lifestyle companies with such iconic brands as Calvin Klein, TOMMY HILFIGER, Van Heusen, and IZOD, PVH was forced to temporarily close its physical stores and, as a result, experienced a sudden massive increase in online sales. The retailer was able to quickly pivot by adjusting its business rules in Manhattan Distributed Order Management (part of Manhattan Active Omni) to expose store inventory to online consumers and reroute its fulfillment processes. Thanks to Manhattan’s solution delivered through Google Cloud, in a matter of days, PVH was able to leverage both its distribution centers and vast store network to fulfill its online orders.

“The events of 2020 have accelerated retail and ecommerce operations forward,” said David Herridge, executive vice president of Global Value Chain Technologies for PVH. “With quick, creative thinking and the right partner, we were able to pivot operations, satisfy our customers and prepare for the future.”

Manhattan’s products have been recognized for their ability to solve real-world challenges through innovation, and used by many of the world’s top brands to solve some of their most complex commerce and supply chain challenges: the latest recognition is Manhattan’s position as sole leader in the 2021 Forrester Wave™ for Order Management Solutions. 

Since December 2018, Google Cloud has been collaborating closely with the team at Manhattan and its ‘always on’, versionless approach to innovation. And, during the last two and a half years, Manhattan has significantly accelerated its cloud-first solutions and market adoption, resulting in tremendous growth in its overall cloud business efforts. 

By building cloud native solutions on Google Cloud, the teams at Manhattan continue to deliver the high-performance, elastic, high-redundancy, secure solutions their customers rely on. Moreover, it means both Google Cloud and Manhattan continue to innovate and push the boundaries of what is possible in terms of the supply chain and omnichannel innovations that underpin global commerce – innovation that is needed more now than maybe ever before.

Our commitment to distributed cloud solutions and ongoing innovation, not to mention the fact Google Cloud operates a net carbon-neutral cloud, means that the working partnership between both industry leading teams continues to be a perfect match of brand values; not just from a technology perspective, but also a long-term sustainability and environmental one too.

More information on the partnership can be found here.

In recent years, machine learning (ML) has gained tremendous popularity as a powerful tool for solving complex problems across various domains. However, building and deploying ML models at scale can be challenging, as it involves a range of tasks such as data preparation, feature engineering, model training, deployment, monitoring, maintenance and so on. 

According to “The Art of AI maturity” report published by Accenture “87% of data science projects never make it into production.” This is where MLOps comes in – it can help to address the core challenges by providing a framework for managing the entire ML lifecycle, from data collection and preparation to model development, testing, and deployment. It also reduces the time from ML model development to production and increases the success rate of ML projects.

In Google Cloud, we understand how important MLOps is to successfully productionize ML models. So we collaborated with Equinix, the world’s digital infrastructure company™ and  a leader in global colocation data center market share, with 248 data centers in 27 countries on five continents. We helped them by providing the advisory services on the MLOps best practices and architecture. 

Let’s take a sneak peek at the MLOps requirements at Equinix and the final architecture that was proposed.

What does MLOps mean for Equinix?

After multiple discovery sessions with the Equinix Team, we identified the core requirements and pain points to address in their new MLOps architecture:

• Reusability: Components such as features and pipeline components should be reused across projects to reduce costs and improve efficiency.
• Foundations: The foundations of the infrastructure, such as environments, folder structure, and project hierarchy, should be well-designed to support scalability and reliability.
• Early identification of problems: Problems should be identified early by including data validation, notifications, and retry mechanisms.
• Cost optimization: Costs should be optimized by paying only for what is used.
• Enterprise CI/CD requirements: Enterprise CI/CD requirements should be met by integrating with GitHub and GitActions.
• Scaling: The infrastructure must scale to support future growth.
• Security: Enterprise security requirements should be met in terms of IAM roles, network, etc.

MLOps Architecture

Based on the above requirements from Equinix, key design considerations were made for example – using Vertex AI Feature Store instead of Big Query for online feature serving, using DataFlow for pre-processing vs using the existing python based pre-processing and so on. After carefully assessing all the alternatives, the below reference architecture for MLOps in GCP was proposed:

Reference architecture for MLOPs using GCP is illustrated in Figure 1. This architecture includes the following pipeline stages:

• Vertex AI Workbench enables data scientists to quickly explore new ideas and develop/experiment new models. The source code is saved in GitHub repository
• Github Actions with self-hosted runners are integrated with Github as the source code repository. This enables continuous integration of code, quality and security scans. The artifacts generated during the process are saved in Artifact Registry and Google Cloud Storage. These artifacts are deployed to implement the pipeline.
• Unit tests and integration tests can be performed during the continuous integration in Github Actions with self-hosted runners. End-to-end tests are performed on demand in the continuous integration pipeline. 
• Metadata about the artifacts is generated and saved in Vertex ML Metadata.
• Automated triggers can be enabled to run pipelines. For example, one trigger is the availability of new training data. These triggers can run the model training pipeline and the new trained model can be pushed to Vertex AI Model Registry.
  • To train a new ML model with new data, the deployed Vertex AI Pipeline is executed. 
  • To train a new ML model with new implementation, a new pipeline will be deployed through CI/CD pipeline.

“The proposed architecture design covers the requirements and scenarios that we were looking for. As our AI and ML portfolio is growing in scale and complexity, it’s important to follow a clear and up-to-date architecture if we want to keep increasing the value delivered by our solutions. As part of the process, the team also acquired the skills required to fully implement it” according to Bernardo Fernandes, Data Science Senior Manager at Equinix.

Below is the snapshot of the features before and after MLOps implementation at Equinix:

As businesses increasingly rely on machine learning to gain a competitive edge, MLOps has become a critical component of their strategy. By adopting MLOps practices, organizations can achieve faster time-to-market, better performance, and higher ROI for their machine learning initiatives.

Fast track end-to-end deployment with Google Cloud AI Services (AIS)

The partnership between Google Cloud and Equinix is just one of the latest examples of how we’re providing AI-powered solutions to solve complex problems to help organizations drive the desired outcomes. To learn more about Google Cloud’s AI services, visit our AI & ML Products page.

^{We’d like to give special thanks to Nitin Aggarwal, Vijay Surampudi, Parag Mhatre and Anantha Narayanan Krishnamurthy for their support and guidance throughout the project. We are also grateful to the super awesome collaboration with the Equinix Team (Ravi Pasula and Brendan Coffey, Bernardo Fernandes, Łukasz Murawski, Jakub Michałowski, Sonia Przygocka-Groszyk, Marek Opechowski, Daria Bondara, Nila Velu, Vijay Narayanan, Dharmendra Kumar, Shailesh Sukare, Arunraj Kumar Raje, Seng Cheong Lee).}