Whether with the cloud or within their own data centers, enterprises have undergone a period of remarkable consolidation and centralization of their compute resources. But with the rise of ever more powerful mobile devices, and increasingly capable cellular networks, application architects are starting to think beyond the confines of the data center, and looking out to the edge. 

What exactly do we mean by edge? Think of the edge as distributed compute happening on a wide variety of non-traditional devices — mobile phones of course, but also equipment sensors in factories, industrial equipment, or even temperature and reaction monitoring in a remote lab. Edge devices are also connected devices, and can communicate back to the mothership over wireless or cellular networks. 

Equipped with increasingly powerful processors, these edge devices are being called upon to perform tasks that have thus far been outside the scope of traditional IT. For enterprises, this could mean pre-processing incoming telemetry in a vehicle, collecting video in kiosks at a mall, gathering quality control data with cameras in a warehouse, or delivering interactive media to retail stores. Enterprises are also relying on edge to ingest data from outposts or devices that have even more intermittent connectivity, e.g., oil rigs or farm equipment, filtering that data to improve quality, reducing it to right-size information load, and processing it in the cloud. New data and models are then pushed back to the edge; in addition, we can also push configuration, software, and media updates and decentralize processing workload.

Edge isn’t all about enabling new use cases – it’s also about right-sizing environments and improving resource utilization. For example, adopting an edge model can also relieve load on existing data centers. 

But while edge computing is full of promise for enterprises, there are many pieces that are still works in progress. Further, developing edge workloads is very different from developing traditional applications, which enjoy the benefits of persistent data connections and run on well-resourced hardware platforms. As such, cloud architects are still in the early days of figuring out how to use and implement edge for their organizations. 

Fortunately, there are tools you can use to help ease the transition to edge computing — and that likely fit into your organization’s existing computing systems. Kubernetes, of course, but also higher level management tools like Anthos, which provides a consistent control plane across cloud, private data center and edge locations. Other parts of the Anthos family – Anthos Config Management and Anthos Service Mesh — go one step further and provide consistent, centralized management to your edge and cloud deployments. And there’s more to come.

For the remainder of this blog post, we’ll dive deeper into the past and current state of edge computing, and the benefits that architects and developers can expect to see from edge computing. In a next post, we’ll take a deeper look at some of the challenges that designing for edge introduces, and some of the advantages the average enterprise has in adopting the edge model. Finally, we’ll look at the Google Cloud tools that are available today to help you build out your edge environment, and look at some early customer examples that highlight what’s possible today — and that will spark your imagination for what to do tomorrow. 

The evolution of edge computing 

The edge is not a new concept. In fact, it’s been around for the last two decades, spanning many use cases that are prevalent today. One of the first applications for edge was to use content delivery networks (CDN) to cache and serve daily static website pages near clients, for example, web servers in California data centers serving financial data to European customers. 

As connectivity has improved and software evolved, the edge has evolved too, and the focus has shifted towards using edge to distribute services. First, simple services expanded from static HTML to javascript libraries or image repositories. Common functions like image transformation, credit and address validation support services followed. Soon, organizations were deploying more complex cloudlet and clustered microservices installations, as well as distributed and replicated datasets. The term “endpoint” became ubiquitous, and APIs profilerated. 

In parallel, there’s been an explosion of creativity in hardware, microcontrollers and dedicated edge devices. Fit-for-purpose products were deployed globally. Services like Google Cloud IoT Core extended our ability to manage and securely connect these dispersed devices, allowing platform managers to register tools and leverage managed services like Pub/Sub and Dataflow for data ingestion. And with Kubernetes, large remote clusters — mini private clouds in and of themselves — operate as self-healing, autoscaling services across the broader internet, opening the door to new models for applications and architectural patterns. In short, both distributed asynchronous systems and economies have blossomed.

What does this mean for enterprises? For the purposes of this series, edge means you can now go beyond the corporate network, beyond cloud VPCs, and beyond hybrid. The modern edge is not sitting at a major remote data center, nor is it a CDN, cloud provider, or in a corporate data center rack — it’s just as likely to look like 100 of these attached to a thousand sensors.

Edge, in short, is about having hardware and devices installed at remote locations that can process and communicate back the information they collect and generate. The edge management challenge, meanwhile, is being able to push configuration and software/model/media updates to these remote locations when they are connected.

Enable new use cases

Today, we have reached a new threshold for edge computing — one where micro-data-processing centers are deployed as the edge of a fractal arm, as it were. Together, they form a broad, geographically distributed, always-on framework for streaming, collecting, processing and serving asynchronous data. This big, loosely coupled application system lives, breathes and grows. Always changing, always learning from the data it collects — and always pushing out updated models when the tendrils are connected. 

Right now, the rise of 5G is pushing the limits of edge even further. Devices enabled with 5G can transmit using a mobile network — no ISP required — enabling connectivity anywhere within reach of a cell tower. Granted, these networks have lower bandwidth, but they are often more than adequate for certain types of data, for example fire sensors in forests bordering remote towns that emit temperature or carbon monoxide data periodically. Recently, Google Cloud partnered with AT&T to enhance business use of 5G edge technology but there is so much more that can be done. 

Reduce data center investments

In addition to enabling the digitization of a broad range of new use cases, adopting edge can also benefit your existing data center.

Let’s face it: data centers are expensive to maintain. Moving some data center load to edge locations can reduce your data center infrastructure investment, as well as compute time spent there. Edge services tend to have much lower service level objectives (SLOs) than data center services, driving lower levels of hardware investment. Edge installations also tend to tolerate disconnectedness, and thus function perfectly well with lower SLOs — and lower costs. 

Let’s look at an example of where edge can really reduce costs: big data. Back in the day, we used to build monolithic serial processors — state machines — that had to keep track of where they were in processing in case of failure. But time and again, we’ve seen that smaller, more distributed processing can break down big, expensive problems into smaller, more cost-effective chunks. 

Starting with the explosion of MapReduce almost 20 years ago, big-data workloads were parallelized across clusters on a network, and state management was simplified with intermediate output to share, wait for, or restart processing from checkpoints. Those monolithic systems were replaced by cheaper, smarter, networked clusters and data repositories where parallel work could be executed and rendered into workable datasets. 

Flash forward to today, and we are seeing those same concepts applied and distributed to edge data-collection points. In this evolution of big data processing, we are scaling up and out to the point where observation data is so massive that it must first be prefiltered, and then preprocessed down to a manageable size and still be actionable. Only then should it be written back to the main data repositories for more resource-intensive processing and model building.

In short, data collection, cleanup, and potentially initial aggregation happens at the edge location, which reduces the amount of junk data sitting in costly data stores. This increases performance of the core data warehouse, and reduces the size and cost of network transfers and storage! 

The edge is a huge opportunity for today’s enterprises. But designing environments that can make effective use of the edge isn’t without its challenges. Stick around for part two of this series, where we look at some of the architectural challenges typically encountered while designing for the edge and how we begin to address them.

The future of AI is better AI—designed with ethics and responsibility built in from the start. This means putting the brakes on AI-driven transformation until you have a well-functioning strategy and process in place to ensure your models deliver fair outcomes. Failing to recognize this imperative is a threat to your bottom line. The following post provides a simple framework to follow to keep your business on the right track as you place more trust in algorithms. 

AI is inherently sociotechnical. AI systems represent the interconnectedness of humans and technology. They are designed to be used by and to inform humans within specific contexts, and the speed and scale of AI means that any lack of responsibility—such as bias, safety, privacy, scientific excellence etc—will also replicate at that same speed and scale. Without ethics and responsibility built in by design, AI systems lack the critical “inputs” or societal context that enable long term success. 

Lawsuits stemming from AI systems that are biased towards certain groups are stacking up. In August 2020, IBM was forced to settle a lawsuit with the city of Los Angeles for misappropriating data it collected for its weather channel app. Health services company, Optum, is being investigated by regulators for creating an algorithm that allegedly recommended that doctors and nurses pay more attention to white patients than to sicker black patients. And Facebook, which granted Cambridge Analytica, a political firm, access to the personal data of more than 50 million people, is buried in legal work.  Google has also run into its share of issues with algorithms making egregious mistakes. 

While lawsuits are real, the foundational reason ethical AI is critical to your bottom line is trust. Without it, increasingly, consumers will ignore you and choose a brand they do trust. Research from Kantar, which runs one of the largest global brand equity studies (4 million consumers, 18,000 brands, across 50 markets), revealed that almost 9% of a brand’s equity is driven by corporate reputation, of which responsibility is a key attribute. Over the last decade, the importance of responsibility to consumers in relation to making brand choices has tripled. 

The study stated brands perceived to be among the world’s most trusted and responsible shared three crucial factors that proved particularly important for building consumer trust and confidence, even when a brand might be new to a market. These are:

• Honesty and openness
• Respect and inclusion
• Identifying with and caring for customers

Brands that develop these associations more strongly tend to outperform their competitors in defending and growing their brand value.

Technology and business leaders need to focus on four areas to accomplish a well-functioning ethical AI strategy. Lopez Research refers to this group of tasks as SEED, which stands for security, ethics, explainability, and data (SEED). Each of these topics could be an article in itself, but this post will define several essential components. 

SECURITY (S)  

It might not seem obvious, but a robust AI strategy requires an embedded security strategy. Companies should look for hardware-level security in components such as GPUs and CPUs. IT leaders should build software security into models to minimize attacks such as poisoning, evasion, deepfakes, backdoors, and model extraction. The threat of adversarial data poisoning attacks machine learning models by maliciously introducing inaccurate data designed to corrupt the model’s ability to be accurate. Another security threat is model extraction, also known as model cloning, where a hacker finds a way to either reconstruct a black-box machine learning model or extract the training data. The first line of defense against all security attacks is to design security at the outset, but the next best step is to frequently test models to ensure they are operating as planned. Business leaders, data science experts, and IT leaders must work together to regularly review the outcomes of AI models.

ETHICS (E)

Today, organizations must understand that ethics should be designed into the solution at its outset. The ethics process starts with defining the potential positive and negative outcomes of the model that your business is creating. Once the team has evaluated potential harmful effects, which means unpacking the systems, beliefs, power hierarchies and dynamics that interconnect with the technology, it’s your responsibility to eliminate or minimize the impact of these outcomes. It’s also critically important to review the impact of models in production and shut down models demonstrating issues. An example of this was the public beta release of the Tay chatbot that Microsoft deployed and rapidly shut down because it propagated negative biases. 

Yet, many organizations aren’t taking this action. The FICO study revealed that 93% of companies said responsible AI was critical for success but only 33% of these companies were measuring AI model outputs to ensure these models were operating as expected (measuring for model drift). Another survey by Pew Research revealed that 68% believe that ethical principles focused primarily on the public good will not be employed in most AI systems by 2030. 

Regulations may turn this tide, regardless of whether organizations plan to adopt an ethical AI framework. Laws governing the ethical use of data in AI are expected to be finalized as soon as 2022, such as the European Commission’s proposed legal framework for AI. Organizations that start with ethical use of AI in mind will be better positioned to deal with customer privacy concerns and regulatory compliance.

EXPLAINABILITY(E)

As models have become more sophisticated, it’s also become increasingly difficult to explain why a model created a specific outcome. In the FICO Responsible AI  report, 65% of respondents could not explain how specific AI model decisions or predictions are made, and only 35% said their organization made an effort to use AI in a way that was transparent and accountable.  However, it’s never been more important to clarify how AI models came to conclusions such as why a loan was denied, why a particular strategy should be implemented, and how AI selected a set of resumes to review for a position. The goal is to create an explainable AI model from the outset but many of today’s models lack this capability. Every business should review its existing models and use open-source toolkits that can be found on Github.com that support the interpretability and explainability of machine learning models. 

Keep in mind that explainability isn’t one-size-fits-all. Different stakeholders need different types of information. Much of explainability to date has focused on “opening the black box” which gets equated to information that is only useful for other data scientists. That’s important, but it doesn’t help the line of business users whose workflows AI is integrated into, or end users who deserve information about how decisions are made; or policymakers who don’t have data science backgrounds, and so on. 

DATA (D) 

An equally important item in ethics is data. Ethics starts with ensuring you have the correct data to create and update models. Three main issues include representative data, inherent biases within existing data, and inaccurate data. A critical issue that most companies miss in creating models is that current data sets frequently lack full market representation. A recent Capgemini Research Institute report revealed that 65% of executives “were aware of the issue of discriminatory bias” with these systems.

Awareness is the first step, but organizations must take action to remedy this issue. Historical data may no longer serve a company’s current needs for model creation. Historical records may contain biases against certain groups. For example, historical criminal data records show an imbalance in ethnic groups’ incarceration, which would lead to model biases. Additionally, laws and societal norms also change. Certain groups were prosecuted for sexual preference in the past, but today this information would create an inaccurate model. 

Companies have also discovered that using demographic data, a common practice in marketing, can also lead to model bias. For example, individuals that primarily used cash for transactions and others that lived in specific zip codes were at a disadvantage in banking models to determine creditworthiness. To minimize these issues, a company needs to augment its data with full representation in areas such as ethnicity, gender, age, behavioral and economic profiles.

Design AI models with a continuous feedback loop

Another, more prominent, yet tricky issue is data accuracy. As the adage says, garbage in, equals garbage out. The least appreciated but arguably the most essential component of the AI model lifecycle is ensuring the model has accurate data at all times. Inaccurate data from either poor data hygiene or data that was tampered with for security purposes can cause model failures. Organizations need to invest the time and resources to ensure they have the correct data. Data privacy is another key element that businesses must address, but the concepts of data privacy, sovereignty, and security are significant enough that we will come back to this in a separate article. 

Overall, it’s clear that while we may have an abundance of data, it most likely doesn’t represent what we want to model for the future. A successful AI strategy is an ethical AI strategy that requires the organization to be thoughtful in its model creation by ensuring it has a broad representation of accurate data and testing the outcomes to ensure the models are secure and operating as expected. 

Organizations that define an AI model lifecycle with a continuous feedback loop will reap the benefits of better intelligence. This will increasingly mean stronger, longer lasting trust with customers and staying on the right side of new laws and regulations.

As organizations continue to produce vast quantities of data, they increasingly need platforms that allow them to analyze, store, and extract meaningful insights from that data. And research from analyst firms like Gartner offer an important way for organizations to evaluate and compare cloud data warehouse providers.

Earlier this year, Gartner named Google a Leader in the 2019 Gartner Magic Quadrant for Data Management Solutions for Analytics (DMSA) (report available here). This evaluation covers Google Cloud’s core data analytics offerings, including BigQuery, a serverless, managed data warehouse, Cloud Dataproc, a managed Spark and Hadoop service, and Cloud Dataflow, which enables you to stream and batch-process your data. Here are a few takeaways:

Simplicity and speed

BigQuery’s performance permits complex queries on large-scale data sets to return in seconds, and a substantial number of BigQuery customers maintain data warehouses that store more than 50 terabytes (and a few customers now use more than 100 petabytes). More than half of these customers are loading data either continuously or many times per day. These customers value the ability to extract, transform, load, and analyze their data on a serverless platform, all without maintaining any underlying infrastructure.

A versatile serverless data warehouse

One of BigQuery’s major advantages is its ability to allow customers to address a wide variety of use cases—from a traditional data warehouse to data science. Over the past year, we’ve worked hard to introduce new features in BigQuery like data types for financial and monetary uses, BigQuery GIS for geospatial data, and machine learning capabilities through BigQuery ML. BigQuery’s continuous ingest capabilities make it suitable for an operational—or a real-time—serverless data warehouse.

An expanding ecosystem

With overall increased market adoption, our analytics offerings continue to benefit from a fast-growing partner ecosystem of service providers, and business intelligence (BI) and data integration vendors. In particular, in 2018 we expanded our partnerships with established industry providers, including Confluent, Dell Boomi, Informatica, Looker, Reltio, Tableau, and ThoughtSpot.

More and more organizations are finding value in Google Cloud’s serverless data warehouse and analytics offerings. If you’d like to learn more, you can download a complimentary copy of the Gartner Magic Quadrant for Data Management Solutions for Analytics on the Google site (requires an email address).

_{Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.}

At Google, we believe anyone should be able to quickly and easily turn their artificial intelligence (AI) idea into reality. Open source software (OSS) has become increasingly important to this goal, heavily influencing the pace of innovation in AI and machine learning (ML) ecosystems. Over the last two decades, ML has transformed Google services including Search, YouTube, Assistant, and Maps, and the basis for this transformation has always been our “open first” approach through investments in projects and ecosystems like TensorFlow, Jax, and PyTorch.

These OSS efforts are important because many AI technologies rely on closed or exclusive approaches. This wall-garden approach creates high barriers to entry for developers; limits efforts to make AI explainable, ethical, equitable; and stunts innovation. We’re committed to open ecosystems, as we firmly believe no one company should own AI/ML innovation. In this blog post, we’ll explore some of Google’s most significant OSS AI and ML contributions from recent years, as well as how our commitment to open technologies can help organizations innovate faster and more flexibly.

Openness is the way to operate as an ecosystem, not a single project

Google’s OSS initiatives extend and enable AI initiatives according to three pillars:

• Access — OSS allows developers, researchers and organizations of all sizes to leverage the latest ML technology. It is a key part of democratizing innovation in ML, fostering software diversity and choice for customers, and lowering operating cost while accelerating scale for everyone.
• Transparency — Open source datasets, ML algorithms, training models, frameworks, and compilers ensure due diligence and validation by the larger community. This is paramount when it comes to ML as it bolsters reproducibility, interpretability, ensures equity, and boosts security.
• Innovation — With more access and transparency, more innovation comes naturally. Our customers and partners take advantage of open source ML toolsets and frameworks to create more innovation in the field by contributing their own OSS.

Google’s ongoing commitment to open source AI

Google’s commitment to open standards spans over two decades of OSS contributions like TensorFlow, JAX, TFX, MLIR, KubeFlow, and Kubernetes, as well as sponsorship for critical OSS data science initiatives like Project Jupyter and NumFOCUS. Initiatives like these have helped Google become the leading Cloud Native Computing Foundation (CNCF) contributor—and by building on these efforts, Google Cloud seeks to be the best platform for the OSS AI community and ecosystem.

The perils of closed technologies can emerge at many points across ML pipelines, which is why Google’s OSS strategy encompasses the entire “idea-to-production” lifecycle, from acquiring data, to training models, to managing infrastructure, to facilitating experimentation and model refinement:

Data acquisition: starting the journey from idea to production-ready ML model

The journey from an idea to a production ML model starts with data. TensorFlow Datasets not only help users acquire ready-to-use, customizable, and highly-optimized datasets (including image, audio, and text), but also provides a set of helpful APIs that make it easy for users to organize their own datasets, regardless of whether they build with TensorFlow, Jax, or other ML frameworks.

Model development and training: shortening the path from data to useful ML

OSS libraries help developers and researchers design, implement, train, test, and debug ML algorithms. Our contributors on this front include:

• The TensorFlow core framework, which offers APIs to help data scientists and developers build and train production-grade ML models on distributed and accelerated infrastructure powered by GPUs or TPUs;
• Google’s founding membership of the PyTorch Foundation, which positions us to increase adoption of ML by building an ecosystem of open-source projects with PyTorch;
• Keras, a simple and powerful ML framework, well integrated with TensorFLow, that makes it easy for developers to quickly build and train ML models, or to leverage pre-trained AI applications;
• Model Garden, which provides implementations of many state-of-the-art computer vision and natural language processing models, maintained by Google and accessible to all, alongside APIs to accelerate training and experiments;
• Jax, a lean, intuitive, and composable system that brings together automatic differentiation (Autograd) and the Accelerated Linear Algebra (XLA) optimizing compiler to offer high-performance ML for fast research and production;
• TensorFlow Hub, a repository of trained ML models ready for fine-tuning and deployment; and,
• MediaPipe open source cross-platform, which lets users leverage customizable ML solutions for live and streaming media, including text and video.

ML infrastructure management: scaling valuable models with powerful backends

Accessing and managing infrastructure for ML, especially at scale, can be a blocker for many organizations, which is why Google has invested in initiatives including:

• The TFX (or TensorFlow Extended) platform, which offers software frameworks and tooling for full MLOps deployments, helping developers with data automation, model tracking, performance monitoring, and model retraining;
• Kubeflow, which makes deployments of ML workflows on Kubernetes simple, portable and scalable; and,
• TRC (TPU Research Cloud), which gives access to a cluster of more than 1,000 Cloud TPU devices at no charge to selected researchers who publish peer-reviewed papers and/or open source code.

Experimentation and model optimization: encouraging discovery and iteration

Data, tools for model training, and infrastructure can achieve only so much without strong processes for experimentation and optimization—which is why we’ve contribution to projects like xManager, which enables anyone to run and keep track of ML experiments locally or on Vertex AI and Tensorboard, which simplifies tracking and visualizing of model performance metrics.

These areas of focus will help not only our customers but the open-source AI community as a whole, and we’re excited to share more OSS news in coming days and months. To start exploring why many organizations choose Google Cloud for their open-source AI needs, visit our “open cloud” page and be sure to register for Google Cloud Next ‘22 for all our latest news.

Thanks to all the contributors to this blog post: Matt Vasey, George Elissaios, Warren Barkley, Manvinder Singh, James Rubin, Abhishek Ratna, Thea Lamkin, Amin Vahdat, Andrew Moore, Max Sapozhnikov, Gandhi, Vikram Kasivajhula