May is a big month for the security industry. It’s been over a year since we gathered for RSA in San Francisco for one of 2020’s last major in-person events. While we likely won’t be together in person this year, it’s an important time for the security community to come together and reflect on many accomplishments, and to consider the challenges still ahead of us. As the world focuses on security incidents and all the risks that still need resolving, it is important to stand back, on occasion, and also note that immense progress has been made by large numbers of small, medium and large enterprises to protect themselves and their customers against increased threats. What is also amazing is to see organizations do this while accelerating their digital transformations, supporting and protecting customers and managing ongoing remote working challenges. We are privileged to play our part in supporting those great teams. 

It’s also been a busy month for us here at Google Cloud since our inaugural CISO perspectives blog post in April. Today, I’ll recap our cloud security and industry highlights, a sneak peak of what’s ahead from Google at RSA and more. 

Thoughts from around the industry 

• Risk Governance of Digital Transformation in the Cloud – In our latest Office of the CISO whitepaper, we shared guidance on both the challenges and opportunities of cloud transformation for Chief Risk Officers, Chief Compliance Officers, Heads of Internal Audit and their teams. A misconception we sometimes see among these executives is that moving to the cloud creates more risk to manage. Having held these leadership positions in previous roles, I believe that the cloud is as much a means of managing security, resilience and other risks as it is a risk in its own right. The whitepaper dives deep into considerations for each of these leadership functions as their organization embarks on a digital transformation journey. 
• The importance of meeting global compliance requirements – Compliance is critical for building trust with customers in regulated industries, especially the public sector. It is worth remembering that in any critical industry, where there can be material impact from incidents, strong industry practices and standards to protect customers are vital (I wrote about this last summer). At Google Cloud, we’re regularly adding new compliance and security certifications to meet our customers’ needs globally. Recently, we expanded our list of FedRAMP High-certified products to include Cloud DNS, and helped our customers in the Asia-Pacific region address various compliance requirements to meet new government regulations for security and data protections. Google Cloud was also the only cloud service provider to complete an annual pooled audit with the Collaborative Cloud Audit Group (CCAG), which is a syndicate of 39 leading European financial institutions and insurance companies who depend on cloud infrastructure and technologies to deliver innovative solutions and experiences for their customers. Having spent most of my career in the financial services industry, I know firsthand the importance of managing risk assessments for outsourced vendors to provide the necessary assurances customers need from their cloud providers. 

RSA 2021 

We have a great lineup of speaking sessions and keynotes from Googlers at RSA this year. Below are the highlights you don’t want to miss: 

• I’ll be doing a session on May 20 about supply chain resilience, where a panel of experts will dive into how we can adjust risk and security initiatives to handle the next “punch to the supply chain.” Additionally, on May 18 I’ll join many of my esteemed CISO leaders from various industries and governments for a keynote discussion on our top security insights, lessons learned and best practices for how we move forward as an industry to address the next wave of challenges. 
• Google’s Senior Director of Information Security Heather Adkins will deliver a session on how to build secure and reliable systems at scale, which will cover principles from Google’s Site Reliability Engineering book with the same title (available for free download here). I’m most looking forward to Heather’s advice for how we as an industry can reshape our security thinking, based on modern architectures and technologies that can help organizations design scalable and reliable systems that are fundamentally secure.
• Nelly Porter, Senior Product Manager at Google Cloud Security, will participate in a panel discussion with security experts on the importance of Confidential Computing technology, how it’s changing the security landscape and where it’s headed. Google Cloud has made great progress in delivering a Confidential Computing portfolio for our customers in regulated industries over the past year, and we’re excited for new milestones in 2021. 

Google cloud security highlights

• Infrastructure and SRE spotlight – Before I joined Google Cloud, I always admired the infrastructure and benefits this organization delivers that are uniquely Google – from the subsea cable innovations to SRE inventions and principles. Security and resiliency are baked into every layer of our infrastructure. Many of the Googlers who build and support our platform have sat in the same seat as our customers, so they understand those needs intimately. Over the last few months it’s been amazing to watch our technical infrastructure team grow, and the direct reliability, operational resilience and security benefits that team brings to our customers. For example, we’ve opened a new region in Poland, announced the first subsea cable that will directly connect the U.S. to Singapore with fiber pairs over an express route, and released an SRE book focused on how organizations can complete a successful cloud migration.
• New security foundations blueprint guide – As part of our mission to deliver the industry’s most trusted cloud, we strive to operate in a shared-fate model for risk management in conjunction with our customers. This includes sharing opinionated step-by-step guidance with key decision points and focus areas for how our customers deploy workloads in Google Cloud. This is why we’ve updated our Google Cloud security foundations guide and corresponding Terraform blueprint scripts. These blueprints are tremendously helpful to many stakeholders within an enterprise, like a CISO that needs to understand our key principles for cloud security, or a C-Suite business leader that needs to quickly identify the skills their teams need to meet an organization’s security, risk, and compliance needs on Google Cloud. 

When we think about the types of features to build into products, we have many principles we follow. But the two that I keep coming back to as crucial are:

1. The need for secure products not just security products. All products should have security built in and while we do build great security products our security and other teams remain focused on constantly enhancing the base levels of security and the security features in all our products. 
2. Defense in Depth. We don’t just focus on defense in depth from attacks – for ourselves and our customers. We also prioritize defense in depth from configuration errors or other hazards. 

As you see below in some of the highlights of new features and products, these represent our commitment to secure products and all forms of defense in depth. 

• Workload identity federation – Service account keys are powerful credentials, and can represent a security risk if they are not managed correctly. A safer approach is to use workload identity federation, using IAM to grant external identities IAM roles, including the ability to impersonate service accounts. This lets you access resources directly and eliminates the maintenance and security burden associated with service account keys. We also offered related overall guidance on the best way to use and authenticate service accounts on Google Cloud.
• VPC-SC Directional Policies – With VPC Service Controls (VPC-SC), admins can define a security perimeter around Google-managed services to control communication to and between those services. Using VPC-SC, you can isolate your production GCP resources from unauthorized VPC networks or the internet. But what if you need to transfer data between isolated environments that you’ve set up? VPC-SC directional policies is a new secure data exchange feature that allows you to configure efficient, private, and secure data exchange between isolated environments. 
• Anthos service mesh supports VMs as well as clusters – Most enterprise compute resources are still in VMs and many will remain there for a long time to come. In Anthos 1.7,  your VM-based workloads can now take advantage of the same mesh functionality as your container-based workloads.
• Cloud Spanner CMEK and Access Approvals – Cloud Spanner is Google Cloud’s fully managed relational database that offers unlimited scale, high performance, strong consistency across regions and high availability. Spanner now supports customer-managed encryption keys (CMEK) and Access Approval, Google Cloud’s industry-leading controls to require approval before access to your content by Google support and engineering teams.
• External Key Manager enhancements – In early 2020 we launched Cloud External Key Manager (Cloud EKM), the industry’s leading Hold-Your-Own-Key (HYOK) product. Using Cloud EKM, the keys used to protect your data stored and processed in Google Cloud are completely hosted and managed outside of Google Cloud infrastructure. Cloud EKM initially launched with support for BigQuery and GCE/PD;  we expanded support for Cloud SQL, GKE, Dataflow Shuffle, and Secret Manager, with CMEK support currently in beta. We also provided in-depth documentation on the functionality, architecture and use cases for Cloud EKM in a new whitepaper.
• Web App and API Protection solution –  Web applications and public APIs are increasingly important to how organizations interface with their customers and partners, and we’ve seen increased investment in tools to protect these resources from fraud and abuse. Google Cloud’s new Web App and API protection solution is based on the same technology Google uses to protect its public-facing services against web application exploits, DDoS attacks, fraudulent bot activity, and API targeted threats. It provides protection across clouds and on-premises environments.
• Threat Intel for Chronicle – Most threat intelligence feeds require security teams to do the implementation and legwork. With our new Threat Intel for Chronicle offering, however, our intelligence insights are applied automatically across your security telemetry to present unique observations within your environment. Threat Intel for Chronicle is exclusively curated for enterprise customers by Uppercase, Google Cloud’s intelligence research and applications team to provide our perspective on threats across the internet and surface them as relevant alerts.

That wraps up another month of thoughts and highlights. If you’d like to have this Cloud CISO Perspectives post delivered every month to your inbox, click here to sign-up, and we’ll see you in June!

Generative AI continues to develop at a blistering pace, making it more important than ever that organizations have access to enterprise-ready capabilities to help them leverage this disruptive technology. 

Harnessing the power of decades of Google’s research, innovation, and investment in AI, Google Cloud continues to make generative AI available with baked-in security, data governance, and scalability across the board. 

To this end, last month, we announced the general availability of Generative AI support on Vertex AI, giving our customers the ability to access powerful foundation models from Google Research and tools for customizing and applying them. 

Today we are announcing the general availability (GA) of four important foundation models for Vertex AI. These include Imagen, PaLM 2 for Chat, Codey, and Chirp. For each of these models, organizations can access APIs on Model Garden and do prompt design and tuning on Generative AI Studio.

• Imagen includes four key features:
  • Image generation for creating studio-grade images at scale
  • Image editing to edit generated or existing images via text prompts 
  • Image captioning for creating captions of images at scale
  • Visual Question & Answering (VQA) for interacting with, analyzing, and explaining images
• PaLM 2 for Chat follows the general availability of PaLM 2 for Text in June 
• Codey supports code generation, completion, and code chat
• Chirp supports multilingual Speech AI 

We’re also announcing Multimodal Embeddings API in preview, which lets customers combine the power of Vertex AI’s generative AI models with their proprietary data, to generate embeddings, or interchangeable vector representations, of their text and image data. These capabilities can enable data science teams to deliver a variety of downstream tasks such as image classification, content recommendations, and visual search. 

In this blog post, we’ll explore what your organization can do with these powerful models and how Vertex AI provides the enterprise-ready capabilities you can use to get up and running with generative AI. 

Helping to drive enterprise value from Generative AI models

Powerful models are the foundation of generative AI, but the software, tools, and infrastructure that surround these models are equally important for enterprise adoption. Organizations face challenges not only accessing these models, but also integrating AI while maintaining protection over intellectual property, adhering to regulations around data security and privacy, and ensuring models and applications are safe to use. Many organizations also want to use generative AI without incurring large costs or managing huge clusters.

We help address these challenges head-on with Vertex AI’s platform capabilities for scalable application integration, purpose-built AI infrastructure, secure and private data customization, and responsible use of this technology. 

Let’s see how each of these pillars can help your organization. 

Access models to build production-ready generative applications 
Vertex AI can make it easy to access foundation models, as today’s model announcements attest. While models are an inextricable part of generative AI, the software that helps enterprises use this technology is equally important—which is why Vertex AI also offers a range of tools for tuning, deploying, monitoring, and maintaining models, so you can build differentiated applications using your own data. 

Turning to today’s announcements, in May we announced Imagen, our foundation model for image generation. Now, we are excited to announce Imagen is generally available with an allowlist (i.e., approved access via your sales representative), letting onboarded customers start using image generation and editing capabilities. Visual Q&A and Captioning for production workloads are also generally available for all customers. Visual Q&A provides new ways to engage with image-based data like retail products or image libraries. This new capability can give you answers to questions about an image, helping you analyze large amounts of data quickly, and it can even help the visually impaired understand images or graphs that they wouldn’t be able to otherwise. Captioning, meanwhile, can make it easy to generate relevant descriptions for your images. Captions can help with indexing and searching, as well as assigning image descriptions to product listings on eCommerce websites. 

“Imagen is beginning to power key capabilities within Omni, Omnicom’s open operating system, that will enable 17,000+ trained and certified users to create audience-driven customized images in minutes. Imagen has been instrumental in offering a scalable platform for image generation and customization. Integrating it into our platform allows us to expand the scope of audience-powered creative inspiration, at a scale that wasn’t previously possible,” said Art Schram, Annalect Chief Product Officer at Omnicom. “We’re starting to adopt the latest features like styles and fine tuning, and engineering data-driven prompts. We look forward to continuing to provide our users relevant visual inspiration in a responsible way.”

“The latest improvements in Imagen’s product preservation capabilities are a perfect match for Typeface’s focus on personalized AI for brands,” explained Vishal Sood, Head of Product at Typeface. “By combining Google Vertex AI’s Imagen with Typeface’s brand-personalized AI, we are able to help enterprises to create 10x personalized content in a fraction of time.”

Google Shopping recently built an application called Product Studio using Imagen on Vertex AI. Product Studio can enable merchants to create rich product images quickly and easily, at a fraction of the time it takes to do professional product photo shoots. “We’re excited about the feedback we’re getting from merchants in our early pilots, who say that Product Studio, which leverages Imagen on Vertex AI, helps them generate and publish lifestyle product photos directly to their product catalogs,” says Jeff Harrell, Google’s Senior Director of Product Management for Merchant Shopping. 

Announced in May, PaLM 2 is a family of models that power dozens of Google products, including Bard and Duet AI in Google Cloud. With the PaLM 2 for Chat model, now generally available, you can leverage Google’s PaLM’s variety of abilities for multi-turn chat applications, such as shopping assistants, customer support agents, and more. 

ThoughtSpot, provider of a widely-adopted business intelligence platform, is using PaLM 2 to build a new feature in ThoughtSpot for Google Sheets called “AI Explain,” which can instantly generate explanations of charts, visuals, and anomalies, and will launch new conversational AI and ML-enabled predictive forecasting capabilities into its analytics platform.

With Codey, your organization’s developers can accelerate a wide variety of coding tasks, helping to empower them to work efficiently and close skills gaps. The model enables not only code completion and code generation capabilities, but also chat to help with debugging, documentation, learning new concepts, and more. Since launching in preview in May, we’ve added additional programming languages including Go, Google Standard SQL, Java, Javascript, Python, and Typescript. We’ve also improved the quality of code responses and increased serving capacity, enabling your developers with the right tools to enter the era of generative engineering.  

“Security and privacy are key to incorporating AI into the software development lifecycle,” said David DeSanto, Chief Product Officer at GitLab. “GitLab leverages Vertex AI to deliver new, AI-powered features with a privacy-first approach, including the ability to run our own models and leverage Codey foundation models built on top of PaLM 2. The GitLab DevSecOps platform empowers organizations to harness the benefits of AI for faster software delivery, while ensuring their data, intellectual property, and source code are protected.”

Originally released in May in preview, Chirp is a version of our 2 billion-parameter speech model, which was trained on millions of hours of audio and supports over 100 languages. Chirp achieves 98% accuracy on English and relative improvement of up to 300% in languages with less than 10 million speakers. Whether the use case involves customer support, transcriptions, or voice control, Chirp can help your organization communicate with customers and constituents inclusively, by engaging audiences in their native languages. 

Last but not least, our Multimodal Embeddings API, now in preview, can unlock an array of new applications, such as image and text-based recommendations, by enabling the processing of text and images interchangeably. This capability complements our Text Embeddings API, which became generally available in June, and remains a recommended choice for those with fully text-based use cases. Multimodal Embeddings API makes it possible to categorize images and text together and can be crucial for use cases like retail recommendation systems that can provide relevant outputs from both images of products and text descriptions.

Match generative AI with infrastructure 
Beyond access to models and tools for building generative AI apps, you need infrastructure to make sure your apps can scale and reliably perform — ideally without running into daunting compute costs or management overhead that distracts your technical talent from building innovative products. Google Cloud offers the choice and power to run smaller models running finite tasks at the lowest latency levels, as well as to run large models capable of cutting-edge experiments. 

As our large language model customers are looking to scale up their projects and applications using our models, they often need assurances that their requests will be serviced with acceptable performance. This is especially critical for delivering real-time applications where customer service is paramount. Starting in August, Vertex AI will support provisioned, dedicated generative AI capacity that can deliver guaranteed throughput. This feature can be especially beneficial to customers who have a high volume of sustained workloads.  

Leverage generative AI while protecting data and privacy 
One capability enabled by Google Cloud is the ability to customize models using your own data. Vertex AI can help customers keep their data protected, secure, and private. When a company tunes a foundation model in Vertex AI, private data, model outputs, and prompts can be kept private, and they are never used in the foundation model training corpus. We recently published a whitepaper, “Adaptation of Large Foundation Models,” which outlines how we help protect customer data. 

Auditability and compliance are essential to helping ensure the security and privacy of customer data. We also engage in comprehensive GDPR privacy efforts, including our transparency commitments for customer data usage and the support for our customer’s Data Protection Impact Assessments (DPIAs). Now, we’re excited to support HIPAA compliance for many of our generally available models on Vertex AI, so that healthcare and life science customers with whom we have a Business Associate Agreement can run workloads with Protected Health Information (PHI) data on Google Cloud. 

Innovate responsibly 
Our AI Principles put beneficial use, user safety, and avoidance of harms above business outcomes and are embedded in how we develop our AI products. We’ve conducted extensive reviews on our generative AI products to identify potential risks and have developed guardrails to mitigate these impacts. For example, to address concerns around safety, we’ve implemented safety filters for bias, toxicity, and other harmful content. We also equip our customers with the tools they need to help reduce risk within their applications and provide recommendations to help navigate responsible AI. 

Bring the power of generative AI to your organization

With both a wide selection of foundation models and extensive, enterprise-grade platform capabilities, Vertex AI continues to unlock ways for your business or organization to access foundation models, tune them on your proprietary data, and leverage them for differentiated apps and digital experiences. To take the next step, visit our product page or reach out to our sales representatives to gain access to our latest capabilities.

How many times have you closed an application when you encounter the “spinning wheel of death” (a melodramatic way of saying an application that is taking too long to load)? In today’s digital economy where many organizations rely on applications as a primary source of revenue, that spinning wheel of death (or poor application performance) can mean lost users or revenue. And just about every modern application relies on APIs as the nervous system across distributed systems, third-party services and microservice architectures. While meeting the need for rapid release cycles and frequent API updates, it is also imperative for IT teams to ensure that your APIs are meeting SLOs, performance requirements, and proactively mitigating issues.

Why are synthetic monitoring tools not enough?

When thousands or even millions of users are making multiple requests to your APIs, just relying on synthetic monitoring tools (that rely on sampling or limited API availability information) is not enough for precision diagnoses or useful forensics. At the same time, monitoring every single aspect just increases your overhead and mean time to diagnosis. Apropos, API monitoring has become absolutely critical — and a fusion of art and science — for operations teams to make sure all APIs are running and performing as intended. If you are worried about monitoring blind spots or overheads, let us look at 3 key practices you can follow to stop dreading your sev1 alerts.

#1 Prioritize alerts for critical events requiring immediate investigation

• Ask any engineer on-call for a critical service and they will tell you about the overheads created by incorrectly prioritized alerts. For example let us imagine a distributed application with 20 APIs. Even if you set up basic alert monitors across latency, error, and traffic for these APIs, you end up monitoring and maintaining 60 alert definitions – which is a lot. To balance avoiding monitoring blindspots and alert fatigue at the same time, operations teams must develop a clear understanding of all events and prioritize configuring alerts for events supporting critical traffic.

• Consider quality over quantity while defining new alert conditions where each new condition is urgent, actionable, and actively or imminently user-visible. Every alert condition created should also contain intelligence that requires active engagement from a user as opposed to a mere robotic response. Apigee’s API monitoring allows creation of alert conditions based on metrics or logs while providing actionable information (Ex: status code, rate etc.,) and playbooks for diagnosi
• In today’s multilayered systems, one team’s symptom (“what’s broken?”) is another downstream system’s cause (“Why?”). Even if some events are not suitable for actionable alerts, a failure needs to create an informational broadcast to a downstream system to mitigate the impact of the upstream dependency. In such cases, investing in automating alerts, grouping multiple incidents in notification channels, and incident tracking. For example, Apigee lets you integrate and group your alert notifications in channels like Slack, Pagerduty, webhooks etc.,
• Modern production systems are ever-evolving where an alert that’s currently rare might become frequent and automatable. Analogous to ticket backlog grooming, alert policies need to be reviewed periodically to make sure new conditions are identified and existing alerts are refined with new thresholds, prioritization and correlation. Controls like Advanced API Ops leverage AI and ML to detect anomalous traffic differentiated from random fluctuations to help define accurate alert definitions

Check out some examples of alerts here

#2 Isolate problem areas quickly with dashboards

Google’s Site Reliability Engineering book presents the case for efficient diagnosis by building dashboards that answer basic questions about every service, normally including some form of the four golden signals — latency, traffic, errors and saturation. But at the same time, capturing just these golden metrics at different levels of granularity can quickly add up. Like all software systems, monitoring can become an endless pit of complexity, complicated to change and a burden to maintain. In the same book, the most effective direction to create a well-functioning standalone system is to collect and aggregate basic metrics, paired with alerting and dashboards.

If you are running a large scale API program with a dedicated team to monitor your APIs, you can leverage the out-of-the-box monitoring dashboards in your API Management solution (like Apigee’s API monitoring) to gather real-time insights into your API performance, availability, latency, and errors. In other cases, you can use solutions like Cloud Monitoring that can provide visibility across your full application stack where individual metrics, events, and metadata can be visualized in a rich query language for rapid analysis. Leveraging a single system for your application stack provides observability in context and can reduce your time spent navigating between systems (Apigee customers can use Cloud Monitoring by default or integrate with other systems using Cloud Monitoring API)

Even after you collect and aggregate the metrics, it is important to have impactful data visualizations to quickly understand the issue and identify correlations during diagnosis. In data visualizations as well, focusing on too many dashboards creates a steep learning curve and increases mean time for every diagnosis. For example, Apigee API Monitoring provides the following visualizations as a standard to balance simplicity and efficiency:

• Timeline view to diagnose traffic (in 1 min intervals), error rates (4xx and 5xx across traffic) and latencies (50th, 90th, 95th and 99th percentiles)
• Pivot tables of metrics and attributes for all API traffic, to help compare activity across different metrics
• Treemaps of recent API traffic by proxy to get a snapshot of incidents, error rates and latencies

#3 Incorporate distributed tracing into your end to end observability strategy

Modern application development accelerated the adoption of technologies and practices like cloud, containers, APIs, microservice architectures, DevOps, SRE etc. While this increases release velocity, it also introduces complexity and more points of failure in an application stack. For example, a slow response to a customer request spans across multiple micro services owned (and monitored) by various teams who might not observe any individual performance issues. Without an end-to-end contextual view of a request, it is nearly impossible to isolate the point of high latency.

In such cases, distributed tracing is the best way for DevOps, Operations and SREs to get answers to questions such as service health, root cause of defects, or performance bottlenecks in a distributed system. Organizations should invest in instrumenting their distributed applications using open source standards such as OpenCensus and Zipkin. Using tools like Cloud Trace with a broad platform, language and environment support can help easily ingest data from any source — open instrumentation or proprietary agents.

While distributed tracing helps in narrowing the issue to a given service, in some cases you might need further context to pinpoint the root cause. For example: Even if you have isolated the source of a performance issue to an API proxy, it is still a tedious process to identify the right bottleneck among multiple policies being executed. Tools like Apigee Debug enable you to zoom into an API proxy flow and probe the details of each step to see internal details like policy executions, performance issues, and routing etc.,

As soon as a request starts to span across a handful of microservices, tools like distributed tracing and Debug will become crucial elements of your monitoring strategy. When every service in a distributed system emits a trace, the amount of data can quickly become overwhelming leading to the classic “needle in the haystack” problem. In such scenarios, it becomes vital to ask the right questions and choose between head-based sampling (randomly selecting which traces will be sampled for analysis) and tail-based sampling (observe all trace information and sample the traces with unusual latency or errors) based on application complexity

Implement effective API monitoring in Apigee

Apigee’s API monitoring (based on metrics exposed by the internals of the system) capabilities work with your existing monitoring infrastructure to help reduce mean time to diagnosis and increase application resiliency. Specifically, operations teams can leverage

• Monitoring dashboards to gain in-depth insights into API availability and performance metrics.
• Debug to precisely diagnose with deeper insights into an API proxy flow without toggling multiple tools.
• Alerts and notifications to gather contextual Insights, facilitate customization, and grouping with first class integrations to various tools (like Slack, PagerDuty, email and support for webhooks).
• Best of Google technologies such as Data Flow, Pub/Sub, Stackdriver, Bigtable and BigQuery to handle massive volumes and complex metrics at scale

Using Apigee’s API monitoring will help you maintain high application resiliency with comprehensive controls to reduce mean time to diagnosis and resolution. Get started with Apigee today or explore Apigee’s API monitoring for free here. Check out our documentation for additional information on API monitoring.