Like virtually every business across the globe, The New York Times had to quickly adapt to the challenges of the coronavirus pandemic last year. Fortunately, our data system with Google Cloud positioned us to perform quickly and efficiently in the new normal. 

How we use data

We have an end-to-end type of data platform; on one side we work very closely with our product teams to collect the right level of data that they’re interested in, such as which articles people are reading, and how long they’re staying onsite. We frequently measure our audience to understand our user segments, and how they come onsite or use our apps. We then provide that data to analysts for end-to-end analytics. 

On the other side, the newsroom is also focused on audience, and we build tools to help them understand how Google Search or different social promotions play a role in a person’s decision to read The New York Times, and also to get a better sense of their behavior on our pages. With this data, the newsroom can make decisions about information that should be displayed on our homepage or in push notifications. 

Ultimately, we’re interested in behavioral analytics—how people engage with our site and our apps. We want to understand different behavioral patterns, and which factors or features will encourage users to register and subscribe with us. 

We also use data to create or curate preferences around personalization, to ensure we’re delivering to our users fresh content, or content that they may not have normally read. Likewise, our data also gets used in our targeting system, so that we can send out the right messaging about our various subscription packages to the right users.

Choosing to migrate to Google Cloud

When I came to The New York Times over five years ago, our data architecture was not working for us. Our infrastructure was gathering data that proved harder for analysts to crunch on a daily basis. We were also hitting hang ups with how that data was streaming into our system and environment. Back then we’d run a query and then go grab some coffee, hoping that the query would finish or give us the right data by the time we came back to our desks. Sometimes it would, sometimes it wouldn’t.

We realized that Hadoop was definitely not going to be the on-premises solution for us, and that’s when we started talking with the Google Cloud team. We began our digital transformation with a migration to BigQuery, their fully managed, serverless database warehouse. We were under a pretty aggressive migration timeline, focusing first on moving over analytics. We made sure our analysts got a top-of-the-line system that treated them the way that they themselves would want to treat the data. 

One significant prominent requirement in our data architecture choice was to enable analysts to be able to work as quickly as they needed to provide high-quality deliverables for their business partners. For our analysts, the transition to BigQuery was night and day. I still remember when my manager ran his very first query on BigQuery and was ready to go grab his coffee, but the query finished by the time he got up from his chair. Our analysts talk about that to this day.

While we were doing the BigQuery transition, we did have concerns about our other systems not scaling correctly. Two years ago, we weren’t sure we’d be able to scale up to the audience we expected on that election day. We were able to band-aid a solution back then, but we knew we only had two more years to figure out a real, dependable solution. 

During that time, we moved our streaming pipeline over to Google Cloud, primarily using App Engine, which has been a flexible environment that enabled quick scaling changes and requirements as needed. Dataflow and Pub/Sub also played significant roles in managing the data. In Q4 of 2020 we had our most significant traffic ever recorded, at 273 million global readers, and four straight days of the highest traffic we’ve had compared to other election weeks. We were proud to see that there was no data loss.

A couple of years ago, on our legacy system, I was up until three in the morning one night trying to keep data running for their needs. This year, for election night, I relaxed and ate a pint of ice cream because I was able to more easily manage our data environment, allowing us to set and meet higher expectations for data ingestion, analysis and insight among our partners in the newsroom.

How COVID-19 changed our 2020 roadmap 

The coronavirus pandemic definitely wasn’t on my team’s roadmap for 2020, and it’s important to mention here that The New York Times is not fundamentally a data company. Our job is to get the news out to our users every single day in paper, on apps, and onsite. Our newsroom didn’t expect the need to build out a giant coronavirus database that would enrich the news they share every day. 

Our newsroom moves quickly, and our engineers have built one of the most comprehensive datasets on COVID-19 in the U.S. With Google, The New York Times decided to make our data publicly available on BigQuery Google’s COVID-19 public dataset. Check out this webinar for more details on our evolution architecture:https://www.youtube.com/embed/mtNlrFpschU?enablejsapi=1&

Flexible approach

We have many different teams that work within Google Cloud, and they’ve been able to pick from the range of available services and tailor project requirements keeping those tools available in mind.

One challenge we think about with the data platform at The New York Times is determining the priorities of what we build. Our ability to engage with product teams at Google though the Data Analytics Customer Council allows us to see into the BigQuery roadmap, or the data analytics roadmap, and plays a significant role in determining where we focus our own development. For example, we’ve built tools like our Data Reporting API, which reads data directly from BigQuery, in order to take advantage of tools like BigQuery BI Engine. This approach encourages our analysts to be better managers of their domains around dimensions and metrics, but not have to focus on building caching mechanisms of their data. Getting that kind of clarity helps us plan how to build The New York Times in the new normal and beyond.

If you are interested to learn more about the data teams at the New York Times, take a look at our open tech roles here and you’ll find many interesting articles at NYT data blog.

Google Cloud VMware Engine delivers an enterprise-grade, cloud-native VMware experience that is built on Google Cloud’s highly performant and scalable infrastructure. By enabling a consistent VMware experience, the service allows customers to adopt Google Cloud rapidly, easily, and with minimal modifications to their vSphere workloads, bringing the best of VMware and Google Cloud together on one platform for a variety of use-cases. These include rapid data center exit, application lift and shift, disaster recovery, virtual desktop infrastructure, or modernization at your own pace.

Here are seven ways VMware Engine outshines alternatives for running your VMware workloads in the cloud, simplify your operations, and help you innovate faster:

1. Dedicated 100Gbps east-west networking
   Google Cloud VMware Engine nodes come with redundant switching and dedicated 100Gbps east-west networking with no oversubscription of bandwidth, unlike other options where there is generally oversubscription. This is especially important when it comes to running latency-sensitive workloads.
2. Four 9’s of availability in a single zone
   The service offers 99.99% uptime SLA for a cluster in a single Zone with five to 16 nodes and FTT=2 or more without the need for stretched clusters, which is higher than the alternatives. Further, dedicated connectivity for core service functions such as vSAN and vMotion enables better solution stability and availability. This enables the service to support the needs of enterprise workloads that require high availability.

Note: “Cluster” means a deployment of three or more dedicated bare metal nodes running VMware ESXi and associated networking managed via management interfaces.

3. Global networking without complex routing
   Google Cloud VMware Engine networking is built based on Google Cloud’s powerful networking architecture. With simplified regional and global routing modes—which allow a VPC’s subnets to be deployed in any region where our service is available—you can architect global networks without the need or overhead of creating and connecting regional network designs. You get instant, direct Layer 3 access between them. In alternative cloud environments, you may have to configure special networking between regions, often requiring VPN-based tunnels over the WAN to enable global uniform network communication. This adds to the deployment and operational complexity, in addition to cost.
4. Integrated multi-VPC networking
   Users often have application deployments in different VPC networks, such as separate dev/test and production environments or multiple administrative domains across business units. The service supports “many-to-many” access from VPC networks to Google Cloud VMware Engine networks with multi-VPC networking, allowing you to retain existing deployed architectures and extend them flexibly to your VMware environments. In addition, by providing multi-VPC networking, you can pool their VMware needs—say for QA and dev—to a smaller set of clusters, effectively reducing their costs.

For more information about the end-to-end networking capabilities and services available in Google Cloud VMware Engine, please refer to the Private Cloud Networking for Google Cloud VMware Engine whitepaper. Here, you’ll find details about network flows, configuration options, and the differentiated benefits of running your VMware workloads in Google Cloud.

5. Unified, cloud-integrated model
   Google Cloud VMware Engine is a fully managed Google first-party service, operated and supported by Google and its world-class team. With fully integrated identities, billing, and access control, you have a simpler end-to-end experience that is different from other services. You access Google Cloud VMware Engine service via the Google Cloud console, like any other Google Cloud service. You can also access other native Google Cloud services privately from your VMware private cloud running in Google Cloud VMware Engine over local connections.
6. Flexibility in third-party ecosystem compatibility
   With Google Cloud VMware Engine, you can set up existing VMware on-premises third-party tools or products that require additional privileges by using a solution user account. This uniquely enables operational consistency, ensuring that the tools you have invested in and used over the years work on Google Cloud VMware Engine. Furthermore, in key areas such as vSAN data encryption, you have the choice of not only using Google Cloud Key Management Service (KMS)—which is turned on by default on vSAN datastores—but also external KMS providers such as HyTrust, Thales, and Fortanix.
7. Dense nodes with high storage:core and memory:core ratios and fast provisioning
   Google Cloud VMware Engine nodes are dense. Each node is powered by Intel® Xeon® Scalable Processors and comes with 36 cores, 72 hyperthreaded cores, 768 GB memory, 19.2 TB NVMe data and 3.2 TB NVMe cache storage. This, along with oversubscription, leads to high consolidation ratios and compelling storage:core per dollar and memory:core per dollar. In addition, you can rapidly spin up these nodes in a VMware private cloud often in under an hour, enabling on-demand, VMware-consistent capacity in Google Cloud for your needs.

These are just a few examples of customer-centric innovation that set Google Cloud VMware Engine infrastructure apart. In addition, migrating to Google cloud can save you up to 38% in TCO. Get started by learning about Google Cloud VMware Engine and your options for migration, or talk to our sales team to join the customers who have embarked upon this journey.

The authors would like to thank the Google Cloud VMware Engine product team for their contributions on this blog.

In times of significant disruption, organizations are faced with three choices: Retrench within legacy solutions, pause and do nothing while waiting for more data or different circumstances, or press ahead, potentially even accelerating to realize the desired outcome. In such an environment, it is critical to ensure you’re delivering the greatest possible impact to the business. 

In Google Cloud’s Office of the CTO, or OCTO, we have the privilege of co-innovating with customers to explore what’s possible and how we can re-imagine and solve their most strategic challenges. These collaborative innovation engagements are often core to critical transformational projects, which often include the rehosting, evolution, and at times re-architecture of existing business solutions. 

We are happy to offer this brand new white paper, where we have distilled the conversations we’ve had with CIOs, CTOs, and their technical staff into several frameworks that can help cut through the hype and the technical complexity, to help devise the strategy that empowers both the business and IT. We called one such framework “up or out.” (And we don’t mean some consulting firm’s hard-nosed career philosophy.) 

One model that we found can help enterprises chart their cloud adoption journey delineates cloud migration along two axes—up and out, and we’ll cover this in much greater detail in the white paper itself.

As you can see, there isn’t a single path to the cloud—not for individual enterprises and not even for individual applications. The up or out framework can help an IT organization and its leadership characterize how they can best benefit from migrating their services or workloads. The framework acts as a general pattern that highlights the continuum of approaches to explore, and you can learn all about it by downloading this detailed white paper.

Or, if you’re really ready to jump start your migration today, you can take advantage of our current offer by signing up for a free discovery and assessment.

May is a big month for the security industry. It’s been over a year since we gathered for RSA in San Francisco for one of 2020’s last major in-person events. While we likely won’t be together in person this year, it’s an important time for the security community to come together and reflect on many accomplishments, and to consider the challenges still ahead of us. As the world focuses on security incidents and all the risks that still need resolving, it is important to stand back, on occasion, and also note that immense progress has been made by large numbers of small, medium and large enterprises to protect themselves and their customers against increased threats. What is also amazing is to see organizations do this while accelerating their digital transformations, supporting and protecting customers and managing ongoing remote working challenges. We are privileged to play our part in supporting those great teams. 

It’s also been a busy month for us here at Google Cloud since our inaugural CISO perspectives blog post in April. Today, I’ll recap our cloud security and industry highlights, a sneak peak of what’s ahead from Google at RSA and more. 

Thoughts from around the industry 

• Risk Governance of Digital Transformation in the Cloud – In our latest Office of the CISO whitepaper, we shared guidance on both the challenges and opportunities of cloud transformation for Chief Risk Officers, Chief Compliance Officers, Heads of Internal Audit and their teams. A misconception we sometimes see among these executives is that moving to the cloud creates more risk to manage. Having held these leadership positions in previous roles, I believe that the cloud is as much a means of managing security, resilience and other risks as it is a risk in its own right. The whitepaper dives deep into considerations for each of these leadership functions as their organization embarks on a digital transformation journey. 
• The importance of meeting global compliance requirements – Compliance is critical for building trust with customers in regulated industries, especially the public sector. It is worth remembering that in any critical industry, where there can be material impact from incidents, strong industry practices and standards to protect customers are vital (I wrote about this last summer). At Google Cloud, we’re regularly adding new compliance and security certifications to meet our customers’ needs globally. Recently, we expanded our list of FedRAMP High-certified products to include Cloud DNS, and helped our customers in the Asia-Pacific region address various compliance requirements to meet new government regulations for security and data protections. Google Cloud was also the only cloud service provider to complete an annual pooled audit with the Collaborative Cloud Audit Group (CCAG), which is a syndicate of 39 leading European financial institutions and insurance companies who depend on cloud infrastructure and technologies to deliver innovative solutions and experiences for their customers. Having spent most of my career in the financial services industry, I know firsthand the importance of managing risk assessments for outsourced vendors to provide the necessary assurances customers need from their cloud providers. 

RSA 2021 

We have a great lineup of speaking sessions and keynotes from Googlers at RSA this year. Below are the highlights you don’t want to miss: 

• I’ll be doing a session on May 20 about supply chain resilience, where a panel of experts will dive into how we can adjust risk and security initiatives to handle the next “punch to the supply chain.” Additionally, on May 18 I’ll join many of my esteemed CISO leaders from various industries and governments for a keynote discussion on our top security insights, lessons learned and best practices for how we move forward as an industry to address the next wave of challenges. 
• Google’s Senior Director of Information Security Heather Adkins will deliver a session on how to build secure and reliable systems at scale, which will cover principles from Google’s Site Reliability Engineering book with the same title (available for free download here). I’m most looking forward to Heather’s advice for how we as an industry can reshape our security thinking, based on modern architectures and technologies that can help organizations design scalable and reliable systems that are fundamentally secure.
• Nelly Porter, Senior Product Manager at Google Cloud Security, will participate in a panel discussion with security experts on the importance of Confidential Computing technology, how it’s changing the security landscape and where it’s headed. Google Cloud has made great progress in delivering a Confidential Computing portfolio for our customers in regulated industries over the past year, and we’re excited for new milestones in 2021. 

Google cloud security highlights

• Infrastructure and SRE spotlight – Before I joined Google Cloud, I always admired the infrastructure and benefits this organization delivers that are uniquely Google – from the subsea cable innovations to SRE inventions and principles. Security and resiliency are baked into every layer of our infrastructure. Many of the Googlers who build and support our platform have sat in the same seat as our customers, so they understand those needs intimately. Over the last few months it’s been amazing to watch our technical infrastructure team grow, and the direct reliability, operational resilience and security benefits that team brings to our customers. For example, we’ve opened a new region in Poland, announced the first subsea cable that will directly connect the U.S. to Singapore with fiber pairs over an express route, and released an SRE book focused on how organizations can complete a successful cloud migration.
• New security foundations blueprint guide – As part of our mission to deliver the industry’s most trusted cloud, we strive to operate in a shared-fate model for risk management in conjunction with our customers. This includes sharing opinionated step-by-step guidance with key decision points and focus areas for how our customers deploy workloads in Google Cloud. This is why we’ve updated our Google Cloud security foundations guide and corresponding Terraform blueprint scripts. These blueprints are tremendously helpful to many stakeholders within an enterprise, like a CISO that needs to understand our key principles for cloud security, or a C-Suite business leader that needs to quickly identify the skills their teams need to meet an organization’s security, risk, and compliance needs on Google Cloud. 

When we think about the types of features to build into products, we have many principles we follow. But the two that I keep coming back to as crucial are:

1. The need for secure products not just security products. All products should have security built in and while we do build great security products our security and other teams remain focused on constantly enhancing the base levels of security and the security features in all our products. 
2. Defense in Depth. We don’t just focus on defense in depth from attacks – for ourselves and our customers. We also prioritize defense in depth from configuration errors or other hazards. 

As you see below in some of the highlights of new features and products, these represent our commitment to secure products and all forms of defense in depth. 

• Workload identity federation – Service account keys are powerful credentials, and can represent a security risk if they are not managed correctly. A safer approach is to use workload identity federation, using IAM to grant external identities IAM roles, including the ability to impersonate service accounts. This lets you access resources directly and eliminates the maintenance and security burden associated with service account keys. We also offered related overall guidance on the best way to use and authenticate service accounts on Google Cloud.
• VPC-SC Directional Policies – With VPC Service Controls (VPC-SC), admins can define a security perimeter around Google-managed services to control communication to and between those services. Using VPC-SC, you can isolate your production GCP resources from unauthorized VPC networks or the internet. But what if you need to transfer data between isolated environments that you’ve set up? VPC-SC directional policies is a new secure data exchange feature that allows you to configure efficient, private, and secure data exchange between isolated environments. 
• Anthos service mesh supports VMs as well as clusters – Most enterprise compute resources are still in VMs and many will remain there for a long time to come. In Anthos 1.7,  your VM-based workloads can now take advantage of the same mesh functionality as your container-based workloads.
• Cloud Spanner CMEK and Access Approvals – Cloud Spanner is Google Cloud’s fully managed relational database that offers unlimited scale, high performance, strong consistency across regions and high availability. Spanner now supports customer-managed encryption keys (CMEK) and Access Approval, Google Cloud’s industry-leading controls to require approval before access to your content by Google support and engineering teams.
• External Key Manager enhancements – In early 2020 we launched Cloud External Key Manager (Cloud EKM), the industry’s leading Hold-Your-Own-Key (HYOK) product. Using Cloud EKM, the keys used to protect your data stored and processed in Google Cloud are completely hosted and managed outside of Google Cloud infrastructure. Cloud EKM initially launched with support for BigQuery and GCE/PD;  we expanded support for Cloud SQL, GKE, Dataflow Shuffle, and Secret Manager, with CMEK support currently in beta. We also provided in-depth documentation on the functionality, architecture and use cases for Cloud EKM in a new whitepaper.
• Web App and API Protection solution –  Web applications and public APIs are increasingly important to how organizations interface with their customers and partners, and we’ve seen increased investment in tools to protect these resources from fraud and abuse. Google Cloud’s new Web App and API protection solution is based on the same technology Google uses to protect its public-facing services against web application exploits, DDoS attacks, fraudulent bot activity, and API targeted threats. It provides protection across clouds and on-premises environments.
• Threat Intel for Chronicle – Most threat intelligence feeds require security teams to do the implementation and legwork. With our new Threat Intel for Chronicle offering, however, our intelligence insights are applied automatically across your security telemetry to present unique observations within your environment. Threat Intel for Chronicle is exclusively curated for enterprise customers by Uppercase, Google Cloud’s intelligence research and applications team to provide our perspective on threats across the internet and surface them as relevant alerts.

That wraps up another month of thoughts and highlights. If you’d like to have this Cloud CISO Perspectives post delivered every month to your inbox, click here to sign-up, and we’ll see you in June!

Google Cloud Results

• Maximizes revenue by allowing customers to place bets faster and more confidently
• Scales for major racing events with 80% less IT involvement and up to $500,000 annual savings
• Improves time to market for new product releases by more than 30x
• Helps enable richer mobile experiences to keep fans engaged
• Processes up to 6,000 bets per minute

The first Saturday of May is the biggest horse racing event in North America each year. Minutes before the race, millions of dollars in online bets will flow in through advanced deposit wagering (ADW) operators such as TVG Network. For TVG’s IT team, it’s a high-stakes game: If wagering systems can’t handle thousands of requests per second, revenue and customers will be lost.

To avoid downtime before a major race, TVG used to bombard its systems with ad-hoc load tests a month in advance. Before each big race event, a team of seven people spent eight hours a week deploying new infrastructure and testing various scenarios. But with complex legacy systems and manual processes, the team’s efforts could only go so far. If an unexpected system issue or undetected bottleneck was found during the run-up to the big race, all bets were off.

After a brush with downtime in 2016, TVG decided to move its ADW application to the cloud, taking the opportunity to rewrite the application to take advantage of modern, container-based architectures. After a short period of development on a different cloud services provider, TVG moved to Google Cloud Platform using Google Kubernetes Engine to automate container management and orchestration.

“We chose Google Cloud Platform because it was the most reliable, cost-effective, and automated cloud solution available,” says Tim Morrow, CTO at TVG Network. “We get better security, strong compliance, and the peace of mind that when the biggest race day rolls around, we won’t have any downtime.”

Placing the right bet

Moving to Google Cloud Platform gives TVG a variety of options in different regions and availability zones to satisfy regulatory requirements. Google Cloud Platform offers continuous availability and transparent maintenance, with no scheduled downtime or patching requirements.

“For our online wagering site, we prefer Google’s philosophy of continuous availability and live migration,” says Tim. “Having to plan for scheduled downtime of cloud instances just seems ridiculous in this day and age. And with Google Cloud Platform, we get much more consistent performance as we scale.”

To keep its IT team focused on value-added tasks, TVG uses Google Cloud managed services such as Cloud Bigtable, a highly scalable NoSQL database, as well as Cloud Storage for backups and Cloud Pub/Sub for real-time messaging between applications.

“We like the software-defined nature of Google Cloud Platform,” says Saeid Vafaeisefat, Vice President of IT, TVG Network. “The managed services are so easy to use. Google Cloud Platform even helps us mitigate and absorb distributed denial of service attacks with its global load balancing features, which we don’t pay extra for.”

TVG worked with SADA Systems, a Google Cloud Premier Partner, for consulting and deployment assistance. “SADA Systems helped us gain a deeper understanding of the advantages of Google Cloud Platform so we could make better decisions about how our application would perform and scale,” says Saeid. “They provided the facilitation, follow-up, and expert advice we needed to make our deployment a success.”

Scaling with 80% less work

With an active-active cloud architecture spanning multiple regions and the ability to conduct continuous, automated load tests, TVG no longer worries about downtime during major racing events—or any time, for that matter. Infrastructure and tools that used to be required to scale and provide resiliency are no longer needed, reducing CapEx. And with autoscaling replacing human intervention, accidental downtime is much less of a concern.

“With Google Cloud Platform, we can scale for major racing events with 80% less IT involvement and up to $500,000 annual CapEx savings,” says Tim. “Google Cloud Platform gives us faster deployment—we’re releasing new enhancements to our wagering site four times a week instead of every two months.”

More profitable user journeys

TVG’s success is being driven by ongoing modernization made possible in part by Google Cloud Platform, with TVG becoming the first U.S. operator to launch native iPhone and iPad apps. Lower latency means that stale data is never an issue, allowing customers to place bets faster and more confidently from anywhere they happen to be—which generates more revenue for TVG.

“Being on Google Cloud Platform has allowed us to develop our customer-facing channel faster while focusing on automated deployment and immutable infrastructure,” says Tim. “We have far greater confidence in our platform, and last year we broke records on our major race days while delivering an excellent customer experience.”