Enterprises want to embrace the pace of innovation and an operational business model of the modern-day cloud, but they don’t want to disrupt their existing IT landscape or upgrade all their legacy applications. This presents a conundrum, as most legacy applications were not designed to run in the cloud, and migrating them can be challenging, risky and cost-prohibitive. 

The complexity, cost and risk of cloud migration often stand in the way of a successful digital transformation. At Google Cloud, we work to meet you where you are and help you craft a migration strategy that lets you run what you want, where you want, how you want. This is why we’re excited to introduce Bare Metal Solution: to jumpstart the migration of applications that have been holding back your cloud adoption. 

Bare Metal Solution consists of all the infrastructure you need to run your specialized workload such as Oracle Database close to Google Cloud. This infrastructure is connected with a dedicated, low-latency and highly resilient interconnect, and connects to all native Google Cloud services. Bare Metal Solution uses OEM hardware that is certified to run multiple enterprise applications, most of which can be migrated to this infrastructure with little or no change, minimizing the risk of migration while simultaneously increasing its velocity.

Bare Metal Solution also comes with automation tools to help you onboard your environment quickly—provisioning your applications, relational databases, configuring popular operating systems and setting up services such as backups and monitoring. The management interface will be familiar to your existing IT teams or systems integrator, allowing you to leverage your investments in existing tools, processes and personnel.

In addition to providing state-of-the-art infrastructure and integrated low-latency access to Google Cloud, Bare Metal Solution also provides the following:

• Completely managed hardware infrastructure: End-to-end infrastructure management such as compute, storage and networking, as well as fully managed and monitored environments such as power, cooling and facilities.
• Google Cloud support and billing: A seamless support experience with support for infrastructure, including defined SLAs for initial response. 24X7 coverage for all Priority 1 and 2 issues. Unified billing across Google Cloud and Bare Metal Solution. 
• Service Level Agreements: Defined enterprise-grade SLA for hardware uptime and interconnect availability.

As an integral part of Google Cloud, Bare Metal Solution lets you off-load provisioning, managing and monitoring of your infrastructure to Google, so you can focus on your own data center modernization. 

The nitty gritty 

Many of the workloads that run on Bare Metal Solution have demanding CPU and I/O requirements. Bare Metal Solution provides a cost-effective and scalable architecture based on state-of-the-art x86 servers and high-performance, resilient storage. The servers are offered in several fixed configurations that support most mainstream enterprise operating systems:

• Dual-socket x86 systems
  • 16 core with 384 GB DRAM
  • 24 core with 768 GB DRAM
  • 56 core with 1536 GB DRAM
• Quad-socket x86 systems
  • 56 core with 1536 GB DRAM 
  • 112 core with 3072 GB DRAM

Bare Metal Solution servers can be used for standalone applications, or configured with application native or database native clustering technologies for high availability. Storage is offered in 1TB volume increments of either hybrid or all-flash disk. For applications that require custom compute shapes or special-purpose hardware, we also offer bespoke hardware configurations. 

Bare Metal Solution uses OEM hardware that is certified for many ISV software applications as well as custom built applications, including those built on Oracle Database. These hardware configurations are offered as a subscription, billed monthly with a preferred term length of 36 months. There are no data ingress and egress charges between Bare Metal Solution and Google Cloud in the same region; customers are responsible for provisioning adequate bandwidth for their business needs.

Legacy apps are no barrier to cloud

Google Cloud is the preferred destination for organizations building cloud-native applications as well as migrating existing on-premises applications, bringing reliable infrastructure, leading data analytics capabilities, and a culture of innovation to your IT environment. Whether you’re looking to build new applications in the cloud or to overhaul your infrastructure, we’re here to help you reach your digital transformation goals. And now, with Bare Metal Solution, we’re excited to extend the power of Google Cloud to specialized, legacy workloads. To learn more please visit our website.

Like virtually every business across the globe, The New York Times had to quickly adapt to the challenges of the coronavirus pandemic last year. Fortunately, our data system with Google Cloud positioned us to perform quickly and efficiently in the new normal. 

How we use data

We have an end-to-end type of data platform; on one side we work very closely with our product teams to collect the right level of data that they’re interested in, such as which articles people are reading, and how long they’re staying onsite. We frequently measure our audience to understand our user segments, and how they come onsite or use our apps. We then provide that data to analysts for end-to-end analytics. 

On the other side, the newsroom is also focused on audience, and we build tools to help them understand how Google Search or different social promotions play a role in a person’s decision to read The New York Times, and also to get a better sense of their behavior on our pages. With this data, the newsroom can make decisions about information that should be displayed on our homepage or in push notifications. 

Ultimately, we’re interested in behavioral analytics—how people engage with our site and our apps. We want to understand different behavioral patterns, and which factors or features will encourage users to register and subscribe with us. 

We also use data to create or curate preferences around personalization, to ensure we’re delivering to our users fresh content, or content that they may not have normally read. Likewise, our data also gets used in our targeting system, so that we can send out the right messaging about our various subscription packages to the right users.

Choosing to migrate to Google Cloud

When I came to The New York Times over five years ago, our data architecture was not working for us. Our infrastructure was gathering data that proved harder for analysts to crunch on a daily basis. We were also hitting hang ups with how that data was streaming into our system and environment. Back then we’d run a query and then go grab some coffee, hoping that the query would finish or give us the right data by the time we came back to our desks. Sometimes it would, sometimes it wouldn’t.

We realized that Hadoop was definitely not going to be the on-premises solution for us, and that’s when we started talking with the Google Cloud team. We began our digital transformation with a migration to BigQuery, their fully managed, serverless database warehouse. We were under a pretty aggressive migration timeline, focusing first on moving over analytics. We made sure our analysts got a top-of-the-line system that treated them the way that they themselves would want to treat the data. 

One significant prominent requirement in our data architecture choice was to enable analysts to be able to work as quickly as they needed to provide high-quality deliverables for their business partners. For our analysts, the transition to BigQuery was night and day. I still remember when my manager ran his very first query on BigQuery and was ready to go grab his coffee, but the query finished by the time he got up from his chair. Our analysts talk about that to this day.

While we were doing the BigQuery transition, we did have concerns about our other systems not scaling correctly. Two years ago, we weren’t sure we’d be able to scale up to the audience we expected on that election day. We were able to band-aid a solution back then, but we knew we only had two more years to figure out a real, dependable solution. 

During that time, we moved our streaming pipeline over to Google Cloud, primarily using App Engine, which has been a flexible environment that enabled quick scaling changes and requirements as needed. Dataflow and Pub/Sub also played significant roles in managing the data. In Q4 of 2020 we had our most significant traffic ever recorded, at 273 million global readers, and four straight days of the highest traffic we’ve had compared to other election weeks. We were proud to see that there was no data loss.

A couple of years ago, on our legacy system, I was up until three in the morning one night trying to keep data running for their needs. This year, for election night, I relaxed and ate a pint of ice cream because I was able to more easily manage our data environment, allowing us to set and meet higher expectations for data ingestion, analysis and insight among our partners in the newsroom.

How COVID-19 changed our 2020 roadmap 

The coronavirus pandemic definitely wasn’t on my team’s roadmap for 2020, and it’s important to mention here that The New York Times is not fundamentally a data company. Our job is to get the news out to our users every single day in paper, on apps, and onsite. Our newsroom didn’t expect the need to build out a giant coronavirus database that would enrich the news they share every day. 

Our newsroom moves quickly, and our engineers have built one of the most comprehensive datasets on COVID-19 in the U.S. With Google, The New York Times decided to make our data publicly available on BigQuery Google’s COVID-19 public dataset. Check out this webinar for more details on our evolution architecture:https://www.youtube.com/embed/mtNlrFpschU?enablejsapi=1&

Flexible approach

We have many different teams that work within Google Cloud, and they’ve been able to pick from the range of available services and tailor project requirements keeping those tools available in mind.

One challenge we think about with the data platform at The New York Times is determining the priorities of what we build. Our ability to engage with product teams at Google though the Data Analytics Customer Council allows us to see into the BigQuery roadmap, or the data analytics roadmap, and plays a significant role in determining where we focus our own development. For example, we’ve built tools like our Data Reporting API, which reads data directly from BigQuery, in order to take advantage of tools like BigQuery BI Engine. This approach encourages our analysts to be better managers of their domains around dimensions and metrics, but not have to focus on building caching mechanisms of their data. Getting that kind of clarity helps us plan how to build The New York Times in the new normal and beyond.

If you are interested to learn more about the data teams at the New York Times, take a look at our open tech roles here and you’ll find many interesting articles at NYT data blog.

Investment management is a heavily data-driven industry—portfolio managers and investment researchers require a large number of data sources to guide them in shaping their investment strategies. 

New cloud capabilities and technologies enable investment managers to process data faster than ever before and iterate on ideas quickly to fuel innovation in the signal generation process and gain a competitive edge. 

Using the cloud for investment research workflows makes it easier to onboard data from data providers, spin up large compute workloads in the midst of market volatility or during heavy research cycles, and manage complex machine learning or natural language workflows to gain market insights.

We hear from industry leaders that they’re exploring new ways to run investment research. “Differentiated investment strategies require new types of information sources, and new ways to process that information,” David Easthope, senior analyst, Market Structure and Technology, Greenwich Associates. “And that, of course, relies heavily on having access to reliable and scalable storage, computational, and AI / ML resources. More specifically, quantitative strategies can benefit from the computational platforms and embedded AI/ML capabilities the cloud can offer.” 

Google Cloud gives investment managers essential components to work and operate faster as they bring their investment research workflows to the cloud. Here are the key highlights:

1. Simplify, speed up your data acquisition, discovery, and analytics

The foundation of any investment strategy starts with data—acquiring it, detecting patterns, and analyzing it for insights. Enabling data providers to easily share large datasets such as tick history within a high-performance analytics engine can greatly reduce the data engineering overhead when possible.

Once data is onboarded, you can tag business and technical metadata related to your datasets and provide portfolio managers the ability to discover these datasets via a search interface.

We further review analytics options for various scenarios, including aggregating massive datasets, creating dashboards, and incorporating streaming analytics workloads.

2. Take advantage of burst compute workloads

Data engineers and researchers require ready access to burst compute capabilities to perform backtesting, portfolio simulations and run risk calculations. Cloud works well for these workloads due to its elasticity, consumption-based models, and hardware evolution.

Many investment managers are shifting to a container-based strategy along with a Kubernetes-based scheduler for greater consistency, scaling and efficiency in environments with a large number of researchers. Cloud managed services and a rich suite of CI/CD tools can make this vision a reality while improving security and developer productivity.

3. Tackle machine learning (ML) and model deployment with the help from cloud

Quantitative researchers scour vast amounts of market and alternative data sources searching for signals and correlations, while ML engineers have the challenge of taking these signals and moving them to production.

Google Cloud empowers users to create and operationalize their models without wasting valuable time with a comprehensive set of MLOps tools. 

In this paper, we explore multiple solutions for ML and model deployment. Those capabilities reduce the amount of time operationalizing ML models, so quants and data scientists have more time to devote to differentiating activities. 

4. Get the data you need in less time with Natural Language and Document AI

Thousands of financial filings, news articles, and sell-side research reports are generated every day, and it’s difficult for humans alone to process this volume of information. These documents are often generated in many languages and the ability to do entity recognition, sentiment or syntactical analysis in those languages, or perhaps translate them into the language of the portfolio manager is of critical importance. Google Cloud provides these capabilities through pre-trained models, or allows you to train high-quality models with your own datasets.

Getting started

There are plenty of emerging technologies, tools, and approaches available to help investment managers today. At Google Cloud, we can help you access, organize, and utilize these essential components to make your research faster, reliable, and more valuable.

To learn more about these four keys to better investment research, check out our whitepaper for more.

Built for a CapEx world, most organizations’ finance systems aren’t set up to take advantage of cloud’s dynamic, OpEx-driven consumption patterns.

It may not be a household name yet, but chances are you’ve crossed paths with OpenX today. OpenX, a leader in programmatic advertising, operates one of the world’s largest ad exchanges, serving over 250 billion ad requests per day, connecting more than 30,000 brands and reaching nearly one billion consumers. To make it happen, in 2019, OpenX migrated entirely out of its data centers and became the first major ad-exchange platform to move completely to the cloud.

The OpenX CTO, Paul Ryan, knew that this cloud transformation initiative had the potential to increase costs faster than its revenues. To be successful, he needed his engineering, finance, and business teams to forge a new “cost-aware” culture, complete with effective cost visibility and controls. In other words, he needed Cloud FinOps — an operational framework and cultural shift that brings technology, finance, and business together to drive financial accountability and realize business benefits through cloud transformation.

Ryan laid out a cloud migration roadmap that included cost governance and controls around project ownership, established cost responsibility with engineering teams to accurately forecast cloud consumption, and challenged developers to lower per-unit costs — while at the same time improving performance, scalability, speed and global reach.

It worked! In just 9 months, OpenX reduced their per-unit cost by over 60%. The framework allowed OpenX to launch new regions in a matter of days, reduce their time to market for new features by over 50%, and complete their migration in record time — seven months! “We are now able to stop worrying about legacy infrastructure and focus more on our growth categories,” said Ryan. “Our tech stack is getting smarter and more sophisticated by the day, and we have the flexibility to scale our infrastructure in real-time as the business scales and evolves.”

Unblocking Cloud’s Potential

Cloud holds the key to a successful digital transformation. In fact, McKinsey forecasts that by 2030, the Fortune 500 alone may realize over $1 trillion of EBITDA value drivers associated with public cloud enablement. But unlike OpenX, many companies struggle to achieve near-term value objectives from their cloud investments. Surveys reflect that more than 30% of cloud spend in 2021 was wasted or inefficient, while upwards of 80% of CIOs have yet to achieve the business benefits of migrating to the cloud.

Traditional IT finance processes are ill-suited for cloud infrastructure: Traditional planning and budgeting processes are challenged to address dynamic consumption patterns and complex migrations. Centralized IT budgets using traditional allocations fail to provide the necessary visibility into sources of cost overruns. CapEx-focused cost controls have little ability to manage largely OpEx-driven spend. Trend-based forecasting often inaccurately predicts cloud costs. And developer teams lack access to cost-aware architecture patterns to deploy the applications more efficiently.

Enter Cloud FinOps

At Google we’ve worked with many companies, like OpenX, to help organizations realize the transformational benefits of the cloud by cultivating a culture of transparency and embedding agile processes to manage costs. We’ve distilled these learnings into a Cloud FinOps operational framework that gives organizations the financial governance and accountability they need to grow their business sustainably.

At a high level, a Cloud FinOps approach depends on five key areas:

1. Accountability and Enablement

Accountability and enablement aim at instilling a cost-conscious culture across the organization. Oftentimes, this means standing up a cross-functional and dedicated team with members from technology, finance and engineering to establish cloud financial best practices and governance. In various organizations, we’ve seen this through an extension of a Cloud Center of Excellence, a Cloud Business Office or simply a Cloud FinOps team. Enablement focuses on empowering IT, finance and business leaders through training to help them better understand the economics of cloud services and the strategies to efficiently deploy and manage them. Cloud financial training guides teams on how to design cost-effective cloud environments, for example, embracing ”cloud-native” design principles such as auto-scaling/elasticity and Infrastructure as a Code.

2. Measurement and Business Value Realization

Effective measurements not only create awareness and enable agile processes, but also support a culture that celebrates success and rewards teams for achieving business objectives. As such, measurement in the service of business value realization is about developing a comprehensive set of long-term benefits and cost KPIs to quantify the total net value of the return on digital transformation. Organizations often start with cost-related KPIs and eventually evolve those KPIs into business value metrics that are mapped to targeted business outcomes.

3. Cloud Cost Optimization

Cloud cost optimization is an iterative and continuous process that provides a consistent methodology to manage cloud consumption cost-effectively. There are three key areas of optimization:

Resource optimization – Model cost-effective cloud usage based on utilization and consumption patterns.
Pricing optimization – Manage cloud spend through a continuous analysis of various pricing models. In a Google Cloud context, that might mean Committed Use Discounts, BigQuery flat rate reservations, etc.
Architecture optimization – Build applications with a cost-aware architecture by leveraging newer generation compute instances (like Tau VMs, which offer an industry-leading 42% better price-performance versus comparable offerings), or using managed services and serverless technology to offload operational overhead.
For example, video hosting, sharing and services platform provider Vimeo built transcoding pipelines by using Google Cloud Spot VMs to optimize their infrastructure spend. To do so, they created fault-tolerant workloads that could withstand preemptions, and in exchange, got up to a 91% discount compared to using regular on-demand instances.

4. Planning and Forecasting

In the cloud, accurately forecasting your finances requires rethinking of traditional approaches to depreciation and trend-based forecasting of maintenance and licensing costs. One way to improve the accuracy of your dynamic cloud needs is to use workload-specific forecasting models that leverage a combination of trend-based models for steady-state workloads, driver-based models for scaling applications, as well as monthly variance analysis. In other words, you can define cloud budgets and forecasts by monitoring cloud consumption trends, allocating cloud cost pools with a proper tagging strategy that’s mapped to a chart of accounts in a general ledger, and conducting a cost-benefit analysis based on cloud infrastructure, implementation, and support costs.

5. Tools and Accelerators

Without the proper tools and processes in place, understanding and managing cloud costs can be complex — and this especially true as organizations scale their business in the cloud. By deploying proper cloud cost management tools and accelerators such as Looker Cloud Cost Management and automation scripts to set guardrails and enforce cost control policies, organizations can effectively manage and track cloud spend with access to near-real-time billing and cost data to make better informed business decisions.

The key objectives of Google Cloud Cost Management tools are to make it as simple as possible for organizations to get visibility into their current and forecasted costs with built-in reporting and customizable dashboards; help drive greater accountability for cloud spending across the organization by providing flexible ways to organize cloud resources and allocate costs; provide strong financial governance controls to reduce the risk of overspending; and offer intelligent recommendations for optimizing cloud costs and usage.

Start Saving with Cloud

Businesses are continuously seeking to better operate and manage their cloud environments and the need is ever increasing to transparently manage cloud spend, optimize costs, and obtain their desired business agility. By enhancing your Cloud FinOps capabilities and adopting principles of continuous cost optimization, you too can accelerate the business value of cloud computing.

Special thanks to Bruce Warner, Daniel Petibone, Nihar Jhawar and FinOps Foundation community for their contributions and sharing their domain expertise to this important Cloud FinOps topic.