The Securities Technology Analysis Center (STAC®), an organization that improves technology discovery and assessment in the finance industry through dialog and research, recently audited the STAC-M3™ benchmark suite on Google Cloud (SUT ID KDB211210). These enterprise tick-analytics benchmarks assess the ability of a solution stack such as database software, servers, and storage, to perform a variety of I/O-intensive and compute-intensive operations on historical market data.

Following up on our previous STAC-M3 benchmark audit (SUT ID KDB181001), a redesigned Google Cloud architecture leveraged the most recent version of kdb+ 4.0, the time-series database from KX, and achieved significant improvements: 35 out of 41 benchmarks ran faster in the new cluster – by up to 18x faster than Google Cloud’s prior results. Key highlights include the following:

Compared to the previous STAC-M3 Antuco suite results on Google Cloud:

• Was faster in 13 of 17 mean response-time benchmarks
• Was 18x faster – a 94% reduction in run time – in the version of Year-High Bid that allows caching (STAC-M3.ß1.1T.YRHIBID-2.TIME), which also set an overall record for all published results
• Had 9x higher throughput in Year-High Bid (STAC-M3.ß1.1T.YRHIBID.MBPS)

Compared to the previous STAC-M3 Kanaga suite results on Google Cloud:

• Was faster in 22 of 24 mean response-time benchmarks
• Was over 10x faster in all four Market Snapshot workloads (STAC-M3.ß1.10T.YR[2,3,4,5]-MKTSNAP.TIME)
• Had 5x the throughput in Year-High Bid involving 2 years of data (STAC-M3.ß1.1T.2YRHIBID.MBPS)

“The STAC-M3 standard was designed by financial firms to reveal the performance of tick analytics stacks. Generational improvements like those exhibited by Google Cloud’s most recent STAC-M3 audit, are important data points for firms evaluating new architectures for performance and scale,” said Peter Nabicht, President of STAC.

These performance results may translate to real-world advantages that may be difficult for investment firms to achieve in static and costly on-premises environments: immediate answers in high data velocity markets, more thoroughly explored research theories by adding data or new quantitative approaches, and reduced costs by releasing cloud resources more quickly.

STAC-M3: High-speed tick analytics

Designing for record-breaking results

In our STAC-M3 audit, the stack under test (SUT) was designed to take advantage of horizontal scalability in the cloud by sharding data across independent compute nodes. The cluster of 12 Google Compute Engine N2 instances was powered by Intel Cascade Lake, with each node using 32 vCPUs, 160GiB of memory, and 9TiB of local NVMe SSDs. The full STAC-M3 Antuco and Kanaga data set was split across the cluster and kdb+ scripts distributed queries between nodes.

This configuration was the sweet spot for this particular workload, but this architecture does not need to be limited to 12 nodes for other workloads – the data sharding algorithm could scale to any number of nodes as required by workload demands. Since scaling out the cluster in this manner increases the total pool of available storage, this architecture can continue scaling out to petabytes of storage across hundreds of nodes.

The ability to spawn large clusters with hundreds of thousands of processors on demand at low cost, and to delete the resources when jobs complete, not only changes the economics of running computations on large financial data sets, it also opens up opportunities to explore solutions to new types of problems that were previously overlooked due to the constraints of fixed hardware on-premises. You can check the pricing of this VM configuration using the Google Cloud Pricing Calculator. The costs can be reduced even further by using preemptible VMs.

While the new cluster used a similar number of nodes, cores, and total memory as the previously-audited cluster, the redesigned architecture allowed us to harness the low latency and high throughput of Local NVMe SSDs.

Resources on demand

The cluster was created on demand using Terraform and Ansible during testing and auditing. The use of infrastructure as code (IaC) techniques ensured that the cluster, fully loaded with the STAC-M3 data set, could be created when needed and then removed when benchmarking was complete. It also meant that the cluster configuration was enforced by code on each deployment, eliminating configuration variance and drift. The full IaC definition to create the cluster can be retrieved from the report in the STAC Vault.

Each time the cluster was created, data was streamed to Local SSDs from Google Cloud Storage, our reliable and secure object storage, at up to the line rate of 32Gbps per node. The entire 57TiB STAC-M3 Antuco and Kanaga data was replicated from Cloud Storage to local storage in approximately 20 minutes.

Since each node was independent and responsible for its own shard of data, doubling the cluster size would cut the synchronization time in half, or copy twice as much data in the same amount of time. Using higher bandwidth options of up to 100Gbps would triple the possible throughput for a relatively small incremental cost, trading an approximately 11%-23% price increase at current list prices for a 200% data synchronization performance increase. Taking advantage of fast networking to cache sharded data in parallel to a large cluster makes storing bulk data in Cloud Storage viable for even the largest workloads.

For quants working on vast data sets in sprawling compute clusters, the ability to fully describe infrastructure as declarative code, create elastic resources on demand, cache data quickly from cheap bulk storage, and turn resources off when computations complete is a dramatic change compared to waiting months to grow on-premises clusters – and a compelling reason to use cloud infrastructure.

To see how we designed and optimized the cluster for API-driven cloud resources, read our new whitepaper.

STAC-A2™: Calculating derivatives risk

In 2018, we showed that cloud instances can outperform bare metal when analyzing large tick history data sets in the demanding suite of STAC-M3 benchmarks. Last year, Google Cloud’s partner Appsbroker showed that the same was true for calculating derivatives risk in STAC-A2 on Google Cloud. You can read about how Appsbroker built its record-breaking STAC-A2 compute cluster on Google Cloud in its blog post, or access the STAC Report directly. Here are the highlights:

Compared to all other publicly reported solutions, this solution, based on a cluster of 10 virtual machines, had:

• The highest throughput (STAC-A2.β2.HPORTFOLIO.SPEED)
• The fastest cold time in the large problem size (STAC-A2.β2.GREEKS.10-100k-1260.TIME.COLD)

Compared to a solution involving an 8-node, on-premises cluster (SUT ID INTC181012), this 10-node, cloud-based solution:

• Had 5 times the maximum paths (STAC-A2.β2.GREEKS.MAX_PATHS)
• Had 10% greater throughput (STAC-A2.β2.HPORTFOLIO.SPEED)
• Was 18% faster in cold runs of the large problem size (STAC-A2.β2.GREEKS.10-100k-1260.TIME)
• Was 9% faster in cold runs of the baseline problem size (STAC-A2.β2.GREEKS.TIME.COLD)

Finding market advantages with Google Cloud

Across the investment management industry, every firm is seeking many of the same competitive advantages. However, finding unique opportunities and managing larger and larger data sets is becoming a major strain. Cloud is fundamentally changing how quants tackle the problem while empowering them to manage risk and generate higher returns.

Building on-premises computing clusters with tens or hundreds of thousands of cores and petabytes of storage requires huge up-front investments and lead time measured in months or years. Google Cloud makes the same scale available to its customers, provisioned on demand and paid per use. More importantly, the elasticity of cloud resources enables agility that is simply not available in a fixed data center cluster – the agility to explore, experiment, iterate, and respond to markets faster than before.

Scaling out to tens of thousands of cores in minutes and then removing the resources immediately not only changes the speed at which questions can be answered; it encourages different and more frequent questions, asked simultaneously on many independent clusters, free from the constraints of fixed on-premises hardware.

It is this flexibility and power that enables financial services firms to leverage larger data sets and get results, backtest, research, and analyze large amounts of data, faster and whenever they need it.

Download our whitepaper to learn more about our latest STAC-M3 tick history analytics benchmark results and how to optimize cloud infrastructure for high-speed market data analysis.

Editor’s Note: In a previous blog, we discussed how home improvement retailer Lowe’s was able to increase the number of releases it supports by adopting Google’s Site Reliability Engineering (SRE) framework on Google Cloud. Lowe’s went from one release every two weeks to 20+ releases daily, helping meet its customer needs faster and more effectively. Today, the Lowe’s SRE team shares how they used SRE principles to decrease their mean-time-to-recovery (MTTR) by over 80 percent.

The stakes of managing Lowes.com have never been higher, and that means spotting, troubleshooting and recovering from incidents as quickly as possible, so that customers can continue to do business on our site. 

To do that, it’s crucial to have solid incident engineering practices in place. Resolving an incident means mitigating the impact and/or restoring the service to its previous condition. The average time it takes to do this is called mean time to recovery (MTTR). Tracking this metric helps us stay on top of the overall reliability of our systems at Lowe’s, while simultaneously improving the speed with which we recover. Our goal is to keep the MTTR metric as low as possible, so that failures don’t negatively impact our business. Here are the four areas we addressed to drive holistic improvement in our MTTR.

Lowe’s incident reporting process

To reduce MTTR, we created a seamless incident reporting process following SRE principles. Our incident reporting process is a workflow that starts at the time an incident occurs, and ends with an SRE captain who closes the action items after a postmortem report. With this approach, we are able to limit the number of critical incidents. The reporting process involves three core components: monitoring, alerting, and blameless postmortems.

Monitoring and alerting

Having proper monitoring and alerting in place is crucial when it comes to incident management. Monitoring and alerting tools let you detect issues as soon as they occur, and notify the right person in the shortest possible time to take action. From a measurement standpoint, we track this as our mean time to acknowledge (MTTA). This is the average time it takes from when an alert is triggered, to when work on the issue begins.

At the time of an incident, our monitoring and alerting tools notify the on-call SRE first responder via PagerDuty in the form of a phone call, text message and email. Our SRE software engineering team has done a lot of automation to enable various Service Level Indicator (SLI) alerts and Service Level Agreement (SLA) notifications. The on-call SRE then initiates a triage call with our service/domain stakeholders to resolve the incident. As a result, we reduced our MTTA from 30 minutes in 2019, to one minute – a 97 percent decrease. 

Blameless postmortems: learning from incidents

A postmortem is a written record of an incident, its impact, the actions taken to resolve it, the root cause and the follow-up actions to prevent the incident from recurring (see example here). A blameless postmortem builds on that and is a core part of an SRE culture, and our culture at Lowe’s. We ensure that individuals are not singled out, and the outcome for all postmortems are directed toward learnings and process improvement.

For us, the postmortem process is the biggest part of our incident workflow. When an SRE creates a new postmortem report, the first step is to conduct a postmortem session with domain stakeholders to review the report. The postmortem then goes into the review stage and gets reviewed by more stakeholders in our weekly postmortem meeting. In the final stage of this process, the SRE captain will close the report once everyone in the weekly meeting agrees that the report is complete.

To conduct a successful postmortem, it is critical to keep the focus on identifying gaps and issues with the system and operations processes, rather than an individual, and generate concrete actions to address the problems we’ve identified. To ensure this, we follow a couple of best practices:

1. We start by gathering the facts from the person who identified the problem, and each SLI owner has to identify a gap or the next SLI upstream owner who created the impact for them.
2. Every SLI owner is provided full opportunity to present their case, and identifying the issue is done as a community exercise. 
3. Once action items and process changes are identified, an owner is nominated to complete the actions, or they will volunteer. 
4. For easy reference, we publish and store postmortems in our incident knowledge base. This process helps SREs continuously improve as future incidents arise. 

Continuous Improvement 

Encouraging a culture of honest, transparent and direct feedback that you need for blameless postmortems is often an iterative process that needs sponsorship from executives, empowering incident captains to lead the entirety of the discussion and outcomes. Running successful postmortems, and completing action items from them, needs to be recognized and accounted for in SRE performance objective assessment. As shared in Google’s SRE book, the best practice is to ensure that writing effective postmortems is a rewarded and celebrated practice, with leadership’s acknowledgement and participation. This is possibly the hardest part to accomplish in an effective postmortem during a cultural transformation unless you have full buy-in from leadership.

However, it’s all well worth it. This process is a key part of how we were able to improve our MTTR over time—from two hours in 2019 to just 17 minutes! 

Our SRE incident reporting process has also transformed how our company solves issues. By streamlining this workflow from alerting, to solving an issue, to blameless postmortems, we have reduced our MTTR by 82 percent and our MTTA by 97 percent. Most importantly, our team is learning from every incident and becoming better engineers as a result. Visit the SRE Google Cloud website to learn more about implementing SRE best practices in the cloud.

Acknowledgement

Special thanks to Rahul Mohan Kola Kandy, Vivek Balivada, and the Digital SRE team at Lowe’s for contributing to this blog post.

As the saying goes, “it’s hard to make predictions, especially about the future.” Some organizations find it challenging to predict what cloud resources they’ll need in months or years ahead. Every organization is on its own unique cloud journey. To help, we’re developing new ways for customers to consume and pay for Google Cloud services. We’re doing this by removing barriers to entry, aligning cost to consumption and providing contractual and product flexibility. Read on to learn how we’re rolling out several new go-to-market programs across these key areas to help our customers purchase and consume Google Cloud services more easily.

Removing barriers to entry with Google Cloud Flex Agreements

Many customers choose multi-year commitments because they provide better line-of-sight into IT spend and budgeting. However, these commitments can create difficulty for those who don’t have clear visibility into their future cloud consumption needs. That’s why today we’re launching Flex Agreements, which enable customers to migrate their workloads to the cloud with no up-front commitments. As part of this new licensing option, Google Cloud customers still get access to unique incentives, such as monthly spend discounts¹, committed use discounts, cloud credits, and access to professional services, based on monthly spend and workloads migrated to Google Cloud.

Flex Agreements are just one example of how we are removing barriers to help customers start using Google Cloud. In 2022, we launched the Innovators Plus annual subscription, which gives developers a curated toolkit to accelerate their expertise, including access to live and on-demand training through Google Cloud Skills Boost, Google Cloud credits, and more. 

We also recently expanded trials for Google Cloud products. For example, the new Spanner free trial instance is good for 90 days, allowing developers to create Google Standard SQL or PostgreSQL databases, explore Spanner capabilities, and prototype applications—with no commitment or contract needed. 

Contractual and feature flexibility  

Contractual flexibility has always been one of our core principles. Committed Use Discounts (CUDs), for example, provide discounted prices in exchange for a commitment to use a minimum level of resources for a specified term. Last year, we introduced Flexible CUD, spend-based commitments that offer predictable and simple flat-rate discounts that apply across multiple virtual machine families and regions.

In addition to contractual flexibility, our customers also need the flexibility to choose features and functionality based on their stages of cloud adoption and the complexity of their business requirements. Therefore, over the next few quarters, we will launch new product pricing editions—Standard, Enterprise, and Enterprise Plus—in parts of our cloud portfolio. This new commercial packaging model will help give customers more choice and flexibility to optimize their cloud spend.

For customers running workloads such as those in regulated industries like banking and public sector, the higher-end Enterprise Plus tier will offer compute, storage, networking and analytics services with high availability, multi-region support, regional failover and disaster recovery, advanced security, and a broad range of regulatory compliance support. The Enterprise pricing tier will include a broad range of features designed for customers with workloads that demand a high level of scalability, flexibility, and reliability. The Standard pricing tier will offer cost-efficient and easy-to-use managed services that include all essential capabilities such as autoscaling to meet the core workload requirements of customers.

Align costs to consumption with autoscaling

At Google Cloud, a core requirement for the products we build is providing customers industry-leading capabilities to automatically scale (autoscale) services up and down to match capacity with real-time demand. Autoscaling improves uptime, reduces infrastructure costs, and removes the operational burden of managing resources.  

Many Google Cloud products include autoscaling capabilities to help customers manage unplanned variations in demand. For example, Dataflow vertical and horizontal autoscaling, in combination with granular adaptive resource configuration (aka “right-fitting”), has resulted in up to 50% saving in infrastructure costs for streaming by automatically choosing the right number of instances required to run the jobs and dynamically re-allocating more or fewer instances during the runtime of jobs. Bigtable also provides native autoscaling capabilities, and Spanner’s autoscale is an open source tool that works across regional and multi-regional Spanner deployments. 

Similarly, we added multiple features such as Cluster Autoscaler, Horizontal Pod Autoscaling, Vertical Pod Autoscaling, and Node Auto-Provisioning to GKE for elasticity and cost efficiency. 

For L.L.Bean, the ability to quickly scale capacity to meet changing usage patterns (e.g., during the holidays), as well as to rapidly perform load tests to test capacity, are “night and day” with Google Cloud compared to L.L.Bean’s legacy on-premises IT system.

“We won’t have to pay for peak capacity to have it available during peak shopping times. We just scale capacity up or down as needed.” — Randy Dyer, Enterprise Architect, L.L.Bean

We are now taking these capabilities to the next level by enabling autoscaling in BigQuery at a more granular level so you never pay more than what you use. This allows you to provision additional capacity in smaller increments, so you never overprovision and overpay for underutilized capacity. BigQuery customers can now try the new BigQuery autoscaler (currently in public preview) in their Google Cloud console.

A commitment to flexibility and choice

At Google Cloud, we remain deeply committed to the success of our customers and partners, and we are uniquely positioned to help organizations transform their business. By providing you with more flexibility and choice in how to purchase our products, we are empowering you to be more efficient and resilient.

Join Google Data Cloud & AI Summit to hear the latest announcements around innovations in Google Data Cloud for databases, data analytics, business intelligence, and AI. Gain expert insights, new solutions, and strategies that can help you transform customer experiences with modern apps, boost revenue, and reduce costs.

^{1. Not available for customers buying through Partner Advantage.}

The past year has given everyone lots to think about—about our priorities as people and as businesses. As the world retreated behind closed doors, we saw how shared interests and experiences can bring us together. As the world grappled with a common enemy, we witnessed just how differently individuals, communities and indeed entire countries can experience a situation. And as we faced seemingly unending obstacles to making it through the pandemic, we saw how making smart decisions based on data can drive meaningful solutions—fast.

That’s why we here at Google Cloud are so proud to partner Univision, the country’s leading Spanish-language content and media company. By partnering with Google Cloud, Univision will be able to accelerate growth across its portfolio of properties, deliver an enhanced user experience for Spanish-speaking audiences and provide the enterprise solutions needed to create the Spanish-language media company of the future.

According to Instituto Cervantes, there are over 580 million Spanish language speakers worldwide. Those viewers, like people everywhere, are avid consumers of streaming content. In Q4 of 2020 alone, viewing time for that content increased by 44%¹, and in 2020, from 50%² more sources. With that surge in demand, Univision needed a cloud provider whose infrastructure could reach Hispanic viewers around the world. With two-plus decades spent building out its network and data centers, as well as global content-delivery capabilities, Google Cloud has the infrastructure Univision needs to reach viewers across the Spanish-speaking world.

At the same time, with such a diverse audience for their content, Univision needs to target that content to viewers’ specific preferences. By applying Google Cloud’s artificial intelligence (AI) and machine learning (ML) technology across its content, Univision intends to personalize content based on shows users have previously watched, enhancing their engagement and viewing experience. 

And as Univision transforms the user experience, it can use Google Cloud’s data and analytics suite to garner deeper insights into its audience and forge stronger relationships with them on an individual basis. With Looker and BigQuery, Univision employees will have access to real-time data to help them make business decisions about programming.

Univision will also migrate video distribution and production operations to Google Cloud, where we’ll help them streamline media workflows and develop innovative new capabilities. Meanwhile, Google Cloud’s tight business and technical integration with other Google services will help ensure Univision reaches viewers on the device of their choice, wherever they are in the world. For example, in the coming years, Univision will expand its global YouTube partnership and will integrate with entertainment features on Google Search that help people better discover TV shows and movies. The company will also use Google Ad Manager for global ad decisioning and Google’s Dynamic Ad Insertion for PrendeTV and future video-on-demand offerings. Finally, Univision will distribute its content and services on Google Play across Android phones and tablets, as well as Google TV and other Android TV OS devices.

We’re thrilled to partner with Univision to help them reach the Spanish-speaking world with their content. With our cloud portfolio, we can help them reach individual viewers around the world, with personalized content that they can consume however they see fit. Best of all, together, we can help them achieve this vision fast, leveraging established cloud, content delivery, and data analytics technologies. You can learn more about the partnership here.