Making Weather Predictions Easy with Weather Research and Forecasting (WRF) Models on Google Cloud!
3589
Of your peers have already read this article.
6:00 Minutes
The most insightful time you'll spend today!
Weather forecasting and climate modeling are two of the world’s most computationally complex and demanding tasks. Further, they’re extremely time-sensitive and in high demand — everyone from weekend travelers to large-scale industrial farming operators wants up-to-date weather predictions. To provide timely and meaningful predictions, weather forecasters usually rely on high performance computing (HPC) clusters hosted in an on-premises data center. These on-prem HPC systems require significant capital investment and have high long-term operational costs. They consume a lot of electricity, have largely fixed configurations, and the underlying computer hardware is replaced infrequently.
Using the cloud instead offers increased flexibility, constantly refreshed hardware, high reliability, geo-distributed compute and networking, and a “pay for what you use” pricing model. Ultimately, cloud computing allows forecasters and climate modelers to provide timely and accurate results on a flexible platform using the latest hardware and software systems, in a cost effective manner. This is a big shift compared with traditional approaches to weather forecasting, and can appear challenging. To help, weather forecasters can now run the Weather Research and Forecasting (WRF) modeling system easily on Google Cloud using the new WRF VM image from Fluid Numerics, and achieve the performance of an on-premises supercomputer for a fraction of the price. With this solution, weather forecasters can get a WRF simulation up and running on Google Cloud in less than an hour!
A closer look at WRF
Weather Research and Forecasting (WRF) is a popular open-source numerical weather prediction modeling system used by both researchers and operational organizations. While WRF is primarily used for weather and climate simulation, teams have extended it to support interactions with chemistry, forest fire modeling, and other use cases. WRF development began in the late 1990s through a collaboration between the National Center for Atmospheric Research (NCAR), National Oceanic and Atmospheric Administration (NOAA), U.S. Air Force, Naval Research Laboratory, University of Oklahoma, and the Federal Aviation Administration. The WRF community comprises more than 48,000 users spanning over 160 countries, with the shared goal of supporting atmospheric research and operational forecasting.
The Google Cloud WRF image is built using Google’s MPI best practices for HPC, with the exception that hyperthreading is not disabled by default, and is easily integrated with other HPC solutions on Google Cloud, including SchedMD’s Slurm-GCP. Normally, installing WRF and its dependencies is a time consuming process. With these new WRF VM images, deploying a scalable HPC cluster with WRF v4.2 pre-installed is quick and easy with our Codelab. OpenMPI 4.0.2 was used throughout this work. Google has had good success with Intel MPI, and we intend to study whether further performance gains can be achieved in this context.
Optimizing WRF
Determining the optimal architecture and build settings for performance and cost was a key part of the process in developing the WRF images. We evaluated how to select the ideal compiler, right CPU platform, and the best file system for handling file IO, so you don’t have to. As a test case for assessing performance, we used the CONUS 2.5km benchmark.
Below, the CONUS 2.5km runtime and cost figure shows the run time required for simulating WRF over a two-hour forecast using 480 MPI ranks (a way of numbering processes) for different machine types available on Google Cloud. For each machine type, we’re showing the lowest measured run time from a suite of tests that varied compiler, compiler optimizations, and task affinity.
We found that compute-optimized c2 instances provided the shortest run time. The Slurm job scheduler allows you to map the MPI tasks to compute hardware using task affinity flags. When optimizing the runtime and cost for each machine type, we compared using srun –map-by core –bind-to core to launch WRF, which maps each MPI process to a physical core (two vCPU per MPI rank), and srun –map-by thread –bind-to thread, which maps each MPI process to a single vCPU. Mapping by core and binding MPI ranks to cores is akin to disabling hyperthreading.
The ideal simulation cost and runtime for CONUS 2.5km for each platform is found when each MPI rank is subscribed to each vCPU. When binding to vCPUs, half as many compute resources are needed when compared to binding to physical cores lowering the per-second cost for the simulation. For CONUS 2.5km, we also found that although mapping MPI ranks to cores results in reduced runtime for the same number of MPI ranks, the performance gains are not significant enough to outweigh the cost savings. For this reason, the WRF-GCP solution does not disable hyperthreading by default.
Runtime and simulation cost can be further reduced by selecting an ideal compiler: the figure below (CONUS 2.5km Compiler Comparisons) shows the simulation runtime for the WRF CONUS 2.5km benchmark on eight c2-standard-60 instances, using GCC 10.30, GCC 11.2.0 and the Intel® OneAPI® compilers (v2021.2.0). In all cases, WRF is built using level 3 compiler optimizations and Cascade Lake target architecture flags. By compiling WRF with the Intel® OneAPI® compilers, the WRF simulation runs about 47% faster than the GCC builds, and at about 68% of the cost, on the same hardware. We’ve used OpenMPI 4.0.2 with each of the compilers as the MPI implementation in this work. With other applications, Google has seen good performance with Intel MPI 2018, and we intend to investigate performance comparisons with this and other MPI implementations.
File IO in WRF can become a significant bottleneck as the number of MPI ranks increases. Obtaining the optimal file IO performance requires using parallel file IO in WRF and leveraging a parallel file system such as Lustre.
Below, we show the speedup in file IO activities relative to serial IO on an NFS file system. For this example, we are running the CONUS 2.5km benchmark on c2-standard-60 instances with 960 MPI ranks. By changing WRF’s file IO strategy to parallel IO, we accelerate file IO time by a factor of 60.
We further speed up IO and reduce simulation costs by using a Lustre parallel file system deployed from open-source Lustre Terraform infrastructure-as-code from Fluid Numerics. Lustre is also available with support from DDN’s EXAScaler solution in the Google Cloud Marketplace. In this case, we use four n2-standard-16 instances for the Lustre Object Storage Server (OSS) instances, each with 3TB of Local SSD. The Lustre Metadata Server (MDS) is an n2-standard-16 instance with a 1TB PD-SSD disk. After mounting the Lustre file system to the cluster, we set the Lustre stripe count to 4 so that file IO can be distributed across the four OSS instances. By switching to the Lustre file system for IO, we speed up file IO by an additional factor of 193, which is orders of magnitude faster than a single NFS server with serial IO.
Adding compute resources and increasing the number of MPI ranks reduces the simulation run time. Ideally, with perfect linear scaling, doubling the number of MPI ranks would cut the simulation time in half. However, adding MPI ranks also increases communication overhead, which can increase the cost per simulation. The communication overhead is due to the increased amount of communication necessitated by splitting the problem more finely across more machines.
To assess the scalability of WRF for the CONUS 2.5km benchmark, we can execute a series of model forecasts where we successively double the number of MPI ranks. Below, we show two- hour forecasts on the c2-standard-60 instances with the Lustre file system, varying the number of MPI ranks from 480 to 1920. In all of these runs, MPI ranks are bound to vCPUs so that the number of vCPUs dedicated to each simulation increases with the increase in MPI ranks. While many HPC workloads run best with simultaneous multithreading (SMT) disabled, we find the best performance for CONUS 2.5km with SMT enabled. Thus, the number of MPI ranks in our runs equals the total number of vCPUs.
As you can see, the CONUS 2.5km Runtime & Cost Scaling figure shows that the run time (blue bars) decreases as the number of MPI ranks and the amount of compute resources increases, at least up to 1920 ranks. When transitioning from 480 to 960 MPI ranks, the run time drops, yielding a speedup of about 1.8x. Doubling again to 1920 MPI ranks, though, we obtained an additional speedup of just 1.5x. This declining trend in the speedup with increasing MPI ranks is a signature of MPI overhead, which increases with more MPI ranks.
Determining your best fit
Most tightly-coupled MPI applications such as WRF exhibit this kind of scaling behavior, where scaling efficiency decreases with increasing MPI ranks. This makes assessing cost-scaling alongside performance-scaling critical when considering Total Cost of Ownership (TCO). Thankfully, per-second billing on Google Cloud makes this kind of analysis a little bit easier. As shown above, a second doubling of the count from 960 cores to 1920 cores can provide an additional 1.5x speedup, but at a 32% higher cost. In some circumstances, this faster turnaround may be needed and worth the extra cost.
If you want to get started with WRF quickly and experiment with the CONUS 2.5km benchmark, we’ve encapsulated this deployment in Terraform scripts and prepared an accompanying codelab.
You can learn more about Google Cloud’s high performance computing offerings at https://cloud.google.com/hpc, and you can find out more about Google’s partner Fluid Numerics at https://www.fluidnumerics.com.
World’s Largest Online-only Grocery Retailer Uses AI to Figure Which Customers Need Most Attention
9487
Of your peers have already read this article.
4:20 Minutes
The most insightful time you'll spend today!
In the United Kingdom, the popularity of online grocery shopping is expected to surge from about 6% of the market today to 9% by 2021, according to market research firm Mintel. One of the pioneers of online-only grocery retailing is Ocado, based in Hatfield, Hertfordshire in the U.K. Since starting commercial deliveries in 2002, the company has grown to 600,000 active customers, 260,000 weekly orders, and £1.39 billion in annual revenue.
Ocado takes supermarket trips out of the equation by enabling shoppers to purchase items online through its convenient web and mobile applications. Items are then picked and packed in automated warehouses and shipped directly to customers in a one-hour time slot of their choosing. Ocado’s delivery punctuality is 95%, order accuracy is 99%, and its service footprint now reaches more than 70% of the U.K. population.
“Google Cloud Platform gives us the flexibility and performance to tackle the large and complex data challenges unique to our business.”
—Paul Clarke, Chief Technology Officer, Ocado
The company achieved its success by building in-house almost all the technology and automation that powers its end-to-end e-commerce, fulfillment, and logistics platform. Ocado also developed a new platform, the Ocado Smart Platform (OSP), which offers large brick-and-mortar grocery retailers around the world access to a best-in-class solution for online grocery.
Democratizing machine learning
The shopping journey for online grocery retailing differs significantly from other e-businesses. Customers often buy dozens of products at once, a single household may have multiple buyers using multiple devices, and product shelf life may only be a couple of days.
“We often say that having built an end-to-end platform that can do online grocery scalably and profitably, we can do other forms of online retail; but the reverse does not necessarily follow,” says Paul Clarke, Chief Technology Officer at Ocado. “Google Cloud Platform gives us the flexibility and performance to tackle the large and complex data challenges unique to our business.”
The Ocado business model takes advantage of consumers’ shifting preferences and the links between digital technology and shopping experiences.
“Google Cloud Machine Learning Engine gives us the agility we need. Our developers were able to try out TensorFlow and see firsthand the benefits of machine learning in the cloud.”
—Paul Clarke, Chief Technology Officer, Ocado
The company has been building machine learning into its systems for over five years. Until recently, Ocado machine learning applications required specialist data scientists, typically with PhDs in machine learning, who would build these solutions from the ground up. It also required the specialist who set up the system and costly on-premises infrastructure to train and run these systems.
However, working with Google as a private alpha testing site for Google Cloud Machine Learning Engine accelerated its adoption of artificial intelligence (AI).
“We’ve been talking about how the cloud could democratize AI for some time,” says Paul. “Google Cloud Machine Learning Engine gives us the agility we need. Our developers were able to try out TensorFlow and see firsthand the benefits of machine learning in the cloud.”
TensorFlow is an open source software library for machine learning developed by the Google Brain team. Ocado developers, engineers, and data scientists now use TensorFlow for many of their machine learning projects. They deploy the models they build on Google Cloud Machine Learning Engine, which lets them train models faster across servers, desktop computers, and mobile devices through a single application program interface (API). Additionally, Google Cloud Machine Learning Engine integrates easily with the other Google Cloud Platform products used widely at Ocado.
What do customers really want?
One of the first TensorFlow models Ocado created was a machine learning algorithm that tags and categorizes customer emails and then prioritizes them for response.
The contact center receives thousands of emails each day and Ocado wanted to automate determining which ones needed to be answered immediately and which ones could wait.
For example, a first-time customer expressing their delight in using Ocado doesn’t need to be responded to with the same urgency as a customer who is missing an item from their order or who won’t be home to receive the delivery.
“Enabling agents to respond without having to sort through less-urgent emails improves Ocado’s responsiveness and customer service.”
—James Donkin, General Manager, Ocado
“We get a lot of emails from customers saying, ‘Our service was great,’ or ‘The driver was very courteous,'” says James Donkin, General Manager, Ocado. “But when issues like weather or road conditions potentially affect delivery, we often get surges of urgent questions. Enabling agents to respond without having to sort through less-urgent emails improves Ocado’s responsiveness and customer service.”
Using Google Cloud Machine Learning Engine, TensorFlow, and a large data set culled from several years’ worth of manually categorized customer emails, Ocado experimented on which kind of neural network architecture would best prioritize emails. After testing its models, Ocado implemented the highest-performing one and has been able to respond to urgent messages four times faster. The company also discovered that 7% of its emails don’t require a response at all, which means call center representatives now have more time to devote to higher priority messages.
“Without Google Cloud Machine Learning Engine, it would have been a lot harder to succeed on a project like email classification,” says Roland Plaszowski, who has recently managed several big data projects and initiatives at Ocado.
“Even if we invested significantly in infrastructure, it would be difficult to manage because of the computational intensity. It’s challenging and expensive to run machine learning projects at the same time without infrastructure that you can scale easily.”
Ocado also uses machine learning to predict customer behavior and improve experiences. By analyzing order data, Ocado makes shopping as frictionless as possible. For example, the ordering system can pre-populate customers’ shopping carts with items they are most likely to purchase, remind customers about items they may have forgotten, and notify them of multi-buy offers they haven’t completed, for example, only buying one of a buy one, get one free offer. Based on machine learning from previous purchase data, the Ocado system can also offer new products that are likely to delight customers.
“You will regularly see items that are more personally relevant to you instead of items that are being promoted more generally,” says James. “I’m a vegetarian, so I’m offered specials for vegetarian products that I normally buy and new ones that I’ve never bought. I’m also less likely to see things that I’m not interested in.”
Machines and machine learning
Within the Internet of Things (IoT), Ocado is looking to enhance its warehouse robots with machine learning. An integral part of the OSP, thousands of robots continually stream data into Google Cloud Storage and Google BigQuery.
Ocado data scientists apply machine learning to create a type of swarm intelligence that enables warehouse robots to work cooperatively to achieve a common goal. Projects include modules to search robot telemetry data, such as whether a battery pack is operating within standard tolerances or whether firmware has been successfully loaded, and use it to optimize maintenance schedules or detect patterns in wear and tear.
“Another challenge we’re looking at is how to embed machine learning directly into robots so they become smarter in terms of self-testing, exception handling, and error recovery,” says Paul. “This is a challenging combination of IoT, data analytics, and machine learning that we believe Google BigQuery and Google Cloud Machine Learning are particularly well suited to helping Ocado achieve.”
The company also discovered that 7% of its emails don’t require a response at all, which means call center representatives now have more time to devote to higher priority messages.
Scaling for new business
Scalability is also a major reason behind some of Ocado’s cloud initiatives, including the migration of all its on-premises data to the cloud. Ocado wanted to improve customer experiences, empower business teams with greater insight, and reduce IT overhead, so it consolidated onto Google Cloud Platform.
“The old databases just weren’t fast enough,” says Paul. “We needed a solution that could scale with the amount of data we generate and how we use it. Google Cloud Storage and Google BigQuery now provide the backbone, from a data point of view, for the Ocado Smart Platform.”
Ocado estimates its business, product, and transaction data is approaching two petabytes. Combining customer and supply chain data helps both internal Ocado operations and the company’s ambitions to commercialize OSP.
“When compared with other options for expansion internationally, selling OSP as a managed service lets us turn companies that could have been competitors into customers,” says Paul. “We want to build OSP once and then turn it on for multiple business-to-business customers.”
Each time Ocado adds a new hosting customer to OSP, it will launch a customized instance to fit that customer’s requirements. The capacity and performance of each new OSP instance must be able to scale quickly as the backend platform for established retailers with large numbers of products, customers, and transactions.
Ocado’s first OSP customer, Morrisons, is already benefiting from this first-of-a kind solution. Morrisons is one of the UK’s four largest supermarkets and uses OSP to power its online retail business. Using Google Cloud Platform, Ocado has stored, processed, and analyzed terabytes of Morrisons’ data using a dedicated data lake and Google BigQuery.
In addition to using Google Cloud Platform for OSP, Ocado also adopted it for its own online grocery retail business operation. Ocado originally used the Apache Spark and Apache Hadoop open-source frameworks on Google Compute Engine for its data platform. Moving to Google BigQuery frees Ocado business analysts from the complex query setup and workflows associated with Spark and Hadoop. Plus, it lets Ocado share data analytics with suppliers and partners.
Google BigQuery is well integrated with TensorFlow on Google Cloud Machine Learning Engine and Google Cloud Dataproc, the Apache Spark and Apache Hadoop service that lets Ocado use open source data tools for batch processing, querying, streaming, and machine learning. Google Cloud Dataflow and Google Cloud Dataproc handle cluster management, and provide an easy-to-use framework so developers can spend less time and money on administration and more time on delivering valuable business features.
Switching from Hadoop to Google BigQuery revealed a series of cost and performance improvements. For example, Ocado no longer needed to decide how many instances to bring up in a cluster or wait for the instances to spin up. Google handled everything.
“We simply ran our queries and paid for the resources that we use,” adds Roland. “One big win with Google BigQuery is we don’t have to do maintenance. Best of all, we saw Google BigQuery outperform our Hadoop cluster by over 80 times on our largest dataset, and for only two-thirds the cost.”
HarbourBridge Schema Assistant Allows Quick, Bulk Migration to Cloud Spanner
5487
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
Today we’re announcing the HarbourBridge Schema Assistant, which provides a guided schema-design workflow for migrating from MySQL or PostgreSQL to Spanner. HarbourBridge imports dump files (from mysqldump or pg_dump) or directly connects to your source database, and converts the source database schema to an equivalent Spanner schema. The new Schema Assistant capability displays the source schema and Spanner schema side-by-side, highlights errors and walks you through a series of steps to validate and optimize your Spanner schema. It also produces a browsable assessment report with an overall migration-fitness score for Spanner, a table-by-table detailed analysis of type mappings and a list of features used in the source database that aren’t supported by Spanner. It supports editing of table and column names, column types, primary keys and constraints, as well as dropping of tables, columns, foreign keys and secondary indexes.
The new Schema Assistant complements HarbourBridge’s existing data and schema migration capabilities and is a critical step towards our goal of building a complete open-source migration toolkit. HarbourBridge continues to support command-line schema and data migration and turn-key Spanner evaluation.
Complementing the bulk data migration capabilities of HarbourBridge, we are also announcing the ability to migrate change events from MySQL to Cloud Spanner.
Supported Features in Schema Assistant
- Global type mapping. Users can customize the global mapping for how types should be mapped to Spanner consistently across the schema. For example, mapping large integers in source schema to Spanner’s NUMERIC.
- Local type mapping. Users can override the custom type mapping for a given table/column.
- Session management. A session keeps track of all the changes made to the schema mapping.
- Customization of secondary indexes. Users can add, edit and delete secondary indexes to optimize their Spanner performance.
- Customization of foreign keys and interleaved tables. Table interleaving is an important design consideration when migrating to Cloud Spanner as explained in more detail in this blog post.
Features in the pipeline
We are already working to further expand the supported set of schema editing features and welcome your feedback. We are particularly excited to expand the Schema Assistant’s design recommendations for optimizing Spanner schemas e.g. in-depth recommendations for primary key design.
HarbourBridge is open source and we gladly accept contributions from the wider community.
4939
Of your peers have already watched this video.
9:37 Minutes
The most insightful time you'll spend today!
L’Oréal: Managing Big-data Complexity with Google Cloud
L’Oreal is a global company with a presence in 150 countries worldwide. Between managing all of its brands and requirements for different countries, L’Oreal looks to data to make insightful business decisions. How does L’Oreal unify its data across all its systems and databases? How does L’Oreal make the data accessible to thousands of employees? In this video, Antoine Castex, Enterprise Architect at L’Oreal, discusses with Martin Omander how L’Oreal built a serverless, multi-cloud warehouse based on Google Cloud.
Chapters:
0:00 – Intro
0:23 – Why does L’Oreal need a new data warehouse?
0:51 – Who is the L’Oreal group?
1:35 – Which systems does L’Oreal use?
2:14 – How does L’Oreal manage complexity?
3:59 – What is ELT?
4:57 – Who are L’Oreal’s data consumers?
5:41 – How L’Oreal built the data warehouse
8:51 – L’Oreal’s future plans
9:10 – Wrap up
Google Cloud Workflows → https://goo.gle/3q20M1V
Cloud Run → https://goo.gle/3CSWbXG
Eventarc → https://goo.gle/3B7qhFy
BigQuery → https://goo.gle/3KHgyJ3
Looker → https://goo.gle/3Rx4Ind
Checkout more episodes of Serverless Expeditions → https://goo.gle/ServerlessExpeditions
Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech
New Enterprise Capabilities and Changes to Cloud Spanner Reduces Cost of Running Workloads by 90 Percent
5408
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
Customers love Cloud Spanner because it gives them the benefits of relational semantics and SQL while also delivering the scale and availability of non-relational databases. Many of these customers want to move even more of their work to Spanner, and have requested smaller instance sizes to support development, testing and small production workloads. We’re happy to announce that they’ll soon get what they asked for: more granular instance sizing is coming to Spanner.
Granular instance sizing will be available in Public Preview soon. With this feature, you can run workloads on Spanner at as low as 1/10th the cost of regular instances, equating to approximately $65/month.
In addition, we are launching new enterprise capabilities to break down operational silos for real-time insights and provide greater database observability to developers:
- Datastream, now in public preview, is a change data capture (CDC) service that allows enterprises to synchronize data across heterogeneous databases and applications. With Spanner support in Datastream, users will be able to stream data from MySQL or Oracle to Spanner reliably and with minimal latency.
- BigQuery federation to Spanner (coming soon) lets users query transactional data residing in Spanner, from BigQuery, without moving or copying data.
- Key Visualizer, available now in public preview, provides interactive monitoring so developers can quickly identify trends and usage patterns in Spanner.
Democratizing access with more granular instance sizing
Today, customers provision Spanner instances by specifying the number of nodes they need to run their workloads. Each node can provide up to 10,000 queries per second (QPS) of reads or 2,000 QPS of writes (writing single rows at 1 KB of data per row) and 2 TB of storage. A Spanner node is replicated across 3 zones for regional instances, and 5 or more zones for multi-regional instances. The choice of node count determines the amount of serving and storage resources that are available to the databases in a given instance.
Historically, the most granular unit for provisioning resources on Spanner has been one node. To enable more granular control, we are introducing Processing Units (PUs); one Spanner node is equal to 1,000 PUs. Customers can now provision in batches of 100 PUs, and get a proportionate amount of compute and storage resources. This will allow teams to run smaller workloads on Spanner at much lower cost. With this feature, customers can start at 100 PUs and scale up as needed in batches of 100 PUs, to up to 1,000 PUs (1 node), all with zero downtime. Subsequently, customers can continue to scale up by adding more nodes, just like what they do today. Customers do have the choice of using either PUs or nodes to provision resources within workloads, when those workloads occupy multiple nodes of capacity.
To illustrate with an example, let’s say a game developer creates a Spanner instance called “baseball-game” in us-central1, with 100 PU compute capacity at $65/month price.
As you can see below, proportional maximum storage of 205 GB is assigned to the 100 PU instance.
Once the game starts becoming popular and resource demands increase, the user edits the instance to increase the compute capacity to 500 PUs with proportional maximum 1 TB of storage.
The game grows in popularity, and the gaming company prepares to launch a much-awaited capability in the game. Anticipating a sharp increase in usage at launch day, the game developers increase the compute capacity to 3,000 PUs. Compute capacity assigned to the instance over a period of time can be viewed in Cloud Monitoring graphs:
Request Access
You can request early access to granular instance sizing feature by filling this form.
Breaking down operational silos with Datastream CDC and BigQuery federation
BigQuery Federation. BigQuery has made analytics easy by bringing together data from multiple sources for seamless analysis. Soon you’ll also be able to analyze data in Spanner directly in BigQuery. With Spanner’s BigQuery federation, you’ll be able to instantly query data residing in Spanner in real-time without moving or copying the data. Simply set up the Spanner as an external data source in BigQuery, as shown below.
Change Data Capture ingest. Now in public preview, Datastream lets you stream change data into Google Cloud from MySQL and Oracle databases. As of today, you can ingest this change data directly into Spanner using a built-in Dataflow template. This lets you migrate data from MySQL and Oracle databases into Spanner in near-real time.
Understanding performance and resource usage with Key Visualizer
Key Visualizer is a new interactive monitoring tool that lets developers and administrators analyze usage patterns in Spanner. It reveals trends and outliers in key performance and resource metrics for databases of any size, helping to optimize queries and reduce infrastructure costs. Designed for performance tuning and instance sizing, Key Visualizer is available today in public preview in the web-based Cloud Console for all Spanner databases at no additional cost. Learn more in the Key Visualizer blog.
Learn more
- To request early access to granular instance sizing in Spanner, fill out this form.
- To get started with Spanner today, create an instance or try it out with a Spanner Qwiklab.
Beany’s Cloud-Based Accounting Solutions Transform Small Business Finance
1827
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
Many people start a small business that aligns with their passions, but soon discover the day-to-day running of a business is very different than anticipated. Dealing with accounting, finance, and other daily activities can quickly overwhelm even the most promising of new businesses.
Recognizing the unique challenges facing small businesses, we founded Beany. With our personalized accounting platform, small business owners can relax knowing that we will look after all their compliance and advisory needs. Business owners who use Beany also benefit because they end up freeing more time to focus on what they enjoy and do best—likely the main reasons they started their business in the first place. We prepare financial statements, minimize tax and keep an eye that our clients are paying the right amount at the right time. We also advise on business purchase and sale, valuations and forecasting for banks. In fact, we do everything that a business owner needs from their accountant, including a help desk filled with domain experts to provide free & unlimited advice and support.
As we expand our team, we’ll continue to introduce new solutions, services, and integrations that make it even easier for small businesses to prioritize business planning, balance budgets, and keep up with ever-changing tax laws. We also plan to launch Beany in new markets and extend our global footprint beyond New Zealand, Australia, and the UK.
Migrating and scaling Beany
Beany was initially developed and hosted on a virtual private server (VPS) with limited capabilities. As our subscriber base grew, we realized we needed a more scalable, secure solution to eliminate downtime and deliver a reliable customer experience. We also understood it would be challenging for our small engineering team to cost-effectively test and deploy new features without a more agile development environment.
With that in mind, we chose the secure-by-design infrastructure of Google Cloud to power Beany and help us scale our innovative accounting platform. We now use Compute Engine to cost-effectively create virtual machines (VMs) that automatically select optimal amounts of processing and memory.
In addition, we encrypt, store, and archive everything on highly secure Cloud Storage, leveraging a combination of solid-state drives (SSDs) and hard disk drives (HDDs) for hot, nearline, and coldline data. With Cloud Storage, we can replicate data between regions in under 15 minutes to enable rapid recovery and business continuity. This has been really useful moving images to data centers on the other side of the planet quickly as we primarily operate out of Sydney and London.
We also connect, create, and collaborate on Google Workspace using Google Gmail, Docs, Calendar, and Meet. Beany smoothly integrates with Google Sheets to provide live reporting from our application direct to our sales, marketing and support staff. It also makes sharing account data with our accounting clients easy and secure.
Analyzing financial data with Google Cloud AI and machine learning
Joining the Google for Startups Cloud Program gave us immediate access to Google Cloud credits which we use to cost-effectively trial and deploy additional Google Cloud solutions. We continue to evaluate Cloud Code which will help our developers more efficiently create, deploy, and integrate applications directly on Google Cloud. We’re also exploring how Google Cloud AI and machine learning products such as Vertex AI and AutoML can further revolutionize accounting with advanced financial analyses and end-to-end automation of business processes.
We can’t wait to see what we accomplish next as we grow our team and introduce new services and solutions that enable small businesses to streamline operations, lower costs, and increase profits. It’s exciting to help small business owners manage accounting and financial planning so they can spend more time doing what they love and realize their business vision.
Beany team members
If you want to learn more about how Google Cloud can help your startup, visit our page here to get more information about our program, and sign up for our communications to get a look at our community activities, digital events, special offers, and more.
More Relevant Stories for Your Company
Mrs. T’s Pierogies Moves SAP Systems to Google Cloud for Faster Analytics Capabilities for its SAP Data
Pierogies might just be the ultimate comfort food. But when Mrs. T’s Pierogies — the leading manufacturer of frozen pierogies in the US — learned it needed to transition its existing on-premises SAP ECC to S/4HANA, the company sought a little comfort for itself. Founded in 1952, Mrs. T’s Pierogies
Know the Leaders of Google Cloud Public Sector Community
At Google Cloud, being a strategic partner is part of our DNA. Whether it’s listening closely to our customers, helping to build team skills for innovation or simply being there (since we know the cloud is 24/7), we get excited about working hands-on with customers to deliver new solutions. As
Financial Firms Can Enjoy These 8 Benefits by Migrating and Running on Google Cloud VMware Engine
The COVID-19 pandemic brought dramatic changes to the financial services industry. Already under pressure from nimble young fintechs to modernize, established banks and insurers were undergoing incremental digital transformation. But in 2020, they hit the gas pedal. Branches closed and remote work became the norm. Almost overnight, employees needed secure remote access
Macy’s Uses Google Cloud to Streamline Retail Operations
As retailers strive to meet the growing expectations of shoppers, they are turning to Google Cloud to transform their businesses and tackle opportunities in an increasingly challenging industry. From optimizing inventory management to increasing collaboration between employees across locations and roles, to helping build omnichannel experiences for their customers, we
