At Google Cloud, our customers are at the forefront of digital transformation—launching entirely new businesses and products built in the cloud, redefining entire industries with data and artificial intelligence, delivering innovative new consumer experiences, or committing to sustainable new ways of doing business.

We’re committed to our customers’ success, and over the past two years we’ve invested significantly in providing ongoing and integrated support through our customer care portfolio, including launching new Premium and Mission Critical Support offerings that allow us to monitor, prevent, and mitigate impacts quickly, while delivering the fastest response times in the industry.

Today, I’m proud to unveil several new resources and offerings for our customers, including a new Custom AI Solutions practice, new Global Delivery Centers, expanded Executive Briefing Centers, and new leaders to help continue to drive our organization forward.

Helping businesses innovate with a new Custom AI Solutions practice

We are investing in services and offerings to help our customers innovate and drive innovative change to business processes, culture, and customer experiences with Google Cloud products and services. Artificial intelligence (AI) and machine learning (ML) technologies are foundational for many such digital transformations, and to help customers create real business value with these technologies, we’re excited to launch a Custom AI offering to address our customers’ most critical innovation needs. 

AI and ML technologies are foundational to digital transformations, yet they are not “one size fits all.” Each customers’ problems and opportunities are unique; a rideshare company will use AI differently than a brick and mortar retailer, or a healthcare company, or an insurance firm. To help organizations deploy AI and ML more effectively, we’re launching a new Custom AI Solutions practice, offering customers custom-built AI and ML solutions built into Vertex AI; access to Google’s engineering expertise; and predictable, subscription-based pricing. 

Our teams are already partnering closely with early customers to build and deploy custom AI solutions. For instance, USAA, the large North American insurer, is using Google Cloud ML to process near-real-time damage estimates based on digital images to create a streamlined operations experience.

Learn more about our Custom AI Solutions practice offering here, and how we work together with our customers here. To get in touch, please contact our sales team.

Launching new Global Delivery Centers

Our drive to digitally transform our customers’ businesses is often manifested through our Global Delivery Centers, which expand our professional consulting and emerging practices capabilities available to customers and partners around the world.

The teams at our Global Delivery Centers help customers get up-and-running on Google Cloud quickly and cost effectively, and consult with customers to rapidly build capacity in areas like data analytics, hybrid and multi-cloud, artificial intelligence, and machine learning. More importantly, they help customers successfully execute projects in support of their most mission-critical business objectives. Critically, these centers also help our global partner ecosystem quickly ramp their Google Cloud practices, and get fast, expert consultation for their customers too.

In 2022, we aim to triple the size of our Global Delivery Center teams in Argentina, Poland, and India. In addition, we’ll invest in building deep Google Cloud talent in Mexico and Portugal, furthering our commitment to the industry’s best expert consultation for customers and partners around the world.

Expanding our Executive Briefing Centers footprint

In addition to our Global Delivery Centers, we’re also pleased to expand our resources and facilities that enable digital and face-to-face meetings and working sessions with our customers. This year, we will launch four new Executive Briefing Centers, located on Google Cloud campuses in London, Paris, Singapore, and Munich. 

These centers provide an opportunity to listen to our customers, share the best of Google Cloud’s solutions, and inspire digital transformation, in conversations facilitated by Google Cloud leadership, engineers, and industry experts. By bringing this experience to our customers in-region we can foster deeper partnerships and develop cloud solutions that meet their requirements for security, privacy, and digital sovereignty without compromising on functionality or innovation. 

Adding new leadership to enable customer success

Finally, I’m excited to welcome two new leaders to Google Cloud on the Customer Experience team, who will help scale our Delivery Centers and deliver exceptional experiences for our customers.

Heading our Global Delivery Center experience is Sunil Rao. Sunil comes to Google Cloud from Accenture, where he spent 18-plus years working with large technology customers across many industry verticals and managed large global teams in Accenture Advanced Technology Center. Sunil will also lead our Technical Onboarding Center, which helps businesses around the world get up to speed with technologies that are critical to understanding customer and business process contexts, and delivering great experiences.

Additionally, Lee Moore is joining Google Cloud to lead Customer Experience in North America. Lee spent nearly 30 years at Accenture in various leadership positions, including services integration for complex problem-solving, product development across a number of industry verticals, and building long-term customer relationships.

You’ll be hearing much more from us in the coming months, as we build out even more powerful and effective cloud-based services and offerings, work with customers to deliver new analytics- and AI-based tools and services, and work with our growing list of partners to help ensure customer success, across the globe.

Organizations have been moving their on-premises data and applications to the cloud for the past several years, driven by reasons as varied as application modernization and content delivery to archival. In particular, we have seen migration momentum pick up in sectors like media and entertainment, where customers are rethinking how they monetize and store their valuable historical content, often while exploring Google Cloud’s many analytical and AI solutions.

Many of these customers are interested in moving their unstructured data from on-premises appliances to Google’s Cloud Storage. Over the past year, we’ve noticed an uptick in larger migrations, where customers move tens of petabytes or more of on-premises file data to Google’s flexible, secure object storage. 

For customers like Telecom Italia/TIM Brasil, Google’s fully managed Storage Transfer Service played a key role in making this transformation possible by moving data from on-premises filesystems to extensible, low-cost Cloud Storage over the network. 

“Storage Transfer Service helped us move petabyte-scale data from on-premises filesystem to Google Cloud in a highly performant and fully-managed way,” said Auana Mattar, CIO at Telecom Italia/TIM Brasil. “Setting up the transfer pipeline required performing some tests to figure out the ideal number of agents and networking settings in our on-prem environment. Once the initial setup was done, transferring data was seamless, and the service was able to saturate a 20 Gbps Partner Interconnect link.” 

While large-scale cloud migrations can be intimidating, there are a number of actions that customers can take to ensure that a multi-petabyte data transfer goes as smoothly as possible. In the past, we’ve shared general architectural guidance for customers new to their cloud journey. And for customers looking for options, Google has multiple paths to move on-premises file data to the cloud, including our fully offline Transfer Appliance.  

In this blog post, we’ll provide an updated perspective focused on how to use Storage Transfer Service to move data from on-prem to the cloud. Specifically, we’ll look at three different factors to consider prior to moving large amounts of data from your on-premises filesystem to Cloud Storage with our Storage Transfer Service. 

Understanding the source files and filesystem

If you are moving data from an on-premises filesystem, you should be aware of how your source files, and your source filesystem, can impact transfer performance. 

Each copy you make to Cloud Storage incurs some overhead from associated operations like metadata transfer, checksumming, and encryption. This means that, for a given amount of storage, transferring large numbers of very small files will take longer. As a rule of thumb, Storage Transfer Service will be most performant when moving files that are 16 MB or larger. 

If you have a large number of smaller files, you may choose to batch them using tools like tar and upload as a single object. This will improve transfer performance but will limit how you can use those files in Cloud Storage. It’s an option best considered for use cases like archival storage, where the data transferred to Google Cloud may not be managed or accessed regularly. Our Nearline, Coldline, and Archive archival tiers offer excellent performance should you ever need to retrieve the archived data. 

The source filesystem may also slow down transfer performance, particularly if the filesystem has limited read throughput. Tools like Fio can be used to test read throughput. We’ve included a command below to run a series of 1MB sequential read operations in Fio and to generate a report:

  #Install fio
> sudo apt install -y fio

#Create a new directory fiotest
> TEST_DIR=/mnt/mnt_dir/fiotest
> sudo mkdir -p $TEST_DIR

#Test read throughput
> sudo fio --directory=$TEST_DIR --direct=1 --rw=randread --randrepeat=0 --ioengine=libaio --bs=1M --iodepth=8 --time_based=1 --runtime=180 --name=read_test --size=1G

Fio will then generate a report. The final line labeled ‘bw’ represents the total aggregate bandwidth of all threads, and it can be used as a proxy for read throughput. In general, you should strive for read throughput (‘bw’) that is 1.5x of your desired upload throughput or speed. (And one easy way to increase read throughput is to ensure that the filesystem itself is not imposing any limits on maximum throughput.)

Optimizing Storage Transfer Service resources

Within the Storage Transfer Service, there are a few settings that we can adjust ahead of time to ensure optimal performance for a larger workload.

First, we should ensure that we have the right number of transfer agents for our source data. We would advise that, for any transfer job larger than 1 GB, you start with at least three agents in separate VMs, with each agent assigned at least 4 vCPU and 8 GB of RAM. In addition to providing a foundation for performant data transfer, this architecture also ensures that the transfer is fault tolerant should one agent machine become unavailable. 

Google Cloud supports up to 100 concurrent agents for a given Google Cloud project. To help you identify the right number of agents to support your workload, you should start your larger transfer first, then wait three minutes after adding each agent to ensure that throughput has stabilized. 

In general, each agent can facilitate roughly 1 Gbps of throughput for up to 10 agents, at which point it may be necessary to add more agents for a very large amount of network bandwidth. For example, in one larger migration, a customer with 20 Gbps of dedicated network capacity ran ~30 agents at once. These numbers illustrate what was required at one enterprise data center. Across all of your environments, it is important to test and monitor your throughput via Cloud Monitoring to ensure you have the right configuration for your transfer goals. 

Another area to optimize is where and how you install your agents. As we mentioned earlier, agents should be installed in separate VMs, and each host machine should dedicate at least 4 vCPUs and 8 GB of memory per agent. This is a starting off point, and larger, long-running transfers may require additional CPU or memory. For those longer jobs, we advise that you monitor CPU utilization and unused memory closely to ensure optimal performance. You should provision more CPUs when utilization exceeds 70%. Similarly, you should be ready to provision additional memory when the agent has less than 1GB of unused memory. 

Preparing your network for large-scale data transfer

The third and final area to consider is your network connectivity. While it can be easy to reduce this to the bandwidth between the source filesystem and the Google Cloud bucket, the network includes two other components that can be easier to configure: first, the network interface from the on-premises agents to the WAN; and second, the agents’ connection to the on-premises filesystem.

For the first component, the network interface from the on-premises agents to the WAN, the general guidance is to not let this become a bottleneck. Specifically, you should ensure that this interface is greater than or equal to the bandwidth you require to read from the filesystem, plus the upload bandwidth to write to Google Cloud. In other words, if you plan on moving 10 Gbps of data from on-premises to Google Cloud, you will need 20 Gbps of bandwidth between the on-premises agents to the WAN: 10 Gbps to read from the networked filesystem, and 10 Gbps to transfer and write to Google Cloud.

On-premises filesystems, and on-premises networks, come in many flavors. For the second component, how the agents connect to an on-premises filesystem, our general rule is to be mindful of latency and to test regularly. It is essential to ensure that agents run on machines that can access a networked filesystem with very low latency. 

Finally, if you are trying to maximize transfer performance, make sure you’ve configured your network to avoid bandwidth restrictions between on-premises filesystem and Google Cloud that might impact transfer speed. Storage Transfer Service will allow you to cap the bandwidth used by transfer, making it easy to minimize any impact on other production applications. Consider using tools like lperf3, tcpdump, and gsutil to measure the network bandwidth available to upload to Cloud Storage. 

In particular, gsutil is worth some additional detail. Gsutil is a Python tool that can help you perform a number of object storage management tasks in Google Cloud, including checking your agent’s connection to the Cloud Storage APIs. It can be installed via the Google Cloud SDK, or separately. In this case, you should also ensure that gsutil is available in the same on-premises VM as the Storage Transfer Service agent. 

If you’d like to use gsutil to test connectivity to Google Cloud, here’s the command:

  gsutil cp test.txt gs://my-bucket

Replace:
my-bucket with the name of your Cloud Storage bucket.

CP is a copy command that lets you copy data from on-premises to the cloud. In this case, gsutil is copying a test document, test.txt, to ensure that you have a connection with the Google Cloud APIs. You will need to create a test document before running this command. 

Test twice, transfer once

One overarching theme across all factors is that testing can help you understand performance. For many customers, a large-scale data transfer from an on-premises filesystem to Google Cloud is an unusual event. And as with any unusual event in enterprise IT, it is a great idea to make sure that each party – from network administrators to filesystem and storage experts to cloud architects – is able to test their domain multiple times, to ensure the event will proceed seamlessly. 

As you fine tune your testing in advance of a data transfer, you may want to learn more about your options for obtaining more network bandwidth, orchestrating transfer from SMB filesystems, or even how to make the right choices to save money on object storage. And we plan to share more in our blog about how customers have used our transfer offerings in the months ahead. Advanced agent setup | Cloud Storage Transfer Service Documentation

For more information about Storage Transfer Service and how to get started, please take a look at our documentation or get started via the Google Cloud console.

Operational resilience continues to be a key focus for financial services firms. Regulators from around the world are refocusing supervisory approaches on operational resilience to support the soundness of financial firms and the stability of the financial ecosystem. Our new white paper discusses the continuing importance of operational resilience to the financial services sector, and the role that a well-executed migration to Google Cloud can play in strengthening it. Here are the key highlights: 

Operational resilience in financial services

Financial services firms and regulators are increasingly focused on operational resilience, reflecting the growing dependency that the financial services industry has on complex systems, automation and technology, and third parties. 

Operational resilience can be defined as the “ability to deliver operations, including critical operations and core business lines, through a disruption from any hazard”¹. Given this definition, operational resilience needs to be thought of as a desired outcome, instead of a singular activity, and as such, the approach to achieving that outcome needs to address a multitude of operational risks including: 

• Cybersecurity: Continuously adjusting key controls, people, processes and technology to prevent, detect and react to external threats and malicious insiders.
• Pandemics: Sustaining business operations in scenarios where people cannot, or will not, work in close proximity to colleagues and customers.
• Environmental and Infrastructure: Designing and locating facilities to mitigate the effects of localised weather and infrastructure events, and to be resilient to physical attacks.
• Geopolitical: Understanding and managing risks associated with geographic and political boundaries between intragroup and third-party dependencies.
• Third-party Risk: Managing supply chain risk, and in particular of critical outsourced functions by addressing vendor lock in, survivability and portability.
• Technology Risk: Designing and operating technology services to provide the required levels of availability, capacity, performance, quality and functionality. 

Operational resilience benefits from migrating to Google Cloud

There is a growing recognition among policymakers and industry leaders that, far from creating unnecessary new risk, a well-executed migration to public cloud technology over the coming years will provide capabilities to financial services firms that will enable them to strengthen operational resilience in ways that are not otherwise achievable.  

Foundationally, Google Cloud’s infrastructure and operating model is of a scale and robustness that can provide financial services customers a way to increase their resilience in a highly commercial way.

Equally important are the Google Cloud products, and our support for hybrid and multi-cloud, that help financial services customers manage various operational risks in a differentiated manner:

• Cybersecurity that is designed in, and from the ground up. From encryption by default, to our Titan security chip, to high-scale DOS defences, to the power of Google Cloud data analytics and Security Command Center our solutions help you secure your environment.
• Solutions that decouple employees and customers from physical offices and premises. This includes zero-trust based remote access that removes the need for complex VPNs, rapidly deployed customer contact center AI virtual agents, and Google Workspace for best-in-class workforce collaboration.
• Globally and regionally resilient infrastructure, data centers and support. We offer a global footprint of 24 regions and 73 zones allowing us to serve customers in over 200 countries, with a globally distributed support function so we can support customers even in adverse circumstances.
• Strategic autonomy through appropriate controls. Our recognition that customers and policymakers, particularly in Europe, strive for even greater security and autonomy is embodied in our work on data sovereignty, operational sovereignty, and software sovereignty.
• Portability, substitutability and survivability, using our open cloud. We understand that from a financial services firm’s perspective, achieving operational resilience may include solving for situations where their third parties are unable, for any reason, to provide the services contracted.
• Reducing technical debt, whilst focusing on great financial products and services. We provide a portfolio of solutions so that financial services firms’ technology organisations can focus on delivering high-quality services and experiences to customers, and not on operating foundational technologies such as servers, networks and mainframes.

Digital tools such as cloud computing are fueling economic transformation across the US, including Ohio. Google continues to invest across cities and communities in Ohio, bringing over 200 jobs to the state, and helping provide $12.85 billion of economic activity for tens of thousands of Ohio businesses, nonprofits, publishers, creators and developers. To further accelerate the transformation of all Ohioan businesses and technologists, we’re thrilled to announce our newest Google Cloud region in Columbus, Ohio is open. The Columbus cloud region brings a second region to the Midwest, the 10th region to North America, and grows our global cloud region count to 33.

A region for the Buckeye State

Now open to Google Cloud customers, the Columbus region (us-east5) provides you with the speed and availability you need to innovate faster, build high-performing applications, and serve local customers — all on the cleanest cloud in the industry. Additionally, the region gives you added flexibility to distribute your workloads across the central, midwest, and eastern US.

The Columbus region offers immediate access to three zones, for high availability workloads, and our standard set of products, including Compute Engine, Google Kubernetes Engine, Cloud Storage, Persistent Disk, CloudSQL, and Cloud Identity. Our private backbone connects Columbus to our global network more quickly and securely. In addition, you can integrate your on-premises workloads with our new region using Cloud Interconnect. This means that Columbus-based customers can expand globally from their front door, and those based outside the region can more easily reach their users in the Midwest.

What customers are saying

Industries including retail, financial services, and IT are investing in Columbus. Organizations across these verticals have turned to the Google Cloud to innovate faster and help solve their most complex challenges

“As Wendy’s continues to innovate in new ways to create fast, frictionless, and fun interactions that redefine the way customers visit and enjoy our restaurants, our partnership with Google Cloud is a key enabler to delivering on our AI/ML and data analytics strategies. The proximity of the new Google Cloud region to Wendy’s headquarters provides the ability for us to move and scale quickly as business needs evolve. Additionally, Google Cloud’s investment in Columbus positions central Ohio as a true technology hub, which further boosts Wendy’s and other regional employers’ ability to recruit innovative talent,” said Kevin Vasconi, Chief Information Officer, Wendy’s.

“Huntington National Bank’s API Architecture is a central component to our growth and technology strategy. As our business segments grow from an offering and geographic perspective, we must evolve our technology to provide the optimal experience for our customers and our partners. Collaborating directly with Google Cloud on the build out of their cloud region in Central Ohio, provides the access our technology teams need to innovatively scale our infrastructure to meet the demands of our business with increased availability, lower latency, and greater resiliency,” said Geoff Preston, Chief Architect, Huntington National Bank.

“Google Cloud has been instrumental in our ability to scale and optimize data management and compute resources. We prioritize scale, elasticity and resilience in cloud services and Google Cloud delivers all three globally and locally. With Google Cloud security, we can efficiently process the quantities of application data required to accelerate alert detection and reduce response times for the critical infrastructure our customers depend on to enable the continuity of their vital applications.” said Sheryl Haislet, Chief Information Officer at Vertiv, a global provider of critical digital infrastructure and continuity systems headquartered in Columbus, Ohio, that leverages Google Cloud solutions to provide resilience for its operations and to better support customers.

“The addition of the new cloud region in Ohio continues to demonstrate Google Cloud’s commitment to the enterprise space and their presence in the region,” said Chris Delong, Chief Technology Officer, Designer Brands Inc. / DSW

What’s next

We are thrilled to welcome you to our new cloud region in Columbus, and eagerly await to see what you build with our platform. Register here for our Cloud Study Jam in June – an event for local developers to get hands-on training with Google Cloud. Stay tuned for more region announcements and launches this year, including our next U.S. region in Dallas, TX. And for more information, contact sales to get started with Google Cloud today.