For the fourth consecutive year, Gartner has positioned Google as a Leader in the 2021 Gartner Magic Quadrant for Cloud Infrastructure and Platform Services (formerly titled as Magic Quadrant for Cloud Infrastructure as a Service upto 2014 Infrastructure as a Service, or (IaaS).

With our customers and communities adjusting to new ways of working and doing business, Google Cloud has remained focused on building services and platforms that help you be more resilient and derive even more value from your cloud infrastructure. We believe Gartner’s analysis and recognition gives our customers the confidence needed to choose Google as the platform for customer-centric innovation. Here are just a few recent examples.

Ready for the most demanding, mission-critical workloads

Our enterprise-ready cloud provides you the uptime, performance, and scale to run even your most demanding workloads. Examples of recent launches: 

• The largest single-node GPU-enabled VM in the industry with up to 16 NVIDIA A100 instances so that our customers can run their ML workloads 
• The only cloud to support scale-out out 96TB SAP HANA so that customers can confidently bring their most critical workloads to GCP
• Strategic partnerships with leading partners like SAP 
• Several regions and an expanded global network footprint including new subsea cables, Firmina, Dunant, Blue and Raman
• High bandwidth 50/75/100Gbps networking for VMs
• Persistent Disk Extreme (block storage) with 120K IOPS 
• Filestore High Scale scale-out NFS for HPC

Saves you money

Save money with a transparent and innovative approach to pricing and intelligent recommendations. In the past year, we’ve launched several innovations to help you save costs:

• Tau VMs, which offer the best price-performance among leading clouds for scale-out workloads 
• Machine-learning-driven predictive auto-scaling for VMs and GKE Autopilot, enabling infrastructure to scale up and down as needed with minimal waste 
• Standard network tier which routes traffic over the internet for cost optimization 

Open

We have a long history of leadership in open technologies—from projects like Kubernetes, the industry standard in container orchestration and interoperability, to TensorFlow, a platform to help anyone develop and train machine learning models. Here are a few recent improvements we’ve made to ensure your cloud is an open cloud: 

• Extended Anthos to bare metal and Microsoft Azure to support customers who want a multi-cloud and hybrid cloud posture. 
• Announced a new network dataplane for Google Kubernetes Engine (GKE) and Anthos that supports eBPF, an open-source Linux kernel technology optimized for Kubernetes. 
• Google Kubernetes Engine (based on the Kubernetes standard) received the top overall score based on 2021 Gartner Solution Scorecard for Google Kubernetes Engine.

Secure

Google Cloud’s trusted infrastructure uses layers of security to protect your data with advanced technologies and operations, keeping your organization secure and compliant. For example, we offer:

• Confidential VMs and Confidential GKE with in-memory encryption and encryption keys controlled by you, with a single checkbox
• Enhanced security for Cloud Run
• Strong support against DDoS attacks. In 2017, our infrastructure absorbed the largest-known DDoS attack at 2.5Tbps with no impact to customers. 

Sustainable

Google Cloud helps customers transform their business sustainably. We operate the cleanest cloud in the industry to make sure your digital footprint doesn’t leave a carbon one. Here are a few proof points:

• Google has been carbon neutral since 2007, and for the past four years has matched 100% of the electricity we consume globally with wind and solar purchases. Everything you run on Google Cloud is net carbon neutral. 
• We continue to innovate towards greater energy efficiency in our data centers, and compared with five years ago, now deliver around seven times as much computing power with the same amount of electrical power.
• Recently we announced new features to help customers reduce the carbon footprint of their applications and infrastructure, including a region picker to help with architecture decisions, and low carbon indicators in the Google Cloud Console.  

Supporting our customers

Most importantly, our field organizations and partner organizations work with a singular focus to ensure customer success. This has made Google Cloud the fastest growing hyperscaler, with a rapidly expanding customer base across all geos and industries.

Since launching Customer Care last year, we consolidated and simplified the post-sales engagement with customers, increased the support channels, created an API to allow programmatic case creation, and combined product specific support into a single package for all of Google Cloud. Enterprises with Customer Care continue to report high levels of satisfaction with their focused technical account managers (TAMs), helping them get the most business value out of Google Cloud.

We are committed to sustaining and accelerating the pace of customer-centric innovation. You can download a complimentary copy of the 2021 Magic Quadrant for Cloud Infrastructure and Platform Services on our website. 

Join us to learn much more about Google Cloud at the upcoming Google Cloud Next ‘21 digital conference.  

^{Gartner, Magic Quadrant for Cloud Infrastructure and Platform Services,  Raj Bala | Bob Gill | Dennis Smith | Kevin Ji | David Wright, 27 July 2021
Gartner, Solution Scorecard for Google Kubernetes Engine,  Tony Iams | Traverse Clayton | Megan Bain, 12 April 2021
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.}

With a global, always-on workforce, the U.S. Navy requires secure collaboration between teams across countries and time zones. This is especially relevant for the 50,000 U.S. Navy sailors deployed aboard approximately 100 ships at any given time, who need to connect with personnel at regional shipyards for everything from routine maintenance, to more serious ship repairs.

Google Public Sector directly assists Naval Sea Systems Command (NAVSEA), the largest of the U.S. Navy’s five “systems commands,” by providing support to the U.S. Naval Ship Repair Facility and Japan Regional Maintenance Center (SRF-JRMC) in Yokosuka and Sasebo, Japan. NAVSEA is using Google Workspace, which harnesses Google’s threat protection and zero trust capabilities, to enable effective, secure, and compliant collaboration between the SRF-JRMC and Navy stakeholders around the world.

The workforce in the SRF-JRMC facility had two unique challenges. The first was around ineffective collaboration and communication channels. Before deploying Google Workspace, a Navy officer in Japan would need to be on-base to communicate over a secure connection, and calls were typically late at night given the time difference. This challenge was even further magnified when the COVID-19 pandemic hit, further restricting staff mobility.

The second challenge was a language barrier. With more than 3,000 Japanese Master Labor Contract (MLC) employees providing critical support to SRF-JRMC’s shipyard, Navy leadership needed an easier way to communicate with their Japanese-speaking counterparts. Breaking down this barrier would ensure work could be completed faster and more effectively, facilitating a more collaborative environment.

Enabling secure collaboration

The Navy partnered with Google Public Sector to enable Google Workspace for SRF-JRMC. Today, Google Workspace assists the Navy’s shipyards by enabling Google Voice for secure voice over Internet Protocol (VoIP) calling options, so personnel can join calls and sessions on-site or at-home to communicate securely and reliably across continents. In addition, early access to solutions like English to Japanese translated captions in Google Meet help break down the language barrier by providing instant translation during meetings. The platform also saves the U.S. Navy thousands of dollars a month in phone bills by providing country-specific dial-in numbers for interviews.

Moreover, Google Workspace tools like Google Drive and Docs also help to simplify human resource workflows by streamlining the hiring for onboarding local Japanese employees for the shipyard. Having a shared Google Drive eliminates the need to send multiple files back and forth among the NAVSEA team, allowing them to minimize on-premises storage space. In addition, Google Docs enables Navy employees to communicate and collaborate with each other and with potential candidates securely, and across any device.

“As the largest overseas ship repair facility of the U.S. Navy, operational readiness and continuity of operations is our top priority,” said Peter Guo, chief information officer at SRF-JRMC. “Cloud collaboration capabilities provide us seamless and secure connectivity across continents and break down language barriers with our colleagues across the globe. We’ve improved our ability to operate anytime, and anywhere and have increased our ability to securely communicate and coordinate especially during network outages and natural disasters.”

Providing pandemic assistance

Google for Government’s Workspace solutions also became useful to the SRF-JRMC during the pandemic, providing valuable communication and collaboration tools during a time of uncertainty. In addition to deploying Google Workspace, SRF-JRMC’s IT department created a COVID-19 Pandemic Dashboard, a Google-based site that consolidated Japanese and international open-source data on COVID-19 outbreaks and provided updated guidance. The Dashboard was built in less than two hours, using Looker Studio, and it leveraged an automated data collection process to track local hospitalization numbers. Before building the site, it took the NAVSEA team hours to compile this information; now, the team can access this information in real-time.

“This Dashboard drastically reduced redundant weekly meetings centered on COVID-19 updates, and saved more than 10 hours per week manually gathering data and presenting it via slide decks,” said Guo. “Additionally, this capability enabled our leadership to make real-time, data-driven decisions and put necessary risk mitigations in place. It empowered supervisors across the shipyard to reference this website at any time and put additional health measures in place to minimize the transmission of COVID-19. Every minute counts at our two shipyards in Japan, so this made a tremendous impact on our operational efficiency and ensured the safety of our sailors.”

Delivering the best possible tools means making life better and work more fulfilling for millions of people, inside and outside of government. To help government agencies maintain access to communications and collaboration tools that they need during and after an incident to keep work going, we are also offering workshops for federal, state and local governments. Learn more about Google for Government solutions for the Department of Defense, and Google Workspace for Government.

Google Cloud VMware Engine delivers an enterprise-grade, cloud-native VMware experience that is built on Google Cloud’s highly performant and scalable infrastructure. By enabling a consistent VMware experience, the service allows customers to adopt Google Cloud rapidly, easily, and with minimal modifications to their vSphere workloads, bringing the best of VMware and Google Cloud together on one platform for a variety of use-cases. These include rapid data center exit, application lift and shift, disaster recovery, virtual desktop infrastructure, or modernization at your own pace.

Here are seven ways VMware Engine outshines alternatives for running your VMware workloads in the cloud, simplify your operations, and help you innovate faster:

1. Dedicated 100Gbps east-west networking
   Google Cloud VMware Engine nodes come with redundant switching and dedicated 100Gbps east-west networking with no oversubscription of bandwidth, unlike other options where there is generally oversubscription. This is especially important when it comes to running latency-sensitive workloads.
2. Four 9’s of availability in a single zone
   The service offers 99.99% uptime SLA for a cluster in a single Zone with five to 16 nodes and FTT=2 or more without the need for stretched clusters, which is higher than the alternatives. Further, dedicated connectivity for core service functions such as vSAN and vMotion enables better solution stability and availability. This enables the service to support the needs of enterprise workloads that require high availability.

Note: “Cluster” means a deployment of three or more dedicated bare metal nodes running VMware ESXi and associated networking managed via management interfaces.

3. Global networking without complex routing
   Google Cloud VMware Engine networking is built based on Google Cloud’s powerful networking architecture. With simplified regional and global routing modes—which allow a VPC’s subnets to be deployed in any region where our service is available—you can architect global networks without the need or overhead of creating and connecting regional network designs. You get instant, direct Layer 3 access between them. In alternative cloud environments, you may have to configure special networking between regions, often requiring VPN-based tunnels over the WAN to enable global uniform network communication. This adds to the deployment and operational complexity, in addition to cost.
4. Integrated multi-VPC networking
   Users often have application deployments in different VPC networks, such as separate dev/test and production environments or multiple administrative domains across business units. The service supports “many-to-many” access from VPC networks to Google Cloud VMware Engine networks with multi-VPC networking, allowing you to retain existing deployed architectures and extend them flexibly to your VMware environments. In addition, by providing multi-VPC networking, you can pool their VMware needs—say for QA and dev—to a smaller set of clusters, effectively reducing their costs.

For more information about the end-to-end networking capabilities and services available in Google Cloud VMware Engine, please refer to the Private Cloud Networking for Google Cloud VMware Engine whitepaper. Here, you’ll find details about network flows, configuration options, and the differentiated benefits of running your VMware workloads in Google Cloud.

5. Unified, cloud-integrated model
   Google Cloud VMware Engine is a fully managed Google first-party service, operated and supported by Google and its world-class team. With fully integrated identities, billing, and access control, you have a simpler end-to-end experience that is different from other services. You access Google Cloud VMware Engine service via the Google Cloud console, like any other Google Cloud service. You can also access other native Google Cloud services privately from your VMware private cloud running in Google Cloud VMware Engine over local connections.
6. Flexibility in third-party ecosystem compatibility
   With Google Cloud VMware Engine, you can set up existing VMware on-premises third-party tools or products that require additional privileges by using a solution user account. This uniquely enables operational consistency, ensuring that the tools you have invested in and used over the years work on Google Cloud VMware Engine. Furthermore, in key areas such as vSAN data encryption, you have the choice of not only using Google Cloud Key Management Service (KMS)—which is turned on by default on vSAN datastores—but also external KMS providers such as HyTrust, Thales, and Fortanix.
7. Dense nodes with high storage:core and memory:core ratios and fast provisioning
   Google Cloud VMware Engine nodes are dense. Each node is powered by Intel® Xeon® Scalable Processors and comes with 36 cores, 72 hyperthreaded cores, 768 GB memory, 19.2 TB NVMe data and 3.2 TB NVMe cache storage. This, along with oversubscription, leads to high consolidation ratios and compelling storage:core per dollar and memory:core per dollar. In addition, you can rapidly spin up these nodes in a VMware private cloud often in under an hour, enabling on-demand, VMware-consistent capacity in Google Cloud for your needs.

These are just a few examples of customer-centric innovation that set Google Cloud VMware Engine infrastructure apart. In addition, migrating to Google cloud can save you up to 38% in TCO. Get started by learning about Google Cloud VMware Engine and your options for migration, or talk to our sales team to join the customers who have embarked upon this journey.

The authors would like to thank the Google Cloud VMware Engine product team for their contributions on this blog.

Data teams across companies have continuous challenges of consolidating data, processing it and making it useful. They deal with challenges such as a mixture of multiple ETL jobs, long ETL windows capacity-bound on-premise data warehouses and ever-increasing demands from users. They also need to make sure that the downstream requirements of ML, reporting and analytics are met with the data processing. And, they need to plan for the future – how will more data be handled and how new downstream teams will be supported?

Checkout how Independence Health Group is addressing their enterprise data warehouse (EDW) migration in the video above.

Why BigQuery?

On-premises data warehouses become difficult to scale so most companies’ biggest goal is to create a forward looking system to store data that is secure, scalable and cost effective. GCP’s BigQuery is serverless, highly scalable, and cost-effective and is a great technical fit for the EDW use-case. It’s a multicloud data warehouse designed for business agility. But, migrating a large, highly-integrated data warehouse from on-premise to BigQuery is not a flip-a-switch kinda migration. You need to make sure your downstream systems dont break due to inconsistent results in migrating datasets, both during and after the migration. So..you have to plan your migration. 

Data warehouse migration strategy

 The following steps are typical for a successful migration: 

• Assessment and planning: Find the scope in advance to plan the migration of the legacy data warehouse 
  • Identify data groupings, application access patterns and capacities
  • Use tools and utilities to identify unknown complexities and dependencies 
  • Identify required application conversions and testing
  • Determine initial processing and storage capacity for budget forecasting and capacity planning 
  • Consider growth and changes anticipated during the migration period 
  • Develop a future state strategy and vision to guide design
• Migration: Establish GCP foundation and begin migration
  • As the cloud foundation is being set up, consider running focused POCs to validate data migration processes and timelines
  • Look for automated utilities to help with any required code migration
  • Plan to maintain data synchronization between legacy and target EDW during the duration of the migration. This becomes a critical business process to keep the project on schedule.
  • Plan to integrate some enterprise tooling to help existing teams span both environments
  • Consider current data access patterns among EDW user communities and how they will map to similar controls available in Big Query. 
  • Key scope includes code integration and data model conversions
  • Expect to refine capacity forecasts and refine allocation design. In Big Query there are many options to balance cost and performance to maximize business value. For example, you can use either on-demand or flat-rate slot pricing or a combination of both. 
• Validation and testing
  •  Look for tools to allow automated, intelligent data validation 
  • Scope must include both schema and data validation
  • Ideally solutions will allow continuous validation from source to target system during migration
  • Testing complexity and duration will be driven by number and complexity of applications consuming data from the EDW and rate of change of those applications 

A key to successful migration is finding Google Cloud partners with experience migrating EDW workloads. For example, our Google Cloud partner Datametica offers services and specialized Migration Accelerators for each of these migration stages to make it more efficient to plan and execute migrations.

Data warehouse migration: Things to consider

• Financial benefits of open source: Target moving to ‘Open Source’ where none of the services have license fees. For example BigQuery uses Standard SQL; Cloud Composer is managed Apache Airflow, Dataflow is based on Apache Beam. Taking these as managed services provides the financial benefits of open source, but avoids the burden of maintaining open source platforms internally. 
• Serverless: Move to “serverless” big data services. The majority of the services used in a recommended GCP data architecture scale on demand allowing more cost effective alignment with needs. Using fully managed services lets you focus engineering time on business roadmap priorities, not building and maintaining infrastructure. 
• Efficiencies of a Unified platform: Any data warehouse migration involves integration with services that surround the EDW for data ingest and pre-processing and advanced analytics on the data stored in the EDW to maximize business value. A cloud provider like GCP offers a full breadth of integrated and managed ‘big data’ services with built-in machine learning. This can yield significantly reduced long-term TCO by increasing both operational and cost efficiency when compared to EDW-specific point solutions. 
• Establishing a solid cloud foundation: From the beginning, take the time to design a secure foundation that will serve the business and technical needs for workloads to follow. Key features include: Scalable Resource Hierarchy, Multi-layer security, multi-tiered network and data center strategy and automation using Infrastructure-as-Code. Also allow time to integrate cloud-based services into existing enterprise systems such as CI/CD pipelines, monitoring, alerting, logging, process scheduling, and service request management. 
• Unlimited expansion capacity: Moving to cloud sounds like a major step, but really look at this as adding more data centers accessible to your teams. Of course, these data centers offer many new services that are very difficult to develop in-house and provide nearly unlimited expansion capacity with minimal up-front financial commitment. . 
• Patience and interim platforms: Migrating an EDW is typically a long running project. Be ready to design and operate interim platforms for data synchronization, validation and application testing. Consider the impact on up-stream and down-stream systems. It might make sense to migrate and modernize these systems concurrent with the EDW migration since they are probably data sources and sinks and may be facing similar growth challenges. Also be ready to accommodate new business requirements that develop during the migration. Take advantage of the long duration to have existing your operational teams learn new services from the partner leading the deployment so your teams are ready to take over post-migration. 
• Experienced partner: An EDW migration can be a major undertaking with challenges and risks during migration, but offers tremendous opportunities to reduce costs, simplify operations and offer dramatically improved capacities to internal and external EDW users. Selecting the right partner reduces the technical and financial risks, and allows you to plan for and possibly start leveraging these long-term benefits early in the migration process.

Example Data Warehouse Migration Architecture

• Setup foundational elements. In GCP these include, IAM for authorization and access, cloud resource hierarchy, billing, networking, code pipelines, Infrastructure as Code using Cloud Build with Terraform ( GCP Foundation Toolkit), Cloud DNS and a dedicated/partner Interconnect to connect to the current data centers.
• Activate monitoring and security scanning services before real user data is loaded using Cloud Operations for monitoring and logging and Security Command Center for security monitoring. 
• Extract files from on-premise legacy EDW and move to Cloud Storage and establish on-going synchronization using Big Query Transfer services. 
• From Cloud Storage, process the data in Dataflow and Load/Export data to BigQuery. 
• Validate the export using Datametica’s validation utilities running in a GKE cluster and Cloud SQL for auditing and historical data synchronization as needed. Application teams test against the validated data sets throughout the migration process. 
• Orchestrate the entire pipeline using Cloud Composer, integrated with on-prem scheduling services as needed to leverage established processes and keep legacy and new systems in sync. 
• Maintain close coordination with teams/services ingesting new data into the EDW and down-streams analytics teams relying on the EDW data for on-going advanced analytics. 
• Establish fine-grained access controls to data sets and start making the data in Big Query available to existing reporting, visualization and application consumption tools using BigQuery data connectors for ‘down-stream’ user access and testing. 
• Incrementally increase Big Query flat-rate processing capacity to provide the most cost-effective utilization of resources during migration. 

To learn more about migrating from on-premises Enterprise Data Warehouses (EDW) to Bigquery and GCP here.