Explore the Complete Startups' Technical Guide on Google Cloud Tech Channel - Build What's Next
Webinar

Explore the Complete Startups’ Technical Guide on Google Cloud Tech Channel

3374

Of your peers have already read this article.

1:30 Minutes

The most insightful time you'll spend today!

Google Cloud has curated many technical guidelines for startups to scale their cloud adoption and implementation of GCP to get their business rolling. Read blog to access The Start Series on Google Cloud Tech channel for insights on cloud journey!

Bootstrap your Startup with our technical guided series


At Google Cloud, we want to provide you with the access to all the tools you need to grow your business. Through the Google Cloud Technical Guides for Startups, leverage industry leading solutions with how-to video guides and resource handbooks curated for startups.

This multi-series contains 3 chapters: Start, Build and Grow, which matches your startup’s stage of growth:

  • The Start Series: Begin by building, deploying and managing new applications on Google Cloud from start to finish.
  • The Build Series: Optimize and scale existing deployments to reach your target audiences.
  • The Grow Series: Grow and attain scale with deployments on Google Cloud.

Kick off with The Start Series

The Start Series is designed to help your startup begin building, deploying and managing new applications on Google Cloud from start to finish. The series contains 12 videos and is dedicated to those who are starting out their cloud journey with Google Cloud. From setting up your project, to choosing the right compute option, to configuring your networking to managing your databases, and understanding support and billing – the Start Series guides you at every step of the journey.

Check out our website and our Google Cloud Technical Guides for Startups full playlist.

Coming up next – The Build Series


Launch into the next part of the journey continuing from the Start Series, with the upcoming Build Series, where we will be focusing on the optimization and scaling of existing deployments to help your startups reach your target audiences.

Join us by checking out the video series on the Google Cloud Tech channel, and subscribe to stay up to date.

See you in the cloud!

Case Study

Canadian Bank’s SAP Workload Moved to BigQuery Helps Unlock New Business Opportunities

7345

Of your peers have already read this article.

2:00 Minutes

The most insightful time you'll spend today!

Canadian ATB Financial's migration of SAP environs that managed its core banking, financial services, payment engine and CRM data to Google Cloud and BigQuery helped them realize business outcomes in millions!

When ATB Financial decided to migrate its vast SAP landscape to the cloud, the primary goal was to focus on things that matter to customers as opposed to IT infrastructure. Based in Alberta, Canada, ATB Financial serves over 800,000 customers through hundreds of branches as well as digital banking options. To keep pace with competition from large banks and FinTech startups and to meet the increasing 24/7 demands of customers, digital transformation was a must. To support this new mandate, in 2019, ATB migrated its extensive SAP backbone to Google Cloud. In addition to SAP S/4 HANA, ATB runs SAP financial services, core banking, payment engine, CRM and business warehouse on Google Cloud. 

In parallel, changes were needed to ATB’s legacy data platform. The platform had stability and reliability issues and also suffered from a lack of historical data governance. Analytics processes were ad hoc and manual. The legacy data environment was also not set up to tackle future business requirements that come with a high dependency on real-time data analysis and insights.

After evaluating several potential solutions, ATB chose BigQuery as a serverless data warehouse and data lake for its next-generation, cloud-native architecture. “BigQuery is a core component of what we call our data exposure enablement platform, or DEEP,” explains Dan Semmens, Head of Data and AI at ATB Financial. According to Semmens, DEEP consists of four pillars, all of which depend on Google Cloud and BigQuery to be successful:

  1. Real-time data acquisition: ATB uses BigQuery throughout its data pipeline, starting with sourcing, processing, and preparation, moving along to storage and organization, then discovery and access, and finally consumption and servicing. So far, ATB has ingested and classified 80% of its core SAP banking data as well as data from a number of its third-party partners, such as its treasury and cash management platform provider, its credit card provider, and its call center software. 
  2. Data enrichment: Before migrating to Google Cloud, ATB managed a number of disconnected technologies that made data consolidation difficult. The legacy environment could handle only structured data, whereas Google Cloud and BigQuery lets the bank incorporate unstructured data sets, including sensor data, social network activity, voice, text, and images. ATB’s data enrichment program has enabled more than 160 of the bank’s top-priority insights running on BigQuery, including credit health decision models, financial reporting, and forecasting, as well as operational reporting for departments across the organization. Jobs such as marketing campaigns and month-end processes that used to take five to eight hours now run in seconds, saving over CA$2.24 million in productivity. 
  3. Self-service analytics: Data for self-service reporting, dashboarding, and visualization is now available for ATB’s 400+ business users and data analysts. Previously, bringing data and analytics to the business users who needed it while ensuring security was burdensome for IT, fraught with recurrent data preparation and other highly manual elements. Now, ATB automates much of its data protection and governance controls through the entire data lifecycle management process. Data access is not only open to more team members but it is faster and easier to acquire without compromising security. And it’s not just raw data that users can access. ATB uses BigQuery to define its enterprise data models and create what it calls its data service layer to make it easier for team members to visualize their data.
  4. AI-assisted analytics and automation: Through Google Cloud and BigQuery, ATB has been able to publish data and ML models that provide alerts and notifications via APIs to customer service agents. These real-time recommendations allow customer service agents to provide more tailored service with contextualized advice and suggested new services. So far, the company has deployed more than 40 ML models to generate over 20,000 AI-assisted conversations per month. Thanks to improved customer advocacy and less churn, the bank has realized more than CA$4 million in operating revenue. During the ongoing COVID crisis, the system was also able to predict when business and personal banking customers were experiencing financial distress so that a relationship manager could proactively reach out to offer support, such as payment deferral or loan restructuring. The AI tools provided by BigQuery are also helping ATB detect fraud that previously evaded rules-based fraud detection by using broader sets of timely and accurate data. 

Thanks to the speed and ease of moving data from SAP to BigQuery, ATB is using artificial intelligence (AI) and machine learning (ML) to do things it previously hadn’t thought possible, including sophisticated fraud prevention models, product recommendations, and enriched CRM data that improves the customer experience. 

Using the power of Google Cloud and BigQuery, ATB Financial has been able to draw more value from its SAP data while lowering cost and improving security and reliability. Speed to provide data sets and insights to internal team members has improved 30%. The bank also has seen a 15x reduction in performance incidents while improving data governance and security. Dan Semmens projects that the digital transformation strategy built on Google Cloud and BigQuery has both saved millions compared to its on-premises environment and has also realized millions in new business opportunities. 

Semmens is looking toward the future that includes initiatives like Open Banking and greater ability to provide real time personalized advice for customers to drive revenue growth. “We see our data platform as foundational to ATB’s 10-year strategy,” he says. “The work we’ve undertaken over the past 18 months has enabled critical functionality for that future.” 

Learn more about how ATB Financial is leveraging BigQuery to gain more from SAP data. Visit us here to explore how Google Cloud, BigQuery, and other tools can unlock the full value of your SAP enterprise data.

Blog

How Google Cloud and SAP Address Global Supply Chain Initiatives

3507

Of your peers have already read this article.

1:30 Minutes

The most insightful time you'll spend today!

Google Cloud and SAP partnership has been driving force for customers to explore innovations and transformations to the cloud. Read the blogpost to learn how the duo continue to remain tour de force in addressing global supply chain challenges!

With SAP Sapphire kicking off today in Orlando, we’re looking forward to seeing our customers and discussing how they can make core processes more efficient and improve how they serve their customers.

One thing is certain to be top of mind – the global supply chain challenges facing the world today. It’s affecting every business across every industry, from common household items that once filled store shelves and are now on backorder, to essential goods and services like food and medical treatments, which are at risk. Even cloud-native companies are making changes to ensure they have the insights, equipment, and other assets they need to continue serving customers.

We are proud to work with SAP on many initiatives that are driving results for our customers and helping them run more intelligent and sustainable companies. I’d like to highlight three of these important initiatives and how they are helping address global supply chain challenges.

Enabling more efficient migrations of critical workloads


We know a key barrier to entry in the cloud is the ability to easily migrate from on-premises environments. Our cloud provides a safe path to help companies including Johnson Controls, PayPal, and Kaeser Compressor to digitize and solve large, complex business problems, reduce costs, scale without cycles of investment, and gain access to key services and capabilities that can unlock value and enable growth.

Singapore-based shipping company Ocean Network Express (ONE) has become more agile by running their mission-critical SAP workloads on Google Cloud and using our data analytics to improve operational efficiency and make faster decisions. They have gone from an on-premises data warehouse solution that would take a full day loading data from SAP S/4HANA, to using our BigQuery solution that delivers business insights in minutes.

Since The Home Depot moved its critical SAP workloads to Google Cloud, the company has been able to shorten the time it takes to prepare a supply chain use case from 8 hours to 5 minutes by using BigQuery to analyze large volumes of internal and external data. This helps improve forecast accuracy and more effectively replenish inventory by being able to create a new plan when circumstances change unexpectedly with demand or a supplier.

Accelerating cloud benefits through RISE and LiveMigration


At Google Cloud, we have dedicated programs to help migrate SAP and other mission-critical workloads to our cloud with our Cloud Acceleration Program for SAP.

For SAP customers moving to Google Cloud, we provide LiveMigration to provide superior uptime and business continuity. LiveMigration eliminates downtime required for planned infrastructure maintenance. This means that your SAP system continues running even when Google Cloud is performing planned infrastructure maintenance upgrades thus ensuring superior business continuity for your mission critical workloads.

We are also proud to be a strategic partner with the RISE with SAP program, which helps accelerate cloud migration for SAP’s global customer base while minimizing risks along the migration journey. This program provides solutions and expertise from SAP and technology ecosystem partners to help companies transform through process consulting, workload migration services, cloud infrastructure, and ongoing training and support. To secure your mission critical workloads, SAP and Google Cloud can provide a 99.9% uptime SLA as part of the RISE with SAP program.

Many large manufacturers have taken advantage of RISE with SAP to forge a secure, proven path to our cloud, including Energizer Holdings Inc., a leading manufacturer and distributor of primary batteries, portable lights, and auto care products. Energizer has turned to RISE with SAP on Google Cloud to power its move to SAP S/4HANA. The company wants to automate essential business processes, improve customer service, and boost innovation. It had been using a private cloud solution but needed to gain flexibility while better containing costs.

“SAP S/4HANA for central finance will help us automate essential business processes, improve customer service, and fuel innovation that grows our company’s leadership position globally. We selected RISE with SAP to begin our journey to SAP S/4HANA and maintain the freedom and flexibility to move at our own pace,” said Energizer Chief Information Officer Dan McCarthy.

Another example is global automotive distributor Inchcape, which moved its mission-critical sales, marketing, finance, and operations systems and data to Google Cloud. With its diverse data sets now in a single, secure cloud platform, Inchcape is applying Google Cloud AI and ML capabilities to manage and analyze its data, automate operations, and ultimately transform the car ownership experience for millions.

“Google Cloud’s close relationship with SAP and its strong technical expertise in this space were a big pull for us,” said Mark Dearnley, Chief Digital Officer at Inchcape. “Ultimately, we wanted a headache-free RISE with SAP implementation and to unlock value for auto makers and consumers in all our regions, while continuing to have the choice and flexibility to modernize our 150-year old business in a way that works for us.”

A new intelligence layer for all SAP Google Cloud customers


When moving mission-critical workloads to the cloud, companies not only need to migrate safely, they also need to quickly realize value, which we enable with Google Cloud Cortex Framework — a layer of intelligence that integrates with SAP Business Technology Platform (SAP BTP). Google Cloud Cortex Framework provides reference architectures, deployment accelerators, and integration services for analytics scenarios.

Like many large e-commerce companies, Mercado Libre experienced skyrocketing transactions that more than doubled in 2020 as people sheltered at home during the pandemic, and they are anticipating more growth. The Google Cloud Cortex Framework is enabling Mercado Libre to respond, run more efficiently, and make faster, data-driven decisions.

Continued partnership to support organizations around the world


Our longstanding partnership with SAP continues to yield exciting innovations for our customers, and we’re honored to work with them to help customers address the ongoing impact of global supply chain challenges. We’re looking forward to sharing new insights and innovations at SAP Sapphire this week, and to listening and learning from you about your plans and challenges, and how we can best support your transformation to the cloud.

Blog

Google Introduces BigQuery Connector for SAP to Power Customers’ Data Analytics Strategy

5467

Of your peers have already read this article.

2:00 Minutes

The most insightful time you'll spend today!

Google Cloud is known for taking steps towards addressing customer requirements. With the new BigQuery Connector for SAP, we offer a fast, simple, cost-effective and massively scalable way to make SAP data accessible on BigQuery!

Google Cloud has a genuine passion for solving technology problems that make a difference for our customers. With the release of our BigQuery Connector for SAP, we’re taking a another big step towards solving a major challenge for SAP customers with a quick, easy, and inexpensive way to integrate SAP data with BigQuery, our serverless, highly scalable, and cost-effective multi cloud data warehouse designed for business agility.

Solving for simplified data integration

Like most businesses today, SAP customers are eager to unlock the immediate insights and opportunities within their ever-growing stores of business data. However, many are discovering just how hard it can be to take the first step in any modern, cloud-enabled data analytics strategy: combining SAP data with other cloud-native, and enterprise data sets in real-time and at scale. According to a 2020 SAPInsider study, more than half of SAP customers surveyed said data integration was their top analytics pain point. These companies urgently need a rapid, sustainable, cost-effective and scalable way to integrate SAP data with modern cloud data analytics solutions.

The BigQuery Connector for SAP gives our customers a solution: a fast, simple, cost-effective and massively scalable way to make SAP data fully accessible within BigQuery by leveraging customers’ existing SAP Landscape Transformation Replication Server (SLT) tooling and skill sets. It’s the first SAP SLT direct near real-time connector for BigQuery without the need to set up additional infrastructure or third-party middleware, and can be deployed using a variety of embedded or stand-alone deployment options. In fact, most customers can install the BigQuery Connector for SAP in less than an hour—a remarkably easy way to start working with our industry-leading analytics solution that delivers proven and quantifiable business advantages for customers. Additionally, the BigQuery Connector for SAP is not restricted to customers who have deployed their SAP applications on Google Cloud. Customer’s who are running their SAP applications on-premise, or on any cloud, can also deploy and realize the analytical benefits of the solution.

Designing a solution with customer requirements and investments in mind

When the Google Cloud team started work on an analytics data integration tool for our SAP customers, we began with a set of requirements designed to root out the usual sources of cost and complexity. These included: 

  • The need for real-time performance with deltas replicated in milliseconds
  • The ability to integrate data from almost any SAP Netweaver based application running today, regardless of its deployment location (on premises, any cloud, Google Cloud)
  • Automatic BigQuery data type mapping with minimal transformation required
  • Generation of target tables in BigQuery directly from source, if required 
  • Application layer integration that avoids the issues of direct database access
  • Leveraging customers’ existing SAP skillsets, change data capture, and infrastructure

An important step towards meeting these requirements came when Alphabet, Google’s parent company, decided to leverage SAP SLT as a foundation for developing direct data replication between SAP and BigQuery for its internal corporate landscape. SLT as part of SAP’s strategic Business Technology Platform, supports real-time replication of data from SAP or third-party systems to SAP HANA, however, one of its limitations was direct integration with targets like BigQuery. 

SAP SLT was a logical foundation for developing the connector for several reasons: 

  • It’s widely adopted among SAP customers who likely already leverage SLT for SAP analytics data integration
  • It works with almost every non-SaaS SAP application environment running today
  • It supports real-time replication performance at massive scale

It was an obvious choice for the Alphabet engineering team who saw immediate value from integrating SAP with BigQuery. 

“The BigQuery Connector for SAP has enabled fast, low latency data replication for billions of records from 500+ tables of our most critical financial and supply chain data. Now in one cost-effective BigQuery data lake, this ERP data can be combined with other data sources for previously impossible real-time analytics and ML use cases. This allows us to drive much deeper strategic insights that support business and operational excellence, management and P&L reporting and more.”—Anil Nagalla, Sr. Engineering Director, Financial Systems, Google

SAP data integration with BigQuery enables new value

By leveraging SAP SLT, the BigQuery Connector for SAP can integrate real-time data streams from any SAP system—while also taking advantage of customers’ existing SAP investments and skillsets. 

At the same time, the BigQuery Connector for SAP does a lot of heavy lifting on its own. For example, it automatically handles the complex, multi-step process of transforming SAP data types for use in BigQuery—mapping data-type transitions between the SAP and BigQuery environments, creating a target table schema on BigQuery for the transformed data types, building the target BigQuery table, and even adapting as new data types appear in your SAP environment.

For teams that want to fine-tune the BigQuery Connector for SAP’s automated recommendations, the connector supports additional levels of customization and choice. But if you simply want to get the job done and give your data analytics team greater support for their high-value work, then you’ll love just how quickly and easily the BigQuery Connector for SAP turns the complicated work of data integration and performance to process large volumes of data into a done deal. By integrating enterprise data sets in real time, customers can drive differentiated value and unlock new insights and actions that drive a competitive advantage. 

The BigQuery Connector for SAP really shines as an enabling tool that transports and transforms your SAP data to power analytics solutions enabled by accelerators like the Google Cloud Cortex Framework: a comprehensive set of reference architectures, deployment accelerators, and integration services designed to give SAP customers a fast and seamless path to value with their data analytics investments. Simply put, the more SAP data you make available within Google Cloud, the easier it is to get meaningful—and often game-changing—insights from these solutions.

Learn more about the BigQuery Connector for SAP

Ready to get started with your own SAP data analytics strategy on Google Cloud? Install the Google Cloud BigQuery Connector for SAP, and discover a faster, simpler, more sustainable way to power your company’s data analytics strategy.

How-to

Recommendations for Modelling SAP Data inside BigQuery

8414

Of your peers have already read this article.

4:00 Minutes

The most insightful time you'll spend today!

SAP-powered organizations can unleash the strength of analytics with BigQuery and follow these guidelines or considerations for modelling SAP data to address business needs.

Over the past few years, many organizations have experienced the benefits of migrating their SAP solutions to Google Cloud. But this migration can do more than reduce IT maintenance costs and make data more secure. By leveraging BigQuery, SAP customers can complement their SAP investments and gain fresh insights by consolidating enterprise data and easily extending it with powerful datasets and machine learning from Google. 

BigQuery is a leading cloud data warehouse, fully managed and serverless, and allows for massive scale, supporting petabyte-scale queries at super-fast speeds. It can easily combine SAP data with additional data sources, such as Google Analytics or Salesforce, and its built-in machine learning lets users operationalize machine learning models using standard SQL — all at a comparatively low cost

If your SAP-powered organization is looking to supercharge its analytics with the strength of BigQuery, read on for considerations and recommendations for modeling with SAP data. These guidelines are based on our real-world implementation experience with customers and can serve as a roadmap to the analytics capabilities your business needs.

Considerations for data replication 

Like most technology journeys, this one should start with a business objective. Keeping your intended business value and goals in mind is critical to making the right decisions in the early steps of the design process.

When it comes to replicating the data from an SAP system into BigQuery, there are multiple ways to do it successfully. Decide which method will work best for your organization by answering these questions:

  • Does your business need real-time data? Will you need to time travel into past data?
  • Which external datasets will you need to join with the replicated data?
  • Are the source structures or business logic likely to change? Will you be migrating the SAP source systems any time soon? For instance, will you be moving from SAP ECC to SAP S/4HANA?

You’ll also need to determine whether replication should be done on a table-by-table basis or whether your team can source from pre-built logic. This decision, along with other considerations such as licensing, will influence which replication tool you should use.

Replicating on a table-by-table basis
Replicating tables, especially standard tables in their raw form, allows sources to be reused and ensures more stability of the source structure and functional output. For example, the SAP table for sales order headers (VBAK) is very unlikely to change its structure across different versions of SAP, and the logic that writes to it is also unlikely to change in a way that affects a replicated table. 

Something else to consider: Reconciliation between the source system and the landing table in BigQuery is linear when comparing raw tables, which helps avoid issues in consolidation exercises during critical business processes, such as period-end closing. Since replicated tables aren’t aggregated or subject to process-specific data transformation, the same replicated columns can be reused in different BigQuery views. You can, for instance, replicate the MARA table (the material master) once and use it in as many models as needed. 

Replicating pre-built logic
If you replicate pre-built models, such as those from SAP extractors or CDS views, you don’t need to build the logic in BigQuery, since you’re using existing logic. Some of these extraction objects have embedded delta mechanisms, which may complement a replication tool that can’t handle deltas. This will save initial development time, but it can also lead to challenges if you create new columns, or if customizations or upgrades change the logic behind the extraction. 

It’s also important to note that different extraction processes may transform and load the same source columns multiple times, which creates redundancy in BigQuery and can lead to higher maintenance needs and costs. However, replicating pre-built models may still be a good choice, since doing so can be especially useful for logic that tends to be immutable, such as flattening a hierarchy, or logic that is highly complex.

How you approach replication will also depend on your long-term plans and other key factors — for example, the availability (and curiosity) of your developers, and the time or effort they can put into applying their SQL knowledge to a new data warehouse. 

With either replication approach, bear in mind when designing your replication process that BigQuery is meant to be an append-always database — so post-processing of data and changes will be required in both cases. 

Processing data changes

The replication tool you choose will also determine how data changes are captured (known as CDC – change data capture). If the replication tool allows for it (for example as SAP SLT does) the same patterns described in the CDC with BigQuery documentation also apply to SAP data. 

Because some data, like transactions, are known to be less static than others (e.g., master data), you need to decide what should be scanned in real time, what will require immediate consistency, and what can be processed in batches to manage costs. This decision will be based on the reporting needs from the business.

Consider the SAP table BUT000, containing our example master data for business partners, where we have replicated changes from an SAP ERP system:

1 SAP table BUT000.jpg

In an append-always replication in BigQuery, all updates are received as new records. For example, deleting a record in the source will be represented as a new record in BigQuery with a deletion flag. This applies to whether the records are coming from raw tables like BUT000 itself or pre-aggregated data, as from a BW extractor or a CDS view.

Let’s take a closer look at data coming particularly from the partners “LUCIA” and “RIZ”. The operation flag tells us whether the new record in BigQuery is an insert (I), update (U) or deletion (D), while the timestamps help us identify the latest version of our business partner.

2 incoming data.jpg

If we want to find the latest updated record for the partners LUCIA and RIZ, this is what the query would look like:

  SELECT partner,
        ARRAY_AGG(i1 ORDER BY i1.recordstamp DESC LIMIT 1) AS row
FROM SAP_ECC.but000 i1 
WHERE partner in ('LUCIA','RIZ')
    GROUP BY partner

With the following result:

3 query results.jpg

After identifying stale records for “LUCIA” and “RIZ” business partners, we can proceed to deleting all stale records for “LUCIA” if we do not want to retain the history. In this example, we are using a different table to which the same replication has been done, for the purpose of comparison and to check that all stale records have been deleted for the selection made and that we only kept last updated records. For example:

  DELETE SAP_HANA.but000 i1
WHERE
i1.recordstamp < TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 3 HOUR) AND
i1.recordstamp < (SELECT MAX(recordstamp) FROM SAP_HANA.but000 i2
WHERE
i1.partner = i2.partner
and partner="LUCIA")

You can also use the following query to retrieve stale records for “LUCIA” partner before moving forward with deletion

  SELECT partner, operation_flag, recordstamp  FROM SAP_HANA.but000 i1
WHERE
i1.recordstamp < TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 3 HOUR) 
AND
i1.recordstamp < (SELECT MAX(recordstamp) FROM SAP_HANA.but000 i2 
WHERE 
i1.partner = i2.partner
and partner="LUCIA")

Which produces all of the records, except the latest update:

4 records.jpg

Partitioning and clustering

To limit the number of records scanned in a query, save on cost and achieve the best performance possible, you’ll need to take two important steps: determine partitions and create clusters. 

Partitioning
partitioned table is one that’s divided into segments, called partitions, which make it easier to manage and query your data. Dividing a large table into smaller partitions improves query performance and controls costs because it reduces the number of bytes read by a query.

You can partition BigQuery tables by:

  • Time-unit column: Tables are partitioned based on a “timestamp,” “date,” or “datetime” column in the table.
  • Ingestion time: Tables are partitioned based on the timestamp recorded when BigQuery ingested the data.
  • Integer range: Tables are partitioned based on an integer column.

Partitions are enabled when the table is created, as in the example below.  A great tip is to always include the partition filter as shown on the left-hand side of the query.

5 Partitions.jpg

Clustering
Clustering can be created on top of partitioned tables by applying the fields that are likely to be used for filtering. When you create a clustered table in BigQuery, the table data is automatically organized based on the contents of one or more of the columns in the table’s schema. The columns you specify are then used to colocate related data.

Clustering can improve the performance of certain query types — for example, queries that use filter clauses or that aggregate data. It makes a lot of sense to use them for large tables such as ACDOCA, the table for accounting documents in SAP S/4HANA. In this case, the timestamp could be used for partitioning, and common filtering fields such as the ledger, company code, and fiscal year could be used to define the clusters.

6 define cluster.jpg

A great feature is that BigQuery will also periodically recluster the data automatically.

Materialized views

In BigQuery, materialized views are precomputed views that periodically cache the results of a query for better performance and efficiency. BigQuery uses precomputed results from materialized views and, whenever possible, reads only the delta changes from the base table to compute up-to-date results quickly. Materialized views can be queried directly or can be used by the BigQuery optimizer to process queries to the base table.

Queries that use materialized views are generally completed faster and consume fewer resources than queries that retrieve the same data only from the base table. If workload performance is an issue, materialized views can significantly improve the performance of workloads that have common and repeated queries. While materialized views currently only support single tables, they are very useful common and frequent aggregations like stock levels or order fulfillment.

Further tips on performance optimization while creating select statements can be found in the documentation for optimizing query computation.

Deployment pipeline and security

For most of the work you’ll do in BigQuery, you’ll normally have at least two delivery pipelines running — one for the actual objects in BigQuery and the other to keep the data staging, transforming, and updated as intended within the change-data-capture flows. Note that you can use most existing tools for your Continuous Integration / Continuous Deployment (CI/CD) pipeline — one of the benefits of using an open system like BigQuery. But, if your organization is new to CI/CD pipelines, this is a great opportunity to gradually gain experience. A good place to start is to read our guide for setting up a CI/CD pipeline for your data-processing workflow.  

When it comes to access and security, most end-users will only have access to the final version of the BigQuery views. While row and column-level security can be applied, as in the SAP source system, separation of concerns can be taken to the next level by splitting your data across different Google Cloud projects and BigQuery datasets. While it’s easy to replicate data and structures across your datasets, it’s a good idea to define the requirements and naming conventions early in the design process so you set it up properly from the start. 

Start driving faster and more insightful analytics

The best piece of advice we can give you is this: Try it yourself. Anyone with SQL knowledge can get started using the free BigQuery tier. New customers get $300 in free credits to spend on Google Cloud during the first 90 days. All customers get 10 GB storage and up to 1 TB queries/month, completely free of charge. In addition to discovering the massive processing capabilities, embedded machine learning, multiple integration tools, and cost benefits, you’ll soon discover how BigQuery can simplify your analytics tasks. 

If you need additional assistance, our Google Cloud Professional Services Organization (PSO) and Customer Engineers will be happy to help show you the best path forward for your organization. For anything else, contact us at cloud.google.com/contact.

Blog

7 Fantastic Ways Google Cloud VMWare Engine Stands Out from the Rest for Running VMWare Workloads in the Cloud!

3581

Of your peers have already read this article.

2:00 Minutes

The most insightful time you'll spend today!

Google Cloud migration for VMware workloads impacts savings 38 percent in TCO. Still considering the pros of Google Cloud VMWare Engine? Here are seven customer-centric innovations in its infrastructure that makes its ideal for Vsphere workloads!

Google Cloud VMware Engine delivers an enterprise-grade, cloud-native VMware experience that is built on Google Cloud’s highly performant and scalable infrastructure. By enabling a consistent VMware experience, the service allows customers to adopt Google Cloud rapidly, easily, and with minimal modifications to their vSphere workloads, bringing the best of VMware and Google Cloud together on one platform for a variety of use-cases. These include rapid data center exit, application lift and shift, disaster recovery, virtual desktop infrastructure, or modernization at your own pace.

Here are seven ways VMware Engine outshines alternatives for running your VMware workloads in the cloud, simplify your operations, and help you innovate faster:

  1. Dedicated 100Gbps east-west networking
    Google Cloud VMware Engine nodes come with redundant switching and dedicated 100Gbps east-west networking with no oversubscription of bandwidth, unlike other options where there is generally oversubscription. This is especially important when it comes to running latency-sensitive workloads.
  2. Four 9’s of availability in a single zone
    The service offers 99.99% uptime SLA for a cluster in a single Zone with five to 16 nodes and FTT=2 or more without the need for stretched clusters, which is higher than the alternatives. Further, dedicated connectivity for core service functions such as vSAN and vMotion enables better solution stability and availability. This enables the service to support the needs of enterprise workloads that require high availability.

Note: “Cluster” means a deployment of three or more dedicated bare metal nodes running VMware ESXi and associated networking managed via management interfaces.

  1. Global networking without complex routing
    Google Cloud VMware Engine networking is built based on Google Cloud’s powerful networking architecture. With simplified regional and global routing modes—which allow a VPC’s subnets to be deployed in any region where our service is available—you can architect global networks without the need or overhead of creating and connecting regional network designs. You get instant, direct Layer 3 access between them. In alternative cloud environments, you may have to configure special networking between regions, often requiring VPN-based tunnels over the WAN to enable global uniform network communication. This adds to the deployment and operational complexity, in addition to cost.
  2. Integrated multi-VPC networking
    Users often have application deployments in different VPC networks, such as separate dev/test and production environments or multiple administrative domains across business units. The service supports “many-to-many” access from VPC networks to Google Cloud VMware Engine networks with multi-VPC networking, allowing you to retain existing deployed architectures and extend them flexibly to your VMware environments. In addition, by providing multi-VPC networking, you can pool their VMware needs—say for QA and dev—to a smaller set of clusters, effectively reducing their costs.

For more information about the end-to-end networking capabilities and services available in Google Cloud VMware Engine, please refer to the Private Cloud Networking for Google Cloud VMware Engine whitepaper. Here, you’ll find details about network flows, configuration options, and the differentiated benefits of running your VMware workloads in Google Cloud.

  1. Unified, cloud-integrated model
    Google Cloud VMware Engine is a fully managed Google first-party service, operated and supported by Google and its world-class team. With fully integrated identities, billing, and access control, you have a simpler end-to-end experience that is different from other services. You access Google Cloud VMware Engine service via the Google Cloud console, like any other Google Cloud service. You can also access other native Google Cloud services privately from your VMware private cloud running in Google Cloud VMware Engine over local connections.
  2. Flexibility in third-party ecosystem compatibility
    With Google Cloud VMware Engine, you can set up existing VMware on-premises third-party tools or products that require additional privileges by using a solution user account. This uniquely enables operational consistency, ensuring that the tools you have invested in and used over the years work on Google Cloud VMware Engine. Furthermore, in key areas such as vSAN data encryption, you have the choice of not only using Google Cloud Key Management Service (KMS)—which is turned on by default on vSAN datastores—but also external KMS providers such as HyTrust, Thales, and Fortanix.
  3. Dense nodes with high storage:core and memory:core ratios and fast provisioning
    Google Cloud VMware Engine nodes are dense. Each node is powered by Intel® Xeon® Scalable Processors and comes with 36 cores, 72 hyperthreaded cores, 768 GB memory, 19.2 TB NVMe data and 3.2 TB NVMe cache storage. This, along with oversubscription, leads to high consolidation ratios and compelling storage:core per dollar and memory:core per dollar. In addition, you can rapidly spin up these nodes in a VMware private cloud often in under an hour, enabling on-demand, VMware-consistent capacity in Google Cloud for your needs.

These are just a few examples of customer-centric innovation that set Google Cloud VMware Engine infrastructure apart. In addition, migrating to Google cloud can save you up to 38% in TCO. Get started by learning about Google Cloud VMware Engine and your options for migration, or talk to our sales team to join the customers who have embarked upon this journey.

The authors would like to thank the Google Cloud VMware Engine product team for their contributions on this blog.

More Relevant Stories for Your Company

Case Study

How The New York Times Increased Speed of Delivery by Using Kubernetes

When New York Times decided a few years ago to move out of its data centers, its first deployments on the public cloud were smaller and less critical applications that were being managed on virtual machines. "We started building more and more tools, and at some point, we realized that

Blog

Accelerating Success: Tips and Techniques for Optimizing and Scaling Your Startup

At Google Cloud, we want to provide you with the access to all the tools you need to grow your business. Through the Google Cloud Technical Guides for Startups, leverage industry leading solutions with how-to video guides and resources curated for startups. This multi-series contains 3 chapters: Start, Build and

Blog

Google Cloud & Optiva Partnership Cements the Future of Telecom for Driving Strong Customer Experience

Editor’s note: Coming out of Mobile World Congress 2022, we are excited to share key learnings from our partner ecosystem on how to leverage advancements in 5G technologies to power customer experiences that seamlessly blend our physical and virtual world into one. The original version of this blog was published

Blog

withVR Uses the Power of VR to Prep People with Speech Disorders for Real-life Speaking Situations

Editor’s note: Meet Gareth Walkom, an entrepreneur dedicated to helping others with speech disorders. Turning life experience into innovation Did you know that 3% of Americans have a speech disorder, while 1% of the world’s population have a stutter? Just getting what they need in everyday interactions can be stressful,

SHOW MORE STORIES