About Tencent Africa

Tencent Africa is dedicated to making every day easier for customers and brands. Opening its doors in 2013 as WeChat Africa, the company is responsible for WeChat operations on the continent.

Industries: Technology

Location: South Africa

About Siatik Systems

Founded in 2008, South Africa-based Siatik Systems is a leading Google Cloud Platform solutions provider with a focus on cloud integration services, information technology consulting services, and bespoke software engineering solutions.

Industries:

Location:

As one of Africa’s leading technology companies, Tencent Africa is responsible for WeChat operations on the continent. WeChat Africa has successfully launched a number of features including the WeChat Wallet, a mobile payment service for smartphones, which enables seamless and secure transactions for friends to send money to one another, cash money out at Standard Bank ATMs, use at accredited retailers, and make purchases from any SnapScan merchant in South Africa. Under the new name, Tencent Africa recently launched two new mobile products, JOOX, a mobile music streaming application, and VOOV, a social live streaming application.

When China’s Tencent brought its hugely popular instant messaging service WeChat to South Africa, it faced a near saturated market. To gain traction, Tencent Africa had to showcase WeChat’s versatility as a platform that goes far beyond just instant messaging. Engaging with local financial partners helped WeChat launch a WeChat Wallet to facilitate electronic payments, while providing exclusive content from South African cultural figures helped bring the platform extra publicity.

As WeChat started to grow its customer base, its on-premises infrastructure was stretched beyond its limits and the company looked for a cloud-based solution. After evaluating all its options, Tencent Africa found Google Cloud Platform best suited its needs.

“We’re a growing business and constantly upgrading our infrastructure. Getting new servers was becoming quite expensive so we wanted a cloud-based solution to reduce costs,” says Christoff Albertyn, Head of Technology at Tencent Africa. “When we did some latency testing, Google came out on top, so it made sense to go with them. Everybody loves Google, it’s a great brand. Just knowing a company like Google is behind it gives you peace of mind.”

Building a platform to scale

Instant messaging services need a stable infrastructure, the ability to scale at speed, and lightning fast performance. When Tencent Africa built an Official Accounts API to run a number of services on WeChat, to help companies engage with users on the platform, latency was its main concern. The Official Accounts services had to deliver replies over WeChat in under five seconds or the connection would drop.

Between the API on-premises servers in South Africa and WeChat’s main servers in China, Tencent Africa’s services regularly hit the limits of the permissible response time. In addition, by 2015, WeChat was starting to garner more customers across South Africa and Tencent Africa had to provision more servers. The on-premises servers were simply being asked to do too much, and expansion was proving expensive, so the company looked for a cloud-based solution.

Before committing to anything, Tencent Services did a thorough latency test of several cloud-based solutions. Google Cloud Platform came top. Tencent also engaged the help of Siatik, a Google Partner, to gauge what solutions best fit the company’s needs and help implement them. Siatik customers span both the public and private sectors and its consultants are regularly engaged in the health care, education, financial and technology industries. In order to provide leading technology solutions, it partners with industry leaders, such as Google, to provide best-of-breed infrastructure and cloud platform solutions to its customers.

“Siatik was a great partner to work with,” says Christoff. “Any questions I had, I could just pick up the phone and whatever it was they’d help me out. They were also able to get some of the team from Google to come to our offices and see how we worked, which was really good.”

With a simple, clean interface in Google Compute Engine, the company was able to spin up virtual instances very quickly and begin migrating its stack to Google. When it became more comfortable with Google Cloud Platform features, Tencent used Google Cloud SQL and Google BigQuery to manage its data streams, and Google Cloud Storage to store them safely.

From day one, it was very smooth. The interface was amazing—Google must have spent a long time perfecting it because it was so easy to use says Christoff. “I set up the first virtual machine in no time without any struggles.”

In addition to powerful hardware from Google Compute Engine, the fast, global network from Google helped to speed up performance. With the infrastructure ported over to Google, Tencent also brought in G Suite as an office productivity solution, which eased the pressure on its servers even more. Throughout the process, Siatik were on hand to advise and provide technical guidance whenever any issues came up.

What I really like is how quickly you can spin servers up. There’s no messing about and the interface is really easy to use, says Christoff. “You can have a virtual machine running in just a few minutes. It’s very impressive.”

Cutting costs, raising expectations

Google Cloud Platform helped Tencent Africa to slash its latency. Before the migration, pings to the Chinese servers and back would regularly take 180 milliseconds. Afterwards, this fell to around 35 milliseconds, helping to ensure smooth service to Tencent’s South African customers. Google offers innovative pricing, such as per-minute billing and sustained use discounts, which helped significantly cut server costs compared to an equivalent on-premises solution.

As its customer base grew, and Tencent Africa began to release new products, the ease of use of Google Compute Engine meant that spinning up new servers took minutes instead of hours and allowed the company to scale up and down at speed. Exploring products such as Kubernetes for rapid deployment or TensorFlow for machine learning, Tencent Africa continues to evolve and experiment with new products to keep WeChat a growing success in South Africa.

“Initially, we just used per-minute billing, but when I saw how much work we were doing with Google BigQuery and Google Cloud SQL, we started to generate sustained-use discounts, which helped save us a lot of money,” says Christoff. “The other great thing is, Google Cloud Platform monitors how you’re using it and tells you what else you can do to reduce costs.”

Today we’re announcing the HarbourBridge Schema Assistant, which provides a guided schema-design workflow for migrating from MySQL or PostgreSQL to Spanner. HarbourBridge imports dump files (from mysqldump or pg_dump) or directly connects to your source database, and converts the source database schema to an equivalent Spanner schema. The new Schema Assistant capability displays the source schema and Spanner schema side-by-side, highlights errors and walks you through a series of steps to validate and optimize your Spanner schema. It also produces a browsable assessment report with an overall migration-fitness score for Spanner, a table-by-table detailed analysis of type mappings and a list of features used in the source database that aren’t supported by Spanner. It supports editing of table and column names, column types, primary keys and constraints, as well as dropping of tables, columns, foreign keys and secondary indexes.

The new Schema Assistant complements HarbourBridge’s existing data and schema migration capabilities and is a critical step towards our goal of building a complete open-source migration toolkit. HarbourBridge continues to support command-line schema and data migration and turn-key Spanner evaluation.

Complementing the bulk data migration capabilities of HarbourBridge, we are also announcing the ability to migrate change events from MySQL to Cloud Spanner.

Supported Features in Schema Assistant

1. Global type mapping. Users can customize the global mapping for how types should be mapped to Spanner consistently across the schema. For example, mapping large integers in source schema to Spanner’s NUMERIC.
2. Local type mapping. Users can override the custom type mapping for a given table/column.
3. Session management. A session keeps track of all the changes made to the schema mapping.
4. Customization of secondary indexes. Users can add, edit and delete secondary indexes to optimize their Spanner performance.
5. Customization of foreign keys and interleaved tables. Table interleaving is an important design consideration when migrating to Cloud Spanner as explained in more detail in this blog post.

Features in the pipeline

We are already working to further expand the supported set of schema editing features and welcome your feedback. We are particularly excited to expand the Schema Assistant’s design recommendations for optimizing Spanner schemas e.g. in-depth recommendations for primary key design.

HarbourBridge is open source and we gladly accept contributions from the wider community.

Google Cloud has a genuine passion for solving technology problems that make a difference for our customers. With the release of our BigQuery Connector for SAP, we’re taking a another big step towards solving a major challenge for SAP customers with a quick, easy, and inexpensive way to integrate SAP data with BigQuery, our serverless, highly scalable, and cost-effective multi cloud data warehouse designed for business agility.

Solving for simplified data integration

Like most businesses today, SAP customers are eager to unlock the immediate insights and opportunities within their ever-growing stores of business data. However, many are discovering just how hard it can be to take the first step in any modern, cloud-enabled data analytics strategy: combining SAP data with other cloud-native, and enterprise data sets in real-time and at scale. According to a 2020 SAPInsider study, more than half of SAP customers surveyed said data integration was their top analytics pain point. These companies urgently need a rapid, sustainable, cost-effective and scalable way to integrate SAP data with modern cloud data analytics solutions.

The BigQuery Connector for SAP gives our customers a solution: a fast, simple, cost-effective and massively scalable way to make SAP data fully accessible within BigQuery by leveraging customers’ existing SAP Landscape Transformation Replication Server (SLT) tooling and skill sets. It’s the first SAP SLT direct near real-time connector for BigQuery without the need to set up additional infrastructure or third-party middleware, and can be deployed using a variety of embedded or stand-alone deployment options. In fact, most customers can install the BigQuery Connector for SAP in less than an hour—a remarkably easy way to start working with our industry-leading analytics solution that delivers proven and quantifiable business advantages for customers. Additionally, the BigQuery Connector for SAP is not restricted to customers who have deployed their SAP applications on Google Cloud. Customer’s who are running their SAP applications on-premise, or on any cloud, can also deploy and realize the analytical benefits of the solution.

Designing a solution with customer requirements and investments in mind

When the Google Cloud team started work on an analytics data integration tool for our SAP customers, we began with a set of requirements designed to root out the usual sources of cost and complexity. These included: 

• The need for real-time performance with deltas replicated in milliseconds
• The ability to integrate data from almost any SAP Netweaver based application running today, regardless of its deployment location (on premises, any cloud, Google Cloud)
• Automatic BigQuery data type mapping with minimal transformation required
• Generation of target tables in BigQuery directly from source, if required 
• Application layer integration that avoids the issues of direct database access
• Leveraging customers’ existing SAP skillsets, change data capture, and infrastructure

An important step towards meeting these requirements came when Alphabet, Google’s parent company, decided to leverage SAP SLT as a foundation for developing direct data replication between SAP and BigQuery for its internal corporate landscape. SLT as part of SAP’s strategic Business Technology Platform, supports real-time replication of data from SAP or third-party systems to SAP HANA, however, one of its limitations was direct integration with targets like BigQuery. 

SAP SLT was a logical foundation for developing the connector for several reasons: 

• It’s widely adopted among SAP customers who likely already leverage SLT for SAP analytics data integration
• It works with almost every non-SaaS SAP application environment running today
• It supports real-time replication performance at massive scale

It was an obvious choice for the Alphabet engineering team who saw immediate value from integrating SAP with BigQuery. 

“The BigQuery Connector for SAP has enabled fast, low latency data replication for billions of records from 500+ tables of our most critical financial and supply chain data. Now in one cost-effective BigQuery data lake, this ERP data can be combined with other data sources for previously impossible real-time analytics and ML use cases. This allows us to drive much deeper strategic insights that support business and operational excellence, management and P&L reporting and more.”—Anil Nagalla, Sr. Engineering Director, Financial Systems, Google

SAP data integration with BigQuery enables new value

By leveraging SAP SLT, the BigQuery Connector for SAP can integrate real-time data streams from any SAP system—while also taking advantage of customers’ existing SAP investments and skillsets. 

At the same time, the BigQuery Connector for SAP does a lot of heavy lifting on its own. For example, it automatically handles the complex, multi-step process of transforming SAP data types for use in BigQuery—mapping data-type transitions between the SAP and BigQuery environments, creating a target table schema on BigQuery for the transformed data types, building the target BigQuery table, and even adapting as new data types appear in your SAP environment.

For teams that want to fine-tune the BigQuery Connector for SAP’s automated recommendations, the connector supports additional levels of customization and choice. But if you simply want to get the job done and give your data analytics team greater support for their high-value work, then you’ll love just how quickly and easily the BigQuery Connector for SAP turns the complicated work of data integration and performance to process large volumes of data into a done deal. By integrating enterprise data sets in real time, customers can drive differentiated value and unlock new insights and actions that drive a competitive advantage. 

The BigQuery Connector for SAP really shines as an enabling tool that transports and transforms your SAP data to power analytics solutions enabled by accelerators like the Google Cloud Cortex Framework: a comprehensive set of reference architectures, deployment accelerators, and integration services designed to give SAP customers a fast and seamless path to value with their data analytics investments. Simply put, the more SAP data you make available within Google Cloud, the easier it is to get meaningful—and often game-changing—insights from these solutions.

Learn more about the BigQuery Connector for SAP

Ready to get started with your own SAP data analytics strategy on Google Cloud? Install the Google Cloud BigQuery Connector for SAP, and discover a faster, simpler, more sustainable way to power your company’s data analytics strategy.