In the past few years, a series of new technologies and regulatory changes has been transforming Brazil’s financial industry. The concept of blockchain added security and agility to financial transactions. The open banking system being deployed by the country’s Central Bank (BC) allows for platform integration and data sharing -with the user’s consent- between financial institutions. Recently, the launch of Pix, also by the BC, has shaken the market by creating a new payment method that is instant, free for individuals and 24/7.

Since 2018, the startup Bit Capital has been working on the development of 100%-digital financial solutions in the cloud to help its clients adapt to this new scenario in a convenient way, without the need to build an infrastructure for that or hiring different providers. Pix was not the exception.

In less than 3 months, the team was able to build and go live with a solution that can be used by both direct participants (i.e. those with banking licenses granted by BC) and indirect participants (i.e. companies depending on direct participants to offer payments on Pix) through Bit Capital’s platform.

Besides this differential, Pix and the startup’s other solutions are based on blockchain and Google Cloud’s cloud, ensuring greater security, scalability and availability for customers and allowing for an easy integration between Bit Capital’s platform’s solutions and those from other companies.

“Now we have a platform with an API and various microsservices, allowing clients to connect and develop their own financial product without having to start from scratch.”

—Francesco Miolo, CFO, Bit Capital

A robust structure to handle Pix’s high demand

Bit Capital was established with a certain urgency to develop its platform. The idea was well-developed and customers were interested. The fast deployment of Google Cloud’s tools was one of the main factors that attracted the company to use it as a basis for its infrastructure.

“We managed to deliver everything we have today in Google Cloud with a small team. That was another challenge: being able to grow with a few people,” says Juliano Souza, the startup’s head of IT infrastructure. “We sought other cloud partners, but they had a steep curve. We chose Google Cloud because we needed quick, quality scaling.”

The same thing happened when they built the solution for Pix, despite the specific challenges involved. The BC had performance requirements that led the company to spend some time experimenting until they reached the best suite of tools to meet those requirements. The startup wanted to create a unique architecture that could help both large and small customers and be integrated to the platform’s other microservices.

With the support from Google Cloud’s team to answer doubts and make the best decisions, the solution was built in a few months, at a cost about two-thirds lower than other providers.

The solution’s architecture is based on apps running in Docker in Google Kubernetes Engine (GKE), with interconnected microservices. The blockchain system runs on Compute Engine, its rows are managed in Pub/Sub and Dataflow, and persistent data, in Cloud SQL. Cloud Interconnect is used for the connection with BC. Cloud KMS and Secrets Management store the company’s pre-credentials. All of this is supported by Cloud Load Balancing for load balancing and autoscaling.

According to the team, GKE was essential for the solution’s success. Using infrastructure as code in the tool made uploading of a group of microservices significantly easier. Developers are able to help set up this environment, which has helped spread the DevOps culture in the team. Besides orchestrating and integrating apps, GKE also provides an elastic structure to handle demand peaks and visibility to monitor internal components.

“Google Kubernetes Engine was the only way to ensure availability, observability and elasticity for all clients, whether small, middle-sized or large.”

—Juliano Souza, Head of IT Infrastructure, Bit Capital

Nowadays, the solution’s environment has 20 clusters with over 1,500 pods – 250GB in data per month, providing a robust structure to support a service involving periods of intense demand such as Black Friday.

Ease to monitor, fix and improve

Just 10 days after Pix’s official launch date, the company had its first test: Black Friday. This allowed a major e-commerce customer to test the scale and see the success of the architecture that had been built. “It worked great regarding what the cloud could deliver. And we found what we needed to fix very quickly. Operations [formerly Stackdriver], in particular Cloud Trace, showed us clearly what needed to be done to improve performance,” Souza explains.

Using Operations added reliability by putting deliveries into production. Checking Cloud Trace to see if there were any performance issues and the exact point where they were happening became routine for Bit Capital’s developers. Google Cloud’s security tools and Google Safety Center provided the resources needed to monitor and secure the environment, with automatic data encryption at rest and in transit.

“Google Cloud has security as a premise. When we upload any kind of component, like a database or a virtual machine, the drive is encrypted by default. With other providers, you must specify that you want it encrypted.

”—Juliano Souza, Head of IT Infrastructure, Bit Capital

The easy service monitoring and management accelerates product and technology development because the team no longer needs to worry about infrastructure. Automated management allows professionals to spend more time on new projects and the company’s business. Also, the deployment of services in a Google Cloud multi-region impacts on customer experience, providing high availability for their solutions.

In the coming months, Bit Capital aims to ramp up service usage and the creation of new projects in Google Cloud by adding more customers to Pix’s solution and Banking as a Service (BaaS) solutions. Anthos will be incorporated to the tool suite to make it easier to connect apps with the customers’ on-prem environments. And the deployment of open banking has the potential to be a business driver for the company.

“We joke that we’re already doing open banking, because we have various connections with different providers, which allows us to offer an integrated solution,” says Francesco Miolo, the startup’s CFO. “With the arrival of the new regulations, something we are waiting for since we started out, we will be able, through our platform, to take our customers to the open finance ecosystem, enabling the development of disruptive business models.”

Companies today are inundated with vast amounts of data from various sources. This overwhelming amount of data is meant to benefit the company, but often leaves data teams feeling overwhelmed, which can create data bottlenecks and result in a slow time to value. In fact, only twenty seven percent of companies agree that data and analytics projects produce insights and recommendations that are highly actionable (Accenture). This means that nearly 3 in 4 companies are not unlocking value in their data, which poses a huge challenge for organizations trying to move the needle and drive real business results. We at Google Cloud, however, saw opportunity in this challenge, which is why we created Data Analytics Design Patterns: cross-product technical solutions designed to accelerate a customer’s path to value realization with their data. These industry solutions bring together product capabilities alongside design methodology, open source deployable code, data models, and reference architectures to accelerate your business outcomes.

With Data Analytics Design Patterns, you get access to more than 30 ready-to-deploy data analytics solutions. Design patterns leverage the best of Google and our rich partner ecosystem, including Technology Partners & System Integrators. In this blog, we will cover 3 examples on how a design pattern can be applied to unlock the value of data:

1. Improve mobile app experience with Unified App Analytics
2. Maximize digital shop’s revenue with Price Optimization 
3. Protect internal systems from security and malware threat with Anomaly Detection 

Unified App Analytics

If mobile apps are part of your go-to-market strategy, you have several data sources that can provide invaluable customer insights. In addition to tools such as CRM (e.g. Salesforce) and customer care (e.g. Zendesk), you likely use Google Analytics to log app events and Firebase Crashlytics to gather data about app errors. But can you easily combine back-end server data with app front-end data to unlock customer insights? 

The Unified App Analytics design pattern makes it easy to plug all the disparate data sources into a single warehouse (BigQuery) and start analyzing it with a Business Intelligence tool (Looker). Once you have a complete and real time view of your customer experience with your app, you can take action. For example, if you notice an increase in app errors, you can quickly combine your Crashlytics data with your CRM data to narrow down the crashes with the highest revenue impact and prioritize their resolution. Further, you can automate your issue resolution workflow by creating a rule for any future crash that impacts a subset of VIP customers.

With the Unified App Analytics design pattern, you’ll gain access to valuable insights about your user experience with your app so you can inform your future app strategy. For example, NPR, an American media company, increased user engagement by showing content that better mapped to listener interests and behaviors.

Price Optimization

In a competitive and hectic global marketplace, strategic pricing matters more than ever, but often projects are consumed by the tedium of standardizing, cleaning, and preparing data—from transactions, inventory, demand, among other sources.

Price Optimization solution allows retailers to build a data driven pricing model. The solution consists of three main components:

• Dataprep by Trifacta: integrates different data sources into a single Common Data Model (CDM). Dataprep is an intelligent data service for visually exploring, cleaning, and preparing structured and unstructured data for analysis, reporting, and machine learning.
• BigQuery: allows you to create and store pricing models in a consistent and scalable way as a serverless Cloud Data Warehouse service
• Looker dashboards: surface insights and enable business teams to take action with enterprise ready BI platform

With the Price Optimization design pattern from Google Cloud and our partner Trifacta, you’ll be able to rapidly unify multiple data sources and create a real-time and ML-powered analysis, leveraging predictive models to estimate future sales. For example, PDPAOLA, an online jewelry company, doubled sales with dynamic pricing adjustments enabled by a single data view.

Anomaly Detection

Organizations need to anticipate and act on risks and opportunities to stay competitive in a digitally transforming society. Anomaly detection helps organizations identify and respond to data points and data trends in high velocity, high volume data sets that deviate from historical standards and expected behaviors, allowing them to take action on changing user needs, mitigate malicious actors and behaviors, and prevent unnecessary costs and monetary losses.

The Anomaly Detection design pattern uses Google Pub/Sub, BigQuery, Dataflow, and Looker to:

• Stream events in real time 
• Process the events, extract useful data points, train the detection algorithm of choice
• Apply the detection algorithm in near-real time to the events to detect anomalies
• Update dashboards and/or send alerts

The challenge of finding the important insights and anomalies in vast amounts of data applies to organizations across all industries and lines of business, but is especially important to protecting the security of an organization. For example, TELUS, a national communications company, modernized their security analytics platform leveraging this pattern, allowing them to detect anomalies in near real time to detect and mitigate suspicious activity.

Get started

Turn your data into business outcomes with Google Cloud and our broad partner ecosystem by deploying Data Analytics Design Patterns at your organization. There are more than 30 Data Analytics Design Patterns ready for you to use. We have more than 200+ more ideas in the pipeline, so be sure to check in regularly as new patterns will be added soon. 

To dive deeper and find out more about how Data Analytics Design Patterns can help your organization accelerate use cases and create faster time to value, check out this video.

As organizations continue to produce vast quantities of data, they increasingly need platforms that allow them to analyze, store, and extract meaningful insights from that data. And research from analyst firms like Gartner offer an important way for organizations to evaluate and compare cloud data warehouse providers.

Earlier this year, Gartner named Google a Leader in the 2019 Gartner Magic Quadrant for Data Management Solutions for Analytics (DMSA) (report available here). This evaluation covers Google Cloud’s core data analytics offerings, including BigQuery, a serverless, managed data warehouse, Cloud Dataproc, a managed Spark and Hadoop service, and Cloud Dataflow, which enables you to stream and batch-process your data. Here are a few takeaways:

Simplicity and speed

BigQuery’s performance permits complex queries on large-scale data sets to return in seconds, and a substantial number of BigQuery customers maintain data warehouses that store more than 50 terabytes (and a few customers now use more than 100 petabytes). More than half of these customers are loading data either continuously or many times per day. These customers value the ability to extract, transform, load, and analyze their data on a serverless platform, all without maintaining any underlying infrastructure.

A versatile serverless data warehouse

One of BigQuery’s major advantages is its ability to allow customers to address a wide variety of use cases—from a traditional data warehouse to data science. Over the past year, we’ve worked hard to introduce new features in BigQuery like data types for financial and monetary uses, BigQuery GIS for geospatial data, and machine learning capabilities through BigQuery ML. BigQuery’s continuous ingest capabilities make it suitable for an operational—or a real-time—serverless data warehouse.

An expanding ecosystem

With overall increased market adoption, our analytics offerings continue to benefit from a fast-growing partner ecosystem of service providers, and business intelligence (BI) and data integration vendors. In particular, in 2018 we expanded our partnerships with established industry providers, including Confluent, Dell Boomi, Informatica, Looker, Reltio, Tableau, and ThoughtSpot.

More and more organizations are finding value in Google Cloud’s serverless data warehouse and analytics offerings. If you’d like to learn more, you can download a complimentary copy of the Gartner Magic Quadrant for Data Management Solutions for Analytics on the Google site (requires an email address).

_{Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.}