In the past few years, a series of new technologies and regulatory changes has been transforming Brazil’s financial industry. The concept of blockchain added security and agility to financial transactions. The open banking system being deployed by the country’s Central Bank (BC) allows for platform integration and data sharing -with the user’s consent- between financial institutions. Recently, the launch of Pix, also by the BC, has shaken the market by creating a new payment method that is instant, free for individuals and 24/7.

Since 2018, the startup Bit Capital has been working on the development of 100%-digital financial solutions in the cloud to help its clients adapt to this new scenario in a convenient way, without the need to build an infrastructure for that or hiring different providers. Pix was not the exception.

In less than 3 months, the team was able to build and go live with a solution that can be used by both direct participants (i.e. those with banking licenses granted by BC) and indirect participants (i.e. companies depending on direct participants to offer payments on Pix) through Bit Capital’s platform.

Besides this differential, Pix and the startup’s other solutions are based on blockchain and Google Cloud’s cloud, ensuring greater security, scalability and availability for customers and allowing for an easy integration between Bit Capital’s platform’s solutions and those from other companies.

“Now we have a platform with an API and various microsservices, allowing clients to connect and develop their own financial product without having to start from scratch.”

—Francesco Miolo, CFO, Bit Capital

A robust structure to handle Pix’s high demand

Bit Capital was established with a certain urgency to develop its platform. The idea was well-developed and customers were interested. The fast deployment of Google Cloud’s tools was one of the main factors that attracted the company to use it as a basis for its infrastructure.

“We managed to deliver everything we have today in Google Cloud with a small team. That was another challenge: being able to grow with a few people,” says Juliano Souza, the startup’s head of IT infrastructure. “We sought other cloud partners, but they had a steep curve. We chose Google Cloud because we needed quick, quality scaling.”

The same thing happened when they built the solution for Pix, despite the specific challenges involved. The BC had performance requirements that led the company to spend some time experimenting until they reached the best suite of tools to meet those requirements. The startup wanted to create a unique architecture that could help both large and small customers and be integrated to the platform’s other microservices.

With the support from Google Cloud’s team to answer doubts and make the best decisions, the solution was built in a few months, at a cost about two-thirds lower than other providers.

The solution’s architecture is based on apps running in Docker in Google Kubernetes Engine (GKE), with interconnected microservices. The blockchain system runs on Compute Engine, its rows are managed in Pub/Sub and Dataflow, and persistent data, in Cloud SQL. Cloud Interconnect is used for the connection with BC. Cloud KMS and Secrets Management store the company’s pre-credentials. All of this is supported by Cloud Load Balancing for load balancing and autoscaling.

According to the team, GKE was essential for the solution’s success. Using infrastructure as code in the tool made uploading of a group of microservices significantly easier. Developers are able to help set up this environment, which has helped spread the DevOps culture in the team. Besides orchestrating and integrating apps, GKE also provides an elastic structure to handle demand peaks and visibility to monitor internal components.

“Google Kubernetes Engine was the only way to ensure availability, observability and elasticity for all clients, whether small, middle-sized or large.”

—Juliano Souza, Head of IT Infrastructure, Bit Capital

Nowadays, the solution’s environment has 20 clusters with over 1,500 pods – 250GB in data per month, providing a robust structure to support a service involving periods of intense demand such as Black Friday.

Ease to monitor, fix and improve

Just 10 days after Pix’s official launch date, the company had its first test: Black Friday. This allowed a major e-commerce customer to test the scale and see the success of the architecture that had been built. “It worked great regarding what the cloud could deliver. And we found what we needed to fix very quickly. Operations [formerly Stackdriver], in particular Cloud Trace, showed us clearly what needed to be done to improve performance,” Souza explains.

Using Operations added reliability by putting deliveries into production. Checking Cloud Trace to see if there were any performance issues and the exact point where they were happening became routine for Bit Capital’s developers. Google Cloud’s security tools and Google Safety Center provided the resources needed to monitor and secure the environment, with automatic data encryption at rest and in transit.

“Google Cloud has security as a premise. When we upload any kind of component, like a database or a virtual machine, the drive is encrypted by default. With other providers, you must specify that you want it encrypted.

”—Juliano Souza, Head of IT Infrastructure, Bit Capital

The easy service monitoring and management accelerates product and technology development because the team no longer needs to worry about infrastructure. Automated management allows professionals to spend more time on new projects and the company’s business. Also, the deployment of services in a Google Cloud multi-region impacts on customer experience, providing high availability for their solutions.

In the coming months, Bit Capital aims to ramp up service usage and the creation of new projects in Google Cloud by adding more customers to Pix’s solution and Banking as a Service (BaaS) solutions. Anthos will be incorporated to the tool suite to make it easier to connect apps with the customers’ on-prem environments. And the deployment of open banking has the potential to be a business driver for the company.

“We joke that we’re already doing open banking, because we have various connections with different providers, which allows us to offer an integrated solution,” says Francesco Miolo, the startup’s CFO. “With the arrival of the new regulations, something we are waiting for since we started out, we will be able, through our platform, to take our customers to the open finance ecosystem, enabling the development of disruptive business models.”

Organizations rely on data warehouses to aggregate data from disparate sources, process it, and make it available for data analysis in support of strategic decision-making. BigQuery is the Google Cloud enterprise data warehouse designed to help organizations to run large scale analytics with ease and quickly unlock actionable insights. You can ingest data into BigQuery either through batch uploading or by streaming data directly to unlock real-time insights. As a fully-managed data warehouse, BigQuery takes care of the infrastructure so you can focus on analyzing your data up to petabyte-scale. BigQuery supports SQL (Structured Query Language), which you’re likely already familiar with if you’ve worked with ANSI-compliant relational databases. 

BigQuery unique features

BI Engine – BigQuery BI Engine is a fast, in-memory analysis service that provides subsecond query response times with high concurrency. BI Engine integrates with Google Data Studio and Looker for visualizing query results and enables integration with other popular business intelligence (BI) tools. 

BigQuery ML: BigQuery ML is unlocking machine learning for millions of data analysts. It  enables data analysts and data scientists to build and operationalize machine learning models directly within BigQuery, using simple SQL.

BigQuery Omni – BigQuery Omni is a flexible, multi-cloud analytics solution powered by Anthos that lets you cost-effectively access and securely analyze data across Google Cloud, Amazon Web Services (AWS), and Azure, without leaving the BigQuery user interface (UI). Using standard SQL and familiar BigQuery APIs, you can break down data silos and gain critical business insights from a single pane of glass. 

Data QnA: Data QnA enables self-service analytics for business users on BigQuery data as well as federated data from Cloud Storage, Bigtable, Cloud SQL, or Google Drive. It uses Dialogflow and enables users to formulate free-form text analytical questions, with auto-suggested entities while users type a question.

Connected Sheets -The native integration between Sheets and BigQuery makes it possible for all business stakeholders, who are already quite familiar with spreadsheet tools, to get their own up-to-date insights at any time.

Geospatial data – BigQuery offers accurate and scalable geospatial analysis with geography data types. It supports core GIS functions – measurements, transforms, constructors, and more – using standard SQL.

How does it work?

Here’s how it works: You ingest your own data into BigQuery or use data from the public datasets. Storage and compute are decoupled and can scale independently on demand. This offers immense flexibility and cost control for your business as you don’t need to keep expensive compute resources up and running all the time. As a result, BigQuery is much more cost-effective than traditional node-based cloud data warehouse solutions or on-premises systems. BigQuery also provides automatic backup and restore of your data. 

You can ingest data into BigQuery in batches or stream real-time data from web, IoT, or mobile devices via Pub/Sub. You can also use Data Transfer Service to ingest data from other clouds, on-premises systems or third-party services. BigQuery also supports ODBC and JDBC drivers to connect with existing tools and infrastructure. 

Interacting with BigQuery to load data, run queries, or create ML models can be done in three different ways. You can use the UI in the Cloud Console, the BigQuery command-line tool, or the API via client libraries available in several languages.

When it comes time to visualize your data, BigQuery integrates with Looker as well as several other business intelligence tools across the Google partner ecosystem.

What about security?

BigQuery offers built-in data protection at scale. It provides security and governance tools to efficiently govern data and democratize insights within your organization. 

• Within BigQuery, users can assign dataset-level and project-level permissions to help govern data access. Secure data sharing ensures you can collaborate and operate your business with trust.
• Data is automatically encrypted both while in transit and at rest, ensuring that your data is protected from intrusions, theft, and attacks. 
• Cloud DLP helps you discover and classify sensitive data assets.
• Cloud IAM provides access control and visibility into security policies.
• Data Catalog helps you discover and manage data. 

How much does it cost?

The BigQuery sandbox lets you explore BigQuery capabilities at no cost and confirm that BigQuery fits your needs. With BigQuery you get predictable price-performance: you pay for storing and querying data, and for streaming inserts. Loading and exporting data are free of charge. Storage costs are based on the amount of data stored, and have two rates based on how often the data is changing. Query costs can be either:

• On-demand – you are charged per query by the amount of data processed
• Flat-rate – if you prefer to purchase dedicated resources 

You can start with the pay-as-you-go, on-demand option and later move to flat-rate if that better suits your usage. Or, start with flat-rate, get a better understanding of your usage and move to the pay-as-you-go models for additional workloads. 

To explore BigQuery and its capabilities a bit more, check out the sandbox; and when you’re ready to modernize your data warehouse with BigQuery then check out the documentation to streamline your migration process here. https://www.youtube.com/embed/So-tVyBQt8E?enablejsapi=1&

For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev.