Compute Engine is a customizable compute service that lets you create and run virtual machines on Google’s infrastructure. You can create a Virtual Machine (VM) that fits your needs. Predefined machine types are pre-built and ready-to-go configurations of VMs with specific amounts of vCPU and memory to start running apps quickly. With Custom Machine Types, you can create virtual machines with the optimal amount of CPU and memory for your workloads. This allows you to tailor your infrastructure to your workload. If requirements change, using the stop/start feature you can move your workload to a smaller or larger Custom Machine Type instance, or to a predefined configuration.

Machine types

In Compute Engine, machine types are grouped and curated by families for different workloads. You can choose from general-purpose, memory-optimized, compute-optimized and accelerator-optimized families. 

• General-purpose machines are used for Day-to-day computing at a lower cost and for balanced price/performance across a wide range of VM shapes. The use cases that best fit here are web serving, app serving, back office applications, databases, cache, media-streaming, microservices, virtual desktops, development environments.
• Memory-Optimized machine are recommended for ultra high-memory workloads such as in-memory analytics and large in-memory databases such as SAP HANA 
• Compute-Optimized machines are recommended for ultra high performance workloads such as High Performance Computing (HPC), Electronic Design Automation (EDA), gaming, video transcoding, single-threaded applications.
• Accelerator-Optimized machines are optimized for high performance computing workloads such as Machine learning (ML), Massive parallelized computations and High Performance Computing (HPC)

How does it work?

You can create a VM instance using a boot disk image, a boot disk snapshot, or a container image. The image can be a public operating system (OS) image or a custom one. Depending on where your users are you can define the zone you want the virtual machine to be created in. By default all traffic from the internet is blocked by the firewall and you can enable the HTTP(s) traffic if needed. 

Use snapshot schedules (hourly, daily, or weekly) as a best practice to back up your Compute Engine workloads. Compute Engine offers live migration by default to keep your virtual machine instances running even when software or hardware update occurs. Your running instances are migrated to another host in the same zone instead of requiring your VMs to be rebooted. 

Availability

For High Availability (HA) Compute Engine offers automatic failover to other regions or zones in event of a failure. Managed instance groups (MIGs) help keep the instances running by automatically replicating instances from a predefined image. They also provide application based autohealing health checks. If an application is not responding on a VM, the auto healer automatically recreates that VM for you. Regional MIGs let you spread app load across multiple zones. This replication protects against zonal failures. MIGs work with load balancing services to distribute traffic across all of the instances in the group. 

Compute Engine offers autoscaling to automatically add or remove VM instances from a managed instance group based on increases or decreases in load. Autoscaling lets your apps gracefully handle increases in traffic, and it reduces cost when the need for resources is lower. You define the autoscaling policy for automatic scaling based on the measured load, CPU utilization, requests per second or other metrics.

Active Assist’s new feature, predictive autoscaling, helps improve response times for your applications–When you enable predictive autoscaling, Compute Engine forecasts future load based on your Managed Instance Group’s (MIG) history and scales it out in advance of predicted load, so that new instances are ready to serve when the load arrives. Without predictive autoscaling, an autoscaler can only scale a group reactively, based on observed changes in load in real time. With predictive autoscaling enabled, the autoscaler works with real-time data as well as with historical data to cover both the current and forecasted load. That makes predictive autoscaling ideal for those apps with long initialization times and whose workloads vary predictably with daily or weekly cycles. For more information, see How predictive autoscaling works or check if predictive autoscaling is suitable for your workload, and to learn more about other intelligent features, check out Active Assist.

Pricing

You pay for what you use. But you can save cost by taking advantage of some discounts! Sustained use saving are automatic discounts applied for running instances for a significant portion of the month. If you know your usage upfront, you can take advantage of committed use discounts which can lead up to significant savings without any upfront cost. And by using short lived preemptive instances you can save up to 80%, they are great for batch jobs and fault tolerant workloads. You can also optimize resource utilization with automatic recommendations. For example if you are using a bigger instance for a workload that can run on a smaller instance you can save costs applying these recommendations.

Security

Compute Engine provides you default hardware security. Using Identity and Access Management (IAM) you just have to ensure that proper permissions are given to control access to your VM resources. All the other basic security principles apply, if the resources are not related and don’t require network communication amongst themselves, consider hosting them on different VPC networks. By default, users in a project can create persistent disks or copy images using any of the public images or any images that project members can access through IAM roles. You may want to restrict your project members so that they can create boot disks only from images that contain approved software that meet your policy or security requirements. You can define an organization policy that only allows Compute Engine VMs to be created from approved images. This can be done by using the Trusted Images Policy to enforce images that can be used in your organization. 

By default all VM families are Shielded VMs. Shielded VMs are virtual machine instances that are hardened with a set of easily configurable security features to ensure that when your VM boots, it’s running a verified bootloader and kernel — is the default for everyone using Compute Engine, at no additional charge. For more details on Shielded VMs refer to the documentation here.

For additional security, you also have the option to use Confidential VM to encrypt your data in use, while it’s being processed in Compute Engine. For more details on Confidential VM refer to the documentation here.

Use cases

There are many use cases Compute Engine can serve in addition to running websites and databases. You can also migrate your existing systems onto Google Cloud, with Migrate for Compute Engine, enabling you to run stateful workloads in the cloud within minutes rather than days or weeks. Windows, Oracle or VMware applications have solution sets enabling a smooth transition to Google Cloud. To run windows applications either bring your own license leveraging Sole-tenant nodes or using the included licenced images. 

Conclusion

Whatever your application use case may be, from legacy enterprise applications to digital native applications, Compute Engine’s families will fit it. For a more in-depth look into Compute Engine check out the documentation. 

For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev

Editor’s note: To launch a new mobile payment platform, Mercari needed a database solution strong on scalability, availability, and performance. Here’s how Cloud Spanner delivered those results. 

E-commerce companies need to connect customers to their services securely, reliably, with zero downtime. When Mercari, Inc. launched a new mobile payment platform, we chose Cloud Spanner as part of our data portfolio, which provided us with easy scalability to handle millions of new users, a fully managed service that minimized overhead costs, and deep integration with other Google Cloud services. 

Mercari, Inc., Japan’s largest C2C marketplace, launched its app in 2013, which allows 18.2million monthly users to easily and securely buy new and used items. Mercari expanded into the United States and in 2014 and launched Merpay, a mobile payment service that can be used through Mercari in Japan in 2017. With more than 85 million users, Merpay is now accepted at 1.8 million merchants and e-commerce sites in Japan, supporting payments via the DOCOMO ID contactless system and QR code.

Prioritizing availability and scalability

When we started building Merpay, we were looking for a new database. In the past, Mercari had used MySQL with bare metal hardware. Because of the amount of data, we required additional expertise to manage and maintain the hardware, software and MySQL implementation. Having built much of the microservices architecture for our Mercari app using Google Kubernetes Engine (GKE), it was natural to look at Google Cloud’s managed services when deciding upon our new database infrastructure for Merpay. 

For the database, we were focusing especially on requirements around availability, scalability, and performance. To do a single payment transaction, there were multiple steps each with writes, requiring high-write-throughput and low-latency from the database. Needing a solution that would support reliable payment processing 24/7/365, we chose Spanner, which offers up to 99.999% availability with zero downtime for planned maintenance and schema changes. 

We worked closely with Google Cloud’s Premium Support, Technical Account Management (TAM), and Strategic Cloud Engineer and Cloud Consultant teams to implement Spanner, including separating the payment processing— originally one of the functions in Mercari—as a microservice.

A nod to easier nodes and better scale

After launch, the number of Merpay users reached 2 million in only a few months. We had 45 Spanner nodes at the time of launch of Merpay and about 50 nodes four months later. Because we’d optimized the application side during those four months, we didn’t have to add many nodes to keep up with the growing traffic. 

Whenever we needed to scale up the serving and storage resources in our instances, Spanner made it easy to increase nodes as needed in the Cloud Console. To optimize costs, it’s easy to add nodes during a marketing campaign and remove them afterward. Even if the traffic count is different from expected, we can change the number of nodes immediately. That’s one incredible convenience of Cloud Spanner.

Building powerful pipelines

Our data pipelines are used for KPI analytics, fraud detection, credit scoring, and customer support use cases. Because Cloud Spanner integrates so easily with other Google Cloud data services, the data in Spanner is easily accessible for analytics. From the Spanner database for each microservice, we create both batch and streaming data pipelines using Pub/Sub and Dataflow, as well as Apache Flink, and load the data into BigQuery and Google Cloud Storage. We’re able to quickly aggregate the data required for data platforms such as BigQuery and Cloud Storage. The original data is stored in Spanner and managed by microservices, but analysis through BigQuery is centrally managed by the Data Platform team. This team determines the confidentiality level of data and centrally manages BigQuery permissions using Terraform. By centrally managing data access permissions on the data platform rather than on individual microservices, we’re able to set appropriate security for individual users.

Our full data portfolio also includes Looker, which we use for product analysis, accounting analysis, test performance visualization, operation monitoring, development efficiency analysis, and HR analysis. We also use Dataproc, Cloud Composer, Data Catalog, and Data Studio. The Dataflow template created for the pipeline is also published as OSS. 

With Spanner and other Google Cloud services, our Merpay platform is flexible, secure, scalable and highly available. As Spanner eliminates overhead, we can devote engineering resources to developing new tools and solutions for our customers. We’re looking ahead at providing cryptocurrency service, for example, and are now working on a project to migrate Mercari’s monolithic system that is still on premises entirely over to Google Cloud.

Read more about Mercari and Merpay. Or check out our recent blog: three reasons to consider Cloud Spanner for your next project.

With 150 million users and counting, Picsart was originally founded as a photo and video editing app we use to enhance our everyday life before uploading them to various social media platforms. As its user numbers continue growing, the company began exploring options for incorporating its easy-to-use editing tools into other applications and services. Now, businesses looking for a low-lift solution to enhance their product offerings with Picsart’s award-winning software suite can do so. Thanks to Apigee, organizations from assorted markets, such as printers, t-shirt makers, and phone case designers, are benefiting from Picsart’s custom design APIs.

Picsart helps companies reach a new audience of customers by offering them graphic design capabilities at an affordable cost, avoiding the resource and time-intensive process of hiring developers and building these functions from scratch. The benefits of APIs are energizing Picsart’s business partners, too; with the addition of photo editing, keeping customers engaged, and returning to their apps.

“Eighty percent of our users are Generation Z and Millennials,” says Jinny Jung, vice-president of Picsart’s API program. “They’re the future of creators all over the globe. They want creative control.”

“Many of the businesses we serve are looking to tap into that audience. They really need solutions like this to take their business to the next level,” Jung adds. Having an API service also takes Picsart to the next level — thanks in part to Apigee.

How to turn a quick pivot

Moving from strictly business-to-consumer (B2C) services to a business-to-business (B2B) offering wasn’t easy for Picsart. The company had to adopt an entirely new business model and develop a new suite of API products — and they did so quickly. 

When a prominent social media site contacted Picsart about offering its editing tools to their users via APIs, Picsart knew it couldn’t miss this opportunity. However, none of their developers knew how to manage APIs for external use, an endeavor involving multiple processes and fraught with logistical complications. Failing to configure APIs correctly allows malicious actors to effortlessly bypass cybersecurity protections, exposing network data.

The company quickly assembled an API business unit to plan and coordinate the creation of a secure API development process. This process would support marketing, sales, distribution, and other facets of the business, all while still delivering great functionality to the company’s original B2C customers. And voila! Picsart was reborn. 

The API BU works as “a scrappy startup within a larger company,” as Jung puts it. Its budget is scrappy too. Among the unit’s first tasks: finding an API management platform that could simplify and streamline all the tasks the team faced at an affordable price. Jung says that Apigee was a clear choice.

“Apigee’s offerings gave us the flexibility to get off the ground without a huge budget and grow our business over time. It’s super easy to use, the dashboard is very intuitive, and the price came in well for us.”

The company had already used APIs internally, but had never provided them to customers or partners.  

“We needed to lean on gateways and management platforms that already knew how to set everything up,” Jung says. “We really needed the shortcut.”

Her team did consider other API platforms, but being part of Google Cloud gave Apigee a natural edge.

“We were already Google Cloud clients. We were already part of the Google Cloud ecosystem. We really liked Apigee’s ease of use and the power of Google behind it.”

Apigee works seamlessly in tandem with its other Google Cloud services, including the Google Cloud Content Delivery Network (CDN) and BigQuery, Google Cloud’s fully-managed data warehouse.

“Apigee’s integration with the whole Google ecosystem makes our lives easier, and makes it quite natural for us to use,” says Tiberiu Stanescu, director of engineering for the Picsart API business unit. “It allows us to also meet our technical needs, including scalability and even improved performance with queries and, when we’re integrating the data, dashboards.” 

Using Apigee for total API control

Picsart uses Apigee to manage all its external APIs, which include those that enable users to remove or change photo backgrounds, apply filters, and add text. Apigee also manages the API features for Picsart’s Editor SDK package.

Key ways in which Picsart uses Apigee include:

• API management. Apigee runs test/stage and production environments, can customize pre-and post-flows, and automatically authorize for services with different methods including API Keys and oAuth.
  Apigee’s pre-built integrated portal has helped Picsart “greatly” in the time the company has been using it, Stanescu says. He praised its flexibility and its rate-limiting function, “which allows us to properly define and secure our products.”

• Developer support. With Apigee, Picsart now takes an “API-first” approach to developing new features by using microservice architectures. They’re exposing APIs via their developer portal, which allows them to describe precisely how developers can use each API.
  
• Apigee helps Picsart’s own developers use APIs to simplify the development process and save the company time and money. In conjunction with other Google products, Apigee is providing Picsart with the data and analytics it needs to view and monitor API-related productivity improvements.

• Security. Apigee’s out-of-the-box security requires no specialized knowledge or skill to implement. Picsart relies on Apigee security features such as including headers to set up security parameters, and custom rate-limiting policies and quotas, allowing the company to avoid stack overflow from API calls.

As a result, the company has experienced zero API-related security incidents, Stanescu notes. “We’re definitely grateful to Apigee for this,” he adds. Recently, Stanescu says, a company performed penetration testing on Picsart’s APIs, looking for vulnerabilities, but failed to discover any security flaws or threats. Apigee’s easy-to-use, but powerful security features are helping Picsart keep its clients and customers safe and secure.

From scrappy startup to business core

In the three years since implementing Google Apigee, Picsart’s API business unit has gone from a seemingly simple request to a critical product offering enabling new and recurring revenue.

“It’s part of the foundation of our business,” says Jinny Jung. “It powers our entire API external-facing business. We wouldn’t be able to sell APIs to developers and partners without the Apigee API gateway. We wouldn’t have gotten off the ground without Apigee.”

Apigee is helping Picsart in another key way, too: its integration with products like BigQuery, Google’s data analytics platform that lets it stay on top of which APIs are most used. That information guides every aspect of the business and helps the company develop and release new generative AI-powered products.

“The data absolutely informs what we choose to develop and where we put our resources,” Jung says. “Apigee makes it super easy for us to track API use and see the trends.”

With Apigee, Picsart is opening up a new world of creative design to audiences across all markets and demographics, while avoiding the stress that comes with a costly development process. Today, generative AI apps continue capturing imaginations, unlocking the ability to create awe-inspiring art with only a few natural language prompts. Picsart and Apigee will continue working together to provide a canvas through APIs no matter where an artist may be.