App Engine is a fully managed serverless compute option in Google Cloud that you can use to build and deploy low-latency, highly scalable applications. App Engine makes it easy to host and run your applications. It scales them from zero to planet scale without you having to manage infrastructure. App Engine is recommended for a wide variety of applications including web traffic that requires low-latency responses, web frameworks that support routes, HTTP methods, and APIs.

Environments

App Engine offers two environments; here’s how to choose one for your application:

• App Engine Standard – Supports specific runtime environments where applications run in a sandbox. It is ideal for apps with sudden and extreme traffic spikes because it can scale from zero to many requests as needed. Applications deploy in a matter of seconds. If your required runtime is supported and it’s an HTTP application, then App Engine Standard is the way to go.
• App Engine Flex – Is open and flexible and supports custom runtimes because the application instances run within Docker containers on Compute Engine. It is ideal for apps with consistent traffic and regular fluctuations because the instances scale from one to many. Along with HTTP applications it also supports applications requiring WebSockets. The max request timeout is 60 minutes. 

How does it work

No matter which App Engine environment you choose, the app creation and deployment process is the same. First write your code, next specify the app.yaml file with runtime configuration, and finally deploy the app on App Engine using a single command: gcloud app deploy.

Notable features

• Developer friendly – A fully managed environment lets you focus on code while App Engine manages infrastructure. 
• Fast responses – App Engine integrates seamlessly with Memorystore for Redis enabling distributed in-memory data cache for your apps.
• Powerful application diagnostics – Cloud Monitoring and Cloud Logging help monitor the health and performance of your app and Cloud Debugger and Error Reporting help diagnose and fix bugs quickly. 
• Application versioning – Easily host different versions of your app, and easily create development, test, staging, and production environments.
• Traffic splitting – Route incoming requests to different app versions for A/B tests incremental feature rollouts, and similar use cases.
• Application security – Helps safeguard your application by defining access rules with App Engine firewall and leverage managed SSL/TLS certificates by default on your custom domain at no additional cost.

Conclusion

Whether you need to build a modern web application or a scalable mobile backend App Engine has you covered. For a more in-depth look, check out the documentation. Click here for demos on how to use serverless technology and free hands-on training.https://www.youtube.com/embed/Xuf3J6SKVV0?enablejsapi=1&

For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev.

Modern developers worldwide have grown accustomed to the comfort of writing code, pushing to a remote Git repository and having that code be deployed at an accessible URL without having to worry about how it is deployed. This was a workflow popularized by Heroku years ago which brought joy and productivity to developers even if it did impose some loss of flexibility for operation teams.

To address that loss of flexibility when meeting security and integration requirements, Heroku introduced Private Spaces. Private Spaces provide network isolation from the internet since any application or datastore provisioned by Heroku is accessible to the internet by default.

Cloud Run is quickly becoming the “swiss army knife” of serverless here at Google Cloud and it’s a natural migration path for developers accustomed to Heroku. The fundamentals are all there:

• Continuously Deploy via Git push using open source Buildpacks or Dockerfiles
• Set CPU and Memory requirements for each instance
• Horizontally scalable apps that scale from zero to thousands of instances to meet traffic demands automatically

So while devs are kept happy, can Cloud Run do something for the Ops folks? Yes. Here are some things available right in the Cloud Run UI:

• Proper Secret Management with IAM based access control. No more setting secrets as environment variables.
• Traffic management between different revisions for blue-green or canary deployments.
• Define SLIs and SLOs with ease. Eg: 90% of requests have to be served under 200ms in a calendar month.
• Secure your service with tools such as Software Delivery Shield, Binary Authorization, and Cloud Armor. Definitely deserves its own blog post.

Recreating Private Spaces on Cloud Run

Let’s focus on network isolation now, let’s say you have an internet-facing app and a private backend API that talks to a private database. Simplest architecture ever, it conceptually looks a bit like this:

Let’s address the database first. If you want to use Postgres then Cloud SQL is most likely what you want, but do keep in mind that we have other datastores that speak Postgres such as AlloyDB and Spanner.

Cloud SQL allows you to create a Postgres instance that’s isolated from the internet by simply unchecking the Public IP checkbox and checking the Private IP checkbox. This will assign an IP address to your Postgres instance on your project’s network.

Once the DB is provisioned you’ll see the IP clearly listed, such as:

Of course there’s so much more to say about CloudSQL, to learn more please take a look at our documentation.

Ok now that you’ve dealt with Postgres, let’s address the private backend API on Cloud Run.

When creating a new Cloud Run service via the Google Cloud Console, Ingress can be limited to “Internal traffic only” so only traffic from internal sources, including your VPC, can access the service. In other words, the internet can not touch it.

As an additional level of security, it’s also possible to enforce that only requests from authorized users be served, In this case a “user” is most likely another service using its associated service account which will need the “roles/run.invoker” in order to call this service.

Now let’s make sure that our Backend API Service can reach the Postgres instance by configuring a VPC Connector. This will allow Cloud Run services to reach into the VPC and therefore, the internal IP for the Postgres instance.

Once the VPC Connector is created, you can associate it with a Cloud Run service.

Then it’s just a matter of configuring your code to use the Postgres instance’s private IP address. A good 12-Factor app friendly spot to do that is with a connection string in an environment variable as part of the Cloud Run service configuration. Better yet, as this may contain a DB password, you can use Secret Manager to mount this environment variable from an encrypted and protected secret.

Finally, let’s now set up that Front End Cloud Run service which will respond to requests from the internet, and securely communicate with the backend API service.

For the frontend service choose to “Allow all traffic” and also “Allow unauthenticated invocations” so anyone on the web can access our URL. We could of course choose the middle option and use Cloud Load Balancing in conjunction with Cloud Armor which provides defenses against DDoS and application attacks, and offers a rich set of WAF rules. However, let’s keep it simple for now.

Keep in mind that our Backend service will only accept requests from within our VPC network, and that we don’t have a private IP address for Cloud Run.

So let’s ensure that all egresses from our Frontend actually get routed to the VPC Connector, this way when our Frontend calls a Backend API via it’s URL endpoint, the Backend will receive the request from within the VPC and allow it in.

PS: If your Backend requires authentication don’t forget to create a Service Account for your Frontend Service and then give it the necessary role following a service-to-service auth pattern.

And that’s it. You now have an operationally acceptable private space like environment with an app composed of two Cloud Run services where the Backend service and Postgres instance are network isolated from the Internet. If after reading this blog you would like to get hands-on experience with the technologies mentioned above, then take a look at Google Cloud Skills Boost. There you will find learning paths, quests, and labs curated to boost your cloud skills in a particular area.

For example here’s a great lab that takes you through developing a REST API on Cloud Run using Go.

At enterprises across industries, documents are at the center of core business processes. Documents store a treasure trove of valuable information whether it’s a company’s invoices, HR documents, tax forms and much more. However, the unstructured nature of documents make them difficult to work with as a data source. We call this “dark data” or unstructured data that businesses collect, process and store but do not utilize for purposes such as analytics, monetization, etc. These documents in pdf or image formats, often trigger complex processes that have historically relied on fragmented technology and manual steps. With compute solutions on Google Cloud and Document AI, you can create seamless integrations and easy to use applications for your users. Document AI is a platform and a family of solutions that help businesses to transform documents into structured data backed by machine learning. In this blog post we’ll walk you through how to use Serverless technology to process documents with Cloud Functions, and with workflows of business processes orchestrating microservices, API calls, and functions, thanks to Workflows.

At Cloud Next 2021, we presented how to build easy AI-powered applications with Google Cloud. We introduced a sample application for handling incoming expense reports, analyzing expense receipts with Procurement Document AI, a DocAI solution for automating procurement data capture from forms including invoices, utility statements and more. Then organizing the logic of a report approval process with Workflows, and used Cloud Functions as glue to invoke the workflow, and do analysis of the parsed document.

We also open sourced the code on this Github repository, if you’re interested in learning more about this application.

In the above diagram, there are two user journeys: the employee submitting an expense report where multiple receipts are processed at once, and the manager validating or rejecting the expense report. 

First, the employee goes to the website, powered by Vue.js for the frontend progressive JavaScript framework and Shoelace for the library of web components. The website is hosted via Firebase Hosting. The frontend invokes an HTTP function that triggers the execution of our business workflow, defined using the Workflows YAML syntax. 

Workflows is able to handle long-running operations without any additional code required, in our case we are asynchronously processing a set receipt files. Here, the Document AI connector directly calls the batch processing endpoint for service. This API returns a long-running operation: if you poll the API, the operation state will be “RUNNING” until it has reached a “SUCCEEDED” or “FAILED” state. You would have to wait for its completion. However, Workflows’ connectors handle such long-running operations, without you having to poll the API multiple times till the state changes. Here’s how we call the batch processing operation of the Document AI connector:

  - invoke_document_ai:
    call: googleapis.documentai.v1.projects.locations.processors.batchProcess
    args:
        name: ${"projects/" + project + "/locations/eu/processors/" + processorId}
        location: "eu"
        body:
            inputDocuments:
                gcsPrefix:
                    gcsUriPrefix: ${bucket_input + report_id}
            documentOutputConfig:
                gcsOutputConfig: 
                    gcsUri: ${bucket_output + report_id}
            skipHumanReview: true
    result: document_ai_response

Machine learning uses state of the art Vision and Natural Language Processing models to intelligently extract schematized data from documents with Document AI. As a developer, you don’t have to figure out how to fine tune or reframe the receipt pictures, or how to find the relevant field and information in the receipt. It’s Document AI’s job to help you here: it will return a JSON document whose fields are: line_item, currency, supplier_name, total_amount, etc. Document AI is capable of understanding standardized papers and forms, including invoices, lending documents, pay slips, driver licenses, and more.

A cloud function retrieves all the relevant fields of the receipts, and makes its own tallies, before submitting the expense report for approval to the manager. Another useful feature of Workflows is put to good use: Callbacks, that we introduced last year. In the workflow definition we create a callback endpoint, and the workflow execution will wait for the callback to be called to continue its flow, thanks to those two instructions:

  - create_callback:
    call: events.create_callback_endpoint
    args:
        http_callback_method: "POST"
    result: callback_details
...
- await_callback:
    try:
        call: events.await_callback
        args:
            callback: ${callback_details}
            timeout: 3600
        result: callback_request
    except:
        as: e
        steps:
            - update_status_to_error:
              ...

In this example application, we combined the intelligent capabilities of Document AI to transform complex image documents into usable structured data, with Cloud Functions for data transformation, process triggering, and callback handling logic, and Workflows enabled us to orchestrate the underlying business process and its service call logic.

Going further 

If you’re looking to make sense of your documents, turning dark data into structured information, be sure to check out what Document AI offers. You can also get your hands on a codelab to get started quickly, in which you’ll get a chance at processing handwritten forms. If you want to explore Workflows, quickstarts are available to guide you through your first steps, and likewise, another codelab explores the basics of Workflows. As mentioned earlier, for a concrete example, the source code of our smart expense application is available on Github. Don’t hesitate to reach out to us at @glaforge and @asrivas_dev to discuss smart scalable apps with us.

Whether they’re reading the newspaper on the way to work, or catching up on the latest headlines on their smartphones, readers expect up-to-the-minute news wherever and whenever makes the most sense for them. As a result, media companies are increasingly looking for ways to improve, expand, and simplify their offerings, and they’re increasingly looking to the cloud to do it.

For more than 160 years The Telegraph has been counted on by readers across the United Kingdom and globally for award-winning news and journalism. An early adopter of cloud technology, it’s been a G Suite customer since 2008 and has already been using Google Cloud Platform to analyze digital behaviors to improve engagement and advertising performance since 2016.

Recently, The Telegraph announced it’s migrating fully to Google Cloud. By migrating all their production and pre-production services, they aim to deliver content faster, provide compelling experiences to readers, and reduce environmental impact.

“We are delighted to announce our newest collaboration with Google Cloud,” said Chris Taylor, Chief Information Officer, The Telegraph. “We have always worked closely with Google as they help us to provide our readers with great experiences on our digital products, collaboration software and internet scale through search. Their continued leadership in projects such as Kubernetes are enabling us to build flexible development environments that truly support DevOps.”

Powering the Digital Publishing Ecosystem

The Telegraph produces large volumes of digital content every day. It was imperative for them to find a cloud provider they could trust to support this ecosystem. By working with Google Cloud they have changed the way they see and engage with data: they can collect new information about their products every second and use that to continually hone their strategy. The Telegraph are placing more confidence and trust in the data captured about their content and now have one of the best available pieces of technology for capturing and analyzing the stories they publish in real-time.

Leveraging AI to support journalists

Time is critical when journalists are on a story, and The Telegraph wants to put important data in the hands of its journalists right when they need it. To do this, it will be using AutoML to classify content for journalists and make it more discoverable. For example, a reporter will be able to bring up relevant assets that link to their stories. It will also apply AutoML to classify Telegraph stock photos to help journalists attach compelling visual content to their stories faster.

Building compelling reader experiences with the help of APIs

Readers have an ever-increasing expectation of personalization. To meet this need, The Telegraph launched My Telegraph, currently live in beta, to offer registered readers personalized news experiences based on their interests or the particular journalists they want to follow. My Telegraph was developed on an API management platform provided by Google Cloud’s Apigee. You can learn more about how it’s applying API management to My Telegraph, in this blog post.

Working for environmental good

The Telegraph is the biggest selling quality newspaper in the UK, an accolade which requires it to print and distribute hundreds of thousands of copies each day. Optimal management of print production is important, and by using a combination of the cloud and machine learning, The Telegraph is better able to predict demand for physical newspapers, maximizing sales and minimizing waste. This makes great business sense for The Telegraph but also has great environmental benefit.