Go is the world’s leading programming language for cloud-based development, used by millions of developers to build and scale their cloud applications and business-critical cloud infrastructure. Whether it’s building CLIs, web applications, or cloud and network services, developers find Go easy to learn, easy to maintain, and packed with useful features such as built in concurrency and a robust standard library.

And now, getting started with Go on Google Cloud is a little bit easier. Following Go’s recent announcement of gonew, an experimental tool for instantiating new projects in Go from predefined templates, Google Cloud is releasing four templates for gonew that developers can use to bootstrap their Go applications across several Google Cloud products. This includes common use cases such as creating a simple HTTP handler Cloud Function, subscribing to a Cloud Pub/Sub topic, or creating an HTTP Server on Cloud Run.

Google Cloud Go templates

• httpfn: A basic HTTP handler (Cloud Function)
• pubsubfn: A function that is subscribed to a PubSub topic handling a Cloud Event (Cloud Function)
• microservice: An HTTP server that can can be deployed to a serverless runtime (Cloud Run)
• taskhandler: An basic app that handles tasks from requests (App Engine)

Let’s get started

• Make sure you have Go installed on your machine, if not you need to download/install from here .
• Start by installing gonew using go install:
  
  $ go install golang.org/x/tools/cmd/gonew@latest
  To generate a basic HTTP Handler Cloud Function, instantiate the existing httpfn template by running gonew in your new project’s parent directory. As of now, gonew’s syntax expects two arguments: first, the path to the template you wish to invoke, and second, the module name of the project you are creating. For example:
  $ gonew github.com/GoogleCloudPlatform/go-templates/functions/httpfn yourdomain.com/httpfn
• Deploy this new go module to Cloud Functions by following the steps listed here, you can also try this in the Cloud Shell editor.

What’s next?

We have big plans for Go on Google Cloud, with several new enhancements on our roadmap. In the meantime, please try out these new templates, deploy them to Google Cloud using the instructions provided and let us know how we can make them better. Please use this to report any issues.

Data-savvy businesses increasingly rely on images and videos for critical functions, and yet are challenged by the sheer mass of information—more than 3.2 billion images and 720,000 hours of video are created daily. This explosion in visual data has paved the way for the growth of computer vision, a form of artificial intelligence (AI) that enables computers to “see” the world similarly to the way people do, but with unblinking consistency, and greater accuracy. 

The transformational impact and value of computer vision solutions are significant and has been a guiding objective for companies and AI developers. And yet, even as the applications for computer vision increase dramatically, architecting and implementing vision AI solutions remain highly complex. Visual data, such as images and video, are made up of thousands of pixels of information that represent millions of different patterns and meanings, which can make interpreting even a single image overwhelming from a computational perspective.

Many organizations struggle with deployments and fail to operationalize vision AI solutions due to development delays, machine learning and data science hiring challenges, inaccurate output, a lack of integration with existing infrastructure, difficulty of use, and high cost. Plainsight, with the power of Google Cloud resources, is addressing all these challenges and helping businesses by enabling the deployment of vision AI within enterprise private networks that can be managed easily and scaled economically.

Plainsight has announced availability of its vision AI platform on Google Cloud Marketplace. Businesses can now easily deploy end-to-end vision AI to private clouds to realize the full value of their video and other visual data for accurate, actionable insights across diverse use cases.

Delivering on the Promise of AI: Seeing What’s Hiding In Plain Sight

For organizations to integrate AI and machine learning into their businesses successfully, the technology must be powerful enough to solve real challenges, yet fast, easy, and accessible enough to ensure the innovation potential is realized. Plainsight on Google Cloud delivers the power of enterprise vision AI that’s quick and easy to use with Google Cloud resources that enable global scale, increased security, bolstered privacy, unified billing, and cost savings. 

To streamline vision AI workflows, Plainsight facilitates the entire pipeline, from visual data ingestion and annotation, through continuous model training, deployment, and monitoring for easier innovation and faster time-to-production. Our platform accelerates vision AI development in a manner that is complete, accurate, and accessible to non-technical business leaders. We believe that AI should be available and accessible to anyone and everyone—so that teams across entire organizations can reap the benefits. 

By integrating Plainsight into their private networks, companies worldwide can now leverage one intuitive platform for centralized control of streamlined vision AI model creation and training with optimized visual data handling for diverse enterprise solutions. These use cases include: social distancing monitoring, medical imaging, drug compound screening, defect detection in manufacturing processes, identifying gas leaks, or even livestock counting and crop health monitoring for agriculture, to name a few.https://www.youtube.com/embed/A7U_0UkjvEg?enablejsapi=1&

We enable customers so they can create successful solutions that enable them to clearly see their business from all angles and to take advantage of the knowledge visual data can reveal by simply and quickly operationalizing practical vision AI applications.

AI-Powered Dataset Creation, Automated Model Training & Easy Deployment Without A Single Line Of Code 

For vision AI applications, success is inextricably dependent on the quality and quantity of the datasets required to train the relevant models. To aid enterprises in this vital stage, the Plainsight platform provides built-in data annotation for the fast and easy creation of datasets. This includes AI-powered features that accelerate the speed and quality of labeling such as SmartPoly, for the automated polygon masking of objects, TrackForward, to predict and automatically label objects from frame to frame in video annotations, and AutoLabel for automated object recognition and labeling based on pre-trained machine learning models, to highlight a few.

In addition, to ensure the success of AI integration, we significantly reduce time-intensive processes with Plainsight vision AI’s automated machine learning with continuous model training and easy deployment capabilities. In just a few clicks, users can leverage optimizations for the most reliable model training without endless experimentation cycles. And, models are easily deployed at scale all within one, easy-to-manage model operationalization process for the business.

Growing With Google Cloud

Plainsight is a vision AI innovation leader, developing solutions that address unmet needs for challenger brands and Fortune 500s across vertical markets. As a team recognized for succeeding where others have failed, our expanding partnership with Google Cloud provides a powerful combination that helps customers see and activate the value of their visual data with a suite of services in a secure and private manner.

Our vision AI Platform simplifies building and operationalizing AI to solve business problems enterprises are facing every day—and the demand is increasing. To accelerate our journey to faster, more accessible AI for enterprises, we knew we needed strong support to grow Plainsight and scale our backend tools to match our vision. 

Google Cloud delivered everything, and more, in one program. The Startup Program by Google Cloud provided the technology and services for scale and the support we needed to maximize the value the Program provided us. The Startup Program has been a springboard for architecting Plainsight vision AI in the cloud, accelerating our goals and optimizing innovation, efficiency, and growth. The team also helped us optimize Google Ads campaigns, fueling adoption of Plainsight. 

After launching the SaaS version of Plainsight Data Annotation in November 2020, we grew our user base by nearly 110x in just three short months. Google Ads has also dramatically increased website traffic, growing new users by nearly 5.75X and page views by over 5X. The Google team helped us identify where Google Cloud offerings could be leveraged instead of developing in-house solutions and offered best practices that enabled us to deliver faster on our initiatives. 

Kubernetes was already the underlying component of our platform and leveraging Google Kubernetes Engine (GKE) as a managed service removed a layer of complexity. By combining GKE and Anthos, we were able to standardize our deployments, aligning to how our customers leverage Anthos for enterprise applications in their own organizations. In addition, as a fast-moving, customer-centric company we use Google Workspace to help us centralize and manage our day-to-day work internally. By leveraging multiple products across Google’s ecosystem, we take advantage of a holistic partnership that has helped our business tremendously as we scale. 

Leveraging Google’s Partners for Strategic Consultation

To facilitate this expansion of our partnership with Google and to maximize our use of Google Cloud services, we are working with DoiT International, a Google Managed Services Provider and 2020 Global Reseller Partner of the Year. DoiT provides us with ongoing technical consultation for cloud-native architecture, Google Cloud Marketplace integration, production-grade Kubernetes support, Google Cloud cost optimization, and technical support. The DoiT team has been invaluable in compiling best practices, tips, and strategies from their vast experience with various cloud customers to ease our Marketplace integration and is providing input for infrastructure strategy to support our continued rapid growth.

Plainsight Delivers Enterprise Vision AI Through The Google Cloud Platform Marketplace

Plainsight vision AI is now available to Google Cloud Customers on Google Cloud Marketplace enabling organizations across industries to deploy private Plainsight instances within their own environments. Marketplace customers will benefit from Google Cloud privacy, security, scalability and unified billing through their Google Cloud account. 

Combining the powerful benefits provided by Google Cloud resources with Plainsight’s vision AI Platform into private networks, enterprises worldwide can now leverage one intuitive platform for centralized control of streamlined vision AI model creation and training with optimized visual data handling for diverse enterprise solutions. 

Through our Google partnership, we’re able to leverage a powerful foundation that allows us to rapidly innovate, scale and accelerate delivery on our vision AI platform capabilities. By executing on our vision to make AI easier, faster and more accessible for all users across entire enterprises, we’re helping businesses see more and by seeing more, they’ll have the power to solve more. 

If you want to learn more about how Google Cloud can help your startup, visit our page here where you can apply for our Startup Program, and sign up for our monthly startup newsletter to get a peek at our community activities, digital events, special offers, and more.

Editor’s note: We’re hearing here how martech provider Wunderkind easily met the scaling demands of their growing customer base on multiple use cases with Cloud Bigtable and other Google Cloud data solutions.

Wunderkind is a performance marketing channel and we mostly have two kinds of customers: online retailers, and publishers like Gizmodo Media Group, Reader’s Digest, The New York Post and more. We help  retailers boost their e-commerce revenue through real-time messaging solutions designed for email, SMS, onsite, and advertising. Brands want to provide a one-to-one experience to more of their customers, and we use our extensive history with best practices in email marketing and technology to help brands reach more customers through targeted messaging and personalized shopping experiences. With  publishers, it’s a different value proposition, we use the same platform to provide a non disruptive and personalized ad experience for their website. For example, if you are on their site and then you left, we might show an ad tailored to you when you come back later – depending on the campaign. 

After running into limitations with our legacy database system, we turned to Cloud Bigtable and Google Cloud, which helped us be more flexible and easily scale for high traffic demand – which can be a stable 40,000 requests per second, and meet the needs of our growing number of data use cases. 

Three different databases power our core product

In our core offering, companies send us user events from their websites. We store these events and later decide (using our secret sauce) if and how to reach out to those users on behalf of our customers. Because many of our customers are retailers, Black Friday and Cyber Monday are big traffic days for us as. On such days, we can get 31 billion events, sometimes as many as  200K events per second. We show 1.6 billion impressions that have seen close to 1 billion pageviews. And at the end of all this, we securely send about 100 million emails. We noticed the same thing for election time; traffic reached the same high volume. We need scalable solutions to support this level of traffic as well as the elasticity to let us pay only for what we use, and that’s where Google Cloud comes in.

So how does this work? Our externally facing APIs, which are running on Google Kubernetes Engine, receive those user events—up to hundreds of thousand per second. All the components in our architecture need to be able to handle this demand. So from our APIs, those events go to Pub/Sub, Dataflow and from there they are written to Bigtable and BigQuery, Google Cloud’s serverless, and highly scalable data warehouse. This business user activity data underpins almost all our products. Events can be things like product views or additions to shopping carts. When we store this data in Bigtable, we use a combination of email address and the customer ID as the Bigtable key and we record the event details in that record. 

What do we do with this information next? It’s important to mention that we also mark the last time we received an event about a user in Memorystore for Redis, Google Cloud’s fully managed Redis service. This is important because we have another service that is periodically checking Memorystore for users that have not been active for a campaign-specific period of time (it can be 30 minutes, for example), then deciding whether to reach out to them.

How we decide when we reach out is an intelligent part of our product offering, based on the channel, message, product, etc. When we do reach out, we use Memorystore for Redis as a rate limiter or token bucket. In order not to overwhelm the email or texting providers we send API requests to, we throttle those requests using Memorystore. (We prefer to preemptively throttle the outgoing API requests as opposed to handling errors later.)

When we do reach out, often we will need details for a specific product—let’s say if the website belongs to a retailer. We usually get that information from the retailer through various channels and we store product information in Cloud SQL for MySQL. We pull that information when we need to send an email with product information, and we use Memorystore for Redis to cache that information, since many of the products are repeatedly called. Our Cloud SQL instance has 16 vCPUs, 60GBs of memory and 0.5TB of disk space and when we perform those product information updates, we have about a thousand write transactions per second. We are also in the process of migrating some tables from a self managed MySQL instance, and we keep those tables synchronized with Cloud SQL using Datastream. 

Our user history database was originally stored in AWS DynamoDB, but we were running into problems with how they structured the data, and we’d often get hot shards but with no way to determine how or why. That led to our decision to migrate to Bigtable. We set up the migration first by writing the data to two locations from Pub/Sub, performed some backfill of data until that was up and running, and then started working on the reading. We performed this over a few short months, then switched everything to Bigtable. 

So, as mentioned, we are using Bigtable for multiple databases. The instance that stores our user events has about 30 TB with about 50 nodes.

Profile management

A second use case for Bigtable is for user profile management, where we track, for example, user attributes based on subscription activity, whether they’ve opted in or out of various lists, and where we apply list-specific rules that determine which targeted emails we send out to users. 

Our very own URL shortener

Our third use case for Bigtable is our URL shortener. When our customers build out campaigns and choose a URL, we append tracking information to the query string of the URLs and they become long. Many times, we are sending them via SMS texts, so the URLs need to be short. We originally used an external solution, but made the determination that they couldn’t support our future demands. Our calls tend to be very bursty in nature, and we needed to plan for a future state of supporting higher throughput. We use a separate table in Bigtable for this shortened URL. We generate the short slug that is 62 bit-encoded and use it as the rowkey. We use the long slug as a Protobuf-encoded data structure in one of the row cells and we also have a cell for counting how many times it was used. We use Bigtable’s atomic increment to increase that counter to track how many times the short slug was used. 

When the user receives a text message on their phone, they click the short URL, which goes through to us, and we expand it to the long slug (from Bigtable) and redirect them to the appropriate site location. Obviously, for the URL shortener use case, we need to make the conversion very quickly. Bigtable’s low latency helps us meet that demand and we can scale it up to meet higher throughput demands.

Meeting the future with Google Cloud

Our business has grown considerably, and as we keep signing up new clients, we need to scale up accordingly, and Bigtable has met our scaling demands easily. With Bigtable and other Google Cloud products powering our data architecture, we’ve met the demand of incredibly high traffic days in the last year, including Black Friday and Cyber Monday. Traffic for these events went much higher than expected, and Bigtable was there, helping us easily scale on demand. 

We are working on leveraging a more cloud native approach and using Google Cloud managed services like GKE, Dataflow, pub/sub, Cloud SQL , Memorystore, BigQuery and more. Google has those 1st party products and we don’t see the value in rolling out or self managing such solutions ourselves..

Thanks to Google Cloud, we now have reliable and flexible data solutions that will help us meet the needs of our growing customer base, and delight their users with fast, responsive, personalized shopping messaging and experiences. 

Learn more about Wunderkind and Cloud Bigtable. Or check out our recent blog exploring the differences between Bigtable and BigQuery.