As Google Cloud has become a choice for more startups, I’ve experienced an increase in founders asking how they should think about cloud services. Though each startup is different and requirements may vary across industries and regions, I’ve seen a few core best practices that help startups to succeed—as well as several traps to avoid. 

For example, If you go with a public cloud provider, you’re ideally not starting from ground zero like you would running your own data center, but it’s important not to introduce similar complexity in a virtualized environment. Just because you’re using public cloud infrastructure doesn’t mean you want to manage it.

Instead, you want to leverage platforms that abstract away complexity so your team can focus on delivering value to customers. That’s where serverless comes in. Serverless platforms are fully managed by the provider, offering automatic scaling for workloads, as well as provisioning, configuring, patching, and management of servers and clusters. Freed from these resource-intensive tasks, your technical talent can focus on the things that differentiate your business, not on IT curation. 

This applies to not only running stateless applications, but also managing and analyzing data. You should be collecting data points about which features are most popular on your platform, what people are buying, the types of activities that help your customers get their jobs done, and so on. This information is essential to building and executing on a product roadmap that will serve your customers. However, it’s not enough to collect this data, you need to make your data accessible, secure, and easy for your team to analyze—so where do you run and host it? 

On Google Cloud, this is where options like Spanner and Cloud SQL can play a large role, as can BigQuery for analysis. You won’t have to worry about standing up infrastructure or patching servers—you can just stream your data to our data management platforms, where it’s available whenever someone needs to run a query. By leveraging a serverless architecture, your startup can be data-driven without having to invest in the traditional complexity of database administration and management—and that can significantly change your playing field. 

Serverless is just one of the factors you should consider as you build out your tech stack. To hear my thoughts on a range of other topics relevant to startups — such as security, cloud credits, and the differences among managed services — check out the below video or visit our Build and Grow page.

https://www.youtube.com/embed/gn4RjLbs8Hc?enablejsapi=1&

Based in Mountain View, California, Khan Academy is a not-for-profit that produces and posts a vast collection of free educational online videos about math and science topics ranging from algebra and trigonometry to biology and economics.

Millions of students, educators, and self-learners around the world watch the videos, both on the Khan Academy’s YouTube channel and on its hugely popular website (www.khanacademy.org), where students answer some 1.5 million practice questions per school day. The Khan Academy development team continually tweaks the site based on how visitors choose to learn.

Challenge

Khan Academy’s beginnings date back to 2004, when Sal Khan’s cousin asked him to remotely tutor her daughter in math. Khan was happy to help and more than qualified, given his three MIT degrees and his Harvard MBA.

As more relatives and friends asked Khan for tutoring assistance, he began videotaping short lessons and posting them to YouTube. His knack for distilling complex concepts into easy-to-follow tutorials helped the videos go viral.

Sal knew he needed to offload the technical and maintenance concerns so he could work on what was important. To grow, it was important for him to be able to stop worrying about things like deployment issues and running his own server.

Ben Kamens, lead developer for Khan Academy

Khan maintained a website for his growing video library for several years, but the platform experienced limitations as traffic increased.

At the same time, media attention was growing, and technology industry leaders, including Microsoft founder Bill Gates and Silicon Valley venture capitalist John Doerr, began lending their support and evangelizing Khan’s mission and work. It was at this point that Khan quit his job as a hedge fund analyst to devote his full-time attention to the site’s growing potential.

“With Google App Engine, we don’t need a system administrator or anyone dedicated to deploying our app, so 99 percent of our time is spent working on our application,” says Ben Kamens, lead developer, Khan Academy.

“Sal knew he needed to offload the technical and maintenance concerns so he could work on what was important, which was to make more videos for the Khan Academy library,” explains Ben Kamens, lead developer for Khan Academy. “To grow, it was important for him to be able to stop worrying about things like deployment issues and running his own server.”

Solution

Khan Academy chose App Engine as its hosting and application development platform because App Engine could easily house its growing collection of 2,000-plus videos, resolving the organization’s overall server and maintenance issues with a single solution.

Using App Engine freed the team to focus on the user experience and the array of content that makes the academy such a powerhouse.

“A lot of what the Khan Academy is about is collecting data on student behavior so we can teach them better,” Kamens says. “Did they use a hint? Have they watched the video before? That data is being stored on Google App Engine so we can figure out what the most effective videos are, or where students struggle the most.”

Khan Academy provides individual profiles to students so they can analyze their learning progress, which means the organization needs systems running in the background to collect and track of all this data. Because App Engine takes care of server support, Khan Academy’s five developers can spend almost all of their time improving site functionality.

“If we didn’t have Google App Engine, we’d be spending a lot more time figuring out server setup and working on routers,” Kamens says. “Our ability to focus on the actual product is the benefit of Google App Engine.”

With the App Engine dashboard, which provides information like response time, uptime, and error rates, the development team also has an easy way to watch over site performance. “It lets us see what the average response time is, so we can keep it really low,” Kamens explains.

Results

During the US school year, Khan Academy receives more than 3.8 million unique visits a month — all served through App Engine. To support this much traffic, a typical company would need an internal system administration staff.

“With Google App Engine, we don’t need a system administrator or anyone dedicated to deploying our app, so 99 percent of our time is spent working on our application,” explains Kamens. “Our application lives on Google App Engine, and we bank all of our scalability and traffic concerns on Google App Engine. Even huge traffic spikes from Sal’s media appearances don’t worry us since we know that Google App Engine will handle the spike.”

Kamens also likes the fact that he and his team can turn to the Google App Engine support team for assistance when they need it. He also finds helpful tips on the Google App Engine blog.

The Khan Academy staff has come to completely rely on App Engine. “We are constantly using more of Google App Engine’s capabilities and have been happy with the performance,” Kamens says. “It’s nice to have an easy deployment process since we deploy the site on average at least once a day, [and] sometimes up to nine or 10 times [a day].”

AgroStar has launched a cloud-based mobile app that is helping to boost crop yields and encourage best practices for small farmers in India. Launched as an on-premises ecommerce platform selling farm tools in 2008, the firm turned to Google Cloud Platform (GCP) to expand its offering. It now uses cloud-based analytics and is deploying ML models to provide timely advice in five languages on everything from seed optimization, crop rotation, and soil nutrition to pest control.

A 2018 survey underscored the demand for agricultural planning for Indian farmers. While farming remains a dominant sector in India, employing half of its labor force, 70 percent of small farmers – those cultivating fewer than three acres – said their crops are damaged by unforeseen weather and pests. An even higher number – 74 percent – say they lack access to farming-related information.

Widening that gap is the relative lack of access to new, higher yield seeds and improved soil analyses for small farmers, who must otherwise rely on traditional methods. “It could take a few years for innovative information to trickle down from universities to small, grassroots farmers,” says Pritesh Gudge, AgroStar Software Engineer. “Today, just by clicking through our Android application, farmers learn about new, effective farming practices and receive advice customized to their crop and soil.”

Connecting a million farmers in the cloud

Operating in the Indian states of Gujarat, Maharashtra, Rajasthan, Orissa, Bihar, and Karnataka, AgroStar is closing the knowledge gap with a full-service, cloud-based SaaS solution – the only one of its kind in India. It combines agronomy, data science, and analytics to help farmers by providing a variety of resources.

AgroStar has reached over a million farmers through its Android app, the AgroStar Agri-Doctor. The mobile client is available as a web-based or full-featured native app. Both provide access to the firm’s knowledge base hosted on GCP, a Q&A forum that connects farmers to each other to help understand and better solve problems and to learn about innovative practices and products. Farmers can also click through to follow local and national market trends that help forecast crop prices.

In addition to the self-service knowledge base, AgroStar provides access to agronomy experts who use cloud-based analytics tools and historical data to provide season-and locale-specific advice to each farmer. “We are now tracking thousands of calls in 5 languages each day,” says Pritesh.

The AgroStar app also provides links to purchase and then track the delivery of farm tools and supplies such as cultivators and fertilizers. An in-house platform manages fulfillment centers and a doorstep delivery network simplifies the supply chain while giving farmers what they need, when they need it. By procuring directly from the manufacturers and primary distributors of farm supplies, Agrostar is achieving cost savings, which it passes on to farmers.

Build fast, pivot faster

From the start, the human and environmental variables of farming in India, not to mention the volume of AgroStar’s few hundred thousand monthly active users, made a highly scalable cloud-based solution inevitable. Farmers rely on the firm’s Agri-Doctor app to provide advice in multiple languages on topics that range widely throughout three growing seasons, each with distinct crop nutrition and rotation cycles and farm implementation requirements.

“For farmers, the focus keeps changing every month, and every season,” says Pritesh. “To serve our growing community, we needed a platform that could process images at high volume, fulfill tools and seed orders across thousands of miles, and respond to multilingual queries. We quickly moved away from spreadsheets and server-based solutions – we needed to build fast and pivot faster.”

Ending late-night deployments

The firm’s first cloud experience was with an AWS solution. At the time, AWS was the only cloud provider in India, but AgroStar wanted to find a solution that was easier to use and offered better integration with Android devices. “Deployment and processing costs were very high, and the developer tools and documentation were not as intuitive as we needed,” says Pritesh.

When GCP service arrived in India in October 2017, AgroStar embarked on a platform re-implementation that made possible dramatic changes in the way it developed and deployed its solution. Using Google Kubernetes Engine (GKE) for crop advice management and Compute Engine for its production application services, the firm built the backend for the Agri-Doctor discussion forum in only three weeks. The platform’s microservice architecture is implemented in Python and Golang and deployed on GCP.

AgroStar began to realize significant efficiencies in its build, deploy, and test cycles. “We previously needed to work overnight to deploy to production,” says Pritesh. “Now using Google for Kubernetes containers and a rolling update strategy, we can deploy during the day without any problems or interruptions to service.”

The move to GCP streamlined AgroStar’s stack. “We were running 12 independent instances on AWS,” says Pritesh. “With Google Kubernetes Engine, we are deployed on a single cluster at a cost savings of $1,300 per month and growing.”

Improving customer response times by 85 percent

With a managed deployment capability, AgroStar can devote more time and resources to executing on its platform and Agri-Doctor app development plan. A strategic goal was managing customer response times as the firm grew its base. GCP has helped the firm meet that goal, achieving an 85 percent improvement in customer response times even as traffic grew significantly.

“With our on-premises solution, we could handle around 100 customers daily, which took 30 to 50 minutes for each customer,” says Pritesh. “We now handle thousands of customers daily, taking only 4 to 5 minutes for each one.”

AgroStar used Firebase to implement its Agri-Doctor app. A real-time cloud database, Firebase provides an API that enables the Agri-Doctor advice forum to be synchronized across all its far-flung mobile clients, effectively sharing knowledge base updates with one million users in near real time.

Using cloud tools to manage and monitor

Cloud Pub/Sub, Kafka, and Cloud Dataflow manage data ingestion and queueing of event and transaction data to the analytics layer. BigQuery fetches and persists data to Cloud Storage. Cloud SQL and dashboards powered by Tableau deliver farmer crop and soil profiles within minutes.

Cloud IAM helps AgroStar control access to all its cloud resources. And Stackdriver, the integrated logging aggregation capability for GCP, helps monitor and speed debugging on every tier of the AgroStar solution.

Machine learning to enhance yields

AgroStar is developing a variety of ML components to improve responsiveness and extend its platform offerings.

To speed up the diagnosis of and treatment for crop blight, AgroStar is building a deep learning pipeline using TensorFlow. The pipeline relies on GoogLeNet models that use multi-layered convolutional visual pattern recognition. It will assess uploaded images to support a disease-detection capability on the mobile app. Based on the commercially successful AI algorithms that automated postal code processing, GoogLeNet offers improved performance and computational efficiencies by using a creative layering technique that distinguishes them from older, sequential recognition engines.

To improve its customer search experience, AgroStar is developing an ML pipeline that shrinks fetch times by suggesting tags mapped to stored data. Processed using TPUs, Cloud Natural Language and Video AI, the tags provide a metadata layer that supports queries in any of the ten natural languages that AgroStar farmers can use.

The AgroStar search pipeline consists of Long Short-Term Memory (LSTM) models of Recurrent Neural Networks. Recurrent networks exhibit “memory” through iterative processing and are distinguished from feedforward networks by a feedback loop connected to their past decisions, ingesting their own outputs moment after moment as input.

Implementing a recommendation engine

The firm is also adapting the Random Forests TensorFlow AI model to develop a crop and product recommendation engine. The model is trained by consuming numerical (rainfall, humidity, water availability per acre) and categorical (soil type, water sources) parameters to suggest appropriate products by season, region, and locale.

To simplify the product suggestion experience, AgroStar developers are testing Cloud Dialogflow, the Google Cloud conversational interface, to build a chatbot capability into its mobile app. The bot will track a farmer’s crop schedules and answer simple questions by linking to the recommendation engine.

AgroStar is also extending its analytics platform with AI-powered sales planning and forecasting. Using linear regression models implemented in TensorFlow and powered by Cloud ML Engine, the capability will enhance supply chain logistics as the company scales its operations across India.

To provide a credit on-demand offering for a range of seed-to-harvest cycle products, AgroStar is attempting to use Vision API to create an AI model that will convert uploaded photos of customer application records into standard data formats. The firm’s credit policy features a grace period in which farmers begin paying back loans after harvested crops go to market.

A versatile and friendly development ecosystem

AgroStar credits the convivial tools and documentation that GCP offers and its incremental, pay-as-you-go pricing model for both the firm’s success and its ability to manage growth.

“What Google Cloud offers is extremely good documentation and extremely simple-to-use tools and interfaces across all services,” says Pritesh. “It helped us initially deploy our platform and at every scale that we have required since then, and its cost effectiveness enabled us to staff up to meet new feature milestones.”