4636
Of your peers have already watched this video.
46:44 Minutes
The most insightful time you'll spend today!
Breaking Down AI for IT Leaders: The No-Nonsense Guide
What is machine learning and, critically, what kinds of problems can it solve? It’s an important question, one that forms the fundamental basis of any AI initiative.
Here’s how the Google Cloud thinks about machine learning: It’s about logic, rather than just data.
Valliappa Lakshmanan, Big Data and Machine Learning, Google Cloud Platform, describes why such a framing is useful when it comes to devising new applications for ML and how to utilize it to expand the capabilities of your business.
From Lakshmanan perspective watching many companies in many industries leverage machine learning, he observes how abstraction levels of machine learning are increasing, how data comprehensiveness is becoming more important than data size and why systems can often build on top of preexisting models.
Lakshmanan also speaks about how your IT infrastructure has to change to enable you to take full advantage of machine learning and achieve tremendous business impact and personalization.
5365
Of your peers have already watched this video.
44:30 Minutes
The most insightful time you'll spend today!
How Go-Jek, Indonesia’s First Billion-dollar Startup, Improved the Productivity of Data Scientists
Go-Jek, Indonesia’s first billion-dollar startup, has seen an incredible amount of growth in both users and data over the past two years. Many of the ride-hailing company’s services are backed by machine learning models hosted on Google Cloud Platform.
Models range from driver
But senior executives at Go-Jek realized something: One of their most important and expensive resources, data scientists, were spending far too much time cleaning data. That wasn’t part of their remit and resulted in a waste of time and money.
As a COO, this a major concern for any company undertaking a machine learning initiative. Data scientists are hard to come by and their salaries have been on the rise for the last few years. Yet according to some reports data scientists spend upto 80% of their time just preparing data—not creating models.
Watch how operational teams at Go-Jek combined the right Google tools and processes to improve the productivity of their data scientists.
Google Cloud named a leader in the Forrester Wave: Streaming Analytics
DOWNLOAD WHITEPAPER4185
Of your peers have already downloaded this article
15:30 Minutes
The most insightful time you'll spend today!
There’s a lot of pressure on companies today to become data-driven, but they need high-performing technology to make that a reality.
As part of Google’s larger cloud data analytics platform, Cloud Pub/Sub and Cloud Dataflow are designed for ease of use, scalability, and performance.
Google Cloud unifies streaming analytics and batch processing the way it should be. No compromises.
Forrester
Forrester has named Google Cloud as a leader in The Forrester Wave™: Streaming Analytics, Q3 2019. This findings reflect Google Cloud’s market momentum, and what Google hears from enterprise customers who are using Cloud Pub/Sub and Cloud Dataflow in production for streaming analytics in conjunction with our broader platform.
According to Forrester, many leading enterprises realize that real-time analytics—the analytics of what’s happening with data in the present—is an incredible competitive advantage. With it, they can act right away to serve customers, fix operational problems, power internet of things (IoT) apps, and respond decisively to competitors.
The report evaluates the top 11 vendors against 26 rigorous criteria for streaming analytics to help enterprise IT teams understand their options and make informed choices for their organizations. Google scored 5 out of 5 in Forrester’s report evaluation criteria of scalability, availability, aggregates, management, security, extensibility, ability to execute, solution roadmap, partners, community, and customer adoption.
We are very excited about the productivity benefits offered by Cloud Dataflow and Cloud Pub/Sub. It took half a day to rewrite something that had previously taken over six months to build using Apache Spark.
Paul Clarke, Director of Technology, Ocado
While stream analytics is a leading business priority, real-life use cases frequently require batch data as an input as well. Google Cloud Platform (GCP) customers can simplify their pipeline development by reusing code across both batch and stream processing using Apache Beam, which also provides pipeline portability to OSS projects. Further, Google Cloud’s data analytics portfolio autoscales across ingestion, processing, and analysis, which eliminates the need for provisioning and makes handling varying volumes of streaming data automatic. We hear from users that they’re able to ingest and process data much more quickly than in the past, helping to get new business insights faster and allowing more users to do self-serve analytics.
The Query Execution Graph: Your Key to Better BigQuery Analytics
3086
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
BigQuery offers strong query performance, but it is also a complex distributed system with many internal and external factors that can affect query speed. When your queries are running slower than expected or are slower than prior runs, understanding what happened can be a challenge.
The query execution graph provides an intuitive interface for inspecting query execution details. By using it, you can review the query plan information in graphical format for any query, whether running or completed.
You can also use the query execution graph to get performance insights for queries. Performance insights provide best-effort suggestions to help you improve query performance. Since query performance is multi-faceted, performance insights might only provide a partial picture of the overall query performance.
Execution graph
When BigQuery executes a query job, it converts the declarative SQL statement into a graph of execution, broken up into a series of query stages, which themselves are composed of more granular sets of execution steps. The query execution graph provides a visual representation of the execution stages and shows the corresponding metrics. Not all stages are made equal. Some are more expensive and time consuming than others. The execution graph provides toggles for highlighting critical stages, which makes it easier to spot the potential performance bottlenecks in the query.
Query performance insights
In addition to the detailed execution graph BigQuery also provides specific insights on possible factors that might be slowing query performance.
Slot contention
When you run a query, BigQuery attempts to break up the work needed by your query into tasks. A task is a single slice of data that is input into and output from a stage. A single slot picks up a task and executes that slice of data for the stage. Ideally, BigQuery slots execute tasks in parallel to achieve high performance. Slot contention occurs when your query has many tasks ready for slots to start executing, but BigQuery can’t get enough available slots to execute them.
Insufficient shuffle quota
Before running your query, BigQuery breaks up your query’s logic into stages. BigQuery slots execute the tasks for each stage. When a slot completes the execution of a stage’s tasks, it stores the intermediate results in shuffle. Subsequent stages in your query read data from shuffle to continue your query’s execution. Insufficient shuffle quota occurs when you have more data that needs to get written to shuffle than you have shuffle capacity.
Data input scale change
Getting this performance insight indicates that your query is reading at least 50% more data for a given input table than the last time you ran the query and hence experiencing query slowness. You can use table change history to see if the size of any of the tables used in the query has recently increased.
What’s next?
We continue to work on improving the visualization of the graph. We are working on adding additional metrics to each step and adding more performance insights that will make query diagnosis significantly easier. We are just getting started.
Google’s Intelligent Products Essentials Assist Manufacturers in Product Development Journey
6944
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
Expectations for both consumer and commercial products have changed. Consumers want products that evolve with their needs, adapt to their preferences, and stay up-to-date over time. Manufacturers, in turn, need to create products that provide engaging customer experiences not only to better compete in the marketplace, but also to provide new monetization opportunities.
However, embedding intelligence into new and existing products is challenging. Updating hardware is costly, and existing connected products do not have the capability to add new features. Furthermore, manufacturers do not have sufficient customer insights due to product telemetry and customer data silos, and may lack the AI expertise to quickly develop and deploy these features.
That’s why today we’re launching Intelligent Products Essentials, a solution that allows manufacturers to rapidly deliver products that adapt to their owners, update features over-the-air using AI at the edge, and provide customer insights using analytics in the cloud. The solution is designed to assist manufacturers in their product development journeys—whether developing a new product or enhancing existing ones.
With Intelligent Products Essentials, manufacturers can:
- Personalize customer experiences: Provide a compelling ownership experience that evolves over the lifetime of the product. For example, a chatbot that contextualizes responses based on product status and customer profile.
- Manage and update products over-the-air: Deploy updates to products in the field, gather performance insights and evolve capabilities over time with monetization opportunities.
- Predict parts and service issues: Detect operating thresholds, anomalies and predict failures to proactively recommend service using AI, reducing warranty claims, decreasing parts shortages and increasing customer satisfaction.
In order to help manufacturers quickly deploy these use cases and many more, Intelligent Products Essentials provides the following:
- Edge connections: Connect and ingest raw or time-series product telemetry from various device platforms utilizing IoT Core or Pub/Sub and enable deployment and management of firmware over-the-air and machine learning models with Vertex AI at the edge.
- Ownership App Template: Easily build connected product companion apps that work on smartphones, tablets, and computers. Use a pre-built API and accompanying sample app that can incorporate product or device registration, identity management, and provide application behavior analytics using Firebase.
- Product fleet management: Manage, update and analyze fleets of connected products via APIs, Google Kubernetes Engine, and Looker.
- AI services: Create new features or capabilities for your products using AI and machine learning products such as DialogFlow, Vision AI, AutoML, all from Vertex AI.
Enterprise data integration: Integrate data sources such as Enterprise Asset Management (EAM), Enterprise Resource Planning (ERP), Customer Relationship Management (CRM) systems and others using Dataflow and BigQuery.
Intelligent Products Essentials helps manufacturers build new features across consumer, industrial, enterprise, and transportation products. Manufacturers can implement the solution in-house, or work with one of our certified solution integration partners like Quantifi and Softserve.
“The focus on intelligent products that Google Cloud is deploying provides a digital option for manufacturers and users. At its heart, systems like Intelligent Product Essentials are all about decision making. IDC sees faster and more effective decision-making as the fundamental reason for the drive to digitize products and processes. It’s how you can make faster and more effective decisions to meet heightened customer expectations, generate faster cash flow, and better revenue realization,” said Kevin Prouty, Group Vice President at IDC. “Digital offerings like Google’s Intelligent Product Essentials potentially go the last mile with the ability to connect the digital thread all the way through to the final user.”
Customers adopting Intelligent Products Essentials
GE Appliances, a Haier company, are enhancing their appliances using new AI-powered intelligent features to enable:
- Intelligent cooking: Help cook the perfect meal to personal preferences, regardless of your expertise and abilities in the kitchen.
- Frictionless service: Build smart appliances that know when they need maintenance and make it simple to take action or schedule services.
- Integrated digital lifestyle: Make appliances useful at every step of the way by integrating them with digital lifestyle services – for example, automating appliance behaviors according to customer calendars, such as oven preheating or scheduling the dishwasher to run in the late evening.
“Intelligent Products Essentials enhances our smart appliances ecosystem, offering richer consumer habit insights. This enables us to develop and offer new features and experiences to integrate with their digital lifestyle.“ —Shawn Stover, Vice-president Smart Home Solutions at GE Appliances.https://www.youtube.com/embed/zaWAJN8aKOw?enablejsapi=1&
Serial 1, Powered by Harley-Davidson, is using Intelligent Product Essentials to manage and update its next generation eBicycles, and personalize its customers’ digital ownership experiences.
“At Serial 1, we are dedicated to creating the easiest and most intuitive way to experience the fun, freedom, and adventure of riding a pedal-assist electric bicycle. Connectivity is a key component of delivering that mission, and working together to integrate Intelligent Product Essentials into our eBicycles will ensure that our customers enjoy the best possible user experience.”— Jason Huntsman, President, Serial 1.
Magic Leap, an augmented reality pioneer with industry-leading hardware and software, is building field service solutions with Intelligent Products Essentials with the goal of connecting manufacturers, dealers, and customers to more proactive and intelligent service.
“We look forward to using Intelligent Products Essentials to enable us to rapidly integrate manufacturers’ product data with dealer service partners into our field service solution. We’re excited to partner with Google Cloud as we continue to push the boundaries of physical interaction with the digital world.” — Walter Delph, Chief Business Officer, Magic Leap
Intelligent Product Essentials is available today. To learn more, visit our website.
5440
Of your peers have already downloaded this article
14:30 Minutes
The most insightful time you'll spend today!
To make great products: do machine learning like the great engineer you are, not like the great machine learning expert you aren’t.
Most of the problems you will face are, in fact, engineering problems. Even with all the resources of a great machine learning expert, most of the gains come from great features, not great machine learning algorithms. So, the basic approach is:
- Make sure your pipeline is solid end to end.
- Start with a reasonable objective.
- Add common-sense features in a simple way.
- Make sure that your pipeline stays solid.
This approach will work well for a long period of time. Diverge from this approach only when there are no more simple tricks to get you any farther. Adding complexity slows future releases.
Once you’ve exhausted the simple tricks, cutting-edge machine learning might indeed be in your future. See the section on Phase III machine learning projects.
This document is arranged as follows:
- The first part should help you understand whether the time is right for building a machine learning system.
- The second part is about deploying your first pipeline.
- The third part is about launching and iterating while adding new features to your pipeline, how to evaluate models and training-serving skew.
- The final part is about what to do when you reach a plateau.
- Afterwards, there is a list of related work and an appendix with some background on the systems commonly used as examples in this document.
More Relevant Stories for Your Company
Tools to Simplify Streaming Analytics
Streaming analytics is transforming how companies interact with their customers and operate with agility. Dataflow supports a number of features to guide you through your journey to adopt streaming, including notebooks for writing your first pipeline, SQL to accelerate your streaming deployments, Flex Templates to scale streaming in your organization,
Vector Search: The Tech Powering Billions of Search Results for Google Users
Recently, Google Cloud partner Groovenauts, Inc. published a live demo of MatchIt Fast. As the demo shows, you can find images and text similar to a selected sample from a collection of millions in a matter of milliseconds: Image similarity search with MatchIt Fast Give it a try — and either select a preset
The Fantastic Story of How BMG Enables a Micropayments Strategy So Music Artists Get Paid
The music industry is rapidly changing. Only 20 years ago, the availability of music and the infrastructure that was required to make an album a sales success were incredibly complex and expensive. With the decline of physical sales and a fundamental shift to digital, music streaming now accounts for more
NCR’s Emerald Leverages Google Cloud to Help Grocers Boost Operational Agility
In recent years, the grocery industry has had to shift to facilitate a wider variety of checkout journeys for customers. This has meant ensuring a richer transaction mix, including mobile shopping, online shopping, in-store checkout, cashierless checkout or any combination thereof like buy online, pickup in store (BOPIS). What’s more,
