The digital era turned the traditional formula for media and entertainment success on its head, ushering in new technologies that have changed how content is produced, distributed, experienced, and monetized. Audiences have more choice, flexibility, and power over what they consume, and today’s media companies have to embrace ongoing transformation or risk falling behind – or becoming irrelevant. 

A new wave of transformation is arriving with generative AI, a type of artificial intelligence that can interact with users in natural language and create novel data, ranging from story outlines, reports, and other text outputs to multimodal content like images, videos, and audio. Media and entertainment are inherently about content creation and creativity—so what does this new technology mean for the industry? 

At Google Cloud, we see tremendous opportunity for creative industries, from more efficient creation methods to improved user experiences. Let’s explore.

AI for media with Google Cloud

Google Cloud has a long history with large language models (LLMs) and other generative AI technologies—from their influence over the years on products like Document AI, to recent announcements like Generative AI support in Vertex AI, which lets businesses access and tune generative AI foundation models, and Generative AI App Builder, which lets developers build chatbots and other generative apps in minutes.  

We’ve helped our global media and entertainment customers with AI for personalization, search and recommendations, predictive analytics, and much more — and with generative AI now on the rise, we have some ideas to help media leaders, technologists, and creators think about and prepare to utilize powerful AI in their work. 

Three lenses on innovation in media

The media and entertainment industry is increasingly diverse and complex, with companies spanning over-the-top (OTT) subscription streaming services, 24-hour linear channels, live broadcasts of sporting events, digital journalism, traditional publishing, short-form user-generated social video, and more. More and more, the boundaries between these segments of the media industry are blurring — but common to them all is the focus on providing compelling content in an engaging audience experience that can be directly or indirectly monetized.

With this in mind, we suggest media and entertainment companies look at the application of innovative technologies like generative AI through the following three lenses:

1. Improving content creation, production, and management
2. Enhancing and personalizing audience experiences
3. Improving monetization

Improving content creation, production, and management

Generative AI democratizes many aspects of content creation, opening new ways to create written material, illustrations, sound effects, special effects, and more. Its recent maturation has been so rapid, some in the media industry have expressed concern that generative AI implies the end of creative professions. We think the opposite is more likely: just as photography, audio recordings, and computer generated images have enabled new modes of creativity, rather than making old ones obsolete, generative AI has the potential to both enable new forms of expression and enhance familiar ones. 

For example, journalists could use generative AI to speed up research by helping them synthesize and analyze large volumes of information, or to help them create initial drafts or summaries of editorial content. Film and television producers could leverage the technology to accelerate the post-production editing process, with new AI-enabled interfaces for rapidly adjusting or enhancing scene details such as lighting and color. Broadcasters could use generative AI to make vast libraries of video footage searchable and accessible for use in telling more compelling stories. The potential use cases go on and on.

Far from undermining incredible creative professions, generative AI is poised to free writers, artists, editors, and many others from the tedious and mundane aspects of their work, empowering them to focus more of their time on creativity.

Enhancing and personalizing audience experiences

Every media organization in the world today faces the reality that for most consumers, switching costs are extremely low. This puts incredible pressure on these companies to invest in delivering low-friction and compelling audience experiences that help mitigate subscribers from churning and viewers from abandoning content experiences for competitive platforms. 

Generative AI can help media companies engage and retain viewers, such as by enabling more powerful search and recommendations on their digital content platforms. With its increasingly multimodal capabilities extending from natural language to both audio and video content, generative AI is well-positioned to power more personalized audience experiences. 

Consumers often complain about “the paradox of choice” or their inability to find something interesting to watch on streaming platforms that have incredibly vast libraries of content available on demand. Imagine a not-too-distant future wherein a consumer can simply ask the content platform they’re using to help them find a specific show to watch based on mood, specific types of scenes, combinations of actors, award nominations, or practically anything they can think to ask. And that’s just the tip of the iceberg — imagine generative AI’s potential to curate, assemble, and even create personalized content for a viewer to consume!

Improving monetization

As consumers’ content consumption further expands from traditional theatrical and linear television programming to include digital offerings across an array of platforms, devices, and content types, media companies face the challenge of maintaining and improving monetization. The conventional economics and approaches to advertising and subscription models are proving, in many cases, not to deliver sufficient ROI. 

Generative AI has the potential to help media companies improve their monetization of audience experiences. As mentioned previously, enhanced personalization can play a role in mitigating churn, which in turn can help sustain and grow subscription and advertising revenues. Going beyond this, generative AI can be leveraged to drive even greater advertising revenues via more targeted, contextual, and personalized advertisements. Imagine both display and video advertisements that are generated on the fly to personalize product specifics, messaging, style, colors, and innumerable other characteristics to drive greater engagement and higher click-through rates (CTR), and thus higher advertising CPMs (cost per thousand impressions).

Coming up next

Generative AI presents a significant opportunity for media companies to fundamentally transform content creation, engagement, and monetization. Compelling services are already on the market — but there is far more to come. 

Google Cloud continues to build on its deep experience and expertise with AI, and we are committed to working with the industry to develop compelling, accessible, trusted, and responsible AI solutions that will drive meaningful business outcomes. We are excited to create the future together with our global media customers and partners across the ecosystem. To learn more about this disruptive topic, read “Debunking five generative AI misconceptions” from Google Cloud vice president of AI & Business Solutions Phil Moyer, or explore our Trusted Tester Program for generative AI.

On Dec 9th and 10th, Wayfair and Google Cloud came together for the inaugural Wayfair-Google Cloud Machine Learning Hackathon. Wayfair firmly believes that hackathons are a great way to fuel a culture of collaboration and experimentation. To fuel Wayfair’s incredible pace of innovation at scale, it’s team of more than 3,000 technologists is constantly experimenting and taking smart risks. It’s test-and-learn culture empowers everyone to think critically and creatively, and take big swings to raise the bar on its world-class experience. 

The Wayfair-Google Cloud Machine Learning Hackathon was all about getting Wayfairians excited about and enabled on new technology. More specifically, this event was a contained test environment for Wayfair innovators to validate AI and ML tools in order to understand how to get better insight from their data. The projects worked on during this hackathon will help Wayfair build new use cases that could impact the business and the end customer in new and productive ways. Google Cloud’s focus for this Hackathon was to enable Wayfairains to harness the power of Machine Learning and AI to enable their own goals around continual improvement and relentless customer focus. 

Prior to the event, Wayfair Data Scientists and Machine Learning Engineers who signed up and submitted ideas for the Hackathon were invited to optional Google Cloud enablement and training sessions. Google Cloud set up classrooms in Qwiklabs on topics including, but not limited to, BigQuery, Vertex AI, and Natural Language AI, so that all Hackathon participants could try out new technologies and tools in a learning environment. Google Cloud subject matter experts were available to field any questions Wayfairians had about the use cases they were hacking on. 

The Hackathon was a hybrid virtual & physical event that hosted 67+ innovators, 49 of which registered for Google Cloud supported ideas. 15 judges from both Google Cloud and Wayfair oversaw the event. The esteemed list of judges included Steven Conine, Wayfair’s Co-Founder and Co-Chairman who has helped pave the way for the development of practical applications of next-generation technologies like augmented reality. The judges measured and evaluated the success of a project based on the following criteria:

• “Wow Factor”: How innovative is the project?
• Impact: How impactful is the project to Wayfair?
• Polish: How complete is the project?
• Presentation Quality: How clear and consistent is the demo? 

The theme of the Hackathon was Machine Learning and AI. Teams were able to collaborate with participants globally, either in-person or virtually, and work together on projects in five categories:  Relentless Customer Focus, Always Improving, Google’s Choice, People’s Choice, and Hackers’ choice. 

At the end of the two days there were winners in all 5 categories. The Google’s Choice award went to “Entity Extraction for Order Matching”. Team “Project Clippy” was named both Hackers’ Choice and the winners of the Relentless Customer Focus category. See below for the results of the 5 categories: 

• Relentless Customer Focus and Hackers’ Choice
  • Winning Team: Project Clippy
  • Hackers: Misha Balyasin, Alex Saad, Leo Smerling, and Gabriele Lanaro
  • This project provided a gamified experience to make the process of leaving a product review even more seamless. 
• Always Improving
  • Winning Team: Customer Causal MetaLeaners
  • Hackers: Colin Gray, Irene Wang, Huy Vo Tran, Wenhao Xu, and Santiago Velez Ferro
  • Team Customer Causal MetaLeaners worked to build lightweight procedure(s) for computationally distributed, multi-target, customized loss functions to make causal meta-learners more applicable to real-world Wayfair problems.
• Google’s Choice Award:
  • Winning Team: Entity Extraction for Order Matching
  • Hackers: Roger Bock, Bradley West, Sina Moeini, and Jonathan de Melker Worms
  • This team built out a solution to use text models to extract and identify the products that customers purchased from user reviews.
• People’s Choice:
  • Winning Team: KNN and ANN on Vertex AI
  • Hackers: Santosh Jhingade, Ashrith Marpaka, Nikhil Bhaip, Adam Schulze, and Brandon Sanders
  • This team used Vertex AI to expand the impact they can have on suppliers and customers by providing accurate and real-time information of products that match either description or image.

Wayfair leaders reflected on the two days and shared their input. Matt Ferrari, Head of Ad Tech, Customer Intelligence, and Machine Learning; Engineering and Product at Wayfair said, “Wayfair has a lot of vendors, but very few strategic partners, and Google is that. Our Partner.” “Thank you all for the participation! I’m grateful to Google, and the many others for helping lead a successful event.” 

Wayfair partners with Google Cloud to optimize performance and resiliency, support scaling data-driven decisions, and Increase employee productivity. “Our category is ripe for innovation, and our partnership with Google Cloud helps us ensure that great ideas can come from anywhere by empowering our technologists with cutting-edge products and solutions,” said Ferrari. “We’re proud to partner on efforts like hackathons that align with our team’s eagerness to work on complex, rewarding problems that push the envelope and challenge us to always think big.”

At first Google Cloud won Wayfair over with the speed, reliability and performance of their technology. Hackathons like this one exemplify that what is equally as important is Google Cloud’s willingness to work side-by-side with Wayfair at every level to enable a culture of innovation. Learn more about the Wayfair-Google Cloud partnership here.

Digital technology offers increasing flexibility and choice to consumers. As a result, the retail industry is dramatically shifting toward more tailored and personalized experiences for shoppers, and businesses are rethinking how they deliver value to customers.

This couldn’t be more true for the beauty retailing industry where leading companies are turning to digital technology to create customized shopping experiences.

At Google Cloud, we’re particularly excited about our work with Ulta Beauty, the largest beauty retailer in the United States with more than 1196 stores in all 50 states, and how the company is using Google Cloud technology solutions to power personalization and redefine beauty retailing.

Established in 1990, Ulta Beauty has had incredible success as a company, and as customers become more discerning and curious about their purchases, the company is finding new ways to meet their changing needs.

Recently, leaders at Ulta Beauty recognized a huge opportunity to complement and enhance the shopping experience by helping beauty enthusiasts navigate through more than 500 brands and 25,000 products carried in their stores and online channel.

They decided to leverage the data from Ulta Beauty’s successful Ultamate Rewards loyalty program to create and offer more unique and personalized user experiences.

With more than 30 million members generating data through sales, transactions, product reviews, and social media engagement, Ulta Beauty’s Loyalty Program creates a comprehensive data set, and the company sought the right technology partner to help organize, analyze and transform that data into valuable insights for its customers.

Ulta Beauty’s leaders knew they had an opportunity to leverage data analytics and machine learning to reach customers in new ways, enhance the guest experience, and continue to grow their active loyalty member base. After considering a number of cloud providers, they chose to expand their existing partnership with Google Cloud.

“Google Cloud listened to our needs and worked in tandem with our engineering team to address our challenges,” said Michelle Pacynski, vice president of digital innovation at Ulta Beauty. “The ease of working with the Google Cloud team and their breadth of experience made the decision a no-brainer, laying the foundation for a great partnership.”  

In 2019, Ulta Beauty announced it was working with Google Cloud Platform to unify and organize its data, using:

• BigQuery to perform data analysis and generate dynamic content, personalized product recommendations, and event-based messages for customers.
• Cloud Storage to provide highly available, secure, resilient and cost-effective access to data across the entire enterprise.
• Compute Engine for the high-performance scalability needed to grow with customer demand while painlessly migrating existing applications to the cloud.
• Anthos to build a hybrid cloud foundation that allows their applications to take advantage of all this data, combining the power and flexibility of GKE with the ability to leverage their existing investment in secure infrastructure on-premises.

Our partnership with Ulta Beauty has enabled increased engagement with customers in store and online, and the creation of new tools and capabilities, including a new Virtual Beauty Advisor tool to deliver tailored recommendations and help shoppers choose the right products, and a Customer Conversation Platform that’s enabling deeper connections with guests, ultimately driving customer loyalty.

“It’s been a really efficient process so far due in part to the ease of working with the Google team,” said Michelle Pacynski, vice president of digital innovation at Ulta Beauty. “They’re experienced, approachable, and their can-do style makes for a great partnership. They listened to our needs and worked in tandem with our engineering team, figuring things out, and getting it done.”

Today, manufacturers are advancing on their factory digitalization journey, betting on innovative technologies to strengthen competitiveness, deliver sustainable growth, and offer new services. Macroeconomic factors – such as high energy costs, increasing labor, and raw material shortages – drive the need for urgent operational optimizations and automation.

Cloud capabilities have matured at an accelerated pace, giving manufacturers practical avenues to achieve these goals. Manufacturers are finding new ways to bring AI and machine learning (ML) to practical use cases, like predictive maintenance, anomaly detection, and asset utilization management. However, manufacturers struggle to adopt AI at scale due to challenges around data accessibility, infrastructure, and technology.

Google Cloud created purpose-built tools and solutions to organize manufacturing data, make it accessible and useful, and help manufacturers quickly take significant steps on this journey by reducing the time to value. In this post, we will explore a practical example of how manufacturers can use Google Cloud manufacturing solutions to train, deploy and extract value from ML-enabled capabilities to predict asset utilization and maintenance needs.

The journey to machine learning insights starts with accessible data

The first step to a successful machine learning project is to unify necessary data in a common repository. For this, we will use Manufacturing Connect, the factory edge platform co-developed with Litmus Automation, to connect to manufacturing assets and stream the asset telemetries to Pub/Sub.

After the telemetry messages are published to Pub/Sub, Dataflow will identify each message based on its structure and apply corresponding normalizations and transformations, which are preconfigured in Manufacturing Data Engine. Once the messages are processed, the messages will be routed to Cloud Storage, BigQuery, and/or Cloud BigTable based on user configuration.

To train a machine learning model, manufacturers can use Vertex AI AutoML to build a no-code model based on the training data stored in the Manufacturing Data Engine.

Then, users can trigger a batch prediction job in Vertex AI or export the AutoML model to run on the edge component of Manufacturing Connect for real-time prediction. Regardless of the model deployment methods, the prediction and explanation results will be ingested into Manufacturing Data Engine, which can be analyzed and visualized in Looker.

A blueprint for classifying asset condition

The following scenario is based on a hypothetical company, Cymbal Materials. This company is a factitious discrete manufacturing company that runs 50+ factories in 10+ countries. 90% of Cymbal Materials manufacturing processes involve milling, which are accomplished using industrial computer numerical control (CNC) milling machines. Although their factories implement routine maintenance checklists, there are unplanned and unknown failures that happen occasionally. However, many of the Cymbal Materials factory workers lack the experience to identify and troubleshoot failures due to labor shortage and high turnover rate in their factories. Hence, Cymbal Materials is working with Google Cloud to build a machine learning model that can identify and analyze failures on top of Manufacturing Connect, Manufacturing Data Engine, and Vertex AI.

For the pilot, Cymbal Materials forms a team of manufacturing engineers and data scientists to evaluate the feasibility of solving the tool wear detection problem. To avoid compliance concern, the Cymbal Materials team chooses to start with a public tool wear detection dataset hosted on Kaggle. This dataset is collected from running machining experiments on 2″ x 2″ x 1.5″ wax blocks in a CNC milling machine. The dataset contains measurements from the 4 motors (X,Y, Z axes and spindle) and program values in the CNC machine, which maps well to the data that Cymbal Materials collects for their CNC milling machines.

To start, the Cymbal Materials data scientists download the tool wear detection dataset from Kaggle and upload the dataset to Cloud Storage. Then, the data scientists use Vertex AI to:

• Perform exploratory data analysis using Vertex AI Workbench
• Train machine learning models using Vertex AI AutoML
• Deploy machine learning models to perform batch prediction and export AutoML models for edge deployment
• Interpret predictions using Vertex Explainable AI

After seeing great performance of the AutoML tabular model, the Cymbal Materials data scientists decide to use the AutoML model to predict on the actual CNC milling machine telemetries from their factories and validate the generalizability of the AutoML model. They ask the manufacturing engineers to deploy Manufacturing Connect in a Cymbal Materials factory and stream telemetries for one CNC milling machine to the Manufacturing Data Engine.

Manufacturing Connect includes an edge component that can gather data from manufacturing assets via an extensive library of 250+ communication protocols. The edge component of Manufacturing Connect comes with built-in Node-RED and Docker runtime, which support running custom workflows and machine learning models at the edge.

Using pre-defined hierarchies, Manufacturing Connect pushes asset telemetries and states to Pub/Sub.

After the factory operational data are ingested into Pub/Sub, Cymbal Materials uses Manufacturing Data Engine to:

• Normalize, transform, and contextualize real-time operational data with slowly changing metadata
• Batch ingest historical operational data and prediction results
• Route data dynamically to Cloud Storage, BigQuery, and/or Cloud BigTable

Using the trained AutoML tabular model and real-time telemetries from CNC milling machines, the data scientists trigger a batch prediction job on the CNC milling machine telemetries in BigQuery. The data scientists configure the batch prediction to output prediction results in Cloud Storage such that Manufacturing Data Engine can batch ingest the prediction results after the batch prediction job completes.

To consume the prediction results, the Cymbal Materials manufacturing engineers use Looker to create visualizations. The dashboard allows the manufacturing engineers to:

• Visualize the CNC milling machine actual and predicted tool conditions over time
• Explain the prediction results by summarizing the top attributing features
• Create alerts based on the predicted tool condition for their assets
• Take actions by contacting the supplier and/or scheduling maintenance for their assets

From edge to cloud, improving production efficiency for manufacturers

To support the entire factory digitalization value journey, manufacturers are looking for capabilities from simple visualizations to predictive ML models. Robust solutions, such as the one covered here, provide rapid paths for engineers to extract insight from their factory data.

Having a common data repository for manufacturing data, industry-leading machine learning platform, and versatile dashboard components accelerate manufacturer’s digital transformation.

This solution brings the best of Google Cloud’s data analytics and artificial intelligence capabilities in an industrial environment. Manufacturing Connect creates the link between industrial machinery and Manufacturing Data Engine, the cloud platform where the manufacturing data are processed, normalized, contextualized, and stored in a ready-to-consume format. Vertex AI can build, deploy, and scale machine learning models using data stored in the Manufacturing Data Engine. Vertex AI includes AutoML and Workbench for training models without code and training custom models with code-first experience respectively.

Learn more about how Google Cloud is transforming manufacturing to meet changing customer expectations at our Google Cloud Next Manufacturing playlist.

What’s next

1. Introducing new Google Cloud manufacturing solutions: smart factories, smarter workers
2. Manufacturing Data Engine | Solutions | Google Cloud
3. GitHub – GoogleCloudPlatform/mfg-ml-examples