ML Workflow Made Simple: How to Automate ML Experiment Tracking with Vertex AI Experiments Autologging
1271
Of your peers have already read this article.
3:30 Minutes
The most insightful time you'll spend today!
Practical machine learning (ML) is a trial and error process. ML practitioners compare different performance metrics by running ML experiments till you find the best model with a given set of parameters. Because of the experimental nature of ML, there are many reasons for tracking ML experiments and making them reproducible including debugging and compliance.
But tracking experiments is challenging: you need to organize experiments so that other team members can quickly understand, reproduce and compare them. That adds overhead that you don’t need.
We are happy to announce Vertex AI Experiments autologging, a solution which provides automated experiment tracking for your models, which streamlines your ML experimentation
With Vertex AI Experiments autologging, you can now log parameters, performance metrics and lineage artifacts by adding one line of code to your training script without needing to explicitly call any other logging methods.
How to use Vertex AI autologging
As a data scientist or ML practitioner, you conduct your experiment in a notebook environment such as Colab or Vertex AI Workbench. To enable Vertex AI Experiments autologging, you call aiplatform.autolog() in your Vertex AI Experiment session. After that call, any parameters, metrics and artifacts associated with model training are automatically logged and then accessible within the Vertex AI Experiment console.
Here’s how to enable autologging in your training session with a Scikit-learn model.
# Enable autologging
aiplatform.autolog()
# Build training pipeline
ml_pipeline = Pipeline(...)
# Train model
ml_pipeline.fit(x_train, y_train)This video shows parameters and training/post-training metrics in the Vertex AI Experiment console.
Vertex AI SDK autologging uses MLFlow’s autologging in its implementation and it supports several frameworks including XGBoost, Keras and Pytorch Lighting. See documentation for all supported frameworks.
Vertex AI Experiments autologging automatically logs model time series metrics when you train models along multiple epochs. That’s because of the integration between Vertex AI Experiments autologging and Vertex AI Tensorboard.
Furthermore, you can adapt Vertex AI Experiments autologging to your needs. For example, let’s say your team has a specific experiment naming convention. By default, Vertex AI Experiments autologging automatically creates Experiment Runs for you without requiring you to call `aiplatform.start_run()` or `aiplatform.end_run()`. If you’d like to specify your own Experiment Run names for autologging, you can manually initialize a specific run within the experiment using aiplatform.start_run() and aiplatform.end_run() after autologging has been enabled.
What’s next
You can access Vertex AI Experiments autologging with the latest version of Vertex AI SDK for Python. To learn more, check out these resources :
- Documentation: Autolog data to an experiment run
- Github: Get started with Vertex AI Experiments autologging
While I’m thinking about the next blog post, let me know if there is Vertex AI content you’d like to see on Linkedin or Twitter.
3002
Of your peers have already watched this video.
26:30 Minutes
The most insightful time you'll spend today!
What’s Next for Personalization on Google Cloud
Customer shopping behavior has changed for good. With fewer in-store shopping visits retailers have had to shore up their digital storefronts and explore new ways to meaningfully engage with their customers.
Delivering a superior customer experience has become even more of a differentiator for the early movers and personalized recommendations have emerged as one of the strongest potential drivers of revenue lift.
But as many retailers have discovered delivering recommendations at scale can actually be quite complex and time consuming.
Learn how to deliver highly-personalized product recommendations with Google Cloud Recommendations AI.
Recommendations AI is now fully open access and self-serve, with more built-in integrations with Google Shopping Merchant Center and Google Analytics, as well as more controls over how you create recommendation pipelines and manage your costs.
You will also hear how Google Cloud partners like Qubit and BigCommerce have successfully deployed Recommendations AI for their customers and made us an integral part of their solution offerings.
Southwire Completes SAP Migration to Google Cloud as a First Step of its Tech Evolution
4951
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
“Talk about tough times, right?”
That’s how Dan Stuart, Senior Vice President of IT Services at Southwire Company, refers to the months following a December 2019 ransomware event, and the COVID crisis that began in spring of 2020. Those events hit just as the company was preparing for an overhaul of their SAP environment. This comprehensive plan included three key elements. First, the company wanted to upgrade their SAP ECC environment to take advantage of the latest functionality available for this critical ERP system. Second, Southwire aimed to deploy SAP Business Warehouse on SAP HANA to accelerate vital reporting for all business users. Third, the company wanted to upgrade to the latest version of SAP Process Orchestration—an essential component that touches key manufacturing interfaces in all Southwire facilities.
Southwire had looked at multiple options for the upgrades, including remaining entirely on-premises, colocation, and full cloud migration. “Going to the cloud seemed a lot more compelling,” says Joe Schleupner, Southwire’s Senior Director of PMO & ITS planning and implementation. “We were going to the cloud eventually, so why take these intermediary steps? Let’s just get it done.”
After looking at several options, Southwire decided to migrate to Google Cloud. “We wanted to be on a platform for SAP that was flexible, scalable, and secure; that we could count on to get up and running quickly,” says Stuart. “We chose Google Cloud not only for those reasons, but also because we recognize that Google has other assets that we may be able to take advantage of down the line, such as technologies like artificial intelligence (AI).”
More stability, less worry
As one of the leading manufacturers of wire and cable used in the transmission and distribution of electricity, Southwire aids the delivery of power to millions of people worldwide. They have more than 30 manufacturing facilities across the United States running 24/7. Any downtime directly affects productivity and revenue. With help from Google Cloud and their implementation partner NIMBL, Southwire completed the SAP migration to Google Cloud over a planned maintenance weekend on July 4th.
The migration itself, while complex, went quickly and smoothly. “Just moving to the cloud was quite a feat because we were dealing with so much data, but in total the SAP system was down for only ~16 hours,” says Schleupner.
“As a project manager, I always felt that Google Cloud had my back” Schleupner says. The Process Orchestration (PO) migration was of particular concern, considering that it controlled all of Southwire’s manufacturing interfaces across the entire company. “Every critical piece of information that goes from SAP down to the manufacturing system goes through that system,” says Schleupner.
Even after migrating, Southwire discovered that making changes to the system was fast, easy, and resulted in no downtime. Normally, certain types of changes would have involved taking down SAP for at least an hour.
The Southwire team also appreciates the fact that the modern cloud architecture means spending less time on routine infrastructure maintenance. “It’s one less thing for me to worry about,” Stuart says, “I can focus on the business side of the house and move the technology and responsibilities to what we do within the Google Cloud Platform.”
What comes next?
While the cloud migration will increase stability, uptime, performance, and security, there is much more to come. Southwire is currently working on a disaster recovery implementation for their SAP environment on Google Cloud. Stuart and Schleupner are excited about where Google Cloud can further take Southwire. They are considering an SAP Hybris e-commerce implementation as well as connected factory and/or factory automation initiatives that can take advantage of artificial intelligence and machine learning.
To Stuart and Schleupner, the migration of Southwire’s SAP environment to Google Cloud, as important as it was, really represents the first step in the company’s tech evolution. Now that much of the heavy lifting is complete, Southwire’s digital transformation can begin in earnest. “There’s no shortage of areas where I think Google Cloud will come into play,” Stuart says, “and we intend to look at these things with an open mind to understand how we can leverage current investments to take our organization where we want to go.”
Learn more about Southwire’s SAP on Google Cloud deployment and how Google Cloud can transform the way you work with your SAP enterprise applications. Visit cloud.google.com/solutions/sap.
5680
Of your peers have already watched this video.
56:20 Minutes
The most insightful time you'll spend today!
How Real Companies Are Innovating with AI Today—and the Benefits They’re Seeing
What do you get when you mix Target, women’s swim wear, and AI? “Joy!” says Mike McNamara, CIO and CDO, Target.
McNamara is just one of the many stories of real businesses conquering old challenges, and new disruptive industry challenges, with artificial intelligence.
McNamara, Nick Rockwell, CTO, The New York Times, Larry Colagiovanni, VP New Product Development, eBay, and Dirk John, CIO, LATAM Airlines, demonstrate how AI solutions allow businesses to find opportunity in chaos, innovation under tough conditions, and revenue in the most unlikely of places.
AI at Target
McNamara shows how AI is being applied to solve some of the most basic, yet critical, challenges that retailers like Target–and any business with large inventories—face. (start at 4:04)
AI at The New York Times
Nick Rockwell, CTO, The New York Times, shares how AI and big data tools are being used to leverage old assets and drive business ideas from them. (start at 12:18)
AI at eBay
Larry Colagiovanni, VP New Product Development, eBay, talks about how AI is helping eBay drive conversational commerce, help buyers sift through 1.1 billion items, and personalize the shopping experience.(start at 28:07)
AI at LATAM Airlines
Dirk John, CIO, LATAM Airlines, discusses how LATAM Airlines adopted advanced analytics tools in just a few weeks to accelerate the company’s understanding of their customers’ needs.(start at 35:58)
1178
Of your peers have already watched this video.
5:30 Minutes
The most insightful time you'll spend today!
Prototyping Language Applications Made Easy with Generative AI
Did you know generative AI allows developers to prototype applications quickly? With Generative AI Studio on Google Cloud, developers can quickly explore and customize AI models that can be leveraged in Google Cloud applications. Watch along and see how developers, with the right tools, can experiment with new ideas in minutes instead of months.
Chapters:
0:00 – Intro
0:29 – Get started with Vertex Generative AI Studio
1:06 – Write your first prompt in Generative AI Studio
1:47 – Prototyping Q&A systems from background text
3:00 – How to save prompts
4:05 – How do LLMs produce output text?
4:41 – Wrap up
Check out more Generative AI for Developers videos → https://goo.gle/GenAIforDevs
Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech
VertexAI #GenerativeAI
Google Cloud’s Recommendation AI Helps Bazaarvoice with 60 Percent increase in CTR
4505
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Not long ago, building AI into recommendation engines was a daunting, expensive task that could take years to get off the ground. But as Bazaarvoice has shown, with the help of cloud services, the time from AI investment to business outcomes is shorter than ever.
Bazaarvoice is the leading provider of product reviews and user-generated content (UGC) solutions that help brands and retailers understand and better serve customers. Its 2019 acquisition of Influenster.com, a community of consumer reviewers 6.5 million strong, expanded the Bazaarvoice portfolio with a platform where consumers can share their candid opinions — and share they have, over 54 million times.
After the acquisition, Bazaarvoice expanded the site’s product diversity by 53%, to more than 5.4 million unique products. To keep user engagement high, Influenster must be seen as both a source of trusted, transparent reviews and a place for customers to discover useful, relevant products for the first time. By introducing shoppers to new products Influenster not only provides value to customers but also helps brands collect consumer insights.
Influenster started out as a place where people gathered to share their honest thoughts on beauty products but quickly expanded to nearly every category, from Art to Wearables. Because of the much smaller scope, the site started and flourished under a rules-based recommendation engine. However, as Influenster expanded its scope under Bazaarvoice, a more robust recommendation system became necessary. In its earliest days, Influenster was successful because of the human perspective it offered: For every product there was a litany of reviews and images that made users feel as if they were getting an endorsement on a product from a friend.
The Bazaarvoice engineering team asked themselves how they could keep that same feeling of personalization with an ever-growing catalog of items and categories. They needed recommendations that could scale with the site, rather than requiring more rules be constructed each time a new product category was introduced. They also needed to ensure the Influenster experience would remain performant even towards unknown members.
Bazaarvoice tested out several recommendation engines, benchmarking each against their current rules-based system. In the end they decided on Google Cloud’s Recommendations AI because of its transparent billing, ease of integration and setup, and naturally, its proven results.
Transparent Billing
“Part of what the engineers loved was they knew exactly what it was going to cost as it scaled” says Nick Shiftan, SVP, Content Acquisition Services Product Unit for Influenster. The goal was to build once and innovate rather than leave a wake of technical debt only to be tackled when costs grew unexpectedly out of control. Google Cloud’s straightforward and pay-as-you-go billing allowed them to anticipate how costs would grow as user interactions did and plan accordingly.
Ease of integration
“I’m positively surprised how Google packed such a complex system in a very easy-to-use API” remarks Eralp Bayraktar, the Software Engineering team lead overseeing the project. Because the original team was made of just one full-time engineer the ease of integration became an even more critical feature. Not only does Recommendations AI pull from years of suggestion expertise in Google Search and YouTube, but in combination with Ad’s Merchant Center, it also creates a streamlined process for importing product metadata. From there, creating a model becomes a matter of picking the preferred recommendation type and then the business objective to optimize for. Once the model is created and the API integrated into the website, the code is already deployed at the global scale: There are no further architectural considerations to ensure recommendations are available to users worldwide. For Bazaarvoice, this meant going from ideation to production in one month.
Proven Results
“We have used it for product recommendations and off-loaded our DB-tiring business logic to Recommendations AI, which resulted in overall faster response times and much better recommendations as proven by our A/B tests,” Eralp continues.
Bazaarvoice began by A/B testing Recommendations AI against their rules-based system. Early on in the experimental phase they noticed a clear and consistent 60% increase in the click-through rate over their original recommendation system.
Even more impressive was the performance on Unknown Members. For every person that signs up for an account on Influenster.com there are many other visitors that come to the website and leave without fully registering. This is typically referred to as the “cold start” problem in the industry — how do you figure out what to recommend to those people without their history, behavior, or preferences? Recommendations AI gives you the option to input and train on unknown users, and by providing metadata on products, it can provide high-quality suggestions to registered members and first-time users alike.
With a mind to the future, Eralp concludes his thoughts on Bazaarvoice’s experience: “It enables discovery by adding an adjustable percentage of cross-category products [for] healthier [traffic distribution] across all our catalog. We are investing in data science and having the Recommendations AI as the baseline is a good challenge for us to thrive.”
To learn more about Recommendations AI and how it can help your organization thrive, check out our recently published 4 part guide which kicks off with an overview on “How to get better retail recommendations with Recommendations AI.” This series also covers data ingestion, modeling, as well as serving predictions & evaluating Recommendations AI. You can also easily get started with our Quickstart Guide.
More Relevant Stories for Your Company
How TeamSnap Improved Return on Ad Spend Significantly
Anyone who has ever coached or played on a sports team, or had a child involved in sports, knows how difficult scheduling and logistics can be. From game and practice schedules, to uniforms and who’s bringing the snacks, it can be a lot for coaches, administrators, parents, and players to
Use No-Cost Solutions To Bring ML Into Your Medical Research
There’s a lot we can learn from combining technology with science to help support the development of amazing discoveries. By using an AI system to predict protein shapes, we have the potential to accelerate research in every field of biology. Inside every cell in your body, billions of tiny molecular
AI-as-a-Service is Here. It’s Really Almost Plug-and-Play
What is the one thing that enterprises want providers to do vis-à-vis AI? To decomplexify it. Today, the path to AI adoption is confusing, and requires skills that are out of reach for most enterprises. That’s what Google Cloud is addressing. “It used to be—still is true—that AI's a pretty
Bra Fit or Brad Pitt? Fun Moments of Using AI for Contact Centers
For many enterprises, the wish to tap into the power of AI is negated only by a lack of know-how: How AI works, what sort and how much data they will need to gather and structure correctly, and how to use the platforms that make AI adoption easier. Google understands
