Google Cloud’s High-performance Compute Speeds Up the Chip Design Process
3497
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Cloud offers a proven way to accelerate end-to-end chip design flows. In a previous blog, we demonstrated the inherent elasticity of the cloud, showcasing how front-end simulation workloads can scale with access to more compute resources. Another benefit of the cloud is access to a powerful, modern and global infrastructure. On-prem environments do a fantastic job of meeting sustained demand but Electronic Design Automation (EDA) tooling upgrades happen much more frequently (every six to nine months) than typical on-prem data center infrastructure upgrades (every three to five years).
What this means is that your EDA tool can provide much better performance if given access to the right infrastructure. This is especially useful in certain phases of the design process.
Take for example, a physical verification workload. Physical verification is typically the last step in the chip design process. In simplified terms, the process consists of verifying design rule checks (or DRCs) against the process design kit (PDK) provided by the foundry. It ensures that the layout produced from the physical synthesis process is ready for handoff to a foundry (in-house or otherwise) for manufacturing. Physical verification workloads tend to require machines with large memories (1TB+) for advanced nodes. Having access to such compute resources enables more physical verification to run in parallel, increasing your confidence in the design that is being taped out (i.e., sent to manufacturing).
At the other end of the spectrum are functional verification workloads. Unlike the physical verification process described above, functional verification is normally performed in the early stages of design and typically requires machines with much less memory. Furthermore, functional verification (dynamic verification in particular) accounts for the most time (translating directly to the availability of compute) in the design cycle. Verifying faster, an ambition for most design teams, is often tied to availability of right-sized compute resources.
The intermittent and varied infrastructure requirements for verification (both functional and physical) can be a problem for organizations with on-prem data centers. On-prem data centers are optimized for maximizing utilization—this does not directly address access to right-sized compute to deliver the best tool performance. Even if the IT and Computer Aided Design (CAD) departments choose to provision additional suitable hardware, the process of provisioning, acquiring and setting up new hardware on-prem typically takes months for even the most modern organizations. A “hybrid” flow that enables use of on-prem clusters most of the time, but provides seamless access to cloud resources as needed would be ideal.
Hybrid chip design in action
You can improve a typical verification workflow simply by utilizing a hybrid environment that provides instantaneous access to better compute. To illustrate, we chose a front-end simulation workflow, and designed an environment that replicates on-prem and cloud clusters. We also took a few more liberties to simplify the environment (described below). The simplified setup is provided in a GitHub repository for you to try out.
In any hybrid chip design flow, there are a few key considerations:
- Connectivity between on-prem infrastructure and the cloud: Establishing connectivity to the cloud is one of the most foundational aspects of the flow. Over the years, this has also become a very well-understood field, and secure, high availability connectivity is a reality in most setups.
In our tutorial, we represent both on-prem and cloud clusters as two different networks in the cloud where all traffic is allowed to pass between these networks. While this is not a real-world network configuration, it is sufficient to demonstrate the basic connectivity model. - Connection to license server: Most chip design flows utilize tools from EDA vendors. Such tools are typically licensed, and you need a license server with valid licenses to operate the tool. License servers may remain on-prem in the hybrid flow, so long as latency to the license server is acceptable. You can also install license servers in the cloud on a Compute Engine VM (particularly sole-tenant nodes) for lower latency. Check with your EDA vendors to understand if you can rehost your license services in the cloud.
In our tutorial, we use an open source tool (Icarus Verilog Simulator) and therefore, do not need a license server. - Identifying data sources and syncing data: There are three important aspects in running EDA jobs: the EDA tools themselves, the infrastructure where the tools run, and the data sources for the tool run. Tools don’t change much, and can be installed on cloud infrastructure. Data sources, on the other hand, are primarily created on-prem and updated regularly. These could be SystemVerilog files that describe the design, the testbenches or the layout files. It is important to sync data between on-prem and cloud to maintain parity. Furthermore, in production environments, it’s also important to maintain a high-performance syncing mechanism.
In our tutorial, we create a file system hierarchy in the cloud that is similar to one you’d find on-prem. We transfer the latest input files before invoking the tool. - Workload scheduler configuration and job submission transparency: Most environments that leverage batch jobs use job schedulers to access a compute farm. An ideal environment finds the balance between cost and performance, and builds parameters in the system to enable predictive (and prescriptive) wrappers to job schedulers (see picture below).
In our tutorial, we use the open-source SLURM job scheduler and an auto-scaling cluster. For simplicity, the tutorial does not include a job submission agent.
Other cloud-native batch processing environments such as Kubernetes can also provide further options for workload management.
Our on-prem network is called ‘onprem’ and the cloud cluster is called ‘burst’. Characteristics of the on-prem and burst clusters are specified below:
Once set up, we ran the OpenPiton regression for single and two-tile configurations. You can see the results below:
Regressions run on “burst” clusters were on average 30% faster than on “onprem”, delivering faster verification sign-off and physical verification turnaround times. You can find details about the commands we used in the repository.
Hybrid solutions for faster time to market
Of course, on-prem data centers will continue to play a pivotal role in chip design. However, things have changed. Cloud-based, high performance compute has proved itself to be a viable and proven technology for extending on-prem data centers during the chip design process. Companies that successfully leverage hybrid chip design flows will be able to better address the fluctuating needs of their engineering teams. To learn more about silicon design on Google Cloud, read our whitepaper “Using Google Cloud to accelerate your chip design process”.
IT Team Figures Out Easiest Way to Build Data Pipelines and Create ML Models
14064
Of your peers have already read this article.
5:15 Minutes
The most insightful time you'll spend today!
Building a strong brand in today’s hyper-competitive business environment takes vision. It also requires a flexible, easily managed approach to digital asset management (DAM), so marketing professionals and other stakeholders can easily share, store, track, and manipulate assets to build the brand.
Many of today’s leading companies, including JetBlue, Slack, TripAdvisor, Lyft, and HealthONE, rely on Brandfolder to deliver consistent, organized, and efficient brand experiences. Brandfolder provides an easy-to-use platform that can scale across an entire company with little end-user training, empowering customers to distribute digital assets wherever they are needed. Customers also gain much greater insight into how those assets are used, and how to use them more effectively in marketing campaigns and brand messaging.
“Google Cloud made it easy to build an ML platform to quickly iterate through different brand intelligence use cases and release data-driven product features into the Brandfolder platform.”
—Ajay Rajasekharan, Head of Data Science, Brandfolder
Brandfolder is constantly advancing its development efforts to introduce new data-driven features without complicating the user experience. Big data, artificial intelligence (AI), and machine learning (ML) are key to meeting customers’ unique business needs, and essential for Brandfolder to compete in the fast-moving DAM industry. To enhance these capabilities, Brandfolder sought a public cloud provider that could help it scale its data pipeline cost effectively while providing access to advanced AI technologies.
After graduating from the Techstars startup accelerator program in 2013, Brandfolder tried two other cloud providers before standardizing on Google Cloud Platform (GCP).
“We saw a difference with Google Cloud from the very beginning because the interactions felt like a strategic relationship,” says Jim Hanifen, Head of Product at Brandfolder. “Google gave us startup credits and a lot of face-to-face support, which we hadn’t experienced with other cloud providers. We decided to move our entire infrastructure to Google Cloud Platform.”
Building an ML platform for brand intelligence
After performing an initial lift-and-shift migration of virtual machines (VMs) onto Compute Engine, Brandfolder built an ML platform using GCP managed services to seamlessly deliver its data products. The platform leverages Cloud SQL, Cloud Storage as the data lake, Cloud Dataproc for cloud-native Apache Spark computing clusters, Cloud Composer as the batch job scheduler, Cloud Pub/Sub as the backbone data pipeline, Container Registry to store Docker images, and Google Kubernetes Engine (GKE) as the application orchestrator. Cloud Dataflow brings data into the data lake and into BigQuery for analysis.
“Google Cloud made it easy to build an ML platform to quickly iterate through different brand intelligence use cases and release data-driven product features into the Brandfolder platform,” says Ajay Rajasekharan, Head of Data Science at Brandfolder, who describes the architecture in a detailed blog. “We simply ingest raw application and event data on one end and output an ML service on the other.”
“Moving to Google Cloud Platform allows us to complete more sophisticated data analysis and ML models much faster, and at a much lower cost. We can create brand-specific ML models 12x faster and get them into production quickly to address our customers’ unique business needs.”
—Brett Nekolny, Head of Engineering, Brandfolder
For many general use cases, Brandfolder does not need to build custom ML models, and instead relies on pre-trained API models from GCP. For example, it uses Vision API and Video Intelligence API to auto-tag creative assets on import to enable fast, intuitive searches across images and videos. When more product- and brand-specific modeling is required to address unique customer use cases, Brandfolder builds and trains custom ML models using its GCP pipeline or Cloud AutoML, a suite of products built on Google transfer learning and neural architecture search technology. For example, if a Brandfolder customer makes different types of grills, Brandfolder can use AutoML Vision to train a model to recognize the different grills.
“Moving to Google Cloud Platform allows us to complete more sophisticated data analysis and ML models much faster, and at a much lower cost,” explains Brett Nekolny, Head of Engineering at Brandfolder. “We can create brand-specific ML models 12x faster and get them into production quickly to address our customers’ unique business needs.”
Industry-leading security and performance
Google Cloud’s security model helps Brandfolder give existing and prospective customers peace of mind that their data will be protected. Cloud Identity & Access Management (Cloud IAM) provides enterprise-grade access control, while Cloud Identity-Aware Proxy (Cloud IAP) enables remote users to work more securely without the hassles of a VPN client. GCP also isolates cloud resources into projects, making it easy to assign permissions and keep data and VMs organized and segregated.
“With Google Cloud, everything begins and ends with security, which makes things very easy for us,” says Jim. “If we’re under a security review, we can submit a Google security white paper. If a potential customer has security concerns, we tell them we are hosted on GCP, and those concerns go away.”
To give customers even better application performance for accessing their brand assets, Brandfolder uses Cloud Memorystore, an in-memory data store service for Redis, to cache data and provide sub-millisecond data access for production applications.
“It was much easier for us to use Cloud Memorystore versus running Redis on our compute instances,” says Brett. “The high availability, replication across zones, and automatic failover with no data loss are big for us.”
Global private network interconnects between Google Cloud and the Fastly content delivery network (CDN) dramatically reduce latency, allowing Brandfolder’s customers to deliver and update even very large creative assets quickly around the world.
“What’s beautiful about the relationship between Google and Fastly is that if one of our customers uploads a new version of an asset, we can propagate that out to Fastly, and the new version will automatically show up in all the places where it’s referenced,” says Brett.
“The ability to quickly solve problems with AI has a substantial impact on our revenue, and that’s more apparent every quarter. Few of our competitors are doing product- or brand-specific modeling because it takes a lot of time and resources. We overcame those hurdles with Google Cloud.”
—Jim Hanifen, Head of Product, Brandfolder
Improving employee and customer productivity
Brandfolder also uses Google solutions for real-time collaboration and productivity, using G Suite to connect employees with intuitive, cloud-based apps. Teams use Gmail, Calendar, Docs, Drive, Sheets, Slides, and Hangouts Meet every day to move the business forward. Many of Brandfolder’s customers are also G Suite users, and Brandfolder offers a plug-in that allows them to view their creative assets inside of Docs and pull images in as needed. Customers can also log into Brandfolder with their G Suite credentials, making the solution even easier to use.
“We’ve been using G Suite since the beginning, and it’s helped us collaborate efficiently to build a successful, growing company,” says Jim. “Our teams expect to have that kind of close collaboration, and everyone here enjoys the G Suite experience.”
Driving 99 percent annual business growth
With automated tagging and other innovative AI-based features, Brandfolder is helping customers locate and distribute assets faster. As a result, Brandfolder is building customer loyalty and increasing sales, growing its business by 99 percent year-over-year. Since moving to GCP, Brandfolder has been able to scale its analytics and data pipeline 50x without a corresponding increase in costs and has not had to expand its development team.
“The ability to quickly solve problems with AI has a substantial impact on our revenue, and that’s more apparent every quarter,” says Jim. “Few of our competitors are doing product- or brand-specific modeling because it takes a lot of time and resources. We overcame those hurdles with Google Cloud.”
3419
Of your peers have already watched this video.
1:30 Minutes
The most insightful time you'll spend today!
Google Cloud Cortex Framework: Innovate on Cloud with Less Risk, Cost and Complexity!
Google Cloud Cortex Framework is a comprehensive approach to cloud innovation that enables users to accelerate value with less risk, complexity and cost! The Cortex Framework includes a comprehensive tools and know-how to build, design and deploy cloud solutions to address business challenges and achieve desired outcomes. Watch the video to get started with Google Cloud Cortex Framework.
VMware Engine’s Exciting New Updates: A Google Cloud Journey
1335
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
IT leaders today are being asked to simultaneously support their company’s infrastructure, find opportunities for growth, and meet their goals with fewer resources and smaller budgets than before. Recently we highlighted three customers who are leveraging Google Cloud VMware Engine to achieve these goals while lowering their TCO and transforming their organization.
It’s because of these successful customer outcomes that we have been awarded the 2023 VMware Cloud Innovation and SaaS Transformation partner achievement award for delivering solutions that accelerate customers’ digital transformation journey. We’re honored to receive this award and continue to stay focused on delivering tremendous value to our customers.
In the past few months, we’ve also made several updates to Google Cloud VMware Engine. Today’s post provides a recap of the latest milestones that make it easier for you to migrate and run your vSphere workloads in a cloud-first, enterprise-class VMware environment in Google Cloud.
Back in September 2022, we announced a number of updates including the preview of API/CLI support (which is now available). In February 2023, we also talked about how to use NetApp CVS as datastores for VMware Engine.
Key updates this time around include:
Availability of VMware Engine in Delhi, Santiago and Milan regions: This brings the availability of VMware Engine to 17 regions worldwide, each supporting 4 9’s of uptime SLA for clusters 5 or more, serving the needs of our regional and multi-national customers. In addition, we have also added a second zone in the London region.
Filestore datastore support for VMware Engine: Generally Available in all VMware Engine regions, you can use Filestore High Scale and Enterprise tier instances as external NFS datastores for VMware Engine nodes. Filestore is VMware certified as an NFS datastore with VMware Engine. You can size compute and storage capacity independently to meet your workload requirements for your storage-intensive VMs. You can also leverage vSAN for low-latency VM requirements and scale Filestore from TBs to PBs for the capacity hungry VMs. If interested in this feature, please contact your Google account team.
Stretched private clouds: These private clouds stretch across two data zones and a witness zone all within the same Google Cloud region. Stretched private clouds use vSphere and vSAN stretched clusters to provide compute and storage high availability against zone-level failures. This capability is now available in Frankfurt and Sydney regions. Learn more here.
Zerto solution version 9.5u1 support: This recovery solution allows critical infrastructure and application virtual machines (VMs) to be replicated continuously from your on-premises vCenter to your private cloud. Learn more about setting up Zerto here.
Google Cloud Backup and Disaster Recovery (GCBDR): GCBDR is available to protect applications running in VMware Engine, and can be managed within the Google Cloud Console. We recently launched GCBDR under Google Cloud Platform Terms of Service simplifying customers’ purchasing and support experience.
vTPM support: Google Cloud VMware Engine private clouds now support the addition of a Trusted Platform Module (TPM) 2.0 virtual cryptoprocessor to a virtual machine. You can add vTPMs to VMs by following VMware instructions or upgrading your existing VMs to include a vTPM. You can read more about this in the VMware blog.
This brings us to the end of our updates this time. For the latest updates to the service, please bookmark our release notes.
No More Migration Challenges: How Nine Clients of Google Cloud Achieved Success
DOWNLOAD E-BOOK3296
Of your peers have already downloaded this article
15:00 Minutes
The most insightful time you'll spend today!
Google Cloud’s hands-on, tailored partnerships and solutions have helped many businesses around the world to ease their cloud migration journey and achieve desired outcomes! Download the E-book to observe the public cloud landscape and understand the common hurdles to cloud adoption, the benefits of multi-phased migration with Google Cloud, cloud migration checklist as well as nine client testimonials.
Make your cloud migration journey cost-effective, fast and easy with Google Cloud!
3262
Of your peers have already downloaded this article
2:30 Minutes
The most insightful time you'll spend today!
Technology leaders have shifted many apps and workloads to public cloud, and they are not finished; 63% plan further expansion in the next 12 months.
According to this newly resealed Forrester report, enterprises are migrating all types of apps and workloads to cloud and the migration provides firms with a wide range of IT benefits.
Download this Forrester Report to learn how to maximize the value of your cloud migration.
More Relevant Stories for Your Company
How TapClicks’ Google Cloud Migration Makes Life Easy for Marketers
Editor’s note: In this blog post we learn how TapClicks migrated to Google Cloud to offer their marketing customers a unified platform for data management, operations, insights, and analysis. TapClicks is a smart marketing cloud, powered by data, that unifies our customer’s marketing. By choosing to migrate our core applications last
Google and NetApp Partnership Expansion to Ease Customers’ Cloud Migration
As organizations continue to evolve to meet the needs of their stakeholders, they are increasingly looking to adopt the cloud and digitally transform their business. Since our partnership with NetApp began in 2018, we’ve been focused on delivering the infrastructure, tools, and services to help customers on their cloud migration journey and
Johnson & Johnson Increases it’s Ability to Find Highly Qualified Staffers for Business Critical Roles by 41% with Easy-to-Use AI
Job seekers can often feel lost or disconnected—like the right opportunity is out there, but they don’t know where or how to look. Employers face a similar challenge when trying to attract the right candidates. Many companies, especially large enterprises, face a talent shortage across a range of critical roles. For global
Cloud and AI Paves the Future of Finance: Excerpts from FIA Boca 2022
Financial markets were among the first to adopt new technologies, and that has certainly been true of the derivatives markets, which were early adopters of electronic trading. Going forward, new capabilities will transform the way industry participants communicate, analyze, and trade. I sat down with Google Cloud’s Phil Moyer and
