At Universe, we serve customers day and night and are always working to make sure they have a great experience, whether online or at one of our live events.

Our technology has to make that possible, and our legacy systems weren’t cutting it anymore. What we needed was a consistent, reliable infrastructure that would help our internal teams provide a fast and innovative ticket-buying experience to customers.

With our data well-managed, we could free up time for our developers to bring new web features to customers, like tailored add-ons at checkout.

Our team of about 20 software engineers needed more flexibility and agility in our infrastructure; we were using various data processing tools, and it wasn’t easy to share data across teams so that everyone saw the same information.

We also needed to incorporate streaming data into the data warehouse to ensure the consistency and integrity of data that’s read in a particular window of time from multiple sources. Our developer teams needed to be able to ship new features faster, and the data back ends were getting in the way.   

In addition, when GDPR regulations went into effect, we needed to make sure all our data was anonymized, and we couldn’t do that with our legacy tools.

Finding the right data tools for the job
To make sure our customers were getting a top-notch online experience, we had to make the right technology choices.

Our first step was to centralize multiple data sources and create a single data warehouse that could serve as the foundation for all of our reporting requirements, both internal and external.

The new technology infrastructure we built had to let us move and analyze data easily, so our teams could focus on using that data and insights to better serve our customers.

Previously, we had lots of siloed systems and applications running in AWS. We did a trial using Redshift, but we needed more flexibility than it offered in how we loaded historical data into our cloud data warehouse. Though we were using MongoDB Atlas for our transactional database, it was important to continue using SQL for querying data.  

The trial task that really sold us on BigQuery was when we wanted to alter a small table that had about 20 million rows, used for internal reporting.

We needed to add a value, but our PostgreSQL system wouldn’t allow it. Using Apache Beam, we set up a simple pipeline that moved data from the original source into BigQuery to see if we could add the column there.

BigQuery ingested the data and let us add the new value in seconds. That was a significant moment that led us to start looking at how we could build end-to-end solutions on Google Cloud. BigQuery gave us multiple options to load our historical data in batches and build powerful pipelines. 

We also explored Google Cloud’s migration tools and data pipeline options. Once we saw how Cloud Dataflow worked, with its Apache Beam back-end, we never looked back. Google Cloud provided us with the data tools we needed to build our data infrastructure. 

Cloud for data, and for users
Introducing new technologies isn’t always simple—companies sometimes avoid it altogether because it’s so hard. But our Google Cloud onboarding process has been easy. 

It took us less than two months to fully deploy our BigQuery data warehouse using the Cloud Dataflow-Apache Beam combination.

Moving to Google Cloud brought us a lot of technology advantages, but it’s also been hugely helpful for our internal users and customers. The data analytics capabilities that we’re now able to offer users has really impressed our internal teams, including developers and DevOps, even those who haven’t used this type of technology before.

Some internal clients are already entirely self-service. We’ve hosted frequent demos, and also hosted some “hack days,” where we share knowledge with our internal teams to show them what’s possible. 

We quickly found that BigQuery helped us solve scale and speed problems.

One of our main pain points had been adding upsell opportunities for customers during the checkout process. The legacy technology hadn’t allowed us to quickly reflect those changes in the data warehouse. With BigQuery, we’re able to do that, and devote fewer resources to making it happen. We’ve also eliminated the time we were spending tuning memory and availability, since BigQuery handles that. Database administration and tuning required specialized knowledge and experience and took up time. With BigQuery, we don’t have to worry about configuring that hardware and software. It just works.

We’ve also eliminated the time we were spending tuning memory and availability, since BigQuery handles that. Database administration and tuning required specialized knowledge and experience and took up time. With BigQuery, we don’t have to worry about configuring that hardware and software. It just works.

Two features in particular that we implemented using BigQuery have helped us improve the performance of our core transactional database. First, using Cloud Dataflow to convert raw MongoDB logs to structured rows under a BigQuery table, which we can then query using SQL to identify slow or underperforming queries. Second, we can now query multiple logging tables using wildcards, since we load Fastly logs to BigQuery. 

Along with MongoDB Atlas as our main transactional database, much of our infrastructure now runs as Google Cloud microservices using Google Kubernetes Engine (GKE), including the home page and our payment system. Kubernetes cron jobs power background scheduled jobs, and we also use Cloud Pub/Sub. Cloud Storage handles any data storage if any space constraints emerge. 

Our overall performance has increased by about 10x with BigQuery. Both our customers and internal clients, like our sales and finance teams, benefit from the new low-latency reporting. Reports that used to be weekly or monthly are now available in near-real time. It’s not only faster to read records, but faster to move the data, too.

We have Cloud Dataflow pipelines that write to multiple places, and the speed of moving and processing data is incredibly helpful. We stream in financial data using Cloud Dataflow in streaming mode, and plan to have different streaming pipelines as we grow. We have several batch pipelines that run every day. We can move terabytes of data without performance issues, and process more than 100,000 rows of data per second from the underlying database. It used to take us a month to move that volume of data into our data warehouse. With BigQuery, it takes two days.

We’re also enjoying how easy and productive these tools are. They make our life as software engineers easier, so we can focus on the problem at hand, not fighting with our tools. 

What’s next for Universe
Our team will continue to push even more into Google Cloud’s data platform. We have plans to explore Cloud Datastore next. We’re also moving our databases to PostgreSQL on GCP, using Cloud Dataflow and Beam. BigQuery’s machine learning tools may also come into play as Universe’s cloud journey evolves, so we can start doing predictive analytics based on our data. We’re looking forward to gaining even more speed and agility to meet our business goals and customer needs.    

“Talk about tough times, right?”

That’s how Dan Stuart, Senior Vice President of IT Services at Southwire Company, refers to the months following a December 2019 ransomware event, and the COVID crisis that began in spring of 2020. Those events hit just as the company was preparing for an overhaul of their SAP environment. This comprehensive plan included three key elements. First, the company wanted to upgrade their SAP ECC environment to take advantage of the latest functionality available for this critical ERP system. Second, Southwire aimed to deploy SAP Business Warehouse on SAP HANA to accelerate vital reporting for all business users. Third, the company wanted to upgrade to the latest version of SAP Process Orchestration—an essential component that touches key manufacturing interfaces in all Southwire facilities. 

Southwire had looked at multiple options for the upgrades, including remaining entirely on-premises, colocation, and full cloud migration. “Going to the cloud seemed a lot more compelling,” says Joe Schleupner, Southwire’s Senior Director of PMO & ITS planning and implementation. “We were going to the cloud eventually, so why take these intermediary steps? Let’s just get it done.”

After looking at several options, Southwire decided to migrate to Google Cloud. “We wanted to be on a platform for SAP that was flexible, scalable, and secure; that we could count on to get up and running quickly,” says Stuart. “We chose Google Cloud not only for those reasons, but also because we recognize that Google has other assets that we may be able to take advantage of down the line, such as technologies like artificial intelligence (AI).” 

More stability, less worry

As one of the leading manufacturers of wire and cable used in the transmission and distribution of electricity, Southwire aids the delivery of power to millions of people worldwide. They have more than 30 manufacturing facilities across the United States running 24/7. Any downtime directly affects productivity and revenue. With help from Google Cloud and their implementation partner NIMBL, Southwire completed the SAP migration to Google Cloud over a planned maintenance weekend on July 4th.

The migration itself, while complex, went quickly and smoothly. “Just moving to the cloud was quite a feat because we were dealing with so much data, but in total the SAP system was down for only ~16 hours,” says Schleupner.

“As a project manager, I always felt that Google Cloud had my back” Schleupner says. The Process Orchestration (PO) migration was of particular concern, considering that it controlled all of Southwire’s manufacturing interfaces across the entire company. “Every critical piece of information that goes from SAP down to the manufacturing system goes through that system,” says Schleupner.

Even after migrating, Southwire discovered that making changes to the system was fast, easy, and resulted in no downtime. Normally, certain types of changes would have involved taking down SAP for at least an hour.

The Southwire team also appreciates the fact that the modern cloud architecture means spending less time on routine infrastructure maintenance. “It’s one less thing for me to worry about,” Stuart says, “I can focus on the business side of the house and move the technology and responsibilities to what we do within the Google Cloud Platform.”

What comes next?

While the cloud migration will increase stability, uptime, performance, and security, there is much more to come. Southwire is currently working on a disaster recovery implementation for their SAP environment on Google Cloud. Stuart and Schleupner are excited about where Google Cloud can further take Southwire. They are considering an SAP Hybris e-commerce implementation as well as connected factory and/or factory automation initiatives that can take advantage of artificial intelligence and machine learning. 

To Stuart and Schleupner, the migration of Southwire’s SAP environment to Google Cloud, as important as it was, really represents the first step in the company’s tech evolution. Now that much of the heavy lifting is complete, Southwire’s digital transformation can begin in earnest. “There’s no shortage of areas where I think Google Cloud will come into play,” Stuart says, “and we intend to look at these things with an open mind to understand how we can leverage current investments to take our organization where we want to go.”

Learn more about Southwire’s SAP on Google Cloud deployment and how Google Cloud can transform the way you work with your SAP enterprise applications. Visit cloud.google.com/solutions/sap.

For more than 15 years, developers have used Google Maps Platform to deliver location-based experiences to their end users and used location intelligence to optimize their businesses. Along this journey, we’ve made a variety of changes to better support our community as needs have changed and new industries and technologies have emerged. We started rolling out a new website experience, at https://mapsplatform.google.com, to help you better understand the products and solutions best suited to address your objectives. Plus, now you can directly connect to the developer documentation for each product to get started quickly, and you can visualize usage and associated costs to have a better idea of what to expect before getting started. 

Getting to your solution faster 

Maps, Routes, Places are building blocks that let you develop implementations for any use case. Building for specific use cases, however, typically requires using a combination of APIs and SDKs. To help you quickly understand what’s possible and what you need to build for your use case, you can now visit the solutions tab to select from a list of popular use cases or industries. Once you’ve selected a use case or industry, you’re taken to a page where you can explore relevant products, read helpful blog posts, see how other customers have deployed for similar use cases, and more.  

Direct access to developer documentation

Did you know there are more than a thousand pages of developer documentation created to help you get started, unblock you when you’re stuck, and share best practices? Now when you explore a product or solution from the Google Maps Platform website, you can easily navigate back and forth between our website and documentation. Just tap on JS, iOS, Android or API under the product name to get to the documentation you need. 

Budgeting for your project

To help you calculate pricing for your project, we’ve introduced a new pricing calculator. Once you find the product and API or SDK you plan to use, pull the slider to reflect your estimated number of monthly requests. This will automatically update the “monthly cost” column for each product and API or SDK you plan to use. If your estimated monthly requests exceed the slider limit, contact our sales team to ​​learn about volume discounts that start at 20% off. 

We hope our new website makes it easier to discover our products and solutions, estimate your budget, and start building with our documentation so you can deliver helpful experiences to your users and optimize your business. 

For more information on Google Maps Platform, visit https://mapsplatform.google.com.

Organizations that collect geospatial data can use that information to understand their operations, help make better business decisions, and power innovation. Traditionally, organizations have required deep GIS expertise and tooling in order to deliver geospatial insights. In this post, we outline some ways that geospatial data can be used in various business applications. 

Assessing environmental risk 

Governments and businesses involved in insurance underwriting, property management, agriculture technology, and related areas are increasingly concerned with risks posed by environmental conditions. Historical models that predict natural disasters like pollution, flooding, and wildfires are becoming less accurate as real-world conditions change. Therefore, organizations are incorporating real-time and historical data into a geospatial analytics platform and using predictive modeling to more effectively plan for risk and to forecast weather.

Selecting sites and planning expansion

Businesses that have storefronts, such as retailers and restaurants, can find the best locations for their stores by using geospatial data like population density to simulate new locations and to predict financial outcomes. Telecom providers can use geospatial data in a similar way to determine the optimal locations for cell towers. A site selection solution can combine proprietary site metrics with publicly-available data like traffic patterns and geographic mobility to help organizations make better decisions about site selection, site rationalization, and expansion strategy.

Planning logistics and transport

For freight companies, courier services, ride-hailing services, and other companies that manage fleets, it’s critical to incorporate geospatial context into business decision-making. Fleet management operations include optimizing last-mile logistics, analyzing telematics data from vehicles for self-driving cars, managing precision railroading, and improving mobility planning. Managing all of these operations relies extensively on geospatial context. Organizations can create a digital twin of their supply chain that includes geospatial data to mitigate supply chain risk, design for sustainability, and minimize their carbon footprint. 

Understanding and improving soil health and yield

AgTech companies and other organizations that practice precision agriculture can use a scalable analytics platform to analyze millions of acres of land. These insights help organizations understand soil characteristics and help them analyze the interactions among variables that affect crop production. Companies can load topography data, climate data, soil biomass data, and other contextual data from public data sources. They can then combine this information with data about local conditions to make better planting and land-management decisions. Mapping this information using geospatial analytics not only lets organizations actively monitor crop health and manage crops, but it can help farmers determine the most suitable land for a given crop and to assess risk from weather conditions.

Managing sustainable development

Geospatial data can help organizations map economic, environmental, and social conditions to better understand the geographies in which they conduct business. By taking into account environmental and socio-economic phenomena like poverty, pollution, and vulnerable populations, organizations can determine focus areas for protecting and preserving the environment, such as reducing deforestation and soil erosion. Similarly, geospatial data can help organizations design data-driven health and safety interventions. Geospatial analytics can also help an organization meet its commitments to sustainability standards through sustainable and ethical sourcing. Using geospatial analytics, organizations can track, monitor, and optimize the end-to-end supply chain from the source of raw materials to the destination of the final product.

What’s next

Google Cloud provides a full suite of geospatial analytics and machine learning capabilities that can help you make more accurate and sustainable business decisions without the complexity and expense of managing traditional GIS infrastructure. Get started today by learning how you can use Google Cloud features to get insights from your geospatial data, see Geospatial analytics architecture.

Acknowledgements: We’d like to thank Chad Jennings, Lak Lakshmanan, Kannappan Sirchabesan, Mike Pope, and Michael Hao for their contributions to this blog post and the Geospatial Analytics architecture.