There’s a new subsea cable in town: Topaz, the first-ever fiber cable to connect Canada and Asia.

Once complete, Topaz will run from Vancouver to the small town of Port Alberni on the west coast of Vancouver Island in British Columbia, and across the Pacific Ocean to the prefectures of Mie and Ibaraki in Japan. We expect the cable to be ready for service in 2023, not only delivering low-latency access to Search, Gmail and YouTube, Google Cloud, and other Google services, but also increasing capacity to the region for a variety of network operators in both Japan and Canada.

Google is spearheading construction of the project, joined by a number of local partners in Japan and Canada to deliver the full Topaz subsea cable system. Other networks and internet service providers will be able to benefit from the cable’s additional capacity, whether for their own use or to provide to third parties. And, similar to other cables we’ve built, with Topaz we will exchange fiber pairs with partners who have systems along similar routes. This is a longstanding practice in the industry that strengthens the intercontinental network lattice for network operators, for Google, and for users around the world.

Network infrastructure investments like Topaz bring significant economic activity to the regions where they land. For example, according to a recent Analysys Mason study, Google’s historical and future network infrastructure investments in Japan are forecasted to enable an additional $303 billion (USD) in GDP cumulatively between 2022 and 2026.

The width of a garden hose, the Topaz cable will house 16 fiber pairs, for a total capacity of 240 Terabits per second (not to be confused with TSPs). It includes support for Wavelength Selective Switch (WSS), an efficient and software-defined way to carve up the spectrum on an optical fiber pair for flexibility in routing and advanced resilience. We’re proud to bring WSS to Topaz and to see the technology is being implemented widely across the submarine cable industry.

While Topaz is the first trans-Pacific fiber cable to land on the West Coast of Canada, it’s not the first communication cable to connect to Vancouver Island. In the 1960s, the Commonwealth Pacific Cable System (COMPAC) was a copper undersea cable linking Vancouver with Honolulu (United States), Sydney (Australia), and Auckland (New Zealand), expanding high-quality international phone connectivity. Today, COMPAC is no longer in service but its legacy lives on. The original cable landing station in Vancouver — the facility where COMPAC made landfall on Canadian soil — has been upgraded to fit the needs of modern fiber optics and will house the eastern end of the Topaz cable.

Traditional and treaty rights, and local communities, are deeply important to our infrastructure projects. The Topaz cable is built alongside the traditional territories of the Hupacasath, Maa-nulth, and Tseshaht, and we have consulted with and partnered with these First Nations every step of the way.

“Tseshaht is very proud of this collaboration and our partnership with Google, who has been very respectful and thoughtful in its engagement with our Nation. That’s how we carry ourselves and that’s how we want business to carry themselves in our territory.“ — Tseshaht First Nation – Elected Chief Councillor-Ken Watts

“The five First Nations of the Maa-nulth Treaty Society are pleased that we have concluded an agreement with Google Canada and have consented to the installation of a new, high-speed fiber optic cable through our traditional territories. This agreement, in which both Google Canada and our Nations benefit, is based on respect for our constitutionally protected treaty and aboriginal rights and enhances the process of reconciliation. We would also like to acknowledge the sensitivity that Google Canada expressed during our talks in regard to the pain and trauma experienced by our people as a result of residential school experience. We look forward to a long and mutually beneficial relationship with Google Canada.” —Chief Charlie Cootes, President of the Maa-nulth Treaty Society

“Google’s respect towards our Nation is appreciated and has good energy behind it.” —Hupacasath First Nation – Elected Chief Councilor – Brandy Lauder

With the addition of Topaz today, we have announced investments in 20 subsea cable projects. This includes Curie, Dunant, Equiano, Firmina and Grace Hopper, and consortium cables like Blue, Echo, Havfrue and Raman — all connecting 29 cloud regions, 88 zones, 146 network edge locations across more than 200 countries and territories. Learn about Google Cloud’s network and infrastructure on our website and in the below video.

As I was preparing for my fireside chat with Dmitri Alperovitch, Founder and Former CTO of CrowdStrike and Executive Chairman of the Silverado Policy Accelerator, for the Google Cloud Government Security Summit taking place on Tuesday, July 20th, I was reflecting about recent developments in cybersecurity, Zero Trust best practices – including aspects of Google’s journey – and the benefits that this approach to cybersecurity brings to the public sector. 

As Dmitri and I discussed what we wanted to share during our session at the Government Security Summit, I quickly realized how pertinent the information was to our public sector audience. For that reason, ahead of Tuesday’s event, I wanted to give you a glimpse of what is to come in our fireside chat. Read below for an exclusive preview of what we plan to discuss. If you are planning to attend on Tuesday, we encourage you to ask questions through our interactive chat function and sign up for 1:1 time with our Ask the Expert program. 

Check out my chat with Dmitri below…

Heather: What do you see as the most significant changes in both the attack surface and cybercriminal tactics in recent years? And why do these changes present unprecedented challenges for public sector? 

Dmitri: In terms of the threat landscape, the more things change, the more they stay the same. The adversaries haven’t changed that much. It’s actually striking. Not all threats are emanating from governments or state actors – either directly or by providing safe harbor to criminals – but a large portion are. That has not changed. The volume and scale of attacks have grown, however, and become massive. And now, the greatest challenge is that everyone is a target. Before, only top-end institutions and government organizations were facing nation state threats. That has changed either through targeted attacks or accidental ones, such as WannaCry. The question now is “How do we protect the most vulnerable and extend much-needed protection to smaller organizations that do not have deep security expertise or resources?”

Heather: Earlier this year, the Biden Administration released its Cybersecurity Executive Order (EO). What do you see as the most important takeaways and potential impact of the EO? 

Dmitri: The Cybersecurity EO is set to have a significant impact. The biggest change is a shift in strategy. For many decades, we all had the mentality that we have to keep attackers out of the network. That is great in theory, but we all know that it is a virtual impossibility with a network of any significant size. Adversaries will find a way in. And, we used to think it was game over once they were in the network.

Then about 10 years ago, we began to see a change. Google led the way with its BeyondCorp strategy as did Lockheed Martin with its sentinel Kill Chain framework. Now, when an adversary penetrates a network, it’s not the end of the game, but the beginning. That is the case if the organization is prepared with a network architecture built around Zero Trust principles. The adversary now has to move laterally, steal credentials, and elevate privileges to get to the resources they want. A Zero Trust architecture can slow them down and give organizations time to detect and eject adversaries from the network. The EO recognizes the power of this approach and has made Zero Trust its fundamental tenet.

Heather: Federal agencies have spent the last 60 days looking at Zero Trust and developing their plans. What do you hope to see? What is critical to accelerating implementation?

Dmitri: The EO establishes a very ambitious timeline, and the US government is not known to move rapidly due to a number of restrictions and considerations. That said, the EO recognizes the idea that logging has to be at the center of modern architecture. Agencies need full visibility into what is going on at endpoints and across the network, have to hunt continuously across networks for adversaries, and must work to rapidly eject them. That is at the core of the EO, and it is a great shift. The private sector should be watching and learning as well.

Heather: What pitfalls should the Federal government look out for in moving forward to implement the EO?

Dmitri: One of the main challenges is the limited authority of the President. Congress is the only branch with the power to change the laws. While the establishment of the Cybersecurity and Infrastructure Security Agency (CISA) a few years ago allows for hunting across agency networks, CISA still has limited authority. It cannot manage the cybersecurity of more than 120 civilian agencies.

Some agencies are performing well when it comes to security, but we have to recognize that this is not the norm. Not all agencies have the capacity to recruit the best cybersecurity talent, but their needs may be just as great. We need to centralize capabilities and leverage the cloud, like Google, to provide all agencies with the best cyber talent and resources.

Join us at the Google Cloud Government Security Summit to hear the rest of our conversation. The event is complimentary. Register today to reserve your spot on July 20th. If you have any questions about the event, please reach out to us at cloudsummitsupport@google.com.

Editor’s note: PedidosYa is the market leader for online food ordering in Latin America, serving 15 markets and over 400 cities. It’s also one of the largest brands within the German multinational company Delivery Hero SE. With over 20 million app downloads, PedidosYa provides the best online delivery experience through 71,000+ online partners, including restaurants, shops, drugstores, and specialized markets. 

Having constant access to fresh customer data is a key requirement for PedidosYa to improve and innovate our customer’s experience. Our internal stakeholders also require faster insights to drive agile business decisions. Back in early 2020, PedidosYa’s leadership tasked the data team to make the impossible possible. Our team’s mission was to democratize data by providing universal and secure access while creating a comprehensive information ecosystem across PedidosYa. We also had to achieve this goal while keeping costs under control— even during the migration stage and removing operational bottlenecks. 

Challenges with legacy cloud infrastructure

PedidosYa first built its data platform on top of AWS. Our data warehouse ran on Redshift, and our data lake was in S3. We used Presto and Hue as the user interfaces for our data analysts. However, maintaining this infrastructure was a daunting task. Our legacy platform couldn’t keep up with the increasing analytics demands. For example, the data stored on S3 complemented by Presto/Hue required high operational overhead. This was because Presto and our IAM (identity access management) didn’t integrate well in our legacy ecosystem. Managing individual users and mapping IAM roles with groups and Kerberos was operationally time-consuming and costly. Further, sharding access on the S3 files was far too complicated to enable seamless ACLs (access control lists).  

There were also challenges with workload management. Our data warehouse had batch data loaded overnight. If one analyst scheduled a query to run during the overnight ETL (extract, transform, load) workload, it would disrupt the current ETL task. This could stop the entire data pipeline. We’d have to wait until data engineers intervened with a manual fix.

It was also difficult to understand whether a query error was due to performance issues or platform resource exhaustion. This lack of clarity affected our data analysts’ ability to autonomously improve querying efficiency. Data team members needed to manually inspect personal queries looking for performance issues. Also,  the current architecture was prone to a ‘tragedy of the commons’ situation; it was seen as an unlimited and free resource. As a result, it was impossible to disentangle the infrastructure from different stakeholder teams, as all had very different needs. 

The decision to modernize our data warehouse

Given the growing challenges from our legacy platform, our tech team decided to transform our analytics environment with a modern data warehouse. They required the following key criteria from their next data platform: 

• Scalability – The ability to grow with elastic infrastructure.
• Cost control – Cost management and transparency. These factors promote efficiency and ownership—both key aspects of data democratization.
• Metadata management – Intuitive data platform focusing on users’ previous SQL knowledge. Plus, being able to enrich the informational ecosystem with metadata,  to diminish data gatekeepers.
• Ease of management – The team needed to reduce operational costs with a serverless solution. Data engineers wanted to focus on their key roles rather than acting as database administrators and infrastructure engineers. The team also wanted much higher availability, and to reduce the impact of maintenance windows and vacuum/analysis.
• Data governance and access rights – With a growing employee base with varying data access requirements, the team needed a simple yet comprehensive solution to understand and track user access to data.

Migrating to Google Cloud

After exploring other alternatives, we concluded Google Cloud had an answer to each of our decision drivers. Google Cloud’s serverless, managed, and integrated data platform, coupled with its seamless integration across open-source solutions, was the perfect answer for our organization. In particular, the natural integration with Airflow as a job orchestrator and Kubernetes for flexible on-demand infrastructure was key.  

We  used Dataflow together with Pub/Sub and Cloud Functions for our data ingestion requirements, which has made our deployment process with Terraform seamless. Because we set up everything in our environment programmatically, operation time has diminished. Google Cloud reduced the deployment process from about 16 hours in our legacy platform to 4 hours.  This is partly due to the friendliness of automating the deployment (such as schema check, load test, table creation, build.) process with Terraform, Cloud Functions, Pub/Sub, Dataflow, and BigQuery on GCP. Input messages processed with Dataflow allow us to abstract and plan the schema changes according to the needs of the functional team. For example, schema changes raise an alarm, and then we can modify the raw layer table schema. By doing this, we ensure that backend modifications that we don’t control do not affect upper layers.

A key reason why we picked Google Cloud was because of its advanced cost and workload management coupled with its transparent log analytics. This information gives us a complete view into any query performance issues to make improvements on the fly. Further, we achieved a significant amount of cost savings by consolidating multiple tools to BigQuery.With BigQuery, we’ve been able to reduce our total cost per query by 5x.

This was due to a number of reasons:

• Automating pipeline deployment made it much simpler to maintain the data processing processes. 
• Analysts are conscious about what queries they’re running, resulting in running better, more optimized queries. 
• Analysts use a Data Studio dashboard to see their queries and all the associated costs. As a result, there’s a lot more transparency for each persona.

 With these changes, we can easily manage and assign costs associated with each workload with their own cost centers using specific Google Cloud projects.

Change management is always challenging. However, BigQuery is intuitive and doesn’t have a steep learning curve from Hue/Hive on SQL basics. BigQuery also allowed the team to expand its capabilities and enabled them to properly work with nested structures, avoiding unnecessary joins and improving query efficiency. Additionally, we now use Data Catalog as our unique point of truth for metadata management. This allows our team to break the data access barriers and enable federation of data across the organization. By using Airflow to orchestrate everything, we keep track of every data stream. With this information, each end user can see their regularly used data entities’ status via the dashboard. This also adds transparency to our everyday data processes.

Finally, with Google Cloud’s IAM rules applied across the different products, data sharing and access is close to a noOps experience. We have programmatically implemented access according to roles and level access within the company. This allows certain pre-validated roles to view more sensitive information. These solutions help drive a more automated data governance experience. 

Up next: Google Cloud AI/ML

The new stack based on BigQuery has created significant productivity gains. Freed from the burden of operational management, PedidosYa’s data team can now focus on adding value through data tools and products.  

• Our data engineers are better equipped to integrate constantly changing transactional and operational data.
• The dataOps team can automate the infrastructure and provide autonomy to the end user.
• Our data quality team can focus on bringing added value to data stakeholders. 
• Data scientists and data analytics can spend more time analyzing data and less time asking data gatekeepers for data access.

PedidosYa can now democratize data access with a well-governed architecture. We are still at the beginning of our journey, but we are closer to achieving our vision of building a data-driven organization. Up next: expanding our artificial intelligence and machine learning capabilities.

Tune in to Google Cloud’s Applied ML Summit on June 10th, 2021, or listen on-demand later, to learn how to apply groundbreaking machine learning technology in your projects.

Commercial trucks often travel partially empty because many shippers don’t have enough cargo to fill an entire container or trailer. Although offering available space to other shippers helps minimize carbon emissions and reduce operating costs, most trucking companies can’t efficiently schedule, track, or deliver multiple freight loads.

Companies have always struggled to ship over-the-road freight efficiently. However,recent economic events have created an unprecedented logistics and transportation crisis that continues to disrupt supply chains, delay deliveries, and significantly raise the price of basic goods. Since some stores can’t keep their shelves fully stocked, many people across the country are finding it more difficult than ever to buy the things they need at an affordable price.

Although exacerbated by the pandemic, many of these supply chain issues have existed for decades. That’s why, in 2015, Flock Freight was started with the mission of reducing waste and inefficiency from the supply chain by reimagining the way freight moves. First to market with advanced algorithms that enable pooling shipments at scale, we create a new standard of service for shippers, increase revenue for carriers and reduce the impact of carbon emissions through shared truckload (STL) service.

Our technology helps lower prices compared to full truckload (FTL) by enabling shippers to only pay for the space they need—and maintain full control over pickup and delivery dates. Flock Freight also optimizes travel routes to speed up deliveries compared to traditional less than truckload (LTL), while eliminating unnecessary shipping hub transfers to minimize damage to cargo.

Today, thousands of shippers and trucking companies across the U.S. use Flock Freight to schedule shared truckloads, lower shipping costs, quickly deliver and track goods, and reduce their carbon footprint by up to 40%. Flock Freight further offsets carbon emissions by buying carbon credits for every FlockDirect™ guaranteed shared truckload shipment—at no extra cost to shippers.

Moving Flock Freight to Google Cloud

We founded Flock Freight with a small team based in southern California. We soon realized we needed a more scalable and affordable technology stack to support our rapidly growing platform and team. After joining the Google for Startups Cloud Program and consulting with dedicated Google startup experts, we decided to move all our data and applications to Google Cloud.

The highly secure-by-design infrastructure of Google Cloud now enables thousands of Flock Freight customers to move their freight faster, cheaper, and with less damage than traditional shipping methods. Specifically, we rely on Google Kubernetes Engine (GKE) to support the combinatorial optimization and machine learning (ML) algorithms and services that identify, pool, and schedule shared truckloads. We also leverage GKE to rapidly develop, deploy, and manage new applications and services.

In addition, we leverage Cloud SQL to automate database provisioning, storage capacity management, and other time-consuming tasks. Cloud SQL easily integrates with existing apps and Google Cloud services such as GKE and Pub/Sub. Lastly, we use Compute Engine to create and run virtual machines, optimize resource utilization, and lower computing costs by up to 91%. These cost savings allow us to shift more resources to R&D and rapidly develop new solutions and services for our customers.

Building a greener, more resilient, and responsive supply chain

The Google for Startups Cloud Program and dedicated Google startup experts were instrumental in helping us manage cloud infrastructure cost and maintaining very high SLAs, helping Flock Freight to focus on developing a comprehensive shipping platform that powers shared truckloads and drives positive industry change.

We especially want to highlight the Google Cloud research credits we relied on to launch Flock Freight and make rapid progress toward transforming the shipping industry. To this day, we continue to work with Google Cloud Managed Services partner DoiT International International to further scale and optimize operations on Google Cloud.

We’re proud of the results we’re delivering for our customers. For example, a home improvement importer now enjoys faster, safer, and easier shipping with 99.9% damage-free service and a 97.5% on-time delivery rate. A packaging supplier continues to maintain a 99% on-time delivery streak and decrease carbon emissions by 37%, while a mineral water company consistently reduces delivery expenses upwards of 50%.

Nationwide demand for shared truckloads continues to increase as the shipping industry works to lower costs and alleviate supply chain disruptions. With the Flock Freight platform, companies are building a more sustainable and resilient supply chain by efficiently combining multiple shipments into shared truckloads.

If you want to learn more about how Google Cloud can help your startup, visit our page here to get more information about our program, and sign up for our communications to get a look at our community activities, digital events, special offers, and more.