This week, the Vera C. Rubin Observatory is launching the first preview of its new Rubin Science Platform (RSP) for an initial cohort of astronomers. The observatory, which is located in Chile but managed by the U.S. National Science Foundation’s NOIRLab in Tucson, AZ and SLAC in California, is jointly funded by the NSF and the U.S. Department of Energy. The platform provides an easy-to-use interface to store and analyze the massive datasets of the Legacy Survey of Space and Time (LSST), which will survey a third of the sky each night for ten years, detecting billions of stars and galaxies, and millions of supernovae, variable stars, and small bodies in our Solar System.

The LSST datasets are unprecedented in size and complexity, and will be far too large for scientists to download to their personal computers for analysis. Instead, scientists will use the RSP to process, query, visualize, and analyze the LSST data archives through a mixture of web portal, notebook, and other virtual data analysis services. An initial launch with simulated data, called Data Preview 0, builds on the Rubin Observatory’s three-year partnership with Google to develop an Interim Data Facility (IDF) on Google Cloud to prototype hosting of the massive LSST dataset. This agreement marks the first time a cloud-based data facility has been used for an astronomy application of this magnitude.

Bringing the stars to the cloud

For Data Preview 0, the IDF leverages Cloud Storage, Google Kubernetes Engine (GKE), and Compute Engine to provide the Rubin Observatory user community access to simulated LSST data in an early version of the RSP. The simulated data were developed over several years by the LSST Dark Energy Science Collaboration to imitate five years of an LSST-like survey over 300 square degrees of the sky (about 1,500 times the area of the moon). The resulting images are very realistic: they have the same instrumental characteristics, such as pixel size and sensitivity to photons, that are expected from the Rubin Observatory’s LSST Camera, and they were processed with an early version of the LSST Science Pipelines that will eventually be used to process LSST data. “This will be the first time that these workloads have ever been hosted in a cloud environment. Researchers will have an opportunity to explore an early version of this platform,” says Ranpal Gill, senior manager and head of communications at the Rubin Observatory.

Broadening access for more researchers

Over 200 scientists and students with Rubin Observatory data rights were selected to participate in Data Preview 0 from a pool of applicants that represents a wide range of demographic criteria, regions, and experience level. Participants will be supported with resources such as tutorials, seminars, communication channels, and networking opportunities—and they will be free to pursue their own science at their own pace using the data in the RSP. 

“The revolutionary nature of the future LSST dataset requires a commensurately innovative system for data access and analysis paired with robust support for scientists,” says Melissa Graham, lead community scientist for the Rubin Observatory and research scientist in the astronomy department at the University of Washington. “I’m personally excited to enhance my own skills by using the RSP’s tools for big data analysis, while also helping others to learn and to pursue their LSST-related science goals during Data Preview 0.” 

At the same time, the fact that the RSP is hosted in the cloud provides researchers at smaller institutions access to state-of-the-art astronomy infrastructure that is comparable to that of the largest national research centers.

The launch benefits the observatory too: the development team can learn what researchers are interested in while also testing and debugging the platform. Graham says that “the platform is still in active development so researchers using it will be able to follow along in the progress, and provide feedback on ways that we can optimize the development of the tools.”

Next steps

The LSST aims to begin the ten-year survey in 2023-24 and expects it to include 500 petabytes of data. Through the cloud, Google aims to help make this extraordinary project scalable and accessible to researchers everywhere. To learn more about Data Preview 0, watch this video.

Want to ramp up your own research in the cloud? We offer research credits to academics using Google Cloud for qualifying projects in eligible countries. You can find our application form on Google Cloud’s website or contact our sales team.

Editor’s note: Engineers from SoFi Stadium, Google Cloud, and Deloitte built a fan-ready Personal Concierge app that uses advanced analytics to tailor game-day experiences for every visitor. SoFi’s 10-year partnership with Google Cloud not only pumps up the fans. Advanced analytics also enable stadium employees to securely store, analyze, and action analytics data, and seamlessly collaborate through tools and services included in Google Cloud Workspace.

SoFi Stadium, an industry-leading, outdoor-indoor events facility and home of the Los Angeles Rams and Los Angeles Chargers, is the centerpiece of Hollywood Park, a near 300-acre sports and entertainment destination. With the capacity to host up to 100,000 spectators across a range of luxury and premium seating options, the 3.1 million square-foot SoFi Stadium is the largest in the NFL. Completed in September 2020, it has already secured a global reputation—scheduled to host Super Bowl LVI in 2022, the College Football National Championship Game in 2023, and the Opening and Closing Ceremonies of the Olympic Games in 2028. Under the same roof canopy as the stadium is the 2.5-acre American Airlines Plaza and the 6,000-seat performance venue, YouTube Theater. 

Through a long-term partnership with Google Cloud, SoFi Stadium is personalizing fan experiences, enabling a more fulfilling, collaborative, and productive work experience for employees, and delivering game-changing technology to the event space.

Personalizing the experience for every fan 

Sports fans are known for their fierce loyalty and camaraderie. While they come to game day ready to cheer with fellow fans, each spectator has a different vision for what will make for a memorable event. SoFi Stadium is helping customers choose their game day flow by extending a personalized, streamlined experience to every stadium goer through several new digital innovations powered by Google Cloud.

First there’s the Personal Concierge App. Powered by Google Cloud and built by Deloitte, this personalized navigation and security tool may be the game day MVP that sports fans have always dreamed of. Hours of precious viewing time can be lost navigating a stadium campus for seating, parking, food, access, and amenities. This application enables fans to quickly locate available parking locations, select the ideal transportation for game day through real-time Google Maps routing, and discover SoFi Stadium events all from their mobile device. Event-goers can also receive digital credentials through the app, enabling them to bypass lines and accelerate their access to luxury viewing suites. 

The advanced data analytics accrued from multiple sources and processed through Google Cloud products such as Cloud Scheduler, App Engine, Dataflow, BigQuery, Looker, Firebase, and more, enable SoFi Stadium to anticipate spectator tastes. These Google Cloud tools can remember preferences and provide attendees with real-time personal recommendations around the clock. 

Performing at the top of their game with Workspace

Advanced data analytics enhance employee experiences, too. Credentialed team members can securely store and analyze data from multiple sources, create operational dashboards with real-time data, track key performance indicators after events, and better forecast attendance, revenue by product, and other important business metrics.

In addition to deploying a number of Google Cloud solutions including Compute Engine, Kubernetes Engine, Cloud CDN, BigQuery, and others to power the Personal Concierge App, SoFi Stadium is empowering its employees by fully migrating to Google Cloud’s Workspace. Workspace streamlines the stadium’s business operations by fostering fast and seamless access to collaboration and performance tools. Built-in software—including Gmail for business, Docs, Drive, Calendar, and Meet—will help employees work smarter and meet securely from anywhere. 

“We couldn’t have picked a better partner to help manage our technology needs,” said Skarpi Hedinsson, Chief Technology Officer, SoFi Stadium and Hollywood Park. “Incorporating Google’s solutions into critical areas of our business ensures we can usher in a new era of innovation in sports and entertainment, raising the bar on what the ultimate fan experience looks like globally.”

Making the big-game even bigger

As Hollywood Park combines the expertise of its team and Google Cloud engineers to power the future of sports and live events, SoFi Stadium prepares to welcome a product of this collaboration with the Infinity Screen by Samsung, the largest video board in sports. With 80 million pixels, weighing 2.2 million pounds, and boasting 120-yards in length, the Infinity Screen by Samsung leverages Google Cloud-based media workflows and optimization technology to power its game-changing screen. 

Through its partnership with Google Cloud, SoFi Stadium is actively developing new media streaming and high-speed wireless solutions. In addition, SoFi Stadium has the world’s first 4K/HDR live production facility. These digital innovations are powering a new standard for sports fans and opening a path to redefine what can take game day from good to great.

NBP, Pakistan’s largest government-owned bank, serves private and commercial customers and also acts as the government treasury bank. This means that it handles all government transactions—including disbursements and cash collection. In the past, every government transaction had to be handled physically through the NBP branch network. But in a populous country like Pakistan, managing the huge volume of financial transactions is a big task, especially if a single bank is the only conduit. While we’re still in the early stages, here are a few ways that we’re actively working to find solutions that overcome these challenges. 

Using APIs to increase access
Our digital banking implementation team, which includes product developers and a small in-house think tank, is responsible for developing new technology and out-of-the-box solutions tailored to the requirements of different areas of the bank. Our implementation partner Abacus played a key role in evaluating and implementing Apigee for NBP.

Recently, we developed a plan to open up the NBP government mandate to other banks and third-party fintech partners. Under this new model, instead of relying solely on our own channels, customers can now transact through fintech apps and other approved Pakistani banks. To be able to roll out a solution that would be reliable, scalable, and secure enough to meet our needs, we adopted Google Cloud’s Apigee API management platform. This platform allows us to accelerate our product development, so that we can compete in the fintech space. It also gives our customers access to a wide range of banking services through our own and partner channels.

As a government bank, NBP has to deal with a lot of procedural hurdles, which have slowed down our entry into the fintech market. Additionally, legacy systems required us to develop solutions for each particular channel and use case. APIs and API management increase the reusability of our services, while also bringing ease and speed of getting our services to market. In fact, we’ve seen the time it takes to offer a new solution reduced by 20%. Apigee not only helps us to achieve our go-to-market goals—it enhances our capacity to capture new and unique use cases across multiple channels. We appreciate the speed, agility, and security that Apigee brings us, along with its many out-of-the-box features. 

Reducing barriers to consumer services
One example of how we’re using Apigee is our passport collection use case. In Pakistan, to obtain a passport, citizens have to visit an NBP branch. This can involve waiting in long lines to deposit the government-required passport insurance fee. The sheer volume of these transactions was overwhelming our local branch teams, and costs for this type of manual transaction were high. Furthermore, the government had problems reconciling the fee collection with the passport transactions. 

To address these concerns, we developed an API that allows not only NBP branches, but also third-party banks and fintechs, to accept passport issuance fees.Customers can now visit the bank or fintech provider of their choice, reducing the load on NBP, while the government passport department inspectors can now easily reconcile these transactions. 

We also developed a bill payment solution with Apigee that gives customers the possibility of paying utility bills online. Previously, the NBP process was inefficient. We had an in-person bill payment mechanism operating in our 1,500+ bank branches. Now, we’ve integrated the payment API with the branch channel so that bill payment is automated, whether it takes place in a branch or online. 

Increasing the API footprint inside and outside the bank
Now that we’ve implemented the Apigee platform, our partner ecosystem is benefiting from it and growing, too. We’ve enrolled a wide range of fintechs that are developing products in the corporate payment space. We’re also planning to partner with several incubation centers to provide our APIs via sandbox environments, once our Apigee developer portal goes live. Other fintechs will be enrolled through the incubation centers as well. Finally, alternative third-party partners, such as fintechs and banks, will consume our APIs and/or partner with us for product development.

We’ve currently published 10 APIs and by the end of this year, we expect to have 25 to 30 live. We’re looking forward to implementing monetization, but not necessarily with revenue as the primary focus. As NBP is a government-owned bank, we have a responsibility to act as a catalyst for smaller players, and we see monetization expertise as an opportunity for fulfilling our mandate to help fintech startups grow and mature. In the long run, API revenues will certainly become more important, but the short-term goal is seeding innovation in the marketplace and providing the best possible retail banking experience for our customers, nationwide.

Secret Manager is a Google Cloud service that provides a secure and convenient way to store API keys, passwords, certificates, and other sensitive data. It is the central place and single source of truth to manage, access, and audit secrets across Google Cloud. Since its launch, Secret Manager has helped secure millions of workloads and continues to provide industry-first features like replication policies and support for VPC perimeters. This blog post explores new Secret Manager capabilities and integrations that will help keep your secrets safer.

New tier, free of charge

No, you’re not dreaming – Secret Manager now has a tier that is free of charge! With this tier, each month per billing account you can have up to:

• 6 secret versions
• 3 rotation events
• 10,000 API calls

This enables you to experience the value of Secret Manager with minimal financial risk and pairs nicely with existing services that offer a free tier like Cloud Run and Cloud Functions. For existing Secret Manager customers, this change will go into effect next billing cycle. Learn more about the Secret Manager free tier in the documentation.

Increased SLA

To meet the growing availability and reliability requirements of our customers, the Secret Manager SLA is now 99.95%! With this update, Secret Manager guarantees that all valid requests will succeed 99.95% of the time. This means you can depend on Secret Manager for even your most critical workloads. Additional details are available in the updated Secret Manager SLA.

Geo-expansion

In addition to the free tier and increased SLA, Secret Manager is now available in all public Google Cloud regions! With Secret Manager’s replication policies, you can choose the specific regions in which to replicate your secret, which means you can store secret payloads in geographical proximity to your workloads or users to reduce latency. This is also very useful if you have legal or regulatory requirements to store data in a particular locality. For more information, check out the list of Secret Manager locations in the documentation.

Compliance certifications

For customers wishing to use Secret Manager to store and process regulated data, Secret Manager is validated for compliance use cases including ISO 27001, ISO 27017, ISO 27018, SOC 1, SOC 2, SOC 3, PCI DSS, and HIPAA. Combined with the increased SLA and geographical availability, this makes Secret Manager suitable for use with regulated workloads.

Customer-Managed Encryption Keys (CMEK)

Secret Manager has always encrypted payloads in transit with TLS and at rest with AES-256. For customers that want additional control over the keys used to encrypt their secret payloads, Secret Manager now supports Customer-Managed Encryption Keys (CMEK). Secret Manager CMEK supports software-backed keys via Cloud KMS, hardware-backed keys via Cloud HSM, and even externally-managed keys via Cloud EKM. Learn how to enable CMEK support for Secret Manager in our tutorial.

Expiration and TTLs

While it was previously possible to expire access to a secret using IAM conditions, the underlying secret would continue to exist. Secret Manager now supports auto-expiring secrets which permanently deletes a secret at a specified timestamp or TTL. Since it is also possible to update a secret’s TTL, services can “lease” a secret and renew their lease on a periodic basis. If the service does not extend the lease by updating the TTL, the secret is automatically deleted.

Expiring secrets can be used in combination with IAM conditions to more safely expire secrets. For more information on expiring secrets and safety measures, see the guide on creating and managing expiring secrets.

Etags and server-side filtering

For customers that create or manage Secret Manager secrets via the API or an SDK, concurrency controls and performance are extremely important. This is why Secret Manager now supports Etags and server-side filtering! Etags help prevent concurrent modifications to the same secret by providing optimistic concurrency controls, while server-side filtering can dramatically reduce payload size and client-side computational overhead. Together, these enable stronger consistency guarantees and performance improvements to your applications. Learn more about Secret Manager Etags and Secret Manager server-side filtering in the documentation.

Code, build, run, deploy, monitor, and orchestrate

Secrets – like API keys, passwords, and certificates – are an integral part of most modern software applications. It is crucial that developers, operators, and security teams are empowered to build, operate, and observe software securely. That is why Secret Manager is now integrated with popular tools and technologies used throughout the application development lifecycle:

• Code – Software engineers can create and access secrets directly from their preferred IDEs with Cloud Code. In VS Code, IntelliJ, or the Cloud Shell Editor, developers can browse secrets and insert code snippets for access secrets, all from the comfort of their local IDE.
• Build – Release engineers can access secrets as part of CI builds using the Cloud Build Secret Manager integration. This could be used, for example, to authenticate to a Docker registry or communicate with the GitHub API. For customers that use other CI systems, there is also a GitHub Action for accessing Secret Manager secrets.
• Run (on serverless) – Developers can mount secrets to be available as environment variables or via the filesystem through the native Cloud Run Secret Manager integration. Since the secrets are resolved in Cloud Run’s control plane, developers can use this integration to avoid a tight coupling between their applications and Secret Manager to enable hybrid cloud deployments or better local development experiences.
• Run (on Kubernetes) – Developers can mount secrets from GKE, Anthos, or any Kubernetes cluster using the Secret Manager CSI driver. This vendor-agnostic driver exposes secrets via environment variables or the filesystem and enables hybrid cloud deployments using the same interface as other public cloud providers and HashiCorp Vault.
• Deploy – To complement the existing Secret Manager Terraform integration, operators can now manage Secret Manager via Kubernetes Config Connector (KCC). KCC allows operators to manage Google Cloud resources through Kubernetes and the familiar Kubernetes APIs.
• Monitor – With the Secret Manager Cloud Asset Inventory (CAIS) integration, security teams can understand secret usage across specific projects, folders, or the entire organization.
• Orchestrate – Secret Manager Event Notifications enable DevOps and security teams to subscribe to Pub/Sub topics for when secrets or secret versions are changed. This enables customers to create deeply-integrated workflows, such as creating a ServiceNow ticket when a new secret version is added. Additionally, Secret Manager Rotation Scheduling enables DevOps teams to build automatic rotation flows like the ones described in the rotation guide.

Best practices

The Secret Manager best practices guide ensures customers get the maximum security benefits from Secret Manager. Security is non-binary, and this guide covers nuanced topics like access controls, coding practices, and secret administration. While not an exhaustive list, the Secret Manager best practices guide answers some of the most common questions and concerns around using Secret Manager in production deployments.

Towards seamless security

Secrets management is an important part of every organization’s security toolkit. With Secret Manager, you can easily manage, audit, and access secrets like API keys and credentials across Google Cloud, Anthos, and on-premises. These new features and integrations make it easy to adopt Secret Manager whether you are a hobbyist working on a side project or a large enterprise with thousands of employees.

To get started, check out the Secret Manager documentation.