Incorporating Custom Holidays into Your Time-Series Models with BigQuery ML
1114
Of your peers have already read this article.
4:30 Minutes
The most insightful time you'll spend today!
About three years ago, JCB, one of the biggest Japanese payment companies, launched a project to develop new high-value services with agility. We set up a policy of starting small from scratch without using the existing system, which we call the concept of “Dejima”, where we focused on improving various aspects such as team structure, risk management, and application and platform development process.
Until now, large Japanese enterprises have built decision-making systems focused on eliminating unnecessary business processes and efficiently increasing quarterly profits. As a result, we are seeing more organizational structures that make it difficult to take on new challenges or experiments with trial and error. We wanted to breathe a new life into this situation, and that is how the concept of Dejima came up. In the Edo period, Japan closed its national border to other countries under its national isolation policy. At the time, Dejima was the only area where special rules were applied and allowed people from different cultures to come and go, and trade. This special rule generated the culture of inclusion and led to Dejima’s prosperity. Like Dejima, we believe that creating an organization that is independent from other business practices can be effective in enabling digital transformation for the organization.
We have been able to make this transformation with the direct help of the Google Cloud and its products such as Google Kubernetes Engine (GKE), Cloud Spanner and Anthos Service Mesh, applying domain-driven design and microservice architecture. We named this the “JCB Digital Enablement Platform (JDEP),” which now hosts multiple business critical production services.
A key benefit of GKE is that the team can easily add resources and release them when they are finished, allowing them to be flexible to accommodate busy periods and off-seasons. Meanwhile, Anthos Service Mesh helps us manage complex environments easily. With containerization and managed services, we are prepared for the future for when more services go into production, as it would be easy to maintain and provide version upgrade support. At the same time, Cloud Spanner ensures that we maintain a 99.99% availability at all times.
Our initial motivation for introducing SRE practices was to break proverbial walls between business, development and operations, which was a success. Now we are focused on ensuring its reliability and maintaining customer satisfaction with our SRE practices.
To ensure the success of SRE practices that we implemented, there were a few categories we needed to address, from defining the organizational culture and practices to ensuring the policies attached to the new models created were practical enough to be implemented on the ground level. This is so that the Dejima concept remains sustainable for the long run.
Instilling a culture of measurement
Here, “appropriate” reliability is the key. According to the conventional way of thinking at JCB, “service failure must not occur” and “SLA should be maintained as high as possible.” We started by discussing what was the “appropriate reliability” that our customers really needed, but it was not as easy as we thought because the level of reliability for user satisfaction differed from application to application.
Eventually, the business, development and operations teams formulated specific SLIs and SLOs together, something we would never have been able to do if we discussed separately. This is because the business is required to compromise on lower service levels, since our reliability standard used to be too high. The collaboration of development and operations teams is necessary to understand how our system works upon our users’ interactions.
After Google Cloud helped us run a series of workshops where all teams participated, we saw change within the organization. The business team started evangelizing SRE to other members in the business department, and the development and operations teams started collaborating autonomously. We felt like we were working at Google speed, accomplishing so much in a short amount of time.
Understanding SRE as an entire company is necessary to progress. We are now working on creating internal training materials to spread the SRE concept throughout the company.
Eliminating ambiguity
With the cooperation of Google Cloud, we have created a Team Charter that defines the team’s mission, values and engagement models. We also created policy documents that include Incident Response Policy, Postmortem Policy, On-call Policy, Toil Policy and Error Budget Policy, to eliminate ambiguity in day-to-day operations.
For example, when an incident occurs, we can identify exactly the level of importance, the roles that are assigned to each person, and in what order they need to follow. When to do a postmortem, who owns it? What to do if the error budget is exhausted? How do other teams reach out to SRE when they have problems? The written policy documents will dramatically improve efficiency and motivate teams to adopt a culture of learning from failures.
The format for such policies are written in Google’s SRE book, but when we adopt it, it needs to take into account the circumstances specific to our company. Simply copying an existing policy won’t work, which is why it’s important to formulate a policy that fits the situation each team is in.
Reformalizing teams
Based on these policies, JCB’s SRE team has two sub-teams. One is called Sheriff which works as the platform SRE, and provides infrastructure services for the application team. The other is called the Diplomat which works as the embedded SRE, and participates in the application team to lead productionisation. There is also a team called Architecture that is separate from the SRE teams whose role is to consult SRE on system design and review architecture.
The SRE team was a single role when it was first launched, but now has two sub-teams. This is because as the number of application teams increases, the number of support tasks for the teams also increases, which can result in a shortage of resources to work on overall improvement. Securing people who are not interrupted from day-to-day support tasks and focusing on the main task improves efficiency.
Whereas both sub-teams share the on-call duty, some engineers are not allowed to do it by contract as they are not allowed to get paged. For those who cannot participate in on-call duties, we created what’s called a Toil Shift, which allows them to focus on resolving tickets in our backlogs instead.
This works well so far, but we will keep evolving as our business grows.
11209
Of your peers have already watched this video.
1:30 Minutes
The most insightful time you'll spend today!
Indian Retailer Figures Optimizes Hyperlocal Delivery to Increase Customer Experience
Anyone who follows the Indian e-commerce scene knows that one of the largest challenges these companies face is hyperlocal delivery.
That was a problem facing Wellness Forever, a retail chain of pharmacies with 150-plus stores across India.
“Exactly a year ago, we started our journey of hyperlocal deliveries. This optimization was a big time challenge for us to understand how to optimize this,” Palani Subbiah, CTO, Wellness Forever.
The problem in front of Wellness Forever was to identify which customer could can be sold from which store, so that a delivery could be made within 90 minutes.
“We handle a large amount of customer data and we wanted to use insights to help and improve the customer satisfaction index,” says Subbiah.
To do that Wellness Forever leveraged Google Big Query to run massive amount of data to come up with the operational insights. They also used Firebase and Google Maps.
“By 2021, we are going to have about 450 stores. Those stores are going to be not only a physical store, which is a digital store.
Google Unveils Topaz, the New Subsea Cable Connecting Asia and Canada
3317
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
There’s a new subsea cable in town: Topaz, the first-ever fiber cable to connect Canada and Asia.
Once complete, Topaz will run from Vancouver to the small town of Port Alberni on the west coast of Vancouver Island in British Columbia, and across the Pacific Ocean to the prefectures of Mie and Ibaraki in Japan. We expect the cable to be ready for service in 2023, not only delivering low-latency access to Search, Gmail and YouTube, Google Cloud, and other Google services, but also increasing capacity to the region for a variety of network operators in both Japan and Canada.
Google is spearheading construction of the project, joined by a number of local partners in Japan and Canada to deliver the full Topaz subsea cable system. Other networks and internet service providers will be able to benefit from the cable’s additional capacity, whether for their own use or to provide to third parties. And, similar to other cables we’ve built, with Topaz we will exchange fiber pairs with partners who have systems along similar routes. This is a longstanding practice in the industry that strengthens the intercontinental network lattice for network operators, for Google, and for users around the world.
Network infrastructure investments like Topaz bring significant economic activity to the regions where they land. For example, according to a recent Analysys Mason study, Google’s historical and future network infrastructure investments in Japan are forecasted to enable an additional $303 billion (USD) in GDP cumulatively between 2022 and 2026.
The width of a garden hose, the Topaz cable will house 16 fiber pairs, for a total capacity of 240 Terabits per second (not to be confused with TSPs). It includes support for Wavelength Selective Switch (WSS), an efficient and software-defined way to carve up the spectrum on an optical fiber pair for flexibility in routing and advanced resilience. We’re proud to bring WSS to Topaz and to see the technology is being implemented widely across the submarine cable industry.
While Topaz is the first trans-Pacific fiber cable to land on the West Coast of Canada, it’s not the first communication cable to connect to Vancouver Island. In the 1960s, the Commonwealth Pacific Cable System (COMPAC) was a copper undersea cable linking Vancouver with Honolulu (United States), Sydney (Australia), and Auckland (New Zealand), expanding high-quality international phone connectivity. Today, COMPAC is no longer in service but its legacy lives on. The original cable landing station in Vancouver — the facility where COMPAC made landfall on Canadian soil — has been upgraded to fit the needs of modern fiber optics and will house the eastern end of the Topaz cable.
Traditional and treaty rights, and local communities, are deeply important to our infrastructure projects. The Topaz cable is built alongside the traditional territories of the Hupacasath, Maa-nulth, and Tseshaht, and we have consulted with and partnered with these First Nations every step of the way.
“Tseshaht is very proud of this collaboration and our partnership with Google, who has been very respectful and thoughtful in its engagement with our Nation. That’s how we carry ourselves and that’s how we want business to carry themselves in our territory.“ — Tseshaht First Nation – Elected Chief Councillor-Ken Watts
“The five First Nations of the Maa-nulth Treaty Society are pleased that we have concluded an agreement with Google Canada and have consented to the installation of a new, high-speed fiber optic cable through our traditional territories. This agreement, in which both Google Canada and our Nations benefit, is based on respect for our constitutionally protected treaty and aboriginal rights and enhances the process of reconciliation. We would also like to acknowledge the sensitivity that Google Canada expressed during our talks in regard to the pain and trauma experienced by our people as a result of residential school experience. We look forward to a long and mutually beneficial relationship with Google Canada.” —Chief Charlie Cootes, President of the Maa-nulth Treaty Society
“Google’s respect towards our Nation is appreciated and has good energy behind it.” —Hupacasath First Nation – Elected Chief Councilor – Brandy Lauder
With the addition of Topaz today, we have announced investments in 20 subsea cable projects. This includes Curie, Dunant, Equiano, Firmina and Grace Hopper, and consortium cables like Blue, Echo, Havfrue and Raman — all connecting 29 cloud regions, 88 zones, 146 network edge locations across more than 200 countries and territories. Learn about Google Cloud’s network and infrastructure on our website and in the below video.
The Cloud-First Imperative to Accelerate Digital Transformation in BFSI
DOWNLOAD WHITEPAPER3374
Of your peers have already downloaded this article
1:30 Minutes
The most insightful time you'll spend today!
In a study of 60 business and technology decision makers and influencers from Indian BFSI organizations, conducted by Forrester Consulting, it is revealed that public cloud adoption in India is on the rise with 78% of the organizations intending to increase the number of secure or compliance-sensitive workloads in the cloud.
According to the study, public cloud platforms help BFSI organizations to:
- Improve customer experience
- Free up IT time to focus on core differentiators instead of commodity workloads
- Reduce reliance on legacy infrastructure
- Help to scale business faster and more easily
Download this infographic to understand why Indian BFSI organizations are adopting a cloud-first approach to accelerate digital transformation.
Five Ways Cloud Computing Helps Companies Optimize Operations and Lower IT Costs
4793
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
There is no one way to recover. The past year has seen unprecedented challenges for business across the world, with social distancing and quarantine measures forcing many organizations to quickly adapt to remote working. A new report from BCG Platinion, titled “Finding a New Normal in the Cloud”, points out that while companies have done well to survive so far, the overall business environment is still a tough one with contracted economies and lowered revenues. CIOs have to find the right balance between technological innovation and lowering their cash burn rate.
The report identifies five key ways in which companies can use cloud computing to optimize their operations and reduce overall IT costs by as much as 10%.
1. Go beyond VPN
The BCG Platinion report states that companies need “rapid, efficient, highly scalable, and device agnostic solutions”. Traditionally, when employees had to work offsite, companies provided them with VPNs. But the sheer scale of the COVID-19 pandemic has shown these solutions to be expensive, slow, inconvenient, and hard to manage when entire workforces are working remotely.
Instead, BCG Platinion emphasizes the need for cloud-based solutions such as BeyondCorp Enterprise for more secure, more effective, lower cost remote access at scale. BeyondCorp Enterprise offers customers a zero trust platform for simple and secure access with continuous end-to-end protection that can be used on any device at any time. Deliveroo, a global food delivery company headquartered in the UK, uses BeyondCorp Enterprise to bring the zero trust model to its distributed workforce.
“Having secure access to applications and associated data is critical for our business,” says Vaughn Washington, VP of Engineering at Deliveroo. “With BeyondCorp Enterprise, we manage security at the app level, which removes the need for traditional VPNs and associated risks.”1
With a low cost cloud subscription model, BeyondCorp Enterprise eliminates the need for hardware, operating, and maintenance costs that come with VPN solutions and can also enable organizations to offer protection to the extended workforce at a fraction of the cost. BCG Platinion estimates that solutions like BeyondCorp Enterprise can save companies as much as 50% versus the cost of a traditional VPN.
2. Use SaaS to keep productivity up
With so much of the workforce fragmented due to social distancing measures, the need for seamless, efficient collaboration is greater than ever. According to BCG Platinion, a fully functional Software-as-a-Service productivity solution such as Google Workspace helps to “alleviate the traditional costs and burdens around availability, backup, and maintenance of on-premise collaboration infrastructure.” BCG Platinion analysts estimate that adopting a SaaS solution can lower computing costs for end users by up to 35%.
But working in the cloud does more than lower costs. The BCG Platinion report cites a 2020 study from Forrester that found adopting Google Workspace boosted revenue growth by 1.5%, reduced the need for on-demand tech support by 20%, and cut the risk of data breaches by more than 95%. Over the last year, companies of all sizes have looked on the conditions of the pandemic as an opportunity to change the way they work.
“Airbus has spent the past year thinking about what it actually means to return to work and we’re looking to support greater flexibility with Google Workspace in a leading role,” says Andrew Plunkett, Airbus Vice President Digital Workplace. “In 2020, we held 5.6M Google Meet sessions and we now have more than 70,000 shared Drives where people collaborate. Google Workspace has changed the way people work at Airbus and that will continue as the solution empowers the hybrid work reality.”2
3. Reduce IT overhead and management costs with cloud-first devices
Working in the cloud can be made even more effective with devices specifically designed for the cloud, says the BCG Platinion report. “Cloud-native devices such as Google’s Chromebooks and Chromeboxes are cost-effective, easy to deploy, simple to use, and highly secure,” says BCG Platinion. Additionally, with “thin client” devices, companies can save on hardware costs compared with traditional laptops and desktops.The report suggests that a thin client approach can produce savings of up to 25% in end-user technology and support.
Organizations and businesses of all sizes have found that Chrome OS and devices have greatly enhanced their capacity to work together in even the most challenging circumstances. Chrome OS provides employees with a modern experience and devices that stay fast, have built-in security, deploy quickly, and reduce the total cost of ownership. “Chromebooks are simple-to-use and cost-effective devices that do everything that our staff need them to do, which is mainly accessing Google Workspace online,” says Henry Lewis, Head of Platform for the London Borough of Hackney. “As soon as the Grab and Go Chromebooks were available, they were well used every day.”3
4. Lift and shift for easier transitions
While migrating to the cloud is a priority for many organizations to succeed, it does not have to happen all at once and in the same way. A total cloud migration can be a huge undertaking, involving redesigns of architecture and refactoring of applications. For many organizations, this process can take several months, even years.
But in times like these, CIOs need to make decisions quickly and reduce cash burn as much as possible. When resources are stretched and time is tight, BCG Platinion recommends a “lift-and-shift” approach for minimal disruption. With Google Compute Engine, for example, organizations can simply rehost their existing workloads on virtual machines without transformation. BCG Platinion reports that moving non-critical workloads to the cloud with lift-and-shift approaches can reduce IT spend by as much as 4% in just three months. Additionally, a quick, effective migration paves the way for more advanced IT infrastructure changes, creating effects that ripple long into the future.
5. Make data work for you with the cloud
Data is central to any industry and making the most out of it is more important now than ever before. With business as usual upended by the pandemic, organizations have turned to data for urgent tasks like demand forecasting or predicting supply-chain disruptions. A McKinsey report from last year points out that while many organisations had already started to engage with analytics and AI technologies, their progress was dramatically accelerated by the urgency of the pandemic: “Analytics capabilities that once might have taken these organizations months or years to build came to life in a matter of weeks.”
With the right setup a company can use data to drive efficiencies, respond quickly to its customers and open new markets. But big wins require huge amounts of information and powerful analytics, which means that effective data handling can be very difficult and expensive to do with on-premises architecture.
Moving to a cloud-based infrastructure minimizes infrastructure costs while opening up cutting edge techniques like machine learning for greater insight. The BCG Platinion report found that using a cloud-based data platform was not only cheaper and more efficient for businesses, but could result in a 70% increase in “effectiveness” which it defined as increased sales, lower costs of procured goods, and reduced inventory holding costs. “Moving to cloud provides competitive advantage as you scale innovation, accelerating the creation of new services to keep ahead of the competition,” explains Norbert Faure, Managing Director, Platinion Western Europe at Boston Consulting Group (BCG).
The key mission of any data platform is to make sure that the right people have access to the right information at the right time. Google Cloud helps to unify data across the entire business, increase agility, and innovate faster with a range of products. BigQuery runs complex analytics at infinite scale while helping organizations save up to 34% on the total cost of ownership compared with alternative cloud-based data warehouse solutions.4 Cloud Spanner provides a fully managed relational database that operates at 99.999% availability. Meanwhile, with Vertex AI, businesses can access the same groundbreaking machine learning tools that Google uses itself for unprecedented insight on a unified AI platform. “The whole Google Cloud suite of products accelerates the process of getting established and up and running, so we can perform the ‘bread-and-butter’ work of data science,” says David Herberich, VP, Head of Data at fintech startup Branch.5
Minimize costs today, innovate for tomorrow
As the world recovers from the pandemic, continued success depends on staying nimble and adaptive, argues the BCG Platinion report. Cloud computing can make companies more resilient by helping them to keep costs low, reducing overall IT spend by as much as 10% overall according to BCG Platinion. “For CIOs, cloud migrations can offer important near-term savings and benefits, while also reinvigorating progress toward the long-term goal of digital transformation.“
To learn more, read the full report from BCG Platinion.
1. BeyondCorp Enterprise: Introducing a safer era of computing
2. Building the future of work with Google Workspace
3. Hackney Council: Empowering 4,000 staff to keep serving their community from home
4. The Economic Advantages of Google BigQuery versus Alternative Cloud-based EDW Solutions
5. Fintech startup, Branch makes data analytics easy with BigQuery
Indian social media platform ShareChat saw a 50% reduction in costs after migrating to Google Cloud
4346
Of your peers have already read this article.
6:30 Minutes
The most insightful time you'll spend today!
One of the key trends that emerged from the lockdown brought about by the COVID-19 outbreak was the significant rise in people seeking entertainment online.
According to data released by Carat India, smartphone usage increased by 1.5 hours a week and social media consumption nearly doubled to 280 minutes a day. Of India’s 670 million internet users, a significant percentage live in rural areas, which saw a surge in demand for local language content.
As India’s premier social media platform, ShareChat saw significant engagement from its 60 million monthly active users (MAU) during the lockdown. The platform now has surpassed 120 million MAU post the government banning 59 Chinese apps in the last week of June.
ShareChat was founded by Ankush Sachdeva (co-founder and Chief Executive Officer), Farid Ahsan (co-founder and Chief Operating Officer), and Bhanu Pratap Singh (co-founder and Chief Technical Officer), in 2015.
In a recent conversation with YourStory, Bhanu spoke about content consumption during COVID-19, how they are reaching out to the country’s vernacular audience and their recent transition to Google Cloud.
Read the Full Story on YourStory
More Relevant Stories for Your Company
2022 is a Big Year for the Gaming Industry!
Editor’s note: This post was originally published in TechPulse Belgium, where Jack Buser, Google Cloud’s Director of Game Industry Solutions, shared his trends for the industry this year. The year 2022 will hold many surprises (with a few already dropping!), but there's one near certainty: By this time next year there will
Synopsys & Google Cloud: Helping Semiconductor Companies Drive Electronic Design Automation Innovation
Google Cloud and Synopsys Inc are partnering to help semiconductor companies drive Electronic Design Automation (EDA) innovation in the cloud to accelerate time to market, and lower costs across the entire semiconductor product life cycle. Synopsys is the industry’s largest provider of EDA technology used in the design and verification
Strategic Consulting, Training & Implementation Guidelines with Public Sector PSO Helps Governments Stay Compliant
Did you know that by 2025, enterprise IT spending on public cloud computing will overtake traditional IT spending? In fact, 51% of IT spend in application software, infrastructure software, business process services, and system infrastructure will transition to the public cloud, compared to 41% in 20221.. As enterprises continue to
Transform Your Business: Comprehensive Cloud Services and Tailored Pricing Plans
As the saying goes, “it’s hard to make predictions, especially about the future.” Some organizations find it challenging to predict what cloud resources they’ll need in months or years ahead. Every organization is on its own unique cloud journey. To help, we’re developing new ways for customers to consume and
