Principles to Make Organizations Data Engineering Driven

4843
Of your peers have already read this article.
4:00 Minutes
The most insightful time you'll spend today!
In the “What type of data processing organisation” paper, we examined that you can build a data culture whether your organization consists mostly of data analysts, or data engineers, or data scientists. However, the path and technologies to become a data-driven innovator are different and success comes from implementing the right tech in a way that matches a company’s culture. In this blog we will expand the data engineering driven organizations and provide how it can be built from the first principles.
Not all organizations are alike. All companies have similar functions (sales, engineering, marketing), but not all functions have the same influence on the overall business decisions. Some companies are more engineering-driven, others are sales-driven, others are marketing-driven. In practice, all companies are a mixture of all these functions. In the same way, the data strategy can be more focused on data analysts, and others on data engineering. Culture is a combination of several factors, business requirements, organizational culture, and skills within the organization.
Traditionally organizations that focused on engineering mainly came from technology driven digital backgrounds. They built their own frameworks or used programming frameworks to build repeatable data pipelines. Some of this is due to the way the data is received, the shape the data is received and the speed of the data arrival as well. If your data allows it, your organization can be more focused on data analysis, and not so much on data engineering. If you can apply an Extract-Load-Transform approach (ELT) rather than the classic Extract-Transform-Load (ETL), then you can focus on data analysis and might not need extensive data engineering capability. For example, data that can be loaded directly into the data warehouse allows data analysts to also do data engineering work and apply transformations to the data.
This does not happen so often though. Sometimes your data is messy, inconsistent, bulky, and encoded in legacy file formats or as part of legacy databases or systems, with a little potential to be actionable by data analysts.
Or maybe you need to process data in streaming, applying complex event processing to obtain competitive insights in near real time. The value of data decays exponentially with time. Most companies can process data by the next day in batch mode. However, not so many are probably obtaining such knowledge the next second data is produced.
In these situations, you need the talent to unveil the insights hidden in that amalgam of data, either messy or fast changing (or both!). And almost as importantly, you need the right tools and systems to enable that talent too.
What are those right tools? Cloud provides the scalability and flexibility for data workloads that are required in such complex situations. Long gone are the times when data teams had to beg for the resources that were required to have an impact in the business. Data processing systems are no longer scarce, so your data strategy should not generate that scarcity artificially.
In this article, we explain how to leverage Google Cloud to enable data teams to do complex processing of data, in batch and streaming. By doing so, your data engineering and science teams can have an impact when (in seconds) after the input data is generated.
Data engineering driven organizations
When the complexity of your data transformation needs is high, data engineers have a central role in the data strategy of your company, leading to data engineering driven organization. In this type of organization, data architectures are organized in three layers: business data owners, data engineers, and data consumers.
Data engineers are at the crossroads between data owners and data consumers, with clear responsibilities:
- Transporting data, enriching data whilst building integrations between analytical systems and operational systems ( as in the real time use cases)
- Parsing and transforming messy data coming from business units into meaningful and clean data, with documented metadata
- Applying DataOps, that is, functional knowledge of the business plus software engineering methodologies applied to the data lifecycle
- Deployment of models and other artifacts analyzing or consuming data

Business data owners are cross-functional domain-oriented teams. These teams know the business in detail, and are the source of data that feeds the data architecture. Sometimes these business units may also have some data-specific roles, such as data analysts, data engineers, or data scientists, to work as interfaces with the rest of the layers. For instance, these teams may design a business data owner, that is the point of contact of a business unit in everything that is related to the data produced by the unit.
At the other end of the architecture, we find the data consumers. Again, also cross-functional, but more focused on extracting insights from the different data available in the architecture. Here we typically find data science teams, data analysts, business intelligence teams, etc. These groups sometimes combine data from different business units, and produce artifacts (machine learning models, interactive dashboards, reports, and so on). For deployment, they require the help of the data engineering team so that data is consistent and trusted.
At the center of these crossroads, we find the data engineering team. Data engineers are responsible for making sure that the data generated and needed by different business units gets ingested into the architecture. This job requires two disparate skills: functional knowledge and data engineering/software development skills. This is often coined under the term DataOps (which evolved from DevOps methodologies developed within the past decades but applied to data engineering practices).
Data engineers have another responsibility too. They must help in the deployment of artifacts produced by the data consumers. Typically, the data consumers do not have the deep technical skills and knowledge to take the sole responsibility for deployment of their artifacts.This is also true for highly sophisticated data science teams. So data engineers must add other skills under their belt: machine learning and business intelligence platform knowledge. Let’s clarify this point, we don’t expect data engineers to become machine learning engineers. Data engineers need to understand ML to ensure that the data delivered to the first layer of a model ( the input ) is correct. They will also become key when delivering that first layer of data in the inference path, as here the data engineering skills around scale / HA etc really need to shine.
By taking the responsibility of parsing and transforming messy data from various business units, or for ingesting in real time, data engineers allow the data consumers to focus on creating value. Data science and other types of data consumers are abstracted away from data encodings, large files, legacy systems, complex message queue configurations for streaming. The benefits of concentrating that knowledge in a highly skilled data engineering team are clear, notwithstanding that other teams (business units and consumers) may also have their data engineers to work as interfaces with other teams. More recently, we even see squads created with members of the business units (data product owners), data engineers, data scientists, and other roles. Effectively creating complete teams with autonomy and full responsibility over a data stream, from the incoming data down to the data driven decision with impact in the business.
Reference architecture – Serverless
The number of skills required for the data engineering team is vast and diverse. We should not make it harder by expecting the team to maintain the infrastructure where they run data pipelines. They should be focusing on how to cleanse, transform, enrich, and prepare the data rather than how much memory or how many cores their solution may require.
The reference architectures presented here are based on the following principles:
- Serverless no-ops technologies
- Streaming-enabled for low time-to-insight
We present different alternatives, based on different products available in Google Cloud:
- Dataflow, the built-in streaming analytics platform in Google Cloud
- Dataproc, the Google Cloud’s managed platform for Hadoop and Spark.
- Data Fusion, a codeless environment for creating and running data pipelines
Let’s dig into these principles.
By using serverless technology we eliminate the maintenance burden from the data engineering team, and we provide the necessary flexibility and scalability for executing complex and/or large jobs. For example, scalability is essential when planning for traffic spikes during mega Friday for retailers. Using serverless solutions allows retailers to look into how they are performing during the day. They no longer need to worry about resources needed to process massive data generated during the day.
The team needs to have full control and write their own code for the data pipelines because of the type of pipelines that the team develops. This is true either for batch or streaming pipelines. In batch, the parsing requirements can be complex and no off the shelf solution works. In streaming, if the team wants to fully leverage the capabilities of the platform, they should implement all the complex business logic that is required, without artificially simplifying the complexity in exchange for some better latency. They can develop a pipeline that achieves a low latency with highly complex business logic. This again requires the team to start writing code from first principles.
However, that the team needs to write code should not imply that they need to rewrite any existing piece of code. For many input/output systems, we can probably reuse code from patterns, snippets, and similar examples. Moreover, a logical pipeline developed by a data engineering team does not necessarily need to map to a physical pipeline. Some parts of the logic can be easily reused by using technologies like Dataflow templates, and use those templates in orchestration with other custom developed pipelines. This brings the best of both worlds (reuse and rewrite), while saving precious time that can be dedicated to higher impact code rather than common I/O tasks. The reference architecture presented has another important feature: the possibility to transform existing batch pipelines to streaming.

The ingestion layer consists of Pub/Sub for real time and Cloud Storage for batch and does not require any preallocated infrastructure. Both Pub/Sub and Cloud Storage can be used for a range of cases as it can automatically scale up with the input workload.
Once the data has been ingested, our proposed architecture follows the classical division in three stages: Extract, Transform, and Load (ETL). For some types of files, direct ingestion into BigQuery (following an ELT approach) is also possible.
In the transform layer, we primarily recommend Dataflow as the data process component. Dataflow uses Apache Beam as SDK. The main advantage of Apache Beam is the unified model for batch and streaming processing. As mentioned before, the same code can be adapted to run in batch or streaming by adapting input and output. For instance, switching the input from files in Cloud Storage to messages published in a topic in Pub/Sub.
One of the alternatives to Dataflow in this architecture is Dataproc, Google Cloud’s solution for managed Hadoop and Spark clusters. The main use case is for those teams that are migrating to Google Cloud but have large amounts of inherited code in Spark or Hadoop. Dataproc enables a direct path to the cloud, without having to review all those pipelines.
Finally, we also present the alternative of Data Fusion, a codeless environment for creating data pipelines using a drag-and-drop interface. Data Fusion actually uses Dataproc as its Compute Engine, so everything we have mentioned earlier applies also to the case of Data Fusion. If your team prefers to create data pipelines without having to write any code, Data Fusion is the right tool.
So in summary, these are the three recommended components for the transform layer:
- Dataflow, powerful and versatile with a unified model for batch and streaming processing. Straightforward path to move from batch processing to streaming
- Dataproc, for those teams that want to reuse existing code from Hadoop or Spark environments.
- Data Fusion, if your team does not want to write any code.
Challenges and opportunities
Data platforms are complex. Having on top of that data responsibility the duty to maintain infrastructure is a wasteful use of valuable skills and talent. Often data teams end up managing infrastructure rather than focusing on analyzing the data. The architecture presented in this article liberates the data engineering team from having to allocate infrastructure and tweak clusters but instead to focus on providing value through data processing pipelines.
For data engineers to focus on what they do best, you need to fully leverage the cloud. A lift & shift approach from any on-premise installation is not going to provide that flexibility and liberation. You need to leverage serverless technologies. As an added advantage, serverless lets you also scale your data processing capabilities with your needs, and be able to respond to peaks of activity, however large these are.
Serverless technologies sometimes face the doubts of practitioners: will I be locked in with my provider if I fully leverage serverless? This is actually a question that you should be asking when deciding whether to set up your architecture on top of a provider.
The components presented here for data processing are based on open source technologies, and fully interoperable with other open source equivalent components. Dataflow uses Apache Beam, which not only unifies batch and streaming, but also offers a widely compatible runner. You can take your code elsewhere to any other runner. For instance, Apache Flink or Apache Spark. Dataproc is a fully managed Hadoop and Spark based on the vanilla open source components of this ecosystem. Data Fusion is actually the Google Cloud version of CDAP, an open source project.
On the other hand, for the serving layer, BigQuery is based on standard Ansi SQL. Whereas in the case of Bigtable and Google Kubernetes Engine, Bigtable is compatible at API level with HBase, and Kubernetes is an open source component.
In summary, when your components are based on open source, like the ones included in this architecture, serverless does not lock you in. The skills required to encode business logic in the form of data processing pipelines are based on engineering principles that remain stable across time. The same principles apply if you are using Hadoop, Spark, or Dataflow or UI driven ETL tooling. In addition, there are now new capabilities, such as low-latency streaming, that were not available before. A team of data engineers that learn the fundamental principles of data engineering will be able to quickly leverage those additional capabilities.
Our recommended architecture separates the logical level, the code of your applications, from the infrastructure where they run. This enables data engineers to focus on what they do best and on where they provide the highest added value. Let your Dataflow and your engineers impact your business, by adopting the technologies that liberate them and allow them to focus on adding business value. To learn more about building an unified data analytics platform, take a look at our recently published Unified Data Analytics Platform paper and Converging Architectures paper.
Qlik and Google Cloud Combo: Extending Integration of SAP Data on BigQuery

5457
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
If your organization is one of the 52% of SAP customers whose top analytics pain point is data integration1, Google Cloud has got you covered. By working with partners like Qlik, we are expanding our integration options and bringing real-time replication capability for SAP to BigQuery.
Integrated data for accelerated insights
BigQuery—our fully managed, enterprise data warehouse that can scale up to petabytes on demand and execute queries in seconds—allows SAP customers to consolidate enterprise data silos and confidently derive more use and value from their data. Customers can accelerate and simplify the delivery of SAP data on BigQuery with the latest Qlik Data Integration platform offering for Google Cloud which allows data integration using an automated, near real-time data pipeline through Qlik Replicate, whether data originates from legacy SAP environments, SAP HANA, or SAP application servers. Additionally, Qlik Compose for Data Warehouses can be used to easily generate and automate logical data models from SAP directly in BigQuery freeing up more time for data analysts to leverage advanced built-in capabilities such as BigQuery ML to derive greater value and insights using standard SQL without the need for advanced programming expertise.
Delivering faster results with real-time
Traditional extract-transform-load (ETL) solutions operate on a batch basis, pulling data sets from SAP daily, hourly, or every minute. These tools often require manual mapping of multiple data fields so that data flows accurately. Given SAP’s highly complex table relationships, in which a single transaction can result in multiple changes, this can be a time consuming and tedious process. ETL can also increase the burden on your SAP systems.
Qlik Replicate simplifies this with its intuitive user interface where you can set up real-time data replication between SAP and BigQuery, eliminating the need for manual coding. And, to prevent system overhead, as soon as a new transaction is entered into SAP, the resulting data is replicated into BigQuery in a process known as change data capture (CDC) from SAP’s log layer. This means that data transfer can benefit from high performance with minimal impact on the source system’s resources. Read our latest white paper to learn how to extract SAP data into BigQuery leveraging Qlik Replicate.
Solution expertise for all core SAP workloads
It doesn’t matter what database your SAP system runs on, Qlik Replicate supports all core SAP systems. It automates real-time data replication and decodes SAP’s complex, application-specific data structures into formats that flow smoothly into BigQuery. This ensures that anyone who depends on data analytics has the most current and relevant SAP data they need. For its robust solution capabilities Qlik has received a new “SAP on Google Cloud Expertise” designation for supporting:
- Fast onboarding and accelerated replication of SAP data into Google Cloud
- Real-time and continuous data replication from SAP applications to BigQuery
- Support for all core SAP modules and a broad set of data sources
- Automated data integration, which cuts resource requirements for initial delivery and ongoing maintenance
Proven customer results
Many SAP customers have experienced the benefits of leveraging Qlik alongside BigQuery for data analytics and AI at scale, including German luxury department store chain Breuninger. In order to meet its customers’ growing and changing expectations,Breuninger needed to accelerate its time to insight from data sources across a highly dispersed landscape of on-premises databases and systems, including SAP. The company uses Qlik Replicate to feed corporate data from modules in its SAP system into BigQuery and integrate its varying on-premises databases with the Google Cloud environment. This has yielded game-changing, real-time customer insights for the retailer.
What could your business do with faster, more integrated insights? Learn more about BigQuery for SAP customers and also how Qlik and Google Cloud can help you modernize and automate data integration and analytics.
4562
Of your peers have already watched this video.
43:30 Minutes
The most insightful time you'll spend today!
What is Cloud Spanner and Is It the Right Database for You?
You’ve probably heard about Cloud Spanner, but what exactly does it do that is so different from other databases out there?
How do you know if it’s worth spending your valuable time trying out?
Cloud Spanner is the first enterprise-grade, globally distributed, and strongly consistent database service built for the cloud specifically to combine the benefits of relational database structure with non-relational database scale.
Phew! That was a mouthful.
So, what really differentiates it? It’s unique in the marketplace in that it combines transactions, SQL query, and relational structure with the scalability you typically associate with non-relational or NoSQL databases.
Databases are part of every application. So it’s very important to pick the right database for your application.
With Cloud Spanner, you enjoy all the traditional benefits of a SQL database; transactions, high availability, schema changes without downtime, and SQL queries. But unlike many relational databases, Cloud Spanner scales horizontally from one to thousands of servers, an arbitrarily large number.
With fully automatic data replication and server redundancy, Cloud Spanner delivers up to five nines of availability in multi-region instances and four nines of availability in regional instances.
In fact, Google’s internal Spanner instance has been handling millions of queries per second from many Google services for years.
In this video, you’ll find out what Cloud Spanner is, what it’s useful for, why it’s unique, and the challenges it solves.
2022 Healthcare Trends: Healthcare Data, M&As, Better Patient Care, AI in Drug Development & Strategic Partnerships

6604
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
The COVID-19 pandemic continues to push the healthcare and life sciences industry in entirely new ways. In record time, we’ve witnessed public health officials, vaccine developers, equipment manufacturers, and essential workers take life saving actions—regularly putting their own lives at risk—to respond to the exceptional challenges of our time.
Yet after all the turmoil and uncertainty of the pandemic, record breaking levels of investment continue to come into the market to fuel innovations.
Vaccine development is now measured in weeks rather than years; providers are leveraging telehealth technologies to improve the physician and patient experience; and individuals have embraced a variety of devices to assume greater ownership and control of their personal health.
With that in mind, here are five of the many innovations I see driving healthcare and life sciences for at least the next 12 months:
1. Unleashing the power of healthcare and life sciences data
People have arguably never had as much access and understanding to their personal health data than they can in 2022. They have the ability to understand their genetic makeup, medical history, family history, and activity levels to ensure they are living a healthy lifestyle. Taken together, this longitudinal profile can become the basis for more personalized medicine. Add wearable devices to the mix and patients gain real- or near-real-time updates on their health status.
Further, global regulatory agencies have continued to mandate the need for providers to maintain a longitudinal health record that provides a holistic view of the patient across all the encounters they have had with a health system. Physician surveys conducted by the The Harris Poll and Google Cloud show that a 360-degree view of the patient, across all provider encounters, leads to faster, more accurate diagnosis, and better outcomes.
If secure data access and interoperability can begin to include insurers, researchers, public health officials, and others in the field, the powerful network effects benefiting patients will only grow. When combined with those longitudinal phenome profiles, the opportunities for personalized and preventative medicine enter a whole new era.
2. Healthcare and life sciences M&A boom continues
Although the pandemic initially slowed activity on mergers and acquisitions in the early part of 2020, the healthcare and life sciences industry has seen a rapid rebound and acceleration of deals ever since. The success of COVID-19 vaccines, the importance of telehealth, and the focus on molecular modeling and genomic-based drug development have all boosted investment as organizations look to enter new markets, develop new therapies, and leverage low interest rates while they last.
In 2021, the total funding of US digital health startups surpassed $29 billion across 729 deals, according to advisory Rock Health. That’s almost double the levels of 2020, which itself set records. Analysts at PWC meanwhile estimate M&A investments in biopharmaceutical and life sciences could approach $400 billion this year across all sub-sectors
Clearly, the pandemic has been a primary driver of investment as the focus on healthcare has dramatically increased. Yet the increased activity also reflects changing business models and emerging technologies that are now required to compete in the rapidly evolving space. For organizations to capitalize on these investments, it will take not only great vision and intellectual property but also the right technologies—like cloud—and the right data interoperability models, to make partnerships and acquisitions more scalable, feasible, and seamless.
3. Transforming the patient experience at a new rate
The pandemic has shined a spotlight on the inefficiencies and complexities that exist in healthcare markets across the globe. As wave after wave has surged, global healthcare systems remain overwhelmed on most every aspect of patients’ treatment journeys. Even before COVID-19, healthcare was already one of the largest spend areas for governments around the world. The pandemic has only exacerbated the known issues.
Clearly, administrators, regulators, physicians, nurses, and patients would agree that the processes and models need to change. There’s a need to maintain this momentum and even increase the tempo to achieve lasting change.
Take telehealth. Within months of the start of the pandemic, providers moved to provide more remote capabilities so physicians could still meet with patients virtually to ensure health and safety on all sides. Payers recognized the importance of telehealth and began to update reimbursement rules. And organizations are now reimagining policies in areas such as prior authorization, submission, and adjudication to reduce complexity and bureaucracy while improving responsiveness.
Looking beyond the system, organizations are also recognizing and deepening their understanding of the structural and social determinants of health that impact patient care and health outcomes, especially for historically underserved communities. Private and public sectors are learning from, and increasingly partnering with, the social sciences, public health, biomedical informatics, computer science, public policy and community groups around how to build a mI’ore equitable and inclusive consumer products and Health IT strategies.
The newfound levels of transparency, visibility, and accountability that patients, caregivers, and organizations are achieving will ultimately increase competition and provide a more effective, equitable, efficient and, above all, healthier marketplace for all patients. As we move past the worst of the pandemic, regulators and organizations should keep fighting for progress over business as usual.
4. AI is now a core competency for Drug Development
The ability of organizations like Pfizer, Moderna, Johnson & Johnson, and Astrazeneca to develop COVID-19 vaccines has been a remarkable accomplishment—particularly the historic speed with which they were created and deployed. This innovation acceleration was largely enabled by the use of new drug development platforms that allow researchers to use artificial intelligence and machine learning to model protein and cellular interactions to rapidly advance the science.
No longer must researchers rely on traditional laboratory testing (and retesting). With their improved understanding of the molecular and genetic structure of a patient and, for example, their tumor, researchers can use AI to enable simulations on computers rather than testing in live conditions. This technology can process thousands, even millions of simulations to help identify high-potential candidates for treatment consideration and subsequent analysis.
AI-enabled drug discovery models can eliminate months and years from the research process, which can reduce the time to develop a drug and accelerate the time to treatment for an individual patient. As just one example, consider the work on AlphaFold2 by Google’s DeepMind unit who leverages AI to predict effective protein shapes for new drugs. Healthcare and life sciences organizations already recognize the potential of AI. Now comes the investments to leverage this rapidly evolving technology to support their efforts now and in the future.
5. Ecosystem partnerships tackle complexity and spur innovation
As the importance and growth of the healthcare and life sciences industry continues, we will see even more new players and partnerships emerging to address old problems in new ways. This trend will touch all aspects of the healthcare value chain and will, increasingly, see three- and four-player partnerships emerge to address the complex challenges of today’s healthcare marketplace.
Technology will continue to play a key role as capabilities and platforms will transform all aspects of the marketplace. Cell phones, wearable devices, and other technologies will provide real-time updates and notifications to patients on everything from glucose levels to payments for healthcare services. Voice recognition software will document physician and patient discussions to reduce the burden of record keeping. Real-world data will be used to simplify and confidentially recruit patients for participation in clinical trials.
New players will continue to enter the market to improve health outcomes and reduce costs. Major retailers are among the companies extending their pharmacies to provide additional diagnostic and concierge services, saving patients from additional appointments while boosting prevention. Community organizations are emerging to help identify and care for underserved communities whose health outcomes are significantly lower than the average patient.
For all the exhausting and heart-wrenching challenges of the past two years, the opportunities the pandemic has laid bare cannot be overlooked. We owe it to those who have worked and fought so hard for every life to forge even more new partnerships—and make it easier to do so—so that the next crisis, when it does arise, will never be as bad as the one we’re now conquering. Technology can be the enabler in this effort and help bring us together to continue to conquer the challenges that lie ahead.
Power up Caching with Memorystore for Memcached

3933
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
In-memory data stores are a fundamental infrastructure for building scalable, high-performance applications. Whether it is building a highly responsive ecommerce website, creating multiplayer games with thousands of users, or doing real-time analysis on data pipelines with millions of events, an in-memory store helps provide low latency and scale for millions of transactions. Redis is a popular in-memory data store for use cases like session stores, gaming leaderboards, stream analytics, API rate limiting, threat detection, and more. Another in-memory data store, open source Memcached, continues to be a very popular choice as a caching layer for databases and is used for its speed and simplicity.
We’re announcing Memorystore for Memcached in beta, a fully managed, highly scalable service that’s compatible with the open source Memcached protocol. We launched Memorystore for Redis in 2018 to let you use the power of open source Redis easily without the burden of management. This announcement brings even more flexibility and choice for your caching layer.
Highlights of Memorystore for Memcached
Memcached offers a simple but powerful in-memory key value store and is popular as a front-end cache for databases. Using Memcached as a front-end store not only provides an in-memory caching layer for faster query processing, but it can also help save costs by reducing the load on your back-end databases.
Using Memorystore for Memcached provides several important benefits:
- Memorystore for Memcached is fully open source protocol compatible. If you are migrating applications using self-deployed Memcached or other cloud providers, you can simply migrate your application with zero code changes.
- Memorystore for Memcached is fully managed. All the common tasks that you spend time on, like deployment, scaling, managing node configuration on the client, setting up monitoring, and patching, are all taken care of. You can focus on building your applications.
- Right-sizing a cache is a common challenge with distributed caches. The scaling feature of Memorystore for Memcached, along with detailed open source Memcached monitoring metrics, allows you to scale your instance up and down easily to optimize for your cache-hit ratio and price. With Memorystore for Memcached, you can scale your cluster up to 5 TB per instance.
- Auto-discovery protocol lets clients adapt to changes programmatically, making it easy to deal with changes to the number of nodes during scaling. This drastically reduces manageability overhead and code complexity.
- You can monitor your Memorystore for Memcached instances with built-in dashboards in the Cloud Console and rich metrics in Cloud Monitoring.
Memorystore for Memcached can be accessed from applications running on Compute Engine, Google Kubernetes Engine (GKE), App Engine Flex, App Engine Standard, and Cloud Functions.

The beta launch is available in major regions across the U.S., Asia, and Europe, and will be available globally soon.
Getting started with Memorystore for Memcached
To get started with Memorystore for Memcached, check out the quick start guide. Sign up for a $300 credit to try Memorystore and the rest of Google Cloud. You can start with the smallest instance and when you’re ready, you can easily scale up to serve performance-intensive applications. Enjoy your exploration of Google Cloud and Memorystore for Memcached.
Carbon Health: Transforming outcomes with Connected Sheets for Looker

3114
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
Everyone wants affordable, quality healthcare but not everyone has it. A 2021 report by the Commonwealth Fund ranked the U.S. in last place among 11 high-income countries in healthcare access.1 Carbon Health is working to change that. We are doing so by combining the best of virtual care, in-person visits, and technology to support patients with their everyday physical and mental health needs.
Rethinking how data and analytics are accessed at Carbon Health
Delivering premium healthcare for the masses that’s accessible and affordable is an ambitious undertaking. It requires a commitment to operating the business in an efficient and disciplined way. To meet our goals, our teams across the company require detailed, daily insights into operating results.
In the last year, we realized our existing BI platform was inaccessible to most of our employees outside of R&D. Creating the analytics, dashboards, and reports needed by our clinic leaders and executives required direct help from our data scientists.
However, this has all changed since deploying Looker as our new BI platform. We initially used Looker to build tables, charts, and graphs that improved how people could access and analyze data about our operating efficiency. As we continued to evaluate how our data and analytics should be experienced by our in-clinic staff, we learned about Connected Sheets for Looker, which has unlocked an entirely new way of sharing insights across the company.
A new way to deliver performance reporting and drive results
Connected Sheets for Looker gives Carbon Health employees who work in Google Sheets—practically everyone—a familiar tool for working with Looker data. For instance, one of our first outputs using the Connected Sheets integration has been a daily and weekly performance push-report for the clinic’s operating leaders, including providers.
Essentially a scorecard, the report tracks the most important KPIs for measuring clinics’ successes, including appointment volume, patient satisfaction such as net promoter score (NPS), reviews, phone call answer rates, and even metrics about billing and collections. To provide easy access, we built a workflow through Google App Script that takes our daily performance report and automatically emails a PDF to key clinic leaders each morning.
Within the first 30 days of the report’s creation, clinic leaders were able to drive noticeable improvements in operating results. For instance, actively tracking clinic volume has enabled us to manage our schedules more effectively, which in turn drives more visits and enables us to better communicate expectations with our patients. Other clinics have dramatically improved their call answer rates by tracking inbound call volume, which has also led to better patient satisfaction.
Greater accountability, greater collaboration
As you can imagine, a report that holds people accountable for outcomes in such a visible way can create some anxiety. We’ve eased those concerns by using the information constructively, with the goal to use reporting as a positive feedback mechanism to bolster open collaboration and identify operational processes that need improvement. For example, data about our call answer rates initiated an investigation that led to an operational redesign of how phones are deployed and managed at more than 120 clinics across the U.S.
Looker as a scalable solution with endless applications
We’re now rolling out Connected Sheets for Looker to deliver performance push-reporting across all teams at Carbon Health. Additionally, we continue to find new ways to leverage Connected Sheets for Looker to meet other needs of the business.
For instance, we’ve recently been able to better understand our software costs by analyzing vendor spend from our accounting systems directly in Google Sheets. Going forward, this will allow us to build a basic workflow to monitor subscription spend and employee application usage, which will lead to us saving money on unnecessary licenses and underutilized software.
We’ve come a long way in the last year. Between Looker and its integration with Google Sheets, we can meet the data needs of all our stakeholders at Carbon Health. Connected Sheets for Looker has been an impactful solution that’s going to help us drive measurable results in how we deliver premium healthcare to the masses.
1. Mirror, Mirror 2021: Reflecting Poorly
2. HEALTHCARE EDITORS’ PICK Meet The Immigrant Entrepreneurs Who Raised $350 Million To Rethink U.S. Primary Care
More Relevant Stories for Your Company

AI Solutions for Government Organizations: How to Get Started
Cloud-native features are helping public sector teams innovate faster than ever. Ideas discussed in a morning meeting can be a working proof of concept later that day. Managed services can remove administrative burden and reduce the steps needed to design and provision cloud infrastructure. Security can be built-in from the

How One Company Uses AI and Data Analysis to Boost Revenue
AI, deep learning, and image recognition is transforming the shopping experience. These technologies enable consumers to use product images or screenshots rather than text to search for similar products. This improves the customer experience and enables retailers with online and offline outlets to provide a genuine omnichannel experience. The lack of

Google Cloud Next 21 for Data Analytics Unplugged
October 23rd (this past Saturday!) was my 4th Googlevarsery and we are wrapping an incredible Google Next 2021! When I started in 2017, we had a dream of making BigQuery Intelligent Data Warehouse that would power every organization’s data driven digital transformation. This year at Next, It was amazing to

How Google Cloud Solutions Help Retail Firms to ABP(Always Be Pivoting)
For years, retailers have been told that they must embrace a litany of new technologies, trends, and imperatives like online shopping, mobile apps, omnichannel, and digital transformation. In search of growth and stability, retailers adopted many of these, only to realize that for every box they ticked, there was another






