Schüttflix, a German logistics start-up, aimed to disrupt the traditional construction supply chain by digitizing the industry, which was primarily reliant on pen and paper processes. The company's mission was to enable data-driven decision-making by providing stakeholders with accurate and timely data. The challenge was to build a modern data stack that could tap into key data sources efficiently and reliably. Alexander Rupp, Head of Data and Business Intelligence at Schüttflix, was tasked with evaluating connectors that could meet these requirements.

Lendi, an Australian mortgage broker with over $12 billion AUD in home loan settlements, was facing a significant challenge in its data management. The company's business model, which involves offering customers a choice of over 2,000 loan products from more than 40 lenders, relies heavily on delivering the right experience to the right person at the right time on the right digital platform. This requires accurate insights into borrowers' needs and preferences, which in turn requires tapping into behavioral data on third-party engagement platforms such as Facebook, Google, and Bing. However, the data from these platforms were siloed and did not integrate easily. Even when the data could be brought into the same repository, the data structure was often inconsistent, necessitating data cleaning before it could be used. The responsibility of ensuring that stakeholders across the company could analyze and create actionable insights from this third-party data fell to Lendi’s Data Architect, Daniel Deng.

JetBlue, a major airline operating over 900 flights daily to more than 110 cities, was grappling with the challenge of managing and analyzing the vast amount of data generated by its operations. Every person, plane, and journey generated data points that could provide insights into customer sentiments, revenue forecasting, fuel consumption, aircraft maintenance, and operational readiness. However, the sheer volume of data, sourced from 130 different systems, was overwhelming and difficult to organize. The airline needed a solution that could centralize this data, making it readily accessible for analysis and decision-making. The challenge was to bring all this data into a single platform quickly and accurately for analysis.

The Denver Broncos, a successful pro football team, faced a significant challenge in maintaining their data pipelines. The team's Senior Director of Ticket Strategy and Analytics, Clark Wray, and his lean team were spending excessive time on home-brewed data integrations. Whenever an original data source or API changed, it would disrupt the data connections they had built, often halting the flow of information for hours. If these issues were not addressed promptly, the business risked operating on inaccurate data. Additionally, the team was constantly adding new data sources to communicate and reach the next generation of fans. It was crucial for them to connect and centralize their email data in Dynamics 365, marketing automation data in Eloqua, and fan feedback in Qualtrics.

Princess Polly, an Australian fashion boutique, was facing challenges in utilizing data effectively during a time of uncertainty. The company was preparing for a critical launch into the U.S. market and needed to support internal departments in making informed decisions. Anand Bhatt, the Head of Business Analytics, was tasked with building an analytics infrastructure that could demonstrate value quickly and efficiently. As the sole member of his team, Anand needed to maximize his time generating value for the business and minimize manual, time-consuming tasks. A key area of focus was cash flow analysis, with the aim of understanding which decisions were impacting the business’ bottom line to make more effective decisions.

Red Ventures (RV), a global company with a focus on positively impacting people’s lives and communities, was facing a challenge in managing marketing data efficiently. The company's Red Digital division provides end-to-end performance marketing services to help business-to-consumer (B2C) services providers attract new customers. To deliver greater value to clients, RV needed to use timely insights from data to reach the right consumers. However, maintaining each client’s data in a separate cloud environment and integrating each client’s data for machine learning predictions was proving to be a tedious and time-consuming task. Data engineers had to write custom scripts to ingest data for each client, which was not an efficient use of their time and skills.

Condé Nast, a global media leader with 37 brands reaching millions of consumers, was faced with the challenge of managing and monetizing trillions of data points generated from its digital assets. The company lacked a central mechanism for managing and maintaining data integration sources, making data not readily available to consumers downstream. The demand to integrate more sources globally continued to grow, and pulling data into the data lake with custom scripts was cost prohibitive. Each marketing technology platform had its own API, data structure and other properties that required its own custom script. Creating the connectors on the fly and managing them on an ongoing basis wasn’t scalable, posing a significant challenge to the company.

Kilo Health, a global leader in digital health and wellness, faced a significant challenge as it rapidly expanded. The company, which started with just seven people in 2013, has grown to over 700 employees managing more than 30 products with over 5 million customers worldwide. This rapid growth led to an exponential increase in data points, which the company needed to manage effectively to become a fully data-driven organization. The challenge was to find a solution that could support this rapid growth and provide intelligent and unbiased insights to stakeholders.

Kuda, a digital bank launched in Nigeria in 2019, experienced a fourfold increase in customers within six months. As a mobile-first digital company, Kuda recognized the need to be data-driven and identified the modern data stack as essential for achieving its goal. Initially, a five-person data team was manually building data pipelines and relied on SQL Server Reporting Services (SSRS) to extract insights from transactional databases. Their data was split into 12 disparate Azure SQL databases with no way to successfully join the data across their internal sources. The team was looking to move from running OLAP queries on an OLTP database, which was proving to be a challenging task. They needed a more scalable solution that would relieve them of having to build and manage data pipelines.

Pitney Bowes, a global technology company that simplifies e-commerce, shipping, and mailing, was facing significant challenges with its data management. The company lacked high-quality, real-time data necessary for critical business decisions. Its Enterprise Information Management (EIM) team was grappling with siloed data, lack of scalability, and inefficient tech spending. Employees were resorting to pasting data into Excel spreadsheets for executive reporting and analytics, which often exacerbated the issues. The company was also experiencing downstream problems, such as late-arriving packages that impacted Service Level Agreement (SLA) targets. They lacked the sophistication to detect delays and notify customers in time, causing reputational risk. The COVID pandemic magnified these data challenges when online shopping increased tenfold, leading to a tenfold increase in parcel volume. The company's legacy data infrastructure was unable to handle event- and email-based data operations for 800 million packages per day. The data captured was critical, but aggregating and consolidating it to the central analytics warehouse took days, making it outdated by the time it reached the leadership team.

Sleeping Duck, an Australian mattress company, was facing the challenge of managing and deriving actionable insights from data scattered across various sources. The data resided in Software as a Service (SaaS) platforms, web apps, marketing platforms such as Facebook and Google Ads, and in the company’s own product. The process of extracting relevant information from these disparate sources was complex and manual. The company's engineers would have had to write and maintain custom scripts to extract data, a practice that was neither scalable nor sustainable. The company needed a solution that could efficiently pull in data from these sources, manage it, and feed it into their business intelligence solutions for data-driven decision making.

Swapfiets, the world’s first ‘bicycle-as-a-service’ company, was facing a challenge in understanding behaviors in the emerging market. The company's growth strategy relied on identifying new cities, winning new subscribers cost-effectively, and establishing an efficient local support network. However, the company was struggling with data management. The data engineering team had built custom Python scripts to extract data into their central Redshift instance, which was manageable when pulling from just a couple of data sources. However, as the business started to expand, this approach proved impractical. Swapfiets needed a more streamlined approach to data ingestion to make sense of critical subscription and usage data. It was crucial for Swapfiets to understand its target demographic and how best to provide local support, carefully target its marketing, and avoid over-provisioning stock.

CarOnSale, a disruptive pan-European platform for car dealers, identified data as a key differentiator in their market. As an online platform, they aimed to cut through the complexity of traditional car trading by harvesting and analyzing data around car auctions. This would provide them with unique market intelligence. The company recognized the need for a centralized architecture, hosted in the cloud, to collect and analyze data at speed and scale. They explored different options to support ELT (Extract, Load and Transform) as opposed to the traditional ETL approach. After selecting Snowflake as their cloud-based data warehouse, they needed to find the ideal data integration solution. Aynaz Bagherynezhad, Data Team Lead, had used Fivetran in a previous role and when Snowflake recommended it as the best way to connect to data sources, it confirmed her own experience.

Chris Klaczynski, a Marketing Analytics Manager at Databricks, was tasked with supporting the primary marketing objectives of driving pipeline generation, growing the database, and improving ROI. However, as Databricks rapidly expanded, the need for centralized and documented data became more and more apparent. Data silos were appearing around the business, including on Chris’ marketing team, where data was stored in its own data warehouse. It was critical that Chris’ newly-built dashboards were supplied with trustworthy, timely data for marketing operations to keep running smoothly. However, without dedicated engineering resources, and in the face of a rapidly expanding marketing team, scaling with demand became next to impossible. Databricks faced a number of challenges with their traditional data warehouse, including issues with their Salesforce and Marketo pipelines, issues appending data natively to existing tables, and schema changes that were always breaking pipelines, resulting in outages and stale, untrustworthy data.

Lufthansa Systems, a division of Lufthansa Airlines, is a leading provider of IT services in the airline industry, serving around 300 airlines worldwide. One of its offerings is Lido/FPLS (flight planning services), which optimizes flight routes in terms of cost, fuel, and time, generating millions of dollars in extra profits for its customers each year. The challenge was that creating these optimized flight plans required massive amounts of data, including up-to-date weather reports, air traffic data, and airline-specific data such as flight schedules, payload, operational conditions, and contracted petrol prices. Lufthansa Systems needed a solution that would allow its central data repository to receive continuous updates from these data sources and distribute optimized flight plans and other data to each customer's site.

Paul Hewitt, a jewellery and accessory brand, was facing challenges in managing and analyzing its advertising spend due to the limitations of its existing tool, Supermetrics. The analytics team, consisting of three employees, had to manually enter data into spreadsheets to determine the most effective marketing channels, a process that was both time-consuming and prone to errors. To meet the needs of an increasingly complex supply chain, the company had invested in an ERP system, Microsoft Dynamics NAV, and began to make data available for analysis with Microsoft Power BI. However, the company wanted to take its data strategy to the next level by integrating data from across the business into one place on a cloud data platform, with the objective of transforming into a data-driven business.

Redwood Logistics, a third-party logistics and transportation management firm, was struggling with managing a complex reporting structure that relied on multiple siloed warehouses. The rapidly growing business needed a modern data stack that could support its mergers and acquisitions strategy, providing leadership with an accurate overview of business performance in near real time. The company was generating 500,000 data points per hour, which was a significant challenge to manage and process. The old system could only load data once a day and was prone to numerous daily failures, becoming a massive maintenance burden. Redwood was initially cautious about using Fivetran’s high-volume data replication because the team needed to understand how it interacted with their existing databases.

Super Dispatch, an online platform for auto transport, was facing challenges in onboarding new users and optimizing experiences for active users. The company's data was decentralized and scattered across various digital properties, business systems, and marketing tools. Employees were relying on spreadsheets shared around the company for different purposes such as marketing, billing, and sales. The data was downloaded from business systems or Software as a Service (SaaS) platforms individually and analyzed in Excel. This posed a significant challenge for Aman Malhotra, a veteran in the marketing, sales, and operations analysis industry, who was hired to improve user activation, retention, and monetization through the use of data.

Vida Health, a digital health company, was facing challenges with its data infrastructure. The company collects data on customers' medical history, past insurance claims, lab test results, and log data from health-tech devices to provide personalized virtual care. However, their custom-built solution using Python scripts and cron jobs to load and transform data in BigQuery was not scalable and often failed when data volume spiked. The pipeline was poorly documented and understood by only a few people on the data team, leading to reporting downtime of 2-3 days when issues arose. The company had recently consolidated its data engineering, data science, and data analytics functions into one team, aiming to improve collaboration. However, the existing data infrastructure was not reliable or accessible enough to best serve their customers and meet their goal of onboarding more than ten new clients in less than six months.

IJsvogel Retail, a Dutch pet and garden products chain with a history of nearly 130 years, was grappling with the challenge of managing and leveraging its vast and disparate data. With over 180 stores, more than 1,600 employees, and over 800 wholesale customers, the company was generating a significant amount of data. However, this data was not being effectively utilized to inform business decisions. Instead, old data and log files were often discarded rather than compiled and analyzed. The company's small IT department found it difficult to promote the adoption of new applications across the company. The lack of a unified, reliable, and stable data source was hindering the company's ability to make informed business decisions.

MyCamper, a Swiss start-up likened to the Airbnb of campervans, faced a significant challenge in managing and analyzing the data collected on its web platform. The company recognized the importance of this data, but initial attempts at analytics were laborious and time-consuming, involving manual extraction of data from Excel spreadsheets. Using Google Analytics proved to be easier but was limited in scope. As an early-stage start-up, MyCamper had more pressing priorities and lacked the in-house skills to manually build out data pipelines. This resulted in a gap in data analytics that needed to be filled. The company also struggled with historicizing data in a way that could be retrieved for analysis. They were unable to track specific data sets, compare historical data to present, or even have a comparable baseline.

Nando’s, a popular fast-food chain known for its flame-grilled peri-peri style chicken, was facing a significant challenge with its existing infrastructure. The company, which operates over 1,200 outlets in 30 countries, was struggling to meet the demands of its data-driven marketing strategies, particularly around customer loyalty and rewards programs. The existing infrastructure was slow and inflexible, making it difficult for the data team to effectively manage data pipelines and make informed business decisions. The team, led by Miquel Puig, Technical Lead on the Engineering team, was manually working on data pipelines and making business decisions based on data at the ingestion stage. One of the key use cases was turning end-of-day data from the outlets into insights that informed loyalty and reward programs.

World Fuel Services (WFS), a Fortune 150 company, faced significant challenges in managing and utilizing its data effectively. The company had grown through numerous acquisitions, each with its own client lists and data sources, making it difficult to gain a comprehensive view of customers across the entire organization. Additionally, the company's existing ETL pipelines pulled data in batches once a day into an on-premise Oracle database, which quickly became too large to run live queries effectively. The company also faced the challenge of managing data from dozens of ERP and billing information services across its subsidiaries, which was particularly critical during the global pandemic when the company needed to increase accounts receivable efforts to maintain revenue.

WeWork, a global provider of flexible office spaces, faced the challenge of securely managing and leveraging its vast data resources to drive business decisions. As a publicly-traded company, WeWork had to meet stringent regulatory compliance requirements, ensuring data governance across disparate sources and silos. The company needed to track data movement and user access over time, demonstrating to auditors and regulators that customer information was safe from internal and external threats. The challenge was not only to pull in data and provide access but also to maintain a historical record of data ingestion, changes to the database, and access. The company also aimed to create a culture of data literacy and innovation, empowering business agility through the use of data.

YipitData, a trusted source of insights using alternative data, faced a significant challenge as it began to grow rapidly. The company's analytics activities were running on dozens of Amazon Redshift clusters, with each team within the company maintaining its own clusters. This arrangement became cumbersome, especially when YipitData needed to share common data sets across teams. The company's product teams analyze billions of data points each day to provide granular insights that drive the successful decision-making of hundreds of investment funds and highly innovative corporations. However, the existing system was slowing down analytics and making it difficult for the company to stay ahead of its market.

Billie.io, a Berlin-based fintech startup, is revolutionizing the way businesses handle payments by providing instant financing for invoices and outsourcing the collections process and default risk coverage. However, the company faced a significant challenge in managing its data architecture. The company needed a solution that could handle the Extract, Load, and Transformation (ELT) process of their production database to the data warehouse efficiently and cost-effectively. The company also needed to avoid latency problems or Service Level Agreement (SLA) issues and prevent transformations from occurring too early. Furthermore, the company wanted to have fine-grain control over when things happen and awareness on tasks that comprise pipelines, their dependencies, and their execution.

Canva, an online design and publishing tool, was under pressure to grow its customer base across three service levels—free, pro, and enterprise. The sales, marketing, and engagement teams needed to identify targets, understand their behavior, and deliver the right message at the right time on the right platform. This required a 360-degree view of the customer across Canva’s digital properties and third-party platforms such as Google, Facebook, and other social media and SaaS tools. The main challenge was the lack of comparison insight. There was no way to analyze Facebook data against Google data or any other platform. Building a point-to-point architecture to pull data into competing platforms would be messy and expensive to maintain. The custom-built solution functioned as intended, but the demand for connectors to more data sources was growing, and the time and resources required to build these connectors were not scalable.

Coupa, a Business Spend Management (BSM) company, provides a cloud platform that digitizes and consolidates spending information across various sectors, creating actionable insights into spending behavior. However, Coupa faced challenges with its own data about its platform and customer usage. The data was siloed, impeding better insights and decision-making. The process of collecting this data and making it accessible to the relevant personnel was complex, costly, and resource-heavy. Coupa had invested in a data team to manage its data, with the goal of pulling data from various sources into a single place for creating actionable insights. However, the analytics strategy was immature and largely consisted of ad hoc procedures. If a UX designer wanted to know how customers were interacting with a particular feature, they’d have to request the engineering team to build a script from scratch, a process that could take weeks.

Daydream, an early-stage startup, provides financial insights to stakeholders across modern businesses. The company's business modeling and planning tool democratizes access to financial information by bringing together processes and data sources that are typically siloed. However, the Head of Engineering for Daydream, Shubham Sinha, faced a critical decision. The success of the startup hinged on its ability to move massive amounts of data from its customers’ cloud-based business systems, each with their own login credentials and access challenges, into the Daydream platform for analysis. The two options were to either ask its customers for login credentials to their business systems, posing a potential security risk, and use custom-built data pipelines to onboard data or to rely on Fivetran to broker the credential sharing exchange and onboard data using its pre-built data pipelines. Maintenance also posed a challenge as each cloud platform has its own APIs, processes, and data structures, many of them requiring custom integrations through scripting.

DPD Polska, a leader in the Polish courier market, was facing challenges with its existing data management system. The company was using a series of on-premises PostgreSQL and Microsoft SQL Server databases to track its trucks, parcels, and people. However, the array of custom SQL databases was preventing DPD from producing timely reports, meeting disaster recovery time objectives, testing new data and analytic products, scaling up its revenues, and increasing its customer base. For instance, one of DPD’s databases had three different usage contexts. The company was in need of a log-based replication solution that would not impact its source systems. The main pain points were the replication time lags, the risk of errors in manual data distribution, and the need for more flexibility, greater reliability, and higher operational scalability.