J.B. Hunt, a Fortune 500 company and one of the largest transportation logistics companies in North America, was in the midst of a company-wide digital transformation journey. A key part of this journey was the implementation of a Microsoft Azure Databricks data lake to modernize the data warehouse for increased efficiencies and data access across the organization. However, as the engineering and technology team started its rollout, they noticed increasing pressure on the operational data stores that served as the backbone for J.B. Hunt 360, their cutting-edge digital freight matching platform. After evaluating the data pipelines, the team pinpointed the need to accelerate the flow of data into the lake to ensure that J.B. Hunt 360 users maintained a quality experience with no performance lag.

Pharmaceutical companies are under increasing pressure to stand out from the crowd in an industry facing pricing pressures, promotional saturation, regulatory scrutiny, reduced product differentiation and increased customer and channel complexity. The healthcare market faces the dual paradox of being awash in data but often lacking in insight. Some of these companies are finding the means to pull ahead of the pack through leveraging analytics in broader and dramatically more effective ways. They are creating the organisational capability to develop and execute against differentiated insights. AsteRx was dealing with large data volumes, comprising millions of scripts from a national sample of hundreds of physicians to be analysed in a myriad of ways that would take weeks to manipulate for a single result. Customers were provided with raw research data, usually in Access or CSV format, making it difficult from which to draw insight. Furthermore, data contained lots of hierarchies, continually changing data frequency with varying levels of data aggregation. Multiple data sources also needed to be integrated with different formats, languages and data structures.

Gwent Police, a force in South Wales, United Kingdom, was facing several challenges. The existing performance management system was incapable of providing information for internal stakeholders. The force needed a system that could provide crime mapping and support intelligence-led policing. The previous system was paper-based and took a whole day to compile monthly staff performance reports. Information was dispersed under different applications in the force’s Command and Control system. The force was also keen to identify the availability of appropriately trained staff and highlight where there were shortages.

The American University of Beirut (AUB), a leading academic institution in Lebanon, faced significant challenges in managing 'big data' and extracting valuable insights. The university was using Oracle's business intelligence solution, which they found difficult to deploy and was only being used for reports rather than for dashboards and business discovery analysis. The university was also seeking to create a roadmap for the use of QlikView throughout the campus. The university and its medical center, AUBMC, were working with a variety of applications and database systems, including IBM DB2, Microsoft SQL Server, FoxPro, Oracle Financials, and specialist systems for hospital clinical needs. They also used three enterprise resource planning (ERP) systems—Oracle ERP, Banner ERP, and IBM—as well as Oracle Business Intelligence Enterprise Edition (OBIEE) for AUBMC.

Loudoun County Public Schools (LCPS) faced a significant challenge during the COVID-19 pandemic. With the closure of schools, the district had to ensure that all students had access to online learning resources. However, a small percentage of families and staff did not have internet access at home. LCPS had procured 1,500 hotspots from various vendors, but the challenge was to distribute these devices equitably. The district needed to identify not just the students, but also the households that required internet access. A triage approach was established to prioritize the distribution based on the disadvantaged status of households, households with multiple students, and households with students in higher grade levels.

Allina Health, a not-for-profit healthcare system, is dedicated to the prevention and treatment of illness and enhancing the health of individuals, families, and communities throughout Minnesota and western Wisconsin. The organization has a comprehensive electronic medical record (EMR) system, Epic, which plays a key role in its success. However, Allina Health is also part of the Center for Medicare & Medicaid (CMS) Innovation’s Pioneer Accountable Care Organization (ACO) model, which requires a strong information infrastructure to manage and visualize the enormous amount of data that will eventually be reported to the government. The goal of this infrastructure is to bring all medical and patient information together, enabling Allina to evaluate its performance, identify areas for improvement, and discover ways for clinicians and care teams to deliver better care to patients.

Design Within Reach (DWR), a multichannel modern furnishings retailer, was struggling with its data management. The company’s sales and inventory data was often analyzed using Excel, and occasionally with custom built reports using SAP’s Crystal Reports. This led to inconsistencies in data analysis as different people used different processes to create reports. The IT team was also burdened with doing one-off pulls from the company’s databases. DWR needed a solution that was flexible enough to be used by all Area Managers, regardless of location, and easy enough that they could find answers on their own, despite their levels of technical ability.

PHOENIX, a premier residential mortgage industry advisor, faced the challenge of collecting and analyzing extensive quantities of data for its over 250 customers. The information required for efficient, effective, and compliant mortgage production and servicing came from multiple sources due to the variety of systems employed by these firms. For instance, a Mortgage Servicing Rights (MSR) owner might use multiple third-party subservicers to manage its MSR portfolio, complicating the process of viewing performance data. It was increasingly difficult to sift through and focus on salient information on a consistent and timely basis to enable informed, smart decisions and to monitor outcomes. A lot of specialized effort was spent gathering the data, understanding what happened with a particular loan, realizing what it means, and then planning a course of action.

MAPCO Express, Inc., a leading convenience store operator in the Southeast United States, was struggling with inaccurate reporting due to the use of Excel as its only reporting tool. The company had significant amounts of valuable data but lacked the ability to extract meaningful information from it. For instance, it was unable to track customer loyalty trends per store. The timeliness of the data was another issue, with reports taking weeks to aggregate, which hindered efficient business decision-making. MAPCO needed a single source of truth and the ability to see the underlying trends and patterns that led to discrepancies.

King.com, a leading global online gaming company, was faced with the challenge of managing and making sense of the massive volumes of gaming data generated by their platform. The company needed a way to make this data accessible to the business for informed decision making. They also aimed to deliver rapid business insights and empower their business users with self-service capabilities. The challenge was not only to handle the data but also to derive meaningful insights that could inform various aspects of the business, from executive decisions to marketing strategies and product development.

Nemours Children’s Health System had a big challenge around electronic information: It needed to change its culture, to move to a model where staff started pursuing data versus just receiving data. The organization wanted to shift from repeatedly building dashboards and reports to really enabling the organization. They wanted to bring all their relevant data from any system, across finance, patient care and the overall business, into one place for analysis.

ForbiddenPlanet.com, a leading science fiction retailer in Europe, was struggling with data analysis. The company had a vast amount of data but lacked a system for effective analysis. This was particularly problematic in their fast-paced comics business, where new editions are released weekly and accurate estimations of product demand are crucial. The company's management account meetings were inefficient, with an 80-page management pack of quarterly account data that was already out of date by the time of the meeting. The lack of real-time data analysis led to meetings ending with more questions than answers.

IndianOil, the largest public sector oil and gas company in India, first turned to data analytics with QlikView in 2012. Dashboards were created, mainly for the company’s chairman and directors. During a review of its IT infrastructure, IndianOil identified an opportunity to make more use of the increasing amount of data being held throughout the organization. The company saw the potential of leveraging this data to drive business value across the organization.

Pizza Pizza, a Canadian food chain, was struggling with managing and making sense of the vast amount of data generated from its operations. The company operates over 750 locations and fulfills over 20 million orders annually, generating a database with billions of fields. The company used to rely on static reports and spreadsheets to analyze its operations, which lacked flexibility and interactivity. The reports were generated daily and if a different perspective was needed, it would take days or weeks for IT to generate a new report. The company was unable to access real-time data and lacked the ability to drill down into the details of its operations.

Sketch Books, Inc., a bookstore and stationery merchandise company with 30 branches across the Philippines, was forced to shift its focus to e-commerce due to the COVID-19 pandemic. The company needed to ensure a high level of customer service despite the shift to remote working. The challenge was to maintain the same level of service and customer satisfaction while transitioning to a new business model.

The Hillsborough County Tax Collector Office (HCTC) is responsible for collecting and distributing local and state taxes and fees to various partner organizations. They serve as an agent for their partner agencies and all of their community members to ensure that the money that flows into their office goes where it needs to go. However, they were facing challenges in managing the vast amount of data they collected and in using it to improve their efficiency and the quality of their work. They initially used Excel to handle performance metrics and insights, but it was time-consuming and prone to errors. They then tried another solution, but it did not allow them to actively manage their data or understand why certain trends were occurring.

Great Ormond Street Hospital Children's Charity was facing a decrease in the performance of warm cash appeals and needed to understand the reasons why and to explore more insight driven approaches to combat this. Previously, fundraising campaign reporting was done after each campaign by pulling data into Excel to create tables and charts, then moving these into Powerpoint for delivery and presentation. This often occurred after the next campaign was already live. The process was costly in terms of the time and resource required to prepare reports and did not allow the team to be as flexible and reactive as they would like.

Gazprom Neft, one of the top three oil-producing companies in Russia, was facing challenges in making data-based decisions due to the lack of a robust Business Intelligence (BI) unit. The company was already a year into their journey with Qlik, a data analytics platform, when the BI Department Head joined in 2013. The task was to further develop the internal Qlik-based processes. However, the allocation of resources was a major challenge, especially server resources. The company was also not always on schedule in the purchasing of new equipment. Change management was another challenge, particularly overcoming internal resistance without clear signaling from company leadership.

Nationwide Building Society, the world’s largest mutual financial institution, was facing a challenge with data management and analytics. The company's strategic vision is driven by its 15 million members, not the market or shareholders, and data underpins every one of their strategic goals. However, the company found that employees often worked within their silos or divisions and designed things for themselves, unaware that others in the organisation were trying to do something similar—often with the same data. This led to a lack of efficiency and a duplication of efforts. Furthermore, the company was using QlikView, but had licenses for only 500 people, limiting the access to data and insights for many employees.

Anurag Group of Institutions (AGI) is a large educational institution in India with a vision to maintain high academic standards and promote analytical thinking and independent judgement. The Department of Computer Science Engineering at AGI identified a serious skills gap for good data analysts in industry and business. With a forecast of 97,000 jobs that need to be filled in the area of data science, there is a huge need for skilled people. The department has been teaching 'Big Data' subjects to students and recognizes that the future for the next 30 years will be data analytics. However, they were seeking the best way to instruct students in data visualization and provide them with practical experience.

PSCU, a credit union-owned cooperative, was in the midst of a multiyear initiative to evolve and broaden its call center services when it realized the need for better data organization and visibility. The company was dealing with disparate data sources and flat reporting that lacked analysis, color-coding, and cross-functionality. The reports served specific needs but did not show all the inputs or outputs due to lack of integration. PSCU recognized the need to better organize its data to more effectively align with its business model and to provide the full suite of products and member experience required by its credit union relationships. The view needed to be cross-functional for the company to optimize and innovate where necessary.

Hendry, an asset protection firm, was facing a challenge of limited value in their services due to a narrow vision of their work. The industry was lagging behind in the adoption of technology and clients were demanding more than what was being delivered. The company realized that they needed to change and adapt to the fast-paced, technologically driven world. They wanted to provide more value to their clients than just services required within statutory law. They saw the potential of data and technology to transform their services and become a full-service advisory business.

Odyssey, a logistics and supply chain management company, was in need of a business intelligence solution that could help them keep up with competitors and add value for their customers. The solution had to be a non-cloud, on-premise solution to allow the company to maintain complete control over the operation. It also needed to be flexible enough to extend into a true 'visibility' platform on the front-end. The company spent almost all of 2016 trying different solutions during the discovery phase of the project, but none of them were the right fit.

Quooker, the creators of a tap that produces boiling water almost instantly, has been experiencing rapid growth, with a 100 percent increase every three years. To sustain this growth and expand operations across a complex global market, the company needed more precise forecasting and analytics tools.

BorgWarner, a global automotive components and parts supplier, was previously using a manual, spreadsheet-based approach to produce reports on total spend, supplier performance, and supply chain risk. This process was time-consuming, taking days to prepare for each supplier. The company faced a significant challenge in 2011 when Japanese suppliers were affected by a tsunami. BorgWarner struggled to determine the impact on their supply chain, products, and customer orders.

The article discusses the challenges faced by organizations in their journey towards data maturity. The first challenge is related to data quality. Organizations often have data quality issues, which become more apparent as they try to utilize the data. Issues can range from misspellings, inconsistent data entry methods, to software compatibility issues. Before any progress towards data-based decisions can be made, the data itself must be trustworthy. This requires creating mechanisms to ensure data quality. The second challenge is related to data literacy. Having great data is just part of the equation. Organizations must also have a genuine curiosity and the ability to ask the right questions. Most organizations are very familiar with asking, “what happened?” but the real power in analyzing data comes from more advanced questions. The final stage of intuitive growth involves reaching a level where insights about data trends lead to an ability to influence outcomes.

Transport fleets are crucial for modern commerce, and their efficiency is paramount. However, as fleets grow in size, their management becomes increasingly complex. To keep track, organizations have equipped their fleet vehicles with sensors. However, the sheer volume of telemetry data they receive is difficult to consolidate and convert into meaningful reports that point to effective business decisions. Montrans, Russia’s leading transport monitoring company, faced this challenge. It started as a fuel consumption tracker but its systems have since grown in sophistication and it wanted to move away from static reports to provide interactive fleet control and data-driven management.

The Chinese Academy of Agricultural Sciences (CAAS) is a national innovation center for agricultural IT. It monitors agricultural information, risk analysis, food safety, and data acquisition. CAAS is in the process of refining big data for smart agriculture through its own application. The application handles heterogeneous data resources from a variety of data sources including local governments, commercial databases, and Internet of Things (IoT). However, the business intelligence (BI) products previously deployed by CAAS were seen as inadequate. They were generally limited to static data analysis and visualized representation, making it difficult to fulfill in-depth data interaction and analysis. CAAS wanted more specialized and powerful functionality. With such a huge volume of data, CAAS was concerned about performance load.

Volvo Group, a global manufacturer of trucks, buses, construction equipment, and marine applications, faced a significant challenge in managing its complex material flow. The company had to connect with the right suppliers for the right spare parts and find the most efficient way of getting the right parts to the right trucks at the right time. The company's existing tools, such as MS Excel, were inefficient and left a lot of room for error. The team spent an excessive amount of time gathering data and making it understandable and presentable. This inefficiency led to a culture where teams and individuals quickly became siloed, missing out on opportunities to improve workflows or identify suppliers whose products weren't up to standards.

HSBC, a global bank serving over 40 million organisations and individuals, was grappling with the complexity and scale of its data. The bank recognised the need for a strong commitment to analytics to thrive in the future. However, the challenge was not just about collecting data but making it usable and accessible across the organisation. The bank needed to ensure data quality, make data-driven decisions easier, and enable every staff member to contribute to business intelligence. The sheer size of HSBC and its commitment to data collection and analysis necessitated a simplified approach to data management.