Progress

The British Broadcasting Corporation (BBC) faced several challenges with its legacy systems. For the BBC iPlayer, the legacy system was slowing productivity, innovation, and content delivery. The growing content required massive scalability and faster response times. For the BBC Sport website, the challenges were the integration of diverse data, such as statistics and videos, rapid application development to meet game deadlines, scalability to support tens of thousands of transactions per second, and the utilization of open standards for mobile delivery. The BBC needed to upgrade the iPlayer to support a growing user base, deliver content that could be viewed on multiple devices, and personalize that content. The BBC also wanted to deliver real-time, customizable, and interactive online content for the Summer Olympics.

The AIRBUS flight test program is a key component of aircraft development. Every part in every possible configuration must be tested in as many scenarios as possible. This creates multiple types of data, all in different silos — plane configuration criteria, flight data criteria, flight event behaviors, crew reports, and other unstructured content criteria. Historically, each test flight required manual research into previous tests to create a set of specific requests, and the effort to leverage data to optimize those new flight tests was considerable. While a lot of data was generated during testing, AIRBUS’ process to search within the data in order to simply retrieve a specific condition and its result was a slow, manual one. And, flight test environments are constantly changing, with a series of maneuvers done at specific locations, which meant that traditional data approaches couldn’t keep up with this complex data challenge.

Condé Nast, a renowned media and magazine brand, was facing challenges with its digital asset management system due to the introduction of over 7 million rich media assets representing more than 220 US and International Brands. The system was struggling in three critical areas: search, usability, and development. The average wait times for asset searches were upwards of 30 seconds per 100 results, making the search too cumbersome for regular use. As a result, editors started using expensive outside firms to source assets. Additionally, metadata and category additions required developer involvement, which delayed content updates by users. Lastly, there were long wait times for programmers to accommodate new features and workflows.

The Royal Society of Chemistry (RSC) had been gathering millions of images, science data files and articles from more than 200,000 authors since the 1840s. All of this information was stored in a wide range of formats at multiple locations and was growing by the day. In 2010, largely due to the huge growth of social media and digital formats, the RSC launched an initiative to make its data more accessible, fluid and mobile. They needed an integrated repository that would make all of their content accessible online to anyone-from teachers to businesses to researchers. The key was finding the right technology.

The US Department of Health and Human Services (HHS) Centers for Medicare and Medicaid Services (CMS) had to develop a Health Insurance Marketplace (HIM) and Data Services Hub (DSH) that would allow millions of Americans to shop for insurance, check their eligibility against dozens of federal and commercial data sources, and provide a way for state health exchanges to connect. This was the first time the US government would tackle an IT project of this kind. CMS had to overcome some daunting challenges: Strict time constraints, Stringent security requirements, Multiple data sources, An unknown data specification, Scalability.

HEROLD's initial challenge was to quickly monetize enormous amounts of addresses, household and corporate data that it acquired from purchasing a directory business. The company was seeking a better way to connect its customer-the businesses that advertise with it -with user of its digital properties, therefore HEROLD needed to create new digital products that could summarize and structure HEROLD's data assets so it could be productized and resold. Additional challenges included: Integrating and Managing Complex and High Volumes of Data. Integrating, accessing and searching the massive amounts of acquired personalized and address data could not be accomplished with HEROLD's existing relational database technology. Much of HEROLD's company data resided in as many as 15 separate systems and included records on more than 360,000 businesses, comprising of unstructured pieces such as product information, user reviews and recommendations, ratings, photos and text descriptions, which came from product feeds and manual entries. Aggressive Timeline to Demonstrate Return on Investment. HEROLD needed to integrate the newly acquired structured and unstructured data into its systems to quickly create and introduce new products that would help them stay ahead of the competition while demonstrating return on investment to management.

The British Broadcasting Corporation (BBC) was tasked with providing comprehensive coverage of the 2012 Summer Olympic Games in London. The challenge was to cover more than 10,000 athletes, cater to a sophisticated audience increasingly reliant on social media, provide continuous live coverage, and deliver content across multiple channels. The traditional method of Static Publishing, which had been relied on for over 15 years, was no longer sufficient. The BBC needed to transition to a Dynamic Publishing infrastructure, which involves creating a collection of related data elements and dynamically serving it as audiences demand. The flow of content was enormous, non-stop, real-time and went across every channel from web, mobile, tablets and broadcast.

Before the introduction of the new database technology, the Haufe Group had distributed its data across different data silos, each created for different applications. A linking of the data and thus an overall view of all content was hardly possible. Due to the silos, users would receive results of their keyword searches which only partly reflected the total availability of specialist information and topical cross-references were largely missing. The scattering losses for the user were significant, and Haufe wanted to fix this issue. The original system was based on the open-source platform Apache SolrTM and relational databases. It stored different types of data such as book metadata, widgets, book covers, product information, and the actual content in different data silos. Queries were made using various search engines and tools.

Lagardère Active, one of the largest media companies in France, faced several challenges in managing its vast and diverse content. The company had about 15 different databases containing various content types, including text, pictures, sound, and video, as well as data from cloud services like Dropbox. The company needed to aggregate all of this content without having to transform the data to make it usable. The company's IT department faced several tight deadlines to create certain new products. For example, it had just one month before the championships to create a live statistics site for the top French professional soccer league (Ligue 1) using heterogeneous data from several sources. The team also only had two months to build an iPad app for Paris Match, one of its biggest properties, for the launch of the French iPad. The rigid relational database technology was causing inter-department rifts, as the editorial business solutions built on a relational platform were not flexible enough.

The bank's clients expected to easily access up-to-the-minute content in whatever format they chose. The bank sought a system that could scale with clients' constantly changing requirements, especially the growing demand for mobile delivery and alerting. The bank's legacy system (based on Oracle and FAST) did not provide analysts with the ability to search and access previous reports at a component level. Adding that capability, which would facilitate reuse of content, and thus improve efficiency as well as information quality, was a significant undertaking. The existing system also did not have the flexibility to quickly add new features. It could not meet the demands of a constantly changing, highly competitive market where being first-to-publish was critical to retaining and growing readership.

In a world of shrinking budgets and rising rates of chronic disease, health information exchanges (HIEs) are an essential element of national and regional efforts to improve healthcare and contain costs. HIEs establish an interoperable framework for healthcare providers to share secure patient information across disparate electronic health record (EHR) platforms and other information systems. However, the challenge lies in creating a flexible, reliable, and scalable analytics platform that could ingest large volumes of diverse data types and provide high performance for batch and interactive processing. Since healthcare providers face steep penalties if they fail to secure protected health information (PHI), the solution also required a robust security architecture that could keep pace with interactive performance requirements as deployments grew.

Smartlogic was looking to support the knowledge management needs of a broader audience and provide an efficient and economical platform for customers and partners. They explored a number of cloud deployment options for their Semaphore platform. The solution needed to have full functionality and integration capabilities, enterprise-grade security, high availability and scalability, all while keeping overall costs and environment management low. They aimed to bring together three leading technologies to provide an efficient and economical platform for customers and partners to address security, scale and on-demand requirements.

The Bank, a major player in the derivatives market, was struggling with a legacy system that was slow, inefficient, and unable to deliver real-time alerts to manage market and counterparty credit positions in the desired timeframe. The existing system, based on a relational database, was comprised of multiple installations around the world, leading to duplicated, redundant, incomplete, and inconsistent data. This made risk management fallible and time-consuming, and the Bank recognized the importance of having a real-time global view of its positions.

Mitchell 1, a leader in providing information solutions for automotive professionals, faced challenges with managing and integrating complex data from multiple sources. The company's online manuals were based on a relational database, which was struggling to handle the growing volume and disparate types of data. The company saw an opportunity to integrate its data and turn it into a competitive advantage. However, the increasing data types and volume were making data integration with a relational database time consuming and costly. To maintain its industry leadership and continue to outpace competitors, Mitchell 1 needed to offer up-to-the-minute, relevant, rapidly searchable and comprehensive repair information.

The American Psychological Association (APA) was facing challenges with its existing search engine, Lucene. The search response times were inconsistent, leading to a poor user experience. Moreover, the delivery of content to APA users was slow, taking up to 48 hours to prepare, deliver, and post content on APA websites. This was unacceptable as APA is the authoritative source for this content, and timely online availability is crucial to the authors, readers, and business partners who depend on it.

Fairfax County wanted to upgrade their existing Police Events application. This application allowed the public to search, filter, and map police service calls by type, location, and date, and access preliminary police investigation data from Fairfax County's 9-1-1 call center. With more than a half-million data points from all over the county, the data overwhelmed the existing Oracle relational database. The county's GIS database contained data from more than 800 different sources, including libraries, schools, hospitals, most of which have unique schemas, making it difficult to sort through and access. Only a small amount of GIS data was available via the previous Fairfax GIS search application, which was JavaScript-based. This legacy application did not provide the flexibility to constrain searches to specific data layers without customization. The existing database had to search layer by layer, one column at a time, making searches slow and difficult. The county has approximately 800 distinct layers in its GIS database and most have their own specific data schema. For non-GIS staff this can make sifting through all of the information very difficult. The ability to add new datasets as they were identified and provide a set of tools for loading and enrichment was tough. Also, returning information in a variety of formats including JSON, XML, and KML was not possible, and data needed to be RESTful to keep in accordance with current practices.

Fairfax County, one of the largest counties in the nation, was facing a challenge in responding to the high-volume of requests from their constituents regarding property history information. The data was stored in disparate databases and file systems and in different formats, making it difficult to effectively search. The county needed a solution that would integrate this data, maintain its integrity, and make it easy to access. They also needed a solution that would fit with their existing IT infrastructure, and have a low and predictable total cost of ownership.

IOP Publishing, a global publisher and distributor of scientific publications, was facing challenges with its legacy solution built on Oracle and Hibernate. The eight-layer stack required a multi-step process to take unstructured content, like journal articles, from ingest to delivery. This often resulted in multiple versions of content in different locations that created a mismatch between versions. Valuable time was lost tracing back to the source; this directly affected the scientists and other users who rely on the IOP platform as a vital research tool. To provide the best service to its user community, IOP Publishing wanted to overhaul its biggest platform, IOPscience. With a limited budget and a mandate to switch systems with no downtime, the IOP Publishing team knew it couldn't rebuild the system from the ground up with traditional technology.

IQ Solutions, a full-service public-health communications and information firm, was facing a challenge in disseminating content from various sources and in disparate formats for the Substance Abuse and Mental Health Services Administration (SAMHSA). The company had to adapt to new technologies and meet mandates from the White House and other government agencies to use shared platforms, focus on digital formats, and expand their reach when it comes to content dissemination. The company was also under pressure to reduce costs associated with content development and delivery. To meet their goals of modernizing and streamlining their business while expanding SAMHSA's reach, IQ Solutions decided to put together a new content repository.

The Press Association (PA) was originally made up of multiple companies brought together to form PA Group. With such diversification in different types of content and services, more technical cohesion and strategy was needed. PA needed a new, centralized solution that could handle XML, while also merging a large volume of structured and unstructured content in a useful and efficient way. Speed and efficiency were also key considerations. With previous solutions, the company was spending 70% of their time managing and manipulating data before they could get any use out of it. It was important that the new platform be fast and flexible.

In 2012, Wiley partnered with the American Geophysical Union (AGU) to create a flagship academic resource in earth, space and environmental sciences. This required integrating all of AGU's data and functionality into the Wiley Online Library. The project posed several challenges. Firstly, the project had an aggressive timeline, with project goals needing to be completed in four months, with just six weeks for actual development. Secondly, Wiley had to migrate existing content, comprised of 160,000 articles from 21 journals, 33 virtual journals based on AGU index terms and 743 special sections in addition to customer, user, product, license and alert data. The project also required a search engine capable of working quickly and supporting content search across a range of index terms and special sections. The articles and journals were not standardized, many articles had no issue or page numbers, and journals came in an assortment of formats, including issues with multiple subsections or special editions. Internally, the production workflow from author contribution to the editorial system had to be well integrated and adapted to the new platform and yet function as it did before.

The Incorporated Council of Law Reporting for England and Wales (ICLR) faced stiff competition as other publishers began to move their data online in the 1990s. At the time, ICLR was lacking a digital delivery platform or experience in digital delivery, and therefore chose to outsource its content through a series of licensing deals with third-party online providers. However, as online customers and information demand increased, ICLR decided to create its own digital delivery platform in-house in order to regain control of its content, and reign in costs. ICLR launched a custom digital platform in 2011, providing basic search and content delivery and was primarily based on duplicating the printed versions of reports. This platform was difficult to maintain, couldn't be changed easily and presented an inconsistent user experience. In addition, ICLR wanted to do more than just deliver English case law: The organization wanted to become the primary source for researching and accessing English case law. To address these issues and add the ability to quickly create new products and services, ICLR needed to create a new, robust, fast, and user-friendly platform.

The global manufacturing company was dealing with a data explosion due to acquisition, organic growth, and product expansion. The company had hundreds of disparate ERP systems, each with massive volumes of data, multiple ERP vendors, and multiple versions of ERP software, which added to the complexity. The company was using relational databases and ETL tools to integrate the data, but the process was slowing innovation and sometimes resulted in data quality issues, which could lead to negative customer experiences. The company needed to integrate its ERP systems, eliminating costly data siloes and time-consuming ETL processes in order to gain a 360-degree view of its data. The organization embarked on a massive integration project using traditional relational tools. After three years, the company had only integrated four of the hundreds of ERP systems. At this pace, the entire system would not be fully integrated for decades, resulting in the organization missing out on numerous opportunities and exposing the company to potential risk.

The bank's management recognized the importance of having a real-time comprehensive global view of its positions in order to quickly and cost-effectively make smart business decisions. The existing system, based on a relational database, was comprised of multiple installations around the world. The legacy system was not fast enough to respond to growing requirements, and the data integration was simply not feasible for the bank's needs. For example, the existing system was unable to deliver real-time alerts to manage market and counterparty credit positions in the desired timeframe. The team was tasked with preventing errors, reducing time and complexity, and creating an agile infrastructure.

Hannover Re, the third-largest reinsurer in the world, wanted to add a powerful Management Information (MI) aspect to its reinsurance service portfolio. The company aimed to help its clients access all relevant data in a 360-degree view to analyze risks and make risk-adequate decisions. The challenge was to integrate and manage massive amounts of diverse data produced at the point of sale, which is heterogeneous and contains unstructured content such as lab reports and scanned images. Hannover Re also wanted to reduce administrative burdens with a fully integrated system that could provide all functionality in a consistent way without adding operational complexity. The company also aimed to deliver an exceptional user experience by making all data points fully available via integrated search to analyze risk profiles, uncover previously hidden risk patterns and offer options to clients to improve their business operations. Lastly, due to the sensitive nature of the data, security was a significant concern for Hannover Re.

ALM's management decided to grow from a legal content provider to a multi-industry information provider through acquisitions and new content, which would help the company maintain leadership in an increasingly competitive landscape. To best manage changes in the marketplace, enable the company to grow, and rapidly reap the benefits of acquisitions, the ALM team needed to invest in the integration and management of customer data. In two years, the company acquired four companies, and their systems and daily data feeds needed to be integrated into existing ALM infrastructure. On the content side, ALM wanted to better create and store content, but also to connect that content with customers, thereby increasing its value and better serving both customers and advertisers. The ultimate goal was to deliver adaptive news and targeted ads tailored for each individual user.

Thieme Publishing Group, a medium-sized company in a competitive landscape, needed a database solution that allows for agile data management, storage, enrichment, and delivery. They had millions of medical and scientific documents, images, and graphics, all of which were available in different formats that existed in many data silos. This situation made product development and deployment labor-intensive. In order to be successful against international competition as well as free search services like Google or Yahoo, Thieme needed to quickly deliver a platform that was capable of making highly annotated, enriched content easily searchable and accessible to a wide range of users. Because there was no standard platform or language to develop on, the heavily siloed nature of Thieme's data and products made scaling a very cumbersome and elaborate process. IT had to start from scratch to create each new product.

The U.S. National Archives and University of Virginia (U.Va.) Press faced the challenge of transforming Founders Online, a scholarly tool traditionally accessed by a limited number of researchers, into a national resource capable of serving the public at large. The original platform was not designed to handle concurrent users at scale. The old platform's performance under load deteriorated quickly, and testing suggested the architecture would only support 100 concurrent users. The design imperative for the original system was to preserve the look and feel of print volumes, which had a minimal impact on smaller files, but longer, outlying collections stressed the system. The old platform recreated each search from scratch, putting an unnecessary burden on computing resources while slowing the system down. To make matters more complex, the organization had the equivalent of just 1.5 full-time programmers to devote to the project.

Warrior Gateway, a nonprofit organization, aimed to create the G.I. Network as a single source of truth connecting veterans to the right support services. However, the existence of so many resources alongside complex Veterans Administration and Department of Defense regulations has created an overwhelming maze that vets must navigate to find basic information about healthcare, education, housing, and employment resources available to them. The challenges included bringing together data from multiple sources, launching resources quickly, and custom application development with limited internal resources.

Springer, a leading scientific and medical publisher, faced challenges with its online content delivery platform, SpringerLink. The platform, which hosts over 8.4 million scientific documents, had become the core of the company's business, accounting for two-thirds of total revenue. However, Springer was relying on a third-party technology provider to power SpringerLink. As the importance of online content grew, Springer needed more control and flexibility over its content distribution. The company also faced a tight deadline to find a new solution, as its contract with the third-party provider was ending soon. Additionally, with the explosion of online content, readers were expecting better functionality from content providers, including faster search times.

The Federal Aviation Administration (FAA) required the Emergency Operations Network (EON) to handle multiple document types and feeds, and make sense of the information as quickly as possible. They needed a solution that provided a backend repository and search for unstructured information-tweets, weather reports, news stories, emails, SharePoint documents, geospatial data, and more that get pulled together for decision-making; and integration with in-house custom built as well as commercially available applications to analyze and share the data rapidly. The FAA AEO-400 team tried to implement EON with traditional SQL technologies, but found it impossible due to the complex mix of data and unplanned informational needs that are standard characteristics of this environment.

NEWBOOKS Solutions and MVB wanted to create a new revenue stream and open up a new market by digitizing published new books, previews, and an ordering system with a new service, VLB-TIX. The developers at NEWBOOKS required an infrastructure that could integrate and manage a wide range of data sources and formats, offer the highest levels of customer service, support future growth with enterprise features, and provide agility. They needed to integrate data from its database and the VLB database, which together contained titles from 23,000 publishers and diverse data types such as bibliographic book metadata, and shared buyers' notes. They also had to incorporate a large number of publishing partners into the process with different workflows and various data formats.