How we built BI on Clickhouse with row-level security in Deutsche Bank Technology Centre
Customer Company Size
Large Corporate
Region
- Europe
Country
- Germany
Product
- ClickHouse
- Tableau
- RShiny
- Kafka
- ABACUS
Tech Stack
- Spark
- Alpakka
- SQL
- ETL
Implementation Scale
- Enterprise-wide Deployment
Impact Metrics
- Digital Expertise
- Productivity Improvements
- Customer Satisfaction
Technology Category
- Analytics & Modeling - Big Data Analytics
- Analytics & Modeling - Predictive Analytics
- Application Infrastructure & Middleware - Data Exchange & Integration
- Application Infrastructure & Middleware - Database Management & Storage
Applicable Industries
- Finance & Insurance
Applicable Functions
- Business Operation
- Quality Assurance
Services
- Data Science Services
- Software Design & Engineering Services
- System Integration
About The Customer
Deutsche Bank Technology Centre is a part of Deutsche Bank, one of the world's leading financial service providers. The Technology Centre focuses on developing and implementing cutting-edge technology solutions to support the bank's operations and services. With a global presence and a diverse range of financial products, Deutsche Bank serves millions of customers worldwide. The Technology Centre plays a crucial role in ensuring the bank's technological infrastructure is robust, secure, and capable of handling the complex demands of the financial industry. The centre is responsible for managing data, developing software solutions, and ensuring compliance with regulatory requirements.
The Challenge
Deutsche Bank Technology Centre faced significant challenges in managing and analyzing data within its Investment Bank division. The natural data silos and the need for a robust Business Intelligence (BI) system were evident. The existing data warehouse solutions were either too expensive or too slow, such as Vertica and Hive, respectively. Additionally, the bank required a data-driven access control mechanism that could provide record-level granularity and full access to SQL, while reusing existing bank-wide access rules. The challenge was to find a solution that could handle heterogeneous data from 72 different data sources and systems, manage over 100 ETL jobs, and provide a seamless user experience across various UIs.
The Solution
To address these challenges, Deutsche Bank Technology Centre implemented a combination of Spark, ClickHouse, and Tableau/RShiny for their BI needs. In 2017, they adopted Spark for its powerful data processing capabilities and ClickHouse for its high-performance columnar storage. Tableau and RShiny were used for data visualization and reporting. By 2018, the solution evolved to include Alpakka for data integration, Kafka for real-time data streaming, and a Web UI for user interaction. The Access-Based Access Control (ABAC) system was integrated to provide data-driven access control with record-level granularity. This comprehensive solution allowed the bank to track changes, manage investment planning, and improve client interaction quality. The use of ClickHouse enabled fast query performance, while Spark and Kafka ensured efficient data processing and real-time analytics.
Operational Impact
Quantitative Benefit
Case Study missing?
Start adding your own!
Register with your work email and create a new case study profile for your business.
Related Case Studies.
Case Study
Real-time In-vehicle Monitoring
The telematic solution provides this vital premium-adjusting information. The solution also helps detect and deter vehicle or trailer theft – as soon as a theft occurs, monitoring personnel can alert the appropriate authorities, providing an exact location.“With more and more insurance companies and major fleet operators interested in monitoring driver behaviour on the grounds of road safety, efficient logistics and costs, the market for this type of device and associated e-business services is growing rapidly within Italy and the rest of Europe,” says Franco.“The insurance companies are especially interested in the pay-per-use and pay-as-you-drive applications while other organisations employ the technology for road user charging.”“One million vehicles in Italy currently carry such devices and forecasts indicate that the European market will increase tenfold by 2014.However, for our technology to work effectively, we needed a highly reliable wireless data network to carry the information between the vehicles and monitoring stations.”
Case Study
Safety First with Folksam
The competitiveness of the car insurance market is driving UBI growth as a means for insurance companies to differentiate their customer propositions as well as improving operational efficiency. An insurance model - usage-based insurance ("UBI") - offers possibilities for insurers to do more efficient market segmentation and accurate risk assessment and pricing. Insurers require an IoT solution for the purpose of data collection and performance analysis
Case Study
Smooth Transition to Energy Savings
The building was equipped with four end-of-life Trane water cooled chillers, located in the basement. Johnson Controls installed four York water cooled centrifugal chillers with unit mounted variable speed drives and a total installed cooling capacity of 6,8 MW. Each chiller has a capacity of 1,6 MW (variable to 1.9MW depending upon condenser water temperatures). Johnson Controls needed to design the equipment in such way that it would fit the dimensional constraints of the existing plant area and plant access route but also the specific performance requirements of the client. Morgan Stanley required the chiller plant to match the building load profile, turn down to match the low load requirement when needed and provide an improvement in the Energy Efficiency Ratio across the entire operating range. Other requirements were a reduction in the chiller noise level to improve the working environment in the plant room and a wide operating envelope coupled with intelligent controls to allow possible variation in both flow rate and temperature. The latter was needed to leverage increased capacity from a reduced number of machines during the different installation phases and allow future enhancement to a variable primary flow system.
Case Study
Automated Pallet Labeling Solution for SPR Packaging
SPR Packaging, an American supplier of packaging solutions, was in search of an automated pallet labeling solution that could meet their immediate and future needs. They aimed to equip their lines with automatic printer applicators, but also required a solution that could interface with their accounting software. The challenge was to find a system that could read a 2D code on pallets at the stretch wrapper, track the pallet, and flag any pallets with unread barcodes for inspection. The pallets could be single or double stacked, and the system needed to be able to differentiate between the two. SPR Packaging sought a system integrator with extensive experience in advanced printing and tracking solutions to provide a complete traceability system.
Case Study
Transforming insurance pricing while improving driver safety
The Internet of Things (IoT) is revolutionizing the car insurance industry on a scale not seen since the introduction of the car itself. For decades, premiums have been calculated using proxy-based risk assessment models and historical data. Today, a growing number of innovative companies such as Quebec-based Industrielle Alliance are moving to usage-based insurance (UBI) models, driven by the advancement of telematics technologies and smart tracking devices.
Case Study
Enhancing Security and Compliance in Remitly's Global Money Transfer Service with Fastly
Remitly, an online remittance service, was faced with the challenge of securing its proprietary global transfer network. The company needed a security solution that could meet PCI requirements and protect customers' sensitive transactions through its mobile application. The solution had to be capable of defending against new and emerging attack types without impacting performance. Remitly also had to deal with irregular traffic patterns, such as a sudden spike in account transfers from a small network segment on the Pacific coastline of South America. The company needed to determine in real time whether such traffic indicated an attack or valid requests. A traditional web application firewall (WAF) would not be able to distinguish this traffic, potentially leading to customer frustration if the IP was blacklisted.
