Gefördert durch die



ICP-Kolloquium - Vortrag im Rahmen des SFB 716

Veranstaltungsdatum:  21. Dezember 2017 13:00 Uhr
Referent/in:  Dr. Axel Arnold


Dr. Axel Arnold,  


And now to something ELSE: Real Time Offer Processing @

Abstract is one of the largest price comparison websites in Germany. How do we develop and operate software for our production systems, and how do physicists contribute to the company's success?

I will both talk about the overall architecture and selected components, to explain how we built a robust software system that can run for weeks without incidents, but at the same time allows for fast adoption of new features. For, millions of offers need to be clustered into products every day, with a considerable amount changing every few hours. I give an overview of the infrastructure we use to process and enrich our offer and product data sheets, both manually and automated. Then, I concentrate on the eventdriven processing system which renders the product data sheets in near real time. It consists of a number of services written in Python, a Redis based queuing system and an Elasticsearch cluster for storage. I talk about our experiences and the challenges we were facing building this scalable processing system, and how it integrates into our existing infrastructure.


Veranstaltungsort:  Campus Vaihingen
Seminarraum 1.079
Allmandring 3