| Abstract |
billiger.de is one of the largest price comparison websites in Germany. How do we develop and
operate software for our production systems, and how do physicists contribute to the company's
success?
I will both talk about the overall architecture and selected components, to explain how we built
a robust software system that can run for weeks without incidents, but at the same time allows for
fast adoption of new features. For billiger.de, millions of offers need to be clustered into
products every day, with a considerable amount changing every few hours. I give an overview of the
infrastructure we use to process and enrich our offer and product data sheets, both manually and
automated. Then, I concentrate on the eventdriven processing system which renders the product data
sheets in near real time. It consists of a number of services written in Python, a Redis based
queuing system and an Elasticsearch cluster for storage. I talk about our experiences and the
challenges we were facing building this scalable processing system, and how it integrates into our
existing infrastructure.
|
