May 15, 2017

From Batch to Stream Processing

At willhaben, a lot of data is passed around between components. At peak times, we handle over 100,000 events every minute. Saving this huge amount of data does not create bottlenecks on modern hardware but carrying out proper (real-time) analysis of data on this scale can become a challenge.

These events sum up to terabytes and getting real-time statistics from them can be really valuable for monitoring either IT infrastructure or business processes. Big Data comes to mind, a buzzword that covers a lot of topics. One of these concepts, often seen in this context, is stream processing. This is a very extensive topic so this post will only talk about some of the ideas behind streams and stream processing, and not go into detail. There are several frameworks for stream processing like Apache Spark, Apache Flink, or Kafka Streams. Each of them has its own advantages and disadvantages. A comparison between them is beyond the scope of this post. Here we will focus on Kafka Streams as it is the simplest one and covers our user cases. We will first cover the basic concepts and then look at the processing pipeline implemented at willhaben.

March 02, 2017

Why Appium is the better option for us

For our monthly “crazy programming Friday”, we decided to try out Appium and maybe get a new testing framework for our end-to-end tests on iOS and Android, which would replace the current system. The system in place is Calabash, which is written in Ruby and, on top of that, feature files in Gherkin and Cucumber. Calabash has the nice ability to be written for both iOS and Android in the same language, so this was something we needed for our new tool as well. Luckily, Appium provides exactly this.

January 23, 2017

Kotlin vs Java: An Introduction

Kotlin is a statically typed programming language for the Java Virtual Machine (JVM), Android, and web browsers crafted by the makers of the famous IDE IntelliJ[1]

The Willhaben Android App relies on traditional Java code and a growing amount of Kotlin code. Right now, we are in a transition period of migrating our legacy Java code to Kotlin. As Java and Kotlin both work on the JVM, they can be used in conjunction within the same codebase.

During my work with Kotlin, I stumbled upon some pretty interesting language details and features, some of which I want to explain briefly in the next few lines and in an upcoming blog post. I’ll also give you a comparison of how the same results can be achieved in the Java world. In this post, I will focus on two features.

January 18, 2017

Building a 144 core Raspberry PI 3 Cluster

A couple of months ago, we built a cluster of 36 Raspberry PI 3 (4 cores each, so 144 cores in total), resulting in 2304 GB of storage and 36 GB of RAM. There are many tutorials about building small Raspberry PI clusters available online, but hardly any of them cover what you can do with them in a business environment. So, why did we spend a lot of time to get this thing to work?

144 Core Raspberry PI 3 Cluster
First, our cluster is like a data center in a box. It just has a network cable and a power cable, so it actually is “plug and play.” Everything else is contained in the cluster shown in the picture. We had to build the hardware, the electronics, and to create the networking, routing, etc. We asked a team of software engineers and operations specialists to work closely together on this project. Participating in a project like this gives each of the members insight into other areas that are usually not part of their daily work; software guys learned more about networking and how to set up the proper administration of servers, and operations guys learned more about the software side.

January 02, 2017

How to integrate Docker into your build process

In this blog post I want to show you how to install Docker on your system and integrate it into your build process. So that it will be easy for you to run your tests against a running Docker container, we will add a Postgres database to the Docker container. This will enable you to test them against a fresh and clean database every time you run tests.

Docker Installation

For general installation instructions you can visit the Docker homepage and follow the steps there. In my example I will install it on a computer running Linux Mint, which is a derivative of Ubuntu, so you can follow the Ubuntu installation steps.

December 30, 2016

JUnit 5! What's new?

JUnit 5 is finally out. I will take this opportunity, to introduce some of its new features. I will point out only the features that I have found most interesting.


The JUnit Team has changed a lot of the annotation names so that it is clearer how and when they should be used. The underlying behaviour is still the same. Methods which are annotated with either @BeforeAll or @AfterAll must be static because each test creates a new instance, and, therefore, it is unclear on which instance they have to be invoked. [1]

December 20, 2016

iOS Deep Linking - Common Challenges and Their Best Attempt to Solve Them


"Deep links" are a powerful mechanism for making web/mobile app content more easily accessible. By automatically navigating to the desired page/screen we are able to improve the user experience dramatically.
Instead of requiring users to click through several pages – starting from the "home screen" – we can point them to a specific location within our app. Usually, a deep link is contained in, for example a push notification, an email, a Facebook advertisement, and so on. Furthermore, we can drive users back to our app from the mobile website with Smart App Banners in the Safari browser. See below for two real world examples. In the next few sections, we will show you what to consider when implementing such functionality on iOS.