September 20, 2017

Monitoring metrics using StatsD

With more than 4.5 million pubic ads and an incredible number of user interactions, we have a huge amount of events to process each day. Events like new incoming content, user registrations, messages, notifications — you know, events for our business of bringing buyers and sellers together.

For this, we run dozens of micro services behind the scenes, small applications that do nothing else than processing events, applications that don’t provide any user interface. Monitoring such services is more than just checking if they are up and running, it’s more than being notified when the memory consumption grows. Usually we also want to know what our application is doing and how it is performing.

August 30, 2017

​ When Hystrix Cannot Save Us


Hystrix with Spring Boot in 2 Minutes


Today, the creation of web applications has been made really easy with frameworks like Spring Boot. Let's say we want to be sure that our application remains responsive even if a third party system does not answer our requests. Here’s how we can solve this problem with Spring Boot in some code snippets coming directly from our payment component:

The EnableHystrix annotation does the following, according to its JavaDoc:
All it does is turn on circuit breakers and let the autoconfiguration find the Hystrix classes if they are available (i.e., you also need Hystrix on the classpath).
Apart from that, we only need to add the HystrixCommand annotation to the methods, which we can cancel, in the case of a timeout or error:

August 17, 2017

Quick Note: ADFS Authentication for Microsoft Dynamics 365

This is a quick note about a problem that took me several hours to figure out because there is few helpful information out there. So if you happen to have a Microsoft Dynamics 365 system and want to fetch an authentication token from ADFS, read on.

The Problem

Microsoft Dynamics 365 is a tool to manage customer relations (CRM), and we use it to track all communication with our B2B customers. The problem at hand was to feed it some information about customer from another system (written in Java), and thankfully, Dynamics has a webservice, so this should be easy.

May 24, 2017

Reactive Java Performance Comparison


This blog post covers the basic concepts of reactive programming and presents a comparison of the performance of a standard Spring MVC WebApp with Cassandra as data layer to a reactive one.

Reactive programming has become a popular topic in recent years. With the upcoming Spring 5 release, reactive programming is finally emerging into the mainstream. Reactive programming should improve the performance of I/O-heavy applications. A regular standard Spring MVC application typically runs in an application server like Jetty or Tomcat on a servlet stack. Each request is bound to a thread that is passed through the servlet container to the user code before it finally gets some data. Normally, the majority of these operations are I/O-bound. The main disadvantage of this model is that the allocated resources (threads) are waiting for these I/O operations to complete, which can take the majority of the thread time. In the meantime, these resources cannot handle any further requests. With reactive programming, the program's code itself is no longer in charge of resource allocation, but instead provides callbacks that are invoked when resources (e.g., new data from the database) are available.

May 15, 2017

From Batch to Stream Processing

At willhaben, a lot of data is passed around between components. At peak times, we handle over 100,000 events every minute. Saving this huge amount of data does not create bottlenecks on modern hardware but carrying out proper (real-time) analysis of data on this scale can become a challenge.

These events sum up to terabytes and getting real-time statistics from them can be really valuable for monitoring either IT infrastructure or business processes. Big Data comes to mind, a buzzword that covers a lot of topics. One of these concepts, often seen in this context, is stream processing. This is a very extensive topic so this post will only talk about some of the ideas behind streams and stream processing, and not go into detail. There are several frameworks for stream processing like Apache Spark, Apache Flink, or Kafka Streams. Each of them has its own advantages and disadvantages. A comparison between them is beyond the scope of this post. Here we will focus on Kafka Streams as it is the simplest one and covers our user cases. We will first cover the basic concepts and then look at the processing pipeline implemented at willhaben.

March 02, 2017

Why Appium is the better option for us


For our monthly “crazy programming Friday”, we decided to try out Appium and maybe get a new testing framework for our end-to-end tests on iOS and Android, which would replace the current system. The system in place is Calabash, which is written in Ruby and, on top of that, feature files in Gherkin and Cucumber. Calabash has the nice ability to be written for both iOS and Android in the same language, so this was something we needed for our new tool as well. Luckily, Appium provides exactly this.

January 23, 2017

Kotlin vs Java: An Introduction

Kotlin is a statically typed programming language for the Java Virtual Machine (JVM), Android, and web browsers crafted by the makers of the famous IDE IntelliJ[1]

The Willhaben Android App relies on traditional Java code and a growing amount of Kotlin code. Right now, we are in a transition period of migrating our legacy Java code to Kotlin. As Java and Kotlin both work on the JVM, they can be used in conjunction within the same codebase.

During my work with Kotlin, I stumbled upon some pretty interesting language details and features, some of which I want to explain briefly in the next few lines and in an upcoming blog post. I’ll also give you a comparison of how the same results can be achieved in the Java world. In this post, I will focus on two features.

January 18, 2017

Building a 144 core Raspberry PI 3 Cluster

A couple of months ago, we built a cluster of 36 Raspberry PI 3 (4 cores each, so 144 cores in total), resulting in 2304 GB of storage and 36 GB of RAM. There are many tutorials about building small Raspberry PI clusters available online, but hardly any of them cover what you can do with them in a business environment. So, why did we spend a lot of time to get this thing to work?

144 Core Raspberry PI 3 Cluster
First, our cluster is like a data center in a box. It just has a network cable and a power cable, so it actually is “plug and play.” Everything else is contained in the cluster shown in the picture. We had to build the hardware, the electronics, and to create the networking, routing, etc. We asked a team of software engineers and operations specialists to work closely together on this project. Participating in a project like this gives each of the members insight into other areas that are usually not part of their daily work; software guys learned more about networking and how to set up the proper administration of servers, and operations guys learned more about the software side.

January 02, 2017

How to integrate Docker into your build process

In this blog post I want to show you how to install Docker on your system and integrate it into your build process. So that it will be easy for you to run your tests against a running Docker container, we will add a Postgres database to the Docker container. This will enable you to test them against a fresh and clean database every time you run tests.

Docker Installation

For general installation instructions you can visit the Docker homepage and follow the steps there. In my example I will install it on a computer running Linux Mint, which is a derivative of Ubuntu, so you can follow the Ubuntu installation steps.