September 20, 2017

Monitoring metrics using StatsD

With more than 4.5 million pubic ads and an incredible number of user interactions, we have a huge amount of events to process each day. Events like new incoming content, user registrations, messages, notifications — you know, events for our business of bringing buyers and sellers together.

For this, we run dozens of micro services behind the scenes, small applications that do nothing else than processing events, applications that don’t provide any user interface. Monitoring such services is more than just checking if they are up and running, it’s more than being notified when the memory consumption grows. Usually we also want to know what our application is doing and how it is performing.

August 30, 2017

​ When Hystrix Cannot Save Us

Hystrix with Spring Boot in 2 Minutes

Today, the creation of web applications has been made really easy with frameworks like Spring Boot. Let's say we want to be sure that our application remains responsive even if a third party system does not answer our requests. Here’s how we can solve this problem with Spring Boot in some code snippets coming directly from our payment component:

The EnableHystrix annotation does the following, according to its JavaDoc:
All it does is turn on circuit breakers and let the autoconfiguration find the Hystrix classes if they are available (i.e., you also need Hystrix on the classpath).
Apart from that, we only need to add the HystrixCommand annotation to the methods, which we can cancel, in the case of a timeout or error:

August 17, 2017

Quick Note: ADFS Authentication for Microsoft Dynamics 365

This is a quick note about a problem that took me several hours to figure out because there is few helpful information out there. So if you happen to have a Microsoft Dynamics 365 system and want to fetch an authentication token from ADFS, read on.

The Problem

Microsoft Dynamics 365 is a tool to manage customer relations (CRM), and we use it to track all communication with our B2B customers. The problem at hand was to feed it some information about customer from another system (written in Java), and thankfully, Dynamics has a webservice, so this should be easy.

May 24, 2017

Reactive Java Performance Comparison

This blog post covers the basic concepts of reactive programming and presents a comparison of the performance of a standard Spring MVC WebApp with Cassandra as data layer to a reactive one.

Reactive programming has become a popular topic in recent years. With the upcoming Spring 5 release, reactive programming is finally emerging into the mainstream. Reactive programming should improve the performance of I/O-heavy applications. A regular standard Spring MVC application typically runs in an application server like Jetty or Tomcat on a servlet stack. Each request is bound to a thread that is passed through the servlet container to the user code before it finally gets some data. Normally, the majority of these operations are I/O-bound. The main disadvantage of this model is that the allocated resources (threads) are waiting for these I/O operations to complete, which can take the majority of the thread time. In the meantime, these resources cannot handle any further requests. With reactive programming, the program's code itself is no longer in charge of resource allocation, but instead provides callbacks that are invoked when resources (e.g., new data from the database) are available.

May 15, 2017

From Batch to Stream Processing

At willhaben, a lot of data is passed around between components. At peak times, we handle over 100,000 events every minute. Saving this huge amount of data does not create bottlenecks on modern hardware but carrying out proper (real-time) analysis of data on this scale can become a challenge.

These events sum up to terabytes and getting real-time statistics from them can be really valuable for monitoring either IT infrastructure or business processes. Big Data comes to mind, a buzzword that covers a lot of topics. One of these concepts, often seen in this context, is stream processing. This is a very extensive topic so this post will only talk about some of the ideas behind streams and stream processing, and not go into detail. There are several frameworks for stream processing like Apache Spark, Apache Flink, or Kafka Streams. Each of them has its own advantages and disadvantages. A comparison between them is beyond the scope of this post. Here we will focus on Kafka Streams as it is the simplest one and covers our user cases. We will first cover the basic concepts and then look at the processing pipeline implemented at willhaben.

March 02, 2017

Why Appium is the better option for us

For our monthly “crazy programming Friday”, we decided to try out Appium and maybe get a new testing framework for our end-to-end tests on iOS and Android, which would replace the current system. The system in place is Calabash, which is written in Ruby and, on top of that, feature files in Gherkin and Cucumber. Calabash has the nice ability to be written for both iOS and Android in the same language, so this was something we needed for our new tool as well. Luckily, Appium provides exactly this.

January 23, 2017

Kotlin vs Java: An Introduction

Kotlin is a statically typed programming language for the Java Virtual Machine (JVM), Android, and web browsers crafted by the makers of the famous IDE IntelliJ[1]

The Willhaben Android App relies on traditional Java code and a growing amount of Kotlin code. Right now, we are in a transition period of migrating our legacy Java code to Kotlin. As Java and Kotlin both work on the JVM, they can be used in conjunction within the same codebase.

During my work with Kotlin, I stumbled upon some pretty interesting language details and features, some of which I want to explain briefly in the next few lines and in an upcoming blog post. I’ll also give you a comparison of how the same results can be achieved in the Java world. In this post, I will focus on two features.