Migrating a 100-submodules monolith from Maven to Gradle

Willhaben started out as one big, monolithic software project. Parts of this monolith are now older than our youngest colleagues.

Some years ago, we started the still ongoing process of splitting the monolith along team lines.

However, today the monolith (or what is left of it) is still an important part of Willhaben’s software landscape. It is made up of a little over 100 submodules and was until recently built with Maven.

A big disadvantage of working with the monolith were the build and the CI/CD times. To merge a merge request, one needed to wait 20 minutes for the pipeline to finish — per push. This was especially annoying when only a single test or a format was changed.

This was the point where I started to think about migrating to Gradle.

Disclaimer: I am myself very proficient with Gradle, less so with Maven. I could be regarded as a “Gradle ninja”.

“There is no way it is viable to migrate such a complex project!”

And that’s what I thought, too. Our Maven build had grown over time and had an awful lot of custom hacks, unclean build processes. I thought that there was no way to do this in a reasonable time.

But the frustration with both local and CI pipeline build times got the better of me, and I tried a timeboxed attempt to get the monolith to compile with Gradle. No testing, no packaging, no publishing — just compiling.
And to my surprise, I got the task done in around 10 hours or work!! The biggest helping factor was gradle init, which did the initial conversion of all 100+ Maven POM files to Gradle, spitting out surprisingly well working Gradle code. The only thing that needed to be fixed were a few dependencies, pinning versions and excluding other dependencies.

This was the point where I proposed a full-blown migration to our architecture team and to the other developers. The proposed 1-month timebox to do a full migration was approved.

I cannot overstate how integral the gradle init command was. It did 99% of the initial work, even adapting some of our workarounds from Maven and solving them “as clean as possible” with Gradle.
For projects that don’t have over-complicated Maven builds, it can easily be possible that Gradle init produces a fully working Gradle build, only missing 3rd party plugins.

How we migrated a live software to a new build system

A team of 2 developers were given 2 months to do the full-blown migration. The goal was that after 1 month the CI pipeline would compile, test and deploy with Gradle. Maven should not be involved in any CI step.

Since this monolith receives multiple commits/merges to its main branch per day, it was important that the migration period would not cause any problems in the pipeline. That’s why we opted to create the Gradle build in parallel to the already working Maven build.
Missed changes to the Maven build would be ported to the Gradle build as the last step of the whole project.

Rather than doing the migration in a big 50 k LoC merge request, individual features were merged in separate MRs like “include NPM frontend build”, “Make WAR deployable local/from IDE” and so on. This way it was easy to track progress and manage or review work.

This approach also allowed us to monitor our build performance gains and losses. On merges to the main branch we executed both the Maven and Gradle build (ignoring the Gradle build and test results) and then compared the build times.

The majority of the work was to get all unit and integration tests running again. There was test (and some production) code that relied on Maven specific behaviour. This was mostly related to resource access (like Paths.get(“target/resources/someImage.png”)) and the way that Gradle resolves dependency versions being different from Maven.

Problems, roadblocks and workarounds

Our Maven build contained quite a few hacks and questionable things, like for example manually copying resources into a checked in src/main/java folder.

The automatically translated pom.xml -> build.gradle.kts files contained code that one would not write in Gradle. Compared to Maven, Gradle is concerned about where dependencies transitively leak into at compile- and runtime, as well as about deterministically resolving dependencies.

• Every Gradle dependency is declared in the api scope since this is the default behaviour of Maven. But in Gradle you should not do that since api is compile-time transitive, which is not normally desirable.
• In Maven you can use test-jar dependencies which pull in the test sources (without its dependencies) of another module. Even Maven admits that is undesirable (see Apache Maven: test-jars the easy vs preferred way). Gradle does not provide this functionality by default.
  With Gradle this functionality was migrated by gradle-init automatically by writing a workaround. It makes it easy to have a 1:1 copy of our Maven project.
  Ideally, the Gradle way to do this would be test-fixtures. We did not migrate the old modules to test fixtures but we made it clear to not use the testJar hack in the future.

Besides, the only other problem was that Gradle picks the highest version number of a dependency, whereas Maven picks the first version it encounters.
Changing the fundamental dependency resolution in Gradle would be way too complicated, so we had to pin all versions using an enforcedPlatform.
This had the nice benefit of being similar to the well known Maven <depenencyManagement> section.

Thanks to the use of Gradle conventional script plugins, we were able to simplify the build.gradle.kts files a lot, so that they only contain dependencies and module specific logic.

Again, we were surprised how little issues we ran into and how well we were able to stick to our time frame. All but one plugin that we used in Maven had a Gradle alternative — alto, sometimes being an open source community plugin rather than maintained by the company the plugin is made for.

Result and Aftermath

Since Gradle is in many aspects quite different from Maven we expected that developers who are only used to Maven would have problems getting comfortable working with Gradle.

Therefore, we prepared two developer presentations where we kept everyone up-to-date and where we gave an introduction to Gradle

After the first wave of expected questions (“My IDE doesn’t recognize Gradle” …), only little help for the teams was needed. After 2 weeks, everyone was smoothly working with Gradle, and we had close to zero Gradle-related questions ever since (despite the most common complaint about Gradle being that it is too complicated!).

Was it worth it?

Yes, definitely.

The numbers do not leave much room for interpretation. Here are the most relevant ones:

• Local Build time: reduced by between 40% and 95% (depending on changed modules)
• Reduced lines of code in build files from 25k to 5k

Pipeline times (Maven):

• Build: 9:46 (53 job average)
• Integration Test: 16:32 (64 jobs average)
• Min: 12:43
• Max: 22:01

Pipeline times (Gradle):

• Build: 3:20 (20 job average)
• Integration Test: 7:32 (17 job average)
• Min: 1:38
• Max: 18:12

This especially shows when doing a simple change (no or only a few source files), the pipeline only takes ~1:30 min (GitLab overhead: 1:20 min).

Also, it didn’t cost a lot of developer time; there were no major hiccups in the process; and it was well received by the developers. Gradle is now in place since February 2024 (since 4 months at the time of publication).