|144 Core Raspberry PI 3 Cluster|
Another very important point is that this mini data center is a playground. We can try out new things without any risk, and at low cost. What if we want to test a service with 20 instances running on multiple hosts? No problem - it’s up and running in a few minutes. No starting of VMs, no starting of any cloud instances that charge by the minute – we can just go and try it out. This gives us the chance to try out new things, play around, and mess things up without any risk. For us, this is really important, as we always want to be able to move forward very quickly. Each month, we spend an entire day building stuff that we engineers want to build and find useful. Some of it needs to be hosted somewhere, and these days, we want to achieve things quickly. Our mini data center is an ideal place to try out and house our ideas on these days (we call them “Crazy Fridays,” even though they held are on a Thursday).
There is another very practical aspect to having a mini data center in the office. Big data, scalable architectures, resilient software design, automatic fail over, and leader election – all of these areas affect software, and that software needs to be tested to respond to cases of errors and other circumstances, like network failures. Have you ever tried to get into a data center and plug in a cable? Slowed down a switch on purpose? I think you can maybe do it once, before you’re put on the blacklist of people not allowed to be there. In your personal mini data center, however, you are free to do whatever you want and to test your software and algorithms in any way you choose.
Furthermore, we use the cluster to host internal applications, like information screen back-ends. Not reliable enough? Running an application on this cluster with a replication factor of ten provides enough room for errors (OK, so there’s no redundant power supply and so on, but that is out of scope and not needed for these kinds of applications). Hosting your own applications on your own mini data center is just fun.
Last but not least, there was a lot of hardware to build, such as the actual blue case shown in the picture and the acrylic glass parts. We used a CNC machine and a laser cutter to get all the parts in place. Building actual, physical things is a lot like building software; there are problems that need to be solved. Also, it broadens the minds of people who spend most of their time just creating virtual things when they are involved in building something they can touch.
If you want to build your own, all plans are provided in our github repository.
So, what do you need to build it?
36 x Raspberry PI 3
36 x 64 GB SD card
36 x CAT6 network cables 1.5m
36 x USB cable (for power supply)
144 x M2.5 screws
144 x M2.5 nuts
6 x High power USB hubs (like this one)
1 x 48 port switch
Building the Hardware
The housing was made of medium-density fiberboard and, of course, you need acrylic glass for the other parts of the tower.
For the bottom housing that contains all the networking and power supplies, we just cut out a lot of rings and glued them together so that we had the shape we wanted.
The acrylic glass parts were cut using a laser cutter:
Engraving our logo
Vertical plate holders
Our rings to hold the raspberry PIs
After that, we just had to get the housing painted ...
... and try to fit everything in it ...
and you are done! It won't take you more than 20-30 hours.
Nevertheless, to make this cluster really usable and easily accessible for everyone who wants to play around with it, we decided to go for a software setup that enables us to deploy any application with just one line of code, including fail over, scaling, etc. We’ll use Docker Swarm to power it.
This software part will be covered in the next installment of this blog article.