What may sound like a step backward is actually huge leap forward for us. I've thrown out the fancy shmancy Ruby/Rake stuff and switched the build system to pure CMake. The previous build system was really comfy: it was capable of parallel, out-of-source builds, could handle code generation, run all unit-tests, did not require the user to manually list all source files... I could probably list another dozen features that I loved about it, but it had a fatal flaw: it did not run on big iron. Most supercomputers don't have Ruby, Ruby gems, Rake, Doxygen and whatnot installed. Of course one could work around this by installing all these packages in his home dir, but that's tedious and cumbersome. Actually, building LibGeoDecomp has always been a hassle. LibGeoDecomp's origin is MuCluDent (Multi Cluster Dendrite [sic]), our first dendrite simulation software. We were a team of young CS (grad) students and were toying with extreme programming. We were looking for a HPC project. Back then few people were interested in parallel computing as growing kept single thread performance growing exponentially. Finally, we found users. Material scientists.

We started off with writing our own makefiles, which was a major PITA when the library was young and we were adding/removing/moving files like frenzy. So we switched to Rake, as we thought this would allow us to script all the largely mechanical parts of the build system. But, while this was true, it did not save us any time. Instead, maintaining the build system was something we spent more and more time on. More than on developing the simulation. For a number of reasons the team disbanded (one being the poor academic performance of our project: writing a makefile doesn't get you papers published). At that time I was looking for a Ph.D. project. I loved what MuCluDent did and thought that there ought to be other applications for the code, only with a different model.So I went ahead and separated the parallelization from the (then dead) model. LibGeoDecomp was born.

But the build system continued to piss me off. Compilation turned out to be more complicated than just calling GCC a couple of times. From other projects I knew that qmake did reasonably well at compiling even larger projects, so I threw out a giant load of Rake code (including some monkey patches. Those were sick!) and started to generate qmake scripts from Rake. It didn't take long for qmake to show its real face, and I was back, looking for alternatives. CMake appeared to be attractive. I had decided against it previously as its syntax alienated me, but a number of larger projects (KDE, Boost) had just started to use CMake, and qmake really did not handle cross-directory dependencies well. With my limited knowledge of CMake I didn't realize its full potential. My solution involved, very similar to the previous qmake approach, creating CMake scripts from Rake. This was the status quo until now. I got it all tuned perfectly. Every dependency triggered just the right builds, every unit test was run at the correct occasion. It felt so perfect. Despite the clumsy combination of tools. Perfect.

I began to realize how bad my solution was when I tried to build LibGeoDecomp on RRZE's woody. No Ruby, no Rake, no nothing. My code was a diva. Working around these problems only meant that in fact my code did not support most major machines. This even went as far as being rejected for testing the library on JAMSTEC's Earth Simulator, partly because it does not support Ruby. In the past weeks I've finally made the cut and threw out all the Rake stuff. Only CMake and Boost are now required to compile LibGeoDecomp. Replicating the behavior I had initially coded in Ruby/Rake in a CMake script wasn't easy and I'm sure that there are still tons of bugs left, but it's better this way. I'm sure we'll get the wrinkles smoothed out in the next couple of days. Starting of revision #227 builds should succeed and run all relevant tests. If not: please see the doc and FAQ for clues, or send me an e-mai right away.

Rake's dead, baby, Rake's dead! (Queue Harley sound here.)

-Andreas

News archive »

last modified: Wed Oct 26 16:51:19 2016 +0200