The distribution of CADET has always been somewhat tricky. We do provide binaries for Windows, but all other platforms need to compile from source. This often makes installing CADET challenging new users.
However, there are more modern approaches, like packaging CADET with conda-forge or distributing it as a docker image. This is also important, since we are planning to use jupterhub in the future which requires docker images (actually there is already a server running that is hosted by the JSC which we might use https://jupyter-jsc.fz-juelich.de/hub/).
Moreover, it would be very convenient if users could simply run
conda install -c conda-forge cadet
For information about building conda packages, see here.
Tagging @j.rao and @j.hassan who offered to support looking into this topic.
There are some questions I’d like to put forth in order to focus our attentions a bit.
Are we going for Docker or Conda (or some combination of both) ?
Should we consider the conda alternatives mentioned in the talk today: Mamba/Boa/Quetz. Link
Since this is to make things easier on the end-user, we’ll have to consider things not only from an easier development/maintenance perspective, but also their ability to install/use either docker or conda (with or without root priveleges).
I believe that setting up a docker container shouldn’t be that hard, and it guarantees that all the necessary dependencies exist in the system. We can just create a script/Dockerfile that does exactly what we need in a controlled environment. But the user might have to wrestle with permissions for the docker daemon, add themselves to a docker group etc. In the end, something like this should work at the end-user level:
docker run -it cadet/cadet
I’m not proficient with Conda or the challenges involved, but I guess the advantage is that we build and run cadet natively. Also, I just checked and conda-forge lists several versions of all the dependencies we need. So it might be as simple as:
I think both. Most people are used to conda and we need Docker for services such as JupyterHub
Should we consider the conda alternatives mentioned in the talk today: Mamba/Boa/Quetz.
Sure, if it is not much additional work, we should definitely have a look!
Since this is to make things easier on the end-user, we’ll have to consider things not only from an easier development/maintenance perspective, but also their ability to install/use either docker or conda (with or without root priveleges).
Conda can (and should) be installed by the user, not system-wide. This way, no root access is required.
I cannot comment on docker but form what I understand about conda-forge is that you basically provide a recipe and that is then built and distributed by them (https://docs.conda.io/projects/conda-build/en/latest/concepts/recipe.html). And yes, as far as we know, all dependencies are there. Especially sundials with the required version 3.2.1.
Because I’m quite busy at the moment, can you guys investigate and experiment a bit with this?
We should probably stick with conda. That is where all the other python dependencies are and trying to get other users, especially corporate ones, to install something else will be more difficult.
I don’t know much about docker either. It is not something I have ever used.
Both conda-forge and docker should be doable in a limited amount of time. We already have scripts for CI (AppVeyor, Travis, Gitlab Actions in some branch). Those can be adapted for conda-forge recipes, I guess.
Since we can statically link to most deps, our package can have very few actual dependencies (if we want to).
For docker, the same scripts or processes can be used for creating the container. I would also add another container that contains a full development environment (all dependencies, compiler toolchains, git, maybe even a preconfigured VS Code). This way, the bar to develop some C++ code is lowered.
I had a talk with Tim Kreuzer from the JSC about jupyterhub. I also asked him about the possiblity to deploy our own images and grant access to external users. Here is his response:
[…] es möglich sein eigene Docker Images zu benutzen. Am einfachsten sind dafür natürlich Images, die auf einem öffentlichen Repository gelagert werden können (z.b. hub.docker.com). Ein privates Repository haben wir momentan noch nicht, wird aber evtl. auch bald kommen.
Ein Feature der neuen Jupyter-JSC Version (die ebenfalls in den nächsten Wochen bis wenigen Monaten) kommen wird, werden auch “Gruppen / Virtuelle Organisationen” sein. Mit diesen Gruppen wird es möglich sein (fast) alles in Jupyter-JSC mit Restriktionen zu versehen. Also auch den von dir genannten UseCase: Dieses Image dürfen nur Leute benutzen, die in einer bestimmen Gruppe sind.
So that sounds promising. We should definitely get that running as it will enable us to have a running version of CADET readily available for everyone.
Interesting article about integrating conda packaging in travis. Preferably we would do the same with docker. This way, we have only one CI system to maintain.
FROM continuumio/miniconda3
COPY /install.sh /
RUN conda config --add channels conda-forge
RUN conda install hdf5 superlu suitesparse sundials=3.1.1 cmake
RUN git clone https://github.com/modsim/CADET /opt/CADET
RUN apt install -y build-essential
RUN chmod +x install.sh
RUN ./install.sh
Where install.sh just jumps into the CADET dir, sets the right environment variables, and builds it.
If we create and push a conda recipe, it’ll be even simpler.
I suggest to exclusively rely on conda-forge. Continuum has changed its license terms and now charges for commercial usage of its channel (i.e., usage of conda within a company). You can still use Miniconda and other channels (i.e., conda-forge) free of charge.
I’ve been working on the Docker side of things, and Jazib on Conda.
So far I’ve tried continuumio/miniconda3 (debian based), ubuntu and archlinux as base images.
Trying to build CADET with the compilers package from conda (gcc-9.3.0) fails everywhere due to some librt.so/glibc issue for testRunner. -DENABLE_TESTS=OFF can be used to sidestep this issue.
Trying to build on continuumio/miniconda3 and ubuntu with os-repo gcc gives a link error for tbb_preview.so.2 when tbb-devel is installed via conda. If we use libtbb-dev from the os-repo everything compiles fine.
The build with archlinux base image works with gcc from os-repo and tbb-devel from conda.
So currently, I’ve pushed the archlinux build to the jugit repo above. More info in the README.
I believe that cadet-cli can be used as an “entrypoint” for the docker container. So instead of opening a shell in the container, we could just type, say docker run <cadet-docker-image> <hdf5-file> and let cadet run in the container while using the file outside. Yet to try this out, though.
A discussion with @j.schmoelder also yielded the idea that we could talk to Martin and Frederick for more help with docker.
From the Conda side of things, building on windows might be a problem because superlu is not available on conda for windows. More on this from @j.hassan
Presently, it seems like our dream of conda install -c conda-forge cadet might not be that easy to achieve on all platforms due to build/link issues and missing dependencies.
@j.rao: CADET only requires either SuperLU or UMFPACK (from SuiteSparse). If both are supplied, the user can choose between them. But UMFPACK is entirely sufficient.
Moreover, the standard TBB should work if it is new enough. We actually do not use preview features.
The libtbb-preview is required for building the native MATLAB MEX binary with multithreading support. Since this will (soon-ish) be replaced by the C API and split into a separate repository, we can move to standard libtbb.
@s.leweke Ah. That should simplify things. I think suitesparse is available on conda for windows.
I don’t think the issue is with libtbb-preview per se. I tried disabling it and forcing libtbb in the CMakelists.txt file at some point. The linker then spat out an error about not finding libtbb.so. CMake finds the libs when configuring. But the linker can’t seem to, even if I force it with ld /path/to/conda/libs. So, essentially, the linker is having trouble finding tbb-devel libs from conda for whatever reason (on debian/ubuntu).
@s.leweke, would you have time to join the zoom meeting at 1700 tomorrow? Maybe we can make some progress then.
@j.hassan We could temporarily avoid using tbb and/or gcc from conda in the recipe. And then when we figure things out, update it.
The notebooks will be hosted on HDFCloud (Helmholtz Data Federation, unrelated to HDF5). We’ll need access to this machine eventually. Tim told me that we can test creating our own Jupyter Kernel on top of their base image on any machine, so we can try this on JURECA-DC for now.
2. Publishing on conda-forge
@j.schmoelder forked the conda-forge/staged-recipes onto the cadet org. on github: staged-recipes. We’ll just try to build the conda-recipes locally (not with a blank docker container). Once the build is successful, we are to submit a pull request to the main conda-forge repo, where they’ll run automated tests. Once it passes, it will get accepted into the conda-forge repo. More info in this issue
