Developer’s call, 21.06.2021

Recap

Fix _nThreads lower bound to 0

See also forum discussion.

When a negative value is passed to NTHREADS, the value is read as int and afterwards assigned to an unsigned int.
During the assignment, the negative value wraps around and becomes a ridiculously high number.
This can cause problems with TBB due to std::bad_alloc.

Now, the value is clipped to the nonnegative range before it is further used.

A fix was merged in c832f68.


Documentation

Feature/move documentation

  • Documentation was migrated and now includes a separate build for every release, as well as the latest version (master).
  • Documentation for CADET-Match was included (also using sphinx-multiversion)

Hotfix/add documentation redirect

(@j.schmoelder)

To redirect cadet.github.io to cadet.github.io/master/index.html, a hotfix needs to be added which includes an html redirect.

Include doxygen documentation in sphinx

(@j.schmoelder)

Short introduction how to build documentation with doxygen:
Install doxygen and change to CADET’s docs/ directory. Then run doxygen cadet.doxyfile . The output is stored in doxy/html/ (can be opened with any browser).

Next step: include doxygen in sphinx.
Open issue upstream: There seems to be a problem with exhale when using @todo in the documentation (see here).

Clean up CI

(@j.schmoelder)

To automate building the documentation on push, we use Github actions. We took a short look at the general setup, webhooks, as well as the actions marketplace.

Github Actions setup for triggering actions of different repositories

There are several options to trigger github actions.
Usually this means on push, on merge, etc.

For building the documentation every time any of the CADET projects is pushed, we need configure some more things.

First, for the CADET-Website repository, we need to add a repository_dispatch.
This adds a trigger for events outside of the respective repository.
Additionally, we add a workflow_dispatch that enables us to manually trigger the workflow.

https://github.com/modsim/CADET-Website/blob/b95c330bc0ff169fba27e61cdc00cc280a1b9045/.github/workflows/deploy-cadet-github-io.yml#L1

Then, we need to configure the CADET project repositories such that on push, the repository_dispatch of the CADET-Website repository is triggered.
For this purpose, we use the dispatch-action from the marketplace and add the following to all the project’s repositories:

jobs:
  dispatch_docs:
    runs-on: ubuntu-latest
    steps:
      - name: Emit repository_dispatch
        uses: mvasigh/dispatch-action@main
        with:
          token: ${{ secrets.DISPATCH_DOCS }}
          repo: CADET-Website
          owner: modsim
          event_type: build_docs

This also requires adding a personal access token (PAT) to legitimize triggering the other repository.

“check” target for Makefile to run testRunner

(@j.rao)

  • Provide tags for different scenarios local, CI, full, …
  • Write draft, present to @s.leweke

Fix testRunner

(@j.rao, @j.hassan)

  • Discuss viability of setting up reference solutions on new machines.

yaml2hdf

@j.rao presented a new handy tool that allows for configuring CADET using yaml.

HDF5 files are a bit of a pain to work with. Especially if we have complicated CADET-python setups and only want to modify some fields of the h5 file at a time. yaml2hdf (tentatively named) makes it easy to work with hdf5 files using yaml configuration files. It currently uses CADET-Python’s save and load functions to read/write h5 files. This tool brings our config files closer to the unix shell, unlocking the use of pipes and other tools to interact with our configurations.

Current Features:

  • input.yaml → output.h5
  • input.h5 → output.yaml
  • base.yaml patch.yaml → output.h5
  • file1.h5 file2.h5 → diff
  • file1.h5 file2.yaml → diff
  • file1.h5 → print yaml | yq '.output.solution.unit_001.solution_outlet_comp_000'

The software currently lies in my ChromaHD/Tools repo, but will eventually be merged into CADET-Python. The script is very rudimentary right now so please forgive any programming sins I might have committed. :wink:

Publish cadet on conda-forge

(@j.rao)
No progress.

Create docker image

(@j.rao)

  • Now using miniconda as base image (instead of arch).
  • The image is now available on dockerhub: docker pull modsim/cadet
  • To make files on the host machine available to the container, we have to mount directories onto the running container.
  • The entrypoint is directly cadet-cli
  • Command to run cadet-cli from the container on a file while mounting the current directory: docker run -v $(pwd):/data modsim/cadet LWE.h5
  • The image on dockerhub is 1.22 GB. Can be reduced by using a leaner base-image, cleaning some conda files and our source files.

Write out state for every unit operation separately

(@j.hassan)

  • Working solution for this problem has been committed to the PR73, but it lacks the consistent way of implementation as we have for other flags.
  • Currently, I am working on a method to access lastState vector in SolutionRecorder and then implement the lastState corresponding to each UNIT_OPERATION in the way which is consistent with CADET implementation of other flags.

Add simple C interface to libcadet

(@w.heymann, @j.schmoelder, @j.hassan)

In CAPIv1, we need to define individual functions for getting access to the data stored within CADET.
Then, a corresponding function needs to be registered in CADET-Python, see for example here.

To do:

  • [ ] Create list of functions (with required arguments)
  • [ ] Naming convention for functions
  • [ ] Map output to addict (check in input/return)

Missing feature

(@s.leweke)

Return information if simulation fails.
Right now, if a simulation fails, the error is written to the terminal and is not accessible for the dll version of CADET-Python.

Idea for interface v2

Add ability to only run single (or multiple) time steps. This enables to pause the simulation, evaluate the results and continue with new time steps (if required).

What do you mean exactly? You can receive log messages (including errors?) in a callback function.

We can already add this to v1.

That was a comment by @w.heymann who mentioned that currently the error messages are cannot yet be read from the Python interface. Maybe he can comment on that.

That would be great. How would that work and how much effort would it take to add it?

  1. Expose this functionality in ISimulator interface class.
  2. Implement this in Simulator class (SimulatorImpl.cpp file).
  3. Create C-API function that calls the corresponding simulator function.
  4. Profit.

You will also want to expose functions that do consistent initialization. Otherwise you’re stuck in the same section.

1 Like