Problem with multithreading with DepositionReader and custom electric field

Simulation / Configuration Questions
1

Hello,
I am trying to use the DepositionReader module along with a custom electric field in allpix-squared. The simulation is running fine, but I noticed that the allpix-squared is not using the multithreading feature even though I specifically added:

multithreading = true
workers = 8

in my configuration file. While running, I see this was printed:

(STATUS) Multithreading enabled, processing events in parallel on 8 worker threads

however, the CPU load is just 100% - strictly single-core. Is it not possible to use DepositionReader along with a custom electric field in a multithreaded way? Do I need to add anything else in my config file (attached below) to use multithreading?

Here are the details of the setup:

  1. Using latest Allpix from CVMFS:

source /cvmfs/clicdp.cern.ch/software/allpix-squared/latest/x86_64-centos7-gcc11-opt/setup.sh

  1. This is the configuration file
    simulationStave01Event1Circle.conf (1.4 KB)

  2. The geometry files are here:
    alpide_no_support.conf (385 Bytes)
    Stave01.conf (98 Bytes)

  3. This is the file with hitTree needed for DepsotionReader module:
    InputForAllPix_Signal_e0ppw_7.0_Stave01Event1.root (362.5 KB)

Thank you very much,
Arka

2

Hi @asantra ,

I am afraid the DepositionReader module is a so called SequentialModule, which does not allow for parallelisation. The reason is the IO from external files, which is not guaranteed to be threadsafe. In principle, the other modules would still be able to run in parallel, so there should be some action on the other workers. However, if DepositionReader is the bottleneck here, I’m afraid it might not lead to a large benefit in computing time.

Cheers
Paul

3

Hi @asantra

adding to what @pschutze said - the data file you provded here seems to only contain a single event. Since our multithreading is event-based, this single event will always be processed only by one core of a CPU. If you start adding more events, the next one will be assigned to a different core and they can be executed in parallel.

Some additional notes on your configs:

  • If you set model_paths = "./" you can place the ALPIDE model file alongside your configuration file.

  • Setting the number of workers explicitly is discouraged - if you do not set the parameter, it will figure out the number of supported threads on your machine automatically

  • In your ROOTObjectWriter module you currently store all objects from the simulation. Especially for DepositedCharge and PropagatedCharge objects this is highly discouraged since you very likely will not need them but they will increase your file size significantly. The single event posted here gives me:

    [F:ROOTObjectWriter] Wrote 2163923 objects to 5 branches in file:
                                   dataFile_Signal_e0ppw_7.0_EFieldV10p7p1pyN17Vpercm_Stave01_Event1.root

    with a total file size of 107M. By changing the configation to:

    [ROTOObjectWriter]
exclude = DepositedCharge, PropagatedCharge

    I end up with:

    [F:ROOTObjectWriter] Wrote 38786 objects to 3 branches in file:
                                 dataFile_Signal_e0ppw_7.0_EFieldV10p7p1pyN17Vpercm_Stave01_Event1.root

    and a total file size of only 1.7M.

Cheers,
Simon