AllPix not saving data properly?

Hi everybody,

I’ve run into a weird issue during my most recent allpix simulations.

I’m trying to simulate the behaviour of a fixed set of charges deposited into a silicon strip sensor, under a TCAD simulated electric field and Ramo potential and a constant magnetic field. My configuration file for reference:

[Allpix]
log_level = “WARNING”
log_format = “DEFAULT”
model_paths = “/eos/user/d/damir/ALLPIX2_Remote/Prototype/Configurations”
detectors_file = “Configurations/detectors_file.conf”
number_of_events = 100
root_file = “histos_prototype_5strip_3DW_transprop_MIP”
random_seed = 0

#The geometry is empty for now
[GeometryBuilderGeant4]

#Magnetic Field Reader
[MagneticFieldReader]
model = “constant”
magnetic_field = 0T 2T 0T

#Depositing the charges directly
[DepositionPointCharge]
source_type = “point”
model = “fixed”
position = 0um 0um 50um
number_of_charges = 1000
output_plots = 1

#Electric Field Reader
[ElectricFieldReader]
model = “mesh”
file_name = “TCAD_Fields/fields_1strip_des_ElectricField_COARSE.init”
output_plots = 1

#Ramo Potential Reader
[WeightingPotentialReader]
model = “mesh”
file_name = “TCAD_Fields/fields_1strip_des_Weighting_WeightingPotential_COARSE.init”
output_plots = 1

#Using Transient Propagation since it uses the Ramo-Schokley Theorem
[TransientPropagation]
temperature = 253K
charge_per_step = 10
mobility_model = “canali”
trapping_model = “cmstracker”
fluence = 4e15/cm/cm
induction_matrix = 3 3
output_plots = 1

#Pulse Transfer
[PulseTransfer]
max_depth_distance = 5um
output_plots = 1

#Default Digitizer
[DefaultDigitizer]
output_plots = 1

#Detector Histogrammer
[DetectorHistogrammer]

#[VisualizationGeant4]
#mode = “gui”

#Define the .root file that contains the tree, and what leaves the tree contains

[ROOTObjectWriter]
file_name = “trees_prototype_5strip_3DW_transprop_MIP”
include = “MCTrack”,“MCParticle”,“PixelCharge”,“PixelHit”,“PropagatedCharge”,“DepositedCharge”

As you can see at the end of the file, I explicitly ask allpix to save “PixelHit” to the output .root file. This tree is meant to contain the hit information I need to derive the charge collection efficiency.

However, when I look in the .root file, there is no such tree to be found. Even more bizarrely, one of the trees that come after it; “PropagatedCharge”, is present in the .root file, despite being dependent on “PixelHit”. I’ve attached a screenshot of the TBrowser I used to check:

Why is this happening? I’m tempted to say that this is some sort of bug, since I remember seeing someone posting on the forum about having a similar issue a couple of months ago.

Hi @damir ,

I don’t think this is a bug, but rather a feature of the digitisation stage; as you don’t set a threshold, the default is used, which is 600 electrons. I don’t know how thick your sensor is, but I can imagine that 400 charges get lost in the propagation at a fluence of 4e15, and thus none of your events reach above the threshold, so no PixeHit objects are created with the current configuration (and thus not saved).

You could check this by retrieving your PixelCharge objects with a macro, and seeing if they reach the threshold value, or just lower the threshold a bit.
You could also look at a few linegraphs, and seeing where charge goes and if it gets trapped.

Kind regards,
Håkan

Good observation, @hwennlof ! Maybe we should remove the default and make it a required parameter? What do you think?

Hi Håkan,

Thanks for the advice. I tried increasing the deposited charge to 5000 electrons, but that didn’t result in what I wanted. After also decreasing the threshold to “1e” the “PixelHit” tree started to appear in the .root file. We’re back on track, but I wonder if I’ve done something weird here since the number of electrons and threshold values I needed seem a bit extreme.

For reference, I’m using a custom detector model with “sensor_thickness = 304um”.

You could check this by retrieving your PixelCharge objects with a macro, and seeing if they reach the threshold value, or just lower the threshold a bit.

I could actually use some help reading the trees I get out of allpix, generally. Is there any documentation anywhere explaining what branches of the trees contain what information?

Cheers,
Damir

Yes, either make it required or default it to zero I think. Not sure what is best, but having it at 600e is a bit funky

Hi @damir

the trees and their content are the simulation objects, described in detail here: Object Types | Allpix Squared

examples for reading and analyzing in C++ or Python are provided here: ROOT Analysis & Helper Macros | Allpix Squared (the scripts are in the repo: tools/root_analysis_macros · master · Allpix Squared / Allpix Squared · GitLab)

Cheers,
Simon

Hi,

Okay, cool that we found the issue of the tree not being produced!
I don’t know enough about your fields and radiation damage to be able to comment on your values and if something is odd, I’m afraid. One thing to check is that your field from TCAD has the right thickness, or to set the field_depth parameter in [ElectricFieldReader] to the thickness of the TCAD field, as it will otherwise be stretched or squished along the sensor depth.

The documentation holds some info on what one can get out of the objects; Object Types | Allpix Squared
The analysisExample.C macro here contains some info on how to get stuff out of the trees; tools/root_analysis_macros · master · Allpix Squared / Allpix Squared · GitLab
It shows how to get the MC hit positions and pixel positions using Allpix Squared objects, so it’s a good starting point. It’s often not very useful to look at these trees using a TBrowser, as a lot of the vbariables are arrays that get squashed when looking like that, so what one sees might not be what one expects… It’s better to use a macro to disentangle things

Kind regards,
Håkan

Hey,

I don’t know enough about your fields and radiation damage to be able to comment on your values and if something is odd, I’m afraid. One thing to check is that your field from TCAD has the right thickness, or to set the field_depth parameter in [ElectricFieldReader] to the thickness of the TCAD field, as it will otherwise be stretched or squished along the sensor depth.

I will say that the values I’ve got are almost certainly incorrect, since I’m using pretty coarse converted electric fields/weighting potentials and my config file still probably needs a bit of tweaking. After running 100 events with 5000 deposited electrons, the average number of charges collected was something like 125 (a charge collection efficiency of around 2.5%). This is way too small, but I’m still fixing my configuration so I don’t put too much weight on it.

I could use some more advice on the TransientPropagation module, though. Here’s what mine currently looks like:

[TransientPropagation]
temperature = 253K
charge_per_step = 10
mobility_model = “canali”
trapping_model = “cmstracker”
fluence = 4e15/cm/cm
induction_matrix = 1 1
output_plots = 1
output_linegraphs = true

I have a few questions about the “induction_matrix” and “output_linegraphs” parameters.

  • The “induction_matrix” parameter isn’t mentioned on the TransientPropagation manual page, but AllPix doesn’t complain when I use it, so I guess everything is working fine. Marco explained it as just representing how many pixels you have in your detector setup. Marco had a 3x3 grid of pixels so he used “induction_matrix = 3 3”, whereas I’m currently using a single strip so I have “induction_matrix = 1 1”. I’m going to try a set of 5 parallel strips next, so I’m guessing I’ll need “induction_matrix = 5 1”.
  • When I added the “output_linegraphs = true” line, I expected that I’d get a diagram of the paths that my 1000 electrons take through the sensor. Instead, AllPix gave me a warning:

|04:55:42.355| (WARNING) Unused configuration keys in section TransientPropagation:detector1:
output_linegraphs

I don’t get why it’s doing this. According to the manual “output_linegraphs” is listed as a parameter of TransientPropagation.

Cheers,
Damir

Hi,

It seems you are using a different Allpix Squared version compared to the manual you’re reading. The website documentation currently shows version 3.0.0 (latest tagged verson), but manuals for all versions can be found here. If you want to keep using the version you are currently using, you should do allpix --version and then find the corresponding manual (certainly one before v3.0.0, as that’s when major changes happened).
The induction_matrix keyword has been replaced by distance in newer versions. It indicates how many pixels the induction is calculated on. Theoretically, all pixels may experience some induction, but practically it doesn’t really happen for pixels further away from where the charge carrier moves, and the calculation is computationally expensive. This keyword can thus be used to limit the number of pixels where the calculation takes place.

The output_linegraphs keyword was also added to [TransientPropagation] only for version 3.0.0, which is why it doesn’t work. I’d suggest either updating your Allpix Squared version, or running with [GenericPropagation] instead just to make the linegraphs.

Kind regards,
Håkan

Hi,

It seems you are using a different Allpix Squared version compared to the manual you’re reading. The website documentation currently shows version 3.0.0 (latest tagged verson), but manuals for all versions can be found here . If you want to keep using the version you are currently using, you should do allpix --version and then find the corresponding manual (certainly one before v3.0.0, as that’s when major changes happened).

Indeed, I am using v2.3.2. It’s the default version that I’ve been using since I started working with Allpix squared, since it’s the one Marco used for parts of his work and I’m trying to mirror some of the stuff that he’s been doing.

I’d be more than happy to move to v3.0.0. I tried to source it (both the centos9 and centos7 versions), but I get some output I don’t understand:

[damir@lxplus7105 ALLPIX2_Remote]$ source /cvmfs/clicdp.cern.ch/software/allpix-squared/3.0.0/x86_64-centos9-gcc12-opt/setup.sh
/cvmfs/sft.cern.ch/lcg/releases/R/4.1.2-75915/x86_64-centos9-gcc12-opt/lib64/R/bin/exec/R: error while loading shared libraries: libtirpc.so.3: cannot open shared object file: No such file or directory
/cvmfs/sft.cern.ch/lcg/releases/R/4.1.2-75915/x86_64-centos9-gcc12-opt/lib64/R/bin/exec/R: error while loading shared libraries: libtirpc.so.3: cannot open shared object file: No such file or directory
/cvmfs/sft.cern.ch/lcg/releases/R/4.1.2-75915/x86_64-centos9-gcc12-opt/lib64/R/bin/exec/R: error while loading shared libraries: libtirpc.so.3: cannot open shared object file: No such file or directory
/cvmfs/sft.cern.ch/lcg/releases/R/4.1.2-75915/x86_64-centos9-gcc12-opt/lib64/R/bin/exec/R: error while loading shared libraries: libtirpc.so.3: cannot open shared object file: No such file or directory
/cvmfs/sft.cern.ch/lcg/releases/R/4.1.2-75915/x86_64-centos9-gcc12-opt/lib64/R/bin/exec/R: error while loading shared libraries: libtirpc.so.3: cannot open shared object file: No such file or directory
gnuplot: error while loading shared libraries: libncurses.so.6: cannot open shared object file: No such file or directory
/eos/user/d/damir/ALLPIX2_Remote
/eos/user/d/damir/ALLPIX2_Remote
/eos/user/d/damir/ALLPIX2_Remote

[damir@lxplus7105 ALLPIX2_Remote]$ source /cvmfs/clicdp.cern.ch/software/allpix-squared/3.0.0/x86_64-centos7-gcc12-opt/setup.sh
gnuplot: /lib64/libc.so.6: version GLIBC_2.32' not found (required by /cvmfs/sft.cern.ch/lcg/views/LCG_103/x86_64-centos9-gcc12-opt/lib/libglib-2.0.so.0) gnuplot: /lib64/libc.so.6: version GLIBC_2.33’ not found (required by /cvmfs/sft.cern.ch/lcg/views/LCG_103/x86_64-centos9-gcc12-opt/lib/libglib-2.0.so.0)
gnuplot: /lib64/libc.so.6: version `GLIBC_2.34’ not found (required by /cvmfs/sft.cern.ch/lcg/views/LCG_103/x86_64-centos9-gcc12-opt/lib/libglib-2.0.so.0)
/eos/user/d/damir/ALLPIX2_Remote
/eos/user/d/damir/ALLPIX2_Remote
/eos/user/d/damir/ALLPIX2_Remote

Sorry for the weird formatting, but it seems to be complaining that it can’t find specific libraries? For reference, the only thing I do before this line is running “setupATLAS” and “lsetup root”.w

The induction_matrix keyword has been replaced by distance in newer versions. It indicates how many pixels the induction is calculated on. Theoretically, all pixels may experience some induction, but practically it doesn’t really happen for pixels further away from where the charge carrier moves, and the calculation is computationally expensive. This keyword can thus be used to limit the number of pixels where the calculation takes place.

This is fine, I’m currently doing a bunch of things to reduce computation time in my setup, so doing some rough simulations in a limited neighbourhood is fine with me.

Cheers,
Damir

Hi @damir

it looks like the sourceing ended up mixing some libraries. SInce you are on LXPlus7, you definitvly have to use the CentOS7 version. But since you sourced the CentOS9 version before, there seem to be some remnants:

gnuplot: /lib64/libc.so.6: version GLIBC_2.33’ not found
(required by /cvmfs/sft.cern.ch/lcg/views
/LCG_103/x86_64-centos9-gcc12-opt/lib/libglib-2.0.so.0)

Could you please try again after logging out and freshly in again?

/Simon

For reference, for me it works on LXplus:

2023-06-14 08:50 simonspa@lxplus7105:~$ source /cvmfs/clicdp.cern.ch/software/allpix-squared/3.0.0/x86_64-centos7-gcc12-opt/setup.sh
2023-06-14 08:53 simonspa@lxplus7105:~$ allpix --version
Allpix Squared version v3.0.0
               built on 2023-05-04, 13:05:52 UTC
               using Boost.Random 1.81.0
                     ROOT 6.28/00
                     Geant4 11.1.1
               running on 10x Intel Core Processor (Broadwell, IBRS)

Copyright (c) 2016-2023 CERN and the Allpix Squared authors.

This software is distributed under the terms of the MIT License.
In applying this license, CERN does not waive the privileges and immunities
granted to it by virtue of its status as an Intergovernmental Organization
or submit itself to any jurisdiction.

Hi Simon,

Yeah, it looks like sourcing them in sequence causes problems, instead of just overwriting stuff like I thought it would. Now that it does work I’ve moved to v3.0.0 for my simulation.

Cheers,
Damir