Efficient Neural Network Training Data Preparation

Preparation of the training data Hosein Karimi 27/11/2019

The goal is to have a neural network which enhances efficiency of the tracking for multi-turn tracks. We need to prepare data using Monte-Carlo simulations to train and test the neural network.

We generate rtraw of the events by simulations, but h5 format is needed to feed the neural network. Steps are: Generating particles of interest (SingleGunGenerator) specifying momentum, angles, and their distribution type Submitting boss jobs to simulate the tracks of these particles in bes3 number of events simulations with different seeds to have enough different events particles and antiparticles Saving information of the particles and the tracks in csv format analyzing generated particles statistics choosing events of interest (number of turns) Reshaping and saving information in h5 files Submitting to the gpu farm to feed and test the neural network

SingleParticleGun algorithm is in the GenModule class of BOSS and can be used to generate particles with specified distribution of cos ? ,?, and momentum. Mode 1: E, theta, and phi are specified Mode 2: Pt, theta, and phi are specified For each one of these three entries there are three choices: FixedMode-1: Generation with a fixed value GaussianMode-2: Generation with a gaussian of specified FlatMode-3: Generation of a flat distribution between a specified minimum and maximum value The output will be stored in the transient event store so it can be passed to the simulation. http://bes3.to.infn.it/Boss/7.0.2/html/SingleParticleGun_8h-source.html http://bes3.to.infn.it/Boss/7.0.2/html/SingleParticleGun_8h-source.html

energy= 40 to 120 MeV events = 75000 100 different seed numbers 7,500,000 events in total All are not needed. Later we choose some based on number of turns and particle identity. (more than 70% remain?) SingleParticleGun

By submitting jobs with these option files, we get the rtraw files of simulations. There are 100 files for electrons and 100 for positrons, each about 1.3 GB. In total there are about 250 GB of data for 7,500,000 events (17 kb/event, about 7000 wires?). (no need for all, maybe we could generate less?) besvis.exe root?

Next is to convert rtraw files to csv type with DumpAlg. Submit job options like: Each rtraw saves in 8 csv files in total 1600 files, about 90 GB mc* files contain generated particles information digi* files contain track data http://cepcgit.ihep.ac.cn/zhangyao/MLTracking/tree/master/multiTurnBESIII/dump

mc* files contain generated particles information digi* files contain track data

http://cepcgit.ihep.ac.cn/zhangyao/MLTracking/blob/master/multiTurnBESIII/describe_csv_data All generated electrons 0 < number of turns <6

HDF stands for Hierarchical Data Format and is used to store and manipulate huge data. h5py (or h5) is the 5thgeneration of this interface. HDF5 file contains two kinds of objects: datasets, which are array-like collections of data groups, which are folder-like containers that hold datasets and other groups Groups work like dictionaries, and datasets work like NumPy arrays We use preProcess.py code to change csv files to h5 files http://cepcgit.ihep.ac.cn/zhangyao/MLTracking/blob/master/multiTurnBESIII/preProcess.py each 13 file which used less that 1GB now have the size of 25 GB

To interpret these files we use http://cepcgit.ihep.ac.cn/zhangyao/MLTracking/blob/master/multiTurnBESIII/describe_h5py_data.ipynb