loading system configuration loading override configuration from /zhome/89/0/38697/mxtest/mctest/../mccodelib/mcxtrace-test/McXtrace_CPU_MPICC_KISS/mccode_config.json INFO: Using directory: "1" INFO: Using existing c-file: ./Test_SAXS.c INFO: Using existing binary: ./Test_SAXS.out INFO: === INFO: Using system default number of mpirun -np processes -------------------------------------------------------------------------- By default, for Open MPI 4.0 and later, infiniband ports on a device are not used by default. The intent is to use UCX for these devices. You can override this policy by setting the btl_openib_allow_ib MCA parameter to true. Local host: n-62-29-34 Local adapter: mlx4_0 Local port: 1 -------------------------------------------------------------------------- -------------------------------------------------------------------------- WARNING: There was an error initializing an OpenFabrics device. Local host: n-62-29-34 Local device: mlx4_0 -------------------------------------------------------------------------- Simulation 'Test_SAXS' (/zhome/89/0/38697/xTESTS/2022-12-01/McXtrace_CPU_MPICC_KISS_5e7/Test_SAXS/Test_SAXS.instr): running on 10 nodes (master is 'n-62-29-34', MPI version 3.1). [Test_SAXS] Initialize [Test_SAXS] Initialize [Test_SAXS] Initialize 3v03.pdb PDB file located... 3v03.pdb PDB file located... 3v03.pdb PDB file located... [Test_SAXS] Initialize [Test_SAXS] Initialize 3v03.pdb PDB file located... 3v03.pdb PDB file located... [Test_SAXS] Initialize [Test_SAXS] Initialize 3v03.pdb PDB file located... [Test_SAXS] Initialize 3v03.pdb PDB file located... 3v03.pdb PDB file located... [Test_SAXS] Initialize 3v03.pdb PDB file located... [Test_SAXS] Initialize 3v03.pdb PDB file located... Calculating scattering from 1160 residues... Calculating scattering from 1160 residues... Calculating scattering from 1160 residues... INFO (Sample7): Creating protein structure from file 3v03.pdb INFO (Sample7): Creating protein structure from file 3v03.pdb INFO (Sample7): Creating protein structure from file 3v03.pdb Calculating scattering from 1160 residues... Calculating scattering from 1160 residues... INFO (Sample7): Creating protein structure from file 3v03.pdb INFO (Sample7): Creating protein structure from file 3v03.pdb Calculating scattering from 1160 residues... INFO (Sample7): Creating protein structure from file 3v03.pdb Calculating scattering from 1160 residues... INFO (Sample7): Creating protein structure from file 3v03.pdb Calculating scattering from 1160 residues... Calculating scattering from 1160 residues... INFO (Sample7): Creating protein structure from file 3v03.pdb Calculating scattering from 1160 residues... INFO (Sample7): Creating protein structure from file 3v03.pdb INFO (Sample7): Creating protein structure from file 3v03.pdb 6lyz.pdb PDB file located... 6lyz.pdb PDB file located... 6lyz.pdb PDB file located... 6lyz.pdb PDB file located... 6lyz.pdb PDB file located... 6lyz.pdb PDB file located... 6lyz.pdb PDB file located... 6lyz.pdb PDB file located... Calculating scattering from 129 residues... Calculating scattering from 129 residues... INFO (Sample11): Creating protein structure from file 6lyz.pdb INFO (Sample11): Creating protein structure from file 6lyz.pdb Calculating scattering from 129 residues... Calculating scattering from 129 residues... INFO (Sample11): Creating protein structure from file 6lyz.pdb INFO (Sample11): Creating protein structure from file 6lyz.pdb Calculating scattering from 129 residues... INFO (Sample11): Creating protein structure from file 6lyz.pdb Calculating scattering from 129 residues... Calculating scattering from 129 residues... INFO (Sample11): Creating protein structure from file 6lyz.pdb INFO (Sample11): Creating protein structure from file 6lyz.pdb Calculating scattering from 129 residues... INFO (Sample11): Creating protein structure from file 6lyz.pdb # : [pid 31078] Signal 11 detected [proc 5] SIGSEGV (Mem Error) # Simulation: Test_SAXS (/zhome/89/0/38697/xTESTS/2022-12-01/McXtrace_CPU_MPICC_KISS_5e7/Test_SAXS/Test_SAXS.instr) # : [pid 31079] Signal 11 detected [proc 6] SIGSEGV (Mem Error) # Simulation: Test_SAXS (/zhome/89/0/38697/xTESTS/2022-12-01/McXtrace_CPU_MPICC_KISS_5e7/Test_SAXS/Test_SAXS.instr) # : [pid 24317] Signal 11 detected [proc 0] SIGSEGV (Mem Error) # Simulation: Test_SAXS (/zhome/89/0/38697/xTESTS/2022-12-01/McXtrace_CPU_MPICC_KISS_5e7/Test_SAXS/Test_SAXS.instr) # Breakpoint: [_Sample11_init] component Sample11=SAXSPDBFast() INITIALISE [/zhome/89/0/38697/McXtrace/mcxtrace/3.x-dev/tools/Python/mxrun/../mccodelib/../../../contrib/SAXSPDBFast.comp:430] 0.00 % ( 0.0/50000000.0) # : [pid 24318] Signal 11 detected [proc 1] SIGSEGV (Mem Error) # Date: Thu Dec 1 03:44:07 2022 # Simulation: Test_SAXS (/zhome/89/0/38697/xTESTS/2022-12-01/McXtrace_CPU_MPICC_KISS_5e7/Test_SAXS/Test_SAXS.instr) # Started: Thu Dec 1 03:44:04 2022 # Last I/O Error: No such file or directory # Breakpoint: [_Sample11_init] component Sample11=SAXSPDBFast() INITIALISE [/zhome/89/0/38697/McXtrace/mcxtrace/3.x-dev/tools/Python/mxrun/../mccodelib/../../../contrib/SAXSPDBFast.comp:430] 0.00 % ( 0.0/50000000.0) # : Simulation stop (abort). # Breakpoint: [_Sample11_init] component Sample11=SAXSPDBFast() INITIALISE [/zhome/89/0/38697/McXtrace/mcxtrace/3.x-dev/tools/Python/mxrun/../mccodelib/../../../contrib/SAXSPDBFast.comp:430] 0.00 % ( 0.0/50000000.0) # Breakpoint: [_Sample11_init] component Sample11=SAXSPDBFast() INITIALISE [/zhome/89/0/38697/McXtrace/mcxtrace/3.x-dev/tools/Python/mxrun/../mccodelib/../../../contrib/SAXSPDBFast.comp:430] 0.00 % ( 0.0/50000000.0) # Date: Thu Dec 1 03:44:07 2022 # Date: Thu Dec 1 03:44:07 2022 # Date: Thu Dec 1 03:44:07 2022 # Started: Thu Dec 1 03:44:04 2022 # Started: Thu Dec 1 03:44:04 2022 # Last I/O Error: No such file or directory # Last I/O Error: No such file or directory # Started: Thu Dec 1 03:44:04 2022 # : Simulation stop (abort). # Last I/O Error: No such file or directory # : Simulation stop (abort). # : Simulation stop (abort). # : [pid 24319] Signal 11 detected [proc 2] SIGSEGV (Mem Error) # Simulation: Test_SAXS (/zhome/89/0/38697/xTESTS/2022-12-01/McXtrace_CPU_MPICC_KISS_5e7/Test_SAXS/Test_SAXS.instr) # Breakpoint: [_Sample11_init] component Sample11=SAXSPDBFast() INITIALISE [/zhome/89/0/38697/McXtrace/mcxtrace/3.x-dev/tools/Python/mxrun/../mccodelib/../../../contrib/SAXSPDBFast.comp:430] 0.00 % ( 0.0/50000000.0) # Date: Thu Dec 1 03:44:07 2022 # Started: Thu Dec 1 03:44:04 2022 # Last I/O Error: No such file or directory # : Simulation stop (abort). # : [pid 31986] Signal 11 detected [proc 7] SIGSEGV (Mem Error) # Simulation: Test_SAXS (/zhome/89/0/38697/xTESTS/2022-12-01/McXtrace_CPU_MPICC_KISS_5e7/Test_SAXS/Test_SAXS.instr) # Breakpoint: [_Sample11_init] component Sample11=SAXSPDBFast() INITIALISE [/zhome/89/0/38697/McXtrace/mcxtrace/3.x-dev/tools/Python/mxrun/../mccodelib/../../../contrib/SAXSPDBFast.comp:430] 0.00 % ( 0.0/50000000.0) # : [pid 31987] Signal 11 detected [proc 8] SIGSEGV (Mem Error) # Simulation: Test_SAXS (/zhome/89/0/38697/xTESTS/2022-12-01/McXtrace_CPU_MPICC_KISS_5e7/Test_SAXS/Test_SAXS.instr) # Breakpoint: [_Sample11_init] component Sample11=SAXSPDBFast() INITIALISE [/zhome/89/0/38697/McXtrace/mcxtrace/3.x-dev/tools/Python/mxrun/../mccodelib/../../../contrib/SAXSPDBFast.comp:430] 0.00 % ( 0.0/50000000.0) # Date: Thu Dec 1 03:44:07 2022 # Started: Thu Dec 1 03:44:04 2022 # Last I/O Error: No such file or directory # Date: Thu Dec 1 03:44:07 2022 # Started: Thu Dec 1 03:44:04 2022 # : Simulation stop (abort). # Last I/O Error: No such file or directory # : [pid 31988] Signal 11 detected [proc 9] SIGSEGV (Mem Error) # Simulation: Test_SAXS (/zhome/89/0/38697/xTESTS/2022-12-01/McXtrace_CPU_MPICC_KISS_5e7/Test_SAXS/Test_SAXS.instr) # Breakpoint: [_Sample11_init] component Sample11=SAXSPDBFast() INITIALISE [/zhome/89/0/38697/McXtrace/mcxtrace/3.x-dev/tools/Python/mxrun/../mccodelib/../../../contrib/SAXSPDBFast.comp:430] 0.00 % ( 0.0/50000000.0) # Date: Thu Dec 1 03:44:07 2022 # Started: Thu Dec 1 03:44:04 2022 # : Simulation stop (abort). # : Simulation stop (abort). # Last I/O Error: No such file or directory -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- 6lyz.pdb PDB file located... Calculating scattering from 129 residues... INFO (Sample11): Creating protein structure from file 6lyz.pdb # : [pid 31076] Signal 11 detected [proc 3] SIGSEGV (Mem Error) # Simulation: Test_SAXS (/zhome/89/0/38697/xTESTS/2022-12-01/McXtrace_CPU_MPICC_KISS_5e7/Test_SAXS/Test_SAXS.instr) # Breakpoint: [_Sample11_init] component Sample11=SAXSPDBFast() INITIALISE [/zhome/89/0/38697/McXtrace/mcxtrace/3.x-dev/tools/Python/mxrun/../mccodelib/../../../contrib/SAXSPDBFast.comp:430] 0.00 % ( 0.0/50000000.0) # Date: Thu Dec 1 03:44:07 2022 # Started: Thu Dec 1 03:44:04 2022 # : Simulation stop (abort). # Last I/O Error: No such file or directory # : [pid 31077] Signal 15 detected [proc 4] SIGTERM (Termination) # Simulation: Test_SAXS (/zhome/89/0/38697/xTESTS/2022-12-01/McXtrace_CPU_MPICC_KISS_5e7/Test_SAXS/Test_SAXS.instr) # Breakpoint: [_Sample10_init] component Sample10=SAXSNanodiscsWithTagsFast() INITIALISE [/zhome/89/0/38697/McXtrace/mcxtrace/3.x-dev/tools/Python/mxrun/../mccodelib/../../../contrib/SAXSNanodiscsWithTagsFast.comp:261] 0.00 % ( 0.0/50000000.0) # Date: Thu Dec 1 03:44:07 2022 # Started: Thu Dec 1 03:44:04 2022 # : Finishing simulation (save results and exit) # : [pid 31077] Signal 11 detected [proc 4] SIGSEGV (Mem Error) # Simulation: Test_SAXS (/zhome/89/0/38697/xTESTS/2022-12-01/McXtrace_CPU_MPICC_KISS_5e7/Test_SAXS/Test_SAXS.instr) # Breakpoint: [_Monitor_save] component Monitor=PSD_monitor() SAVE [/zhome/89/0/38697/McXtrace/mcxtrace/3.x-dev/tools/Python/mxrun/../mccodelib/../../../monitors/PSD_monitor.comp:125] 0.00 % ( 0.0/50000000.0) # Date: Thu Dec 1 03:44:07 2022 # Started: Thu Dec 1 03:44:04 2022 # : Simulation stop (abort). # Last I/O Error: No such file or directory -------------------------------------------------------------------------- mpirun noticed that process rank 6 with PID 31079 on node n-62-29-11 exited on signal 3 (Quit). -------------------------------------------------------------------------- [n-62-29-34:24310] 9 more processes have sent help message help-mpi-btl-openib.txt / ib port not selected [n-62-29-34:24310] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages [n-62-29-34:24310] 9 more processes have sent help message help-mpi-btl-openib.txt / error in device init