ORCA
Job submission with ubORCA script
Important
The script submits an ORCA 5.0.4 job with filename.inp as input to the cpu queue of Komondor. Job submission can be executed as follows:
/opt/software/packages/orca/ubORCA filename.inp
In order to make it convenient, the following alias can be applied:
alias ubORKA='/opt/software/packages/orca/ubORCA'
In this case, the syntax of the submission will be handier (and then ubORCA will be ubORKA in all the commands below).
ubORKA filename.inp
The ubORCA script only accepts ORCA input files with the .inp extension, and the ORCA input file must include information about the number of CPUs and allocated memory size in the following format (otherwise job will fail and the size of output file remains zero):
%pal
nprocs 4
end
%maxcore 5000
Since ORCA 5.0.4 allows running with parallel OpenMPI processes, the number of parallel processes must be declared in the ORCA input. This information is automatically passed to SLURM through the ubORCA script. Of course, one can also use this script for single-process jobs, but the process declaration must still be provided in the same way. Currently, it is impractical to use more than 16 processes (maximum 782 in cpu queue of Komondor). Memory allocation is also handled automatically using the maxcore value provided in the ORCA input, this value is give in MB as memory per processes. ORCA may use more memory than the maxcore value specified. Therefore, allocated memory in SLURM has overhead for ORCA input. In other words, SLURM allocates 100/70 times as much memory as the maxcore value specified in the ORCA input. Additionally, it is important to provide memory for the operating system on the computing node as well. The maximum physical memory is 128 GB per cpu for the cpu queue. Also, be aware that storage can be an issue since this script submits the job to the scratch folder, which currently has a hard limit of 1.1 TB for each account. You can check the current storage usage by using squota. In the case of proper job submission, the script will output the following message: Running 8 parallel processes 7142MB of memory is used per process 57136MB total memory allocated for calculation
Overallocation is prohibited, and the script will display the following message: sbatch: error: Batch job submission failed: Requested node configuration is not available
The output file will have the same name as the input file, but with the extension .out. Additionally, if it is specified in the input, then the following additional outputs will also be copied back to the submission folder: .xyz, .hess, .spectrum, .gbw, .cpcm. A basic input format is:
! B3LYP/G cc-pVDZ Opt Freq
%pal
nprocs 8
end
%maxcore 5000
*xyz 0 1
C 0.765317 0.000000 0.000000
H 1.164483 1.006642 0.170694
H 1.164478 -0.355494 -0.957128
H 1.164488 -0.651149 0.786427
C -0.765317 0.000000 0.000000
H -1.164488 0.651723 -0.785951
H -1.164483 -1.006517 -0.171430
H -1.164478 0.354794 0.957387
*
Benchmarking
General benchmarking for ORCA jobs cannot be done since the results affected by several factors (e.g. level of theory chosen); therefore, you must experience it yourself. For the above-mentioned input, the specified CPU and memory allocation provided the shortest run time, but only 3 times faster than the single cpu job, therefore nprocs 4 would be even more economical choice since the speed-up is about 2.5 in that case if maxcore 7500MB applied. Alternatively, your own specific SLURM script can be written by pointing to the folder of /opt/software/packages/orca/orca
ORCA 5.0.4 specific information can be available via:
https://www.orcasoftware.de/tutorials_orca/ https://orcaforum.kofo.mpg.de/app.php/portal
update: 03/08/2024