Child pages
  • Hyak FDTD Solutions

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The trial license of FDTD Solutions requires access to the Internet. The following steps set it up so this is possible. You must first set up X11 forwarding on your workstation before proceeding.

No Format
 login$ qsubsrun -l nodes=1:ppn=16 -I -V
 p build --time=2:00:00 --mem=100G --pty /bin/bash
 node$ module load fdtd_8.12
 node$ fdtd-solutions

...

You will need to change parameters which are bold and red for appropriately your job. Depending on your application, you may need to change parameters which are bold and blue. In most cases, you should not change any text that is not colored.

No Format
 #\!/bin/bash
 ##
 ## \!\! _NEVER_ remove # signs from in front of PBS or from the line above \!\!
 ##
 ## RENAME FOR YOUR JOB
 #PBS -N <font color=red>*test-fdtd*</font>
No Format
 ## EDIT FOR YOUR JOB
 ## Request 16 CPUs (cores) on 2 nodes, 32 total cores
 #PBS -l nodes=<font color=red>*2*</font>:ppn=<font color=red>*16*</font>,mem=<font color=red>*22gb*</font>
No Format
 ## WALLTIME DEFAULTS TO ONE HOUR - ALWAYS SPECIFY FOR LONGER JOBS
 ## If the job doesn't finish in 10 minutes, cancel it
 #PBS -l walltime=<font color=red>*00:10:00*</font>
No Format
 ## EDIT FOR YOUR JOB
 ## Put the output from jobs into the below directory
 #PBS -o <font color=red>*/gscratch/GROUPNAME/USERNAME/JOB_DIR*</font>
 ## Put both the stderr and stdout into a single file
 #PBS -j oe
No Format
 ## EDIT FOR YOUR JOB
 ## Specify the working directory for this job
 #PBS -d <font color=red>*/gscratch/GROUPNAME/USERNAME/JOB_DIR*</font>

 

 

Use the command "module avail" to find the version number for below commands.

See end of below link for a sample slurm script.

Mox_scheduler

 

No Format
 ## Some applications, particularly FORTRAN applications require
 ##  a larger than usual data stack size. Uncomment if your
 ##  application is exiting unexpectedly.
 #ulimit -s unlimited
  
 ## Disable regcache
 export MX_RCACHE=0
No Format
 ### Debugging information
 ### Include your job logs which contain output from the below commands
 ###  in any job-related help requests.
 # Total
Number of processors (cores) to be used by the job
 HYAK_NPE=$(wc -l < $PBS_NODEFILE)
 # Number of nodes used
 HYAK_NNODES=$(uniq $PBS_NODEFILE | wc -l )
 echo "**** Job Debugging Information ****"
 echo "This job will run on $HYAK_NPE total CPUs on $HYAK_NNODES different nodes"
 echo ""
 echo "Node:CPUs Used"
 uniq -c $PBS_NODEFILE | awk '\{print $2 ":" $1\}'
 echo "SHARED LIBRARY CHECK"
 ldd `which fdtd-engine-impi-lcl`
 echo "ENVIRONMENT VARIABLES"
 set
 echo "**********************************************"
 ### End Debugging information
No Format
 # Prevent
tasks from exceeding the total RAM of the node
 # Requires HYAK_NPE and HYAK_NNODE or HYAK_TPN to be set.
 HYAK_TPN=$((HYAK_NPE/HYAK_NNODES))
 NODEMEM=`grep MemTotal /proc/meminfo | awk '\{print $2\}'`
 NODEFREE=$((NODEMEM-2097152))
 MEMPERTASK=$((NODEFREE/HYAK_TPN))
 ulimit -vs $MEMPERTASKunlimited
  
 module load icc_15.0.3-impi_5.0.3
 module load fdtd_8.12
No Format
 ###
 # EDIT FOR YOUR JOB
 #
 mpiexec.hydrampirun -bootstrapn rsh32 -rmk pbs fdtd-engine-impi-lcl *<font color=red>inputfileinputfile.fsp*</font>