TRACE provides a rich programming environment for the development of applications.

C, C++ and Fortran

AMD (AOCC), Intel, Gnu and NVIDIA HPC compilers for C, C++ and Fortan are available on TRACE. Be sure to load the module for the compiler set that you want to use. Once the module is loaded, you will have access to the compiler commands:

		Compiler command for
	Module name	C	C++	Fortran
AMD	aocc	clang	clang	flang
Intel	intel	icc	icpc	ifort
Gnu	gcc	gcc	g++	gfortran
NVIDIA	nvhpc	nvcc	nvc++	nvfortran

Compiler options

AMD provides a Compiler Options Quick Reference Guide for AMD, Gnu and Intel compilers on their EPYC processors.
There are man pages for each of the compilers.
See also:

AMD Optimizing C/C++ Compiler (AOCC)
NVIDIA compilers web site
GNU compilers web site
Module documentation for information on what modules are available and how to use them.

OpenMP programming

To compile OpenMP programs you must add an option to your compile command:

Compiler	Option
Intel	-qopenmpfor example: icc -qopenmp yprog.c
Gnu	-fopenmpfor example: gcc -fopenmp myprog.c
NVIDIA	-mpfor example: nvcc -mp myprog.c

MPI programming

Three types of MPI are supported on TRACE: MVAPICH2, OpenMPI and Intel MPI. The three MPI types may perform differently on different problems or in different programming environments. If you are having trouble with one type of MPI, please try using another type. Contact trace-it-help@andrew.cmu.edu for more help.
To compile an MPI program, you must:

load the module for the compiler that you want
load the module for the MPI type you want to use – be sure to choose one that uses the compiler that you are using. The module name will distinguish between compilers.
issue the appropriate MPI wrapper command to compile your program

To run your previously compiled MPI program, you must load the same MPI module that was used in compiling.
To see what MPI versions are available, type module avail mpi or module avail mvapich2. Note that the module names include the MPI family and version ("openmpi/4.0.2"), followed by the associated compiler and version ("intel20.4"). (Modules for other software installed with MPI are also shown.)

Wrapper commands

To use the Intel compilers with	Load an intel module plus	Compile with this wrapper command
		C	C++	Fortran
Intel MPI	intelmpi/version-intelversion	mpiicc note the "ii"	mpiicpc note the "ii"	mpiifort note the "ii"
OpenMPI	openmpi/version-intelversion	mpicc	mpicxx	mpifort
MVAPICH2	mvapich2/version-intelversion	mpicc code.c -lifcore	mpicxx code.cpp -lifcore	mpifort code.f90 -lifcore

To use the Gnu compilers with	Load a gcc module plus	Compile with this command
		C	C++	Fortran
OpenMPI	openmpi/version-gccversion	mpicc	mpicxx	mpifort
MVAPICH2	mvapich2/version-gccversion	mpicc	mpicxx	mpifort

To use the NVIDIA compilers with	Load an nvhpc module plus	Compile with this command
		C	C++	Fortran
OpenMPI	openmpi/version-nvhpcversion	mpicc	mpicxx	mpifort
MVAPICH2	Not available

Custom task placement with Intel MPI

If you wish to specify custom task placement with Intel MPI (this is not recommended), you must set the environment variable I_MPI_JOB_RESPECT_PROCESS_PLACEMENT to 0. Otherwise the mpirun task placement settings you give will be ignored. The command to do this is:
For the BASH shell:
export I_MPI_JOB_RESPECT_PROCESS_PLACEMENT=0
For the CSH shell:
setenv I_MPI_JOB_RESPECT_PROCESS_PLACEMENT 0
See also:

Intel MPI web site
MVAPICH2 web site
OpenMPI web site
Module documentation for information on what modules are available and how to use them.

Other languages

Other languages, including Java, Python, R, and MATLAB, are available. See the software page for information.

Debugging and performance analysis

DDT is a debugging tool for C, C++ and Fortran 90 threaded and parallel codes. It is client-server software. Install the client on your local machine and then you can access the GUI on TRACE to debug your code.
See the DDT page for more information.

TRACE Public Resources

Programming Environment

Analytics