Several methods are available to transfer files into and from TRACE.

Data Transfer Nodes

The primary data transfer node for TRACE is shown below

Paths for TRACE file space Home directory   

See https://cmu-enterprise.atlassian.net/l/cp/jFDV0K6E for more information
 

Commands to transfer files

You can use rsync, scp, sftp to copy files to and from TRACE.

rsync

You can use the rsync command to copy files to and from TRACE. A sample rsync command to copy to a TRACE directory is

The rsync man page contains full command usage details.

Substitute your username for 'username'. Make sure you use the correct groupname in your target directory. By default, rsync will not copy older files with the same name in place of newer files in the target directory. It will overwrite older files.

We recommend the following rsync options

We also recommend the option

If you use this option, your transfer will use a faster data validation algorithm.
You may want to put your rsync command in a loop to insure that it completes. A sample loop is

This loop will try your rsync command 20 times. If it succeeds it will exit. If an rsync invocation is unsuccessful the system will try again and pick up where it left off. It will copy only those files that have not already been transferred. You can put this loop, with your rsync command, into a batch script and run it with sbatch.

scp

To use scp for a file transfer you must specify a source and destination for your transfer. The format for either source or destination is

For transfers involving TRACE-HPC,  username is your TRACE username. The machine-name should be given as the TRACE data transfer node. We recommend using it for all file transfers using scp involving TRACE. Using it prevents file transfers from disrupting interactive use on TRACE's login nodes.

File transfers using scp must specify full paths for TRACE file systems. See https://cmu-enterprise.atlassian.net/l/cp/jFDV0K6E for more information.

sftp

To use sftp, first connect to the remote machine:

When TRACE is the remote machine, use your TRACE userid as  username. The TRACE machine-name should be specified as data.trace.cmu.edu. This is the name for a high-speed data connector at TRACE.  We recommend using it for all file transfers using sftp involving TRACE. Using it prevents file transfers from disrupting interactive use on TRACE's login nodes.
You will be prompted for your password on the remote machine. If TRACE is the remote machine, enter your TRACE password.
You can then enter sftp subcommands, like put to copy a file from the local system to the remote system, or get to copy a file from the remote system to the local system.
To copy files into TRACE, you must either cd to the proper directory or use full pathnames in your file transfer commands. See See https://cmu-enterprise.atlassian.net/l/cp/jFDV0K6E for more information.

Graphical SSH client

If you are using a graphical SSH client, configure it to connect to the TRACE data transfer node on port 22/TCP. Login using your TRACE Portal username and password.