Several methods are available to transfer files into and from TRACE.

...

Paths for TRACE file space Home directory   

See https://cmu-enterprise.atlassian.net/l/cp/jFDV0K6E for more information
 

Commands to transfer files

You can use rsync, scp, sftp to copy files to and from TRACE.

rsync

You can use the rsync command to copy files to and from TRACE. A sample rsync command to copy to a TRACE directory is

rsync -rltpDvp -e 'ssh -l username' source_directory data.trace.cmu.edu:target_directory

The rsync man page contains full command usage details.

Substitute your username for 'username'. Make sure you use the correct groupname in your target directory. By default, rsync will not copy older files with the same name in place of newer files in the target directory. It will overwrite older files.

We recommend the following rsync options

-rltDvp

We also recommend the option

-oMACS=umac-64@openssh.com

If you use this option, your transfer will use a faster data validation algorithm.
You may want to put your rsync command in a loop to insure that it completes. A sample loop is

RC=1
n=0
while [[ $RC -ne 0 && $n -lt 20 ]] do
    rsync source-file target-file
    RC = $?
    let n = n + 1
    sleep 10
done

This loop will try your rsync command 20 times. If it succeeds it will exit. If an rsync invocation is unsuccessful the system will try again and pick up where it left off. It will copy only those files that have not already been transferred. You can put this loop, with your rsync command, into a batch script and run it with sbatch.

scp

To use scp for a file transfer you must specify a source and destination for your transfer. The format for either source or destination is
username@machine-name:path/filename
For transfers involving TRACE-HPC,  username is your TRACE username. The machine-name should be given as data.trace.cmu.edu. This is the name for a high-speed data connector at TRACE. We recommend using it for all file transfers using scp involving TRACE. Using it prevents file transfers from disrupting interactive use on TRACE's login nodes.
File transfers using scp must specify full paths for TRACE file systems. See Paths for TRACE file spaces for details.

sftp

To use sftp, first connect to the remote machine:

sftp

...

 username@machine-name

When  TRACE When TRACE is the remote machine, use your TRACE userid as  username. The TRACE machine-name should be specified as data.trace.cmu.edu. This is the name for a high-speed data connector at TRACE.  We recommend using it for all file transfers using sftp involving TRACE. Using it prevents file transfers from disrupting interactive use on TRACE's login nodes.
You will be prompted for your password on the remote machine. If TRACE is the remote machine, enter your TRACE password.
You can then enter sftp subcommands, like put to copy a file from the local system to the remote system, or get to copy a file from the remote system to the local system.
To copy files into TRACE, you must either cd to the proper directory or use full pathnames in your file transfer commands. See Paths for TRACE file spaces for detailsSee See https://cmu-enterprise.atlassian.net/l/cp/jFDV0K6E for more information.

Graphical SSH client

If you are using a graphical SSH client, configure it to connect to data.trace.cmu.edu the TRACE data transfer node on port 22/TCP. Login using your TRACE Portal username and password.