UFTP Service

UFTP is a file transfer tool and has a client/server architecture. Its main features include high-performance file transfers from client to server (and vice versa), list directories, make/remove files or directories, sync files and data sharing. Users can easily share their data even with users who do not have Unix level access to the data.

The data access server (uftpd) is running on JUDAC, while users authenticate with their SSH Public key to an authentication server.

Authentication server: https://uftp.fz-juelich.de:9112/UFTP_Auth

Client setup

You can install the 'uftp' client on your desktop machine (Linux/Mac OS) to be able to transfer data from your desktop to supercomputers and vice versa. The latest version is available for download here. The detailed documentation of the client is available here. On JUWELS and JURECA the client is already installed and can be loaded with:

$> module purge && module use $OTHERSTAGES && module load Stages/Devel-2019a uftp

By default, the uftp client will use SSH key authentication to authenticate to the Auth server. In order to use UFTP, you must have SSH access to JUDAC. If not, you'll have to first upload your SSH Public key via JuDoor.

You can use ssh agent forwarding with "ssh -A" when logging in to JURECA or JUWELS.

If not specified on the command line with the "-u username" option, the client will use the environment variable $UFTP_USER as the username, if that is not defined, the current username ($USER) will be used. The value should be your JuDoor user name.

Examples

We recommend to set environment variables to store the URL of the Auth server:

$> export SERVER=https://uftp.fz-juelich.de:9112/UFTP_Auth/rest/auth/JUDAC

Get information about the remote server

$> uftp info $SERVER

If you want to explicitely set a different username, use the "-u" option:

$> uftp info -u username $SERVER

List contents of a remote directory

$> uftp ls $SERVER:/p/home/jusers/$USER/jureca

Download a single file to the current directory

$> uftp cp $SERVER:/p/home/jusers/$USER/jureca/test .

You can also download multiple files using wildcards.

$> uftp cp $SERVER:/p/home/jusers/$USER/jureca/testdir/* .

Uploading files using wildcards

$> uftp cp "/tmp/test/*" $SERVER:/p/home/jusers/$USER/jureca

Data sharing

By default, files will be shared for "anonymous" access. This will allow anyone who knows the sharing link to access the file using normal HTTP tools like wget or curl. Shares can be limited to certain users.

First we need to set the following environment variable. On JURECA and JUWELS it is already set when loading the uftp module.

$> export UFTP_SHARE_URL=https://uftp.fz-juelich.de:9112/UFTP_Auth/rest/share/JUDAC

List shares with

$> uftp share --list

To share a file with anybody

$> uftp share /p/home/jusers/$USER/jureca/test

This will print the shared link on the screen. You can use curl or wget to download it. If you want to restrict the access to a specific user use "--access" option. For example,

$> uftp share --access "CN=schuller1, OU=ssh-local-users" /p/home/jusers/$USER/jureca/test

Note that the "CN=..." part contains the Unix user ID of the target user and "OU=ssh-local-users" is same for all users.

To delete a share

$> uftp share --delete /p/home/jusers/$USER/jureca/test
$> uftp share --access "CN=..." --delete /p/home/jusers/$USER/jureca/test