UFTP Service
UFTP is a file transfer tool and has a client/server architecture. Its main features include high-performance file transfers from client to server (and vice versa), list directories, make/remove files or directories, sync files and data sharing. Users can easily share their data even with users who do not have Unix level access to the data.
The data access server (uftpd) is running on JUDAC, while users authenticate with their SSH Public key to an authentication server.
Authentication server: https://uftp.fz-juelich.de:9112/UFTP_Auth
Client setup
You can install the 'uftp' client on your desktop machine (Linux/Mac OS) to be able to transfer data from your desktop to supercomputers and vice versa. The latest version is available for download here. The detailed documentation of the client is available here. On JUWELS and JURECA the client is already installed and can be loaded with:
$> module purge && module use $OTHERSTAGES && module load Stages/Devel-2019a uftp/1.1.0
By default, the uftp client will use SSH key authentication to authenticate to the Auth server. In order to use UFTP, you must have SSH access to JUDAC. If not, you'll have to first upload your SSH Public key via JuDoor.
You can use ssh agent forwarding with "ssh -A" when logging in to JURECA or JUWELS.
If not specified on the command line with the "-u username" option, the client will use the environment variable $UFTP_USER as the username, if that is not defined, the current username ($USER) will be used. The value should be your JuDoor user name.
Examples
We recommend to set environment variables to store the URL of the Auth server:
$> export SERVER=https://uftp.fz-juelich.de:9112/UFTP_Auth/rest/auth/JUDAC
Get information about the remote server
$> uftp info $SERVER
If you want to explicitely set a different username, use the "-u" option:
$> uftp info -u username $SERVER
List contents of a remote directory
$> uftp ls $SERVER:/p/home/jusers/$USER/jureca
Download a single file to the current directory
$> uftp cp $SERVER:/p/home/jusers/$USER/jureca/test .
You can also download multiple files using wildcards.
$> uftp cp $SERVER:/p/home/jusers/$USER/jureca/testdir/* .
Uploading files using wildcards
$> uftp cp "/tmp/test/*" $SERVER:/p/home/jusers/$USER/jureca
Data sharing
By default, files will be shared for "anonymous" access. This will allow anyone who knows the sharing link to access the file using normal HTTP tools like wget or curl. Shares can be limited to certain users.
First we need to set the following environment variable. On JURECA and JUWELS it is already set when loading the uftp module.
$> export UFTP_SHARE_URL=https://uftp.fz-juelich.de:9112/UFTP_Auth/rest/share/JUDAC
List shares with
$> uftp share --list
To share a file with anybody
$> uftp share /p/home/jusers/$USER/jureca/test
This will print the shared link on the screen. You can use curl or wget to download it. If you want to restrict the access to a specific user use "--access" option. For example,
$> uftp share --access "CN=schuller1, OU=ssh-local-users" /p/home/jusers/$USER/jureca/test
Note that the "CN=..." part contains the Unix user ID of the target user and "OU=ssh-local-users" is same for all users.
To delete a share
$> uftp share --delete /p/home/jusers/$USER/jureca/test
$> uftp share --access "CN=..." --delete /p/home/jusers/$USER/jureca/test
