Best practices on magnitUDE

In the following, you’ll find a collection of best practices that we found to work well on magnitUDE. However, none of these are mandatory to follow. You are free to establish your own workflows.

Using the UDE login server

If you are a member of UDE you have access from the internet to the universities login server. The server address is either student.uni-due.de or staff.uni-due.de, depending on your current status. For the connection, you just use your UDE account username and password. After the first login, you can put an ssh-key there to avoid typing your password every time.

Connection to magnitUDE via UDE login server

As the server is registered to the universities domain space, you can reach the magnitUDE via ssh from there, without using a VPN, respectively.

This becomes handy when using the UDE login server as a proxy for your ssh connection to the magnitude. On UNIX systems you can set up instructions for the ssh connection to do so. One way is to modify (or create if not present) the file ~/.ssh/config to contain the following entry:

Host magnitUDE
    HostName login01.magnitude.uni-due.de
    User <username>
    IdentityFile <path tu your private key>
    ProxyJump <username>@staff.uni-due.de

Now from the terminal you just have to ssh magnitUDE to establish the connection to magnitUDE. In case you have not put an ssh-key on the UDE login server, you will be asked for a password. Here the UDE id password is required to establish the connection to the login server first and then from there to the magnitUDE.

Notice, that you still have to follow the two-factor authentification as described here in detail.

Internet via the UDE login server

For security reasons the magnitude is not connected to the internet. Some software or workflows, however, require an internet connection at least for configuration steps. The UDE login server mounts the same home directory as the magnitude and is connected to the internet. Hence, we can login to the server and use the internet, e.g., to download software and install to our home directory and use this software later on the magnitude. This approach can be taken, e.g., for installing python packages from the internet as discussed here.

Transferring data to magnitUDE with rsync

To transfer files from and to the magnitUDE we recommend rsync. It efficiently copies and sync files to or from a remote system and supports copying links and permissions. It’s faster than scp (Secure Copy) because rsync uses a remote-update protocol which allows transferring just the differences between two sets of files. The first time, it copies the whole content of a file or a directory from source to destination but from next time, it copies only the changed blocks and bytes to the destination. Rsync consumes less bandwidth utilization as it uses compression and decompression method while sending and receiving data on both ends.

Usage:

rsync -avzh /path/to/local/folder/research-data magnitUDE:/scratch/$UDE-user-id/

The meaning of the options is as follows:

-a : archive mode, which allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships, and timestamps.
-v : verbose
-z : compress file data.
-h : human-readable, output numbers in a human-readable format.

Please note that in order to transfer the whole directory instead of only its contents there is no trailing / at the end of the path /path/to/local/folder/research-data.

Sharing files with other users on magnitUDE

If you (<username>) want to share data with another user of magnitUDE (<user_a>) you can grant <user_a> access to your scratch partition using setfacl, e.g.

setfacl -m u:<user_a>:rx /scratch/<username>/

It will grant <user_a> read and execute permissions but not writing/deleting. Any new files you create will keep the permissions you have for <user_a> as well. If you need <user_a> to write files, simply change the :rx part to :rwx.

To remove the permissions use:

setfacl -x u:<user_a> /scratch/<username>/
setfacl -b /scratch/<username>/

Attention: This way you grant another user access to your files which is a potential security issue.

SFTP on magnitUDE

You can use the SFTP protocoll to browse, upload and download files from mthe magnitude.

Visual Studio-Code