Temporary storage needed quickly

Situation: You want to download a big dataset to have a look at it or to do a quick test/computation with it. But you don't have storage for that and requesting storage is too much of a hassle for a quick test.

Solution: Use the /tmp folder--possibly on another computer.

Explanation: There's a folder /tmp on every Linux computer. In the MPI CBS it's at least 25GB in size (workstations) but some compute servers are equipped with 5TB. To utilize a bigger temporary storage, follow these steps:
  1. Connect to a compute server, set an additional getserver /tmp capacity constraint ( -T ):
    user@host > getserver -sTL
    Find help about getserver via getserver --help= .
  2. Go to /tmp , make a folder and use it.
  3. Remove the folder after you're finished.

Be aware that:
  • /tmp is cleared whenever the computer start. Possible reasons for a computer starting:
    • Any crash of the computer (happens sometimes, usually when users use too much memory and and computer runs out of it)
    • Urgent security updates (very unusual)
    • Operating system kernel update once a month, see MaintenanceWindow.
  • /tmp is a shared storage. Other users (or even yourself) might have already put data there.
  • There are almost no performance and protection guarantees for this folder. Find more information here.

This topic: EDV/FuerUser > WebHome > QuickTempStorage
Topic revision: 05 Aug 2024, wherbst
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback