This page list the information about how to optimize I/O operations on the code and file system level.
Rules for I/O Handling on JUROPA/HPC-FF
In order to optimize the I/O performance of programs on JUROPA/HPC-FF and to avoid any disturbance of other users you should follow a few rules when writing your software for the system:
- Write your data in large blocks instead of small portions to disk.
- Use buffered I/O whenever possible and avoid to flush your output frequently.
Depending on the programming language used, there are some specific rules:
When using the ifort compiler, make sure that the environment variable
FORT_BUFFERED = true
is set. This is the case in the default environment, when the module parastation is loaded; in case you unload this module make sure that you set the FORT_BUFFERED variable properly.
Alternatively, you can use the compiler option -assume buffered_io to switch on I/O buffering for FORTRAN programs. The ifort compiler offers the non-standard parameters BUFFERED, BLOCKSIZE and BUFFERCOUNT for the open statement in order to customize the buffering of the output. Here are typical values, recommended for JUROPA/HPC-FF. For details, please see the Intel manual.
|BUFFERED=YES||enables buffering of WRITE commands|
|BLOCKSIZE=1048576||sets the size of the buffer to 1 MB|
|BUFFERCOUNT=n||uses n buffers; in general n=1 is appropriate|
open(unit=20,file=test.out,status='new',form='formatted', blocksize=1048576, buffered='yes', buffercount=1)
Do not use fflush (except at the end of your output/program).
Do not use flush (except at the end of your output/program). Do not use endl, because this performs an additional flush each time called. If you need a line break, please use \n instead.