Search

link to homepage

Institute for Advanced Simulation (IAS)

Navigation and service


SIONlib

Scalable I/O library for parallel access to task-local files

SIONlib Logo

SIONlib is a library for writing and reading binary data to/from several thousands of processors into one or a small number of physical files. For parallel access to files only the open and close functions are collective while writing and reading file can be done asynchronously. SIONlib provides a SIONlib file handle which substitutes the C file pointer.

SIONlib provides a simplified file handling for parallel programs which formerly read or write binary data in parallel into separate files (task-local files). After opening the file with SIONlib, the I/O can be done with the SIONlib equivalents to standard C-I/O functionality (fwrite -> sion_fwrite, fread -> sion_fread) which have the same syntax as their C counterparts. Instead of maintaining thousands of separate files, there is only one large file containing all data.

Given that each processor writes its data to same file, SIONlib assigns different regions (e.g., data blocks) of the file to a processor. Hence, for each processor, SIONlib needs the estimated (or known) data size which will be written to the file (chunk size). SIONlib can also handle extensions (additional chunks), if the end of a data block is reached.

Both, the estimated chunk size and file system block size are used to calculate the individual start position inside the file for each processor. Ensuring contention-free access to file system blocks enables efficient parallel writing and reading.

SIONlib is a joint project of:
Forschungszentrum Jülich, Jülich Supercomputing Centre
German Research School for Simulation Sciences
Laboratory for Parallel Programming

Download current version (see also release notes)

If you are interested in upcoming SIONlib versions, we recommend to add your mail address to the SIONlib-news mailing list.

Contact and WWW

sionlib_jsc@fz-juelich.de

http://www.fz-juelich.de/jsc/sionlib


Servicemeu

Homepage

Logo

 

 

 

YOUR OPINION MATTERS!

 

Dear visitor,

To make our website suit your needs even more and to give it a more appealing design, we would like you to answer a few short questions.

Answering these questions will take approx. 10 min.

Start now Close window

Thank you for your support!

 

In case you have already taken part in our survey or in case you have no time to take part now, you can simply close the window by clicking "close".

If you have any questions on the survey, please do not hesitate to contact: webumfrage@fz-juelich.de.

 

Your Team at Forschungszentrum Jülich

 

Note: Forschungszentrum Jülich works with the market research institute SKOPOS to anonymously conduct and analyze the survey. SKOPOS complies with the statutory requirements on data protection as well as with the regulations of ADM (Arbeitskreis Deutscher Markt- und Sozialforschungsinstitute e.V.) and ESOMAR (Europäische Gesellschaft für Meinungs- und Marketingforschung). Your data will not be forwarded to third parties.