High-Level APIs

The high-level APIs are designed for simple tasks for which performance is not critical. Unlike the Full APIs, the high-level APIs only require a single object handler resembling a C++ fstream or a Python file I/O idiom. The high-level APIs are recommended to both first-time and advanced users; the low-level APIs being recommended only when performance testing identifies a bottleneck or when more control is needed.

Typical scenarios for using the simple high-level APIs are:

• Reading a file to perform data analysis with libraries (matplotlib, scipy, etc.)

• Interactive: few calls make interactive usage easier.

• Saving data to files is small or personal projects

• Online frameworks: e.g. Jupyter notebooks, see python-mpi examples running on MyBinder

The designed functionality syntax is closely related to the native language IO bindings for formatted text files e.g. C++ fstream getline, and Python file IO. The main function calls are: open (or constructor in C++), write, read and close (or destructor in C++). In addition, ADIOS2 borrows the corresponding language native syntax for advancing lines to advance the step in write mode, and for a “step-by-step” streaming basis in read mode. See each language section in this chapter for a write/read example.

Note

The simplified APIs are based on language native file IO interface. Hence write and read calls are always synchronized and variables data memory is ready to use immediately after these calls.

Currently ADIOS2 support bindings for the following languages and their minimum standards:

Language	Standard	Interface	Based on
C++	11/newer	#include adios2.h	fstream
Python	2.7/3	import adios2	Python IO
Matlab

The following sections provide a summary of the API calls on each language and links to Write and Read examples to put it all together.

C++ High-Level API

C++11 High-Level APIs are based on a single object adios2::fstream

Caution

DO NOT place use namespace adios2 in your C++ code. Use adios2::fstream directly to prevent conflicts with std::fstream.

C++11 Write example

#include <adios2.h>
...

int rank, size;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);

// Nx, Ny from application, std::size_t
const adios2::Dims shape{Nx, Ny * static_cast<std::size_t>(size)};
const adios2::Dims start{0, Ny * static_cast<std::size_t>(rank)};
const adios2::Dims count{Nx, Ny};

adios2::fstream oStream("cfd.bp", adios2::fstream::out, MPI_COMM_WORLD);

// NSteps from application
for (std::size_t step = 0; step < NSteps; ++step)
{
    if(rank == 0 && step == 0) // global variable
    {
        oStream.write<int32_t>("size", size);
    }

    // physicalTime double, <double> is optional
    oStream.write<double>( "physicalTime", physicalTime );
    // T and P are std::vector<float>
    oStream.write( "temperature", T.data(), shape, start, count );
    // adios2::endl will advance the step after writing pressure
    oStream.write( "pressure", P.data(), shape, start, count, adios2::end_step );

}

// Calling close is mandatory!
oStream.close();

C++11 Read “step-by-step” example

#include <adios2.h>
...

int rank, size;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);

// Selection Window from application, std::size_t
const adios2::Dims start{0, 0};
const adios2::Dims count{SelX, SelY};

if( rank == 0)
{
   // if only one rank is active use MPI_COMM_SELF
   adios2::fstream iStream("cfd.bp", adios2::fstream::in, MPI_COMM_SELF);

   adios2::fstep iStep;
   while (adios2::getstep(iStream, iStep))
   {
       if( iStep.currentstep() == 0 )
       {
           const std::size_t sizeOriginal = iStep.read<std::size_t>("size");
       }
       const double physicalTime = iStream.read<double>( "physicalTime");
       const std::vector<float> temperature = iStream.read<float>( "temperature", start, count );
       const std::vector<float> pressure = iStream.read<float>( "pressure", start, count );
   }
   // Don't forget to call close!
   iStream.close();
}

adios2::fstream API documentation

class adios2::fstream

Public Types

enum openmode

Available open modes for adios2::fstream constructor or open calls

Values:

enumerator out

write

enumerator in

read

enumerator in_random_access

read_random_access

enumerator app

append, not yet supported

Public Functions

fstream(const std::string &name, adios2::fstream::openmode mode, MPI_Comm comm, const std::string engineType = "BPFile")

High-level API MPI constructor, based on C++11 fstream. Allows for passing parameters in source code.

Parameters

• name – stream name

• mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)

• comm – MPI communicator establishing domain for fstream

• engineType – available adios2 engine

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

fstream(const std::string &name, const adios2::fstream::openmode mode, MPI_Comm comm, const std::string &configFile, const std::string ioInConfigFile)

High-level API MPI constructor, based on C++11 fstream. Allows for runtime config file.

Parameters

• name – stream name

• mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)

• comm – MPI communicator establishing domain for fstream

• configFile – adios2 runtime configuration file

• ioInConfigFile – specific io name in configFile

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

fstream(const std::string &name, const adios2::fstream::openmode mode, const std::string engineType = "BPFile")

High-level API non-MPI constructor, based on C++11 fstream. Allows for passing parameters in source code.

Parameters

• name – stream name

• mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)

• engineType – available adios2 engine

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

fstream(const std::string &name, const adios2::fstream::openmode mode, const std::string &configFile, const std::string ioInConfigFile)

High-level API MPI constructor, based on C++11 fstream. Allows for runtime config file.

Parameters

• name – stream name

• mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)

• configFile – adios2 runtime configuration file

• ioInConfigFile – specific io name in configFile

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

fstream() = default

Empty constructor, allows the use of open later in the code

~fstream() = default

Using RAII STL containers only

explicit operator bool() const noexcept

Checks if fstream object is valid

void open(const std::string &name, const openmode mode, MPI_Comm comm, const std::string engineType = "BPFile")

High-level API MPI open, based on C++11 fstream. Allows for passing parameters in source code. Used after empty constructor.

Parameters

• name – stream name

• mode – adios2::fstream::in (Read), adios2::fstream::out (Write), adios2::fstream::app (Append)

• comm – MPI communicator establishing domain for fstream

• engineType – available adios2 engine

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

void open(const std::string &name, const openmode mode, MPI_Comm comm, const std::string configFile, const std::string ioInConfigFile)

High-level API MPI constructor, based on C++11 fstream. Allows for runtime config file. Used after empty constructor.

Parameters

• name – stream name

• mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)

• comm – MPI communicator establishing domain for fstream

• configFile – adios2 runtime configuration file

• ioInConfigFile – specific io name in configFile

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

void open(const std::string &name, const openmode mode, const std::string engineType = "BPFile")

High-level API non-MPI open, based on C++11 fstream. Allows for passing parameters in source code. Used after empty constructor.

Parameters

• name – stream name

• mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)

• engineType – available adios2 engine

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

void open(const std::string &name, const openmode mode, const std::string configFile, const std::string ioInConfigFile)

High-level API non-MPI constructor, based on C++11 fstream. Allows for runtime config file. Used after empty constructor.

Parameters

• name – stream name

• mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)

• configFile – adios2 runtime configuration file

• ioInConfigFile – specific io name in configFile

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

void set_parameter(const std::string key, const std::string value) noexcept

Set a single stream parameter based on Engine supported parameters. MUST be passed before the first call to write or read. See: https://adios2.readthedocs.io/en/latest/engines/engines.html

Parameters

• key – input parameter key

• value – input parameter value

void set_parameters(const adios2::Params &parameters) noexcept

Set stream parameters based on Engine supported parameters. MUST be passed before the first call to write or read. See: https://adios2.readthedocs.io/en/latest/engines/engines.html

Parameters

parameters – input key/value parameters

template<class T>
void write_attribute(const std::string &name, const T &value, const std::string &variableName = "", const std::string separator = "/", const bool endStep = false)

Define attribute inside fstream or for a variable after write. Single value input version.

Parameters

• name – unique attribute identifier IO object or for a Variable if variableName is not empty (associated to a variable)

• value – single data value

• variableName – default is empty, if not empty attributes is associated to a variable after a write

• separator – default is “/”, hierarchy between variable name and attribute, e.g. variableName/attribute1, variableName::attribute1. Not used if variableName is empty.

• endStep – similar to std::endStep, end current step and flush (default). Use adios2::endStep for true.

template<class T>
void write_attribute(const std::string &name, const T *data, const size_t size, const std::string &variableName = "", const std::string separator = "/", const bool endStep = false)

Define attribute inside fstream or for a variable after write. Array input version.

Parameters

• name – unique attribute identifier IO object or for a Variable if variableName is not empty (associated to a variable)

• data – pointer to user data

• size – number of data elements

• variableName – default is empty, if not empty attributes is associated to a variable after a write

• separator – default is “/”, hierarchy between variable name and attribute, e.g. variableName/attribute1, variableName::attribute1. Not used if variableName is empty.

• endStep – similar to std::endStep, end current step and flush (default). Use adios2::endStep for true.

template<class T>
void write(const std::string &name, const T *data, const adios2::Dims &shape = adios2::Dims(), const adios2::Dims &start = adios2::Dims(), const adios2::Dims &count = adios2::Dims(), const bool endStep = false)

writes a self-describing array variable

Parameters

• name – variable name

• data – variable data data

• shape – variable global MPI dimensions. Pass empty for local variables.

• start – variable offset for current MPI rank. Pass empty for local variables.

• count – variable dimension for current MPI rank. Local variables only have count.

• endStep – similar to std::endStep, end current step and flush (default). Use adios2::endStep if true.

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

template<class T>
void write(const std::string &name, const T *data, const adios2::Dims &shape, const adios2::Dims &start, const adios2::Dims &count, const adios2::vParams &operations, const bool endStep = false)

write overload that allows passing supported operations (e.g. lossy compression “zfp”, “mgard”, “sz”) to a self-described array variable

Parameters

• name – variable name

• data – variable data data

• shape – variable global MPI dimensions. Pass empty for local variables.

• start – variable offset for current MPI rank. Pass empty for local variables.

• count – variable dimension for current MPI rank. Local variables only have count.

• operations – vector of operations, each entry is a std::pair:

• endStep – similar to std::endStep, end current step and flush (default). Use adios2::endStep if true.

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

template<class T>
void write(const std::string &name, const T &value, const bool isLocalValue = false, const bool endStep = false)

Write a self-describing single-value variable

Parameters

• name – variable name

• value – variable data value (can be r-value)

• isLocalValue – true: local value (returned as GlobalArray), false: global value (returned as global value)

• endStep – similar to std::endStep, end current step and flush (default). Use adios2::endStep for true.

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

template<class T>
void read(const std::string &name, T *data, const size_t blockID = 0)

Reads into a pre-allocated pointer. When used with adios2::getstep reads current step

Parameters

• name – variable name

• data – pre-allocated pointer to hold read data

• blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

template<class T>
void read(const std::string &name, T &value, const size_t blockID = 0)

Reads a value. When used with adios2::getstep reads current step value

Parameters

• name – variable name

• value – output value, if variable is not found (name and type don’t match) the returned value address becomes nullptr

• blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

template<class T>
void read(const std::string &name, T *data, const size_t stepsStart, const size_t stepsCount = 1, const size_t blockID = 0)

Read accessing steps in random access mode. Not be used with adios2::getstep as it throw an exception when reading in stepping mode.

Parameters

• name – variable name

• data – pre-allocated pointer to hold read data, if variable is not found (name and type don’t match) it becomes nullptr

• stepsStart – variable initial step (relative to the variable first appearance, not absolute step in stream)

• stepsCount – variable number of steps form step_start, don’t have to be contiguous, necessarily

• blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

template<class T>
void read(const std::string &name, T &value, const size_t step, const size_t blockID = 0)

Reads into a single value for a single step. Not be used with adios2::getstep as it throws an exception when reading in stepping mode.

Parameters

• name – variable name

• value – filled with value, if variable is not found (name, type and step don’t match) the returned value address becomes nullptr

• step – selected single step

• blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

template<class T>
void read(const std::string &name, T *data, const adios2::Dims &start, const adios2::Dims &count, const size_t blockID = 0)

Reads into a pre-allocated pointer a selection piece in dimension. When used with adios2::getstep reads current step

Parameters

• name – variable name

• data – pre-allocated pointer to hold read data, if variable is not found (name and type don’t match) it becomes nullptr

• start – variable local offset selection

• count – variable local dimension selection from start

• blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

template<class T>
void read(const std::string &name, T *data, const adios2::Dims &start, const adios2::Dims &count, const size_t stepsStart, const size_t stepsCount, const size_t blockID = 0)

Reads into a pre-allocated pointer a selection piece in dimensions and steps. Not be used with adios2::getstep as it throws an exception when reading in stepping mode.

Parameters

• name – variable name

• data – pre-allocated pointer to hold read data, if variable is not found (name and type don’t match) it becomes a nullptr

• start – variable local offset selection

• count – variable local dimension selection from start

• stepsStart – variable initial step (relative to the variable first appearance, not absolute step in stream)

• stepsCount – variable number of steps form step_start, don’t have to be necessarily contiguous

• blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

template<class T>
std::vector<T> read(const std::string &name, const size_t blockID = 0)

Reads entire variable for current step (streaming mode: step by step)

Parameters

• name – variable name

• blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

Returns

data of variable name for current step. Single data will have a size=1 vector

template<class T>
std::vector<T> read(const std::string &name, const size_t stepsStart, const size_t stepsCount = 1, const size_t blockID = 0)

Returns a vector with full variable dimensions for the current step selection. Not be used with adios2::getstep as it throw an exception when reading in stepping mode.

Parameters

• name – variable name

• stepsStart – variable initial step (relative to the variable first appearance, not absolute step in stream)

• stepsCount – variable number of steps form step_start, don’t have to be contiguous, necessarily

• blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

Returns

data of variable name for current step, empty if exception is thrown

template<class T>
std::vector<T> read(const std::string &name, const Dims &start, const Dims &count, const size_t blockID = 0)

Reads a selection piece in dimension for current step (streaming mode: step by step)

Parameters

• name – variable name

• start – variable local offset selection

• count – variable local dimension selection from start

• blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

Returns

data of variable name for current step, empty if exception is thrown

template<class T>
std::vector<T> read(const std::string &name, const Dims &start, const Dims &count, const size_t stepsStart, const size_t stepsCount, const size_t blockID = 0)

Reads a selection piece in dimension and a selection piece in steps (non-streaming mode). Not be used with adios2::getstep as it throw an exception when reading in stepping mode.

Parameters

• name – variable name

• start – variable local offset selection

• count – variable local dimension selection from start

• stepsStart – variable initial step (relative to the variable first appearance, not absolute step in stream)

• stepsCount – variable number of steps form step_start, don’t have to be contiguous, necessarily

• blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

Returns

variable data, empty if exception is thrown

template<class T>
std::vector<T> read_attribute(const std::string &name, const std::string &variableName = "", const std::string separator = "/")

Reads an attribute returning a vector For single data vector size = 1

Parameters

• name – attribute name

• variableName – default is empty, if not empty look for an attribute associated to a variable

• separator – default is “/”, hierarchy between variable name and attribute, e.g. variableName/attribute1, variableName::attribute1. Not used if variableName is empty.

Returns

vector containing attribute data

void end_step()

At write: ends the current step At read: use it in streaming mode to inform the writer that the reader is done consuming the step. No effect for file engines.

void close()

close current stream becoming inaccessible

size_t current_step() const noexcept

Return current step when getstep is called in a loop, read mode only

Returns

current step

Friends

friend bool getstep(adios2::fstream &stream, adios2::fstep &step)

Gets step from stream Based on std::getline, enables reading on a step-by-step basis in a while or for loop. Read mode only

Parameters

• stream – input stream containing steps

• step – output object current step, adios2::fstep in an alias to adios2::fstream with scope narrowed to one step

Returns

true: step is valid, false: step is invalid (end of stream).

Python High-Level API

Python simple bindings follow closely Python style directives. Just like the full APIs, they rely on numpy and, optionally, on mpi4py, if the underlying ADIOS2 library is compiled with MPI.

For online examples on MyBinder :

• Python-MPI Notebooks

• Python-noMPI Notebooks

Python Write example

from mpi4py import MPI
import numpy as np
import adios2

comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()
...
shape = [size * nx]
start = [rank * nx]
count = [nx]

# with-as will call adios2.close on fh at the end
with adios2.open("cfd.bp", "w", comm) as fh:

   # NSteps from application
   for i in range(0, NSteps):

      if(rank == 0 and i == 0):
         fh.write("size", np.array([size]))

      fh.write("physical_time", np.array([physical_time]) )
      # temperature and pressure are numpy arrays
      fh.write("temperature", temperature, shape, start, count)
      # advances to next step
      fh.write("pressure", pressure, shape, start, count, end_step=True)

Python Read “step-by-step” example

from mpi4py import MPI
import numpy as np
import adios2

comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()
...
shape = [size * nx]
start = [rank * nx]
count = [nx]

if( rank == 0 ):
   # with-as will call adios2.close on fh at the end
   # if only one rank is active pass MPI.COMM_SELF
   with adios2.open("cfd.bp", "r", MPI.COMM_SELF) as fh:

      for fstep in fh:

         # inspect variables in current step
         step_vars = fstep.available_variables()

         # print variables information
         for name, info in step_vars.items():
             print("variable_name: " + name)
             for key, value in info.items():
                print("\t" + key + ": " + value)
             print("\n")

         # track current step
         step = fstep.current_step()
         if( step == 0 ):
            size_in = fstep.read("size")

         # read variables return a numpy array with corresponding selection
         physical_time = fstep.read("physical_time")
         temperature = fstep.read("temperature", start, count)
         pressure = fstep.read("pressure", start, count)

Caution

When reading in stepping mode with the for-in directive, as in the example above, use the step handler (fstep) inside the loop rather than the global handler (fh)

File class API

adios2.open(*args, **kwargs)

Overloaded function.

1. open(name: str, mode: str, comm: MPI4PY_Comm, engine_type: str=’BPFile’) -> adios2::py11::File

   Simple API MPI open, based on python IO. Allows for passing parameters in source code.

   Parameters

   name

   stream name

   mode

   “w” : write, “r” : read, “a” : append (append not yet supported)

   comm (mpi4py)

   MPI communicator

   engine_type

   adios2 engine type, default=BPFile

   Returns

   file (adios2 stream)

   handler to adios File for the simple Python API

2. open(name: str, mode: str, comm: MPI4PY_Comm, config_file: str, io_in_config_file: str) -> adios2::py11::File

   Simple API MPI open, based on python IO. Allows for passing a runtime configuration file in xml format and the name of the io element related to the returning File.

   Parameters

   name

   stream name

   mode

   “w” : write, “r” : read, “a” : append (append not yet supported)

   comm (mpi4py)

   MPI communicator

   config_file

   adios2 runtime configuration file name, in xml format

   io_in_config_file

   io element in configfile related to returning File

   Returns

   file (adios2 stream)

   handler to adios File for the simple Python API

3. open(name: str, mode: str, engine_type: str=’BPFile’) -> adios2::py11::File

High-level API, file object open

4. open(name: str, mode: str, config_file: str, io_in_config_file: str) -> adios2::py11::File

High-level API, file object open with a runtime config file

class adios2.File

add_transport(self: adios2.File, type: str, parameters: Dict[str, str] = {}) → int

Adds a transport and its parameters to current IO. Must be supported by current engine type.

Parameters

type

must be a supported transport type for current engine.

parameters

acceptable parameters for a particular transport CAN’T use the keywords “Transport” or “transport” in key

Returns

transport_index

handler to added transport

available_attributes(self: adios2.File) → Dict[str, Dict[str, str]]

Returns a 2-level dictionary with attribute information. Read mode only.

Returns

attributes dictionary

key

attribute name

value

attribute information dictionary

available_variables(self: adios2.File) → Dict[str, Dict[str, str]]

Returns a 2-level dictionary with variable information. Read mode only.

Returns

variables dictionary

key

variable name

value

variable information dictionary

close(self: adios2.File) → None

Closes file, thus becoming unreachable. Not required if using open in a with-as statement. Required in all other cases per-open to avoid resource leaks.

current_step(self: adios2.File) → int

Inspect current step when using for-in loops, read mode only

Returns

current step

end_step(self: adios2.File) → None

Write mode: advances to the next step. Convenient when declaring variable attributes as advancing to the next step is not attached to any variable.

Read mode: in streaming mode releases the current step (no effect in file based engines)

read(*args, **kwargs)

Overloaded function.

1. read(self: adios2.File, name: str, block_id: int=0) -> array

   Reads entire variable for current step (streaming mode step by step)

   Parameters

   name

   variable name

   block_id

   required for local array variables

   Returns

   array

   values of variable name for current step. Single values will have a shape={1} numpy array

2. read(self: adios2.File, name: str, start: List[int]=[], count: List[int]=[], block_id: int=0) -> array

   Reads a selection piece in dimension for current step (streaming mode step by step)

   Parameters

   name

   variable name

   start

   variable local offset selection (defaults to (0, 0, …)

   count

   variable local dimension selection from start defaults to whole array for GlobalArrays, or selected Block size for LocalArrays

   block_id

   required for local array variables

   Returns

   array

   values of variable name for current step empty if exception is thrown

3. read(self: adios2.File, name: str, start: List[int], count: List[int], step_start: int, step_count: int, block_id: int=0) -> array

   Random access read allowed to select steps, only valid with File Engines

   Parameters

   name

   variable to be read

   start

   variable offset dimensions

   count

   variable local dimensions from offset

   step_start

   variable step start

   step_count

   variable number of steps to read from step_start

   block_id

   required for local array variables

   Returns

   array

   resulting array from selection

read_attribute(self: adios2.File, name: str, variable_name: str = '', separator: str = '/') → array

Reads a numpy based attribute

Parameters

name

attribute name

variable_name

if attribute is associated with a variable

separator

concatenation string between variable_name and attribute e.g. variable_name + separator + name (var/attr) Not used if variable_name is empty

Returns

array

resulting array attribute data

read_attribute_string(self: adios2.File, name: str, variable_name: str = '', separator: str = '/') → List[str]

Read a string attribute

Parameters

name

attribute name

variable_name

if attribute is associated with a variable

separator

concatenation string between variable_name and attribute e.g. variable_name + separator + name (var/attr) Not used if variable_name is empty

Returns

list

resulting string list attribute data

read_string(*args, **kwargs)

Overloaded function.

1. read_string(self: adios2.File, name: str, block_id: int=0) -> List[str]

   Reads string value for current step (use for streaming mode step by step)

   Parameters

   name

   string variable name

   block_id

   required for local variables

   Returns

   list

   data string values. For global values: returns 1 element For local values: returns n-block elements

2. read_string(self: adios2.File, name: str, step_start: int, step_count: int, block_id: int=0) -> List[str]

   Reads string value for a certain step (random access mode)

   Parameters

   name

   string variable name

   step_start

   variable step start

   step_count

   variable number of steps to read from step_start

   block_id

   required for local variables

   Returns

   string

   data string values for a certain step range.

set_parameter(self: adios2.File, key: str, value: str) → None

Sets a single parameter. Overwrites value if key exists.

Parameters

key

input parameter key

value

parameter value

set_parameters(self: adios2.File, parameters: Dict[str, str]) → None

Sets parameters using a dictionary. Removes any previous parameter.

Parameters

parameters

input key/value parameters

value

parameter value

steps(self: adios2.File) → int

Inspect available number of steps, for file engines, read mode only

Returns

steps

write(*args, **kwargs)

Overloaded function.

1. write(self: adios2.File, name: str, array: array, shape: List[int]=[], start: List[int]=[], count: List[int]=[], end_step: bool=False) -> None

   writes a self-describing array (numpy) variable

   Parameters

   name

   variable name

   array

   variable data values

   shape

   variable global MPI dimensions. Pass empty numpy array for local variables.

   start

   variable offset for current MPI rank. Pass empty numpy array for local variables.

   count

   variable dimension for current MPI rank. Pass a numpy array for local variables.

   end_step

   end current step, begin next step and flush (default = false).

2. write(self: adios2.File, name: str, array: array, shape: List[int], start: List[int], count: List[int], operations: List[Tuple[str, Dict[str, str]]], end_step: bool=False) -> None

   writes a self-describing array (numpy) variable with operations e.g. compression: ‘zfp’, ‘mgard’, ‘sz’

   Parameters

   name

   variable name

   array

   variable data values

   shape

   variable global MPI dimensions. Pass empty numpy array for local variables.

   start

   variable offset for current MPI rank. Pass empty numpy array for local variables.

   count

   variable dimension for current MPI rank. Pass a numpy array for local variables.

   end_step

   end current step, begin next step and flush (default = false).

3. write(self: adios2.File, name: str, array: array, local_value: bool=False, end_step: bool=False) -> None

   writes a self-describing single value array (numpy) variable

   Parameters

   name

   variable name

   array

   variable data single value

   local_value

   true: local value, false: global value

   end_step

   end current step, begin next step and flush (default = false).

4. write(self: adios2.File, name: str, string: str, local_value: bool=False, end_step: bool=False) -> None

   writes a self-describing single value string variable

   Parameters

   name

   variable name

   string

   variable data single value

   local_value

   true: local value, false: global value

   end_step

   end current step, begin next step and flush (default = false).

write_attribute(*args, **kwargs)

Overloaded function.

1. write_attribute(self: adios2.File, name: str, array: array, variable_name: str=’’, separator: str=’/’, end_step: bool=False) -> None

   writes a self-describing single value array (numpy) variable

   Parameters

   name

   attribute name

   array

   attribute numpy array data

   variable_name

   if attribute is associated with a variable

   separator

   concatenation string between variable_name and attribute e.g. variable_name + separator + name (“var/attr”) Not used if variable_name is empty

   end_step

   end current step, begin next step and flush (default = false).

2. write_attribute(self: adios2.File, name: str, string_value: str, variable_name: str=’’, separator: str=’/’, end_step: bool=False) -> None

   writes a self-describing single value array (numpy) variable

   Parameters

   name

   attribute name

   string_value

   attribute single string

   variable_name

   if attribute is associated with a variable

   separator

   concatenation string between variable_name and attribute e.g. variable_name + separator + name (“var/attr”) Not used if variable_name is empty

   end_step

   end current step, begin next step and flush (default = false).

3. write_attribute(self: adios2.File, name: str, string_array: List[str], variable_name: str=’’, separator: str=’/’, end_step: bool=False) -> None

   writes a self-describing single value array (numpy) variable

   Parameters

   name

   attribute name

   string_array

   attribute string array

   variable_name

   if attribute is associated with a variable

   separator

   concatenation string between variable_name and attribute e.g. variable_name + separator + name (“var/attr”) Not used if variable_name is empty

   end_step

   end current step, begin next step and flush (default = false).

Matlab simple bindings

The ADIOS Matlab API supports reading data from ADIOS BP files with a simplified API that consists of three functions:

• ADIOSOPEN returns a structure with information on an ADIOS BP File (variables and attributes).

• ADIOSREAD reads in a variable from the file. It expects the info structure returned by ADIOSOPEN.

• ADIOSCLOSE closes the file.

Organization of an ADIOS BP file

An ADIOS BP file contains a set of variables and attributes. Each variable in the group has a path, which defines a logical hierarchy of the variables within the file.

Time dimension of a variable

Variables can be written several times from a program, if they have a time dimension. The reader exposes the variables with an extra dimension, i.e. a 2D variable written over time is seen as a 3D variable. In MATLAB, the extra dimension is the last dimension (the slowest changing dimension). Since the reader allows reading an arbitrary slice of a variable, data for one timestep can be read in with slicing.

Min/max of arrays

The ADIOS BP format stores the min/max values in each variable. The info structure therefore contains these min/max values. There is practically no overhead to provide this information (along with the values of all attributes) even for file sizes of several terabytes.

In the Matlab console use help for these functions

>>> help adiosopen
>>> help adiosread
>>> help adiosclose

ADIOSOPEN

FILE = adiosopen(PATH)

Open a file for reading pointed by PATH and return an information structure (FILE).

The returned FILE structure contains the following information

Name              File path

Handlers          Object handlers to pass on to ADIOS functions
  FileHandler        uint64 file handler
  GroupHandler       uint64 IO group object handler
  ADIOSHandler       uint64 ADIOS object handler

Variables         Structure array of variables
     Name            Path of variable
     Type            Matlab type class of data
     Dims            Array of dimensions
     StepsStart      First step's index for this variable in file, always at least 1
     StepsCount      Number of steps for this variable in file, always at least 1
     GlobalMin       Global minimum  of the variable (1-by-1 mxArray)
     GlobalMax       Global maximum of the variable

Attribute         Structure array of attributes
     Name            Path of attribute
     Type            Matlab type class of data
     Value           Attribute value

ADIOSREAD

Read data from a BP file opened with adiosopen. Provide the structure returned by adiosopen as the first input argument, and the path to a variable. Inspect file.Variables and file.Attributes for the list of variables and attributes available in a file.

data = adiosread(file, VARPATH)

Read the entire variable VARPATH from a BP file. file is the output of ADIOSOPEN. VARPATH is a string to a variable or attribute. If an N-dimensional array variable has multiple steps in the file this function reads all steps and returns an N+1 dimensional array where the last dimension equals the number of steps.

data = adiosread(file, INDEX)

Read the entire variable from a BP file. INDEX points to a variable in the file.Variables array.

data = adiosread(..., START, COUNT, STEPSTART, STEPCOUNT)

Read a portion of a variable.

START and COUNT:
A slice is defined as two arrays of N integers, where N is the
number of dimensions of the variable, describing the
"start" and "count" values. The "start" values start from 1.
    E.g. [1 5], [10 2] reads the first 10 values in the first dimension
and 2 values from the 5th position in the second dimension resulting in
a 10-by-2 array.
    You can use negative numbers to index from the end of the array
as in python. -1 refers to the last element of the array, -2 the one
before and so on.
    E.g. [-1], [1] reads in the last value of a 1D array.
         [1], [-1] reads in the complete 1D array.

STEPSTART and STEPCOUNT:
Similarly, the number of steps from a specific step can be read instead
of all data. Steps start from 1. Negative index can be used as well.
    E.g. -1, 1  will read in the last step from the file
         n, -1  will read all steps from 'n' to the last one

ADIOSCLOSE

adiosclose(file)

Close file and free internal data structures. file is the structure returned by adiosopen.