High-Level APIs

The high-level APIs are designed for simple tasks for which performance is not critical. Unlike the Full APIs, the high-level APIs only require a single object handler resembling a C++ fstream or a Python file I/O idiom. The high-level APIs are recommended to both first-time and advanced users; the low-level APIs being recommended only when performance testing identifies a bottleneck or when more control is needed.

Typical scenarios for using the simple high-level APIs are:

Reading a file to perform data analysis with libraries (matplotlib, scipy, etc.)
Interactive: few calls make interactive usage easier.
Saving data to files is small or personal projects
Online frameworks: e.g. Jupyter notebooks, see python-mpi examples running on MyBinder

The designed functionality syntax is closely related to the native language IO bindings for formatted text files e.g. C++ fstream getline, and Python file IO. The main function calls are: open (or constructor in C++), write, read and close (or destructor in C++). In addition, ADIOS2 borrows the corresponding language native syntax for advancing lines to advance the step in write mode, and for a “step-by-step” streaming basis in read mode. See each language section in this chapter for a write/read example.

Note

The simplified APIs are based on language native file IO interface. Hence write and read calls are always synchronized and variables data memory is ready to use immediately after these calls.

Currently ADIOS2 support bindings for the following languages and their minimum standards:

Language	Standard	Interface	Based on
C++	11/newer	`#include adios2.h`	`fstream`
Python	2.7/3	`import adios2`	Python IO
Matlab

The following sections provide a summary of the API calls on each language and links to Write and Read examples to put it all together.

C++ High-Level API

C++11 High-Level APIs are based on a single object adios2::fstream

Caution

DO NOT place use namespace adios2 in your C++ code. Use adios2::fstream directly to prevent conflicts with std::fstream.

C++11 Write example

#include <adios2.h>
...

int rank, size;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);

// Nx, Ny from application, std::size_t
const adios2::Dims shape{Nx, Ny * static_cast<std::size_t>(size)};
const adios2::Dims start{0, Ny * static_cast<std::size_t>(rank)};
const adios2::Dims count{Nx, Ny};

adios2::fstream oStream("cfd.bp", adios2::fstream::out, MPI_COMM_WORLD);

// NSteps from application
for (std::size_t step = 0; step < NSteps; ++step)
{
    if(rank == 0 && step == 0) // global variable
    {
        oStream.write<int32_t>("size", size);
    }

    // physicalTime double, <double> is optional
    oStream.write<double>( "physicalTime", physicalTime );
    // T and P are std::vector<float>
    oStream.write( "temperature", T.data(), shape, start, count );
    // adios2::endl will advance the step after writing pressure
    oStream.write( "pressure", P.data(), shape, start, count, adios2::end_step );

}

// Calling close is mandatory!
oStream.close();

C++11 Read “step-by-step” example

#include <adios2.h>
...

int rank, size;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);

// Selection Window from application, std::size_t
const adios2::Dims start{0, 0};
const adios2::Dims count{SelX, SelY};

if( rank == 0)
{
   // if only one rank is active use MPI_COMM_SELF
   adios2::fstream iStream("cfd.bp", adios2::fstream::in, MPI_COMM_SELF);

   adios2::fstep iStep;
   while (adios2::getstep(iStream, iStep))
   {
       if( iStep.currentstep() == 0 )
       {
           const std::size_t sizeOriginal = iStep.read<std::size_t>("size");
       }
       const double physicalTime = iStream.read<double>( "physicalTime");
       const std::vector<float> temperature = iStream.read<float>( "temperature", start, count );
       const std::vector<float> pressure = iStream.read<float>( "pressure", start, count );
   }
   // Don't forget to call close!
   iStream.close();
}

`adios2::fstream` API documentation

class fstream

Public Types

enum openmode

Available open modes for adios2::fstream constructor or open calls

Values:

enumerator out: write

enumerator in: read

enumerator in_random_access: read_random_access

enumerator app: append, not yet supported

Public Functions

fstream(const std::string &name, adios2::fstream::openmode mode, MPI_Comm comm, const std::string engineType = "BPFile")

High-level API MPI constructor, based on C++11 fstream. Allows for passing parameters in source code.

Parameters

name – stream name
mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)
comm – MPI communicator establishing domain for fstream
engineType – available adios2 engine

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

fstream(const std::string &name, const adios2::fstream::openmode mode, MPI_Comm comm, const std::string &configFile, const std::string ioInConfigFile)

High-level API MPI constructor, based on C++11 fstream. Allows for runtime config file.

Parameters

name – stream name
mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)
comm – MPI communicator establishing domain for fstream
configFile – adios2 runtime configuration file
ioInConfigFile – specific io name in configFile

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

fstream(const std::string &name, const adios2::fstream::openmode mode, const std::string engineType = "BPFile")

High-level API non-MPI constructor, based on C++11 fstream. Allows for passing parameters in source code.

Parameters

name – stream name
mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)
engineType – available adios2 engine

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

fstream(const std::string &name, const adios2::fstream::openmode mode, const std::string &configFile, const std::string ioInConfigFile)

High-level API MPI constructor, based on C++11 fstream. Allows for runtime config file.

Parameters

name – stream name
mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)
configFile – adios2 runtime configuration file
ioInConfigFile – specific io name in configFile

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

fstream() = default: Empty constructor, allows the use of open later in the code

~fstream() = default: Using RAII STL containers only

explicit operator bool() const noexcept: Checks if fstream object is valid

void open(const std::string &name, const openmode mode, MPI_Comm comm, const std::string engineType = "BPFile")

High-level API MPI open, based on C++11 fstream. Allows for passing parameters in source code. Used after empty constructor.

Parameters

name – stream name
mode – adios2::fstream::in (Read), adios2::fstream::out (Write), adios2::fstream::app (Append)
comm – MPI communicator establishing domain for fstream
engineType – available adios2 engine

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

void open(const std::string &name, const openmode mode, MPI_Comm comm, const std::string configFile, const std::string ioInConfigFile)

High-level API MPI constructor, based on C++11 fstream. Allows for runtime config file. Used after empty constructor.

Parameters

name – stream name
mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)
comm – MPI communicator establishing domain for fstream
configFile – adios2 runtime configuration file
ioInConfigFile – specific io name in configFile

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

void open(const std::string &name, const openmode mode, const std::string engineType = "BPFile")

High-level API non-MPI open, based on C++11 fstream. Allows for passing parameters in source code. Used after empty constructor.

Parameters

name – stream name
mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)
engineType – available adios2 engine

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

void open(const std::string &name, const openmode mode, const std::string configFile, const std::string ioInConfigFile)

High-level API non-MPI constructor, based on C++11 fstream. Allows for runtime config file. Used after empty constructor.

Parameters

name – stream name
mode – fstream::in (Read), fstream::out (Write), fstream::app (Append)
configFile – adios2 runtime configuration file
ioInConfigFile – specific io name in configFile

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

void set_parameter(const std::string key, const std::string value) noexcept

Set a single stream parameter based on Engine supported parameters. MUST be passed before the first call to write or read. See: https://adios2.readthedocs.io/en/latest/engines/engines.html

Parameters

key – input parameter key
value – input parameter value

void set_parameters(const adios2::Params &parameters) noexcept

Set stream parameters based on Engine supported parameters. MUST be passed before the first call to write or read. See: https://adios2.readthedocs.io/en/latest/engines/engines.html

Parameters: parameters – input key/value parameters

template<class T> void write_attribute(const std::string &name, const T &value, const std::string &variableName = "", const std::string separator = "/", const bool endStep = false)

Define attribute inside fstream or for a variable after write. Single value input version.

Parameters

name – unique attribute identifier IO object or for a Variable if variableName is not empty (associated to a variable)
value – single data value
variableName – default is empty, if not empty attributes is associated to a variable after a write
separator – default is “/”, hierarchy between variable name and attribute, e.g. variableName/attribute1, variableName::attribute1. Not used if variableName is empty.
endStep – similar to std::endStep, end current step and flush (default). Use adios2::endStep for true.

template<class T> void write_attribute(const std::string &name, const T *data, const size_t size, const std::string &variableName = "", const std::string separator = "/", const bool endStep = false)

Define attribute inside fstream or for a variable after write. Array input version.

Parameters

name – unique attribute identifier IO object or for a Variable if variableName is not empty (associated to a variable)
data – pointer to user data
size – number of data elements
variableName – default is empty, if not empty attributes is associated to a variable after a write
separator – default is “/”, hierarchy between variable name and attribute, e.g. variableName/attribute1, variableName::attribute1. Not used if variableName is empty.
endStep – similar to std::endStep, end current step and flush (default). Use adios2::endStep for true.

template<class T> void write(const std::string &name, const T *data, const adios2::Dims &shape = adios2::Dims(), const adios2::Dims &start = adios2::Dims(), const adios2::Dims &count = adios2::Dims(), const bool endStep = false)

writes a self-describing array variable

Parameters

name – variable name
data – variable data data
shape – variable global MPI dimensions. Pass empty for local variables.
start – variable offset for current MPI rank. Pass empty for local variables.
count – variable dimension for current MPI rank. Local variables only have count.
endStep – similar to std::endStep, end current step and flush (default). Use adios2::endStep if true.

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

template<class T> void write(const std::string &name, const T *data, const adios2::Dims &shape, const adios2::Dims &start, const adios2::Dims &count, const adios2::vParams &operations, const bool endStep = false)

write overload that allows passing supported operations (e.g. lossy compression “zfp”, “mgard”, “sz”) to a self-described array variable

Parameters

name – variable name
data – variable data data
shape – variable global MPI dimensions. Pass empty for local variables.
start – variable offset for current MPI rank. Pass empty for local variables.
count – variable dimension for current MPI rank. Local variables only have count.
operations – vector of operations, each entry is a std::pair:
endStep – similar to std::endStep, end current step and flush (default). Use adios2::endStep if true.

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

template<class T> void write(const std::string &name, const T &value, const bool isLocalValue = false, const bool endStep = false)

Write a self-describing single-value variable

Parameters

name – variable name
value – variable data value (can be r-value)
isLocalValue – true: local value (returned as GlobalArray), false: global value (returned as global value)
endStep – similar to std::endStep, end current step and flush (default). Use adios2::endStep for true.

Throws

std::invalid_argument – (user input error) or std::runtime_error (system error)

template<class T> void read(const std::string &name, T *data, const size_t blockID = 0)

Reads into a pre-allocated pointer. When used with adios2::getstep reads current step

Parameters

name – variable name
data – pre-allocated pointer to hold read data
blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

template<class T> void read(const std::string &name, T &value, const size_t blockID = 0)

Reads a value. When used with adios2::getstep reads current step value

Parameters

name – variable name
value – output value, if variable is not found (name and type don’t match) the returned value address becomes nullptr
blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

template<class T> void read(const std::string &name, T *data, const size_t stepsStart, const size_t stepsCount = 1, const size_t blockID = 0)

Read accessing steps in random access mode. Not be used with adios2::getstep as it throw an exception when reading in stepping mode.

Parameters

name – variable name
data – pre-allocated pointer to hold read data, if variable is not found (name and type don’t match) it becomes nullptr
stepsStart – variable initial step (relative to the variable first appearance, not absolute step in stream)
stepsCount – variable number of steps form step_start, don’t have to be contiguous, necessarily
blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

template<class T> void read(const std::string &name, T &value, const size_t step, const size_t blockID = 0)

Reads into a single value for a single step. Not be used with adios2::getstep as it throws an exception when reading in stepping mode.

Parameters

name – variable name
value – filled with value, if variable is not found (name, type and step don’t match) the returned value address becomes nullptr
step – selected single step
blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

template<class T> void read(const std::string &name, T *data, const adios2::Dims &start, const adios2::Dims &count, const size_t blockID = 0)

Reads into a pre-allocated pointer a selection piece in dimension. When used with adios2::getstep reads current step

Parameters

name – variable name
data – pre-allocated pointer to hold read data, if variable is not found (name and type don’t match) it becomes nullptr
start – variable local offset selection
count – variable local dimension selection from start
blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

template<class T> void read(const std::string &name, T *data, const adios2::Dims &start, const adios2::Dims &count, const size_t stepsStart, const size_t stepsCount, const size_t blockID = 0)

Reads into a pre-allocated pointer a selection piece in dimensions and steps. Not be used with adios2::getstep as it throws an exception when reading in stepping mode.

Parameters

name – variable name
data – pre-allocated pointer to hold read data, if variable is not found (name and type don’t match) it becomes a nullptr
start – variable local offset selection
count – variable local dimension selection from start
stepsStart – variable initial step (relative to the variable first appearance, not absolute step in stream)
stepsCount – variable number of steps form step_start, don’t have to be necessarily contiguous
blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

template<class T> std::vector<T> read(const std::string &name, const size_t blockID = 0)

Reads entire variable for current step (streaming mode: step by step)

Parameters

name – variable name
blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

Returns

data of variable name for current step. Single data will have a size=1 vector

template<class T> std::vector<T> read(const std::string &name, const size_t stepsStart, const size_t stepsCount = 1, const size_t blockID = 0)

Returns a vector with full variable dimensions for the current step selection. Not be used with adios2::getstep as it throw an exception when reading in stepping mode.

Parameters

name – variable name
stepsStart – variable initial step (relative to the variable first appearance, not absolute step in stream)
stepsCount – variable number of steps form step_start, don’t have to be contiguous, necessarily
blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

Returns

data of variable name for current step, empty if exception is thrown

template<class T> std::vector<T> read(const std::string &name, const Dims &start, const Dims &count, const size_t blockID = 0)

Reads a selection piece in dimension for current step (streaming mode: step by step)

Parameters

name – variable name
start – variable local offset selection
count – variable local dimension selection from start
blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

Returns

data of variable name for current step, empty if exception is thrown

template<class T> std::vector<T> read(const std::string &name, const Dims &start, const Dims &count, const size_t stepsStart, const size_t stepsCount, const size_t blockID = 0)

Reads a selection piece in dimension and a selection piece in steps (non-streaming mode). Not be used with adios2::getstep as it throw an exception when reading in stepping mode.

Parameters

name – variable name
start – variable local offset selection
count – variable local dimension selection from start
stepsStart – variable initial step (relative to the variable first appearance, not absolute step in stream)
stepsCount – variable number of steps form step_start, don’t have to be contiguous, necessarily
blockID – required for local variables, specify current block to be selected

Throws

throws – exception if variable name, dimensions or step not found

Returns

variable data, empty if exception is thrown

template<class T> std::vector<T> read_attribute(const std::string &name, const std::string &variableName = "", const std::string separator = "/")

Reads an attribute returning a vector For single data vector size = 1

Parameters

name – attribute name
variableName – default is empty, if not empty look for an attribute associated to a variable
separator – default is “/”, hierarchy between variable name and attribute, e.g. variableName/attribute1, variableName::attribute1. Not used if variableName is empty.

Returns

vector containing attribute data

void end_step(): At write: ends the current step At read: use it in streaming mode to inform the writer that the reader is done consuming the step. No effect for file engines.

void close(): close current stream becoming inaccessible

size_t current_step() const noexcept

Return current step when getstep is called in a loop, read mode only

Returns: current step

Friends

friend bool getstep(adios2::fstream &stream, adios2::fstep &step)

Gets step from stream Based on std::getline, enables reading on a step-by-step basis in a while or for loop. Read mode only

Parameters

stream – input stream containing steps
step – output object current step, adios2::fstep in an alias to adios2::fstream with scope narrowed to one step

Returns

true: step is valid, false: step is invalid (end of stream).

Python High-Level API

Python simple bindings follow closely Python style directives. Just like the full APIs, they rely on numpy and, optionally, on mpi4py, if the underlying ADIOS2 library is compiled with MPI.

For online examples on MyBinder :

Python Write example

from mpi4py import MPI
import numpy as np
import adios2

comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()
...
shape = [size * nx]
start = [rank * nx]
count = [nx]

# with-as will call adios2.close on fh at the end
with adios2.open("cfd.bp", "w", comm) as fh:

   # NSteps from application
   for i in range(0, NSteps):

      if(rank == 0 and i == 0):
         fh.write("size", np.array([size]))

      fh.write("physical_time", np.array([physical_time]) )
      # temperature and pressure are numpy arrays
      fh.write("temperature", temperature, shape, start, count)
      # advances to next step
      fh.write("pressure", pressure, shape, start, count, end_step=True)

Python Read “step-by-step” example

from mpi4py import MPI
import numpy as np
import adios2

comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()
...
shape = [size * nx]
start = [rank * nx]
count = [nx]

if( rank == 0 ):
   # with-as will call adios2.close on fh at the end
   # if only one rank is active pass MPI.COMM_SELF
   with adios2.open("cfd.bp", "r", MPI.COMM_SELF) as fh:

      for fstep in fh:

         # inspect variables in current step
         step_vars = fstep.available_variables()

         # print variables information
         for name, info in step_vars.items():
             print("variable_name: " + name)
             for key, value in info.items():
                print("\t" + key + ": " + value)
             print("\n")

         # track current step
         step = fstep.current_step()
         if( step == 0 ):
            size_in = fstep.read("size")

         # read variables return a numpy array with corresponding selection
         physical_time = fstep.read("physical_time")
         temperature = fstep.read("temperature", start, count)
         pressure = fstep.read("pressure", start, count)

Caution

When reading in stepping mode with the for-in directive, as in the example above, use the step handler (fstep) inside the loop rather than the global handler (fh)

File class API

adios2.open(*args, **kwargs)

Overloaded function.

open(name: str, mode: str, engine_type: str = ‘BPFile’) -> adios2::py11::File

High-level API, file object open

open(name: str, mode: str, config_file: str, io_in_config_file: str) -> adios2::py11::File

High-level API, file object open with a runtime config file

class adios2.File

add_transport(self: adios2.adios2.File, type: str, parameters: Dict[str, str] = {}) → int

Adds a transport and its parameters to current IO. Must be supported by current engine type.

Parameters

type: must be a supported transport type for current engine.
parameters: acceptable parameters for a particular transport CAN’T use the keywords “Transport” or “transport” in key

Returns

transport_index: handler to added transport

available_attributes(self: adios2.adios2.File) → Dict[str, Dict[str, str]]

Returns a 2-level dictionary with attribute information. Read mode only.

Returns

attributes dictionary

key: attribute name
value: attribute information dictionary

available_variables(self: adios2.adios2.File, keys: List[str] = []) → Dict[str, Dict[str, str]]

Returns a 2-level dictionary with variable information. Read mode only.

Parameters

keys: list of variable information keys to be extracted (case insensitive) keys=[‘AvailableStepsCount’,’Type’,’Max’,’Min’,’SingleValue’,’Shape’] keys=[‘Name’] returns only the variable names as 1st-level keys leave empty to return all possible keys

Returns

variables dictionary

key: variable name
value: variable information dictionary

close(self: adios2.adios2.File) → None: Closes file, thus becoming unreachable. Not required if using open in a with-as statement. Required in all other cases per-open to avoid resource leaks.

current_step(self: adios2.adios2.File) → int

Inspect current step when using for-in loops, read mode only

Returns: current step

end_step(self: adios2.adios2.File) → None

Write mode: advances to the next step. Convenient when declaring variable attributes as advancing to the next step is not attached to any variable.

Read mode: in streaming mode releases the current step (no effect in file based engines)

read(*args, **kwargs)

Overloaded function.

read(self: adios2.adios2.File, name: str, block_id: int = 0) -> numpy.ndarray

Reads entire variable for current step (streaming mode step by step)

Parameters

name
variable name

block_id
required for local array variables

Returns

array
values of variable name for current step. Single values will have a shape={1} numpy array
read(self: adios2.adios2.File, name: str, start: List[int] = [], count: List[int] = [], block_id: int = 0) -> numpy.ndarray

Reads a selection piece in dimension for current step (streaming mode step by step)

Parameters

name
variable name

start
variable local offset selection (defaults to (0, 0, …)

count
variable local dimension selection from start defaults to whole array for GlobalArrays, or selected Block size for LocalArrays

block_id
required for local array variables

Returns

array
values of variable name for current step empty if exception is thrown
read(self: adios2.adios2.File, name: str, start: List[int], count: List[int], step_start: int, step_count: int, block_id: int = 0) -> numpy.ndarray

Random access read allowed to select steps, only valid with File Engines

Parameters

name
variable to be read

start
variable offset dimensions

count
variable local dimensions from offset

step_start
variable step start

step_count
variable number of steps to read from step_start

block_id
required for local array variables

Returns

array
resulting array from selection

read_attribute(self: adios2.adios2.File, name: str, variable_name: str = '', separator: str = '/') → numpy.ndarray

Reads a numpy based attribute

Parameters

name: attribute name
variable_name: if attribute is associated with a variable
separator: concatenation string between variable_name and attribute e.g. variable_name + separator + name (var/attr) Not used if variable_name is empty

Returns

array: resulting array attribute data

read_attribute_string(self: adios2.adios2.File, name: str, variable_name: str = '', separator: str = '/') → List[str]

Read a string attribute

Parameters

name
attribute name

variable_name
if attribute is associated with a variable

separator
concatenation string between variable_name and attribute e.g. variable_name + separator + name (var/attr) Not used if variable_name is empty

Returns

list: resulting string list attribute data

read_string(*args, **kwargs)

Overloaded function.

read_string(self: adios2.adios2.File, name: str, block_id: int = 0) -> List[str]

Reads string value for current step (use for streaming mode step by step)

Parameters

name
string variable name

block_id
required for local variables

Returns

list
data string values. For global values: returns 1 element For local values: returns n-block elements
read_string(self: adios2.adios2.File, name: str, step_start: int, step_count: int, block_id: int = 0) -> List[str]

Reads string value for a certain step (random access mode)

Parameters

name
string variable name

step_start
variable step start

step_count
variable number of steps to read from step_start

block_id
required for local variables

Returns

string
data string values for a certain step range.

set_parameter(self: adios2.adios2.File, key: str, value: str) → None

Sets a single parameter. Overwrites value if key exists.

Parameters

key: input parameter key
value: parameter value

set_parameters(self: adios2.adios2.File, parameters: Dict[str, str]) → None

Sets parameters using a dictionary. Removes any previous parameter.

Parameters

parameters: input key/value parameters
value: parameter value

steps(self: adios2.adios2.File) → int

Inspect available number of steps, for file engines, read mode only

Returns: steps

write(*args, **kwargs)

Overloaded function.

write(self: adios2.adios2.File, name: str, array: numpy.ndarray, shape: List[int] = [], start: List[int] = [], count: List[int] = [], end_step: bool = False) -> None

writes a self-describing array (numpy) variable

Parameters

name
variable name

array
variable data values

shape
variable global MPI dimensions. Pass empty numpy array for local variables.

start
variable offset for current MPI rank. Pass empty numpy array for local variables.

count
variable dimension for current MPI rank. Pass a numpy array for local variables.

end_step
end current step, begin next step and flush (default = false).
write(self: adios2.adios2.File, name: str, array: numpy.ndarray, shape: List[int], start: List[int], count: List[int], operations: List[Tuple[str, Dict[str, str]]], end_step: bool = False) -> None

writes a self-describing array (numpy) variable with operations e.g. compression: ‘zfp’, ‘mgard’, ‘sz’

Parameters

name
variable name

array
variable data values

shape
variable global MPI dimensions. Pass empty numpy array for local variables.

start
variable offset for current MPI rank. Pass empty numpy array for local variables.

count
variable dimension for current MPI rank. Pass a numpy array for local variables.

end_step
end current step, begin next step and flush (default = false).
write(self: adios2.adios2.File, name: str, array: numpy.ndarray, local_value: bool = False, end_step: bool = False) -> None

writes a self-describing single value array (numpy) variable

Parameters

name
variable name

array
variable data single value

local_value
true: local value, false: global value

end_step
end current step, begin next step and flush (default = false).
write(self: adios2.adios2.File, name: str, string: str, local_value: bool = False, end_step: bool = False) -> None

writes a self-describing single value string variable

Parameters

name
variable name

string
variable data single value

local_value
true: local value, false: global value

end_step
end current step, begin next step and flush (default = false).

write_attribute(*args, **kwargs)

Overloaded function.

write_attribute(self: adios2.adios2.File, name: str, array: numpy.ndarray, variable_name: str = ‘’, separator: str = ‘/’, end_step: bool = False) -> None

writes a self-describing single value array (numpy) variable

Parameters

name
attribute name

array
attribute numpy array data

variable_name
if attribute is associated with a variable

separator
concatenation string between variable_name and attribute e.g. variable_name + separator + name (“var/attr”) Not used if variable_name is empty

end_step
end current step, begin next step and flush (default = false).
write_attribute(self: adios2.adios2.File, name: str, string_value: str, variable_name: str = ‘’, separator: str = ‘/’, end_step: bool = False) -> None

writes a self-describing single value array (numpy) variable

Parameters

name
attribute name

string_value
attribute single string

variable_name
if attribute is associated with a variable

separator
concatenation string between variable_name and attribute e.g. variable_name + separator + name (“var/attr”) Not used if variable_name is empty

end_step
end current step, begin next step and flush (default = false).
write_attribute(self: adios2.adios2.File, name: str, string_array: List[str], variable_name: str = ‘’, separator: str = ‘/’, end_step: bool = False) -> None

writes a self-describing single value array (numpy) variable

Parameters

name
attribute name

string_array
attribute string array

variable_name
if attribute is associated with a variable

separator
concatenation string between variable_name and attribute e.g. variable_name + separator + name (“var/attr”) Not used if variable_name is empty

end_step
end current step, begin next step and flush (default = false).

Matlab simple bindings

The ADIOS Matlab API supports reading data from ADIOS BP files with a simplified API that consists of three functions:

ADIOSOPEN returns a structure with information on an ADIOS BP File (variables and attributes).

ADIOSREAD reads in a variable from the file. It expects the info structure returned by ADIOSOPEN.

ADIOSCLOSE closes the file.

Organization of an ADIOS BP file

An ADIOS BP file contains a set of variables and attributes. Each variable in the group has a path, which defines a logical hierarchy of the variables within the file.

Time dimension of a variable

Variables can be written several times from a program, if they have a time dimension. The reader exposes the variables with an extra dimension, i.e. a 2D variable written over time is seen as a 3D variable. In MATLAB, the extra dimension is the last dimension (the slowest changing dimension). Since the reader allows reading an arbitrary slice of a variable, data for one timestep can be read in with slicing.

Min/max of arrays

The ADIOS BP format stores the min/max values in each variable. The info structure therefore contains these min/max values. There is practically no overhead to provide this information (along with the values of all attributes) even for file sizes of several terabytes.

In the Matlab console use help for these functions

>>> help adiosopen
>>> help adiosread
>>> help adiosclose

`ADIOSOPEN`

FILE = adiosopen(PATH): Open a file for reading pointed by PATH and return an information structure (FILE).

The returned FILE structure contains the following information

Name              File path

Handlers          Object handlers to pass on to ADIOS functions
  FileHandler        uint64 file handler
  GroupHandler       uint64 IO group object handler
  ADIOSHandler       uint64 ADIOS object handler

Variables         Structure array of variables
     Name            Path of variable
     Type            Matlab type class of data
     Dims            Array of dimensions
     StepsStart      First step's index for this variable in file, always at least 1
     StepsCount      Number of steps for this variable in file, always at least 1
     GlobalMin       Global minimum  of the variable (1-by-1 mxArray)
     GlobalMax       Global maximum of the variable

Attribute         Structure array of attributes
     Name            Path of attribute
     Type            Matlab type class of data
     Value           Attribute value

`ADIOSREAD`

Read data from a BP file opened with adiosopen. Provide the structure returned by adiosopen as the first input argument, and the path to a variable. Inspect file.Variables and file.Attributes for the list of variables and attributes available in a file.

data = adiosread(file, VARPATH)

Read the entire variable VARPATH from a BP file. file is the output of ADIOSOPEN. VARPATH is a string to a variable or attribute. If an N-dimensional array variable has multiple steps in the file this function reads all steps and returns an N+1 dimensional array where the last dimension equals the number of steps.

data = adiosread(file, INDEX)

Read the entire variable from a BP file. INDEX points to a variable in the file.Variables array.

data = adiosread(..., START, COUNT, STEPSTART, STEPCOUNT)

Read a portion of a variable.

START and COUNT:
A slice is defined as two arrays of N integers, where N is the
number of dimensions of the variable, describing the
"start" and "count" values. The "start" values start from 1.
    E.g. [1 5], [10 2] reads the first 10 values in the first dimension
and 2 values from the 5th position in the second dimension resulting in
a 10-by-2 array.
    You can use negative numbers to index from the end of the array
as in python. -1 refers to the last element of the array, -2 the one
before and so on.
    E.g. [-1], [1] reads in the last value of a 1D array.
         [1], [-1] reads in the complete 1D array.

STEPSTART and STEPCOUNT:
Similarly, the number of steps from a specific step can be read instead
of all data. Steps start from 1. Negative index can be used as well.
    E.g. -1, 1  will read in the last step from the file
         n, -1  will read all steps from 'n' to the last one

`ADIOSCLOSE`

adiosclose(file): Close file and free internal data structures. file is the structure returned by adiosopen.

High-Level APIs

C++ High-Level API

C++11 Write example

C++11 Read “step-by-step” example

adios2::fstream API documentation

Python High-Level API

Python Write example

Python Read “step-by-step” example

File class API

Matlab simple bindings

Organization of an ADIOS BP file

Time dimension of a variable

Min/max of arrays

ADIOSOPEN

ADIOSREAD

ADIOSCLOSE

`adios2::fstream` API documentation

`ADIOSOPEN`

`ADIOSREAD`

`ADIOSCLOSE`