Operators
In the previous tutorial we learned how to write and read attributes.
For this example to work, you would need to have a compression library installed, which ADIOS automatically detects. The easiest way to install SZ or SZ3 is with Spack, and you can do that as follows:
git clone https://github.com/spack/spack.git ~/spack
cd ~/spack
. share/spack/setup-env.sh
spack install sz # for SZ (SZ2)
# or
spack install sz3 # for SZ3
spack load sz # or sz3
In this tutorial we will learn how to use operators. Operators are used for Data compression/decompression, lossy and lossless. They act upon the user application data, either from a variable or a set of variables in a IO object.
Additionally, we will explore how to simply write variables across multiple steps.
So, let’s dig in!
Start editing the skeleton file ADIOS2/examples/hello/bpOperatorSZWriter/bpOperatorSZWriter_tutorialSkeleton.cpp.
In an MPI application first we need to always initialize MPI. We do that with the following lines:
int rank, size;
int rank, size;
int provided;
// MPI_THREAD_MULTIPLE is only required if you enable the SST MPI_DP
MPI_Init_thread(&argc, &argv, MPI_THREAD_MULTIPLE, &provided);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
This application has command line arguments for the size of the data, and the compression accuracy, which we can read as follows:
const std::size_t Nx = static_cast<std::size_t>(std::stoull(argv[1]));
const double accuracy = std::stod(argv[2]);
Now we need to create some application variables which will be used to define ADIOS2 variables.
std::vector<float> myFloats(Nx);
std::vector<double> myDoubles(Nx);
std::iota(myFloats.begin(), myFloats.end(), 0.);
std::iota(myDoubles.begin(), myDoubles.end(), 0.);
Now we need to create an ADIOS2 instance and IO object.
adios2::ADIOS adios(MPI_COMM_WORLD);
adios2::IO bpIO = adios.DeclareIO("BPFile_SZ");
Now we need to define the variables we want to write.
adios2::Variable<float> bpFloats = bpIO.DefineVariable<float>(
"bpFloats", {size * Nx}, {rank * Nx}, {Nx}, adios2::ConstantDims);
adios2::Variable<double> bpDoubles = bpIO.DefineVariable<double>(
"bpDoubles", {size * Nx}, {rank * Nx}, {Nx}, adios2::ConstantDims);
Now we need to define the compression operator we want to use. In this case we will use the SZ compressor.
adios2::Operator op = bpIO.DefineOperator("SZCompressor", "sz");
varFloats.AddOperation(op, {{"accuracy", std::to_string(accuracy)}});
varDoubles.AddOperation(op, {{"accuracy", std::to_string(accuracy)}});
Note
DefineOperator()’s second parameter can be zfp, sz, or sz3. For more information regarding operators and their
properties you can look at Basics: Interface Components: Operator.
Let’s also create an attribute to store the accuracy value.
adios2::Attribute<double> attribute = bpIO.DefineAttribute<double>("accuracy", accuracy);
Now we need to open the file for writing.
adios2::Engine bpWriter = bpIO.Open("SZexample.bp", adios2::Mode::Write);
Now we need to write the data. We will write the data for 3 steps, and edit them in between.
for (unsigned int step = 0; step < 3; ++step)
{
bpWriter.BeginStep();
bpWriter.Put<double>(bpDoubles, myDoubles.data());
bpWriter.Put<float>(bpFloats, myFloats.data());
bpWriter.EndStep();
// here you can modify myFloats, myDoubles per step
std::transform(myFloats.begin(), myFloats.end(), myFloats.begin(),
[&](float v) -> float { return 2 * v; });
std::transform(myDoubles.begin(), myDoubles.end(), myDoubles.begin(),
[&](double v) -> double { return 3 * v; });
}
Now we need to close the file.
bpWriter.Close();
Finally we need to finalize MPI.
MPI_Finalize();
The final code should look as follows (excluding try/catch and optional usage of MPI), and it was derived from the example ADIOS2/examples/hello/bpOperatorSZWriter/bpOperatorSZWriter.cpp.
/*
* SPDX-FileCopyrightText: 2026 Oak Ridge National Laboratory and Contributors
*
* SPDX-License-Identifier: Apache-2.0
*/
#include <algorithm> //std::transform
#include <ios> //std::ios_base::failure
#include <iostream> //std::cout
#include <numeric> //std::iota
#include <stdexcept> //std::invalid_argument std::exception
#include <vector>
#include "adios2.h"
#if ADIOS2_USE_MPI
#include <mpi.h>
#endif
void Usage()
{
std::cout << "\n";
std::cout << "USAGE:\n";
std::cout << "./adios2_hello_bpOperatorSZWriter Nx sz_accuracy\n";
std::cout << "\t Nx: size of float and double arrays to be compressed\n";
std::cout << "\t sz_accuracy: absolute accuracy e.g. 0.1, 0.001, to skip "
"compression: -1\n\n";
}
int main(int argc, char *argv[])
{
if (argc != 3)
{
Usage();
return EXIT_SUCCESS;
}
int rank, size;
#if ADIOS2_USE_MPI
int provided;
// MPI_THREAD_MULTIPLE is only required if you enable the SST MPI_DP
MPI_Init_thread(&argc, &argv, MPI_THREAD_MULTIPLE, &provided);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
#else
rank = 0;
size = 1;
#endif
try
{
const std::size_t Nx = static_cast<std::size_t>(std::stoull(argv[1]));
const double accuracy = std::stod(argv[2]);
/** Application variable */
std::vector<float> myFloats(Nx);
std::vector<double> myDoubles(Nx);
std::iota(myFloats.begin(), myFloats.end(), 0.);
std::iota(myDoubles.begin(), myDoubles.end(), 0.);
/** ADIOS class factory of IO class objects */
#if ADIOS2_USE_MPI
adios2::ADIOS adios(MPI_COMM_WORLD);
#else
adios2::ADIOS adios;
#endif
/*** IO class object: settings and factory of Settings: Variables,
* Parameters, Transports, and Execution: Engines */
adios2::IO bpIO = adios.DeclareIO("BPFile_SZ");
adios2::Variable<float> varFloats = bpIO.DefineVariable<float>(
"bpFloats", {size * Nx}, {rank * Nx}, {Nx}, adios2::ConstantDims);
adios2::Variable<double> varDoubles = bpIO.DefineVariable<double>(
"bpDoubles", {size * Nx}, {rank * Nx}, {Nx}, adios2::ConstantDims);
if (accuracy > 1E-16)
{
adios2::Operator op = adios.DefineOperator("SZCompressor", "sz");
varFloats.AddOperation(op, {{"accuracy", std::to_string(accuracy)}});
varDoubles.AddOperation(op, {{"accuracy", std::to_string(accuracy)}});
}
adios2::Attribute<double> attribute = bpIO.DefineAttribute<double>("SZ_accuracy", accuracy);
// To avoid compiling warnings
(void)attribute;
/** Engine derived class, spawned to start IO operations */
adios2::Engine bpWriter = bpIO.Open("SZexample.bp", adios2::Mode::Write);
for (unsigned int step = 0; step < 3; ++step)
{
bpWriter.BeginStep();
bpWriter.Put(varFloats, myFloats.data());
bpWriter.Put(varDoubles, myDoubles.data());
bpWriter.EndStep();
// here you can modify myFloats, myDoubles per step
std::transform(myFloats.begin(), myFloats.end(), myFloats.begin(),
[&](float v) -> float { return 2 * v; });
std::transform(myDoubles.begin(), myDoubles.end(), myDoubles.begin(),
[&](double v) -> double { return 3 * v; });
}
/** Create bp file, engine becomes unreachable after this*/
bpWriter.Close();
}
catch (std::invalid_argument &e)
{
std::cerr << "Invalid argument exception: " << e.what() << "\n";
#if ADIOS2_USE_MPI
std::cerr << "STOPPING PROGRAM from rank " << rank << "\n";
MPI_Abort(MPI_COMM_WORLD, 1);
#endif
}
catch (std::ios_base::failure &e)
{
std::cerr << "IO System base failure exception: " << e.what() << "\n";
#if ADIOS2_USE_MPI
std::cerr << "STOPPING PROGRAM from rank " << rank << "\n";
MPI_Abort(MPI_COMM_WORLD, 1);
#endif
}
catch (std::exception &e)
{
std::cerr << "Exception: " << e.what() << "\n";
#if ADIOS2_USE_MPI
std::cerr << "STOPPING PROGRAM from rank " << rank << "\n";
MPI_Abort(MPI_COMM_WORLD, 1);
#endif
}
#if ADIOS2_USE_MPI
MPI_Finalize();
#endif
return 0;
}
You can compile and run it as follows:
cd Path-To-ADIOS2/examples/hello/bpOperatorSZWriter
mkdir build
cd build
cmake -DADIOS2_DIR=Path-To-ADIOS2/build/ ..
cmake --build .
mpirun -np 2 ./adios2_hello_bpOperatorSZWriter_mpi 20 0.000001
You can check the content of the output file “SZexample.bp” using bpls as follows:
Path-To-ADIOS2/build/bin/bpls ./SZexample.bp
double bpDoubles 3*{40}
float bpFloats 3*{40}