Install from Source

ADIOS2 uses CMake for building, testing and installing the library and utilities.

Building, Testing, and Installing ADIOS 2

To build ADIOS v2.x, clone the repository and invoke the canonical CMake build sequence:

$ git clone https://github.com/ornladios/ADIOS2.git ADIOS2
$ mkdir adios2-build && cd adios2-build
$ cmake ../ADIOS2 -DADIOS2_BUILD_EXAMPLES=ON
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
...

ADIOS2 build configuration:
  ADIOS Version: 2.10.1
  C++ Compiler : GNU 9.4.0
    /usr/bin/c++

  Fortran Compiler : GNU 9.4.0
    /usr/bin/f95

  Installation prefix: /usr/local
        bin: bin
       lib: lib
    include: include
     cmake: lib/cmake/adios2
    python: lib/python3/dist-packages

  ...
  Features:
    Library Type: shared
    Build Type:   Release
    Testing: OFF
    Examples: ON
    Build Options:
      DataMan            : ON
      DataSpaces         : OFF
      HDF5               : OFF
      HDF5_VOL           : OFF
      MHS                : ON
      SST                : ON
      Fortran            : ON
      MPI                : ON
      Python             : ON
      PIP                : OFF
      Blosc2             : OFF
      BZip2              : ON
      LIBPRESSIO         : OFF
      MGARD              : OFF
      MGARD_MDR          : OFF
      PNG                : OFF
      SZ                 : OFF
      ZFP                : ON
      DAOS               : OFF
      IME                : OFF
      O_DIRECT           : ON
      Sodium             : ON
      Catalyst           : OFF
      SysVShMem          : ON
      UCX                : OFF
      ZeroMQ             : ON
      Profiling          : ON
      Endian_Reverse     : OFF
      Derived_Variable   : OFF
      AWSSDK             : OFF
      GPU_Support        : OFF
      CUDA               : OFF
      Kokkos             : OFF
      Kokkos_CUDA        : OFF
      Kokkos_HIP         : OFF
      Kokkos_SYCL        : OFF
      Campaign           : OFF

If a desired feature is OFF in the report above, tell cmake where to find the required dependencies for that feature and manually turn it on. E.g.:

$ cmake ... -DADIOS2_USE_Blosc2=ON   -DCMAKE_PREFIX_PATH="<path to c-blosc2 installation>"

Then compile using

$ make -j 16

Optionally, run the tests (need to configure with -DBUILD_TESTING=ON cmake flag)

$ ctest
Test project /home/wgodoy/workspace/build
        Start   1: ADIOSInterfaceWriteTest.DefineVar_int8_t_1x10
  1/295 Test   #1: ADIOSInterfaceWriteTest.DefineVar_int8_t_1x10 .........................   Passed    0.16 sec
        Start   2: ADIOSInterfaceWriteTest.DefineVar_int16_t_1x10
  2/295 Test   #2: ADIOSInterfaceWriteTest.DefineVar_int16_t_1x10 ........................   Passed    0.06 sec
        Start   3: ADIOSInterfaceWriteTest.DefineVar_int32_t_1x10

      ...

        Start 294: ADIOSBZip2Wrapper.WrongParameterValue
294/295 Test #294: ADIOSBZip2Wrapper.WrongParameterValue .................................   Passed    0.00 sec
        Start 295: ADIOSBZip2Wrapper.WrongBZip2Name
295/295 Test #295: ADIOSBZip2Wrapper.WrongBZip2Name ......................................   Passed    0.00 sec

100% tests passed, 0 tests failed out of 295

Total Test time (real) =   95.95 sec

And finally, use the standard invocation to install (setting the install path beforehand):

$ cmake ../ADIOS2 -DCMAKE_INSTALL_PREFIX=/path/to/where/adios/will/be/installed
$ make install

CMake Options

The following options can be specified with CMake’s -DVAR=VALUE syntax. The default option is highlighted.

VAR

VALUE

Description

ADIOS2_USE_MPI

ON/OFF

MPI or non-MPI (serial) build.

ADIOS2_USE_ZeroMQ

ON/OFF

ZeroMQ for the DataMan engine.

ADIOS2_USE_HDF5

ON/OFF

HDF5 engine. If HDF5 is not on the syspath, it can be set using -DHDF5_ROOT=/path/to/hdf5

ADIOS2_USE_Python

ON/OFF

Python bindings. Python 3 will be used if found. If you want to specify a particular python version use -DPYTHON_EXECUTABLE=/path/to/interpreter/python -DPython_FIND_STRATEGY=LOCATION

ADIOS2_USE_Fortran

ON/OFF

Bindings for Fortran 90 or above.

ADIOS2_USE_SST

ON/OFF

Simplified Staging Engine (SST) and its dependencies, requires MPI. Can optionally use LibFabric/UCX for RDMA transport. You can specify the LibFabric/UCX path manually with the -DLIBFABRIC_ROOT=… or -DUCX_ROOT=… option.

ADIOS2_USE_BZip2

ON/OFF

BZIP2 compression.

ADIOS2_USE_ZFP

ON/OFF

ZFP compression (experimental).

ADIOS2_USE_SZ

ON/OFF

SZ compression (experimental).

ADIOS2_USE_MGARD

ON/OFF

MGARD compression (experimental).

ADIOS2_USE_PNG

ON/OFF

PNG compression (experimental).

ADIOS2_USE_Blosc2

ON/OFF

Blosc compression (experimental).

ADIOS2_USE_Endian_Reverse

ON/OFF

Enable endian conversion if a different endianness is detected between write and read.

ADIOS2_USE_IME

ON/OFF

DDN IME transport.

In addition to the ADIOS2_USE_Feature options, the following options are also available to control how the library gets built:

CMake VAR Options

Values

Description |

BUILD_SHARED_LIBS

ON/OFF

Build shared libraries.

ADIOS2_BUILD_EXAMPLES

ON/OFF

Build examples.

BUILD_TESTING

ON/OFF

Build test code.

CMAKE_INSTALL_PREFIX

/path/to/install (/usr/local)

Installation location.

CMAKE_BUILD_TYPE

Debug/Release/RelWithDebInfo/MinSizeRel

Compiler optimization levels.

CMAKE_PREFIX_PATH

Semi-colon separeated list of paths

Location of extra dependencies

Example: Enable Fortran, disable Python bindings and ZeroMQ functionality

$ cmake -DADIOS2_USE_Fortran=ON -DADIOS2_USE_Python=OFF -DADIOS2_USE_ZeroMQ=OFF ../ADIOS2

Notes:

To provide search paths to CMake for dependency searching:

  • Use a PackageName_ROOT variable to provide the location of a specific package.

  • Add an install prefix to the CMAKE_PREFIX_PATH which is searched for all packages.

  • Both the PackageName_ROOT and CMAKE_PREFIX_PATH can be used as either environment variables or CMake variables (passed via -D), where the CMake variable takes prescedence.

# Several dependencies are installed under /opt/foo/bar and then a
# single dependency (HDF5 in this case) is installed in /opt/hdf5/1.13.0
$ export CMAKE_PREFIX_PATH=/opt/foo/bar
$ cmake -DHDF5_ROOT=/opt/hdf5/1.13.0 ../ADIOS2

Example: the following configuration will build, test and install under /opt/adios2/2.9.0 an optimized (Release) version of ADIOS2.

$ cd build
$ cmake -DADIOS2_USE_Fortran=ON -DCMAKE_INSTALL_PREFIX=/opt/adios2/2.9.0 -DCMAKE_BUILD_Type=Release ../ADIOS2
$ make -j16
$ ctest
$ make install

For a fully configurable build script, click here.

Building on HPC Systems

  1. Modules: Make sure all “module” dependencies are loaded and that minimum requirements are satisfied. Load the latest CMake module as many HPC systems default to an outdated version. Build with a C++11-compliant compiler, such as gcc >= 4.8.1, Intel >= 15, and PGI >= 15.

  2. Static/Dynamic build: On Cray systems such as Titan, the default behavior is static linkage, thus CMake builds ADIOS2 creates the static library libadios2.a by default. Read the system documentation to enable dynamic compilation, usually by setting an environment variable such as CRAYPE_LINK_TYPE=dynamic. Click here for a fully configurable script example on OLCF systems.

  3. Big Endian and 32-bit systems: ADIOS2 hasn’t been tested on big endian and generally will not build on 32-bit systems. Please be aware before attempting to run.

  4. PGI compilers and C++11 support: Version 15 of the PGI compiler is C++11 compliant. However it relies on the C++ standard library headers supplied by the system version of GCC, which may or may support all the C++11 features used in ADIOS2. On many systems (Titan at OLCF, for example) even though the PGI compiler supports C++11, the configured GCC and its headers do not (4.3.x on Cray Linux Environment, and v5 systems like Titan). To configure the PGI compiler to use a newer GCC, you must create a configuration file in your home directory that overrides the PGI compiler’s default configuration. On Titan, the following steps will re-configure the PGI compiler to use GCC 6.3.0 provided by a module:

$ module load gcc/6.3.0
$ makelocalrc $(dirname $(which pgc++)) -gcc $(which gcc) -gpp $(which g++) -g77 $(which gfortran) -o -net 1>${HOME}/.mypgirc 2>/dev/null

  1. Enabling RDMA for SST data transfers: The SST engine in ADIOS2 is capable of using RDMA networks for transfering data between writer and reader cohorts, and generally this is the most performant data transport. However, SST depends upon libfabric to provide a generic interface to the underlying RDMA capabilities of the network, and properly configuring libfabric can be a difficult and error-prone task. HPC computing resources tend to be one-off custom resources with their own idiosyncracies, so this documentation cannot offer a definitive guide for every situation, but we can provide some general guidance and some recommendations for specific machines. If you are unable to configure ADIOS2 and libfabric to use RDMA, the best way to get help is to open an issue on the ADIOS2 github repository.

Pre-build concerns of note:

  • on some HPC resources, libfabric is available as a loadable module. That should not be taken as an indication that that build of libfabric will work with SST, or even that it is compatible with the system upon which you find it. Your mileage may vary and you may have to build libfabric manually.

  • libfabric itself depends upon other libraries like libibverbs and librdmacm. If you build libfabric with a package manager like spack, spack may build custom versions of those libraries as well, which may conflict with the system versions of those libraries.

  • MPI on your HPC system may use libfabric itself, and linking your application with a different version of libfabric (or its dependent libraries) may result failure, possibly including opaque error messages from MPI.

  • libfabric is structured in such a way that even if it is found during configuration, ADIOS cannot determine at compile time what providers will be present at run-time, or what their capabilities are. Therefore even a build that seems to successfully include libfabric and RDMA may be rejected at runtime as unable to support SST data transfer.

Configuration:

ADIOS2 uses the CMake find_package() functionality to locate libfabric. CMake will automatically search system libraries, but if you need to specify a libfabric location other than in a default system location you can add a “-DLIBFABRIC_ROOT=<directory>” argument to direct CMake to libfabric’s location. If CMake finds libfabric, you should see the line “RDMA Transport for Staging: Available” near the end of the CMake output. This makes the RDMA DataTransport the default for SST data movement. (More information about SST engine parameters like DataTransport appears in the SST engine description.) If instead you see “RDMA Transport for Staging: Unconfigured”, RDMA will not be available to SST.

Run-time:

Generally, if RDMA is configured and the libfabric provider has the capabilities that SST needs for RDMA data transfer, SST will use RDMA without external acknowledgement. However, if RDMA is configured, but the libfabric provider doesn’t have the capabilities that SST needs, ADIOS will output an error : ‘Warning: Preferred DataPlane “RDMA” not found.’ If you see this warning in a situation where you expect RDMA to be used, enabling verbose debugging output from SST may provide more information. The SstVerbose environment variable can have values from 1 to 5, with 1 being minimal debugging info (such as confirming which DataTransport is being used), and 5 being the most detailed debugging information from all ranks.

Installing the ADIOS2 library and the C++ and C bindings

By default, ADIOS2 will build the C++11 libadios2 library and the C and C++ bindings.

  1. Minimum requirements:

    • A C++11 compliant compiler

    • An MPI C implementation on the syspath, or in a location identifiable by CMake.

  2. Linking make install will copy the required headers and libraries into the directory specified by CMAKE_INSTALL_PREFIX:

    • Libraries:

      • lib/libadios2.* C++11 and C bindings

    • Headers:

      • include/adios2.h C++11 namespace adios2

      • include/adios2_c.h C prefix adios2_

    • Config file: run this command to get installation info

      • bin/adios2-config

Enabling the Python bindings

To enable the Python bindings in ADIOS2, based on PyBind11, make sure to follow these guidelines:

  • Minimum requirements:

    • Python 2.7 and above.

    • numpy

    • mpi4py

  • Running: If CMake enables Python compilation, an adios2.so library containing the Python module is generated in the build directory under lib/pythonX.X/site-packages/

    • make sure your PYTHONPATH environment variable contains the path to adios2.so.

    • make sure the Python interpreter is compatible with the version used for compilation via python --version.

    • Run the Python tests with ctest -R Python

    • Run helloBPWriter.py and helloBPTimeWriter.py via

    $ mpirun -n 4 python helloBPWriter.py
$ python helloBPWriter.py

Enabling the Fortran bindings

  1. Minimum requirements:

    • A Fortran 90 compliant compiler

    • A Fortran MPI implementation

  2. Linking the Fortran bindings: make install will copy the required library and modules into the directory specified by CMAKE_INSTALL_PREFIX

    • Library (note that libadios2 must also be linked) - lib/libadios2_f.* - lib/libadios2.*

    • Modules - include/adios2/fortran/*.mod

  3. Module adios2: only module required to be used in an application use adios

Running Tests

ADIOS2 uses googletest to enable automatic testing after a CMake build. To run tests just type after building with make, run:

$ ctest
  or
$ make test

The following screen will appear providing information on the status of each finalized test:

Test project /home/wfg/workspace/build
Start  1: ADIOSInterfaceWriteTest.DefineVarChar1x10
1/46 Test  #1: ADIOSInterfaceWriteTest.DefineVarChar1x10 .......................   Passed    0.06 sec
Start  2: ADIOSInterfaceWriteTest.DefineVarShort1x10
2/46 Test  #2: ADIOSInterfaceWriteTest.DefineVarShort1x10 ......................   Passed    0.04 sec
Start  3: ADIOSInterfaceWriteTest.DefineVarInt1x10
3/46 Test  #3: ADIOSInterfaceWriteTest.DefineVarInt1x10 ........................   Passed    0.04 sec
Start  4: ADIOSInterfaceWriteTest.DefineVarLong1x10
...
128/130 Test #128: ADIOSZfpWrapper.UnsupportedCall ..........................................   Passed    0.05 sec
    Start 129: ADIOSZfpWrapper.MissingMandatoryParameter
129/130 Test #129: ADIOSZfpWrapper.MissingMandatoryParameter ................................   Passed    0.05 sec
    Start 130: */TestManyVars.DontRedefineVars/*
130/130 Test #130: */TestManyVars.DontRedefineVars/* ........................................   Passed    0.08 sec

100% tests passed, 0 tests failed out of 130

Total Test time (real) = 204.82 sec

Running Examples

ADIOS2 is best learned by examples.

A few very basic examples are described below:

Directory

Description

ADIOS2/examples/hello

very basic “hello world”-style examples for reading and writing .bp files.

ADIOS2/examples/heatTransfer

2D Poisson solver for transients in Fourier’s model of heat transfer. Outputs bp.dir or HDF5.

ADIOS2/examples/basics

covers different Variable use cases classified by the dimension.

As Package

Conda

ADIOS2 can be obtained from anaconda cloud: conda-forge adios2

PyPI

ADIOS2 pip package can be downloaded with pip3 install adios2 or python3 -m pip install adios2. This is contains the serial build only, so MPI programs cannot use it. See adios2 on PyPi

Spack

ADIOS2 is packaged in Spack. See adios2 spack package

Docker

Docker images including building and installation of dependencies and ADIOS 2 containers for Ubuntu 20 and CentOS 7 can be found in: under the directory scripts/docker/

Linking ADIOS 2

From CMake

ADIOS exports a CMake package configuration file that allows its targets to be directly imported into another CMake project via the find_package command:

cmake_minimum_required(VERSION 3.12)
project(MySimulation C CXX)

find_package(MPI REQUIRED)
find_package(ADIOS2 REQUIRED)
#...
add_library(my_library src1.cxx src2.cxx)
target_link_libraries(my_library PRIVATE adios2::cxx11_mpi MPI::MPI_C)

When configuring your project you can then set the ADIOS2_ROOT or ADIOS2_DIR environment variables to the install prefix of ADIOS2.

From non-CMake build systems

If you’re not using CMake then you can manually get the necessary compile and link flags for your project using adios2-config:

$ /path/to/install-prefix/bin/adios2-config --cxxflags
ADIOS2_DIR: /path/to/install-prefix
-isystem /path/to/install-prefix/include -isystem /opt/ohpc/pub/mpi/openmpi3-gnu7/3.1.0/include -pthread -std=gnu++11
$ /path/to/install-prefix/bin/adios2-config --cxxlibs
ADIOS2_DIR: /path/to/install-prefix
-Wl,-rpath,/path/to/install-prefix/lib:/opt/ohpc/pub/mpi/openmpi3-gnu7/3.1.0/lib /path/to/install-prefix/lib/libadios2.so.2.4.0 -pthread -Wl,-rpath -Wl,/opt/ohpc/pub/mpi/openmpi3-gnu7/3.1.0/lib -Wl,--enable-new-dtags -pthread /opt/ohpc/pub/mpi/openmpi3-gnu7/3.1.0/lib/libmpi.so -Wl,-rpath-link,/path/to/install-prefix/lib

Use on DOE machines

ADIOS2 is installed as part of the E4S software stack and access to adios2 is the same as access to the many other packages.

NERSC Perlmutter

To use adios2 on Perlmutter,

  • load the e4s module

  • pick your compiler environment with spack

  • load adios2 with spack

~> module load e4s
  _____________________________________________________________________________________
   The Extreme-Scale Scientific Software Stack (E4S) is accessible via the Spack package manager.

   In order to access the production stack, you will need to load a spack
   environment. Here are some tips to get started:


   'spack env list' - List all Spack environments
   'spack env activate gcc' - Activate the "gcc" Spack environment
   'spack env status' - Display the active Spack environment
   'spack load amrex' - Load the "amrex" Spack package into your user environment

   For additional support, please refer to the following references:

     NERSC E4S Documentation: https://docs.nersc.gov/applications/e4s/
     E4S Documentation: https://e4s.readthedocs.io
     Spack Documentation: https://spack.readthedocs.io/en/latest/
     Spack Slack: https://spackpm.slack.com

  _____________________________________________________________________________________

~> spack env list
==> 4 environments
  cce  cuda  gcc  nvhpc
~> spack env activate gcc
~> spack load adios2

~> which bpls
/global/common/software/spackecp/perlmutter/e4s-23.08/94543/spack/opt/spack/linux-sles15-zen3/gcc-12.3.0/adios2-2.9.1-iwv5lkkc5gyagr4uqrqr4v2fds7x66pk/bin/bpls

~> bpls -Vv
  blps: ADIOS file introspection utility

  Build configuration:
  ADIOS version: 2.9.1
  C++ Compiler:  GNU 12.3.0 (CrayPrgEnv)
  Target OS:     Linux-5.14.21-150400.24.81_12.0.87-cray_shasta_c
  Target Arch:   x86_64
  Available engines = 10: BP3, BP4, BP5, SST, SSC, Inline, MHS,
  ParaViewADIOSInSituEngine, Null, Skeleton
  Available operators = 4: BZip2, SZ, ZFP, PNG
  Available features = 16: BP5, DATAMAN, MHS, SST, FORTRAN, MPI, BZIP2, PNG,
  SZ, ZFP, O_DIRECT, CATALYST, SYSVSHMEM, ZEROMQ, PROFILING, ENDIAN_REVERSE

OLCF Frontier

OLCF installs the E4S packages in individual modules, hence adios2 is also available as a module.

$ module avail adios2
----- /sw/frontier/spack-envs/base/modules/spack/cray-sles15-x86_64/cray-mpich/8.1.23-j56azw5/cce/15.0.0 -----
 adios2/2.8.1    adios2/2.8.3 (D)

Where:
 D:  Default Module

$ module load adios2
$ bpls -Vv
  blps: ADIOS file introspection utility

  Build configuration:
  ADIOS version: 2.8.3
  C++ Compiler:  GNU 12.2.0 (CrayPrgEnv)
  Target OS:     Linux-5.14.21-150400.24.11_12.0.57-cray_shasta_c
  Target Arch:   x86_64

ALCF Aurora

To use adios2 on Aurora,

  • Load the default oneAPI (loaded automatically on login)

  • module use /soft/modulefiles

  • module load spack-pe-oneapi/0.5-rc1

This is a “metamoduile” that makes many software packages from E4S loadable as modules.

$ module use /soft/modulefiles
$ module load spack-pe-oneapi/0.5-rc1
$ module avail adios2

---------- /soft/packaging/spack/oneapi/0.5-rc1/modulefiles/Core -----------
adios2/2.9.0-oneapi-mpich-testing