ADIOS 2: The Adaptable Input/Output System version 2

Funded by the Exascale Computing Project (ECP), U.S. Department of Energy

Introduction

• What’s new in 2.9?
  • Summary
    • General
    • File I/O
    • Staging
    • Experimental features
• Introduction
  • What ADIOS2 is and isn’t
  • Adaptable IO beyond files in Scientific Data Lifecycles

Setting Up

• Install from Source
  • Building, Testing, and Installing ADIOS 2
  • CMake Options
  • Building on HPC Systems
  • Installing the ADIOS2 library and the C++ and C bindings
  • Enabling the Python bindings
  • Enabling the Fortran bindings
  • Running Tests
  • Running Examples
• As Package
  • Conda
  • Spack
  • Docker
• Linking ADIOS 2
  • From CMake
  • From non-CMake build systems

Basics

• Interface Components
  • Components Overview
  • ADIOS
  • IO
    • Setting a Particular Engine and its Parameters
    • Adding Supported Transports with Parameters
    • Defining, Inquiring and Removing Variables and Attributes
    • Opening an Engine
  • Variable
    • Data Types
    • Shapes
  • Attribute
  • Engine
    • BeginStep
    • EndStep
    • Close
    • Put: modes and memory contracts
    • PerformsPuts
    • PerformsDataWrite
    • Get: modes and memory contracts
    • PerformsGets
    • Engine usage example
    • Available Engines
  • Operator
  • Runtime Configuration Files
    • XML
    • YAML
• Supported Virtual Engine Names
  • Virtual Engine Setups
• Supported Engines
  • BP5
  • BP4
  • BP3
  • HDF5
  • SST Sustainable Staging Transport
  • SSC Strong Staging Coupler
  • DataMan for Wide Area Network Data Staging
  • Inline for zero-copy
  • Null
  • Plugin Engine
• Supported Operators
  • CompressorZFP
    • CompressorZFP Specific parameters
    • CompressorZFP Execution Policy
  • Plugin Operator
  • Encryption
• Full APIs
  • C++11 bindings
    • ADIOS2 components classes
    • ADIOS class
    • IO class
    • Variable <T> class
    • Attribute <T> class
    • Engine class
    • Operator class
  • Fortran bindings
    • ADIOS2 typed handlers
    • ADIOS subroutines
    • IO subroutines
    • Variable subroutines
    • Engine subroutines
    • Operator subroutines
    • Attribute subroutines
  • C bindings
    • adios2_adios handler functions
    • adios2_io handler functions
    • adios2_variable handler functions
    • adios2_attribute handler functions
    • adios2_engine handler functions
    • adios2_operator handler functions
  • Python bindings
    • ADIOS class
      • ADIOS
        • ADIOS.AtIO()
        • ADIOS.DeclareIO()
        • ADIOS.DefineOperator()
        • ADIOS.FlushAll()
        • ADIOS.InquireOperator()
        • ADIOS.RemoveAllIOs()
        • ADIOS.RemoveIO()
    • IO class
      • IO
        • IO.AddTransport()
        • IO.AvailableAttributes()
        • IO.AvailableVariables()
        • IO.DefineAttribute()
        • IO.DefineVariable()
        • IO.EngineType()
        • IO.FlushAll()
        • IO.InquireAttribute()
        • IO.InquireVariable()
        • IO.Open()
        • IO.Parameters()
        • IO.RemoveAllAttributes()
        • IO.RemoveAllVariables()
        • IO.RemoveAttribute()
        • IO.RemoveVariable()
        • IO.SetEngine()
        • IO.SetParameter()
        • IO.SetParameters()
    • Variable class
      • Variable
        • Variable.AddOperation()
        • Variable.BlockID()
        • Variable.Count()
        • Variable.Name()
        • Variable.Operations()
        • Variable.SelectionSize()
        • Variable.SetBlockSelection()
        • Variable.SetSelection()
        • Variable.SetShape()
        • Variable.SetStepSelection()
        • Variable.Shape()
        • Variable.ShapeID()
        • Variable.Sizeof()
        • Variable.Start()
        • Variable.Steps()
        • Variable.StepsStart()
        • Variable.Type()
    • Attribute class
      • Attribute
        • Attribute.Data()
        • Attribute.DataString()
        • Attribute.Name()
        • Attribute.Type()
    • Engine class
      • Engine
        • Engine.BeginStep()
        • Engine.BlocksInfo()
        • Engine.Close()
        • Engine.CurrentStep()
        • Engine.EndStep()
        • Engine.Flush()
        • Engine.Get()
        • Engine.LockReaderSelections()
        • Engine.LockWriterDefinitions()
        • Engine.Name()
        • Engine.PerformDataWrite()
        • Engine.PerformGets()
        • Engine.PerformPuts()
        • Engine.Put()
        • Engine.Steps()
        • Engine.Type()
    • Operator class
      • Operator
        • Operator.Parameters()
        • Operator.SetParameter()
        • Operator.Type()
• High-Level APIs
  • C++ High-Level API
    • C++11 Write example
    • C++11 Read “step-by-step” example
    • adios2::fstream API documentation
  • Python High-Level API
    • Python Write example
    • Python Read “step-by-step” example
    • File class API
      • open()
      • File
        • File.add_transport()
        • File.available_attributes()
        • File.available_variables()
        • File.close()
        • File.current_step()
        • File.end_step()
        • File.read()
        • File.read_attribute()
        • File.read_attribute_string()
        • File.read_string()
        • File.set_parameter()
        • File.set_parameters()
        • File.steps()
        • File.write()
        • File.write_attribute()
  • Matlab simple bindings
    • Organization of an ADIOS BP file
    • Time dimension of a variable
    • Min/max of arrays
    • ADIOSOPEN
    • ADIOSREAD
    • ADIOSCLOSE

Advanced Topics

• Aggregation
  • Aggregation in BP5
• Memory Management
  • BP4 buffering
  • BP5 buffering
  • Span object in internal buffer
• GPU-aware I/O
  • Building ADIOS2 with a GPU backend
    • Building with CUDA enabled
    • Building with Kokkos enabled
  • Writing GPU code
    • Using Kokkos buffers
• Plugins
  • Writing Your Plugin Class
    • Engine Plugin
    • Operator Plugin
  • Build Shared Library
  • Using Your Plugin in an Application
    • Engine Plugins
    • Operator Plugins
• ADIOS2 in ECP hardware
  • OLCF CRUSHER
    • SST MPI Data Transport

Ecosystem Tools

• HDF5 API Support through VOL
  • Disclaimer
  • External
  • Internal
• Command Line Utilities
  • bpls : Inspecting Data
  • adios_reorganize
  • adios2-config
  • sst_conn_tool : SST network connectivity tool
• Visualizing Data
  • Using VTK and Paraview
    • Saving the vtk.xml data model

Additional Information

• FAQ
  • MPI vs Non-MPI
  • APIs
  • Building on Titan
  • Building and Running on Fujitsu FX100
  • FAQs Answered
    • Can I use the same library for MPI and non-MPI code?
    • Can I use ADIOS 2 C++11 library with C++98 codes?
    • Why are C and Fortran APIs missing functionality?
    • C++11: Why are std::string arguments passed sometimes by value and sometimes by reference?
    • C++11: Should I pass adios2:: objects by value or by reference?
    • Fortran: Can I pass slices and temporary arrays to adios2_put?
    • My application uses PGI compilers on Titan, can I link ADIOS 2?
    • How do I enable the Python bindings on Titan?
    • How do I build ADIOS 2 on Fujitsu FX100?
    • SST engine hangs on Fujitsu FX100. Why?
• Advice

• Search Page