Glossary of Core NWB Tools
The glossary shown here provides a quick overview of the key software packages of the core NWB software stack. For a more general discussion of the overall organization of the core NWB software stack see the NWB Software Ecosystem page on the main NWB website.
Read/Write NWB File APIs
The NWB reference APIs provide full support for reading and writing all components of the NWB standard, including support for extensions. The APIs are interoperable, i.e., files created with PyNWB can be read in MatNWB and vice versa. Both PyNWB and MatNWB support advanced read/write for efficient interaction with very large data files (i.e., data too large for main memory), via lazy data loading, iterative data write, and data compression among others.
PyNWB:
MatNWB
Converting Data to NWB
NWB Conversion Tools
The NWB Conversion Tools is a Python library for automatic conversion from proprietary data formats to NWB. Docs Source
Validating NWB Files
NWB provides tools both to check that files comply with the NWB standard schema as well as to check whether the data complies with NWB Best Practices. Validating compliance with the NWB schema ensures that files are structurally correct and can be read by NWB APIs. Validating compliance with best practices helps improve data quality and (re-)usability.
PyNWB: Validate schema compliance
The PyNWB reference Python API includes classes and command line tools for validating compliance of files with the core NWB schema and the schema of NWB Neurodata Extensions (NDX). Validation Docs
NWB Inspector: Validate best practice
The NWB Inspector is a python library and command-line tool for inspecting NWB files for adherence to NWB best practices Docs Source
Extending NWB
Neurodata Extensions (NDX) are used to extend the NWB data standard, e.g., to integrate new data types with NWB or define standard for lab- or project-specific metadata. The collection of tools listed here are used to create, document, publish extensions. To learn more about how create extensions see the Extending NWB section.
NDX Template
The NDX Template provides a template for creating Neurodata Extensions (NDX) for the NWB data standard. Source
When creating a new extension, the NDX-template will create a detailed NEXTSTEPS.md file describing how to create an extension and how to submit it to the NDX catalog.
NDX Catalog
The Neurodata Extensions Catalog (NDX Catalog) is a community-led catalog of Neurodata Extensions (NDX) to the NWB data standard. The NDX Catalog provides a central portal to search, publish, and review of NDX. Catalog Source
Publishing NDX
The staged-extensions GitHub repository is used to register new extensions for publication in the Neurodata Extensions Catalog (NDX Catalog). Source
Documentation Utilities
The HDMF Documentation Utilities (hdmf-docuils) provides utility tools for creating documentation for extension schema defined using the NWB Schema Language. Source
The NDX Template automatically sets up the documentation. As such, developers of extensions will commonly hdmf-docuils as part of the standard setup of NDX code repositories without having to interact with the tool directly.
NWB Format Specification
The NWB data standard is governed by the NWB Format Specification. When creating new extensions we typically build on and reuse existing
neurodata_types
already available in NWB. The NWB Format Specification provides a reference definition for all types available in NWB. The NWB schema itself includes/builds on the HDMF Common Schema. Docs Source
HDMF Common Schema
The HDMF Common Schema defines the schema of common, general data structures, which are used throughout the NWB Standard Schema but which are not specific to neurophysiology. Example types defined in the HDMF common schema incude, e.g., all types related to
DynamicTable
for defining data tables. Docs Source
Core Development
Understanding core development tools (e.g., HDMF) is useful for developers in particular when we need to dive deeper into the core data infrastructure for NWB, e.g., when changing or creating new storage methods or when developing features for common data types (e.g., DynamicTable
) that are defined in HDMF and used in NWB.
HDMF
The Hierarchical Data Modeling Framework (HDMF) is a python package for working with hierarchical data. It provides APIs for specifying data models, reading and writing data to different storage backends, and representing data with Python object. HDMF builds the foundation for the PyNWB Python API for NWB. Docs Source