Tables

The DynamicTable data type stores tabular data. It also allows you to define custom columns, which offer a high degree of flexibility.

Table Values

String-valued table entries should not contain JSON. Instead, these values should be unpacked and used as additional columns of that table.

Check function: check_table_values_for_dict()

Empty Tables

An empty DynamicTable is one that does not have any rows. When adding tables to an NWB file, do not write empty tables.

Tables With Only a Single Row

It is not common to save a table with only a single row entry. Consider other neurodata_types, such as a one-dimensional TimeSeries.

Check function: check_single_row()

Tables With an Entire Column as NaN

If an entire column of a DynamicTable would be filled with NaN values, then it should not be written.

Check function: check_col_not_nan()

Table Region Data

Store data with long columns rather than long rows. When constructing dynamic tables, keep in mind that the data is stored by column, so it will be less efficient to store data in a table that has many more columns than rows.

Check function check_dynamic_table_region_data_validity()

Boolean Columns

Use boolean values where appropriate. Although boolean values (True/False) are not used in the core schema, they are a supported data type, and we encourage the use of DynamicTable columns with boolean values. It is also encouraged practice for boolean columns to be named is_condition where condition is whatever the positive state of the variable is, e.g. you might create a column called is_correct that has boolean values.

The reason for this practice is two-fold:

(i) It allows for easier user comprehension of the information by intuitively restricting the range of possible values for the column; a user would otherwise have to extract all the values and calculate the unique set to see that there are only two values.

(ii) For large amounts of data, it also saves storage space for the data within the HDF5 file by using the minimal number of bytes per item. This can be especially importance if the repeated values are long strings or float casts of 1 and 0.

An example of a violation of this practice would be for a column of strings with the following values everywhere;

hit_or_miss_col = ["Hit", "Miss", "Miss", "Hit", ...]

This should instead become

is_hit = [True, False, False, True, ...]

Check function check_column_binary_capability()

Note

If the two unique values in your column are float types that differ from 1 and 0, the reported values are to be considered as additional contextual information for the column, and this practice does not apply.

Note

HDF5 does not natively store boolean values. h5py handles this by automatically transforming boolean values into an enumerated type, where 0 maps to “TRUE” and 1 maps to “FALSE”. Then on read these values are converted back to the np.bool type. pynwb does the same, so if you are reading and writing with pynwb you may not need to worry about this. However, this will be important to know if you write using PyNWB and read with some other language.

Timing Columns

Times are always stored in seconds in NWB. In TimeIntervals tables such as the TrialsTable and EpochsTable, start_time and stop_time should both be in seconds with respect to the timestamps_reference_time of the NWBFile (which by default is the session_start_time, see Global Date and Time Reference for more details).

The start_time values should reflect when each interval began during the recording session, should be in non-decreasing order, and should not all be the same value. Each stop_time value should be strictly greater than its corresponding start_time. Common signs that times were not properly converted to the session reference frame are all start_time and stop_time values being zero, zero-duration intervals where stop_time equals start_time, or other time columns in the table containing trial-relative offsets instead of session-absolute times.

Additional time columns in TimeIntervals tables, such as the TrialsTable should have _time as a suffix to the name. E.g., if you add more times in TrialsTable, such as a subject response time, name it response_time and store the time values in seconds from the timestamps_reference_time of the NWBFile, just like start_time and stop_time. This convention is used by downstream processing tools. For instance, NWBWidgets uses these times to create peri-stimulus time histograms relating spiking activity to trial events. See Global Date and Time Reference for more details.

Check functions: check_time_intervals_duration(), check_time_intervals_start_time_not_constant(), check_time_intervals_stop_after_start()

Unique ids

The values of the id attribute of any DynamicTable should be unique. This includes descendants of DynamicTable such as TimeIntervals and ElectrodesTable. In PyNWB, rows of DynamicTable increment as you add rows, so this variable is unique by default. If you would like to make values of id non-unique, a better solution would be to store these values as a custom column and use the default id values.