Time Series

When using TimeSeries or any subtype (e.g., ElectricalSeries, SpatialSeries, ImageSeries, etc.) please ensure the following practices are followed.

Data Orientation

For the data field of TimeSeries, the first dimension of the array should always be time.

Keep in mind that the dimensions are reversed in MatNWB, so in memory in MatNWB the time dimension must be last.

In PyNWB the order of the dimensions is the same in memory as on disk, so the time index should be first.

Check functions: check_data_orientation() and check_timestamps_match_first_dimension()

Units of Measurement

Time-related values should always be in seconds. This includes rate (if applicable), which should be in Hz.

Every TimeSeries instance has unit as an attribute, which is meant to indicate the unit of measurement for that data, using the appropriate type from the International System of Units (SI).

A common mistake is to provide timestamps in milliseconds or microseconds instead of seconds, which results in unusually long durations (e.g., appearing to span years instead of hours). Similarly, providing the sampling period (time between samples) instead of the rate (frequency in Hz) leads to suspiciously low rate values.

Check functions: check_missing_unit(), check_time_series_duration(), check_rate_not_below_threshold()

Time Series: Time References

The timestamps or rate of a TimeSeries should be in seconds with respect to the global timestamps_reference_time of the nwb-schema:NWBFile.

Subtypes

ElectricalSeries are reserved for neural data. An ElectricalSeries holds signal from electrodes positioned in or around the brain that are monitoring neural activity, and only those electrodes should be in the ElectrodeTable.

For non-neural electrodes that still may store and report raw values in Volts, simply use a general TimeSeries object with units set to “Volts”.

Breaks in Continuity

The data field of TimeSeries should generally be stored as one continuous stream as it was acquired, not by trial as is often reshaped for analysis.

Data can be trial-aligned on-the-fly using the TrialTable.

Storing measured data as a continuous stream ensures that other users have access to the inter-trial data, and that we can align the data within any specifiable window.

If you only have data spanning specific segments of time, then only include those timepoints in the data, see Timestamps vs. Start & Rate for more information.

A primary implication is that the values in nwb-schema:TimeSeries.timestamps, as well as the corresponding ordering of their indices in the nwb-schema:TimeSeries.data array, should always be strictly increasing.

Check function: check_timestamps_ascending()

Timestamps without NaNs

The timestamps field of a TimeSeries should not contain NaN values, as this can lead to ambiguity in time references and potential issues in downstream analyses.

Ensure that all timestamps are valid numerical values. If gaps in time need to be represented, consider segmenting the data into separate TimeSeries objects with appropriate starting_time or use the timestamps vector to explicitly represent time gaps.

Check function: check_timestamps_without_nans()

Timestamps vs. Start & Rate

TimeSeries allows you to specify time using either timestamps or rate together with starting_time (which defaults to 0). If the sampling rate is constant, then specify the rate and starting_time instead of writing the full timestamps vector.

For segmented data, refer to the section covering Breaks in Continuity;

1. If the sampling rate is constant within each segment, each segment can be written as a separate TimeSeries with the starting_time incremented appropriately.

2. Even if the sampling rate is constant within each segment, a single TimeSeries can be written using the timestamps vector to appropriately indicate the gaps between segments.

Check function: check_regular_timestamps()

Avoid Negative Timestamps

When writing TimeSeries data, avoid using negative timestamps.

All timestamps in the NWBFile are written with respect to a global reference point (either timestamps_reference_time or session_start_time).

While negative timestamps are technically valid, they might introduce unnecessary complications for future users of the file. In most cases, negative timestamps may be indicative of an alignment error or a problem with the source data. As much as possible, re-align the session start time so that all timestamps are positive and correctly referenced to either timestamps_reference_time or session_start_time. See Global Date and Time Reference for more details.

Check function: check_timestamp_of_the_first_sample_is_not_negative()

Chunk Data

Use chunking to optimize reading of large data for your use case.

By default, when using the HDF5 backend, TimeSeries data are stored on disk using column-based ordering.

This means that if the data of a TimeSeries has multiple dimensions, then all data from a single timestamp are stored contiguously on disk, followed by the next timestamp, and so on.

This storage scheme may be optimal for certain uses, such as slicing TimeSeries by time; however, it may be sub-optimal for other uses, such as reading data from all timestamps for a particular value in the second or third dimension.

This is especially important when writing NWBFiles that are intended to be uploaded to the DANDI Archive for storage, sharing, and publication.

For more information about how to enable chunking and compression on your data, consult the PyNWB tutorial or the MatNWB instructions.

Compress Data

Data writers can optimize the storage of large data arrays for particular uses by using compression applied to each chunk individually. This is especially important when writing NWBFiles that are intended to be uploaded to the DANDI Archive for storage, sharing, and publication. For more information about how to enable compression on your data, consult the PyNWB tutorial or the MatNWB instructions

Check functions: :py:check_large_dataset_compression(), :py:check_small_dataset_compression()

Unknown Resolution

If the resolution of a TimeSeries is unknown, use -1.0 or NaN to indicate this.

Check function: :py:check_resolution()

Zero Rate

If the data field of TimeSeries has more than one frame, and according to Data Orientation this axis ought to be time, then the rate field should not be 0.0.

Check function: :py:check_rate_is_not_zero()

. _best_practice_positive_rate:

Positive Rate

The rate field of TimeSeries must be positive when specified. Negative sampling rates are not physically meaningful and indicate an error in the data.

Check function: :py:check_rate_is_positive()