Data Manager
The DataManager has 3 core functionalities
Assigns TimeFrames, or temporal coordinate systems to different datatypes. It keeps track of these relationships
Operates a callback system that allows observers to know when data has been added to or removed from the datamanager.
Checks in and serves data of different types to the user
Time Frame Structure for Multi-Rate Data Visualization
An important feature of the program is being able to simultaneously visualize data that was collected at different sampling rates. The time frame structure is designed to indicate when time events occur at a particular sampling frequency. The data manager is responsible for keeping track of which data is located in what time frame. Multiple data objects can belong to one time frame, but one data object can only belong to a single time frame. However, if the relationship between time frame objects is specified, then data requested in one coordinate system can be converted to another.
Illustrative Example: Electrophysiology and Video Data
Consider the following example: an NI-DAQ box samples at 30000 Hz for electrophysiology. This results in analog data with 30000 samples per second as well as event data for sorted spikes at 30000 Hz resolution. The experiment may have also had a high-speed video camera collecting frames at 500 Hz. A digital event for each frame was recorded with the same NI-DAQ. The user may have processed the video frames to categorize behavior, which would also be at the 500 Hz resolution and be interval data.
Default Time Frame and Customization
The default time frame is simply called “time” and defaults to numbers between 1 and the number of frames in a loaded video. The scroll bar operates in this time frame and consequently sends signals in this time frame. The user can override this default time frame and replace it with an event structure with the same number of samples but where each event corresponds to the 30000 Hz resolution digitized camera frame exposures. The user can then create a new clock called “master” that again counts from 1,2,3,… up to the total number of samples collected by the NI-DAQ.
Data Indexing within Time Frames
All data manager data types have a notion of “index,” and these correspond to the time frame they are associated with. This index property is important because data may be sparsely labeled and not have the same number of samples as their time frame.
Widget Considerations for Data Synchronization
Widgets that represent different data simultaneously must be aware of accounting for these differences.
Performance Notes
The user should also be aware that pointer indirections sample by sample for large vectors will be quite inefficient. If the user needs to find a series of values in a range in a different coordinate system it is most likely important to.
Observer Framework
Data Types
Point
A Point represents a 2 dimensional (x, y) coordinate, which may vary in time. The PointData structure can hold multiple points per unit time.
| Examples of Point Data |
|---|
Line
A Line represents a 2 dimensional (x,y) collection of points, which may vary with time. The collection of points is ordered, meaning that each point is positioned relative to its neighbors. The LineData structure can hold multiple lines per unit time.
| Examples of Line Data |
|---|
Currently, lines are represented by a raw list of points. One could imagine lines being parameterized in different ways, such as the coefficients of a polynomial fit. For many calculations, the parameterized version of a line may be less memory intensive and more efficient.
Mask
A mask represents a 2 dimensional collection of points, which may vary with time. Compared to a Line, the points in a Mask are not ordered. These would correspond to something like a binary segmentation mask over an image. The MaskData structure can hold multiple masks per unit time.
| Examples of Mask Data |
|---|
| Binary Semantic Segmentation Labels |
A mask may just be thought as a raw pixel, by pixel definition of a shape. Shapes could be defined as bounding boxes, circles, polygons, etc. It may one day be useful to describe shapes in other ways compared to the raw pixel-by-pixel definition.
Tensors
Tensors are N-Dimensional data structures, and consequently very flexible containers for complex data. A concrete use would be to store a Height x Width x Channel array for different timepoints during an experiment. These may be the features output from an encoder neural network that processes a video.
| Examples of Tensor Data |
|---|
Analog Time Series
An analog time series has values that can vary continuously
| Examples of Analog Time Series Data |
|---|
| Voltage traces from electrode recordings |
| Fluorescence intensity traces from Calcium imaging |
Digital Event Series
A digital event represents an ordered sequence of events, where each event is represented as a single instance in time. For instance, spike times from electrophysiology
| Examples of Event Data |
|---|
Digital Interval Series
| Examples of Interval Data |
|---|
Media
Media is a sequence of images.
Image
| Examples of Image Data |
|---|
| Image sequence from two photon calcium imaging experiment |
Video
| Examples of Video Data |
|---|
| MP4 video from high speed scientific camera of behavior |