Heatmap Widget Roadmap

HeatmapWidget Development Roadmap

This document tracks remaining development tasks for the Heatmap Widget. For documentation of completed features and current architecture, see the main developer documentation.

Completed Work Summary

The following features are complete and documented in index.qmd:

  • Widget skeleton: EditorState pattern, registration, state/view/properties factories
  • State management: HeatmapState with PlotAlignmentState, RelativeTimeAxisState, VerticalAxisState composition
  • Axis system: RelativeTimeAxisWidget (X) and VerticalAxisWidget (Y) with bidirectional sync
  • Interaction: Separated X/Y zoom, pan, double-click time navigation via shared PlotInteractionHelpers
  • Properties panel: Alignment controls (PlotAlignmentWidget), axis range controls in collapsible sections
  • OpenGL lifecycle: SceneRenderer initialization, background color, projection/view matrix management
  • State serialization: Full round-trip JSON via rfl::json
  • Unit selection state: HeatmapStateData::unit_keys with full add/remove/batch API and unitsChanged() signal
  • Unit selection UI: Feature_Tree_Widget embedded in properties panel, filtered to DigitalEventSeries, with prefix-based group toggling
  • Rate estimation infrastructure: EventRateEstimation library in Plots/Common/EventRateEstimation/ providing createUnitGatherContext() / createUnitGatherContexts() for batched aligned gathering and estimateRate() / estimateRates() returning RateEstimate (paired times[] + values[]) with a std::variant<BinningParams, GaussianKernelParams, CausalExponentialParams>-based variation point. See Rate Estimation Redesign for the full API design.
  • Colormap infrastructure: CorePlotting::Colormaps library in CorePlotting/Colormaps/ providing ColormapFunction, ColormapPreset (Inferno, Viridis, Magma, Plasma, Coolwarm, Grayscale, Hot), LUT-based O(1) evaluation with linear interpolation, mapValue() / mapValues() / mapMatrix() utilities. Shared with SpectrogramWidget and any future external-feature coloring use cases.
  • Heatmap scene building: HeatmapMapper in CorePlotting/Mappers/ converting HeatmapRowData rows to RenderableRectangleBatch with Auto / Manual / Symmetric color range modes. HeatmapOpenGLWidget::rebuildScene() now gathers units via createUnitGatherContexts(), estimates rates via estimateRates(), applies scaling via applyScaling(), converts RateEstimate objects to HeatmapRowData, and uploads the scene.
  • Y-axis auto-fit: HeatmapWidget auto-fits the vertical axis when the unit count changes. Y bounds are set to [0, num_units], Y zoom is reset to 1.0, and Y pan is reset to 0.0 so all units are visible. The VerticalAxisWidget uses identityAxis("Unit", 0) for integer unit-index labels. Signal renamed from trialCountChanged to unitCountChanged for semantic clarity.
  • Scaling and normalization: Five scaling modes (Firing Rate Hz, Z-Score, Normalized [0,1], Raw Count, Count/Trial). Uses shared ScalingMode/applyScaling() from RateEstimate.hpp. HeatmapStateData extended with scaling and color_range (Auto/Manual/Symmetric). Auto-switches to Coolwarm colormap for z-score and Symmetric color range. Properties panel has “Scaling & Color Range” collapsible section with ScalingModeControls widget and manual vmin/vmax spinboxes.
  • Rate estimation options : EstimationParams variant (std::variant<BinningParams, GaussianKernelParams, CausalExponentialParams>) stored directly in HeatmapStateData::estimation_params. Lightweight EstimationParams.hpp header (no heavy transitive deps) safe to include from state headers. RateEstimationControls library provides composable Qt widgets: EstimationMethodControls (method combo + stacked per-method parameter pages) and ScalingModeControls (scaling mode combo). HeatmapPropertiesWidget uses both widgets. Renderer passes estimationParams() directly to estimateRates() — no intermediate conversion.
  • Row sorting: Five sorting modes (Manual, TimeToPeak, PeakRate, MeanRate, Alphabetical) with ascending/descending toggle. HeatmapSortMode enum and sortModeLabel() / allSortModes() utilities defined in HeatmapState.hpp. HeatmapStateData extended with sort_mode and sort_ascending. Sorting logic in HeatmapDataPipeline.hpp/cpp via computeSortOrder() (returns index permutation) and applySortOrder() (reorders rows, rate estimates, and unit keys in-place). Applied after runHeatmapPipeline() in renderer before colormap mapping. Properties panel has “Sorting” collapsible section with sort mode combo and ascending checkbox. Full serialization round-trip.
  • Colormap selection UI: ColormapControls composable widget in Plots/Common/ColormapControls/ with colormap preset combo (gradient icon previews from LUT), color range mode combo (Auto/Manual/Symmetric), and vmin/vmax spinboxes. HeatmapStateData::colormap field with colormapChanged() signal. Auto-switches to Coolwarm for z-score scaling. Renderer reads colormap from state.

Not yet implemented: colorbar display (Phase 4) and all cross-widget linking described below.

Phase 4: Colorbar Display

Status: Not started. Phases 2 and 3 are complete; this phase can proceed.

Goal: Render a color legend alongside the heatmap showing the value-to-color mapping.

4.1 ColorbarWidget

Create a lightweight QWidget that renders a vertical color gradient with tick labels:

CorePlotting/Widgets/
├── ColorbarWidget.hpp
└── ColorbarWidget.cpp

The ColorbarWidget needs:

  • A ColormapFunction to render the gradient
  • A value range (vmin, vmax) for tick label computation
  • An axis label (e.g., “Firing Rate (Hz)”, “Z-Score”)
  • Optional tick formatting (integer, decimal, scientific notation)

4.2 Integration with HeatmapWidget

Place the colorbar to the right of the OpenGL canvas in the HeatmapWidget layout:

┌──────────────────────────────────┬────┐
│                                  │    │
│       HeatmapOpenGLWidget        │ CB │
│                                  │    │
├──────────────────────────────────┴────┤
│         RelativeTimeAxisWidget        │
└───────────────────────────────────────┘

The colorbar updates when the colormap, scaling mode, or value range changes.

4.3 Shared Consumer

The ColorbarWidget should be reusable by SpectrogramWidget, which also needs a color legend for its power spectrum display.

Phase 7: Cross-Widget Linking

Status: Design complete. Related to EventPlot Roadmap Phase 6 and PSTH Roadmap Phase 4.

Goal: Clicking a heatmap row updates PSTHWidget and EventPlotWidget to show the selected unit’s data.

7.1 SelectionContext Integration

Following the existing SelectionContext pattern used by DataInspector widgets:

// When user clicks a heatmap row:
void HeatmapOpenGLWidget::mouseReleaseEvent(QMouseEvent * event) {
    if (!_is_panning) {
        int unit_index = worldToUnitIndex(screenToWorld(event->pos()));
        auto const & unit_config = _state->units().at(unit_index);
        // Emit via SelectionContext:
        SelectionContext::instance().setSelectedData(
            unit_config.event_key, SelectionSource::HeatmapWidget);
    }
}

7.2 Receiving Widgets

In PSTHWidget and EventPlotWidget (if not pinned):

void PSTHPropertiesWidget::_onSelectionChanged(SelectionSource const & source) {
    if (source.type != SelectionSourceType::HeatmapWidget) return;
    if (_state->isPinned()) return;
    auto key = source.data_key;
    auto type = _data_manager->getDataType(key);
    if (type == DataType::DigitalEventSeries) {
        _state->clearPlotEvents();
        _state->addPlotEvent(key, PSTHEventOptions{.event_key = key});
        // Scene rebuilds automatically via stateChanged()
    }
}

7.3 Row Click → Unit Selection Flow

The primary cross-widget workflow:

HeatmapWidget row click
        ↓
SelectionContext::setSelectedData("unit_42", source)
        ↓
PSTHWidget (if not pinned) receives selectionChanged
        ↓
Clears existing events, adds "unit_42" → rebuilds histogram
        ↓
EventPlotWidget (if not pinned) also receives selectionChanged
        ↓
Updates raster plot for "unit_42"

This enables rapid exploration of neural data: click through units in the heatmap while the PSTH and raster plot update in sync. See EventPlot Roadmap Phase 6 and PSTH Roadmap Phase 4 for the complementary receiver implementations.

7.4 Visual Row Highlighting

When a row is selected (either by click or via incoming SelectionContext), highlight the selected row with a border or background overlay to provide visual feedback.

7.5 Pin Button

Add a pinned field to HeatmapStateData and a pin/unpin toggle button to the properties panel toolbar. When pinned, the widget ignores SelectionContext changes, preserving its current view. The pinned state is serialized for workspace restoration.

Phase 8: Tooltip with Stacked PSTH + Raster

Status: In progress. Basic tooltip wiring is complete — hovering over a heatmap row shows the unit key name via PlotTooltipManager. The tooltip correctly reflects the current sort order. PSTH and raster glyph rendering in the tooltip is not yet implemented. Depends on Phase 1 and Phase 7.

Goal: When hovering over a heatmap row, display a tooltip containing a miniature stacked PSTH histogram and raster plot for that unit.

8.1 Concept

┌─────────────────────────────────────────────────────┐
│                   Heatmap                           │
│  unit_A  ██████████████████████████████████████████  │
│  unit_B  ██████████████████████████████████████████  │
│  unit_C  ████████▓▓▓▓████████████████████████████  ← hover
│  unit_D  ██████████████████████████████████████████  │
│                                                     │
│          ┌─────────────────────┐                    │
│          │  unit_C PSTH        │                    │
│          │  ▄▄ ██ ▄▄ ..       │                    │
│          │─────────────────────│                    │
│          │  unit_C Raster      │                    │
│          │  | || |  || |       │                    │
│          │  ||  | | |  ||      │                    │
│          └─────────────────────┘                    │
└─────────────────────────────────────────────────────┘

8.2 Lightweight Rendering

The tooltip must render quickly without creating full widget instances. Two approaches:

Option A: Pre-rendered QPixmap Cache

When the scene rebuilds, pre-render PSTH and raster thumbnails for each unit into QPixmap objects. The tooltip simply displays the cached pixmap. This has memory cost proportional to the number of units but provides instant tooltips.

Option B: On-Demand QPainter Rendering

When the hover event fires, compute the PSTH bins and raster tick positions for the hovered unit using the cached GatherResult data, and render directly with QPainter into a tooltip widget. This avoids the memory cost but requires fast rendering.

Recommendation: Option B with the cached GatherResult data from the scene rebuild. The rate computation uses the shared estimateRate() from the EventRateEstimation library (the same function used by the heatmap itself), and QPainter rendering of a simple histogram + tick marks is lightweight.

8.3 Tooltip Content

The tooltip should show:

  1. Header: Unit label / event key
  2. PSTH subplot: Bar or line histogram of the unit’s rate profile (same data as the heatmap row)
  3. Raster subplot: Tick marks for each trial’s events (using the unit’s GatherResult)
  4. Optional statistics: Peak rate, time to peak, mean rate

8.4 Implementation Approach

The tooltip lifecycle (dwell timer, suppression during pan/zoom, show/hide) is handled by the shared PlotTooltipManager (Plots/Common/TooltipManager/). The HeatmapWidget provides two callbacks:

  1. Hit test provider: Converts the screen position to a unit index (row lookup from Y coordinate). Returns a PlotTooltipHit with the unit index in user_data.
  2. Content provider: Receives the hit, renders the PSTH + raster thumbnail into a QPixmap via QPainter, and returns it as PlotTooltipContent. The PlotTooltipManager displays the pixmap in its built-in PixmapPopup widget.
_tooltip_mgr = std::make_unique<PlotTooltipManager>(this);

_tooltip_mgr->setHitTestProvider([this](QPoint pos) -> std::optional<PlotTooltipHit> {
    QPointF world = screenToWorld(pos);
    int unit_index = worldToUnitIndex(world);
    if (unit_index < 0) return std::nullopt;
    PlotTooltipHit hit;
    hit.world_x = static_cast<float>(world.x());
    hit.world_y = static_cast<float>(world.y());
    hit.user_data = unit_index;
    return hit;
});

_tooltip_mgr->setContentProvider([this](PlotTooltipHit const & hit) -> PlotTooltipContent {
    int unit_index = std::any_cast<int>(hit.user_data);
    PlotTooltipContent content;
    content.pixmap = renderUnitTooltip(unit_index);  // QPainter PSTH + raster
    return content;
});

The renderUnitTooltip() method:

  • Reuses the shared estimateRate() from the EventRateEstimation library for the PSTH portion
  • Reuses RasterMapper logic for the raster portion (a simplified version without full scene building)
  • Renders both subplots into a single QPixmap via QPainter

This is related to PSTH Roadmap Phase 6 (Embeddable PSTH for Tooltips), which proposes decoupling the PSTH computation and rendering from the full widget machinery.

Phase 9: Colormap Selection UI

Status: Complete. Implemented as a shared ColormapControls library in Plots/Common/ColormapControls/. See ColormapControls documentation.

Completed work:

  • ColormapControls widget: Composable widget with colormap preset combo box (gradient icon previews rendered from LUT) and color range controls (Auto / Manual / Symmetric mode with vmin/vmax spinboxes). Embedded in HeatmapPropertiesWidget within the “Rate Estimation & Scaling” collapsible section.
  • State extension: HeatmapStateData::colormap field (ColormapPreset, default Inferno). HeatmapState provides colormapPreset() / setColormapPreset() with colormapChanged() signal. Full JSON serialization round-trip via rfl::json.
  • Auto-suggestion: When switching to z-score scaling, the colormap automatically changes to Coolwarm and the color range switches to Symmetric. Reverting from z-score restores Inferno and Auto.
  • Renderer integration: HeatmapOpenGLWidget::rebuildScene() reads the colormap from _state->colormapPreset() instead of hard-coding based on scaling mode.

Phase 10: PNG Export

Status: Not started. Qt provides the underlying infrastructure via QOpenGLWidget::grabFramebuffer().

Goal: Export the current heatmap view as a publication-quality PNG image.

10.1 Basic Grab

The simplest approach uses Qt’s built-in framebuffer grab:

void HeatmapWidget::exportPNG(QString const & path) {
    QImage image = _opengl_widget->grabFramebuffer();
    image.save(path, "PNG");
}

This captures exactly what is displayed, including the current zoom/pan state.

10.2 High-Resolution Export

For publication quality, render at a higher resolution than screen:

void HeatmapWidget::exportPNG(QString const & path, int scale_factor) {
    // Resize OpenGL widget to scaled dimensions
    // Render at high resolution
    // Grab framebuffer
    // Restore original size
}

10.3 Composite Export with Axes and Colorbar

For a complete figure, composite the OpenGL framebuffer with axis labels and colorbar:

  1. Render the heatmap at target resolution
  2. Render axes using QPainter at matching resolution
  3. Render colorbar using QPainter
  4. Composite into a single QImage
  5. Save as PNG

10.4 UI Integration

Add an “Export PNG…” action to:

  • The properties panel (button or menu)
  • A right-click context menu on the plot

The dialog should offer:

  • File path selection
  • Resolution scale factor (1x, 2x, 4x)
  • Include axes checkbox
  • Include colorbar checkbox

10.5 SVG Export (Future)

The PSTH Roadmap Phase 5 proposes shared SVG export infrastructure for all plot widgets using SceneRenderer. The HeatmapWidget should adopt this when available, as SVG provides resolution-independent output for publications.

Phase 11: AnalogTimeSeries Heatmap Variant (Future)

Status: Early conceptual design. Separate from the DigitalEventSeries heatmap.

Goal: A sibling widget that creates heatmaps from AnalogTimeSeries data rather than discrete events. This enables visualization of continuous signals like ΔF/F calcium imaging traces across multiple cells, aligned to events.

11.1 Concept

Where the current HeatmapWidget computes rate from discrete events (analogous to stacking PSTHWidget rows), the AnalogTimeSeries variant would display raw continuous signals (analogous to stacking LinePlotWidget rows).

Feature Event Heatmap (current) Analog Heatmap (future)
Data source DigitalEventSeries AnalogTimeSeries
Per-unit computation Binning + rate estimation Direct value sampling / interpolation
Analogous single-unit view PSTHWidget LinePlotWidget
Typical use case Population spike rate ΔF/F, LFP, EMG across units
Normalization Firing rate, z-score Z-score, ΔF/F₀, % change

11.2 Shared Infrastructure

Despite the different data sources, significant infrastructure is shared:

  • Colormap system (Phase 2)
  • Colorbar widget (Phase 4)
  • Sorting modes (Phase 6) — time-to-peak, peak value, etc.
  • Cross-widget linking (Phase 7) — click row → update linked widgets
  • PNG/SVG export (Phase 10)
  • Scaling/normalization (Phase 3) — z-score, 0-1 normalization

11.3 Different Infrastructure

Key differences that may warrant a separate widget rather than a mode switch:

  • Data gathering: GatherResult<AnalogTimeSeries> instead of GatherResult<DigitalEventSeries>
  • Value computation: Direct sampling/interpolation instead of binning + rate estimation
  • Alignment: May align to DigitalIntervalSeries only (baseline period for ΔF/F₀ normalization)
  • Time resolution: Native sampling rate rather than binned; may need downsampling for display
  • Tooltip: Would show a line trace rather than a PSTH histogram

11.4 Recommendation

Implement as a separate widget (AnalogHeatmapWidget) that reuses the colormap, colorbar, sorting, linking, and export infrastructure. The rendering and data gathering paths are sufficiently different that forcing them into a single widget would add unwanted complexity. The two widgets can coexist in the Plot menu, clearly labeled:

  • Plot → Heatmap Plot (Events) — current widget
  • Plot → Heatmap Plot (Analog) — future widget

Open Questions

  1. Row label rendering: Should unit labels be rendered as axis tick labels via VerticalAxisWidget, or as a separate label column? The axis approach reuses existing infrastructure but may not handle long labels well.

  2. Performance at scale: What is the target unit count? 100 units × 200 bins = 20K rectangles is trivial for OpenGL. 1000 units × 1000 bins = 1M rectangles may need texture-based rendering instead of rectangle batches.

  3. AnalogTimeSeries variant timing: Should the analog heatmap be a mode of this widget or a completely separate widget? The shared infrastructure is substantial, but the data pipelines are fundamentally different.

See Also