Kwik format

Cortex Lab, University College London

Cyrille Rossant

Overview

Description of the format
Performance considerations
Analysis and visualization framework

Einevoll 2012

Spike sorting

The Kwik format

.kwik

.kwx

.kwd

tens of MB

tens of GB

spike times

clustering

metadata

probe

features

masks

waveforms

raw data

filtered data

3 HDF5 files

# https://github.com/klusta-team/kwiklib/wiki/Kwik-format

/kwik_version* [=2]
/name*
/application_data
    spikedetekt
        MY_SPIKEDETEKT_PARAM*
        ...
/user_data
/channel_groups
    [X]  # channel group index
        name*
        channel_order*  # list of channels
        adjacency_graph* [Kx2 array of integers]
        application_data
        user_data
        channels
            [X]  # Relative channel index from 0 to shanksize-1
                name*
                ignored*
                position*
                voltage_gain*
                display_threshold*
                application_data
                    klustaviewa
                    spikedetekt
                user_data
        spikes
            time_samples* [N-long EArray of UInt64]
            time_fractional* [N-long EArray of UInt8]
            recording* [N-long EArray of UInt16]
            clusters
                main* [N-long EArray of UInt32]
                original* [N-long EArray of UInt32]
            features_masks
                hdf5_path* [='{kwx}/channel_groups/X/features_masks']
            waveforms_raw
                hdf5_path* [='{kwx}/channel_groups/X/waveforms_raw']
            waveforms_filtered
                hdf5_path* [='{kwx}/channel_groups/X/waveforms_filtered']
        clusters
            [clustering_name]
                [X]  # Cluster number
                    application_data
                        klustaviewa
                            color*
                    cluster_group*
                    mean_waveform_raw*
                    mean_waveform_filtered*
                    quality_measures
                        isolation_distance*
                        matrix_isolation*
                        refractory_violation*
                        amplitude*
                    user_data
                        ...
        cluster_groups
            [clustering_name]
                [X]  # Cluster group number
                    name*
                    application_data
                        klustaviewa
                            color*
                    user_data
/recordings
    [X]  # Recording index from 0 to Nrecordings-1
        name*
        start_time*
        start_sample*
        sample_rate*
        bit_depth*
        band_high*
        band_low*
        raw
            hdf5_path* [='{raw.kwd}/recordings/X']
        high
            hdf5_path* [='{high.kwd}/recordings/X']
        low
            hdf5_path* [='{low.kwd}/recordings/X']
        user_data
/event_types
    [X]  # The name of the event type.
        user_data
        application_data
            klustaviewa
                color*
        events
            time_samples* [N-long EArray of UInt64]
            recording* [N-long EArray of UInt16]
            user_data [group or EArray]

The .kwik file

The .kwx file


/channel_groups
  [0]
    features_masks # [(N x NFEATURES x 2) EArray of Float32]
    waveforms_raw # [(N x NWAVESAMPLES x NCHANNELS) EArray of Int16]
    waveforms_filtered # [(N x NWAVESAMPLES x NCHANNELS) EArray of Int16]

The .kwd files

# experiment.raw.kwd
/recordings
  [0]
    data # [(Nsamples x Nchannels) EArray of Int16]
    [and some metadata...]

# idem for:
# experiment.high.kwd
# experiment.low.kwd

Probe format

channel_groups = {
    0: {
        # list of channels to keep
        "channels": [0, 1, 2, 3],
        # list of pairs of connected (nearby) channels
        "graph": [[0, 1], [2, 3], ...],  
        "geometry": {
            0: [0.1, 0.2],
            1: [0.3, 0.4], ...} 
        },
    1: {
        "channels": [4, 5, 6, 7],
        "graph": [[4, 5], [6, 7], ...],
        "geometry": {
            4: [0.1, 0.2], 
            5: [0.3, 0.4], ...}
        }
    }