How well does probe_layer
cover braindecode?

Layer-wise linear probing tested across the entire braindecode.models.util.models_dict registry: every model, a sample of its named submodules, one forward pass per (model, layer) pair. The dots below are the result — green for a passing hook, vermillion for a failure, hatched for skipped.

56
Models in registry
51/56
Built successfully
21
100% layer pass-rate
394/488
Layers passed · 80.7%

The matrix

Click a row to enumerate its layers
All Perfect Partial Build failed Skipped 56 shown
Nothing matches that filter.

Why layers fail, when they do

Pattern analysis

By cause

By model (top offenders)

REVE concentrates motor-imagery information mid-way through its transformer Linear-probe AUROC across frozen REVE layers · BNCI 2014-001 (4-class) · 3 seeds 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 AUROC · test set chance · 0.50 model.to_patch_embedding.0 AUROC: 0.633 ± 0.005 bal_acc: 0.369 ± 0.014 n_seeds: 3 model.mlp4d AUROC: 0.500 ± 0.000 bal_acc: 0.250 ± 0.000 n_seeds: 3 model.transformer.layers.0.1 AUROC: 0.639 ± 0.024 bal_acc: 0.372 ± 0.022 n_seeds: 3 model.transformer.layers.3.1 AUROC: 0.669 ± 0.010 bal_acc: 0.386 ± 0.004 n_seeds: 3 model.transformer.layers.7.1 AUROC: 0.759 ± 0.002 bal_acc: 0.483 ± 0.003 n_seeds: 3 model.transformer.layers.11.1 AUROC: 0.802 ± 0.003 bal_acc: 0.498 ± 0.008 n_seeds: 3 model.transformer.layers.15.1 AUROC: 0.807 ± 0.002 bal_acc: 0.536 ± 0.006 n_seeds: 3 model.transformer.layers.21.1 AUROC: 0.693 ± 0.001 bal_acc: 0.398 ± 0.003 n_seeds: 3 model.ln AUROC: 0.500 ± 0.000 bal_acc: 0.250 ± 0.000 n_seeds: 3 T15 0.807 ± 0.002 probe collapses (feature norm) canonical probe 0.788 ± 0.014 final output Patch MLP T0 T3 T7 T11 T15 T21 LN Patch + input MLP Transformer (22 blocks · sampled) Output norm Frozen REVE-base (69.7M parameters), 9 subjects, seeds 33/34/35. Shaded band = mean ± 1 SD across seeds. All probes use aggregation=flatten on batch-first captures. Sampled 7 of the 22 transformer blocks.
show numbers · REVE
probe_layerAUROC ± SDbal_acc ± SDn
canonical (probe at REVE final output)0.788 ± 0.0140.500 ± 0.0323
model.to_patch_embedding.00.633 ± 0.0050.369 ± 0.0143
model.mlp4d0.500 ± 0.0000.250 ± 0.0003
model.transformer.layers.0.10.639 ± 0.0240.372 ± 0.0223
model.transformer.layers.3.10.669 ± 0.0100.386 ± 0.0043
model.transformer.layers.7.10.759 ± 0.0020.483 ± 0.0033
model.transformer.layers.11.10.802 ± 0.0030.498 ± 0.0083
model.transformer.layers.15.10.807 ± 0.0020.536 ± 0.0063
model.transformer.layers.21.10.693 ± 0.0010.398 ± 0.0033
model.ln0.500 ± 0.0000.250 ± 0.0003
Motor-imagery information peaks in early BENDR convolutional layers Linear-probe AUROC across frozen BENDR layers · BNCI 2014-001 (4-class) · 3 seeds 0.45 0.50 0.55 0.60 0.65 0.70 AUROC · test set chance · 0.50 aggregation changes: flatten · then mean_tokens for transformer encoder.encoder.Encoder_0 AUROC: 0.651 ± 0.023 bal_acc: 0.372 ± 0.018 n_seeds: 3 encoder.encoder.Encoder_1 AUROC: 0.639 ± 0.006 bal_acc: 0.368 ± 0.006 n_seeds: 3 encoder.encoder.Encoder_2 AUROC: 0.625 ± 0.005 bal_acc: 0.355 ± 0.007 n_seeds: 3 encoder.encoder.Encoder_3 AUROC: 0.626 ± 0.006 bal_acc: 0.348 ± 0.002 n_seeds: 3 encoder.encoder.Encoder_4 AUROC: 0.581 ± 0.014 bal_acc: 0.317 ± 0.001 n_seeds: 3 encoder.encoder.Encoder_5 AUROC: 0.544 ± 0.013 bal_acc: 0.296 ± 0.006 n_seeds: 3 contextualizer.input_conditioning.0 AUROC: 0.519 ± 0.038 bal_acc: 0.265 ± 0.026 n_seeds: 3 contextualizer.input_conditioning.1 AUROC: 0.508 ± 0.015 bal_acc: 0.260 ± 0.005 n_seeds: 3 contextualizer.input_conditioning.3 AUROC: 0.544 ± 0.044 bal_acc: 0.284 ± 0.039 n_seeds: 3 contextualizer.relative_position.0 AUROC: 0.498 ± 0.034 bal_acc: 0.270 ± 0.021 n_seeds: 3 contextualizer.transformer_layers.0 AUROC: 0.504 ± 0.011 bal_acc: 0.253 ± 0.008 n_seeds: 3 contextualizer.transformer_layers.1 AUROC: 0.500 ± 0.009 bal_acc: 0.253 ± 0.007 n_seeds: 3 contextualizer.transformer_layers.2 AUROC: 0.506 ± 0.006 bal_acc: 0.252 ± 0.015 n_seeds: 3 contextualizer.transformer_layers.3 AUROC: 0.502 ± 0.004 bal_acc: 0.245 ± 0.006 n_seeds: 3 contextualizer.transformer_layers.4 AUROC: 0.504 ± 0.006 bal_acc: 0.262 ± 0.013 n_seeds: 3 contextualizer.transformer_layers.5 AUROC: 0.501 ± 0.015 bal_acc: 0.255 ± 0.020 n_seeds: 3 contextualizer.transformer_layers.6 AUROC: 0.508 ± 0.016 bal_acc: 0.259 ± 0.013 n_seeds: 3 contextualizer.transformer_layers.7 AUROC: 0.499 ± 0.017 bal_acc: 0.249 ± 0.016 n_seeds: 3 Enc 0 0.651 ± 0.023 ic.3 +0.036 vs ic.1 canonical probe 0.509 ± 0.017 final output Enc 0 Enc 1 Enc 2 Enc 3 Enc 4 Enc 5 ic.0 ic.1 ic.3 rel-pos T0 T1 T2 T3 T4 T5 T6 T7 Convolutional encoder Projection Transformer Frozen BENDR (157.1M parameters), 9 subjects, seeds 33/34/35. Shaded band = mean ± 1 SD across seeds. Conv probes use aggregation=flatten; transformer probes use aggregation=mean_tokens.
show numbers
probe_layerAUROC ± SDbal_acc ± SDn
canonical (probe at BENDR final output)0.509 ± 0.0170.259 ± 0.0183
encoder.encoder.Encoder_00.651 ± 0.0230.372 ± 0.0183
encoder.encoder.Encoder_10.639 ± 0.0060.368 ± 0.0063
encoder.encoder.Encoder_20.625 ± 0.0050.355 ± 0.0073
encoder.encoder.Encoder_30.626 ± 0.0060.348 ± 0.0023
encoder.encoder.Encoder_40.581 ± 0.0140.317 ± 0.0013
encoder.encoder.Encoder_50.544 ± 0.0130.296 ± 0.0063
contextualizer.input_conditioning.00.519 ± 0.0380.265 ± 0.0263
contextualizer.input_conditioning.10.508 ± 0.0150.260 ± 0.0053
contextualizer.input_conditioning.30.544 ± 0.0440.284 ± 0.0393
contextualizer.relative_position.00.498 ± 0.0340.270 ± 0.0213
contextualizer.transformer_layers.00.504 ± 0.0110.253 ± 0.0083
contextualizer.transformer_layers.10.500 ± 0.0090.253 ± 0.0073
contextualizer.transformer_layers.20.506 ± 0.0060.252 ± 0.0153
contextualizer.transformer_layers.30.502 ± 0.0040.245 ± 0.0063
contextualizer.transformer_layers.40.504 ± 0.0060.262 ± 0.0133
contextualizer.transformer_layers.50.501 ± 0.0150.255 ± 0.0203
contextualizer.transformer_layers.60.508 ± 0.0160.259 ± 0.0133
contextualizer.transformer_layers.70.499 ± 0.0170.249 ± 0.0163

NeuralBench-EEG-Core — Cross-FM evaluation

Layer-wise linear probes on six EEG foundation models across the nine tasks with public-data YAMLs in facebookresearch/neuroai. The Core spec lists 36 EEG tasks, but only 9 ship dataset configs in the public release — the other 27 require manual access to gated corpora (TUH EEG, THINGS-images, etc.). So “NeuralBench-EEG-Core v1.0” here is the full public-data sweep: 6 FMs × 9 tasks × 10–11 probe layers × 3 seeds. Three (FM×task) cells are structurally impossible: LaBraM × {mental_arithmetic, mental_workload, motor_execution}, where the dataset has channels not in LABRAM_CHANNEL_ORDER (only fixable with InterpolatedLaBraM).

0.50 0.50 0.53 0.60 0.52 0.52 0.84 0.62 0.85 0.79 0.88 0.87 0.65 0.64 0.69 0.63 0.74 0.72 0.72 0.73 0.69 0.80 0.53 0.56 0.64 0.61 0.70 0.64 0.61 0.71 0.63 0.70 0.81 0.63 0.51 0.65 0.58 0.63 0.64 0.73 0.57 0.77 0.69 0.75 0.76 0.93 0.70 0.94 0.87 0.83 0.97 c-VEP ERN Mental Arith. Mental Workload Motor Exec. Motor Imag. N2pc P300 SSVEP BENDR BIOT CBraMod LaBraM LUNA REVE Best AUROC per FM × task (linear probe at best intermediate layer)
FM ranking — mean best-AUROC across 9 task(s) 0.5 0.6 0.7 0.8 0.9 REVE 0.756 CBraMod 0.722 LaBraM 0.693 LUNA 0.693 BENDR 0.685 BIOT 0.604

Per-FM probe depth — motor imagery / Tangermann 2012

BENDR — AUROC by probe layer (motor_imagery / Tangermann 2012) chance 0.5 0.6 0.7 0.8 0 1 3 0 0 1 2 3 4 5 6 7 Encoder_0 Encoder_1 Encoder_2 Encoder_3 Encoder_4 Encoder_5 null
BIOT — AUROC by probe layer (motor_imagery / Tangermann 2012) chance 0.5 0.6 0.7 0.8 encoder patch_embeddin 0 1 0 1 0 1 0 1
CBraMod — AUROC by probe layer (motor_imagery / Tangermann 2012) chance 0.5 0.6 0.7 0.8 0 11 2 3 5 6 8 9 patch_embeddin rearrange null
LaBraM — AUROC by probe layer (motor_imagery / Tangermann 2012) chance 0.5 0.6 0.7 0.8 0 1 10 11 3 4 6 7 8 patch_embed null
LUNA — AUROC by probe layer (motor_imagery / Tangermann 2012) chance 0.5 0.6 0.7 0.8 0 1 2 3 4 5 6 7 channel_locati patch_embed null
REVE — AUROC by probe layer (motor_imagery / Tangermann 2012) chance 0.5 0.6 0.7 0.8 ln mlp4d 0 1 1 1 1 1 1 null

Depth analysis — does the best probe layer generalise?

Each foundation model was probed at every layer of its own architecture across all nine NeuralBench-EEG-Core tasks. The four views below answer one question: if I have to pick a probe layer without knowing the downstream task, where should I tap?

Optimal probe depth varies by FM — no universal layer BENDR peaks at the input, LaBraM at the output, REVE and CBraMod near mid-depth, LUNA close to the input. Mean AUROC across the 9 tasks; each curve uses its own model's normalised depth axis. 0.5 0.6 0.7 0.8 chance 0.00 0.25 0.50 0.75 1.00 input depth fraction within each FM's architecture output mean AUROC across the 9 tasks REVE 0.744 best: trf.15 (depth 0.50) CBraMod 0.714 best: enc.6 (depth 0.57) LUNA 0.692 best: patch (depth 0.10) LaBraM 0.684 best: blk.11 (depth 1.00) BENDR 0.676 best: Enc.0 (depth 0.00) BIOT 0.589 best: 0.1 (depth 0.22) REVE CBraMod LUNA LaBraM BENDR BIOT
REVE & LUNA cluster their best layers tightly; BIOT and LaBraM scatter For each task, we mark where the best probe layer sits on the FM's normalised depth axis. Dot area = AUROC. Tight clusters mean one layer generalises; spread means tuning per task helps. 0.00 0.25 0.50 0.75 1.00 depth fraction within each FM (0 → 1) BENDR c-VEP: AUROC 0.502 @ Enc.0 (depth 0.00) ERN: AUROC 0.837 @ Enc.0 (depth 0.00) M. arith.: AUROC 0.649 @ Enc.0 (depth 0.00) M. workld.: AUROC 0.718 @ Enc.0 (depth 0.00) M. exec.: AUROC 0.534 @ Enc.0 (depth 0.00) M. imag.: AUROC 0.640 @ Enc.0 (depth 0.00) N2pc: AUROC 0.625 @ Enc.1 (depth 0.06) P300: AUROC 0.726 @ Enc.0 (depth 0.00) SSVEP: AUROC 0.931 @ Enc.3 (depth 0.17) 0.02±0.06 BIOT c-VEP: AUROC 0.505 @ 3.0 (depth 0.78) ERN: AUROC 0.618 @ 0.1 (depth 0.22) M. arith.: AUROC 0.642 @ 2.1 (depth 0.67) M. workld.: AUROC 0.719 @ 0.1 (depth 0.22) M. exec.: AUROC 0.565 @ 1.1 (depth 0.44) M. imag.: AUROC 0.606 @ 0.1 (depth 0.22) N2pc: AUROC 0.508 @ 1.1 (depth 0.44) P300: AUROC 0.571 @ 1.0 (depth 0.33) SSVEP: AUROC 0.698 @ 2.0 (depth 0.56) 0.43±0.20 CBraMod c-VEP: AUROC 0.532 @ enc.2 (depth 0.31) ERN: AUROC 0.849 @ rearr (depth 0.00) M. arith.: AUROC 0.687 @ rearr (depth 0.00) M. workld.: AUROC 0.729 @ patch (depth 0.08) M. exec.: AUROC 0.639 @ enc.2 (depth 0.31) M. imag.: AUROC 0.706 @ enc.6 (depth 0.62) N2pc: AUROC 0.648 @ enc.6 (depth 0.62) P300: AUROC 0.766 @ enc.6 (depth 0.62) SSVEP: AUROC 0.944 @ enc.3 (depth 0.38) 0.32±0.26 LaBraM c-VEP: AUROC 0.598 @ blk.7 (depth 0.67) ERN: AUROC 0.789 @ blk.11 (depth 1.00) M. imag.: AUROC 0.634 @ blk.11 (depth 1.00) N2pc: AUROC 0.576 @ blk.11 (depth 1.00) P300: AUROC 0.691 @ blk.6 (depth 0.58) SSVEP: AUROC 0.873 @ blk.11 (depth 1.00) 0.88±0.20 LUNA c-VEP: AUROC 0.523 @ blk.1 (depth 0.33) ERN: AUROC 0.879 @ patch (depth 0.11) M. arith.: AUROC 0.631 @ patch (depth 0.11) M. workld.: AUROC 0.686 @ patch (depth 0.11) M. exec.: AUROC 0.615 @ patch (depth 0.11) M. imag.: AUROC 0.696 @ patch (depth 0.11) N2pc: AUROC 0.629 @ patch (depth 0.11) P300: AUROC 0.746 @ blk.1 (depth 0.33) SSVEP: AUROC 0.828 @ patch (depth 0.11) 0.16±0.10 REVE c-VEP: AUROC 0.525 @ trf.11 (depth 0.39) ERN: AUROC 0.875 @ trf.7 (depth 0.26) M. arith.: AUROC 0.739 @ trf.15 (depth 0.52) M. workld.: AUROC 0.798 @ trf.11 (depth 0.39) M. exec.: AUROC 0.698 @ trf.15 (depth 0.52) M. imag.: AUROC 0.807 @ trf.15 (depth 0.52) N2pc: AUROC 0.641 @ trf.7 (depth 0.26) P300: AUROC 0.756 @ trf.7 (depth 0.26) SSVEP: AUROC 0.965 @ trf.7 (depth 0.26) 0.37±0.12 dot area ∝ AUROC; the small horizontal bar above each row spans mean ± 1 SD of the best-layer position across tasks

Per-FM depth profiles

AUROC at each probe layer, averaged across the nine tasks. Shaded band: ±1 SD across tasks (narrow = robust choice, wide = task-specific). Stage brackets show the architecture's natural blocks.

BENDR Best at the input: Enc.0 → 0.68. 0.5 0.6 0.7 0.8 0.9 AUROC, mean ± SD across 9 tasks Enc.0 encoder ctx in-cond ctx rel-pos ctx trf output
BIOT Best mid-depth: 0.1 → 0.59. 0.5 0.6 0.7 0.8 0.9 AUROC, mean ± SD across 7 tasks 0.1 patch emb transformer blocks pooled encoder
CBraMod Best mid-depth: enc.6 → 0.71. 0.5 0.6 0.7 0.8 0.9 AUROC, mean ± SD across 9 tasks enc.6 input encoder layers output
LaBraM Best mid-depth: blk.11 → 0.68. 0.5 0.6 0.7 0.8 0.9 AUROC, mean ± SD across 6 tasks blk.11 patch emb blocks output
LUNA Best mid-depth: patch → 0.69. 0.5 0.6 0.7 0.8 0.9 AUROC, mean ± SD across 9 tasks patch chan + patch blocks output
REVE Best mid-depth: trf.15 → 0.74. 0.5 0.6 0.7 0.8 0.9 AUROC, mean ± SD across 9 tasks trf.15 patch emb transformer norm/mlp output

Layer × task drilldown

Each cell is AUROC for one (probe layer, task) pair. Tasks ordered by FM mean, so the strongest tasks lie on the left of every panel.

BENDR probe layer × task. Cell = mean AUROC; hatched = < 3 seeds; blank = no run. coverage: 60/171 cells (35%), of which 30 have the full 3-seed grid · best cell 0.931 Enc.0 Enc.1 Enc.2 Enc.3 Enc.4 Enc.5 ctx-ic.0 ctx-ic.1 ctx-ic.3 ctx-rp ctx-tl.0 ctx-tl.1 ctx-tl.2 ctx-tl.3 ctx-tl.4 ctx-tl.5 ctx-tl.6 ctx-tl.7 null SSVEP ERN P300 M. workld. N2pc M. arith. M. imag. M. exec. c-VEP
BIOT probe layer × task. Cell = mean AUROC; hatched = < 3 seeds; blank = no run. coverage: 52/90 cells (58%), of which 30 have the full 3-seed grid · best cell 0.719 0.61 0.55 0.49 0.50 0.50 patch 0.57 0.57 0.52 0.50 0.50 0.0 0.72 0.62 0.61 0.50 0.50 0.1 0.60 0.59 0.53 0.57 0.50 0.50 1.0 0.60 0.57 0.56 0.51 0.50 1.1 0.70 0.58 0.54 0.50 0.50 2.0 0.64 0.60 0.59 0.55 0.51 0.50 2.1 0.54 0.52 0.52 0.49 0.50 3.0 0.60 0.53 0.54 0.51 0.51 0.50 0.50 3.1 0.52 0.50 0.50 encoder* M. workld. SSVEP M. arith. ERN M. imag. P300 M. exec. N2pc c-VEP
CBraMod probe layer × task. Cell = mean AUROC; hatched = < 3 seeds; blank = no run. coverage: 99/99 cells (100%), of which 99 have the full 3-seed grid · best cell 0.948 0.91 0.85 0.76 0.52 0.69 0.64 0.65 0.63 0.53 rearr 0.79 0.72 0.64 0.73 0.61 0.63 0.59 0.60 0.52 patch 0.91 0.81 0.75 0.70 0.66 0.65 0.64 0.63 0.52 enc.0 0.90 0.82 0.77 0.69 0.68 0.69 0.65 0.64 0.53 enc.2 0.94 0.83 0.69 0.70 0.66 0.69 0.65 0.64 0.53 enc.3 0.94 0.84 0.77 0.71 0.67 0.67 0.65 0.61 0.49 enc.5 0.93 0.83 0.77 0.71 0.67 0.71 0.65 0.64 0.53 enc.6 0.90 0.81 0.75 0.73 0.65 0.67 0.64 0.60 0.53 enc.8 0.81 0.72 0.70 0.67 0.64 0.59 0.59 0.58 0.52 enc.9 0.90 0.76 0.67 0.62 0.64 0.61 0.62 0.57 0.52 enc.11 0.95 0.78 0.68 0.57 0.60 0.57 0.63 0.52 0.52 null SSVEP ERN P300 M. workld. M. arith. M. imag. N2pc M. exec. c-VEP
LaBraM probe layer × task. Cell = mean AUROC; hatched = < 3 seeds; blank = no run. coverage: 66/66 cells (100%), of which 66 have the full 3-seed grid · best cell 0.873 0.64 0.59 0.63 0.58 0.51 0.50 patch 0.63 0.57 0.65 0.54 0.50 0.51 blk.0 0.77 0.63 0.65 0.54 0.54 0.52 blk.1 0.71 0.70 0.66 0.57 0.54 0.54 blk.3 0.68 0.65 0.68 0.58 0.56 0.55 blk.4 0.70 0.66 0.69 0.59 0.54 0.51 blk.6 0.79 0.69 0.66 0.60 0.53 0.54 blk.7 0.72 0.63 0.68 0.54 0.55 0.53 blk.8 0.79 0.69 0.64 0.54 0.56 0.54 blk.10 0.87 0.79 0.67 0.56 0.63 0.58 blk.11 0.81 0.76 0.64 0.57 0.65 0.59 null SSVEP ERN P300 c-VEP M. imag. N2pc
LUNA probe layer × task. Cell = mean AUROC; hatched = < 3 seeds; blank = no run. coverage: 99/99 cells (100%), of which 99 have the full 3-seed grid · best cell 0.879 0.50 0.50 0.50 0.50 0.50 0.50 0.50 0.50 0.50 ch-emb 0.88 0.75 0.83 0.69 0.63 0.70 0.61 0.63 0.52 patch 0.76 0.75 0.63 0.62 0.62 0.61 0.58 0.56 0.50 blk.0 0.76 0.75 0.62 0.62 0.63 0.61 0.58 0.56 0.52 blk.1 0.75 0.74 0.61 0.62 0.62 0.62 0.58 0.56 0.52 blk.2 0.75 0.74 0.61 0.62 0.62 0.61 0.58 0.56 0.52 blk.3 0.75 0.74 0.61 0.62 0.62 0.61 0.58 0.57 0.52 blk.4 0.76 0.74 0.60 0.61 0.62 0.61 0.57 0.56 0.52 blk.5 0.74 0.73 0.60 0.61 0.62 0.61 0.57 0.56 0.52 blk.6 0.74 0.73 0.60 0.61 0.63 0.61 0.56 0.55 0.52 blk.7 0.72 0.72 0.59 0.60 0.63 0.62 0.57 0.53 0.53 null ERN P300 SSVEP M. workld. M. arith. M. imag. M. exec. N2pc c-VEP
REVE probe layer × task. Cell = mean AUROC; hatched = < 3 seeds; blank = no run. coverage: 90/90 cells (100%), of which 79 have the full 3-seed grid · best cell 0.965 0.94 0.73 0.52 0.63 0.73 0.62 0.57 0.63 0.51 patch 0.94 0.85 0.76 0.64 0.75 0.65 0.60 0.64 0.51 trf.0 0.96 0.83 0.80 0.67 0.75 0.67 0.57 0.63 0.50 trf.3 0.97 0.87 0.80 0.76 0.76 0.68 0.66 0.64 0.52 trf.7 0.91 0.81 0.80 0.80 0.68 0.69 0.69 0.58 0.52 trf.11 0.95 0.86 0.79 0.81 0.74 0.74 0.70 0.62 0.50 trf.15 0.86 0.75 0.77 0.69 0.67 0.68 0.60 0.57 0.52 trf.21 0.50 0.50 0.50 0.50 0.50 0.50 0.50 0.50 0.50 mlp4d 0.50 0.50 0.50 0.50 0.50 0.50 0.50 0.50 0.50 ln 0.94 0.83 0.75 0.79 0.69 0.68 0.66 0.62 0.50 null SSVEP ERN M. workld. M. imag. P300 M. arith. M. exec. N2pc c-VEP