Gain fields (attention fields)

You have stimulus images on a pixel grid, and on every frame (a TR or a trial) some locations are special: attended, salient, rewarded, expected, where the target or a distractor is. The question is whether the stimulus near those locations drives visual cortex more or less than the same stimulus elsewhere. A gain-field PRF model answers it on the stimulus grid: a field around each special location multiplies the image before every voxel’s ordinary PRF pools it.

\[\begin{split}F_t(g) &= \sum_{\text{fields}} \sum_k \sum_\ell W_{t\ell k}\, \gamma_{k}\, A(g; \ell) \\ M_t(g) &= \max\bigl(1 + F_t(g),\, 0\bigr) \\ r_v(t) &= \text{baseline}_v + \text{amplitude}_v \sum_g S_t(g)\, M_t(g)\, \mathrm{RF}_v(g)\end{split}\]
  • \(W_{t\ell k}\) (in the paradigm): weight of gain type \(k\) at location \(\ell\) on frame \(t\), usually 0 or 1.

  • \(\gamma_k\): one gain per gain type, signed (> 0 enhances, < 0 suppresses). Shared by all voxels.

  • \(A(g;\ell)\): the field shape centred on location \(\ell\), peak 1 (a Gaussian with a shared width sigma by default).

  • \(\mathrm{RF}_v\): the voxel’s receptive field from the model underneath (GaussianPRF2D, DifferenceOfGaussiansPRF2D, …), with the same amplitude convention.

With every gain at 0, the model predicts exactly what the PRF model underneath predicts. So you can fit the PRFs first and then ask how much the gain fields add.

Tutorial: Gain fields: attention modulates population receptive fields simulates a population, fits shared gains with the PRFs fixed, combines two fields and adds a field shape of your own (also Lesson 8: Gain fields: attention modulates PRFs).

Step 1: the paradigm

The paradigm holds the images and the per-frame weights. braincoder.stimuli.make_gain_field_paradigm() (or model.make_paradigm) builds it:

import numpy as np
from braincoder.models import GainFieldPRF2D

# Pixel grid: (n_grid, 2) x and y in degrees.
x = np.linspace(-6, 6, 41)
gx, gy = np.meshgrid(x, x)
grid = np.stack([gx.ravel(), gy.ravel()], 1).astype(np.float32)

# Every location a field can sit on: any number, anywhere.
angles = np.arange(8) * np.pi / 4
field_positions = 4 * np.stack([np.cos(angles), np.sin(angles)], 1)    # 8 items at 4 deg

model = GainFieldPRF2D(grid, field_positions, gain_labels=['target', 'distractor'])
model.parameter_labels
# ['x', 'y', 'sd', 'baseline', 'amplitude',
#  'field_sigma', 'field_gain_target', 'field_gain_distractor']

# images: (n_trials, n_grid) or (n_trials, n_x, n_y), flattened like grid.
# One dict per trial: gain label -> location index, (x, y) tuple,
# list of locations, or {location: weight}.
assignments = [{'target': 2, 'distractor': 5},
               {'target': 0},                       # no distractor on this trial
               ...]
paradigm = model.make_paradigm(images, assignments, index=betas.index)

Columns: ('image', x, y) for every pixel, then ('field', gain_label, location_index). An array W of shape (n_frames, n_locations, n_gains) works instead of the list of dicts, which is handy for fractional weights (e.g. the fraction of a TR a distractor was on). If the special things move freely, pass every position that occurs as field_positions.

Step 2: fit

The field parameters (widths and gains) belong to the model, not to a voxel. The parameters DataFrame repeats them on every row, and they must be equal across rows. Fit them with shared_pars. The robust route is to fit the PRFs first with the model underneath, then fit the shared terms with the PRFs held fixed:

from braincoder.models import GaussianPRF2D
from braincoder.optimize import ParameterFitter

# (a) PRFs from the mapping data, as usual, with GaussianPRF2D.
prf_pars = ...                                   # columns x, y, sd, baseline, amplitude

# (b) Gain fields with the PRFs fixed. Gains start at 0 (= the PRF model).
init = model.add_field_parameters(prf_pars, field_sigma=1.0)
fitter = ParameterFitter(model, betas, paradigm)
pars = fitter.fit(init_pars=init,
                  fixed_pars=model.prf_parameter_labels(),
                  shared_pars=model.field_parameter_labels(include_tied=False),
                  learning_rate=0.02, max_n_iterations=2000, r2_atol=1e-7)
pars.iloc[0][model.field_parameter_labels()]

Leave out fixed_pars to refine PRFs and gains jointly. Check recovery on simulated data at your noise level (model.simulate(paradigm, pars, noise=...)) before trusting a gain: gains and widths trade off when the field is much wider or narrower than the items.

Time courses with an HRF

GainFieldPRF2DWithHRF (and GainFieldDoGPRF2DWithHRF) apply the modulation to the images frame by frame, then the HRF. Several fields can act at once. Below, a sustained field marks the location attended in each block, with a zero-sum constraint (the gain at the attended location and the gain at the 3 others sum to 0, so a uniform change of the whole display is left to the PRFs), and a transient field, which shares its width, responds to brief onsets. The transient field sits outside the clip at 0. A third term scales the drive of the items against a mapping bar, since both are painted as 1 in the images:

from braincoder.hrf import SPMHRFModel
from braincoder.models import GainFieldDoGPRF2DWithHRF, GainField

positions = 4 * np.array([[1, 1], [-1, 1], [-1, -1], [1, -1]]) / np.sqrt(2)
item_mask = np.hypot(grid[:, 0], grid[:, 1]) > item_radius           # your geometry

model = GainFieldDoGPRF2DWithHRF(
    grid, positions, ['attended', 'unattended', 'onset'],
    hrf_model=SPMHRFModel(tr=1.5),
    fields=[GainField('sustained', gain_labels=['attended', 'unattended']),
            GainField('transient', gain_labels=['onset'], clamped=False)],
    image_scales={'item_scale': item_mask},
    ties={'sustained_gain_unattended': {'sustained_gain_attended': -1 / 3},
          'transient_sigma': 'sustained_sigma'})

# W: (n_TRs, 4, 3). Column 0: 1 at the attended location of the block;
# column 1: 1 elsewhere; column 2: fraction of the TR an onset was on.
paradigm = model.make_paradigm(images, W, index=bold.index)

ties makes a parameter a function of others: a number (a constant, e.g. 0. to switch a gain off), another label (equal), or {label: coef} (a linear combination). Tied parameters are filled in by the model, so predictions and fitted values always satisfy the constraint, and you leave them out of shared_pars (field_parameter_labels(include_tied=False)). image_scales adds a positive model-level parameter per mask; its null value is 1.

Extending

Several fields. fields is a list of GainField (name, shape, gain labels, sign constraint, inside or outside the clip). The parameters are named f'{name}_{shape_parameter}' and f'{name}_gain_{label}'.

A new field shape is a subclass of FieldShape with parameter labels, their transforms (unconstrained → native) and one method, written with keras.ops so it runs on every backend:

from keras import ops
from braincoder.models import FieldShape, GainField, GainFieldPRF2D

class AnnulusField(FieldShape):
    parameter_labels = ['radius', 'width']
    transformations = ['softplus', 'softplus']

    def profile(self, dx, dy, radius, width):
        # dx, dy: (1, n_locations, n_grid); parameters: (n_batches, 1, 1)
        r = ops.sqrt(dx ** 2 + dy ** 2 + 1e-12)
        return ops.exp(-(r - radius) ** 2 / (2 * width ** 2))

model = GainFieldPRF2D(grid, positions, ['attended'],
                       fields=[GainField('ring', AnnulusField())])
# parameters ... 'ring_radius', 'ring_width', 'ring_gain_attended'

GaussianField and DoGField (centre minus a wider surround) ship with braincoder.

Another combination rule. Pass combination=f with f(images, field_sums) -> images (one field sum per field, in order), e.g. divisive: lambda S, F: S / (1 + ops.softplus(sum(F))); or subclass and override _combine.

Another PRF kernel. Combine GainFieldMixin with any image PRF model:

from braincoder.models import GainFieldMixin, DivisiveNormalizationGaussianPRF2D

class GainFieldDNPRF2D(GainFieldMixin, DivisiveNormalizationGaussianPRF2D):
    pass

Limitations

  • Field parameters are model-level (one value for all voxels in the parameters DataFrame). Fit different areas as separate models if their fields may differ.

  • Decoding (get_stimulus_pdf) is not meaningful with these models: decode with the PRF model underneath.

  • ParameterFitter has no bounds; constrain gains with GainField(gains='positive' | 'negative') and widths with GaussianField(sigma_min=...).

  • Memory: the modulated images are n_frames x n_grid (shared fields never build a voxels x frames x pixels tensor), so the cost is close to the PRF model underneath.

API

class braincoder.models.GainFieldPRF2D(grid_coordinates=None, field_positions=None, gain_labels=None, fields=None, combination='multiplicative', clamp=True, image_scales=None, ties=None, positive_image_values_only=True, verbosity=20, **kwargs)

Bases: GainFieldMixin, GaussianPRF2D

Gaussian 2D PRF with gain fields. See braincoder.models.gain_field.

One line for the common case (one Gaussian field, signed gains):

model = GainFieldPRF2D(grid_coordinates, field_positions, ['attended', 'ignored'])
model.parameter_labels
# ['x', 'y', 'sd', 'baseline', 'amplitude',
#  'field_sigma', 'field_gain_attended', 'field_gain_ignored']
class braincoder.models.GainFieldMixin(grid_coordinates=None, field_positions=None, gain_labels=None, fields=None, combination='multiplicative', clamp=True, image_scales=None, ties=None, positive_image_values_only=True, verbosity=20, **kwargs)

Adds gain fields to any image PRF model (GaussianPRF2D and subclasses). Put it first in the bases:

class GainFieldDNPRF2D(GainFieldMixin, DivisiveNormalizationGaussianPRF2D):
    pass
Parameters:
  • grid_coordinates (array (n_grid, 2)) – Pixel coordinates, as for GaussianPRF2D.

  • field_positions (array (n_locations, 2)) – Every location a field can be centred on, in grid units. Any number, anywhere (they need not lie on the grid).

  • gain_labels (list of str) – Names of the gain types, e.g. ['attended', 'ignored']. The paradigm says per frame which location carries which label, with which weight.

  • fields (list of GainField, optional) – Default: one Gaussian field named 'field' with signed gains for all labels.

  • combination ('multiplicative' or callable) –

    'multiplicative' (default):

    M = max(1 + sum(clamped fields), 0) + sum(unclamped fields)
    S_eff = S * M
    

    (without clamp the max is dropped). A callable gets (images, field_sums), with field_sums one (n_batches, n_frames, n_grid) tensor per field in fields order, and returns the modulated images. Or override _combine().

  • clamp (bool, default True) – Clip the clamped part of the modulation at 0, so that strong suppression cannot invert the stimulus.

  • image_scales (dict {name: mask}, optional) – Extra model-level parameters that scale part of the image before the fields act: S <- S * (1 + sum_s (scale_s - 1) * mask_s). mask is (n_grid,) (bool or weights). Each adds a positive parameter name (softplus); its null value is 1. Use it when two kinds of stimulus share one image but their relative drive is unknown (e.g. a mapping bar and task items).

  • ties (dict {label: tie}, optional) – Parameters computed from others instead of being free. tie is a number (constant), another label (equal to it), or {label: coef} (linear combination of native values). Examples: a zero-sum pair {'field_gain_b': {'field_gain_a': -1/3}}, two fields with one width {'b_sigma': 'a_sigma'}, a switched-off gain {'field_gain_c': 0.0}. Tied columns ignore their raw value (no gradient); their native value is filled in by the forward transform, so predictions and fitted parameters always satisfy the tie.

  • **kwargs – Passed to the PRF model (e.g. rf_normalization).

add_field_parameters(prf_parameters, **values)

Append the field columns to per-voxel PRF parameters.

Gains default to 0 and image scales to 1 (the null model, which predicts exactly what the PRF model underneath predicts); shape parameters such as widths need a value:

pars = model.add_field_parameters(prf_pars, field_sigma=1.0)
apply_ties(parameters)

Return a copy of a parameter DataFrame with the tied columns filled in.

field_parameter_labels(include_tied=True)

Labels of the model-level parameters (fields, image scales): pass them as shared_pars.

get_modulated_images(paradigm, parameters)

The images as the PRFs see them, S_t(g) * M_t(g): DataFrame frames x pixels.

make_paradigm(images, assignments, index=None)

braincoder.stimuli.make_gain_field_paradigm() for this model.

prf_parameter_labels()

Labels of the per-voxel PRF parameters.

class braincoder.models.GainField(name='field', shape=None, gain_labels=None, gains='signed', clamped=True)

One field: a shape, the gain labels it responds to, its gain sign.

Parameters:
  • name (str) – Prefix of its parameter labels (f'{name}_sigma', …).

  • shape (FieldShape, default GaussianField())

  • gain_labels (list of str or None) – Which of the model’s gain labels this field responds to (None = all). Each gives a parameter f'{name}_gain_{label}'.

  • gains ({'signed', 'positive', 'negative'}) – Constraint on the gains. 'signed' (default) leaves them unbounded, so the data decide between enhancement (> 0) and suppression (< 0). With 'positive'/'negative' do not initialise a gain at exactly 0.

  • clamped (bool, default True) – With the default combination rule and clamp=True, a clamped field sits inside the clip max(1 + F, 0); an unclamped one is added outside it (e.g. a brief transient on top of a sustained field: M = max(1 + F_sustained, 0) + F_transient).

class braincoder.models.FieldShape

Spatial profile of a field, centred on a field location.

A new shape is a subclass with

  • parameter_labels: names of its parameters (e.g. ['sigma']);

  • transformations: one per parameter, 'identity', 'softplus', 'sigmoid' or a (forward, inverse) tuple of keras.ops functions from unconstrained to native values;

  • profile().

Example: an annulus (ring of radius radius and width width):

class AnnulusField(FieldShape):
    parameter_labels = ['radius', 'width']
    transformations = ['softplus', 'softplus']

    def profile(self, dx, dy, radius, width):
        r = ops.sqrt(dx ** 2 + dy ** 2 + 1e-12)
        return ops.exp(-(r - radius) ** 2 / (2 * width ** 2))
profile(dx, dy, **parameters)

Field value at offsets (dx, dy) from the field location.

dx, dy have shape (1, n_locations, n_grid); every parameter has shape (n_batches, 1, 1). Return (n_batches, n_locations, n_grid). Peak 1 keeps gains interpretable as the modulation at the field centre.

class braincoder.models.GaussianField(sigma_min=0.01)

Isotropic Gaussian, peak 1: exp(-d^2 / (2 sigma^2)).

sigma = sigma_min + softplus(raw), in grid units.

class braincoder.models.DoGField(sigma_min=0.01)

Difference of Gaussians (“Mexican hat”): a centre of width sigma minus surround_weight times a surround of width surround_ratio * sigma. Peak 1 - surround_weight.

sigma = sigma_min + softplus, surround_ratio = 1 + softplus (surround wider than centre), surround_weight = sigmoid (0..1).

braincoder.stimuli.make_gain_field_paradigm(images, assignments, field_positions, gain_labels, grid_coordinates=None, index=None, atol=0.0001)

Build the paradigm of a gain-field PRF model.

A gain-field model needs, for every frame (TR or trial), the stimulus image and which field locations carry which gain. This function puts both in one DataFrame.

Parameters:
  • images (array (n_frames, n_grid) or (n_frames, n_x, n_y), or DataFrame) – Stimulus intensity on the model’s pixel grid, flattened in the same order as grid_coordinates.

  • assignments (array (n_frames, n_locations, n_gains), or list of dicts) – The weight W[t, l, k] of gain k at location l on frame t (usually 0/1). As a list (one entry per frame), each entry maps a gain label to a location, a list of locations, or a {location: weight} dict. A location is an int index into field_positions or an (x, y) tuple that must match one of them. Missing labels and None mean weight 0 (no modulation on that frame).

  • field_positions (array (n_locations, 2)) – Every location a field can sit at, in grid units. If the attended or salient things move freely, pass all positions that occur.

  • gain_labels (list of str) – Names of the gain types, e.g. ['attended', 'ignored'].

  • grid_coordinates (array (n_grid, 2), optional) – The model’s pixel coordinates, only used to label the image columns ('image', x, y); without it they are ('image', pixel_index, nan).

  • index (pandas Index, optional) – Row index (defaults to the index of images if it is a DataFrame, else frame = 0..n_frames-1). Give it the index of your data.

Returns:

Columns ('image', x, y) for the pixels, then ('field', gain_label, location_index).

Return type:

DataFrame (n_frames, n_grid + n_gains * n_locations)