deepmreye package

Subpackages

• deepmreye.util package
  • Submodules
  • deepmreye.util.data_generator module
    • create_cv_generators()
    • create_generators()
    • create_holdout_generators()
    • create_leaveoneout_generators()
    • data_generator()
    • get_all_subject_data()
    • get_nonan_indices()
    • get_single_data_generators()
    • get_start_end_tr()
    • get_subject_data()
    • get_tr_indices()
  • deepmreye.util.data_io module
    • download_mask()
    • get_all_subject_labels()
    • get_all_subject_labels_bmd()
    • get_all_subject_labels_ign()
    • get_all_subject_labels_mmd()
    • get_subject_labels()
  • deepmreye.util.model_opts module
    • get_opts()
  • deepmreye.util.util module
    • Arg
    • angle_between_points()
    • augment_input()
    • calculate_scores()
    • color
      • color.BLUE
      • color.BOLD
      • color.CYAN
      • color.DARKCYAN
      • color.END
      • color.GREEN
      • color.PURPLE
      • color.RED
      • color.UNDERLINE
      • color.YELLOW
    • euclidean_distance()
    • get_model_scores()
    • mish()
    • quantify_predictions()
    • smooth_signal()
    • step_decay_schedule()
  • Module contents

Submodules

deepmreye.analyse module

deepmreye.analyse.visualise_input_data(X, y, color='rgb(0, 150, 175)', cut_at=151, bg_color='rgb(247,247,247)', ylim=None, num_functionals=78)

deepmreye.analyse.visualise_predictions_click(evaluation, scores, color='rgb(0, 150, 175)', bg_color='rgb(247,247,247)')

deepmreye.analyse.visualise_predictions_slider(evaluation, scores, color='rgb(0, 150, 175)', bg_color='rgb(247,247,247)', line_color='rgb(240,240,240)', ylim=None, subTR=False)

deepmreye.architecture module

class deepmreye.architecture.GroupNormalization(*args, **kwargs)

Bases: Layer

Group normalization layer.

Group Normalization divides the channels into groups and computes within each group the mean and variance for normalization. GN’s computation is independent of batch sizes, and its accuracy is stable in a wide range of batch sizes

Parameters

groups: Integer

The number of groups for Group Normalization.

axis: Integer

the axis that should be normalized (typically the features axis). For instance, after a Conv2D layer with data_format="channels_first", set axis=1 in BatchNormalization.

epsilon:

Small float added to variance to avoid dividing by zero.

center:

If True, add offset of beta to normalized tensor. If False, beta is ignored.

scale:

If True, multiply by gamma. If False, gamma is not used. When the next layer is linear (also e.g. nn.relu), this can be disabled since the scaling will be done by the next layer.

beta_initializer:

Initializer for the beta weight.

gamma_initializer:

Initializer for the gamma weight.

beta_regularizer:

Optional regularizer for the beta weight.

gamma_regularizer:

Optional regularizer for the gamma weight.

beta_constraint:

Optional constraint for the beta weight.

gamma_constraint:

Optional constraint for the gamma weight.

Input shape

Arbitrary. Use the keyword argument input_shape (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.

Output shape

Same shape as input.

References

• Group Normalization

build(input_shape)

Creates the variables of the layer (for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call. It is invoked automatically before the first execution of call().

This is typically used to create the weights of Layer subclasses (at the discretion of the subclass implementer).

Args:

input_shape: Instance of TensorShape, or list of instances of

TensorShape if the layer expects a list of inputs (one instance per input).

call(inputs, **kwargs)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state, including tf.Variable instances and nested Layer instances,

in __init__(), or in the build() method that is

called automatically before call() executes for the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

compute_output_shape(input_shape)

Computes the output shape of the layer.

This method will cause the layer’s state to be built, if that has not happened before. This requires that the layer will later be used with inputs that match the input shape provided here.

Args:

input_shape: Shape tuple (tuple of integers) or tf.TensorShape,

or structure of shape tuples / tf.TensorShape instances (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

A tf.TensorShape instance or structure of tf.TensorShape instances.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Note that get_config() does not guarantee to return a fresh copy of dict every time it is called. The callers should make a copy of the returned dict if they want to modify it.

Returns:

Python dictionary.

deepmreye.architecture.compute_standard_loss(out_confidence, real_reg, pred_reg)

deepmreye.architecture.confidence_block(input_layer, num_fc, activation, dropout_rate, inner_timesteps, mc_dropout)

deepmreye.architecture.conv3d_block(input_layer, filters, kernel_size, strides, activation)

deepmreye.architecture.create_standard_model(input_shape, opts)

Create convolutional model for training and inference.

Parameters

input_shapelist

Input shape for each sample (X,Y,Z). Size given by shape of smallest eye mask

optsdict

All model options used for creating and training the model. See util.model_opts for available options

Returns

modelKeras Model

Full model instance, used for training uncertainty estimate

model_inferenceKeras Model

Model instance used for inference, provides uncertainty estimate (unsupervised model)

deepmreye.architecture.downsample_block(input_layer, filters, depth, multiplier, groups, activation)

deepmreye.architecture.euclidean_distance(y_true, y_pred)

deepmreye.architecture.mean_squared_error(y_true, y_pred)

deepmreye.architecture.regression_block(input_layer, num_dense, num_fc, activation, dropout_rate, inner_timesteps, mc_dropout, dense_out=2)

deepmreye.architecture.res_block(input_layer, filters, groups, activation)

deepmreye.architecture.upsampling_block(input_layer, size=2)

deepmreye.preprocess module

deepmreye.preprocess.cut_mask(to_mask, mask, x_edges, y_edges, z_edges, replace_with=0, save_overview=True, fp_func=None, verbose=0)

Cut mask into given shape given edges.

Parameters

to_maskants Image

Image to mask

maskants Image

Mask as numpy array

x_edgeslist

Edges of mask in x-dimension

y_edgeslist

Edges of mask in y-dimension

z_edgeslist

Edges of mask in z-dimension

replace_withint, optional

Values outside of mask are set to this, by default 0

save_overviewbool, optional

Saves report / quality control figure when set to True, by default True

fp_funcstr, optional

Filepath to new functional, by default None

verboseint, optional

Verbosity level of this function, by default 0

Returns

original_inputants Image

Returns to_mask

masked_eyeants Image

masked_eye as numpy array

maskants Image

Return mask

deepmreye.preprocess.get_mask_edges(mask, split=True)

Get edges of mask.

Parameters

fp_maskfilepath, optional

Filepath to mask

splitbool, optional

Splits masks into hemispheres, by default True

Returns

mask:

Array of extracted mask edges

x_edges, y_edges, z_edges:

Edges in (x,y,z)-dimension

deepmreye.preprocess.get_masks(data_path='')

Load masks for whole brain, big eye mask and small eye mask.

Parameters

data_pathstr, optional

Path to where masks are stored in .nii format, by default ‘../deepmreye/masks/’

Returns

eyemask_smallants Image

Eyemask containing voxels within the eye

eyemask_bigants Image

Square eye mask centered on both eyes

dme_templateants Image

Template brain using centered gaze positions

maskants Image

Mask which is used to cut 3D shape for model (in this case the same as eyemask_small)

x_edgeslist

Edges of mask in x-dimension

y_edgeslist

Edges of mask in y-dimension

z_edgeslist

Edges of mask in z-dimension

deepmreye.preprocess.load_label(label_path, label_type='calibration_run')

Load label for experiment, which should return X,Y coordinates for each timepoint.

This function can be exchanged for experiment specific loading of labels, or by using different label types.

Parameters

label_pathstr

Path to file with labels

label_typestr, optional

Which type of labels are used in the experiment, by default ‘calibration_run’

Returns

this_labelnumpy array

X,Y coordinates for each functional describing gaze position during this timepoint.

deepmreye.preprocess.normalize_img(img_in, mad_time=False, standardize_tr=True, std_cut_after=5)

Normalize the 4D input across different dimensions.

Parameters

img_inants Image

Image to normalize

mad_timebool, optional

Determines if median absolute deviation should be used across time dimension, by default False

standardize_trbool, optional

Determines if each image should be normalized across spatial dimensions, by default True

std_cut_afterint, optional

Gets rid of outliers after normalization, by default 5

Returns

img_inants Image

Normalized output image

deepmreye.preprocess.plot_subject_report(fn_subject, original_input, masked_eye, mask, color='rgb(0, 150, 175)', bg_color='rgb(14, 17, 23, 0)')

Plot quality check figure for given subject.

Parameters

fn_subjectstring

Filepath to subject

original_inputants Image

Filepath to functional image of subject

masked_eyearray

Numpy array of masked eye

maskants Image

ants mask

colorstr, optional

Boxplot color, by default “rgb(0, 150, 175)”

bg_colorstr, optional

Background color, by default “rgb(0,0,0)”

deepmreye.preprocess.register_to_eye_masks(dme_template, func, masks, verbose=1, transforms=None, metric='GC')

Register functional to DeepMReye template (dme_template) using different sized masks.

Parameters

dme_templateants Image

Ants image with template file

funcants Image

Functional image to register to dme_template

maskslist

List of Ants image objects containing variable sized masks

verboseint, optional

Verbosity level of function, by default 1

transformsstring, optional

Which transforms should be used to transform image, by default None & set to Similarity

metricstr, optional

Which metric to quantify fit, by default ‘GC’

Returns

funcants Image

Functional image registered to dme_template

transformation_statsarray

Statistics of transformation, used for dataset report

deepmreye.preprocess.run_participant(fp_func, dme_template, eyemask_big, eyemask_small, x_edges, y_edges, z_edges, replace_with=0, transforms=None)

Run preprocessing for one participant with templates and masks preloaded to avoid computational overhead.

Parameters

fp_funcstring

Filepath to participant functional

dme_templateants Image

Preloaded Image to dme_template

eyemask_bigants Image

Big eyemask as ants Image

eyemask_smallants Image

Small eyemask as ants Image

x_edgeslist

Edges of mask in x-dimension

y_edgeslist

Edges of mask in y-dimension

z_edgeslist

Edges of mask in z-dimension

replace_withint, optional

Values outside of mask are set to this, by default 0

deepmreye.preprocess.save_data(participant, participant_data, participant_labels, participant_ids, processed_data, center_labels=False)

Save participant data to npz file for fast (lazy) loading during model training.

Parameters

participantstr

Participant label

participant_datalist

4D (X,Y,Z,t) data for participant across runs

participant_labelslist

3D (t,X,Y) data, with corresponding labels to participant data

participant_idsstr

Participant identifier with run id

processed_datastr

Filepath to where processed data should be stored

center_labelsbool, optional

Centers labels to (0,0) which can improve performance of model, by default False

deepmreye.train module

deepmreye.train.evaluate_model(dataset, model, generators, save=False, model_path='./', model_description='', verbose=0, **args)

Evaluate model performance given model and generators used for training the model.

Evaluate only on test set.

Parameters

datasetstr

Description of dataset, used for saving dataset to file

modelKeras Model

Full model instance, used for training uncertainty estimate

generatorsgenerator

Cross validation, Hold out or Leave one out generator, yielding X,y pairs

savebool, optional

If true, save test set predictions to file, by default False

model_pathstr, optional

Filepath to where model weights should be stored, by default ‘./’

model_descriptionstr, optional

Description of model used for saving the model evaluations, by default ‘’

verboseint, optional

Verbosity level, by default 0

Returns

evaluation: dict

Raw gaze coordinates, returned for each participant

scores: pandas DataFrame

Evaluation metrics for gaze coordinates (Pearson, R2-Score, Euclidean Error)

deepmreye.train.train_model(dataset, generators, opts, clear_graph=True, save=False, model_path='./', workers=4, use_multiprocessing=True, models=None, return_untrained=False, verbose=0)

Train the model given a cross validation, hold out or leave one out generator, given model options.

Parameters

datasetstr

Description of dataset, used for saving dataset to file

generatorsgenerator

Cross validation, Hold out or Leave one out generator, yielding X,y pairs

optsdict

Model options for training

clear_graphbool, optional

If computational graph should be reset before each run, by default True

savebool, optional

If model weights should be saved to file, by default False

model_pathstr, optional

Filepath to where model weights should be stored, by default ‘./’

workersint, optional

Number of workers used when using multiprocessing, by default 4

use_multiprocessingbool, optional

If multiprocessing should be used, can speed up training by 10x if data loader is bottleneck, by default True

modelsKeras Model instance, optional

Can be provided if already trained model should be used instead of training a new one, by default None

return_untrainedbool, optional

If true, returns untrained but compiled model, by default False

verboseint, optional

Verbosity level, by default 0

Returns

modelKeras Model

Full model instance, used for training uncertainty estimate

model_inferenceKeras Model

Model instance used for inference, provides uncertainty estimate (unsupervised model)

Module contents