Pipeline process class

This class is used to run the entire preprocessing pipeline - from CZI files to a pyramid of tiles that can be viewed in neuroglancer.

Args are animal, self.channel, and downsample. With animal being the only required argument. All imports are listed by the order in which they are used in the

class library.image_manipulation.pipeline_process.Pipeline(animal, channel='C1', downsample=False, scaling_factor=32.0, task='status', debug=False)

Bases: ElastixManager, HistogramMaker, ImageCleaner, MaskManager, MetaUtilities, NgPrecomputedMaker, Normalizer, OmeZarrManager, ParallelManager, PrepCreater, TiffExtractor

This is the main class that handles the preprocessing pipeline responsible for converting Zeiss microscopy images (.czi) into neuroglancer viewable formats. The Zeiss module can be swapped out to make the pipeline compatible with other microscopy setups

TASK_ALIGN = 'Creating alignment process'

TASK_CELL_LABELS = 'Creating centroids for cells'

TASK_CLEAN = 'Applying masks'

TASK_EXTRACT = 'Extracting TIFFs and meta-data'

TASK_EXTRA_CHANNEL = 'Creating separate channel'

TASK_HISTOGRAM = 'Making histogram'

TASK_MASK = 'Creating masks'

TASK_NEUROGLANCER = 'Creating Neuroglancer data'

TASK_OMEZARR = 'Creating multiscaled ome zarr'

TASK_REALIGN = 'Creating alignment improvements'

TASK_SHELL = 'Creating 3D shell outline'

affine_align()

Perform the section to section alignment (registration) This method needs work. It is not currently used.

align()

Perform the section to section alignment (registration)

align_masks()

check_ram()

I set an arbitrary limit of 50GB of RAM for the full resolution images

check_settings()

Make sure there is a ./src/settings.py file and make sure there is enough RAM I set an arbitrary limit of 50GB of RAM for the full resolution images

check_status()

static check_url(animal)

clean()

extra_channel()

This step is in case self.channel X differs from self.channel 1 and came from a different set of CZI files. This step will do everything for the self.channel, so you don’t need to run self.channel X for step 2, or 4. You do need to run step 0 and step 1. TODO fix for channel variable name

extract()

get_section_count()

Retrieves the count of sections for the specified animal. If the section count from the database is zero, it calculates the count based on the number of thumbnail files in the specified directory. I put this back in as test_dir requires it. :returns: int: The total count of sections.

histogram()

mask()

neuroglancer()

This is the main method to run the entire neuroglancer process. We also define the input, output and progress directories. This method may be run from the command line as a task, or it may also be run from the align and realign methods

omezarr()

realign()

Perform the improvement of the section to section alignment. It will use fiducial points to improve the already aligned image stack from thumnbail_aligned. This only needs to be run on downsampled channel 1 images. With the full resolution images, the transformations come from both iterations of the downsampled images and then scaled. While the transformations are only created on channel 1, the realignment needs to occur on all channels

report_status()

shell()