video699.screen.semantic_segmentation.fastai_detector

This module implements automatic detection and localization of projector screens on video using semantic segmentation U-Net architecture implemented with FastAI library and post-processed into ConvexQuadrangles

Module Contents

video699.screen.semantic_segmentation.fastai_detector.LOGGER

video699.screen.semantic_segmentation.fastai_detector.ALL_VIDEOS

video699.screen.semantic_segmentation.fastai_detector.CONFIGURATION

video699.screen.semantic_segmentation.fastai_detector.VIDEOS_ROOT

video699.screen.semantic_segmentation.fastai_detector.DEFAULT_VIDEO_PATH

video699.screen.semantic_segmentation.fastai_detector.DEFAULT_LABELS_PATH

video699.screen.semantic_segmentation.fastai_detector.DEFAULT_MODEL_PATH

video699.screen.semantic_segmentation.fastai_detector.DEFAULT_DEVICE

class video699.screen.semantic_segmentation.fastai_detector.SegLabelListCustom

Bases: fastai.vision.SegmentationLabelList

Semantic segmentation custom label list that opens labels in binary mode. It is inherited from fastai class with RGB images.

open(self, fn)

class video699.screen.semantic_segmentation.fastai_detector.SegItemListCustom

Bases: fastai.vision.SegmentationItemList

Semantic segmentation custom item list with binary labels.

_label_cls

class video699.screen.semantic_segmentation.fastai_detector.FastAIScreenDetectorVideoScreen(frame, screen_index, coordinates)

Bases: video699.interface.ScreenABC

A projection screen shown in a frame, detected by :class: FastAIScreenDetector.

Parameters:

• frame (FrameABC) – A video frame containing the projection screen.
• coordinates (ConvexQuadrangleABC) – A map between frame and screen coordinates.
• screen_index (int) – An index of screen in the frame.

frame

FrameABC – A video frame containing the projection screen.

coordinates

ConvexQuadrangleABC – A map between frame and screen coordinates.

screen_index

int – An index of screen in the frame.

frame

coordinates

class video699.screen.semantic_segmentation.fastai_detector.FastAIScreenDetector(filtered_by: Callable = lambda fname: 'frame' in str(fname), valid_func: Callable = None, progressbar: bool = True, train=True)

Bases: video699.interface.ScreenDetectorABC

A projection screen shown in a frame, detected by :class: FastAIScreenDetector.

Parameters:

• filtered_by (function) – A function used for filtering frames inside training videos.
• valid_func (function) – A function used for splitting frames into validation and training dataset.
• progressbar (bool) – A flag to turn on fastai training progressbar.

model_path

Path – A path to the model to load and save.

labels_path

Path – A path to the labels to generate and load.

videos_path

Path – A path to the dataset videos.

post_processing_params

dict – A methods used for post_processing.

train_params

dict – A params used for training a model.

filtered_by

function – A function used for filtering frames inside training videos.

valid_func

function – A function used for splitting frames into validation and training dataset.

device

string – A device used for training : ‘cpu’ or ‘cuda’

progressbar

bool – A flag to turn on fastai training progressbar.

train

bool – A flag to try the train the network if cannot be loaded by default where it is initialized. When off, initialization does not train but only load a model.

src_shape

tuple – A tuple consisting of height and width respectively.

is_fitted

bool – A flag to check is model is fitted already.

self.learner

Learner – A fastai model.

init_model(self)

Initialize learner with parameters set in constructor.

update_params(self, **kwargs)

train(self, **kwargs)

Train self.learner model.

detect(self, frame: FrameABC, **kwargs)

A screen detection: semantic segmentation and post-processing parts of algorithm merged in one function. :param frame: A frame from a video. :type frame: FrameABC :param kwargs: keyword parameters to rewrite default post-processing parameters. :type kwargs: dict

Returns:screens – A screens detected by FastAIScreenDetector. Return type:array-like

semantic_segmentation(self, frame: FrameABC)

Semantic segmentation part of detecting the screens from frame :param frame: A single frame from a video. :type frame: FrameABC

Returns:pred – A prediction from semantic segmentation for single frame. Return type:np.array

post_processing(self, pred, frame: FrameABC, **kwargs)

A post-processing part of screen detection algorithm. :param pred: A prediction from semantic segmentation. :type pred: np.array :param frame: A frame from a video. :type frame: FrameABC :param kwargs: Keyword arguments of different post-processing method. Not specified methods use default value. Optional :type kwargs: dict

Returns:screens – The detecteed screens in left-sorted order for single frame. Return type:array_like[FastAIScreenDetectorVideoScreen]

save(self, model_path: PathLike = None, chunk_size: int = 10000000)

Save the model into directory divided to multiple files. :param model_path: A path to the model to load and save. Default set in __init__. :type model_path: Path :param chunk_size: A size of chunks we divided model before saving. :type chunk_size: int

load(self, model_path: PathLike = None)

Load model from previously saved chunks. :param model_path: A path to the model to load and save. Default set in __init__. :type model_path: Path

semantic_segmentation_batch(self, frames: List[FrameABC])

Semantic segmentation part of detecting the screens from multiple frames. :param frames: The frames from a video. :type frames: array-like

Returns:preds – The semantic segmentation predictions for multiple frames. Return type:np.array

post_processing_batch(self, preds, frames: List[FrameABC], **kwargs)

A post-processing part of screen detection algorithm for multiple predictions. :param preds: The predictions from semantic segmentation for multiple frames. :type preds: np.array :param frames: The frames from a video. :type frames: np.array

Returns:screens – The detected screens in left-sorted order for single frame for multiple predictions. Return type:array_like

detect_batch(self, frames: List[FrameABC], **kwargs)

A screen detection: semantic segmentation and post-processing parts of algorithm merged in one function for multiple frames.

Parameters:

• frames (FrameABC) – The frames from a video.
• methods (dict) – A dictionary of different post-processing method. Not specified methods use default value. Optional

Returns:

screens – Screens detected by FastAIScreenDetector for multiple frames.

Return type:

array-like

delete(self)

Delete saved model.