VisualPropertiesExtractor Documentation

Table of Contents

• Overview
• Key Features
• Getting Started
• Constructor
• Arguments
• Calibration System
• Output Dataset
• Practice Examples
• License
• Author

Overview

VisualPropertiesExtractor is a comprehensive utility for extracting size, shape, color, and texture features from segmented objects in RGB images using their associated masks. Designed for image-based machine learning tasks such as object tracking, quality inspection, and digital twin modeling, it enables:

• Extraction of object-level visual descriptors
• Group-wise aggregation over time (e.g., drying intervals)
• Optional calibration of color and size metrics

It integrates seamlessly with TensorFlow pipelines and works alongside the companion utility
ImageAndMaskDatasetBuilder
for efficient preprocessing.

Key Features

• Region-based property extraction using scikit-image
• Textural analysis via grey-level co-occurrence matrices (GLCM)
• Conversion from RGB to Lab color space
• Calibration of color (Lab) and size metrics via rational models
• Grouping and aggregation of visual metrics by time intervals
• JSON and CSV output for further analysis or ML ingestion

Getting Started

Prerequisites

• Python 3.10+
• pip ≥ 23.3.1

a. Clone the Repository

git clone https://github.com/anthony-iheonye/visual-properties-extractor.git
cd visual-properties-extractor

b. Set Up Environment

python3 -m venv vpe_venv
source vpe_venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

Constructor

VisualPropertiesExtractor(
    start_image_index: int,
    images_directory: str,
    masks_directory: str,
    image_mask_channels: Tuple[int, int],
    exclude_partial_objects: bool = False,
    cache_visual_attributes: bool = False,
    calibrator: VisualPropertiesCalibrator = None,
    image_group_size: int = 1,
    interval_per_image_group: int = 15,
    initial_group_interval: int = 0,
    save_location: str = None,
    overwrite_existing_record: bool = True,
    realtime_update: bool = False,
    image_cache_directory: Optional[str] = None,
)

Arguments

Class Attributes

Calibration System

When VisualPropertiesExtractor is supplied a VisualPropertiesCalibrator object, it automatically calibrates both color and size features:

• Color Calibration: Converts camera-measured Lab values (L*, a*, b*) to colorimeter-equivalent values using a rational polynomial model.
• Size Calibration: Converts pixel-based measurements (e.g., area, perimeter, equivalent diameter) to metric units like mm² or mm.

Calibration Parameters

The calibrator receives its configuration through a CalibrationParameters object that encapsulates the coefficients for:

• Color calibration (L*, a*, b*)
• Size scaling (area, diameter, perimeter)

Example Parameters:

realtime_parameters = CalibrationParameters(
    l_index={
        'a': -5.04461047576763E+06,
        'b': 5.13323641056063E+05,
        'c': 1.15843677123398E+04,
        'd': -7.50471807730221E+01
    },
    a_index={
        'a': -4.37022874993402E+00,
        'b': 7.66127417312916E-01,
        'c': 5.72699875489546E-03,
        'd': -1.01255846619522E-04
    },
    b_index={
        'a': -1.15950594393709E+01,
        'b': 7.12135024826324E-01,
        'c': -2.46027845221479E-03,
        'd': -2.19862222769195E-05
    },
    size_factor={
        'area': 0.00312,       # pixel^2 to mm^2
        'diameter': 0.05583,   # pixels to mm
        'perimeter': 0.05617   # pixels to mm
    }
)

These parameter objects are passed into VisualPropertiesCalibrator, which in turn is passed to VisualPropertiesExtractor:

calibrator = VisualPropertiesCalibrator(calibration_parameters=realtime_parameters)
extractor = VisualPropertiesExtractor(..., calibrator=calibrator)

This setup was used in a real-world machine learning project to predict the color, size, and texture evolution of green peas during fluidized bed drying. Visual features were extracted from both real-time images captured inside the dryer and offline images sampled externally.

Output Dataset

After running .process_data(), results are stored as:

• comprehensive_visual_properties.json/csv: Full object-wise dataset per image
• mean_visual_properties.json/csv: Mean visual attributes per image
• visual_props_per_time_interval.json/csv: Grouped averages by drying interval (if grouping enabled)

Practice Examples

Explore a real-world use case in the provided Jupyter notebook:

Exercise: Extract Ground Truth Visual Attributes from Segmented Images

Learn how to use VisualPropertiesExtractor to compute calibrated object-level visual features from segmented RGB images of green peas undergoing fluidized bed drying.

The notebook walks you through:

• Real-time and offline image processing

• Calibration using two domain-specific calibrators - realtime_calibrator and offline_calibrator

• Grouping and time-based visual summaries (e.g., 10 images per 15-minute interval)

• Extracting and previewing:

  • Mean visual attributes per image
  • Object-level properties
  • Group-wise aggregated summaries

Example Snippet:

from app.services.visual_attributes import VisualPropertiesExtractor, realtime_calibrator
from app.utils import dirs

# image and masks directories
image_dir = dirs.realtime_image_dir            # 'assets/realtime/images'
mask_dir = dirs.realtime_mask_dir              # 'assets/realtime/masks'

# Location where visual attribute results will be saved.
save_dir = dirs.realtime_visual_attr_dir       # 'assets/realtime/visual_attributes'

online_extractor = VisualPropertiesExtractor(
    start_image_index=1,
    images_directory=image_dir,
    masks_directory=mask_dir,
    image_mask_channels=(3, 1),
    exclude_partial_objects=False,
    cache_visual_attributes=False,
    calibrator=realtime_calibrator,
    image_group_size=3,                       # 3 images per drying interval
    interval_per_image_group=15,
    initial_group_interval=0,
    save_location=save_dir,
    overwrite_existing_record=True,
)

online_extractor.process_data()

For the full example, visit the Jupyter notebook.

License

This project is licensed under the MIT License.

Author

Developed by Anthony Iheonye | LinkedIn