Evaluating LaMa for AI-Powered Image Inpainting in Sairone

Introduction

Image inpainting is the process of reconstructing missing or unwanted regions of an image while preserving visual consistency with the surrounding content. It is useful in a wide range of computer vision applications, including removing unwanted objects, cleaning image artifacts, restoring damaged regions, and preparing images for further analysis.

As part of the development of Fraime, we evaluated and integrated a LaMa-based image inpainting pipeline to provide automated object and region removal. The implementation uses IOPaint, an open-source image inpainting framework that provides an interface for several image editing models, including LaMa.

The objective of this implementation was not to generate new visual content from textual prompts, but to reconstruct masked regions using the visual information available in the surrounding image. This makes LaMa particularly suitable for cases where the unwanted content should be removed while maintaining the original appearance of the scene.

LaMa

LaMa (Large Mask Inpainting) is an image inpainting approach designed to reconstruct missing image regions, including relatively large masked areas. Unlike generative image-editing approaches that rely heavily on semantic prompts, LaMa performs reconstruction based primarily on the surrounding visual context.

The model is particularly useful for removing objects, defects, and other unwanted regions where the background can be inferred from the remaining image.

For the Sairone implementation, LaMa is accessed through the IOPaint framework. IOPaint provides a unified interface for image inpainting and supports both interactive and batch processing workflows. Its LaMa implementation loads a pre-trained TorchScript model and applies it to an input image together with a binary mask.

IOPaint

IOPaint is an open-source image inpainting and outpainting framework developed around multiple image editing models. In addition to LaMa, it provides support for other model families and utilities such as segmentation and super-resolution.

For the Sairone implementation, we focused specifically on the LaMa component rather than using the complete IOPaint interface.

The framework exposes LaMa as an erase/inpainting model. Its implementation performs the following main operations:

1. Loads the pre-trained LaMa model.

2. Converts the input image into the expected normalized representation.

3. Converts the supplied mask into a binary mask.

4. Runs the image and mask through the LaMa network.

5. Converts the generated tensor back into an image.

6. Returns the reconstructed image.

The implementation also uses a padding configuration based on a multiple of eight, matching the spatial requirements of the underlying model.

Sairone Integration

The main purpose of integrating LaMa into Sairone was to provide an automated inpainting component that could operate as part of an existing computer vision pipeline.

Instead of exposing the complete IOPaint graphical interface, the relevant model functionality can be incorporated into a processing pipeline where both the image and the region to be removed are already available.

The resulting workflow can be represented as:

Input Image → Region/Mask Detection → Mask Preparation → LaMa Inpainting → Output Image

The separation between mask generation and inpainting is particularly useful for Sairone. The inpainting model does not need to determine which object should be removed. A preceding computer vision component can generate the mask, and LaMa can then focus exclusively on reconstructing the masked region.

This allows the same inpainting component to be used with different detection and segmentation methods.

Input and Mask

The primary inputs to the inpainting process are:

• An RGB image

• A binary mask defining the region to be reconstructed

The mask determines which pixels are considered missing or unwanted.

Pixels outside the mask remain part of the visual context used by the model, while pixels inside the mask are reconstructed.

This makes mask quality an important part of the overall pipeline. An inaccurate mask can directly affect the resulting image, even when the inpainting model itself performs correctly.

For this reason, mask generation should be treated as a separate stage of the Sairone workflow rather than as an implicit part of the inpainting model.

Processing Pipeline

The implemented pipeline consists of several stages.

1. Image Preparation

The input image is converted into the format expected by the inpainting model.

IOPaint's LaMa implementation normalizes the image before converting it into a PyTorch tensor. The image is then passed to the selected computation device, such as CPU or GPU.

2. Mask Preparation

The supplied mask is normalized and converted into a binary representation.

The implementation treats pixels greater than zero as part of the masked region:

mask = (mask > 0) * 1

This provides a simple interface between upstream segmentation/detection components and the inpainting model.

3. Model Inference

The prepared image and mask are passed to the LaMa model:

inpainted_image = self.model(image, mask)

The model reconstructs the masked region using the surrounding image context.

4. Output Conversion

The resulting tensor is converted back into an image representation and the values are mapped to the standard 8-bit image range.

IOPaint subsequently converts the resulting RGB image into BGR representation for compatibility with its image-processing pipeline.

Why LaMa for Sairone?

One of the main advantages of LaMa for Sairone is that it provides an inpainting solution without requiring a text prompt or an object-generation workflow.

For object removal, the desired operation is often straightforward: remove this region and reconstruct what should be behind it.

Our product, Fraime enables users to create polygonal masks that can cover multiple parts of an image, offering plenty of flexibility. LaMa is well suited to this type of task.

It is also possible to run IOPaint in a self-hosted environment and use GPU acceleration through PyTorch, making it suitable for integration into an existing computer vision infrastructure. The original project documents CPU, GPU, and Apple Silicon support and provides both interactive and batch-processing interfaces.

Another useful property is the separation between the mask and the inpainting model. This allows Sairone to combine LaMa with other computer vision systems rather than requiring the inpainting system to perform detection itself.

Batch Processing

IOPaint also supports batch processing, where input images and their corresponding masks can be supplied as directories. This is useful when inpainting needs to be applied to a large collection of images rather than a single interactive image.

This makes the approach suitable for automated Sairone processing services where images are processed without manual interaction.

Results

The integration was evaluated on images containing unwanted regions that needed to be removed while maintaining the visual structure of the surrounding scene.

The results show that LaMa can reconstruct masked regions while maintaining continuity with neighboring textures and structures. Its effectiveness is particularly visible when the region being removed is surrounded by sufficient contextual information.

For relatively simple backgrounds, the reconstruction can appear seamless. More complex scenes, highly structured objects, or regions containing significant semantic information can present more challenging cases.

The quality of the final result therefore depends on both the capabilities of the model and the location and shape of the supplied mask.

Limitations

Although LaMa provides an effective solution for image inpainting, several limitations should be considered when integrating it into production pipelines.

Mask Quality

The inpainting model depends directly on the supplied mask. Poorly localized masks can result in unnecessary image modification or incomplete object removal.

Complex Structures

Regions containing complex geometry, text, repeated patterns, or important semantic structures can be more difficult to reconstruct accurately.

Large Missing Regions

As the masked area becomes larger, the amount of information available from the surrounding image decreases. Consequently, reconstruction becomes increasingly dependent on the model's ability to infer plausible content.

Resolution and Computational Requirements

Processing large images increases memory consumption and inference time. Production pipelines should therefore consider image resolution, available GPU memory, and whether images need to be resized or processed in sections.

Conclusion

LaMa provides a practical approach to automated image inpainting for Sairone. Through IOPaint, the pre-trained model can be incorporated into an existing computer vision workflow using an image and a corresponding mask as the primary inputs.

The main strength of this approach is its simplicity: an upstream component determines the region that should be removed, while LaMa reconstructs the corresponding image content.

This makes the technology useful for Sairone applications involving object removal, image cleanup, defect removal, and other workflows where preserving the surrounding visual context is more important than generating entirely new image content.

The implementation also provides a foundation for combining inpainting with Sairone's other computer vision capabilities, allowing object detection, segmentation, and image reconstruction to operate as separate but complementary stages of a larger processing pipeline.

References

[1] Fraime inpainting service: https://saiwa.ai/app/base-service/inpainting

[2] Sanster. IOPaint: Image Inpainting Tool Powered by AI Models. GitHub.
GitHub — IOPaint Repository

[3] Suvorov, R., Logacheva, E., Mashikhin, A., Remizova, A., Ashukha, A., Silvestrov, A., Kong, N., Goka, H., Park, K., & Lempitsky, V. LaMa: Resolution-robust Large Mask Inpainting with Fourier Convolutions. WACV 2022.
GitHub — Official LaMa Implementation