In the previous article, maadaa.ai covered the panoptic segmentation in the autonomous driving field with open datasets. 

Please read: Panoptic Segmentation enables Real-World Computer Vision in Autonomous Driving.

As we know, panoptic segmentation, this annotation type is widely used in various scenario applications, especially in life-saving fields such as medical imaging and helping visually impaired people to provide a holistic understanding of their surroundings.

In this article, we will cover the following topics:

1. What is panoptic segmentation?
2. Panoptic segmentation for medical imaging
3. Panoptic segmentation for the visually impaired
4. maadaa.ai: strengths and experience in healthcare industries
5. The related open and commercial datasets

1. What is panoptic segmentation?

In brief, the task definition is simple: Each pixel of an image must be assigned a semantic label and an instance id. Pixels with the same label and id belong to the same object; for stuff labels, the instance id is ignored.

For more professional and detailed explanations of panoptic segmentation, please read:

Repost | Overview of Panoptic Segmentation — towards Real-World Computer Vision (Part.1)

2. Panoptic segmentation for medical imaging

Using panoptic segmentation for medical images is a crucial application. 

Medical images often have complex environments requiring the model to recognize interesting objects and backgrounds. 

Since the progress of panoptic segmentation, great interest has been paid to using this task in medical images. They employ panoptic segmentation to combine background and foreground knowledge to improve cell segmentation, cancer cell detection, lesion detection, etc.

For example, for radiologists who deal with large volumes of visual data, identifying cancer cells with human eyes is extremely challenging due to factors such as occlusion or saturation.

By using panoptic segmentation, radiologists can easily recognize tumor cells, because this method allows them to separate the background from the foreground. The key role is that both the instances and the amorphous regions help shape the context of the disease.

Furthermore, especially for diagnosing diseases like cancer, it’s often difficult to accurately detect cells that overlap and are diverse during the screening. 

Semantic segmentation models were commonly used but showcased gaps in data and inaccuracies in the case of overlapping cells. Panoptic segmentation, specifically with deep learning, plays a more accurate and efficient role than previous technologies.

For example, [3] proposes to use panoptic segmentation to overcome the high variability of object appearances, numerous overlapping objects, and ambiguous object boundaries in medical images. They introduce a Panoptic Feature Fusion Net (PFFNet) to perform broad applicability on various biomedical and biological datasets, including histopathology images, fluorescence microscopy images, and plant phenotype images.[4] propose a panoptic segmentation model which incorporates an auxiliary semantic segmentation branch with the instance branch to integrate global and local features. The model can incorporate complementary information at both global and local levels. As shown in Fig 4, they apply panoptic segmentation in cancer diagnosis and prognosis, which boosts performance by a large margin. In [137] the issue of segmenting overlapping nuclei was addressed.  A bending loss regularised network for nuclei segmentation was developed. Large curvatures were penalized with high penalties, whereas smaller curvatures were penalized with low penalties and employed as bending loss. This has helped reduce the bending loss and prevent the formation of several nuclei-surrounded outlines.

 Fig4: panoptic segmentation annotation on different organs (a)breast (b)annotation (c) bladder (d)annotation([4]Fig1)

3. Panoptic segmentation for the visually impaired 

[11] propose a new panoptic segmentation system to provide a holistic understanding of the surroundings for the visually impaired. Such as in Fig 8, they use panoptic segmentation to assist the navigation of visually impaired people by efficiently offering both things and stuff awareness in the proximity of the visually impaired.

Fig 8 panoptic segmentation for visually impaired

4. maadaa.ai: Strengths and experience in Healthcare industries

maadaa.ai has joined many AI projects in the healthcare field for several years, accumulating extensive experiences, resources, and a large number of experts in various fields as well as front-line medical practitioners, including clinicians, pharmacists, nurses, healthcare professionals, etc.

maadaa.ai has worked in specific medical fields like cellular data annotation projects, brain segmentation data annotation projects, retinal data annotation projects and other medical-type data annotation projects.

For instance, for cellular data annotation projects, maadaa.ai has the capability for cancer cell body labeling, cell annotation, cancer cell annotation, chromosome labeling, pathology image section annotation and fine-grained multi-classification labeling for multiple disease types, and labeling of experimental animal pathology.

For brain segmentation data annotation projects, maadaa.ai has extensive experience in labeling specific organ sites and correctly labeling brain tumors.

For retinal data annotation projects, maadaa.ai has worked on Diabetic Retinopathy (DR) recognition and diabetic fundus retinal lesion recognition for several years.

Last, but not least, other medical-type data annotation projects, such as Microscopic Images Annotation, Parkinson’s Disease Detection, Facial Skin Detection, OCR labeling of medical documents, medical document attribute labeling, wound annotation, X-Ray labeling, ECG labeling, etc, maadaa.ai has several years of experience working with top hospitals and researchers.

5. The related open datasets

AI is driven by data—not code.

1. COVID-19 X-Ray Dataset (V7)

It is V7’s original dataset containing 6500 images of AP/PA chest X-Rays with pixel-level polygonal lung segmentations. There are 517 cases of COVID-19 amongst these. 

Lung annotations are polygons following pixel-level boundaries. You can export them in COCO, VOC, or Darwin JSON formats. Each annotation file contains a URL to the original full-resolution image and a reduced-size thumbnail.

For more details, check out: https://github.com/v7labs/covid-19-xray-dataset 

2. NIH 

100,000 chest X-rays with diagnoses, labels, and annotations.

For more details, check out: https://nihcc.app.box.com/v/ChestXray-NIHCC 

3. OASIS

The Open Access Series of Imaging Studies (OASIS) is a project aimed at making neuroimaging data sets of the brain freely available to the scientific community.

For more details, check out:  https://www.oasis-brains.org/ 

Reference List：

1. [3] Panoptic Feature Fusion Net: A Novel Instance Segmentation Paradigm for Biomedical and Biological Images
2. [4]Nuclei Segmentation via a Deep Panoptic Model with Semantic Feature Fusion
3. [11] Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impairedhear

Further reading:

 Panoptic Segmentation enables Real-World Computer Vision in Autonomous Driving

 Overview of Panoptic Segmentation — towards Real-World Computer Vision (Part.1)

 Overview of Panoptic Segmentation — towards Real-World Computer Vision (Part.2)

 Overview of Panoptic Segmentation — towards Real-World Computer Vision (Part.3)