Biomedical images are often referenced for clinical decision support (CDS), educational purposes, and research. The task of automatically finding the images in a scientific article that are most useful for the purpose of determining relevance to a clinical situation is traditionally done using text and is quite challenging. We propose to improve this by associating image features from the entire image and from relevant regions of interest with biomedical concepts described in the figure caption or discussion in the article. However, images used in scientific article figures are often composed of multiple panels where each sub-figure (panel) is referenced in the caption using alphanumeric labels, e.g. Figure 1(a), 2(c), etc. It is necessary to separate individual panels from a multi-panel figure as a first step toward automatic annotation of images. In this work we present methods that add make robust our previous efforts reported here. Specifically, we address the limitation in segmenting figures that do not exhibit explicit inter-panel boundaries, e.g. illustrations, graphs, and charts. We present a novel hybrid clustering algorithm based on particle swarm optimization (PSO) with fuzzy logic controller (FLC) to locate related figure components in such images. Results from our evaluation are very promising with 93.64% panel detection accuracy for regular (non-illustration) figure images and 92.1% accuracy for illustration images. A computational complexity analysis also shows that PSO is an optimal approach with relatively low computation time. The accuracy of separating these two type images is 98.11% and is achieved using decision tree.

Meeting Name

SPIE 7874, Document Recognition and Retrieval XVIII (2011: Jan. 23, San Francisco, CA)


Electrical and Computer Engineering

Keywords and Phrases

Automatic annotation of images; Automatic segmentations; biomedical article retrieval; Biomedical image analysis; Biomedical images; CdS; Clinical decision support; Clinical situations; Computation time; Computational complexity analysis; content-based image retrieval; Detection accuracy; Document Retrieval; figure panels; Fuzzy logic controllers; Hybrid clustering algorithm; Image features; Multi-modal; Regions of interest; Scientific articles; Biomedical engineering; Clustering algorithms; Computational complexity; Content based retrieval; Decision support systems; Decision trees; Fuzzy logic; Image analysis; Information retrieval; Particle swarm optimization (PSO); Image segmentation

International Standard Book Number (ISBN)


International Standard Serial Number (ISSN)


Document Type

Article - Conference proceedings

Document Version

Final Version

File Type





© 2011 SPIE -- The International Society for Optical Engineering, All rights reserved.

Publication Date

01 Jan 2011