Exploiting Spatial Attention and Contextual Information for Document Image Segmentation

Yuman Sang; Yifeng Zeng; Ruiying Liu; Fan Yang; Zhangrui Yao; Yinghui Pan

doi:10.1007/978-3-031-05981-0_21

Exploiting Spatial Attention and Contextual Information for Document Image Segmentation

Yuman Sang, Yifeng Zeng^*, Ruiying Liu, Fan Yang, Zhangrui Yao, Yinghui Pan

^*Corresponding author for this work

Computer Information Sciences Department

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Citations (Scopus)

Abstract

We propose a new framework of combining an attention mechanism with a conditional random field to deal with a document image segmentation task. The framework aims to recognize homogeneous regions, e.g. text, figures, or tables, in document images through a pixel-wise spatial attention module. The attention module obtains essential global information and gathers long-distance pixel dependencies. To get extra knowledge around images, we use a conditional random field to model contextual information in the document. The new framework enables an effective combination of pixel features with their contextual information in the document image segmentation task. We conduct extensive experiments over multiple challenging datasets and demonstrate the performance of our new framework in comparison to a series of state-of-the-art segmentation methods.

Original language	English
Title of host publication	Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings
Editors	João Gama, Tianrui Li, Yang Yu, Enhong Chen, Yu Zheng, Fei Teng
Publisher	Springer
Pages	261-274
Number of pages	14
ISBN (Print)	9783031059803
DOIs	https://doi.org/10.1007/978-3-031-05981-0_21
Publication status	Published - 2022
Event	26th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2022 - Chengdu, China Duration: 16 May 2022 → 19 May 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13282 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	26th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2022
Country/Territory	China
City	Chengdu
Period	16/05/22 → 19/05/22

Keywords

Conditional random field
Document image segmentation
Spatial attention mechanism

Access to Document

10.1007/978-3-031-05981-0_21

Cite this

Sang, Y., Zeng, Y., Liu, R., Yang, F., Yao, Z., & Pan, Y. (2022). Exploiting Spatial Attention and Contextual Information for Document Image Segmentation. In J. Gama, T. Li, Y. Yu, E. Chen, Y. Zheng, & F. Teng (Eds.), Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings (pp. 261-274). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13282 LNAI). Springer. https://doi.org/10.1007/978-3-031-05981-0_21

Sang, Yuman ; Zeng, Yifeng ; Liu, Ruiying et al. / Exploiting Spatial Attention and Contextual Information for Document Image Segmentation. Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings. editor / João Gama ; Tianrui Li ; Yang Yu ; Enhong Chen ; Yu Zheng ; Fei Teng. Springer, 2022. pp. 261-274 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{842ea836d8a540a2952228b193c24c10,

title = "Exploiting Spatial Attention and Contextual Information for Document Image Segmentation",

abstract = "We propose a new framework of combining an attention mechanism with a conditional random field to deal with a document image segmentation task. The framework aims to recognize homogeneous regions, e.g. text, figures, or tables, in document images through a pixel-wise spatial attention module. The attention module obtains essential global information and gathers long-distance pixel dependencies. To get extra knowledge around images, we use a conditional random field to model contextual information in the document. The new framework enables an effective combination of pixel features with their contextual information in the document image segmentation task. We conduct extensive experiments over multiple challenging datasets and demonstrate the performance of our new framework in comparison to a series of state-of-the-art segmentation methods.",

keywords = "Conditional random field, Document image segmentation, Spatial attention mechanism",

author = "Yuman Sang and Yifeng Zeng and Ruiying Liu and Fan Yang and Zhangrui Yao and Yinghui Pan",

note = "Funding Information: Supported by the NSF in China (NSF: 62176225 and 61836005).; 26th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2022 ; Conference date: 16-05-2022 Through 19-05-2022",

year = "2022",

doi = "10.1007/978-3-031-05981-0_21",

language = "English",

isbn = "9783031059803",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "261--274",

editor = "Jo{\~a}o Gama and Tianrui Li and Yang Yu and Enhong Chen and Yu Zheng and Fei Teng",

booktitle = "Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings",

address = "Germany",

}

Sang, Y, Zeng, Y, Liu, R, Yang, F, Yao, Z & Pan, Y 2022, Exploiting Spatial Attention and Contextual Information for Document Image Segmentation. in J Gama, T Li, Y Yu, E Chen, Y Zheng & F Teng (eds), Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13282 LNAI, Springer, pp. 261-274, 26th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2022, Chengdu, China, 16/05/22. https://doi.org/10.1007/978-3-031-05981-0_21

Exploiting Spatial Attention and Contextual Information for Document Image Segmentation. / Sang, Yuman; Zeng, Yifeng; Liu, Ruiying et al.
Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings. ed. / João Gama; Tianrui Li; Yang Yu; Enhong Chen; Yu Zheng; Fei Teng. Springer, 2022. p. 261-274 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13282 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Exploiting Spatial Attention and Contextual Information for Document Image Segmentation

AU - Sang, Yuman

AU - Zeng, Yifeng

AU - Liu, Ruiying

AU - Yang, Fan

AU - Yao, Zhangrui

AU - Pan, Yinghui

N1 - Funding Information: Supported by the NSF in China (NSF: 62176225 and 61836005).

PY - 2022

Y1 - 2022

N2 - We propose a new framework of combining an attention mechanism with a conditional random field to deal with a document image segmentation task. The framework aims to recognize homogeneous regions, e.g. text, figures, or tables, in document images through a pixel-wise spatial attention module. The attention module obtains essential global information and gathers long-distance pixel dependencies. To get extra knowledge around images, we use a conditional random field to model contextual information in the document. The new framework enables an effective combination of pixel features with their contextual information in the document image segmentation task. We conduct extensive experiments over multiple challenging datasets and demonstrate the performance of our new framework in comparison to a series of state-of-the-art segmentation methods.

AB - We propose a new framework of combining an attention mechanism with a conditional random field to deal with a document image segmentation task. The framework aims to recognize homogeneous regions, e.g. text, figures, or tables, in document images through a pixel-wise spatial attention module. The attention module obtains essential global information and gathers long-distance pixel dependencies. To get extra knowledge around images, we use a conditional random field to model contextual information in the document. The new framework enables an effective combination of pixel features with their contextual information in the document image segmentation task. We conduct extensive experiments over multiple challenging datasets and demonstrate the performance of our new framework in comparison to a series of state-of-the-art segmentation methods.

KW - Conditional random field

KW - Document image segmentation

KW - Spatial attention mechanism

UR - http://www.scopus.com/inward/record.url?scp=85130236179&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-05981-0_21

DO - 10.1007/978-3-031-05981-0_21

M3 - Conference contribution

AN - SCOPUS:85130236179

SN - 9783031059803

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 261

EP - 274

BT - Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings

A2 - Gama, João

A2 - Li, Tianrui

A2 - Yu, Yang

A2 - Chen, Enhong

A2 - Zheng, Yu

A2 - Teng, Fei

PB - Springer

T2 - 26th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2022

Y2 - 16 May 2022 through 19 May 2022

ER -

Sang Y, Zeng Y, Liu R, Yang F, Yao Z, Pan Y. Exploiting Spatial Attention and Contextual Information for Document Image Segmentation. In Gama J, Li T, Yu Y, Chen E, Zheng Y, Teng F, editors, Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings. Springer. 2022. p. 261-274. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-05981-0_21