Boosting Semantic Segmentation by Conditioning the Backbone with Semantic Boundaries

Haruya Ishikawa, Yoshimitsu Aoki

研究成果: Article査読


In this paper, we propose the Semantic-Boundary-Conditioned Backbone (SBCB) framework, an effective approach to enhancing semantic segmentation performance, particularly around mask boundaries, while maintaining compatibility with various segmentation architectures. Our objective is to improve existing models by leveraging semantic boundary information as an auxiliary task. The SBCB framework incorporates a complementary semantic boundary detection (SBD) task with a multi-task learning approach. It enhances the segmentation backbone without introducing additional parameters during inference or relying on independent post-processing modules. The SBD head utilizes multi-scale features from the backbone, learning low-level features in early stages and understanding high-level semantics in later stages. This complements common semantic segmentation architectures, where features from later stages are used for classification. Extensive evaluations using popular segmentation heads and backbones demonstrate the effectiveness of the SBCB. It leads to an average improvement of (Formula presented.) in IoU and a (Formula presented.) gain in the boundary F-score on the Cityscapes dataset. The SBCB framework also improves over- and under-segmentation characteristics. Furthermore, the SBCB adapts well to customized backbones and emerging vision transformer models, consistently achieving superior performance. In summary, the SBCB framework significantly boosts segmentation performance, especially around boundaries, without introducing complexity to the models. Leveraging the SBD task as an auxiliary objective, our approach demonstrates consistent improvements on various benchmarks, confirming its potential for advancing the field of semantic segmentation.

出版ステータスPublished - 2023 8月

ASJC Scopus subject areas

  • 分析化学
  • 情報システム
  • 原子分子物理学および光学
  • 生化学
  • 器械工学
  • 電子工学および電気工学


「Boosting Semantic Segmentation by Conditioning the Backbone with Semantic Boundaries」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。