Local PatchMatch Based on Superpixel Cut for Efficient High-resolution Stereo Matching

Cheng, Xianjing; Zhao, Yong; Raj, Raja Soosaimarian Peter; Hu, Zhijun; Yu, Xiaomin; Yang, Wenbang

doi:10.1590/1678-4324-202210409

Acessibilidade / Reportar erro

Brasil

Brazilian Archives of Biology and Technology

Español English

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

Article - Engineering, Technology and Techniques • Braz. arch. biol. technol. 65 • 2022 • https://doi.org/10.1590/1678-4324-202210409 copy

Local PatchMatch Based on Superpixel Cut for Efficient High-resolution Stereo Matching

Authorship SCIMAGO INSTITUTIONS RANKINGS

HIGHLIGHTS

A new method of extracting the feature points in superpixels is proposed to compute the candidate disparities using non-local cost aggregation.
A novel two-stage optimization framework for superpixel proposals by first getting the labels of feature points quickly and then consists the candidate label sets for updating the labels of pixels within the corresponding superpixel instead of assigning a label randomly to those pixels.
The weight combination of intensity, gradient and binary image is designed for constructing an optimal minimum spanning tree to compute the aggregated matching cost.

Abstract:

Obtaining the accurate disparity of each pixel quickly is the goal of stereo matching, but it is very difficult for the 3D labels-based methods due to huge search space of 3D labels, especially for highresolution images. We present an novel two-stage optimization strategy to get the accurate disparity map for high-resolution stereo image efficiently, which includes feature points optimization and superpixel optimization. In the first stage, we construct the support points including edge points and robust points for triangulation, which is used to extract feature points and then perform spatial propagation and random refinement to get the candidate 3D label sets. In the stage of superpixel optimization, we update per pixel labels of the corresponding superpixels using the candidate label sets, and then perform spatial propagation and random refinement. In order to provide more prior information to identify weak texture and textureless areas, we design the weight combination of “intensity + gradient + binary image” for constructing an optimal minimum spanning tree (MST) to compute the aggregated matching cost, and obtain the labels of minimum aggregated matching cost. We also design local patch surrounding the corresponding superpixel to accelerate our algorithm in parallel. The experimental result shows that our method achieves a good trade-off between running time and accuracy, including KITTI and Middlebury benchmark, which are the standard benchmarks for testing the stereo matching methods.

Keywords:
feature points; superpixel optimization; prior information; weight combination; stereo matching

Method	Ours	Mesh[²⁸28 Chi Zhang, Zhiwei Li, Yanhua Cheng, Rui Cai, Hongyang Chao, Yong Rui. Meshstereo: A global stereo model with mesh alignment regularization for view interpolation. Proceedings of the IEEE International Conference on Computer Vision. 2015:2057-65. https://doi.org/10.1109/ICCV.2015.238 https://doi.org/10.1109/ICCV.2015.238... ]	AdaStereo [⁴³43 Xiao Song, Guorun Yang, Xinge Zhu, Hui Zhou, Zhe Wang, Jianping Shi. AdaStereo: a simple and efficient approach for adaptive stereo matching. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021:10328-37. https://arxiv.org/abs/2004.04627 https://arxiv.org/abs/2004.04627... ]	AANet++[³⁸38 Xu H, Zhang J. Aanet: Adaptive aggregation network for efficient stereo matching. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020:1959-68. https://doi.org/10.1109/CVPR42600.2020.00203 https://doi.org/10.1109/CVPR42600.2020.0... ]	SGM[¹⁵15 Hirschmuller H. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on pattern analysis and machine intelligence, 2007,30(2):328-41. https://doi.org/10.1109/TPAMI.2007.1166 https://doi.org/10.1109/TPAMI.2007.1166... ]	LPS[³³33 Sinha S N, Scharstein D, Szeliski R. Efficient high-resolution stereo matching using local plane sweeps. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014:1582-9. https://doi.org/10.1.1.648.5914 https://doi.org/10.1.1.648.5914... ]	Deeppruner [⁴²42 Shivam Duggal, Shenlong Wang, Wei-Chiu Ma, Rui Hu, Raquel Urtasun. Deeppruner: Learning efficient stereo matching via differentiable patchmatch. Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019:4384-93. https://doi.org/10.1109/ICCV.2019.00448 https://doi.org/10.1109/ICCV.2019.00448... ]	ELAS[²⁹29 Geiger A, Roser M, Urtasun R. Efficient large-scale stereo matching. Asian conference on computer vision. Springer, Berlin, Heidelberg, 2010:25-38. https://doi.org/10.1007/978-3-642-19315-6_3 https://doi.org/10.1007/978-3-642-19315-... ]
Res	H	H	H	H	H	H	H	F
Avg	10.7	13.2	13.7	15.4	18.4	19.2	30.1	32.3
Austr	5.15	5.90	19.6	17.5	40.3	6.14	34.2	50.9
AustrP	4.23	4.88	7.41	8.37	4.54	5.34	19.9	9.17
Bicyc2	5.48	10.8	10.6	10.2	8.03	9.24	24.3	11.0
Class	6.38	12.9	14.5	9.86	22.9	7.53	23.8	33.0
ClassE	16.5	10.6	15.7	23.9	40.5	96.0	47.2	88.2
Compu	7.84	11.0	7.85	9.82	11.4	12.3	26.1	18.3
Crusa	9.56	12.2	22.6	17.7	24.7	9.61	26.1	47.3
CrusaP	10.3	9.01	9.32	15.9	10.1	9.40	22.8	26.8
DjembL	4.02	5.39	7.00	3.25	5.40	5.18	18.4	11.7
Hoops	19.0	23.5	22.4	27.1	28.5	27.4	36.5	37.4
Liv	17.7	17.7	14.5	16.2	23.9	24.3	23.2	23.7
Nkuba	18.5	21.0	17.8	18.4	20.0	23.0	31.7	28.8
Plants	9.73	15.4	14.8	20.0	14.2	10.0	48.3	63.0
Stairs	18.0	20.9	24.2	37.7	30.9	25.6	44.8	42.8

Method	D1-bg	D1-fg	D1-all
Pretrained MC-CNN	13.20	28.26	15.71
MST [¹⁶16 Yang Q. A non-local cost aggregation method for stereo matching. 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2012:1402-9. https://doi.org/10.1109/CVPR.2012.6247827 https://doi.org/10.1109/CVPR.2012.624782... ]	45.83	38.22	44.57
ELAS [²⁹29 Geiger A, Roser M, Urtasun R. Efficient large-scale stereo matching. Asian conference on computer vision. Springer, Berlin, Heidelberg, 2010:25-38. https://doi.org/10.1007/978-3-642-19315-6_3 https://doi.org/10.1007/978-3-642-19315-... ]	7.86	19.04	9.72
Ours	5.18	17.81	7.28
3DMST [³⁵35 Lincheng Li, Xin Yu, Shunli Zhang, Xiaolin Zhao, Li Zhang. 3D cost aggregation with multiple minimum spanning trees for stereo matching. Appl. Opt., 2017, 56(12): 3411-20. https://doi.org/10.1007/978-3-030-41404-7_25 https://doi.org/10.1007/978-3-030-41404-... ]	3.36	13.03	4.97

Method	Bad 2.0 nonocc all	Bad 1.0 nonocc all	Bad 4.0 nonocc all	Aggerr nonocc all
Intensity	12.4 18.7	24.7 30.8	7.68 13.4	3.37 7.36
Intensity+gradient	12.2 18.6	24.7 30.8	7.57 13.2	3.31 7.40
Ours	11.8 18.2	24.3 30.4	7.26 12.9	3.06 7.06

Instituto de Tecnologia do Paraná - Tecpar Rua Prof. Algacyr Munhoz Mader, 3775 - CIC, 81350-010 Curitiba PR Brazil, Tel.: +55 41 3316-3052/3054, Fax: +55 41 3346-2872 - Curitiba - PR - Brazil
E-mail: babt@tecpar.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] *Correspondence: chengxianjing2014@126.com; Tel.: +86-13765958535 (X.C.).