Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report
Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report
As the quality of mobile cameras starts to play a crucial role in modern smartphones, more and more attention is now being paid to ISP algorithms used to improve various perceptual aspects of mobile photos. In this Mobile AI challenge, the target was to develop an end-to-end deep learning-based image signal processing (ISP) pipeline that can replace classical hand-crafted ISPs and achieve nearly real-time performance on smartphone NPUs. For this, the participants were provided with a novel learned ISP dataset consisting of RAW-RGB image pairs captured with the Sony IMX586 Quad Bayer mobile sensor and a professional 102-megapixel medium format camera. The runtime of all models was evaluated on the MediaTek Dimensity 1000+ platform with a dedicated AI processing unit capable of accelerating both floating-point and quantized neural networks. The proposed solutions are fully compatible with the above NPU and are capable of processing Full HD photos under 60-100 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.
Heena Patel、Grigory Malivenko、Kiran Raja、Jingyang Peng、Kwan-Lin Chen、Hao Dejun、Changbeom Kang、Anjali Sarvaiya、Raghavendra Ramachandra、Etienne de Stoutz、Shusong Xu、Min-Hung Chen、Chao-Hung Chen、Vishal Chudasama、Andrey Ignatov、Yu-Syuan Xu、Jin Guo、Xie Fen、Tao Wang、Mei-Hsuang Wu、Zheng Shaolong、Minsu Kwon、Radu Timofte、Yipeng Ma、Anastasia Sycheva、Hsien-Kai Kuo、Cheng-Ming Chiang、Shinjo Wang、Feng Zhuang、Myungje Lee、Man-Yu Lee、Christoph Busch、Chih-Chung Hsu、Wen-Chia Tsai、Fenglong Song、Ming-Chun Hsyu、Yu Tseng、Jaeyoon Yoo、Kishor Upla、Kalpesh Prajapati、Chao-Wei Chen
微电子学、集成电路计算技术、计算机技术电子技术应用
Heena Patel,Grigory Malivenko,Kiran Raja,Jingyang Peng,Kwan-Lin Chen,Hao Dejun,Changbeom Kang,Anjali Sarvaiya,Raghavendra Ramachandra,Etienne de Stoutz,Shusong Xu,Min-Hung Chen,Chao-Hung Chen,Vishal Chudasama,Andrey Ignatov,Yu-Syuan Xu,Jin Guo,Xie Fen,Tao Wang,Mei-Hsuang Wu,Zheng Shaolong,Minsu Kwon,Radu Timofte,Yipeng Ma,Anastasia Sycheva,Hsien-Kai Kuo,Cheng-Ming Chiang,Shinjo Wang,Feng Zhuang,Myungje Lee,Man-Yu Lee,Christoph Busch,Chih-Chung Hsu,Wen-Chia Tsai,Fenglong Song,Ming-Chun Hsyu,Yu Tseng,Jaeyoon Yoo,Kishor Upla,Kalpesh Prajapati,Chao-Wei Chen.Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report[EB/OL].(2021-05-17)[2025-08-21].https://arxiv.org/abs/2105.07809.点此复制
评论