基于知识蒸馏的云端协同框架

With the rapid development of mobile devices, using deep learning models to provide users with high-quality services is no longer a privilege for servers. Previous research focused on training high-performance models on cloud servers to provide services to users, or preprocessing the data at the edge server before sending the data to the cloud, where the cloud server completes the training task. However, these methods have certain limitations.First, network link between the user and the cloud server is long so it is difficult to ensure a low network delay. Secondly, users need to upload personal data such as photos they have taken, which poses a potential risk of privacy leakage. In response to this problem, a cloud-device collaboration framework based on knowledge distillation is proposed, which can provide users with fast and high-performance services while protecting user privacy. The framework delivers the teacher model trained on cloud servers to the mobile devices to guide the training of the student model deployed on mobile devices and to obtain the final model. The final model has better performance than the original and faster inference time compared to the teacher model. At the same time, the final model is dynamically updated. As the data collected by users increases, the performance of the final model will gradually improve and approach the performance of the teacher model. Experimental results show that the final model can increase the accuracy of the service by 17.01%, and the inference speed is reduced by 31.67% compared to the teacher model. At the same time, at the update stage, the final model only uses 16.67% of the training samples, and can obtain an accuracy rate that is only 3.18% lower than the teacher model.