基于数字孪生的端边云协同联邦强化学习在多小区车联网中的应用
Digital-Twin-Based End-Edge-Cloud Collaborative Federated Reinforcement Learning for Multi-Cell IoV Networks
普阳 1高博 1耿光磊 1熊轲 1王煜炜2
作者信息
- 1. 北京交通大学计算机科学与技术学院，北京 100044
- 2. 中国科学院计算技术研究所，北京100190
- 折叠
摘要
由于多小区车联网中频谱资源有限,亟需高效的资源分配算法。强化学习是目前被广泛采用的方法,但现有的基于强化学习的方法通常存在收敛速度慢和通信开销大的问题。联邦学习作为一种分布式范式,可以通过在小区间共享策略来加速收敛,并通过传输模型参数而非原始数据来降低通信负载。然而,在本地训练的联邦学习会在车辆端产生显著能耗。本文提出了一种基于数字孪生的端边云协同联邦强化学习框架 DT-PFRL,用于多小区车联网中的频谱资源分配。通过利用数字孪生,本文构建了一个虚拟车联网环境,并将训练任务从实体车辆转移到云端的虚拟副本。实体车辆仅传输实时状态信息以校正模拟误差。本文构建了一个优化问题,旨在最小化能耗,同时保持模型性能,并通过设计一种采用部分参数聚合策略的联邦强化学习算法来解决该问题。大量实验表明,与现有基线方法相比,DT-PFRL 在保持有效频谱分配性能的同时,显著降低了实体车辆的能耗。
Abstract
Due to limited spectrum resources in multi-cell Internet of vehicles (IoV) networks, dynamic and efficient allocation algorithms are essential. While reinforcement learning (RL) has been widely adopted, existing RL-based methods often suffer from slow convergence and high communication overhead. Federated learning (FL), as a distributed paradigm, can accelerate convergence by sharing optimized policies across cells and reduce communication load by transmitting model parameters instead of raw data. However, FL incurs significant energy consumption at the vehicle side due to local training. In this paper, we propose DT-PFRL, a digital-twin-based end-edge-cloud collaborative federated reinforcement learning framework for spectrum resource allocation in multi-cell IoV networks. By leveraging digital twin (DT), we construct a virtual IoV environment and offload training from physical vehicles to cloud-based digital replicas. Physical vehicles only transmit real-time state information to correct simulation errors. We formulate an optimization problem to minimize energy consumption while preserving model performance and solve it by designing a federated reinforcement learning algorithm with a partial parameter aggregation policy. Extensive experiments demonstrate that DT-PFRL significantly reduces the energy consumption of physical vehicles while maintaining effective spectrum allocation performance compared to existing baselines.关键词
计算机系统结构/数字孪生/联邦学习/车联网/频谱资源分配Key words
computer architecture/ digital twin/ federated learning/ Internet of Vehicles/ spectrum resource allocation引用本文复制引用
普阳,高博,耿光磊,熊轲,王煜炜.基于数字孪生的端边云协同联邦强化学习在多小区车联网中的应用[EB/OL].(2026-03-24)[2026-03-27].http://www.paper.edu.cn/releasepaper/content/202603-237.学科分类
计算技术、计算机技术
评论