Flexible Personalization Quantization for Heterogeneous Resource-Constrained Devices with Edge Collaboration
Qianru Wang , Guangtian Zhang , Qingyang Li , Yanguo Peng , Jiangtao Cui , Bin Guo
Smart devices embedded with increasingly intelligent applications have become ubiquitous in daily life. Considering the resource-limited devices, Quantization techniques have emerged to compress deep learning models employed in these applications. Traditionally, quantization relies heavily on cloud servers, functioning as a one-on-one service. However, such approaches encounter significant limitations when applied to heterogeneous devices with diverse computational and storage capabilities, as well as varying data distributions. The reliance on cloud servers leads to growing computational burdens and communication overheads, as the servers must allocate considerable time and resources to train separate quantized models tailored to each device’s specific requirements. To address this issue, this paper proposes leveraging edge server collaboration to relieve the computational burden on cloud servers. Unlike traditional one-on-one quantization, we design a hierarchical quantization framework named PerFlexQ, which joint considers network overhead and device capabilities to optimize both model quantization and distribution efficiency. PerFlexQ consists of three phases: Cloud2edge quantization quantizes the original model to meet the requirements of edge servers. Then edge2device quantization efficiently re-quantizes the quantized model into lower bit-widths, significantly reducing the costs of training different quantized models for heterogeneous devices. After receiving the quantized model from the connected edge server, on-device local personalization fine-tunes only a small part of the parameters in order to adapt to the local data distribution. In the experiments, we simulate device heterogeneity using CIFAR-100 and HARTH datasets with varying data distributions. Extensive evaluation across three different base recognition models demonstrates that our hierarchical quantization framework achieves significant cost reductions compared to SOTA baseline methods, while the resulting quantized models improve accuracy. Even under ultra-low target bit-widths, personalized quantized models can be efficiently generated through PerFlexQ.
Model compression for mobile devices / Edge collaboration / Model quantization
Higher Education Press 2026
/
| 〈 |
|
〉 |
PDF
(1729KB)
