FPGA sharing in the cloud: a comprehensive analysis

Abstract

Cloud vendors are actively adopting FPGAs into their infrastructures for enhancing performance and efficiency. As cloud services continue to evolve, FPGA (field programmable gate array) systems would play an even important role in the future. In this context, FPGA sharing in multi-tenancy scenarios is crucial for the wide adoption of FPGA in the cloud. Recently, many works have been done towards effective FPGA sharing at different layers of the cloud computing stack.

In this work, we provide a comprehensive survey of recent works on FPGA sharing. We examine prior art from different aspects and encapsulate relevant proposals on a few key topics. On the one hand, we discuss representative papers on FPGA resource sharing schemes; on the other hand, we also summarize important SW/HW techniques that support effective sharing. Importantly, we further analyze the system design cost behind FPGA sharing. Finally, based on our survey, we identify key opportunities and challenges of FPGA sharing in future cloud scenarios.

Graphical abstract

Keywords

cloud FPGA / FPGA sharing / efficiency / design cost

Jinyang Guo received the bachelor degree in software engineering from Wuhan University, China. He is working toward the master degree in the Department of Computer Science and Engineering, Shanghai Jiao Tong University, China. His main research interests include graph processing, FPGA accelerator, and edge computing

Lu Zhang received the BS degree from the Northwestern Polytechnical University, China in 2016. He is working toward the PhD degree in the Department of Computer Science and Engineering, Shanghai Jiao Tong University, China. His research interests include edge computing, network function virtualization, and serverless computing

José Romero Hung is an International PhD Candidate in the Department of Micro/Nano Electronics, Shanghai Jiao Tong University, China. His most recent research contributions include topics on hardware acceleration for complex machine learning applications

Chao Li received his PhD degree from the University of Florida, USA in 2014. He is currently an professor with tenure in the Department of Computer Science and Engineering, Shanghai Jiao Tong University, China. His research mainly focuses on computer architecture and systems for emerging applications. He is a senior member of IEEE/ACM/CCF

Jieru Zhao received the Ph.D. degree in electronic and computer engineering from the Hong Kong University of Science and Technology, China in 2020. She is an Assistant Professor with the Department of Computer Science and Engineering, Shanghai Jiao Tong University, China. Her current research interests include reconfigurable system, high-level synthesis and HW/SW co-design for emerging applications. Dr. Zhao was a recipient of the Best Paper Award at ICCAD 2017 and Best Paper Nominations at DATE 2022, SC 2021, and CASES 2018

Minyi Guo received his PhD degree in computer science from the University of Tsukuba, Japan. He is currently Zhiyuan Chair professor, Shanghai Jiao Tong University, China. His research interests include parallel/distributed computing, compiler optimizations, cloud computing, and big data. He is now on the editorial board of IEEE Transactions on Parallel and Distributed Systems and Journal of Parallel and Distributed Computing. Dr. Guo is IEEE fellow and CCF fellow

References

[1] Amazon Cloud. Amazon EC2 F1 Instances. See Aws.amazon.com/cn/ec2/instance-types/f1/ website, 2016

[2] Huawei Cloud. FPGA Accelerated Cloud Server. See Huaweicloud.com/product/facs.html website, 2017

[3] Alibaba Cloud. FPGA Cloud Server. See Aliyun.com/product/ecs/fpga website, 2017

[4] Baidu AI Cloud. FPGA Cloud Server. See Cloud.baidu.com/product/fpga.html website, 2017

[5] Tencent Cloud. FPGA Cloud Server. See Cloud.tencent.com/product/fpga website, 2018

[6] Tourad E H C, Eleuldj M. Survey of deep learning neural networks implementation on FPGAs. In: Proceedings of the 5th International Conference on Cloud Computing and Artificial Intelligence: Technologies and Applications. 2020, 1−8

[7] Wu C, Fresse V, Suffran B, Konik H . Accelerating DNNs from local to virtualized FPGA in the cloud: a survey of trends. Journal of Systems Architecture, 2021, 119: 102257

[8] Shahzad H, Sanaullah A, Herbordt M C. Survey and future trends for FPGA cloud architectures. In: Proceedings of 2021 IEEE High Performance Extreme Computing Conference. 2021, 1−10

[9] Quraishi M H, Tavakoli E B, Ren F . A survey of system architectures and techniques for FPGA virtualization. IEEE Transactions on Parallel and Distributed Systems, 2021, 32( 9): 2216–2230

[10] Trimberger S, McNeil S. Security of FPGAs in data centers. In: Proceedings of IEEE the 2nd International Verification and Security Workshop. 2017, 117−122

[11] Turan F, Verbauwhede I . Trust in FPGA-accelerated cloud computing. ACM Computing Surveys, 2021, 53( 6): 128

[12] Jin C, Gohil V, Karri R, Rajendran J. Security of cloud FPGAs: a survey. 2020, arXiv preprint arXiv: 2005.04867

[13] Dessouky G, Sadeghi A R, Zeitouni S. SoK: Secure FPGA multi-tenancy in the cloud: challenges and opportunities. In: Proceedings of 2021 IEEE European Symposium on Security and Privacy. 2021, 487−506

[14] Chen F, Shan Y, Zhang Y, Wang Y, Franke H, Chang X, Wang K. Enabling FPGAs in the cloud. In: Proceedings of the 11th ACM Conference on Computing Frontiers. 2014, 3

[15] Putnam A, Caulfield A M, Chung E S, Chiou D, Constantinides K, Demme J, Esmaeilzadeh H, Fowers J, Gopal G P, Gray J, Haselman M, Hauck S, Heil S, Hormati A, Kim J Y, Lanka S, Larus J, Peterson E, Pope S, Smith A, Thong J, Xiao P Y, Burger D. A reconfigurable fabric for accelerating large-scale datacenter services. In: Proceedings of 2014 ACM/IEEE 41st International Symposium on Computer Architecture. 2014, 13−24

[16] Intel. Hardware accelerator research program. See Communityintel.com/t5/Software-Tuning-Performance/HARP-Hardware-Accelerator-Research-Program/m-p/1126570 website, 2018

[17] Iordache A, Pierre G, Sanders P, De F. Coutinho J G, Stillwell M. High performance in the cloud with FPGA groups. In: Proceedings of the 9th IEEE/ACM International Conference on Utility and Cloud Computing. 2016, 1−10

[18] Zha Y, Li J. Virtualizing FPGAs in the cloud. In: Proceedings of the 25th International Conference on Architectural Support for Programming Languages and Operating Systems. 2020, 845−858

[19] Byma S, Steffan J G, Bannazadeh H, Leon-Garcia A, Chow P. FPGAS in the cloud: booting virtualized hardware accelerators with OpenStack. In: Proceedings of the 22nd IEEE Annual International Symposium on Field-Programmable Custom Computing Machines. 2014, 109−116

[20] Lavin C, Kaviani A. RapidWright: enabling custom crafted implementations for FPGAs. In: Proceedings of the 26th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines. 2018, 133−140

[21] Zhu Z, Liu A X, Zhang F, Chen F . FPGA resource pooling in cloud computing. IEEE Transactions on Cloud Computing, 2021, 9( 2): 610–626

[22] Guo L, Chi Y, Wang J, Lau J, Qiao W, Ustun E, Zhang Z, Cong J. AutoBridge: coupling coarse-grained floorplanning and pipelining for high-frequency HLS design on multi-die FPGAs. In: Proceedings of the 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 2021, 81−92

[23] Moghaddamfar M, Färber C, Lehner W, May N, Kumar A. Resource-efficient database query processing on FPGAs. In: Proceedings of the 17th International Workshop on Data Management on New Hardware. 2021, 4

[24] Chen X, Tan H, Chen Y, He B, Wong W F, Chen D. ThunderGP: HLS-based graph processing framework on FPGAs. In: Proceedings of 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 2021, 69−80

[25] Liu C, Shao Z, Li K, Wu M, Chen J, Li R, Liao X, Jin H. ScalaBFS: a scalable BFS accelerator on FPGA-HBM platform. In: Proceedings of 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 2021, 147

[26] Wu Y W, Wang Q G, Zheng L, Liao X F, Jin H, Jiang W B, Zheng R, Hu K . FDGLib: a communication library for efficient large-scale graph processing in FPGA-accelerated data centers. Journal of Computer Science and Technology, 2021, 36( 5): 1051–1070

[27] Bacis M, Brondolin R, Santambrogio M D. BlastFunction: an FPGA-as-a-service system for accelerated serverless computing. In: Proceedings of the 23rd Conference on Design, Automation and Test in Europe. 2020, 852−857

[28] Hong B, Kim H Y, Kim M, Suh T, Xu L, Shi W . FASTEN: an FPGA-based secure system for big data processing. IEEE Design & Test, 2018, 35( 1): 30–38

[29] Skhiri R, Fresse V, Malek J, Suffran B, Jamont J. An approach for an efficient sharing of IP as a service in cloud FPGA. In: Proceedings of the 18th International Multi-Conference on Systems, Signals & Devices. 2021, 784−789

[30] Khawaja A, Landgraf J, Prakash R, Wei M, Schkufza E, Rossbach C J. Sharing, protection, and compatibility for reconfigurable fabric with Amorphos. In: Proceedings of the 13th USENIX Conference on Operating Systems Design and Implementation. 2018, 107−127

[31] Zha Y, Li J. Hetero-ViTAL: A virtualization stack for heterogeneous FPGA clusters. In: Proceedings of the 48th ACM/IEEE Annual International Symposium on Computer Architecture. 2021, 470−483

[32] Xilinx. Alveo data center accelerator card platforms. See Docs.xilinx.com/r/en-US/ug1120-alveo-platforms website, 2021

[33] Intel. FPGA Cloud Server, High Bandwidth Memory (HBM2) Interface Intel® FPGA IP user guide. See Intel website, 2021

[34] Choi Y K, Cong J, Fang Z, Hao Y, Reinman G, Wei P . In-depth analysis on microarchitectures of modern heterogeneous CPU-FPGA platforms. ACM Transactions on Reconfigurable Technology and Systems, 2019, 12( 1): 4

[35] Voss N, Quintana P, Mencer O, Luk W, Gaydadjiev G. Memory mapping for multi-die FPGAs. In: Proceedings of the 27th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines. 2019, 78−86

[36] Xilinx. TAlveo U280 data center accelerator card user guideT. See Xilinx.com/products/boards-and-kits/alveo/u280 website, 2021

[37] Hung J R, Li C, Wang P, Shao C, Guo J, Wang J, Shi G . ACE-GCN: a fast data-driven FPGA accelerator for GCN embedding. ACM Transactions on Reconfigurable Technology and Systems, 2021, 14( 4): 21

[38] Xilinx. Xilinx UltraScale: The next-generation architecture for your next-generation architecture. See Docs.xilinx.com/v/u/en-US/wp435-Xilinx-UltraScale website. 2014

[39] Ma J, Zuo G, Loughlin K, Cheng X, Liu Y, Eneyew A M, Qi Z, Kasikci B. A hypervisor for shared-memory FPGA platforms. In: Proceedings of the 25th International Conference on Architectural Support for Programming Languages and Operating Systems. 2020, 827−48444

[40] Tarafdar N, Eskandari N, Lin T, Chow P . Designing for FPGAs in the cloud. IEEE Design & Test, 2018, 35( 1): 23–29

[41] Vaishnav A, Pham K D, Koch D, Garside J. Resource elastic virtualization for FPGAs using OpenCL. In: Proceedings of the 28th International Conference on Field Programmable Logic and Applications. 2018, 111−41184

[42] Gonzalez I, Lopez-Buedo S, Sutter G, Sanchez-Roman D, Gomez-Arribas F J, Aracil J. Virtualization of reconfigurable coprocessors in HPRC systems with multicore architecture. Journal of Systems Architecture, 2012, 58(6−7): 247−256

[43] Huang C H, Hsiung P A . Virtualizable hardware/software design infrastructure for dynamically partially reconfigurable systems. ACM Transactions on Reconfigurable Technology and Systems, 2013, 6( 2): 11

[44] Wang W, Bolic M, Parri J. pvFPGA: accessing an FPGA-based hardware accelerator in a paravirtualized environment. In: Proceedings of 2013 International Conference on Hardware/Software Codesign and System Synthesis. 2013, 10

[45] Kidane H L, Bourennane E B, Ochoa-Ruiz G. NoC based virtualized accelerators for cloud computing. In: Proceedings of the 10th IEEE International Symposium on Embedded Multicore/Many-core Systems-on-Chip. 2016, 133−137

[46] Vatsolakis C, Pnevmatikatos D. RACOS: transparent access and virtualization of reconfigurable hardware accelerators. In: Proceedings of 2017 International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation. 2017, 11−19

[47] Zhao Q, Amagasaki M, Iida M, Kuga M, Sueyoshi T . Enabling FPGA-as-a-service in the cloud with hCODE platform. IEICE Transactions on Information and Systems, 2018, 101-D( 2): 335–343

[48] Mbongue J M, Hategekimana F, Kwadjo D T, Bobda C. FPGA virtualization in cloud-based infrastructures over virtio. In: Proceedings of the 36th IEEE International Conference on Computer Design. 2018, 242−245

[49] Mbongue J, Hategekimana F, Kwadjo D T, Andrews D, Bobda C. FPGAVirt: a novel virtualization framework for FPGAs in the cloud. In: Proceedings of the 11th IEEE International Conference on Cloud Computing. 2018, 862−865

[50] Coole J, Stitt G. Intermediate fabrics: virtual architectures for circuit portability and fast placement and routing. In: Proceedings of the 8th International Conference on Hardware/Software Codesign and System Synthesis. 2010, 13−22

[51] Cong J, Wei P, Yu C H, Zhou P. Bandwidth optimization through on-chip memory restructuring for HLS. In: Proceedings of the 54th ACM/EDAC/IEEE Design Automation Conference. 2017, 43

[52] Yang J, Yan L, Ju L, Wen Y, Zhang S, Chen T. Homogeneous NoC-based FPGA: the foundation for virtual FPGA. In: Proceedings of the 10th IEEE International Conference on Computer and Information Technology. 2010, 62−67

[53] Huang C, Hsiung P A . Hardware resource virtualization for dynamically partially reconfigurable systems. IEEE Embedded Systems Letters, 2009, 1( 1): 19–23

[54] Asiatici M, George N, Vipin K, Fahmy S A, Ienne P. Designing a virtual runtime for FPGA accelerators in the cloud. In: Proceedings of the 26th International Conference on Field Programmable Logic and Applications. 2016, 1−2

[55] Giechaskiel I, Rasmussen K, Szefer J. Reading between the dies: Cross-SLR covert channels on multi-tenant cloud FPGAs. In: Proceedings of the 37th IEEE International Conference on Computer Design. 2019, 1−10

[56] Knodel O, Lehmann P, Spallek R G. RC3E: reconfigurable accelerators in data centres and their provision by adapted service models. In: Proceedings of the 9th IEEE International Conference on Cloud Computing. 2016, 19−26

[57] Huang M, Wu D, Yu C H, Fang Z, Interlandi M, Condie T, Cong J. Programming and runtime support to blaze FPGA accelerator deployment at datacenter scale. In: Proceedings of the 7th ACM Symposium on Cloud Computing. 2016, 456−469

[58] Stangl J, Lor ̈unser T, Dinakarrao S M P. A fast and resource efficient FPGA implementation of secret sharing for storage applications. In: Madsen J, Coskun A K, eds, 2018 Design, Automation & Test in Europe Conference & Exhibition. 2018, 654–659

[59] Weerasinghe J, Abel F, Hagleitner C, Herkersdorf A. Enabling FPGAs in hyperscale data centers. In: Proceedings of 2015 IEEE the 12th Intl Conf on Ubiquitous Intelligence and Computing and 2015 IEEE the 12th Intl Conf on Autonomic and Trusted Computing and 2015 IEEE the 15th Intl Conf on Scalable Computing and Communications and its Associated Workshops. 2015, 1078−1086

[60] Fahmy S A, Vipin K, Shreejith S. Virtualized FPGA accelerators for efficient cloud computing. In: Proceedings of the 7th IEEE International Conference on Cloud Computing Technology and Science. 2015, 430−435

[61] Li X, Wang X, Liu F, Xu H. DHL: enabling flexible software network functions with FPGA acceleration. In: Proceedings of the 38th IEEE International Conference on Distributed Computing Systems. 2018, 1−11

[62] Papadimitriou K, Dollas A, Hauck S . Performance of partial reconfiguration in FPGA systems: a survey and a cost model. ACM Transactions on Reconfigurable Technology and Systems, 2011, 4( 4): 36

[63] Xilinx. UltraScale Architecture Configuration. See Docs.xilinx.com/v/u/en-US/ug570-ultrascale-configuration website, 2022

[64] Zhang J, Xiong Y, Xu N, Shu R, Li B, Cheng P, Chen G, Moscibroda T. The feniks FPGA operating system for cloud computing. In: Proceedings of the 8th Asia-Pacific Workshop on Systems. 2017, 22

[65] Knodel O, Genssler P R, Spallek R G . Migration of long-running tasks between reconfigurable resources using virtualization. ACM SIGARCH Computer Architecture News, 2016, 44( 4): 56–61

[66] Vaishnav A, Pham K, Koch D. Live migration for OpenCL FPGA accelerators. In: Proceedings of 2018 International Conference on Field-Programmable Technology. 2018, 38−45

[67] Korolija D, Roscoe T, Alonso G. Do OS abstractions make sense on FPGAs? In: Proceedings of the 14th USENIX Conference on Operating Systems Design and Implementation. 2020, 991−1010

[68] Asiatici M, George N, Vipin K, Fahmy S A, Ienne P . Virtualized execution runtime for FPGA accelerators in the cloud. IEEE Access, 2017, 5: 1900–1910

[69] Fowers J, Ovtcharov K, Papamichael M, Massengill T, Liu M, Lo D, Alkalay S, Haselman M, Adams L, Ghandi M, Heil S, Patel P, Sapek A, Weisz G, Woods L, Lanka S, Reinhardt S K, Caulfield A M, Chung E S, Burger D. A configurable cloud-scale DNN processor for real-time AI. In: Proceedings of the 45th ACM/IEEE Annual International Symposium on Computer Architecture. 2018, 1−14

[70] Yazdanshenas S, Betz V. Improving confidentiality in virtualized FPGAs. In: Proceedings of 2018 International Conference on Field-Programmable Technology. 2018, 258−261

[71] Yazdanshenas S, Betz V . The costs of confidentiality in virtualized FPGAs. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2019, 27( 10): 2272–2283

[72] Zhao M, Gao M, Kozyrakis C. ShEF: shielded enclaves for cloud FPGAs. In: Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 2022, 1070−1085

[73] Krautter J, Gnad D R E, Tahoori M B . Mitigating electrical-level attacks towards secure multi-tenant FPGAs in the cloud. ACM Transactions on Reconfigurable Technology and Systems, 2019, 12( 3): 12

[74] Krautter J, Gnad D, Tahoori M . CPAmap: On the complexity of secure FPGA virtualization, multi-tenancy, and physical design. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2020, 2020( 3): 121–146

[75] Duncan A, Nahiyan A, Rahman F, Skipper G, Swany M, Lukefahr A, Farahmandi F, Tehranipoor M. SeRFI: Secure remote FPGA initialization in an untrusted environment. In: Proceedings of the 38th IEEE VLSI Test Symposium. 2020, 1−6

[76] Krautter J, Gnad D R E, Schellenberg F, Moradi A, Tahoori M B. Active fences against voltage-based side channels in multi-tenant FPGAs. In: Proceedings of 2019 IEEE/ACM International Conference on Computer-Aided Design. 2019, 1−8

[77] Jayasinghe D, Ignjatovic A, Parameswaran S. RFTC: runtime frequency tuning countermeasure using FPGA dynamic reconfiguration to mitigate power analysis attacks. In: Proceedings of the 56th ACM/IEEE Design Automation Conference. 2019, 1−6

[78] Provelengios G, Holcomb D, Tessier R. Characterizing power distribution attacks in multi-user FPGA environments. In: Proceedings of the 29th International Conference on Field Programmable Logic and Applications. 2019, 194−201

[79] Walder H, Platzner M. A runtime environment for reconfigurable hardware operating systems. In: Proceedings of the 14th International Conference on Field Programmable Logic and Application. 2004, 831−835

[80] Ko H J, Li Z, Midkiff S. Optimizing data layout and system configuration on FPGA-based heterogeneous platforms. In: Proceedings of 2018 IEEE/ACM International Conference on Computer-Aided Design. 2018, 1−8

[81] Koch D. Partial Reconfiguration on FPGAs: Architectures, Tools and Applications. New York: Springer, 2013

[82] Caulfield A M, Chung E S, Putnam A, Angepat H, Fowers J, Haselman M, Heil S, Humphrey M, Kaur P, Kim J Y, Lo D, Massengill T, Ovtcharov K, Papamichael M, Woods L, Lanka S, Chiou D, Burger D. A cloud-scale acceleration architecture. In: Proceedings of the 49th Annual IEEE/ACM International Symposium on Microarchitecture. 2016, 1−13

[83] Yazdanshenas S, Betz V. Quantifying and mitigating the costs of FPGA virtualization. In: Proceedings of the 27th International Conference on Field Programmable Logic and Applications. 2017, 1−7

[84] Kirchgessner R, Stitt G, George A, Lam H. VirtualRC: a virtual FPGA platform for applications and tools portability. In: Proceedings of the 20th ACM/SIGDA International Symposium on Field Programmable Gate Arrays. 2012, 205−208

[85] Stitt G, Karam R, Yang K, Bhunia S . A uniquified virtualization approach to hardware security. IEEE Embedded Systems Letters, 2017, 9( 3): 53–56

[86] Schellenberg F, Gnad D R E, Moradi A, Tahoori M B. An inside job: remote power analysis attacks on FPGAs. In: Proceedings of 2018 Design, Automation & Test in Europe Conference & Exhibition. 2018, 1111−1116

[87] Trimberger S . Security in SRAM FPGAs. IEEE Design & Test of Computers, 2007, 24( 6): 581

[88] Zhou P, Sheng J, Yu C H, Wei P, Wang J, Wu D, Cong J. TMOCHA: multinode cost optimization in heterogeneous clouds with acceleratorsT. In: Proceedings of 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 2021, 273−279

[89] Shepovalov M, Akella V . FPGA and GPU-based acceleration of ML workloads on amazon cloud - A case study using gradient boosted decision tree library. Integration, 2020, 70: 1–9

[90] Eguro K, Venkatesan R. FPGAs for trusted cloud computing. In: Proceedings of the 22nd International Conference on Field Programmable Logic and Applications. 2012, 63−70

[91] Wu C, Buyya R, Ramamohanarao K . Cloud pricing models: taxonomy, survey, and interdisciplinary challenges. ACM Computing Surveys, 2020, 52( 6): 108

[92] Landgraf J, Yang T, Lin W, Rossbach C J, Schkufza E. Compiler-driven FPGA virtualization with SYNERGY. In: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 2021, 818−831

Acknowledgements

We thank all the reviewers for their valuable comments and feedbacks. This work was sponsored by the National Natural Science Foundation of China (Grant No. 61972247).

RIGHTS & PERMISSIONS

2023 Higher Education Press