Simulation and experimental investigation of liquid-cooling thermal management for high-bandwidth co-packaged optics

Abstract

This study explores the application of cold plate liquid cooling technology in co-packaged optics (CPO). By integrating optical modules and the switch chip on the same substrate, CPO shortens the electrical interconnection distance, effectively solving the problems of high power consumption and poor signal integrity of traditional pluggable optical modules under high bandwidth. However, the surge in power density and the thermal crosstalk resulting from high integration density make thermal management one of the key challenges that constrain the reliability of high-capacity co-packaged optics. For the unique architecture of CPO, this study analyzes its heat dissipation needs in detail, and a thermal management scheme is designed. The thermal management scheme is simulated and optimized based on the Navier−Stokes equation. The simulation results show that, in a 51.2 Tbit/s CPO system, the junction temperature of the switch chip is 97.3 °C, the maximum junction temperature of the optical modules is 31.3 °C, and the temperature difference between the optical modules is 2.4 °C to 1.2 °C. To verify the simulation results, a thermal test experimental platform is built, and the experimental results show that the temperature simulation difference is within 4% and the pressure change trend is consistent with the simulation. Combining the experimental data and simulation results, the designed heat sink can satisfy the heat dissipation demands of the 51.2 Tbit/s bandwidth CPO system. This conclusion demonstrates the potential of liquid-cooling technology in CPO, providing support for research on liquid-cooling technology in the CPO. The design provides a theoretical and practical basis for the high performance and reliability of optoelectronic integration technology in wavelength division multiplexing (WDM) systems and micro-ring device applications, contributing to the application of next-generation optical communication networks.

Graphical abstract

Keywords

Co-packaged optics / Optoelectronic integration / Thermal management / Liquid cooling / Manifold microchannel heat sink

References

[1] Jarek,K., Mazurek, G.: Marketing and artificial intelligence. Cent. Eur. Bus. Rev. 8(2), 46 (2019)

[2] Sevilla,J., Heim,L., Ho,B.T.A., Hobbhahn, M., Villalobos,P.: Compute trends across three eras of machine learning. In: Sevilla, J., (ed.) 2022 International joint conference on neural networks (IJCNN), pp. 1–8. IEEE, Padua (2022)

[3] Stone,R., Chen,R., Rahn,J., Venkataraman, S., Wang,X., Schmidtke,K., Stewart, J.: Co-packaged optics for data center switching. In: Stone, R., (ed.) 2020 European conference on optical communications (ECOC), pp. 1–3. IEEE, Brussels (2020)

[4] Mahajan,R., Li,X., Fryman,J., Zhang, Z., Nekkanty,S., Tadayon,P., Jaussi, J., Shumarayev,S., Agrawal,A., Jadhav, S., Singh,K.A., Alduino,A., Gujjula, S., Chiu,C.P., Nordstog,T., Hosseini, K.J., Sane,S., Deshpande,N., Aygun,K., Sarkar,A., Dobriyal, P., Pothukuchi,S., Pogue,V., Hui,D.: Co-packaged photonics for high performance computing: status, challenges and opportunities. J. Lightwave Technol. 40(2), 379–392 (2022)

[5] Papatryfonos,K., Selviah, D.R., Maman,A., Hasharoni,K., Brimont, A., Zanzi,A., Kraft,J., Sidorov, V., Seifried,M., Baumgartner,Y., Horst,F., Offrein,B.J., Lawniczuk,K., Broeke, R.G., Terzenidis,N., Mourgias-Alexandris,G., Tang, M., Seeds,A.J., Liu,H., Sanchis, P., Moralis-Pegios,M., Manolis,T., Pleros, N., Vyrsokinos,K., Sirbu,B., Eichhammer, Y., Oppermann,H., Tekin,T.: Co-package technology platform for low-power and low-cost data centers. Appl. Sci. (Basel. 11(13), 6098 (2021)

[6] Razdan,S., Traverso, M., Torza,A.: Co-packaged optics integration for hyperscale networking. Cisco Systems Inc IEEE, :San Jose (2023)

[7] Sun,Y., Liu,F., Xue,H.: High-speed and high-density optoelectronic co-package technologies. ZTE Technol. J 24(4), 27–32 (2018)

[8] Xiang,C., Bowers, S.M., Bjorlin,A., Blum,R., Bowers, J.E.: Perspective on the future of silicon photonics and electronics. Appl. Phys. Lett. 118(22), 220501 (2021)

[9] Takemura,K., Ohshima, D., Noriki,A., Okamoto,D., Ukita,A., Ushida,J., Tokushima, M., Shimizu,T., Ogura,I., Shimura, D., Aoki,T., Amano,T., Nakamura, T.: Silicon-photonics-embedded interposers as co-packaged optics platform. Transact. Jpn. Instit. Electron. Packag. 15, E21–012 (2022)

[10] Hosseini,K., Kok,E., Shumarayev,S.Y., Chiu,C.P., Sarkar, A., Toda,A., Ke,Y., Chan,A., Jeong,D., Zhang, M., Raman,S., Tran,T., Singh,K.A., Bhargava,P., Zhang, C., Lu,H., Mahajan,R., Li,X., Deshpande,N., Keeffe,C., Hoang,T., Krishnamoorthy,U., Sun,C., Meade,R., Stojanovic,V., Wade,M.: 8 Tbps copackaged FPGA and silicon photonics optical IO. In: Hosseini, K., (ed.) 2021 Optical fiber communications conference and exhibition (OFC). IEEE, San Francisco (2021)

[11] Hosseini,K., Kok,E., Shumarayev,S.Y., Jeong,D., Chan,A., Katzin,A., Liu, S., Roucka,R., Raval,M., Mac,M., Chiu,C.P., Tran, T., Singh,K.A., Raman,S., Ke,Y., Li,C., Yang, L.F., Chao,P., Lu,H., Luna,F., Li,X., Hoang, T.T., Sarkar,A., Toda,A., Mahajan, R., Deshpande,N., O’Keeffe,C., Krishnamoorthy, U., Stojanovic,V., Madden,C., Zhang,C., Sysak,M., Bhargava, P., Sun,C., Wade,M.: 5.12 Tbps co-packaged FPGA and silicon photonics interconnect I/O. In: Hosseini, K., (ed.) 2022 IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits). IEEE, Honolulu (2022)

[12] Wade,M., Anderson, E., Ardalan,S., Bhargava,P., Buchbinder, S., Davenport,M.L., Fini,J., Lu,H., Li,C., Meade, R., Ramamurthy,C., Rust,M., Sedgwick, F., Stojanovic,V., VanOrden, D., Zhang, C., Sun,C., Shumarayev,S.Y., O’Keeffe, C., Hoang,T.T., Kehlet,D., Mahajan, R.V., Guzy,M.T., Chan,A., Tran,T.: Teraphy: a chiplet technology for low-power, high-bandwidth inpackage optical I/O. IEEE Micr. 40(2), 63–71 (2020)

[13] Meade,R., Ardalan, S., Davenport,M., Fini,J., Sun,C., Wade,M., Wright-Gladstein, A., Zhang,C.: TeraPHY: a high-density electronic-photonic chiplet for optical I/O from a multi-chip module. In: Meade, R., (ed.) 2019 Optical fiber communications conference and exhibition (OFC). IEEE, San Diego (2019)

[14] Mahajan,R., Sankman, R., Patel,N., Kim,D.W., Aygun,K., Qian,Z., Mekonnen, Y., Salama,I., Sharan,S., Iyengar, D., Mallik,D.: Embedded multi-die interconnect bridge (EMIB)–a high density, high bandwidth packaging inter-connect. In: Mahajan, R., (ed.) 2016 IEEE 66th Electronic components and technology conference (ECTC). IEEE, Las Vegas (2016)

[15] Nagarajan,R., Ding,L., Coccioli,R., Kato, M., Tan,R., Tumne,P., Patterson, M., Liu,L.: 2.5 d heterogeneous integration for silicon photonics engines in optical transceivers. IEEE J Select Top Quant Elect. 29(3), 1–10 (2022)

[16] Tian,W., Hou,H., Dang,H., Cao, X., Li,D., Chen,S., Ma,B.: Progress in research on co-packaged optics. Micromachines (Basel). 15(10), 1211 (2024)

[17] Tartaglia,A., Cavaliere, F., Lostedt,M., Bigongiari,A., Palagi, A., Parkholm,U., Tavemark,A., Stracca, S., D’Errico,A., Lessard,S., Johansson, M.: Perspectives for co-packaged optics in radio access networks. In: Tartaglia, A., (ed.) 2023 23rd International conference on transparent optical networks (ICTON). IEEE, Bucharest (2023)

[18] Mehta,M.: Using silicon photonics in a co-packaged optical interconnect. Chiplet Summit, San Jose (2024)

[19] Minkenberg,C., Krishnaswamy, R., Zilkie,A., Nelson,D.: Copackaged datacenter optics: opportunities and challenges. IET Optoelectron. 15(2), 77–91 (2021)

[20] Aleksic,S.: Power consumption issues in future high-performance switches and routers. In: Aleksic, S., (ed.) 2008 10th anniversary international conference on transparent optical networks, vol. 3. IEEE, Athens (2008)

[21] Pomeroy,J.W., Uren,M.J., Lambert,B., Kuball, M.: Operating channel temperature in GaN HEMTs: DC versus RF accelerated life testing. Microelectron. Reliab. 55(12), 2505–2510 (2015)

[22] Darwish,A.M., Huebschman, B.D., Viveiros,E., Hung,H.A.: Dependence of GaN HEMT millimeter-wave performance on temperature. IEEE Trans. Microw. Theory Tech. 57(12), 3205–3211 (2009)

[23] Haghshenas,K., Setz,B., Blosch,Y., Aiello, M.: Enough hot air: the role of immersion cooling. Energy Inform. 6(1), 14 (2023)

[24] Rodgers,P., Eveloy, V., Pecht,M.G.: Limits of air-cooling: status and challenges. In: Rodgers, P., (ed.) Semiconductor thermal measurement and management IEEE twenty first annual IEEE symposium. IEEE, San Jose (2005)

[25] Dong,L., Gu,X., Hu,S., Koyama, F.: Densely packed 1.1 µm band vertical cavity surface emitting laser array for co-packaged optics. Jpn. J. Appl. Phys. 61 1011 (2022)

[26] Baehr-Jones,T., Hochberg, M., Walker,C., Chan,E., Koshinz, D., Krug,W., Scherer,A.: Analysis of the tuning sensitivity of silicon-on-insulator optical ring resonators. J. Lightwave Technol. 23(12), 4215–4221 (2005)

[27] Fathololoumi,S., Nguyen, K., Mahalingam,H., Sakib,M., Li,Z., Seibert,C., Montazeri, M., Chen,J., Doylend,J.K., Jayatilleka, H., Jan,C., Heck,J., Venables, R., Frish,H., Defrees,R., Appleton, R., Hollingsworth,S., McCargar,S., Jones,R., Liao,L.: 1.6 Tbps silicon photonics integrated circuit for co-packaged optical-IO switch applications. In: Fathololoumi, S., (ed.) Optical fiber communication conference. Optica Publishing Group, Washington, DC (2020)

[28] Fu,Y., Zhang,X., Hraimel,B., Liu, T., Shen,D.: Mach-zehnder: a review of bias control techniques for mach-zehnder modulators in photonic analog links. IEEE Microw. Mag. 14(7), 102–107 (2013)

[29] Marcatili,E.A.: Bends in optical dielectric guides. Bell Syst. Tech. J. 48(7), 2103–2132 (1969)

[30] Bogaerts,W., De Heyn, P., Van Vaerenbergh,T., De Vos,K., Kumar Selvaraja, S., Claes,T., Dumon,P., Bienstman, P., Van Thourhout,D., Baets,R.: Silicon microring resonators. Laser Photonics Rev. 6(1), 47–73 (2012)

[31] Nellis,G., Klein,S.: Heat transfer. Cambridge University Press, New York (2008)

[32] Munson,B.R., Okiishi, T.H., Huebsch,W.W., Rothmayer,A.P.: Fluid mechanics. Wiley, Singapore (2013)

[33] Hansson,J., Nilsson, T.M., Ye,L., Liu,J.: Novel nanostructured thermal interface materials: a review. Int. Mater. Rev. 63(1), 22–45 (2018)

[34] Hua,Y., Luo,L., Le Corre,S., Fan, Y.: Heat spreading effect on the optimal geometries of cooling structures in a manifold heat sink. Energ. 308 132948 (2024)

[35] Zhang,Y.F., Han,D., Zhao,Y.H., Bai, S.L.: High-performance thermal interface materials consisting of vertically aligned graphene film and polymer. Carbo. 109 552–557 (2016)

[36] Lee,D., Mayberry, R., Mackie,A., Hable,B., Heller, D., Jarrett,B., Zhao,X., Nash,T.: Optimizing reflowed solder TIM (sTIMs) processes for emerging heterogeneous integrated packages. In: Lee, D., (ed.) 2022 IEEE 72nd Electronic components and technology conference (ECTC). IEEE, San Diego (2022)

[37] Erp,R., Kampitsis, G., Matioli,E.: A manifold microchannel heat sink for ultra-high power density liquid-cooled converters. In: Erp, R., (ed.) 2019 IEEE applied power electronics conference and exposition (APEC). IEEE, Anaheim (2019)

[38] Hua,Y.C., Shen,Y., Tang,Z.L., Tang, D.S., Ran,X., Cao,B.Y.: Near-junction thermal managements of electronics. Adv. Heat Transf. 56 355–434 (2023)

[39] Maycock,P. D.: Thermal conductivity of silicon, germanium, III–V compounds and III–V alloys. Solid-State Electron. 10(3), 161–168 (1967)

[40] Kuchta,D.M.: Developments of VCSEL-based transceivers for co-packaging. In: Kuchta, D.M., (ed.) Optical fiber communication conference. Optica Publishing Group, Washington, DC (2023)

[41] Mehta,M.: An AI compute ASIC with optical attach to enable next generation scale-up architectures. In: Mehta, M., (ed.) 2024 IEEE hot chips 36 symposium (HCS). IEEE, Stanford (2024)

[42] Samalam,V.K.: Convective heat transfer in microchannels. J. Electron. Mater. 18(5), 611–617 (1989)

[43] Kim,S.J.: Methods for thermal optimization of microchannel heat sinks. Heat Transf. Eng. 25(1), 37–49 (2004)

[44] Ryu,J., Choi,D.H., Kim,S.J.: Numerical optimization of the thermal performance of a microchannel heat sink. Int. J. Heat Mass Transf. 45(13), 2823–2827 (2002)

[45] Mohammed,H., Gunnasegaran, P., Shuaib,N.: Numerical simulation of heat transfer enhancement in wavy microchannel heat sink. Int. Commun. Heat Mass Transf. 38(1), 63–68 (2011)

[46] Li,P., Luo,Y., Zhang,D., Xie, Y.: Flow and heat transfer characteristics and optimization study on the water-cooled microchannel heat sinks with dimple and pin-fin. Int. J. Heat Mass Transf. 119 152–162 (2018)

RIGHTS & PERMISSIONS

The Author(s) 2025