DTFab: A Digital Twin based Approach for Optimal Reticle Management in Semiconductor Photolithography

Abstract

Photolithography is among the key phases in chip manufacturing. It is also among the most expensive with manufacturing equipment valued at the hundreds of millions of dollars. It is paramount that the process is ran efficiently, guaranteeing high resource utilization and low product cycle times. A key element in the operation of a photolithography system is the effective management of the reticles that are responsible for the imprinting of the circuit path on the wafers. Managing reticles means determining which are appropriate to mount on the very expensive scanners as a function of the product types being released to the system. Given the importance of the problem, several heuristic policies have been developed in the industry practice in an attempt to guarantee that the expensive tools are never idle. However, such policies have difficulties reacting to unforeseen events (e.g., unplanned failures, unavailability of reticles). On the other hand, the technological advance of the semiconductor industry in sensing at system and process level should be harnessed to improve on these “expert policies”. In this manuscript, we develop a system for the real time reticle management that not only is able to retrieve information from the real system, but also is able to embed commonly used policies to improve upon them. We develop a new digital twin for the photolithography process that efficiently and accurately predicts the system performance, thus allowing our system to make predictions for future behaviors as a function of possible decisions. Our results demonstrate the validity of the developed model, and the feasibility of the overall approach demonstrating a statistically significant improvement of performance as compared to the current policy.

Keywords

Semiconductor manufacturing / reinforcement learning / reticle management / digital twin

References

[1] Allgeier H, Flechsig C, Lohmer J, Lasch R, Schneider G, Zettler B (2020). Simulation-based evaluation of lot release policies in a power semiconductor facility - A case study. 2020 Winter Simulation Conference (WSC). USA.

[2] Benzoni A, Yugma C, Bect P, Planchais A (2020). Allocating reticles in an automated stocker for semiconductor manufacturing facility. 2020 Winter Simulation Conference (WSC). USA.

[3] Bertsekas D. Rollout, Policy Iteration, and Distributed Reinforcement Learning, 2020, Belmont, Massachusetts, USA: Athena Scientific

[4] Bixby R, Burda R, Miller D (2006). Short-interval detailed production scheduling in 300mm semiconductor manufacturing using mixed integer and constraint programming. The 17th Annual SEMI/IEEE ASMC 2006 Conference. USA.

[5] Bradley S, Hax A, Magnanti T (1977). Applied Mathematical Programming. Addison-Wesley, USA.

[6] Byrne P (2007). An analysis of semiconductor reticle management using discrete event simulation. Proceedings of the 2007 Summer Computer Simulation Conference. USA.

[7] Carranza R (1986). Silicon Run LITE, an overview film that introduces semiconductor manufacturing. NationalElectrical.https://sihconrun.com/our-films/silicon-run-lite/.

[8] Dantzig G, Thapa M. Linear Programming, 1997, USA: Springer.

[9] Derbyshire K (2015). EUV: Cost killer or savior? Semiconductor Engineering. https://semiengineering.com/euv-cost-killer-or-cost-savior/.

[10] De Diaz S, Fowler J, Pfund M, Mackulak G, Hickie M. Evaluating the impacts of reticle requirements in semiconductor wafer fabrication. IEEE Transactions on Semiconductor Manufacturing, 2005, 18: 622-632.

[11] EESEMI (2005). Mask and ceticles: Tools for pattern formation on semiconductor wafers. EESEMI. https://eesemi.com/masks-reticles.html.

[12] Fathi Y, Barnette K. Heuristic procedures for the parallel machine problem with tool switches. International Journal Of Production Research, 2002, 40: 151-164.

[13] Fox M (1983). Constraint-directed search: A case study of Job-Shop scheduling. PhD thesis, Carnegie-Mellon University.

[14] Hadash G, Kermany E, Carmeli B, Lavi O, Kour G, Jacovi A (2018). Estimate and replace: A novel approach to integrating deep neural networks with existing applications. ArXiv Preprint ArXiv:1804.09028.

[15] Ham A, Cho M. A practical two-phase approach to scheduling of photolithography production. IEEE Transactions on Semiconductor Manufacturing, 2015, 28: 367-373.

[16] Haße H, Li B, Weißenberg N, Cirullies J, Otto B. Digital twin for real-time data processing in logistics. Artificial Intelligence and Digital Transformation in Supply Chain Management: Innovative Approaches for Supply Chains. Proceedings of the Hamburg International Conference of Logistics (HICL), 2019, 27: 4-28.

[17] Hickie M (1999). Improving photolithography reticle management with network modeling and discrete event simulation. PhD thesis, Arizona State University.

[18] Hung Y, Chen I. A simulation study of dispatch rules for reducing flow times in semiconductor wafer fabrication. Production Planning & Control, 1998, 9: 714-722.

[19] Kim T, Kim H, Lee T, Morrison J, Kim, E (2021). On scheduling a photolithograhy toolset based on a deep reinforcement learning approach with action filter. 2021 Winter Simulation Conference (WSC). USA.

[20] Klemmt A, Lange J, Weigert G, Lehmann F, Seyfert J (2010). A multistage mathematical programming based scheduling approach for the photolithography area in semiconductor manufacturing. Proceedings of the 2010 Winter Simulation Conference. USA.

[21] Kour G, Saabne R (2014). Real-time segmentation of on-line handwritten Arabic script. 2014 14th International Conference on Frontiers in Handwriting Recognition (ICFHR). Greece.

[22] Kour G, Saabne R (2014). Fast classification of handwritten on-line Arabic characters. 2014 6th International Conference of Soft Computing and Pattern Recognition (SoCPaR). Tunisia.

[23] Leachman R, Glassey R, Solorzano J. A queue management policy for the release of factory work orders, 1988, USA: Engineering Systems Research Center, University of California Berkeley Research Report

[24] Li Y, Jiang Z, Jia W. An integrated release and dispatch policy for semiconductor wafer fabrication. International Journal of Production Research, 2014, 52: 2275-2292.

[25] Mishra S, Prakash, Tiwari M, Lashkari R. A fuzzy goal-programming model of machine-tool selection and operation allocation problem in FMS: A quick converging simulated annealing-based approach. International Journal of Production Research, 2006, 44: 43-76.

[26] Morra J (2017). At ASML, orders pile up for extreme ultraviolet lithography. Electronic Design. https://www.electronicdesign.com/technologies/embedded-revolution/article/21805342/at-asml-orders-pile-up-for-extreme-ultraviolet-lithography.

[27] Park S, Fowler J, Carlyle M, Hickie M (1999). Assessment of potential gains in productivity due to proactive reticle management using discrete event simulation. Proceedings of the 31st Conference on Winter Simulation: Simulation - A Bridge to the Future - Volume 1. Phoenix, Arizona, USA.

[28] Park I, Huh J, Kim J, Park J. A reinforcement learning approach to robust scheduling of semiconductor manufacturing facilities. IEEE Transactions on Automation Science and Engineering, 2020, 17: 1420-1431.

[29] Peters M, Puharic B (2003). Extending reticle life through better cleaning budgets. Semiconductor Manufacturing: 89613095.

[30] Raghunathan V (2019). Digital Twins vs simulation: Three key differences. Entrepreneur. https://www.entrepreneur.com/article/333645.

[31] Rai R, Kameshwaran S, Tiwari M. Machine-tool selection and operation allocation in FMS: Solving a fuzzy goal-programming model using a genetic algorithm. International Journal of Production Research, 2002, 40: 641-665.

[32] Randhawa S, Kuo C. Evaluating scheduling heuristics for non-identical parallel processors. International Journal of Production Research, 1997, 35: 969-981.

[33] Randhawa S, Zeng Y. Job shop scheduling: An experimental investigation of the performance of alternative scheduling rules. Production Planning & Control, 1996, 7: 47-56.

[34] Schoolov K. CNBC., 2022

[35] Simens (2022). PlantSight. https://new.siemens.com/global/en/products/automation/ind-ustry-software/plantsight.html.

[36] Sterling T (2022). Intel orders ASML system for well over $340 mln in quest for chipmaking edge. Reuters. https://www.reuters.com/technology/intel-orders-asml-machine-still-drawing-board-chipmakers-look-an-edge.

[37] Sun L, Chen X, Tomizuka M (2014). Selective iterative learning control to deal with iteration-dependent disturbance. Proceedings Of ISCIE/ASME International Symposium On Flexible Automation.

[38] Systèmes D. Dassault Systemes., 2021

[39] Tang C, Denardo E. Models arising from a flexible manufacturing machine, part II: Minimization of the number of switching instants. Operations Research, 1998, 36: 778-784.

[40] Turkcan A, Akturk M, Storer R. Non-identical parallel CNC machine scheduling. International Journal Of Production Research, 2003, 41: 2143-2168.

[41] Turkcan A, Akturk M, Storer R. Due date and cost-based FMS loading, scheduling and tool management. International Journal of Production Research, 2007, 45: 1183-1213.

[42] TWI Global (2022). Simulation vs digital twin (what is the difference between them?). https://www.twi-global.com/technical-knowledge/faqs/simulation-vs-digital-twin.aspx.

[43] Vitelli P (2021). The reticle allocation problem and how to approach it [tech paper review]. Flexciton. https://www.flexciton.com/single-post/the-reticle-allocation-problem-technical-paper-review.

[44] Waschneck B, Reichstaller A, Belzner L, Altenmüller T, Bauernhansl T, Knapp A, Kyek A (2018). Deep reinforcement learning for semiconductor production scheduling. 2018 29th Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC).

[45] Wikichip.com (2023). Mask / Reticle. Wikichip. https://en.wikichip.org/wiki/mask.

[46] Zeballos L. A constraint programming approach to tool allocation and production scheduling in flexible manufacturing systems. Robotics and Computer-Integrated Manufacturing, 2010, 26: 725-743.

[47] Zhang W, Dietterich T. A reinforcement learning approach to job-shop scheduling. IJCAI, 1995, 95: 1114-1120.

[48] Zhang Z, Zheng L, Weng M. Dynamic parallel machine scheduling with mean weighted tardiness objective by Q-Learning. The International Journal of Advanced Manufacturing Technology, 2007, 34: 968-980.

[49] Zhang Z, Zheng L, Li N, Wang W, Zhong S, Hu K. Minimizing mean weighted tardiness in unrelated parallel machine scheduling with reinforcement learning. Computers & Operations Research, 2012, 39: 1315-1324.