A diffusion-based machine learning method for 3D architectural form-finding

Abstract

Form-finding is a process in architectural design. Architects create and manipulate the morphology of a building by finding the form using digital tools and algorithms, such as machine learning. Recent research indicates that existing machine learning methods for architectural form-finding are not efficient for training and cannot generate multiple 3D forms under the constraints of users. Therefore, in this research, we develop a method to train and apply low-rank adaptation (LoRA) models in Stable Diffusion (SD) to generate 3D architectural forms based on morphological heat maps. Furthermore, the generated 3D forms can be directly used to precisely control the generation of realistic architectural renderings using pre-trained LoRA and SD models. In conclusion, our method can help architects generate 3D architectural models with consistent renderings. It can serve as a useful tool to improve efficiency and creativity in the architectural design practice of form-finding.

Keywords

Architectural form finding / 3D architectural model / Machine learning / Stable diffusion / Low-rank adaptation

References

[1] Aitrepreneur , 2023. Ultimate free lora training in stable diffusion! less than 7gb vram! Youtube.

[2] As, I. , Pal, S. , Basu, P. , 2018. Artificial intelligence in architecture: generating conceptual design via deep learning. Int. J. Architect. Comput. 16 (4), 306- 327.

[3] Battiato, S. , Curti, S. , La Cascia, M. , Tortora, M. , Scordato, E. , 2004. Depth map generation by image classification. In: ThreeDimensional Image Capture and Applications VI. SPIE, pp. 95-104.

[4] Belniak, S. , Leśniak, A. , Plebankiewicz, E. , Zima, K. , 2013. The influence of the building shape on the costs of its construction. Journal of Financial Management of Property and Construction 18 (1), 90- 102.

[5] Carpo, M. , 2017. The Second Digital Turn: Design beyond Intelligence. MIT press.

[6] Chaillou, S. , 2020. Archigan: artificial intelligence x architecture. In: Architectural Intelligence: Selected Papers from the 1st International Conference on Computational Design and Robotic Fabrication (CDRF 2019). Springer, pp. 117-127.

[7] Chatterjee, S. , Hazra, D. , Byun, Y.-C. , Kim, Y.-W. , 2022. Enhancement of image classification using transfer learning and gan-based synthetic data augmentation. Mathematics 10 (9), 1541.

[8] Ching, F.D. , 2023. Architecture: Form, Space, and Order. John Wiley & Sons.

[9] Demir, G. , Çekmis‚, A. , Yeşilkaynak, V.B. , Unal, G. , 2021. Detecting visual design principles in art and architecture through deep convolutional neural networks. Autom. ConStruct. 130, 103826.

[10] Ding, L. , Goshtasby, A. , 2001. On the canny edge detector. Pattern Recogn. 34 (3), 721- 725.

[11] Ekici, B. , Cubukcuoglu, C. , Turrin, M. , Sariyildiz, I.S. , 2019. Performative computational architecture using swarm and evolutionary optimisation: a review. Build. Environ. 147, 356- 371.

[12] Ennemoser, B. , Mayrhofer-Hufnagl, I. , 2023. Design across multiscale datasets by developing a novel approach to 3dgans. Int. J. Architect. Comput. 21 (2), 358- 373.

[13] Gal, R. , Alaluf, Y. , Atzmon, Y. , Patashnik, O. , Bermano, A.H. , Chechik, G. , Cohen-Or, D. , 2022. An image is worth one word: personalizing text-to-image generation using textual inversion. arXiv preprint arXiv: 2208.01618.

[14] Gandikota, R. , Materzynska, J. , Fiotto-Kaufman, J. , Bau, D. , 2023. Erasing concepts from diffusion models. arXiv preprint arXiv: 2303.07345.

[15] Grobman, Y.J. , Yezioro, A. , Capeluto, I.G. , 2009. Computer-based form generation in architectural designda critical review. Int. J. Architect. Comput. 7 (4), 535- 553.

[16] Hiesinger, K.B. , Hadid, Z. , Schumacher, P. , 2011. Zaha Hadid: Form in Motion, vol. 2011. Philadelphia Museum of Art, Philadelphia, PA.

[17] Hu, E.J. , Shen, Y. , Wallis, P. , Allen-Zhu, Z. , Li, Y. , Wang, S. , Wang, L. , Chen, W. , 2021. Lora: low-rank adaptation of large language models. arXiv preprint arXiv: 2106.09685.

[18] Huang, S. , Zheng, H. , 2022. Morphological regeneration of the industrial waterfront based on machine learning. In: Proceedings of the 27th International Conference on Computer-Aided Architectural Design Research in Asia (CAADRIA), pp. 475-484.

[19] Huang, W. , Zheng, H. , 2018. Architectural drawings recognition and generation through machine learning. In: Proceedings of the 38th Annual Conference of the Association for Computer Aided Design in Architecture (ACADIA), pp. 156-165.

[20] Isola, P. , Zhu, J.-Y. , Zhou, T. , Efros, A.A. , 2017. Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125-1134.

[21] Jain, A. , Mildenhall, B. , Barron, J.T. , Abbeel, P. , Poole, B. , 2022. Zero-shot text-guided object generation with dream fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 867-876.

[22] Karras, T. , Aittala, M. , Hellsten, J. , Laine, S. , Lehtinen, J. , Aila, T. , 2020. Training generative adversarial networks with limited data. Adv. Neural Inf. Process. Syst. 33, 12104- 12114.

[23] Kim, F.C. , Johanes, M. , Huang, J. , 2023. Text2form diffusion: framework for learning curated architectural vocabulary. In: 41st Conference on Education and Research in Computer Aided Architectural Design in Europe, eCAADe 2023, pp. 79-88. Education and research in Computer Aided Architectural Design in Europe.

[24] Lu, R.T.W. , 2023. Sentiments Re-imaged by AI. PhD thesis. New York University Tandon School of Engineering.

[25] Lu, Y. , Wu, W. , Geng, X. , Liu, Y. , Zheng, H. , Hou, M. , 2022. Multiobjective optimization of building environmental performance: an integrated parametric design method based on machine learning approaches. Energies 15 (19), 7031.

[26] Lynn, G. , 2013. Folding in architecture (1993). The Digital Turn in Architecture 1992-2012 28-47.

[27] Ma, H. , 2023. Text semantics to image generation: a method of building facades design base on stable diffusion model. arXiv preprint arXiv: 2303.12755. McNeel, 2023. Rhino-rhinoceros 3d.

[28] Miao, Y. , Koenig, R. , Knecht, K. , 2020. The development of optimization methods in generative urban design: a review. In: Proceedings of the 11th Annual Symposium on Simulation for Architecture and Urban Design, pp. 1-8.

[29] MinusType , 2023. modernarchi15 stable diffusion lora civitai.

[30] Newton, D. , 2019. Generative deep learning in architectural design. Technology Architecture+ Design 3 (2), 176- 189.

[31] Oh, S. , Jung, Y. , Kim, S. , Lee, I. , Kang, N. , 2019. Deep generative design: integration of topology optimization and generative models. J. Mech. Des. 141 (11), 111405.

[32] Park, J.-H. , 2000. Subsymmetry analysis of architectural designs: some examples. Environ. Plann. Plann. Des. 27 (1), 121- 136.

[33] Ploennigs, J. , Berger, M. , 2023. Ai art in architecture. AI in Civil Engineering 2 (1), 8.

[34] Rabagliati, J. , Huber, C. , Linke, D. , 2014. Balancing complexity and simplicity. Fabricate 2014: Negotiating Design and Making 44-51.

[35] Retsin, G. , 2019. Discrete: Reappraising the Digital in Architecture. John Wiley & Sons, ISBN 978-1-119-50034-6.

[36] Richardson, E. , Metzer, G. , Alaluf, Y. , Giryes, R. , Cohen-Or, D. , 2023. Texture: text-guided texturing of 3d shapes. arXiv preprint arXiv: 2302.01721.

[37] Roich, D. , Mokady, R. , Bermano, A.H. , Cohen-Or, D. , 2022. Pivotal tuning for latent-based editing of real images. ACM Trans. Graph. 42 (1), 1- 13.

[38] Rombach, R. , Blattmann, A. , Lorenz, D. , Esser, P. , Ommer, B. , 2022. High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684-10695.

[39] Ruiz, N. , Li, Y. , Jampani, V. , Pritch, Y. , Rubinstein, M. , Aberman, K. , 2022. Dreambooth: fine tuning text-to-image diffusion models for subject-driven generation. arXiv preprint arXiv: 2208.12242.

[40] Shen, J. , Liu, C. , Ren, Y. , Zheng, H. , 2020. Machine learning assisted urban filling. In: Proceedings of the 25th International Conference on Computer-Aided Architectural Design Research in Asia (CAADRIA), pp. 679-688. SketchUp, 2023. 3d warehouse.

[41] Steinfeld, K. , Park, K. , Menges, A. , Walker, S. , 2019. Fresh eyes: a framework for the application of machine learning to generative architectural design, and a report of activities at smartgeometry 2018. In: Computer-Aided Architectural Design. "Hello, Culture" 18th International Conference, CAAD Futures 2019, Daejeon, Republic of Korea, June 26-28, 2019, Selected Papers 18. Springer, pp. 32-46.

[42] Sun, H. , Burton, H.V. , Huang, H. , 2021. Machine learning applications for building structural design and performance assessment: state-of-the-art review. J. Build. Eng. 33, 101816.

[43] Sun, Y. , Yang, C.-H. , Lyu, Y. , Lin, R. , 2022. From pigments to pixels: a comparison of human and ai painting. Appl. Sci. 12 (8), 3724.

[44] Tian, R. , 2021. Suggestive site planning with conditional gan and urban gis data. In: Proceedings of the 2020 DigitalFUTURES: the 2nd International Conference on Computational Design and Robotic Fabrication (CDRF 2020). Springer, pp. 103-113.

[45] Wu, X. , 2022. Creative painting with latent diffusion models. arXiv preprint arXiv: 2209.14697.

[46] Zhang, H. , Huang, Y. , 2021. Machine learning aided 2d-3d architectural form finding at high resolution. In: Proceedings of the 2020 DigitalFUTURES: the 2nd International Conference on Computational Design and Robotic Fabrication (CDRF 2020). Springer, pp. 159-168.

[47] Zhang, L. , Agrawala, M. , 2023. Adding conditional control to text-to-image diffusion models. arXiv preprint arXiv: 2302.05543.

[48] Zhao, H.-x. , Magoulès, F. , 2012. A review on the prediction of building energy consumption. Renew. Sustain. Energy Rev. 16 (6), 3586- 3592.

[49] Zheng, H. , 2020. Form finding and evaluating through machine learning. In: Architectural Intelligence. Springer, pp. 207-217.

[50] Zheng, H. , Moosavi, V. , Akbarzadeh, M. , 2020. Machine learning assisted evaluations in structural design and construction. Autom. ConStruct. 119, 103346.

[51] Zheng, H. , Ren, Y. , 2020a. Architectural layout design through simulated annealing algorithm. In: Proceedings of the 25th International Conference on Computer-Aided Architectural Design Research in Asia (CAADRIA), pp. 275-284.

[52] Zheng, H. , Ren, Y. , 2020b. Machine learning neural networks construction and analysis in vectorized design drawings. In: Proceedings of the 25th International Conference on Computer-Aided Architectural Design Research in Asia (CAADRIA), pp. 707-716.

[53] Zheng, H. , Yuan, P.F. , 2021. A generative architectural and urban design method through artificial neural networks. Build. Environ. 205, 108178.

[54] Zhou, Q. , Jacobson, A. , 2016. Thingi10k: a dataset of 10,000 3D-printing models. arXiv preprint arXiv: 1605.04797.

RIGHTS & PERMISSIONS

The Author(s). Publishing services by Elsevier B.V. on behalf of Higher Education Press and KeAi.