Beyond 2D Rasterization: Text-to-3D Sketch Generation with Direct 3D Coons Patch Optimization

Abstract

Generating 3D sketches from textual descriptions represents a fundamental yet relatively underexplored problem spanning computer vision. This task provides a compelling avenue for translating human conceptual imagination into explicit 3D geometric form. However, existing text-to-3D sketch approaches typically circumvent direct 3D reasoning by projecting 3D structures into 2D space, performing gradient-based optimization in the 2D raster space, and subsequently re-lifting the results back into 3D. Such proxy optimization pipelines introduce significant challenges, including unstable gradients, reduced geometric fidelity, and substantial computational overhead during both training and inference. To address these limitations, we propose D3Sketch, a text-to-3D sketch generation framework that performs optimization natively in 3D space, eliminating the conventional 3D → 2D → 3D optimization loop. D3Sketch takes a text prompt as input and leverages a text-to-3D model to synthesize a 3D mesh, which is treated as the optimization target. We then initialize a parametric 3D geometric template with a set of learnable vertices as the Bézier control points, whose surface is defined by a collection of Bézier curves. D3Sketch optimizes the positions of vertices to minimize the surface discrepancy between the target mesh and the optimized template. After the template optimization converges, we extract its boundary curves, which constitute the final generated 3D sketch. We theoretically prove that our D3Sketch has provably stronger representation capacity compared to existing pipeline. And we provide extensive experiments to demonstrate our D3Sketch generates view-consistent 3D sketches with superior geometric fidelity, and reducing inference time to just 6 seconds. Code will be released later.

Keywords

3D sketch generation / Bézier curves / EdgeConv / 3D design / Text-to-3D

RIGHTS & PERMISSIONS

Higher Education Press 2026

Just Accepted

This article has successfully passed peer review and final editorial review, and will soon enter typesetting, proofreading and other publishing processes. The currently displayed version is the accepted final manuscript. The officially published version will be updated with format, DOI and citation information upon launch. We recommend that you pay attention to subsequent journal notifications and preferentially cite the officially published version. Thank you for your support and cooperation.