A Unified Lightweight Compute Engine for Softmax, GELU, and SiLU Based on a Shared Exponential Unit

i

This article has been updated. A Correction to this article was published on .

i

The Original Article was published on

i

The Original Article was published on

i

This article has been updated

i

This article was retracted on

i

This article has been updated

Abstract

Transformer inference evaluates Softmax in attention and GELU or SiLU in feed-forward blocks. When GELU is approximated in the sigmoid form z σ(1.702z) of the original GELU paper, it differs from SiLU only in the scale of the sigmoid argument; both activations then reduce to z • softmax_2,1 ( [k, 0]) and share one log-domain softmax datapath with Softmax, extending the Softmax/GELU reuse of Peltekis et al. to a third operator. The mapping shrinks the pre-scaling front end from three 32 × 32 multipliers to one constant multiplication. The datapath uses a 16-segment minimax piecewise-linear exponential, a Mitchell logarithm with a truncated quadratic correction, and signed-digit constants; an optional time-division-multiplexed (TDM) configuration halves the exponential cores. The accuracy gains stem from this arithmetic, not from the operator unification. In SMIC 55 nm at N = 8, against a same-library reconstruction of the dual-mode reuse baseline, the full-throughput and TDM configurations are 33.1% and 47.8% smaller (the latter at half the throughput), close timing at 15 ns, whereas the reconstruction fails at 16 ns, and draw 44–57% less dynamic power under identical activation traces. The area savings persist in open 7 nm and 45 nm libraries. A gated mode adds GeGLU and SwiGLU; on an FPGA board all five operators run at 66.7 MHz without a mismatch in 1.8 × 10¹⁰ vector comparisons.

Keywords

Transformer accelerator / Softmax / GELU / SiLU / Sigmoid approximation / Hardware reuse / Time-division multiplexing (TDM) / Fixed-point arithmetic

CHANGE HISTORY

A Correction to this paper has been published:
https://doi.org

The Retraction Note was updated to reflect reception of statements of disagreement from all authors.

This article has been retracted.Please see the Retraction Notice for more detail:
https://doi.org

Rights & permissions

Higher Education Press 2026

Just Accepted

This article has successfully passed peer review and final editorial review, and will soon enter typesetting, proofreading and other publishing processes. The currently displayed version is the accepted final manuscript. The officially published version will be updated with format, DOI and citation information upon launch. We recommend that you pay attention to subsequent journal notifications and preferentially cite the officially published version. Thank you for your support and cooperation.