|国家预印本平台
首页|N-TORC: Native Tensor Optimizer for Real-time Constraints

N-TORC: Native Tensor Optimizer for Real-time Constraints

N-TORC: Native Tensor Optimizer for Real-time Constraints

来源:Arxiv_logoArxiv
英文摘要

Compared to overlay-based tensor architectures like VTA or Gemmini, compilers that directly translate machine learning models into a dataflow architecture as HLS code, such as HLS4ML and FINN, generally can achieve lower latency by generating customized matrix-vector multipliers and memory structures tailored to the specific fundamental tensor operations required by each layer. However, this approach has significant drawbacks: the compilation process is highly time-consuming and the resulting deployments have unpredictable area and latency, making it impractical to constrain the latency while simultaneously minimizing area. Currently, no existing methods address this type of optimization. In this paper, we present N-TORC (Native Tensor Optimizer for Real-Time Constraints), a novel approach that utilizes data-driven performance and resource models to optimize individual layers of a dataflow architecture. When combined with model hyperparameter optimization, N-TORC can quickly generate architectures that satisfy latency constraints while simultaneously optimizing for both accuracy and resource cost (i.e. offering a set of optimal trade-offs between cost and accuracy). To demonstrate its effectiveness, we applied this framework to a cyber-physical application, DROPBEAR (Dynamic Reproduction of Projectiles in Ballistic Environments for Advanced Research). N-TORC's HLS4ML performance and resource models achieve higher accuracy than prior efforts, and its Mixed Integer Program (MIP)-based solver generates equivalent solutions to a stochastic search in 1000X less time.

Suyash Vardhan Singh、Iftakhar Ahmad、David Andrews、Miaoqing Huang、Austin R. J. Downey、Jason D. Bakos

计算技术、计算机技术

Suyash Vardhan Singh,Iftakhar Ahmad,David Andrews,Miaoqing Huang,Austin R. J. Downey,Jason D. Bakos.N-TORC: Native Tensor Optimizer for Real-time Constraints[EB/OL].(2025-04-06)[2025-07-21].https://arxiv.org/abs/2504.04661.点此复制

评论