A Physical-Structure-Driven Back-Propagation Framework for Deep Networks
CHEN Shengchang
A Physical-Structure-Driven Back-Propagation Framework for Deep Networks
A Physical-Structure-Driven Back-Propagation Framework for Deep Networks
摘要
Back-Propagation (BP) computes exact gradients of differentiable computational graphs via the chain rule. Its reverse sweep, however, transports products of transposed weight matrices and activation derivatives, and is sequential across depth. This paper proposes Physics-Structure-Driven Back-Propagation (PSDBP), an alternative credit-assignment framework motivated by the interpretation of a layered network as a structured physical system. PSDBP consists of three structural projections: adjoint demand transport ($W^T$), projected constitutive inversion ($\sigma^{-1}$ with range projection), and least-squares approximate inverse projection ($X^T$). We establish a precise relationship between these three operations and the Alternating Direction Method of Multipliers (ADMM): in the strong-penalty limit, the ADMM iterations for a lifted constrained formulation reduce to the three PSDBP projections, a relationship that serves as a theoretical justification for the architecture rather than a proof of convergence. We also prove three rigorous conditional results: the local inverse gain scales as the reciprocal constitutive slope, the transported demand contains no activation-derivative factors but remains sensitive to weight singular values, and the single-sample update is non-negatively aligned with the derivative-scaled local update. None of these results implies global descent or immunity to weight-induced signal attenuation. The framework establishes a structural credit-assignment mechanism that completely replaces the chain-rule-based gradient propagation of BP with three physically motivated projections: adjoint demand transport via the weight transpose, constitutive inversion via the inverse activation function, and weight updates via least-squares projection onto the input. This structural perspective provides a self-contained mathematical foundation for credit assignment in deep networks, orthogonal to the differential calculus underlying BP.