The authors present the first continuous-variable (CV) thermodynamic computer and a stochastic processing unit (SPU) that fits on a circuit board. The SPU can sample from an 8-dimensional Gaussian distribution and perform matrix inversion. The authors also implemented Gaussian process regression and uncertainty quantification through spectral-normalized neural Gaussian processes for neural network class prediction on the SPU. They expect that on a large scale, the SPU will have advantages over classical hardware, particularly in tasks involving a large number of dimensions and in terms of energy consumption.
Key takeaways:
- The work from Normal Computing introduces a new class of hardware: the thermodynamic computer and stochastic processing unit (SPU), which successfully implements one of the fundamental operations in linear algebra widely used in ML, matrix inversion.
- The fundamental building block of Thermodynamic AI hardware is the stochastic unit (s-unit) - a continuous variable undergoing Brownian motion, which can be implemented on an analog electrical circuit with a noisy resistor and capacitor.
- The authors aim to unify modern AI algorithms, many of which use stochasticity and are inspired by physics, based on thermodynamics. Examples of thermodynamic algorithms include generative diffusion models, Hamiltonian Monte Carlo, and simulated annealing.
- The authors created a stochastic processing unit (SPU) that fits on a circuit board and can sample from an 8-dimensional Gaussian distribution. It also implemented Gaussian process regression and Uncertainty quantification through spectral-normalized neural Gaussian processes for neural network class prediction.