Probabilistic circuits maintain uncertainty instead of…
0 reactions
Probabilistic circuits maintain uncertainty instead of collapsing it
There's a paper from UAI 2024 that really caught my attention about Addition As Int (AAI) — approximating floating-point multiplication as integer addition to make probabilistic circuits run on milliwatt devices. That's 357-649× energy reduction compared to right. What does that mean? Real-time, streaming, stateless inferencing in your smartphone. Or, honestly, something even smaller.
But to me, the more interesting part is what probabilistic circuits actually do differently from neural networks:
**Neural networks:** Compute through layers → collapse to single output at softmax → probability distribution is gone
**Probabilistic circuits:** The circuit IS the distribution. You can query from any angle:
* P(disease | symptoms) — diagnosis
* P(symptoms | disease) — what to expect
* P(disease AND complication) — joint probability
* MAP query — most likely explanation
Product nodes only connect independent variables. The structure guarantees that the covariance "ghost" is zero by construction.
This matters for:
* **Explainability:** The circuit topology IS the explanation
* **Edge AI:** Milliwatt-scale reasoning under uncertainty
* **AI-to-AI negotiation:** Two PCs can share calibrated distributions, not just point estimates
* **Missing data:** Handle gracefully without imputation
I wrote up the connection between covariance, factorization, and why brains might work similarly — maintained uncertainty as continuous process rather than compute-collapse-output.
Paper: Yao et al., "On Hardware-efficient Inference in Probabilistic Circuits" (UAI 2024) https://proceedings.mlr.press/v244/yao24a.html
Full post: https://www.williamsoutherland.com/tech/ghost-in-the-formula-probabilistic-circuits/
#technology
source
There's a paper from UAI 2024 that really caught my attention about Addition As Int (AAI) — approximating floating-point multiplication as integer addition to make probabilistic circuits run on milliwatt devices. That's 357-649× energy reduction compared to right. What does that mean? Real-time, streaming, stateless inferencing in your smartphone. Or, honestly, something even smaller.
But to me, the more interesting part is what probabilistic circuits actually do differently from neural networks:
**Neural networks:** Compute through layers → collapse to single output at softmax → probability distribution is gone
**Probabilistic circuits:** The circuit IS the distribution. You can query from any angle:
* P(disease | symptoms) — diagnosis
* P(symptoms | disease) — what to expect
* P(disease AND complication) — joint probability
* MAP query — most likely explanation
Product nodes only connect independent variables. The structure guarantees that the covariance "ghost" is zero by construction.
This matters for:
* **Explainability:** The circuit topology IS the explanation
* **Edge AI:** Milliwatt-scale reasoning under uncertainty
* **AI-to-AI negotiation:** Two PCs can share calibrated distributions, not just point estimates
* **Missing data:** Handle gracefully without imputation
I wrote up the connection between covariance, factorization, and why brains might work similarly — maintained uncertainty as continuous process rather than compute-collapse-output.
Paper: Yao et al., "On Hardware-efficient Inference in Probabilistic Circuits" (UAI 2024) https://proceedings.mlr.press/v244/yao24a.html
Full post: https://www.williamsoutherland.com/tech/ghost-in-the-formula-probabilistic-circuits/
#technology
source
| 4,000 mlx | |
| 8 | |
| 0 |