Haru

Ap­prox­i­mate Nat­u­ral La­tents Have Ex­act Prices

• Haru 19 Jun 2026 23:19 UTC

  1 point

  0

  in reply to: Salmonus_Kim’s comment on: In­tro­duc­tion: Gaus­sian Nat­u­ral Latents

  Thanks for reading! The exact formula exists, but it might not be explicitly written out in any future post. So, here’s an incomplete exposition which will make more sense when the later posts go up.
  
  Write as the waterline set by the mediation-error budget .
  
  I claim^[1] it must be the unique solution of , where modes with are active and are brought down to the common conditional correlation , while modes with contribute their full dependence to the mediation budget.
  
  If we want the exact formula for , choose a set of active modes without loss of generality.
  We fix the condition . Then, the threshold equation is

  
  
  Rearranging, the exact formula comes out to

  
  
  The annoying part is that you have to choose the right . If you choose the wrong one, it won’t satisfy the condition, so you try until it does (which is why it’s nicer to just box the implicit budget equation).

  1. ^

     This I’ll explain in a later post. It’s essentially a sum of the leftover mutual information in each canonical mode, and it implicitly determines the waterline.

• Haru 18 Jun 2026 20:30 UTC

  3 points

  0

  in reply to: cubefox’s comment on: Two Clas­si­cal An­swers to “What do Two Vari­ables Share?”

  Thanks for the question, I should have been thinking more about pedagogy. Consider a situation where Alice and Bob have two different camera feeds observing the same room, as below.
  

  

  
  Alice’s feed is , Bob’s is . They are correlated, and the shared latent variable is whatever’s actually going on inside the room.
  
  Gács–Körner asks if Alice and Bob can each apply some function to their own camera feed—without communicating—and be certain they get the same answer? The answer is basically no. Alice sees the front of the elephant; Bob sees the back. Any image Alice sees is compatible with many possible images on Bob’s screen, and vice versa, because the feeds are a lossy projection of the underlying latent. The only functions guaranteed to agree are constant ones which carry no information. That’s what means here.
  
  Mutual Information measures how much one feed tells you about the other statistically. In this example, MI is large: if Alice see the front of an elephant through her feed, she can infer a lot about what Bob’s feed should look like. It quantifies how many bits of uncertainty about Bob’s feed get eliminated by seeing Alice’s.
  
  Wyner asks: what’s the simplest explanation that fully accounts for the correlation? I.e., what variable do you need such that once you know , Alice’s feed tells you nothing further about Bob’s? Here, I think you need the full data-generating process (exact position of the elephant to a high degree of precision) which includes information hidden from both feeds. This is way more information than the two feeds share, because the explanation has to cover everything that could have generated correlated observations, not just what did.
  
  I just published another post, the introduction to my research direction, where I talk about why I think this is relevant!

  What links here?

  • Ap­prox­i­mate Nat­u­ral La­tents Have Ex­act Prices by Haru (4 Jul 2026 1:57 UTC; 25 points)

In­tro­duc­tion: Gaus­sian Nat­u­ral Latents

Two Clas­si­cal An­swers to “What do Two Vari­ables Share?”