NOESIS

Implementation status

The thesis in one paragraph

Reasoning LLMs (o1, R1, the Claude Extended-Thinking family) have shown that test-time compute scaling works: spending more tokens “thinking” reliably improves accuracy on hard problems. The current production form, CoT in token space, has a fundamental inefficiency: most thinking tokens are linguistic glue (transitions, hedges, restatements) rather than load-bearing reasoning steps. Coconut (Hao et al., NeurIPS 2024) attacks this directly: instead of decoding each reasoning step to a word token, feed the last hidden state back as the next input embedding — stay in latent space. NOESIS makes latent reasoning practical by combining four ideas: continuous-thought backbone, learned adaptive think-budget, latent verifier, and a clean continuous-thought RL formulation. The result is a reasoning model that thinks in latent space, knows when to stop thinking, verifies its conclusions, and can be RL-trained end-to-end.

Architecture in one figure

           prompt embeddings
                  │
                  ▼
        ┌─────────────────────┐
        │ Transformer encoder │
        └─────────┬───────────┘
                  │
                  ▼
        ┌──────────────────────┐
        │ Adaptive K-thoughts  │  policy π_K: how many latent
        │ policy π_K(K | h_0)  │  steps to spend? K ∈ {1,2,4,…}
        └─────────┬────────────┘
                  │  K
                  ▼
        ┌──────────────────────┐
        │ Latent thought loop  │  for k=1..K:
        │ (Coconut + noise)    │    h_k = LM(h_{k-1}) + σ ε
        │                      │    (noise ε enables clean RL)
        └─────────┬────────────┘
                  │
                  ▼
        ┌──────────────────────┐
        │ Latent verifier      │  confidence(h_1,…,h_K) → c
        │ (small MLP)          │  if c < τ: extra thinking round
        └─────────┬────────────┘
                  │
                  ▼
              answer head
                  │
                  ▼
              y_1, y_2, …

The reference implementation in Appendix B of the master prompt is the stochastic latent loop with REINFORCE trajectory_log_prob. The score-function gradient d/dμ log p = ε / σ matches both the analytical form and a properly-set-up finite-difference numerical gradient. The σ-gradient correctly comes only from the log-normalization term — a subtle bug caught and fixed in v1 of the reference during development.

Key contributions

• Continuous-thought backbone (Coconut-style) — reasoning happens in the residual stream, not the token stream.
• Learned adaptive think-budget — a small policy decides how many latent steps to spend, trained with RL where the reward is accuracy − λ · steps.
• Latent verifier — a separate small network reads the continuous-thought trajectory and emits a confidence score; low confidence triggers an extra round of thinking (the latent-space analog of self-consistency).
• Continuous-thought RL — a clean policy-gradient formulation through continuous thoughts. Treats the latent trajectory as a stochastic process whose log-probability is derived from injected perturbation noise. Unlocks R1-style post-training without converting thoughts to tokens.

Phased plan

Required reading

1. Hao et al. 2024/2025 — Coconut (arxiv 2412.06769 v3) — the anchor
2. Pfau et al. 2024 — “Let's Think Dot by Dot” (latent tokens as null padding boost reasoning)
3. Goyal et al. 2024 — Pause tokens / Filler tokens
4. Hu et al. 2024 — Quiet-STaR (self-taught reasoning with rationales)
5. Akyürek et al. 2024 — TTT for few-shot reasoning
6. OpenAI 2024 — o1 system card; DeepSeek 2025 — R1 (discrete-CoT-with-RL baseline)

Target marquee result goal — not yet measured