fullwrong http://arbdwj.github.io/ Thu, 02 Jul 2026 12:46:50 +0000 Thu, 02 Jul 2026 12:46:50 +0000 Jekyll v3.10.0 <p>A post on LessWrong tracing how a model’s awareness of being evaluated emerges across the training stages of OLMo 3: negligible during pretraining, rising with supervised fine-tuning, dropping with DPO, and rising again substantially during RL.</p> <p><a href="https://www.lesswrong.com/posts/c2tqL9xPbttisAHtt/tracing-eval-awareness-emergence-through-training-of-olmo-3">Read it on LessWrong →</a></p> Wed, 10 Jun 2026 12:00:00 +0000 http://arbdwj.github.io/2026/06/10/tracing-eval-awareness-olmo3/ http://arbdwj.github.io/2026/06/10/tracing-eval-awareness-olmo3/ <p>How LLMs compress text, image, and speech, and the universal power laws that govern compression ratio as a function of model and data scale.</p> <p><a href="https://www.lesswrong.com/posts/rWBYQXJTHBmCBCLEk/scaling-laws-for-llm-based-data-compression">Read it on LessWrong →</a></p> Wed, 23 Jul 2025 10:00:00 +0000 http://arbdwj.github.io/2025/07/23/scaling-compression/ http://arbdwj.github.io/2025/07/23/scaling-compression/ <p>Testing whether the Platonic Representation Hypothesis — that models across modalities converge to a shared statistical model of reality — still holds once you move beyond in-distribution data.</p> <p><a href="https://www.lesswrong.com/posts/Su2pg7iwBM55yjQdt/exploring-the-platonic-representation-hypothesis-beyond-in">Read it on LessWrong →</a></p> Sun, 27 Oct 2024 10:00:00 +0000 http://arbdwj.github.io/2024/10/27/prh-ood/ http://arbdwj.github.io/2024/10/27/prh-ood/ <p>Investigating how transformers keep reasoning when chain-of-thought steps are replaced by filler tokens, using the 3SUM task — and a modified greedy decoding scheme that recovers the hidden computation with 100% consistency.</p> <p><a href="https://www.lesswrong.com/posts/3duPFzDDX8myY6q4K/understanding-hidden-computations-in-chain-of-thought">Read it on LessWrong →</a></p> Wed, 28 Aug 2024 01:49:00 +0000 http://arbdwj.github.io/2024/08/28/fillertokens/ http://arbdwj.github.io/2024/08/28/fillertokens/ <p>An argument that solving goal misgeneralization in the worst case reduces to Eliciting Latent Knowledge: any adversarial-training scheme that relies on a non-deceiving prediction head runs straight into the ELK problem.</p> <p><a href="https://www.lesswrong.com/posts/MWSCqzPrAbNrYoqWv/adversarial-training-against-goal-misgeneralization-is-elk">Read it on LessWrong →</a></p> Fri, 24 Mar 2023 10:00:00 +0000 http://arbdwj.github.io/2023/03/24/goalmisgen/ http://arbdwj.github.io/2023/03/24/goalmisgen/ <p>On why certifying that a superintelligent system is <em>intelligent</em> first requires proving that it is <em>honest</em> — and why honesty is the hard-to-find global optimum among easy-to-find deceptive ones.</p> <p><a href="https://www.lesswrong.com/posts/hqzHbew35Jx4xoDhE/the-agi-needs-to-be-honest">Read it on LessWrong →</a></p> Sat, 16 Oct 2021 10:00:00 +0000 http://arbdwj.github.io/2021/10/16/truth/ http://arbdwj.github.io/2021/10/16/truth/