Tag: AI

All the articles with the tag "AI".

• Swirling Thoughts on AI, LLMs, Novel Hypothesis Generation and Crypto?

  Published:Apr 3, 2025 |  at 09:45 PM |  in 14 min read

  In this post, I swirl my own thoughts on the very topic of swirling thoughts in the context of AI, LLMs and their potential for novel hypothesis generation. I ground this discussion in some of the other ongoing parallel lines of thoughts/investigation including GRPO, reinforcement learning, the future of science and scientific peer review and the notion of being able to "purchase" scientific innovation.

• Multistep Reasoning Agents (with GRPO & RLEF) - Project Euler Edition

  Published:Mar 23, 2025 |  at 02:38 AM |  in 21 min read

  Converting chat LLMs into reasoning agents through GRPO and reinforcement learning from execution feedback — achieving performance improvements on Project Euler's algorithmic challenges with multi-step reasoning, tool use, and code execution.

• Flash Attention in a Flash

  Published:Dec 27, 2024 |  at 02:49 AM |  in 8 min read

  Flash Attention is a new attention mechanism that can be used to speed up training and inference in large-scale AI models. This article provides an overview of Flash Attention and its applications in AI research and development.

• o3 and the Future of Science

  Published:Dec 23, 2024 |  at 10:04 PM |  in 23 min read

  o3 and Human-AI scientific collaboration

• Estimating Transformer Model Properties: A Deep Dive

  Published:Dec 7, 2024 |  at 05:44 AM |  in 8 min read

  In this post, we'll explore how to estimate the size of a Transformer model, including the number of parameters, FLOPs, peak memory footprint, and checkpoint size.

• Neural Scaling Laws (Then and Now)

  Published:Dec 3, 2024 |  at 04:07 AM |  in 12 min read

  Scaling laws have been a cornerstone of deep learning research for decades. In this post, we'll explore the history of scaling laws in deep learning, and how they've evolved over time.