Tag: AI

All the articles with the tag "AI".

• Flash Attention in a Flash

  Published:Dec 27, 2024 |  at 02:49 AM |  in 8 min read

  Flash Attention is a new attention mechanism that can be used to speed up training and inference in large-scale AI models. This article provides an overview of Flash Attention and its applications in AI research and development.

• Estimating Transformer Model Properties: A Deep Dive

  Published:Dec 7, 2024 |  at 05:44 AM |  in 8 min read

  In this post, we'll explore how to estimate the size of a Transformer model, including the number of parameters, FLOPs, peak memory footprint, and checkpoint size.

• Neural Scaling Laws (Then and Now)

  Published:Dec 3, 2024 |  at 04:07 AM |  in 12 min read

  Scaling laws have been a cornerstone of deep learning research for decades. In this post, we'll explore the history of scaling laws in deep learning, and how they've evolved over time.

• Feeding the Beast - Data Loading Secrets for Hungry Neural Networks

  Published:Dec 3, 2024 |  at 02:53 AM |  in 11 min read

  Data loading is a critical part of training deep learning models. In this post, we'll explore the best practices for loading data into neural networks, with a focus on PyTorch.

• Can AI Write about AI Personality With Personality?

  Published:Nov 23, 2024 |  at 07:33 AM |  in 31 min read

  Should AI have "Personality" or will "Bland and Mid" suffice?

• o1 and Reasoning

  Published:Nov 20, 2024 |  at 12:53 AM |  in 62 min read

  Organizing research presumably related to OpenAI's o1 reasoning model.