DAY 1 – Architecture and Training of Large Language Models
1.1 — Introduction and Background
* Early language models: the Word2Vec approach (Milokov et al.), CharRNN (Karpathy)
* The 2017 “war” between convolutional and recurrent models, before the arrival of the Transformer
* Adaptation of the Transformer architecture (Vaswani et al.): positional embeddings, attention mechanism, autoregressive generation, normalization, and the overall encoder-decoder architecture
1.2 — Position Embeddings and Attention Mechanisms
* The challenge of position embedding depending on the specific task.
* Presentation of Rotary Positional Embedding (Su et al), mechanism and results. Presentation of “Attention with Linear Biases” (ALiBi, Press et al)
* Bottleneck in attention computation and quadratic complexity. Overview of approaches used in time series forecasting
* Focus on “Flash Attention” approaches (Dao et al., Shah et al.). Presentation of three incremental studies, highlighting the advances and results achieved.
* Approaches proposed by DeepSeek: Multihead Latent Attention (v2), DeepSeek Sparse Attention (v3)
1.3 — Mixture of Experts: Balancing Model Size and Inference Speed
* Dynamic Load Adaptation: Overview of the Comprehensive Approach, Example of Universal Transformers
* Initial MoE and foundational work: Switch Transformers, GShard
* Foundational principles of Mixture of Experts approaches: selection (Top-K Routing, Soft Routing), architecture (Hierarchical, Orthogonal, Calibrated MoE)
* DeepSeek MOE Study (Dai et al.)
1.4 — Architecture Case Study: DeepSeek v3
* Walkthrough of the DeepSeek v3 architecture components in relation to the points covered in court to gain a comprehensive understanding of the model.
1.5 — Training a Large Language Model
* Initial Training of the Transformer: Implementation and Limitations
* Overview of the Different Training Phases of an LLM Today. Importance of pretraining (general) versus specialized training (task-specific)
* Deep Reinforcement Learning: introduction to the fundamentals: modeling, reward, overall objective, policy function.
* Reinforcement Learning via Human Feedback (Ouyang et al): overview of research since Proximal Policy Optimization. Implementation and limitations.
* Direct Preference Optimization & Reinforcement Learning with Verifiable Rewards: algorithms and results (Lambert et al)
* DeepSeek-Math: overview of Group Relative Policy Optimization.
1.6 — Architecture Case Study: DeepSeek-R1
* Analysis of the model’s architecture and training in relation to the points discussed in class to gain a comprehensive understanding.
DAY 2 – The Agent-Based Paradigm: Optimization, Safety, and Interpretability of LLMs
2.1 — From LLM to Agent-Based Systems
* The principle of iteration in LLMs
* Introduction to tools in LLM research: application to software development, with a focus on the AutoCodeRover approach (Zhang et al). Ways to invoke a tool (MCP, CLI, code).
* Iteration and source of truth: illustration via the AlphaGeometry approach (Trinh et al.) to isolating the LLM as an exploration engine when compared to a symbolic engine
* The origins of agents: a look back at Stanford’s work (Generative Agents: Interactive Simulacra of Human Behavior), challenges in information propagation and memory management
* From Chain of Thought to agent-based planning: principles and applications
2.2 — Adapting LLMs for Agent-Based Applications
* Case Study: Agentic Specificities for the GLM-5 Model
* Case Study: DeepSeek-v4, and RAG vs. Agentic Search
* The Memory Bottleneck in an Agentic System: A Study of Various Memory Systems, Each with Their Known Limitations. Focus on “A-MEM” (Xu et al.) and critical analysis.
2.3 — Optimization and Fine-Tuning of an LLM
* Simple optimizations in deep learning. Differences in computational requirements between training and inference. Impacts of quantization
* Optimization and fine-tuning approaches: “Low-Rank Factorization” (LoRA) and distillation. Application examples.
* Inference Optimization: Limitations of the autoregressive model. Challenges of the “Key-Value Cache” (KV Cache). Optimizations at the token, model, or system level
* Case Study: TurboQuant.
2.4 — Interpretability of LLMs
* Interpretability in Deep Learning: A Review of the Context and Major Challenges Facing Neural Networks. Issues with Embeddings, Limitations of Observing Attention
* Sparse Autoencoding Approaches: Principles, Observable Results, and Limitations.
* Circuit Extraction from a Model: Approach and Results
* Focus on Anthropic’s Jacobian-Based Approach: “J-Space”
* Taking a Step Back to Examine Internal Interpretability and the Pursuit of Traceability in an AI System
2.5 — LLM Security
* An Introduction to Adversarial Attacks and Their Applications to Large Language Models.
* Prompt injection and jailbreaking: an overview of various approaches. Mitigation and testing solutions. Presentation of NVIDIA’s “Garak” tool (LLM PenTesting)
* Data protection and LLMs: extraction of training data (Model Inversion / Membership Inference), or poisoning of a training/fine-tuning dataset.
* Taking a step back and gaining a holistic view of the security of an agent-based system. Recommendations.
2.6 — Future Developments in the Models
* General Knowledge Track: World Models as a Complement to LLMs. State-Space Architectures (Mamba). Large Diffusion Language Models. Overview of the current status and future prospects.