LLM and agent-based tools are indispensable, but they remain difficult to master due to their novelty and the field’s constant evolution. The challenge is to provide a well-reasoned and substantiated overview—free of fantasy—to offer the keys to managing and steering a project based on these technologies. Managing “hallucinations,” controlling costs, choosing a model, testing an AI tool, and cybersecurity challenges are among the topics covered during this event and illustrated by research findings and real-world experience.

Moyens techniques

Course materials presented during the training and distributed to all participants at the end of the training; case studies and practical examples selected based on the participants’ areas of interest

Suivi de l’exécution

All trainees are required to sign in for each half-day. Assessment: Learning assessment questionnaire at the end of the training

Appréciation des résultats

Post-Training Satisfaction Survey

Objectifs pédagogiques

 

1.1 — Review of the Basics

* From Large Language Models to Agent-Based Systems: A Brief Overview of Developments Since GPT-3
* Bias and hallucinations are inherent to AI: fundamental limitations of our approaches and impacts to consider.
* The primary method for evaluating an AI tool: limitations and blind spots.
* A detailed overview of three key benchmarks and their limitations: SWE-Bench, ARC-AGI, τ2-bench
* The hidden human cost! A focus on AI pitfalls in software development and open-source projects.
* Considerations for AI-driven change management: oversight, shared information. Pitfalls of uncertainty metrics.

 
 

1.2 — Overview of the AI/Agents Ecosystem

Please note: The schedule provided here is for August 2026. This section of the training program is updated monthly.
* Anthropic: Analysis of the latest models and Claude extensions (Claude Legal). Model evolution and challenges encountered with the latest models. Focus on Claude Fable: conflict with the U.S. government, communication, usage limitations, and costs. IPO implications.
* OpenAI: A look back at the feud with Anthropic. Exploring revenue streams (advertising, marketplace). Analysis of the latest models and potential directions for development. IPO implications
* Google: Analysis of a different market positioning. Internal control over the hardware used. Differentiation of Gemma models: Diffusion, Differential Privacy, Edge AI. Other projects (robotics, World Models, etc.)
* Chinese “Open Source” models? Analysis of the DeepSeek phenomenon, with a focus on the proposed model and hardware optimization.
* Chinese “Open Source” Models? Analysis by GLM & Kimi.
* Is “Software as a Service” Doomed? Analysis and Discussion.

 
 

1.3 — AI Safety: Key Considerations

* AI Security? Overview of the Main Security Vulnerabilities in a Model: Adversarial Attacks, Jailbreaking, Prompt Injection, Model Inversion, Membership Inference, Poisoned Datasets
* AI Security in the Context of Agent-Based Systems: Overview of Publicly Discovered Vulnerabilities. Analysis of the conditions for the secure operation of an agent-based system.
* A Look Back at Antropic Mythos: AI-Based Security Vulnerability Detection. Results by Open-Source Models. Impacts to Consider.
* Case Study: Misuse of an Agent-Based Search System (RAG): Essential Precautions and Best Practices

 
 

1.4 — Solutions: Investments in Data

* AI as the Realm of Proof of Concept: Which Initiatives Constitute True Investments in AI?
* Methodological principles of data architecture: iterative processes aimed at understanding the data, whether during training or during the testing and validation of a system.
* Data architecture as a meeting point between business experts and data scientists: a case study.
* How to Test an AI Tool in a Stable and Iterative Manner? Black-Box Testing Approaches in Data Architecture
* Leveraging Synthetic Data to Expand Test Coverage.
* Automated and Customized Testing of an External Tool

 
 

1.5 — Solutions: Investments in AI

* Large-scale model training using the “black box” approach: major limitations and potential solutions.
* AI Architecture: Key Principles, Comparison with the Software Paradigm. The Value of Smaller Models in Terms of Cost and Traceability.
* Architecture-based controlled quality iteration in AI. Adapting to the various available models.
* Sovereignty and AI: Challenges and Solutions.
* Cost-control levers in AI: estimates, adjustments, and economic management of the system.

Technologies abordées

LLMs via API: Anthropic/OpenAI/Gemini/Mistral
“Open Source” LLMs: DeepSeek, Kimi, Qwen
Agent-based systems

Compétences visées

Understanding the challenges and risks associated with LLM and agent-based tools
In-depth knowledge of the ecosystem and future development paths
Defining and ensuring the quality of an AI tool
Implementing a project and related investments: data architecture and AI architecture