Published
Updated
This guide helps you choose between RAG, fine-tuning and simply improving the prompt for a business use. It assumes the use has been chosen: the guide prioritising AI use cases covers that choice.
Key takeaways
RAG gives the model facts that change; fine-tuning changes a behaviour or a format.
Try a better-written prompt first, and measure each option on the same evaluation set.
A RAG system is only as good as its documents: quality, updates and access rights determine the result.
What is the difference between RAG and fine-tuning?
RAG (in French) retrieves the relevant passages from your documents at the moment the question is asked and passes them to the model. Fine-tuning (in French) retrains the model on your examples and changes its parameters. The authors of RAG (Lewis et al., 2020) credit it with two advantages: knowledge that is easier to update and better traceability of sources. OpenAI’s guide on model accuracy likewise distinguishes two dimensions. RAG addresses context, and therefore the accuracy of answers; fine-tuning addresses behaviour, and therefore its consistency.
Should you start by improving the prompt?
Yes: a better-written prompt often solves the problem, at a much lower cost and in much less time. Specify the role, the expected format and a few examples of good answers: the prompt engineering (in French) page details these techniques. At the same time, build an evaluation set (in French): real questions and their expected answers. OpenAI’s guide suggests gathering at least twenty or so. This set will serve to compare all the options, then to check every update.
When should you choose RAG?
Choose RAG when the model lacks information that is specific to your company or that changes: procedures, prices, contracts, product sheets. A study by Ovadia and co-authors (EMNLP 2024) compares the two methods for teaching facts to a model. RAG outperforms unsupervised fine-tuning, including for new knowledge.
A RAG system is only as good as its documents. Remove outdated versions and duplicates, and appoint an owner for each source: the data quality and governance (in French) page describes these rules. Apply existing access rights. OWASP describes the risk of a RAG system returning to a user documents that they are not entitled to read. ANSSI (the French national cybersecurity agency) asks for need-to-know to be addressed from the design stage (recommendation R8).
RAG does not eliminate errors. Magesh and co-authors (Journal of Empirical Legal Studies, 2025) tested legal research tools built on RAG. According to this article, the tools from LexisNexis and Thomson Reuters gave between 17% and 33% incorrect or poorly sourced answers. The hallucination (in French) page explains why review remains necessary.
Organisation and assessments
Make organisational decisions with a clear assessment
Our assessments identify practical changes to help your teams and processes work together.
When should you choose fine-tuning?
Choose fine-tuning when the problem lies in the model’s behaviour: a format to follow strictly, a consistent tone, a classification repeated thousands of times. According to the same Ovadia study, unsupervised fine-tuning struggles to learn new facts. OpenAI’s guide advises starting with around fifty high-quality examples, then validating on a separate set.
Check the offer before you commit. On 7 May 2026, OpenAI announced the phased end of its self-serve fine-tuning. New customers no longer have access to it. Its active customers will no longer be able to start a new training run from 6 January 2027. Since 2 July 2026, customers who have not used any fine-tuned model in the last 60 days have already been excluded. Models already fine-tuned remain usable until their base model is retired. OpenAI plans to retire GPT-4o and GPT-4.1 on 14 January 2027.
On the AI Act side, fine-tuning rarely makes you the provider of a general-purpose AI model. According to the Commission’s FAQ (update of 11 November 2025), this requires using more than one third of the compute of the original training. Article 25 still needs to be checked if the system is high-risk.
What are the security risks for a RAG system?
The main risk comes from the documents themselves: a booby-trapped text can contain instructions that the model executes. In its Top 10 risks for LLM applications, published on 3 August 2026, OWASP ranks prompt injection (in French) first. This risk includes the indirect form, through documents, emails or web pages. The danger increases if the system can act, for example send an email: the guide AI agents in business covers this case. Limit sources to documents whose origin you know, and keep human approval before any action.
What does maintaining each method cost?
A RAG system requires continuous upkeep: updating the index, removing outdated documents, checking access rights, running the evaluation set again. A fine-tuned model requires new training every time the base model or the expected format changes. ANSSI recommends never retraining a model directly in production (recommendation R21). In both cases, put a figure on your teams’ time, not just on the supplier’s invoice.
How do you decide for your use cases?
Start from the problem observed on the evaluation set, then choose the method that addresses it. This grid is a Spentia proposal, with no success statistics or client portfolio claimed.
Fictional exercise. An industrial mid-sized company examines four uses.
Use | Problem observed | Method chosen |
|---|---|---|
Support answers on product sheets | The model does not know recent product references | RAG on the up-to-date catalogue |
Data extraction from purchase orders | Irregular output format | Prompt with examples, then fine-tuning if the gap persists |
Legal assistant for contracts | Facts specific to each contract, strict access rights | RAG with per-user access rights and review by the legal counsel |
Writing ads in the brand’s tone | Uneven tone | Prompt and examples; the result is sufficient, no fine-tuning |
Frequently asked questions
Can you train an AI on your company’s data?
Can you train an AI on your company’s data?
Yes, but it is often not the right method for teaching it facts. RAG gives the model your up-to-date documents at the moment of the question, without retraining it. Fine-tuning serves rather to set a format or a tone. In both cases, check your rights over the data used.
How many documents do you need to set up a RAG system?
How many documents do you need to set up a RAG system?
There is no threshold. Quality matters more than volume: up-to-date documents, no duplicates, with an identified owner. Start with the scope of a single use and measure the result on your evaluation set.
Is a chatbot on internal documents safe?
Is a chatbot on internal documents safe?
It becomes safe under three conditions. Respect each user’s access rights, limit sources to documents of known origin and keep human review for answers that commit the company. Without these rules, it can disclose documents or follow hidden instructions.
Does fine-tuning make you a provider under the AI Act?
Does fine-tuning make you a provider under the AI Act?
Rarely for the model itself: the Commission sets the threshold at more than one third of the compute of the original training. However, substantially modifying a high-risk system, or diverting it to a high-risk purpose, can make you its provider (Article 25).
Sources
P. Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv, 2020 (v4, 2021): the founding RAG paper, knowledge updates and provenance: arXiv
O. Ovadia, M. Brief, M. Mishaeli, O. Elisha, Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs, Proceedings of EMNLP 2024, pp. 237-250: RAG outperforms unsupervised fine-tuning for injecting knowledge: ACL Anthology
V. Magesh et al., Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, Journal of Empirical Legal Studies, vol. 22, 2025, pp. 216-242: three RAG-based legal tools tested, between 17% and 33% incorrect or poorly sourced answers: Wiley
OpenAI, Optimizing LLM Accuracy guide: prompt and evaluation set first; RAG and fine-tuning as two separate levers (context, behaviour): OpenAI
OpenAI, Deprecations page, section on self-serve fine-tuning, accessed 26 October 2026: timeline for the phased end announced on 7 May 2026: OpenAI
European Commission, FAQ on the guidelines for providers of general-purpose AI models, update of 11 November 2025: one-third compute threshold: European Commission
Consolidated AI Act, Article 25: cases in which a deployer becomes the provider of a high-risk AI system: AI Act Service Desk
OWASP Gen AI Security Project, Top 10 for LLM Applications, 2026 edition, 3 August 2026: LLM01:2026 Prompt Injection, ranked first, direct and indirect injection; LLM02:2026 Sensitive Information Disclosure (authorise access before retrieval in a RAG system); LLM09:2026 Vector and Embedding Weaknesses: OWASP GenAI
ANSSI, Recommandations de sécurité pour un système d’IA générative (Security recommendations for a generative AI system), 29 April 2024: R8 (need-to-know) and R21 (no retraining in production): ANSSI
The grid and the fictional exercise are a Spentia methodological construction, with no success statistics or client portfolio claimed.
