Prompting, retrieval or fine-tuning: choose the fix for the failure
Identify whether the problem is instructions, missing evidence or repeated output behaviour before committing data and time to training.
In brief
Fine-tuning is not the first answer to every weak AI result. Better instructions can address an unclear task; retrieval can supply changing facts; training may fit repeated behaviour when the platform supports it and data quality justifies it.
Diagnosing the failure avoids training a model to reproduce inaccurate examples or memorize facts that should remain updateable.
Choose by failure type
If the model omits a required field, improve the output contract and validation. If it lacks a current policy, supply the source through retrieval. If a consistent style or narrow behaviour remains difficult across many examples, investigate a supported training route after simpler fixes.
Evaluate before committing
Keep training examples separate from evaluation. Remove sensitive material you lack permission to use and correct bad labels. Compare with the untrained baseline on the same task; training a polished format does not independently verify facts.
Plan lifecycle and support
Inspect the provider’s current eligibility, supported models and deprecation policy. Budget dataset maintenance and evaluation, not only one training job. OpenAI’s wind-down is a concrete reason to recheck older advice; other platforms and local methods have their own constraints.
Before you act
Availability is provider-specific. OpenAI’s current documentation says its self-serve fine-tuning platform is winding down and is not open to new users; do not plan a new integration from an old tutorial.
What to do next
- Collect examples of the actual failure and define success.
- Test a clear prompt and relevant evidence on the same held-back examples.
- Consider supported training only if the remaining behaviour problem and data quality justify its cost and maintenance.
