AI Course Recommendations Over Your Catalog: RAG Explained
AI course recommendation with retrieval-augmented generation (RAG) works in four steps: filter the catalog down to sessions a learner can actually book, retrieve the best matches by meaning and keywords, re-rank them, then let a language model explain the result using only those retrieved records. The model writes the answer. Your data decides what it may say.
That order matters for training providers. A course catalog is not a static library: sessions have dates, venues, seat limits and prerequisites, and they change daily. A recommender that suggests a full or past session looks broken even when its writing is fluent. This guide walks through each stage with a small illustrative catalog, then shows how to test the result.
Key takeaways
- Apply hard constraints (dates, location, seats, prerequisites) as filters before similarity search, never as hints in a prompt.
- Index stable course text. Read volatile session facts from your database at query time.
- Combine keyword and vector search, then re-rank, then require the model to cite a session ID for every recommendation.
- Measure precision@k, the rate of constraint violations, and groundedness, not just whether answers sound good.
How does RAG work for AI course recommendation?
RAG pairs a language model with a searchable store of your own records. At question time the system retrieves the most relevant records and hands them to the model, which writes its answer from them. For a training catalog, the store holds course and session records, so answers reflect your offer instead of the model’s general memory.
The idea comes from Lewis et al. (2020). They combined a pretrained generator’s stored knowledge (“parametric memory”) with a dense vector index of Wikipedia (“non-parametric memory”) reached through a pretrained retriever. They also named provenance and updating knowledge as open problems for models that rely on their weights alone.
A catalog is a smaller, faster-changing version of that outside memory. When a course is retired, you edit a record. You do not retrain anything.
How should you prepare a course catalog for retrieval?
Split each course record into a few small chunks by meaning (overview, outcomes, audience and prerequisites, agenda), prefix each chunk with the course title and ID, and convert each to an embedding. Index only text that rarely changes. Dates, seats and prices stay in your database.
On chunk size, Anthropic’s contextual retrieval write-up describes chunks of “usually no more than a few hundred tokens” and notes that size, boundaries and overlap all affect retrieval. The same write-up prepends a short, chunk-specific context to each chunk before embedding, so a bare “Day 2: role plays” chunk still says which course it belongs to. Course records are short, so three to five chunks per course is a sensible starting point to test.
An embedding turns text into a list of numbers so that related texts sit close together. OpenAI’s embeddings guide ranks documents by cosine similarity between the query vector and each document vector. As of November 2026 it lists text-embedding-3-small (1,536 dimensions by default) and text-embedding-3-large (3,072), both accepting up to 8,192 input tokens. Embed queries and chunks with the same model, and re-embed everything if you switch.
Pro tip: Keep session facts out of the index. A seat count baked into an embedding is wrong an hour later. Store only a course ID in the index, then join to live session data after retrieval. This one rule prevents most stale recommendations.
Why does AI course recommendation need hard filters before similarity?
Embeddings measure how related two texts are, not whether a session can be booked. Date, location, format, seats and prerequisites are yes-or-no conditions, so apply them as database filters. Similarity then ranks only the options a learner could actually take, which stops the closest match from being a session that already ran.
Separate the request into two kinds of constraint:
- Hard constraints: the session has not started, seats remain, the format and city match, the learner meets the prerequisites.
- Soft preferences: a two-day format, a particular weekday, a named instructor, a lower price.
A language model can extract these from free text into a fixed schema, but validate the output and resolve relative dates such as “next month” against the real date and the learner’s time zone. If a hard constraint is ambiguous, ask.
Filter-first also avoids an empty-result trap. If you retrieve the ten most similar chunks and filter afterward, all ten may belong to past sessions, and you return nothing even though valid courses sit lower down. Microsoft’s hybrid search documentation says a filter can apply at the beginning or end of query processing, and that with its semantic ranker you may prefer post-filtering. It advises testing which works better for your queries. Test both on your own catalog.
How do hybrid retrieval and re-ranking pick the best matches?
Hybrid retrieval runs keyword search and vector search in parallel, then merges the two ranked lists. Keyword search catches exact terms such as course codes and certification names. Vector search catches meaning, such as “handling difficult coworkers” matching a course titled conflict management. A re-ranker then reorders the shortlist.
The Azure documentation says hybrid queries merge results with Reciprocal Rank Fusion (RRF), where each list scores a document as 1/(rank + k) and the scores are summed. Its ranking article says experiments favor a small constant such as k = 60. Our own arithmetic shows the effect:
- Course A ranks 1st in keyword search and 3rd in vector search: 1/61 + 1/63 is about 0.0323.
- Course B ranks 2nd in keyword search and 1st in vector search: 1/62 + 1/61 is about 0.0325.
B wins narrowly because it is strong in both lists. Anthropic’s tests found that combining embeddings with keyword (BM25) search beat embeddings alone. Its full pipeline, which added contextual chunks and reranked 150 candidates down to 20, cut the top-20 retrieval failure rate from 5.7% to 1.9% on its own datasets. Your catalog will differ, so measure before you trust those numbers.
After relevance ranking, apply soft preferences as a separate, logged step: soonest start, most seats, the learner’s language. Keeping relevance and business rules apart lets you see which one moved a result.
How do you keep the answer grounded in retrieved records?
Give the model only the eligible records, tell it to answer from them alone, and require a session ID for every recommendation. Then check the output in code: each cited ID must be in the retrieved set, and each date must match its record. Reject and retry anything that fails.
A minimal prompt skeleton looks like this:
You are a course advisor for a training provider.
Use ONLY the records below. Do not use outside knowledge.
Recommend at most 3 sessions. For each, give the session_id,
one sentence on why it fits, and the start date exactly as
written in the record.
If no record fits, say so and name the constraint that
excluded the nearest options.
REQUEST: {learner request}
RECORDS: {eligible session and course records}
The last instruction matters most. Without a permitted way to say “nothing qualifies,” a model can stretch a poor match into a recommendation.
What does a worked example look like?
Take a learner who wants an in-person conflict management course in Chicago, starting between November 16 and 30, 2026. The illustrative catalog below has six sessions. Filters remove four, similarity ranks the remaining two, and the model explains the result. All names, dates and seat counts are invented for teaching.
| Session | Course | Format and city | Dates (2026) | Seats left | Prerequisite |
|---|---|---|---|---|---|
| S-101 | Conflict Management for New Team Leads | In person, Chicago | Nov 18-19 | 3 | None |
| S-102 | Conflict Management for New Team Leads | In person, Chicago | Oct 7-8 | 6 | None |
| S-103 | Conflict Management for New Team Leads | In person, Chicago | Nov 24-25 | 0 | None |
| S-104 | Conflict Management for New Team Leads | Virtual | Nov 19-20 | 9 | None |
| S-105 | Difficult Conversations Workshop | In person, Chicago | Nov 17 (one day) | 5 | None |
| S-106 | Advanced Negotiation | In person, Chicago | Nov 18-19 | 8 | Conflict Management for New Team Leads |
The system then works through five steps, assuming today is November 2, 2026:
- Parse. Hard: in person, Chicago, start between Nov 16 and Nov 30, at least one seat, prerequisites met. Soft: two days. Topic text: “handling conflict.”
- Filter. S-102 already ran, S-103 is full, S-104 is virtual, and S-106 needs a prerequisite this learner lacks. S-101 and S-105 remain.
- Retrieve. Hybrid search over the chunks of the two remaining courses ranks Conflict Management first.
- Re-rank. S-101 matches the two-day preference. S-105 stays as a one-day alternative.
- Generate and validate. The model cites S-101 and S-105 with dates from the records. Code confirms both IDs came from the retrieved set.
With six sessions the ranking step is trivial. In a real catalog, filtering may leave hundreds of candidates, and ranking quality becomes the whole game. Note also that S-101 has three seats. Re-check availability at the moment of booking, because the number you showed may be stale by then.
How does AI course recommendation handle cold start?
Cold start means the system has no history to learn from, either for a new course or a new learner. Retrieval over course descriptions copes well with both. A new course is searchable the day its record exists, and a new learner’s request is itself the signal, unlike models that need enrollment history first.
The weak point is thin course records. If a course has a one-line description and no stated outcomes, audience or prerequisites, retrieval has little to match against. Set a minimum record standard before indexing: outcomes, intended audience, level, duration and prerequisites.
When a request is vague, such as “something on leadership,” ask one or two questions about role, goal, city and timing instead of guessing.
How do you evaluate AI course recommendation?
Build a test set of real requests, each labeled with the sessions a trained advisor would accept. Then measure four things: precision@k for relevance, a constraint-violation rate for bookability, groundedness for faithfulness to the records, and whether the system correctly says nothing qualifies when that is true.
| Metric | Question it answers | How to compute it |
|---|---|---|
| Precision@k | Of the k sessions shown, how many are acceptable? | Accepted sessions in the top k, divided by k, averaged over test requests |
| Constraint-violation rate | Did any shown session break a hard filter? | Sessions shown that are past, full, wrong format or ineligible, divided by sessions shown. The target is zero |
| Groundedness | Is every claim in the answer supported by the records? | Supported claims divided by total claims, the approach behind the faithfulness metric in evaluation tools such as Ragas |
| Correct abstention | When nothing qualifies, does it say so? | Share of unsatisfiable test requests answered with no recommendation |
The constraint-violation rate is our proposed metric, not an industry standard. It exists because the other two can pass while the system fails. An answer can be perfectly grounded in a stale record, so test against live availability data.
Pro tip: Seed the test set with traps. Add requests built to fail: a month with no sessions, a city you do not serve, a prerequisite the learner lacks. A system that always finds something to recommend will look excellent on easy requests and fail exactly these.
Which failure modes should you test for?
Many failures come from letting the language model decide things that code should decide. The table lists the six that matter most for a training catalog, with the usual cause and the guard that prevents each. Run each as a named test case before launch.
| Failure | Usual cause | Guard |
|---|---|---|
| Recommends a past session | Dates stored in the index, or filtered after retrieval | Filter on live session data first; validate dates in code |
| Recommends a full session | Seat counts cached or embedded | Read seats at query time; re-check at booking |
| Ignores a prerequisite | Prerequisite exists only as prose in the description | Store prerequisites as structured fields and check them against the learner record |
| Invents a course or date | Model answers from general memory | Records-only prompt; reject any ID not in the retrieved set |
| Misses an exact term | Vector-only search on a course code or acronym | Hybrid retrieval with keyword search |
| Wrong start time for a virtual session | Time zone dropped when dates are stored or shown | Store times with a zone; display in the learner’s zone |
What should you do next?
Start with the data, not the model. Check that every course has outcomes, audience, level and structured prerequisites, and that session dates and seat counts can be queried live. If they cannot, fixing that will improve any recommender you build later.
Then build the smallest version: filters, hybrid search over about a hundred courses, a records-only prompt and a validator. Write the trap test requests before you tune anything.
If your sessions and enrollments live in a training management platform, that is where the live data comes from. SimpliTrain, built by Mundrisoft, describes multi-tenant client management, white-label portals and e-commerce with auto-enrollment on payment for training companies. This post describes a technique, not a SimpliTrain feature. For context, Mundrisoft’s article on custom enterprise LMS platforms and AI integration covers the strategic side, and its AI Solutions page lists custom AI tool development and AI system integration among its services.
Frequently asked questions
Is RAG the same as a recommendation engine?
No. A recommendation engine usually learns from behavior, such as enrollments and completions, to predict what a learner will want. RAG retrieves records that match a request and has a model explain them. It can serve as the front end of a recommender, and the two can be combined, but they solve different problems.
Do I need a vector database for AI course recommendation?
Not necessarily. A catalog of a few thousand courses produces a small index, and search services such as Azure AI Search run vector and keyword queries together. What you need is metadata filtering, keyword search and live joins to session data. Choose the tool that supports those, then test it on your own queries.
How often should the course index be refreshed?
Re-embed a course only when its descriptive text changes, and re-embed everything if you change embedding models. Session dates, seats and prices should never be refreshed in the index at all, because they are read live from your database each time a request arrives.
Can the model recommend a course that does not exist?
Yes, if it is allowed to answer from general knowledge. Prevent it by passing only retrieved records, instructing the model to use nothing else, and verifying in code that every cited ID exists in the retrieved set. If a recommendation fails that check, reject the answer and retry.
Does RAG personalize recommendations?
Only as far as you give it personal data. RAG uses the learner’s request plus any profile fields you pass, such as role, location or completed courses, and applies them as filters or ranking signals. It does not build a model of the learner from past behavior unless you add that separately.