Skip to content

RAG tutorial: retrieval-augmented generation in TypeScript

· AI Agents · · 7 min read

Retrieval-augmented generation (RAG) makes a language model answer from your documents instead of from memory. Before the model is called, the question is used to find the most relevant passages, and those passages go into the prompt with an instruction to answer only from them. This tutorial builds the retrieval half end to end with a local embedding model, so it runs without an API key, and ends with the prompt you would send to any model.

Tested on 21 November 2026 with Node.js 26, @huggingface/transformers 4.3.2 and the Xenova/all-MiniLM-L6-v2 embedding model, run with tsx 4. The script type-checks with TypeScript 5 in strict mode.

1. Set up the project

mkdir rag-demo && cd rag-demo
npm init -y
npm pkg set type=module
npm install @huggingface/transformers@4.3.2
npm install -D tsx@4 typescript@5 @types/node@24

2. The whole pipeline

Five steps: chunk the documents, load an embedding model, embed every chunk once, embed the question and rank chunks by similarity, then build the prompt. The documents here are a small shop's delivery, returns and opening-hours notes.

import { pipeline } from "@huggingface/transformers";

// 1. Your documents, split into small chunks with a source for each.
const chunks = [
  { source: "delivery.md", text: "We deliver within 10 km of the shop. Orders placed before 2 pm arrive the same day." },
  { source: "delivery.md", text: "Delivery is free on orders over 3,000 rupees. Below that it costs 200 rupees." },
  { source: "returns.md", text: "Unopened items can be returned within 7 days with the receipt." },
  { source: "returns.md", text: "Medicines cannot be returned once they have left the pharmacy." },
  { source: "hours.md", text: "The shop is open 9 am to 11 pm every day, including public holidays." },
];

// 2. A local embedding model. It downloads once and then runs offline.
const embed = await pipeline("feature-extraction", "Xenova/all-MiniLM-L6-v2");

async function vector(text: string): Promise<number[]> {
  const output = await embed(text, { pooling: "mean", normalize: true });
  return Array.from(output.data as Float32Array);
}

// Vectors are normalised, so the dot product is the cosine similarity.
const similarity = (a: number[], b: number[]) => a.reduce((sum, value, i) => sum + value * b[i], 0);

// 3. Index: embed every chunk once and keep the vectors.
const index = await Promise.all(chunks.map(async (chunk) => ({ ...chunk, vector: await vector(chunk.text) })));

// 4. Retrieve: embed the question and take the closest chunks.
async function retrieve(question: string, k = 2) {
  const q = await vector(question);
  return index
    .map((chunk) => ({ source: chunk.source, text: chunk.text, score: similarity(q, chunk.vector) }))
    .sort((a, b) => b.score - a.score)
    .slice(0, k);
}

// 5. Build the prompt for any LLM: context first, then the rule, then the question.
function buildPrompt(question: string, context: { source: string; text: string }[]) {
  const sources = context.map((c, i) => `[${i + 1}] (${c.source}) ${c.text}`).join("\n");
  return `Answer using only the sources below. If they do not answer the question, say you don't know.\n\nSources:\n${sources}\n\nQuestion: ${question}`;
}

for (const question of ["Do you deliver on the same day?", "Can I return medicine?", "Do you sell laptops?"]) {
  const hits = await retrieve(question);
  console.log(`\nQ: ${question}`);
  for (const hit of hits) console.log(`  ${hit.score.toFixed(2)}  ${hit.source}  ${hit.text}`);
}

console.log("\n--- Prompt for the first question ---");
const question = "Do you deliver on the same day?";
console.log(buildPrompt(question, await retrieve(question)));

Run it with npx tsx rag.ts. The first run downloads the model; after that it runs offline. This is the output it produced:

Q: Do you deliver on the same day?
  0.61  delivery.md  We deliver within 10 km of the shop. Orders placed before 2 pm arrive the same day.
  0.35  hours.md  The shop is open 9 am to 11 pm every day, including public holidays.

Q: Can I return medicine?
  0.70  returns.md  Medicines cannot be returned once they have left the pharmacy.
  0.33  returns.md  Unopened items can be returned within 7 days with the receipt.

Q: Do you sell laptops?
  0.17  delivery.md  Delivery is free on orders over 3,000 rupees. Below that it costs 200 rupees.
  0.14  returns.md  Unopened items can be returned within 7 days with the receipt.

--- Prompt for the first question ---
Answer using only the sources below. If they do not answer the question, say you don't know.

Sources:
[1] (delivery.md) We deliver within 10 km of the shop. Orders placed before 2 pm arrive the same day.
[2] (hours.md) The shop is open 9 am to 11 pm every day, including public holidays.

Question: Do you deliver on the same day?

3. Reading the scores

The two questions the documents can answer found the right chunk first, with scores of 0.61 and 0.70. The question about laptops, which nothing in the documents covers, still returned two chunks, because retrieval always returns the closest matches, but its best score was only 0.17.

That is why a RAG system needs a minimum score. Below it, skip the model and say you don't know, or hand the question to a person. The right threshold depends on the embedding model and your documents, so measure it on real questions rather than copying a number.

4. Chunking decides most of the quality

  • Keep one idea per chunk. A chunk that mixes delivery and returns will match both kinds of question weakly.
  • Keep the source with every chunk, so answers can cite it and staff can check it.
  • Re-embed a document whenever it changes. Stale chunks give confident, outdated answers.
  • Write the documents for retrieval: short sentences that state the fact plainly retrieve better than long marketing copy.

5. Generating the answer

The last block prints the prompt: the retrieved sources, numbered, an instruction to answer only from them, and the question. Send that to the model you use through its API. Asking the model to cite source numbers makes its answers easy to check.

6. From example to production

  • Store vectors in a database once you have more than a few thousand chunks; PostgreSQL with pgvector is a common choice.
  • Build a test set of real questions with the chunk each should find, and re-run it whenever you change chunking, the model or the threshold.
  • Log the question, the chunks retrieved and their scores for every answer.
  • Send anything below the threshold, and anything sensitive, to a person.

Want a chatbot that answers only from your own documents? AI chatbot development