Project: Ask My Notes (a mini RAG bot)
Build a question-answering bot over your own notes: split them into chunks, find the relevant ones, build a grounded prompt, and get the answer from a free AI model on your laptop.
Why it matters: RAG is the most common GenAI pattern in companies. Building it from scratch, without a framework, is the best way to explain it in an interview.
Step 1
What you will build
A chat bot that answers from your notes, not from whatever the model remembers:
- Chunk: split the notes into paragraphs
- Retrieve: find the paragraphs that share words with the question
- Augment: put those paragraphs into the prompt as context
- Generate: ask the model, on your laptop with Ollama (set up in the "Your First Real AI Call" lesson)
You build steps 1 to 3 here. Step 4 runs on your laptop.
Step 2
Chunk the notes
Small chunks keep the prompt short and on topic. Real systems split by tokens with some overlap. Splitting on blank lines is enough for notes.
Tiny example
A blank line separates paragraphs.
notes = "Docker packages apps.\n\nKubernetes runs containers."
chunks = [part.strip() for part in notes.split("\n\n") if part.strip()]
print(len(chunks), chunks)Step 3
Retrieve by shared words
Score each chunk by how many words it shares with the question, ignoring filler words like "what" and "the". Production systems use embeddings and a vector database instead, but the idea is the same: find the closest text first.
Tiny example
Set intersection counts shared words.
question = {"docker", "containers"}
chunk = {"docker", "packages", "apps"}
print(len(question & chunk), "shared word(s)")Step 4
Ground the prompt
Tell the model to answer only from the notes, and to say "I don't know" when the notes do not cover the question. That one line cuts made-up answers a lot.
Tiny example
Context first, question last, and a way out.
hits = ["Docker packages an application with its dependencies."]
context = "\n".join(f"- {hit}" for hit in hits)
print(f"Notes:\n{context}\n\nQuestion: What does Docker do?")Build it here · answer checked automatically
Build the project
Finish the three functions. split_into_chunks(text) splits on blank lines and returns the stripped, non-empty paragraphs. retrieve(question, chunks, k=2) scores each chunk with len(words(question) & words(chunk)) and returns the top k chunks with a score above 0, best first. build_prompt(question, hits) returns a prompt that includes the hits as context, the question, and tells the model to say "I don't know" if the notes do not cover it.
Edit the code, then press Run & check. Passing it marks this project done.
python console
Project
~/lesson.py
Ctrl+Enter
console output
ready
$ waiting for Run...On your laptop
Run the real thing on your computer
Now connect it to a real model. You need Ollama with llama3.2 downloaded, from the "Your First Real AI Call" lesson. Nothing else to install.
- 1
Make a folder called
ask-my-notes, save the files below into it, and open a terminal there. On Mac and Linux, typepython3instead ofpython. - 2
Replace the text in
notes.txtwith your own class notes. Keep a blank line between paragraphs. - 3
Make sure Ollama is running (open the app, or run
ollama servein another terminal).
ask_my_notes.py
"""Ask My Notes: a tiny RAG bot over your own notes, answered by a free local model."""
import json
import urllib.error
import urllib.request
from pathlib import Path
OLLAMA_URL = "http://localhost:11434/v1/chat/completions"
MODEL = "llama3.2"
STOP_WORDS = {
"a", "an", "the", "is", "it", "and", "so", "on", "of", "to", "what", "how",
"does", "do", "why", "in", "with", "can", "be", "before", "i", "me", "my",
}
def words(text):
cleaned = text.lower()
for mark in ".,?!:;()":
cleaned = cleaned.replace(mark, " ")
return {word for word in cleaned.split() if word not in STOP_WORDS}
def split_into_chunks(text):
return [part.strip() for part in text.split("\n\n") if part.strip()]
def retrieve(question, chunks, k=2):
question_words = words(question)
scored = []
for chunk in chunks:
score = len(question_words & words(chunk))
if score > 0:
scored.append((score, chunk))
scored.sort(key=lambda pair: pair[0], reverse=True)
return [chunk for _, chunk in scored[:k]]
def build_prompt(question, hits):
context = "\n".join(f"- {hit}" for hit in hits)
return (
"Answer the question using only the notes below. "
"If the notes do not cover it, say \"I don't know\".\n\n"
f"Notes:\n{context}\n\nQuestion: {question}"
)
def ask_model(prompt):
body = json.dumps({
"model": MODEL,
"messages": [{"role": "user", "content": prompt}],
"temperature": 0,
"stream": False,
}).encode("utf-8")
request = urllib.request.Request(OLLAMA_URL, data=body, headers={"Content-Type": "application/json"})
with urllib.request.urlopen(request, timeout=120) as response:
data = json.loads(response.read().decode("utf-8"))
return data["choices"][0]["message"]["content"].strip()
def main():
chunks = split_into_chunks(Path("notes.txt").read_text(encoding="utf-8"))
print(f"Loaded {len(chunks)} notes. Ask a question, or press Enter on an empty line to quit.")
while True:
question = input("\nYou: ").strip()
if not question:
break
hits = retrieve(question, chunks)
if not hits:
print("Bot: I don't know. None of your notes mention that.")
continue
try:
answer = ask_model(build_prompt(question, hits))
except urllib.error.HTTPError as err:
print(f"Bot: Ollama answered with an error ({err.code}). Run: ollama pull {MODEL}")
return
except urllib.error.URLError:
print("Bot: I could not reach Ollama. Open the Ollama app, then try again.")
return
print("Bot:", answer)
print("Sources:", " | ".join(hit[:40] + "..." for hit in hits))
if __name__ == "__main__":
main()notes.txt
MLOps is the practice of deploying, monitoring and retraining machine learning models in production. Docker packages an application with its dependencies so it runs the same on every machine. Kubernetes runs and scales containers across many servers. RAG means retrieval augmented generation. It finds relevant notes and adds them to the prompt before the model answers. A vector database stores embeddings so similar text can be found quickly.
README.md
# Ask My Notes
A small RAG (retrieval augmented generation) bot. It finds the notes that match
a question and asks a local AI model (Llama 3.2 through Ollama) to answer from
those notes only.
## How to run
ollama pull llama3.2
python ask_my_notes.py
## How it works
1. Split notes.txt into paragraphs
2. Score each paragraph by shared words with the question
3. Put the best paragraphs into a grounded prompt
4. Send the prompt to the local model and print the answer with its sources
## What I learned
- The four steps of RAG, built without a framework
- Grounded prompts and saying "I don't know" instead of guessing
- Calling an OpenAI-compatible chat API from plain Python
## Next ideas
- Use embeddings (for example nomic-embed-text in Ollama) instead of shared words
- Serve it as an API with FastAPI.gitignore
__pycache__/ .venv/
Run it from the same folder:
python ask_my_notes.py
You should see something like:
Loaded 5 notes. Ask a question, or press Enter on an empty line to quit. You: What does Docker do? Bot: Docker packages an application with its dependencies so it runs the same on every machine. Sources: Docker packages an application with its... You: Who won the cricket match? Bot: I don't know. None of your notes mention that.
Show it to recruiters
Put it on GitHub
- 1
Create a free account at github.com if you do not have one yet.
- 2
Click + (top right), then New repository. Name it
ask-my-notes, choose Public, and click Create repository. - 3
Easiest way: on the new repository page, click uploading an existing file, drag in
ask_my_notes.py,notes.txt,README.md,.gitignore, and click Commit changes. Done. - 4
The way companies work: install Git from git-scm.com, open a terminal in your project folder, and run these. Replace
YOUR-USERNAMEwith your GitHub username. The twogit configlines are needed only once per computer. The first push asks you to sign in: on Windows a browser window opens. On Mac or Linux, GitHub does not accept your account password here, so use a personal access token as the password, or publish with GitHub Desktop instead.git config --global user.name "Your Name" git config --global user.email "you@example.com" git init git add . git commit -m "First version of ask-my-notes" git branch -M main git remote add origin https://github.com/YOUR-USERNAME/ask-my-notes.git git push -u origin main
- 5
Put the repository link on your resume and LinkedIn under Projects, with one line on what it does and one on what you learned. Recruiters open these.
Takeaways
- 1RAG is chunk, retrieve, augment, generate.
- 2Grounded prompts tell the model to use only the given notes.
- 3Return 'I don't know' when nothing relevant is found, instead of guessing.