Skip to content
Back to Projects
Archived
AI/ML
Open Source
Jan 2024

Talker

An open source teaching assistant that answers course questions from the syllabus, slides and lecture materials — with citations, and YouTube timestamps when a video covers the answer.

View Source
FAISS
Retrieval
Local OLlama2
Model
Citations-first
Answer style
Text + video
Modalities
Talker project visual

The constraint

Students struggled to get timely answers to course-specific questions outside of office hours, leading to bottlenecks and disengagement.

The approach

Built a RAG-powered teaching assistant using OLlama2 and FAISS that answers questions based on class syllabus, slides, and materials. Added multi-modal support including YouTube videos with timestamp links.

Architecture
RAG pipeline with retrieval-first responses and citations.
High level
LearnerQuestion + contextRetrieverFAISS vector indexSourcesSlides, syllabus, notesGeneratorOllama2 + guardrailsAnswerCitationsembed + searchgroundingcontextsfinalOptionalYouTube timestampsmulti-modal
Process
  1. 1
    Source corpus

    Collected course syllabus, slides, and materials; normalized formats for consistent chunking.

  2. 2
    Index + retrieval

    Chunked content into semantically coherent spans and indexed with FAISS for fast retrieval.

  3. 3
    Grounded generation

    Used retrieved context to generate answers with citations and optional video timestamps when available.

  4. 4
    Iteration

    Measured usage/feedback, tightened prompts/guardrails, and improved citations and fallback behavior.

Outcomes

  • Answers grounded in course materials, returned with the source cited
  • Multi-modal responses that deep-link to the timestamp in a lecture video
  • Ran on a local OLlama2 model, so no course material left the machine
  • Open source, with outside contributions merged
What I'd do next
  • Add an eval harness for retrieval quality (recall@k, faithfulness) and regression tests.
  • Introduce caching + rate limiting for peak traffic periods and more robust context selection.
  • Improve citation UX: quote snippets, highlight sources, and add per-source confidence.
Details

Built an open source teaching assistant on a local OLlama2 model with a FAISS knowledge base, answering student questions from the course syllabus, slides and materials.

Added multi-modal retrieval that returns lecture videos deep-linked to the timestamp covering the question, rather than the whole recording.

Related projects