The LoRA Workshop: Building Specialized Minds

Sunday evening, and the whiteboard has turned into a hardware bill again.

That is usually how I know an idea has become serious. The concept is no longer just a diagram or a paragraph in a product note. It has started asking for GPUs.

Tutor works today as a memory-and-retrieval problem: course context, recent conversation, student progress, prior explanations, and the model response stitched together carefully enough that the lesson feels continuous. That RAG layer matters. It gives Tutor memory. It gives it context. It keeps it from treating every question like the first time it has met the student.

But RAG is not the end state.

The next step is specialization.

Beyond RAG

RAG can retrieve the right material. It can pull in the relevant course notes, the prior lesson, the assignment instructions, or the student's earlier mistake. That is useful, but it does not fully change how the model teaches.

A general model with retrieved context is still a general model.

For Tutor, I want something more specific: one powerful local AI model as the base, then subject-specific LoRA adapters layered over it for different courses. Calculus gets one adapter. Intro programming gets another. Biology, writing, physics, language learning, each with its own tuned teaching behavior.

Not different products. Not separate tutors with separate memories.

One Tutor. One strong base. Different lenses.

What a LoRA Would Do

LoRA stands for Low-Rank Adaptation. In practical terms, it means training a small adapter that changes how a larger model behaves without retraining the whole thing.

That is the part that makes it interesting.

The base model keeps the general intelligence: language, reasoning, conversation, tool use, and the ability to follow the student's thread. The LoRA nudges that intelligence toward a particular domain and teaching style.

A calculus LoRA should not merely know calculus. The base model already knows a lot of calculus. The adapter should shape how the model teaches calculus: when to show algebraic steps, when to pause for a conceptual explanation, when to ask the student to try the next move, when to point out a common mistake without making them feel stupid.

A writing LoRA would be different. It would need to understand drafts, tone, structure, argument, voice, revision, and when to stop correcting and start encouraging.

The coursework matters because a generic subject label is not enough. A high school Algebra II course, a college Calculus I course, and a self-paced adult refresher course are not the same teaching problem. The adapter should be fine-tuned on the actual material and examples for that course.

That is the workshop idea.

The Architecture I Want

The architecture in my head looks like this:

student session
  ↓
recent conversation context
  ↓
RAG retrieval from course materials and student history
  ↓
strong local base model
  ↓
subject/course LoRA adapter
  ↓
tutoring response

The RAG layer provides facts and context.

The LoRA layer shapes behavior.

The base model provides the intelligence.

That separation matters. If the student asks about a specific assignment, RAG should retrieve the assignment. If the student is working through a proof, a future LoRA could help the model behave like a patient math tutor. If the student is frustrated, the base model still needs enough general conversational ability to respond like a person instead of a worksheet generator.

A good Tutor cannot be a pile of facts. It has to teach.

The Hardware Problem

Here is the part where the whiteboard becomes a parts list.

We do not have the hardware to make this real yet. Not properly.

The workstation experiments have been useful. The RTX 3090 can teach us a lot about local models, quantization, retrieval, and small experiments. But the system I want for Tutor needs more than "it runs if I close everything else and wait." It needs enough GPU memory and throughput to serve a powerful base model while experimenting with adapters in a way that does not make every test painful.

That is why the server build keeps coming back into the conversation. The Proxmox box, the big power supply, the motherboard choices, the GPU planning, all of it connects to this. We can prototype the idea now, but actually making it useful will require the hardware to catch up with the plan.

The frustrating part is that the architecture feels right before the budget does.

Why Not Just Use RAG Forever?

Because RAG answers a different question.

RAG asks: what information should the model see right now?

LoRA asks: how should the model behave after it sees it?

For Tutor, both matter. If a student is learning derivatives, the model needs the relevant lesson material, but it also needs the right teaching posture. Does it lead with intuition? Does it give the formula first? Does it ask a question? Does it notice that the student keeps making the same algebra mistake? Does it slow down?

Those are not just retrieval problems.

They are behavior problems.

And behavior is exactly where fine-tuning becomes interesting.

Course-Specific Adapters

The long-term goal is not one generic "math" adapter or one generic "writing" adapter. The better version is course-specific.

A Tutor instance for a particular class would have:

• the course syllabus
• lesson materials
• assignments and rubrics
• example problems
• preferred explanations
• student history
• a subject/course LoRA that could eventually be trained on the teaching style and content patterns that fit that course

The adapter should not replace the course material. It should help the model teach in the right shape.

That could make Tutor feel less like a chatbot answering homework questions and more like a teacher who understands the course.

That is the goal.

Where We Are Now

We are not there yet.

Right now, this is the direction we want to go. RAG first, because memory and course retrieval are foundational. LoRAs next, because specialization is how Tutor becomes more than a general model with documents pasted into the prompt.

The concept is clear: one powerful local AI, layered with LoRA adapters fine-tuned on specific coursework for specific subjects.

The missing piece is hardware.

So for now I keep refining the architecture, testing the retrieval layer, pricing parts I do not want to buy yet, and writing down the plan before the details blur.

Some infrastructure starts as code.

Some of it starts as a bill you cannot quite afford yet.