# From Project to Product: How cant.study → wehelpyou.study Transforms Student Note‑Taking and Meetings with Open‑Source LLMs

For three semesters I worked as an **on‑campus student note‑taker**, clocking in at 8 a.m. every Monday, Wednesday, and Friday. While the paycheck helped, the routine exposed a deeper accessibility gap: many courses *never* secure a note‑taker, leaving NTID classmates to piece together scattered slides, hurried phone photos, or—too often—nothing at all.

> **Why can’t note‑taking be automated, reliable, and context‑aware?**

That question became **CANT — the Context‑Aware Note Taker**. ([http://cant.study/](http://cant.study/))

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1746638963035/e921c673-bfe9-4966-a43f-f83d7c8e9775.png align="center")

---

### From Manual Notes to Multimodal Intelligence

CANT’s earliest prototype hinged on three technical insights:

1. **Context Correction Layer**  
    Raw audio is first transcribed by **NVIDIA Canary**. A *Context Correction Layer* then aligns jargon, acronyms, and speaker disfluencies with the instructor’s actual slides and course materials, transforming transcripts into task‑ready text.
    
2. **Iterative Chunk Summarization**  
    Lectures are streamed in ~10 k‑token chunks. Each chunk is summarized, then adjacent summaries are recursively refined to build a cohesive document—reducing LLM hallucinations and keeping sections tight and well‑labeled.
    
3. **24‑Hour Retention Window**  
    To balance study needs with privacy, **only the generated structured notes are retained** on the server after the first 24 hours. The system performs an auto‑purge unless a student explicitly opts to archive them.
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1746724169417/7d8ac106-7025-40e7-a653-322227228e48.jpeg align="center")

---

### Design Decisions & Why They Won

| Challenge | Design Decision | Why It Won |
| --- | --- | --- |
| **Transcription accuracy** | **Context Correction Layer over Canary output** | Injects slide headings, technical terms, and abbreviations for near‑human readability. |
| **Token limits in LLMs** | **Chunk‑then‑iterate pipeline** | Produces tighter summaries and scales to multi‑hour seminars. |
| **Observability & debugging** | **Langfuse tracing + log streams (file & Elasticsearch)** | One-click replay of any session’s LLM calls; Kibana dashboards spot drift or latency spikes. |
| **Student trust & compliance** | **24‑hour note retention** | Guarantees automatic cleanup without manual admin intervention. |

---

### A (Very) Brief Tour of the Deployment Pipeline

1. **Frontend** – React 18 + Tailwind. A single “Start Session” button streams 30 s audio chunks and visualizes progress.
    
2. **API Layer (Node.js + Express)** – Handles JWT auth, writes Langfuse traces, handles other business logic.
    
3. **Sandbox Container**
    
    * **Speech**: NVIDIA Canary
        
    * **Context Parser**: OCR for slides/PDFs
        
    * **LLM**: Llama 3 8B‑Instruct‑128k behind an Nginx reverse proxy
        
4. **Storage** – **MongoDB** for transcripts & structured notes (ephemeral collections respect the 24‑hour policy).
    
5. **CI/CD** – GitHub Actions → Docker Buildx → zero‑downtime blue‑green deploys on **Linode’s dedicated GPU cloud**.
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1746639079748/b8478e83-c957-491c-ad55-5a993af35cc7.png align="center")

---

### Launch Day: 30 April 2025 — Capstone Poster & Public Release

We unveiled CANT during the RIT Data Science Capstone poster session on **30 April 2025** and simultaneously opened public access:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1746639103959/eaa4e4db-705c-4374-bc29-a20cc401f1dd.png align="center")

| Domain | Purpose |
| --- | --- |
| [**cant.study**](http://cant.study) | The problem space—where students *can’t* study without equitable notes. |
| [**wehelpyou.study**](http://wehelpyou.study) | The solution space—CANT’s live app and onboarding hub. |

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">Try our app for free @ https://app.cant.study</div>
</div>

**First‑week impact**

* **42 new registered users**
    
* **100 + unique visitors**
    
* **8 + hours of audio captured**
    
* **14 structured note sets generated**
    

**Improvements and Results**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1746724215128/1e4ddcab-e7ad-41a0-89b0-51a88f066825.png align="center")

---

### What’s Next?

1. **Federated Learning**  
    Train the Context Correction Layer on‑device so each cohort’s jargon improves future transcripts—without centralizing raw data.
    
2. **Edge Deployment using Service Workers**  
    Packaging the full pipeline for offline field trips and patchy Wi‑Fi environments.
    
3. **Open‑Source Modules**  
    Releasing the chunk‑iterative summarizer, Context Aware ASR Correction so researchers can plug in custom LLMs.
    

Thank you for following our journey from sleepy lecture halls to an AI‑powered accessibility platform. Have feedback or want to contribute? Visit [**cant.study**](http://app.cant.study)—because *when you* [*CANT.study*](http://CANT.study)*,* [*WEHELPYOU.study*.](https://wehelpyou.study/)

**Email: gmail@sanathswaroop.com**

**Contact: +1 (585)-565-2567**