Parsing Megabytes Without Freezing the UI

Browsers are surprisingly good at data — if you respect their model. A 50 MB CSV is a few hundred thousand rows: parsing it synchronously on the main thread will freeze the interface for seconds, which users experience as a crash. The fix is a pattern of three techniques that client-side tools like KPI Master rely on to analyse files without ever touching a server.

1. Read in Chunks, Not All at Once

The FileReader API can read a whole file into memory in one call — and that is usually the wrong move. Reading in slices keeps memory growth predictable and lets you show progress:

const CHUNK = 2 * 1024 * 1024; // 2 MB slices
let offset = 0;
while (offset < file.size) {
    const slice = file.slice(offset, offset + CHUNK);
    const text = await slice.text();          // or slice.arrayBuffer()
    // feed the slice to a streaming parser
    offset += CHUNK;
}

For line-oriented formats like CSV, JSONL and TSV, a streaming parser that holds a partial line across chunk boundaries gives you a single pass over the file with constant memory.

2. Do the Heavy Work in a Worker

Web Workers run JavaScript on a separate thread. Move the parse loop — and any heavy statistics — into a worker and the interface keeps rendering, scrolling and responding to clicks the whole time. The worker communicates through postMessage:

// main thread
const worker = new Worker('/workers/parse.js');
worker.postMessage({ file, chunkSize: 2 * 1024 * 1024 });
worker.onmessage = (e) => updateProgress(e.data);

Workers cannot touch the DOM, but parsing, type detection and aggregation do not need it. That boundary is a feature: it forces the data logic to stay pure and testable.

3. Transfer Buffers, Don't Copy Them

Passing an ArrayBuffer to a worker normally copies it — for a 50 MB file that is another 50 MB of churn and a visible pause. The transferable option moves the buffer without copying; the sending side loses access, which is fine once the data is handed over:

worker.postMessage(buffer, [buffer]); // zero-copy transfer

Combined with typed arrays (Float64Array, Int32Array) instead of strings for numeric columns, a 50 MB file can live in memory at a fraction of its text size — and aggregation over typed arrays is dramatically faster than string math.

What the User Should See

Performance work is invisible unless it fails. The interface should communicate the pipeline's state:

• Progress, not a spinner. "Reading 23/50 MB · parsing rows 1,204,000" — a concrete number is a promise the process is alive.
• First results early. Stream the first N rows through as soon as they are parsed, not after the whole file completes. A preview table that fills in progressively feels instant even on big files.
• Graceful failure. A malformed row mid-file should surface as a message with the line number, not abort the whole parse. Most real-world files are 98% parseable; the analysis should use the 98% and tell the user about the 2%.

The constraint that makes all of this worthwhile: it runs locally, so file size is bounded by the device's memory, not by any server's patience — and the data never leaves the machine.