Skip to content

Large files and streaming

Read seekable files and write bounded chunks while retaining explicit XLSX resource limits.
On this page
Stream a workbook to a destination
import { toXlsxWorkbookStream } from "@sheetwrite/core";
import "@sheetwrite/xlsx/register";
declare const destination: WritableStream<Uint8Array>;
const controller = new AbortController();
signal: controller.signal,
onWarning: (warning) => console.warn(warning.code, warning.message),
}).pipeTo(destination);
OutputResult
Destination receives bytesBounded Uint8Array chunks, not one complete file buffer.
Destination slows downBackpressure slows the producer.
Reader cancelsExport stops reading the store.

The output contains the same XLSX parts as buffered export. ZIP entry order and container bytes can differ.

Try it live: resource-limit failures. The lab demonstrates typed limits, not a whole-file memory guarantee.

Full setup

This browser example reads a selected File without calling arrayBuffer() on the whole compressed input. Its writable consumer counts bytes instead of collecting them:

Seekable import and bounded output consumption
import { fromXlsxWorkbookFile, toXlsxWorkbookStream } from "@sheetwrite/core";
import "@sheetwrite/xlsx/register";
const input = document.querySelector<HTMLInputElement>('input[type="file"]');
const file = input?.files?.[0];
if (!file) throw new Error("Choose an XLSX file first");
const options = {
onWarning: (warning: import("@sheetwrite/core").XlsxWorkbookWarning) =>
console.warn(warning.code, warning.message),
};
const snapshot = await fromXlsxWorkbookFile(file, options);
let outputBytes = 0;
await toXlsxWorkbookStream(snapshot, options).pipeTo(new WritableStream<Uint8Array>({
write(chunk) { outputBytes += chunk.byteLength; },
}));
console.log(outputBytes); // Total depends on the selected workbook.

Replace the counting consumer with the host's actual writable destination to save bytes. Collecting chunks into a Blob or using new Response(stream).arrayBuffer() buffers the output again.

Streaming functions

FunctionInputResult
toCsvStream(sheet, store, options?)Visible table viewReadableStream<Uint8Array>.
toTsvStream(range, store, options?)Canonical data-space rangeReadableStream<Uint8Array>.
toXlsxTableStream(workbook, store, options?)Active-sheet tableReadableStream<Uint8Array>.
toXlsxWorkbookStream(snapshot, options?)Workbook snapshotReadableStream<Uint8Array>.
fromXlsxWorkbookFile(file, options?)Seekable File or BlobPromise<WorkbookSnapshot>.

CSV/TSV remain core-only; XLSX requires backend registration. Existing string and Uint8Array APIs remain available.

Both buffered and Blob XLSX import retain the ZIP directory and parse each worksheet incrementally from bounded inflated chunks. Neither builds a whole-worksheet string or XML tree. Snapshot cells are built as rows stream, with one cell representation rather than a second parsed record. The snapshot still holds every imported cell: streaming is not a lazy or constant-memory workbook store.

Resource limits

Every XLSX table/workbook path shares XlsxWorkbookOptions:

OptionMeaning
maxCellsLogical accepted cells; default 1_000_000.
resourceLimitsOverride specific input, archive, XML, dimension, preservation and output budgets.
signalAbort between bounded codec stages.
onWarningSurface structured fidelity losses.

Defaults are exported as DEFAULT_XLSX_RESOURCE_LIMITS:

ResourceDefault
Input bytes32 MiB
Output bytes128 MiB
Archive entries1,024
Per-entry uncompressed bytes64 MiB
Total uncompressed bytes256 MiB
Kept unmodelled part bytes64 MiB compressed
Compression ratio×100
Sheets256
Rows per sheet1,048,576
Columns per sheet16,384
Merges100,000
Shared strings1,000,000
Style records65,536
XML elements2,000,000
XML depth64
XML attributes per element128
XML text bytes16 MiB

Hard resource bounds are checked before unsafe decompression or allocation. Exceeding one rejects with XlsxResourceError, which names the resource, limit and observed value. The kept-part budget instead drops over-budget graphs with warnings. Do not raise limits merely to suppress failures; budget for the resulting snapshot as well as the codec.

Memory evidence and limits

PR #186 measured seven interleaved fresh-process rounds per side with one CPU slot:

Generated workbookImport pathPeak RSS median before → afterCandidate maximum
200,000 × 8Buffered1,095 → 800 MiB814 MiB
200,000 × 8Blob1,112 → 779 MiB791 MiB
1,000,000 × 8Buffered5,061 → 3,352 MiB3,512 MiB
1,000,000 × 8Blob4,475 → 3,336 MiB3,504 MiB

These are measured workloads with adjusted budgets, not default-limit promises. The later results supersede the earlier #179 import peak-risk observation; they do not establish a universal memory ceiling.

See also

Documentation