How to Handle Large JSON Files Without Crashing Your Browser

By Deepak Kumar Developer & builder of JSON Formatter Hub · About the author

May 2025 · Published by JSON Formatter Hub · Report an issue

JSON.parse() loads the entire file into memory before returning anything. For small files this is fine. For a 100MB database export, it will exhaust browser memory and crash the tab. This guide covers the practical strategies that actually work when JSON is too large to handle with standard tools.

The Problem: In-Memory Parsing

Every standard JSON parser — in browsers, Node.js, Python, and Java — works by reading the entire input, building a complete in-memory representation, and then returning it. A 100MB JSON file can expand to 500MB–1GB in memory because the in-memory object graph has overhead for each key, value, and reference. Browser tabs typically have 512MB–2GB of memory limits and no swap. The result is a hard crash.

Rule of thumb:

• Under 5MB: any browser formatter works fine, including this one
• 5–20MB: browser tools are slow but may work; use the strategies below for speed
• Over 20MB: use command-line tools or streaming parsers

Strategy 1: jq — The JSON Swiss Army Knife

jq is a lightweight command-line JSON processor. It streams through the file without loading everything into memory and lets you filter, transform, and extract data. Install it from jqlang.org.

Essential jq commands

# Pretty-print the first 10 array elements
jq '.[0:10]' large-data.json

# Get all keys of the root object
jq 'keys' large-data.json

# Count array items
jq 'length' large-data.json

# Extract a specific field from every object in an array
jq '.[].email' users.json

# Filter: only objects where age > 30
jq '[.[] | select(.age > 30)]' users.json

# Extract just the first item
jq '.[0]' large-array.json

# Get only specific fields from each object
jq '[.[] | {name, email}]' users.json

# Convert array of objects to CSV
jq -r '.[] | [.name, .email, .age | tostring] | join(",")' users.json

Extract just the part you need, then use a browser formatter

# Extract a single nested object
jq '.config.database' app-config.json > database-config.json

# Now paste database-config.json into the browser formatter

Strategy 2: Streaming Parsers (Node.js)

The stream-json npm package streams JSON token by token, so memory usage stays constant regardless of file size:

const { createReadStream } = require('fs');
const { chain } = require('stream-chain');
const { parser } = require('stream-json');
const { streamArray } = require('stream-json/streamers/StreamArray');

let count = 0;
chain([
  createReadStream('large-users.json'),
  parser(),
  streamArray()
]).on('data', ({ value }) => {
  // Process one object at a time — constant memory
  console.log(value.email);
  count++;
}).on('end', () => {
  console.log(`Processed ${count} records`);
});

Strategy 3: Streaming Parsers (Python)

ijson is the Python equivalent — it iterates over items in a large JSON array without loading the whole file:

import ijson

with open('large-users.json', 'rb') as f:
    for user in ijson.items(f, 'item'):
        # 'item' = each element of the top-level array
        print(user['email'])
        # Process one record at a time

For deeply nested paths:

import ijson

with open('response.json', 'rb') as f:
    # Extract items from data.users array
    for user in ijson.items(f, 'data.users.item'):
        process(user)

Strategy 4: Split the File into Chunks

If you need to work with all the data in a browser tool, split the large array into smaller files first:

Using jq

# Split a 100,000-item array into chunks of 1,000
jq -c '.[]' large.json | split -l 1000 - chunk_
# Then wrap each chunk back into an array
for f in chunk_*; do
  jq -s '.' "$f" > "${f}.json"
done

Using Python

import json

CHUNK_SIZE = 1000

with open('large.json') as f:
    data = json.load(f)  # Still loads all into memory once

for i in range(0, len(data), CHUNK_SIZE):
    chunk = data[i:i + CHUNK_SIZE]
    with open(f'chunk_{i // CHUNK_SIZE}.json', 'w') as out:
        json.dump(chunk, out, indent=2)

print(f'Created {len(data) // CHUNK_SIZE + 1} chunk files')

Strategy 5: Import into a Database

For very large files (hundreds of MB or more) that you need to query repeatedly, importing into a database is the right long-term approach.

SQLite with JSON support

-- Load a JSON array into a SQLite table
-- Using the json_each() table-valued function
CREATE TABLE users AS
SELECT
  json_extract(value, '$.id')    AS id,
  json_extract(value, '$.name')  AS name,
  json_extract(value, '$.email') AS email
FROM json_each(readfile('users.json'));

PostgreSQL JSONB

-- Import JSON file then query with operators
COPY staging(raw) FROM '/path/to/data.json';
SELECT raw->>'name', raw->>'email'
FROM staging
WHERE (raw->>'age')::int > 30;

Strategy 6: JSONL (Newline-Delimited JSON)

If you control the data source, use JSONL format for large datasets. Each line is a valid, standalone JSON object. This enables line-by-line processing with standard Unix tools without any special parser:

# Each line is one JSON object:
{"id":1,"name":"Alice","email":"alice@example.com"}
{"id":2,"name":"Bob","email":"bob@example.com"}

# Count lines (= record count)
wc -l data.jsonl

# Extract a field from every line with jq
jq -r '.email' data.jsonl

# Filter with jq
jq 'select(.age > 30)' data.jsonl

# Process line by line in Python
with open('data.jsonl') as f:
    for line in f:
        record = json.loads(line.strip())
        process(record)

Summary: Which Strategy to Use

Compressing Large JSON for Storage and Transfer

A JSON file that is 100MB uncompressed often compresses to 10–20MB with gzip, because JSON is highly repetitive text (repeated key names, whitespace, common values). Compression is almost always worth enabling for large JSON files transmitted over HTTP.

# Compress a JSON file with gzip
gzip -k large-data.json        # creates large-data.json.gz (keeps original)
gzip -d large-data.json.gz     # decompress

# Pipe jq output directly into gzip
jq '.[]' large-data.json | gzip > extracted.jsonl.gz

# Node.js — stream with compression
const { createReadStream, createWriteStream } = require('fs');
const { createGzip } = require('zlib');

createReadStream('data.json')
  .pipe(createGzip())
  .pipe(createWriteStream('data.json.gz'))
  .on('finish', () => console.log('Compressed'));

For HTTP APIs, enable gzip in your server and set Accept-Encoding: gzip on the client. Most frameworks (Express, FastAPI, Spring Boot) have a one-line setting to compress responses automatically.

Diagnosing Memory Problems

If your script or browser tool is running out of memory when processing JSON, these steps help identify the problem:

• Check the file size first — run ls -lh data.json (Linux/Mac) or check properties in Windows Explorer. Anything over 10MB should not go into a browser formatter.
• Count records without loading — use jq 'length' data.json to count top-level array items, or wc -l data.jsonl for JSONL files. This tells you what you are dealing with before you commit to loading everything.
• Sample first — examine just the first record before processing all of them: jq '.[0]' data.json. Understanding the structure first prevents wasted processing.
• Profile Node.js memory — run node --max-old-space-size=4096 script.js to increase the V8 heap limit. If the process still crashes, you need a streaming approach.
• Use Python's memory_profiler — install with pip install memory-profiler then run mprof run script.py to see per-line memory usage.

Large JSON in Cloud Storage

When large JSON files live in cloud storage (Amazon S3, Google Cloud Storage, Azure Blob), you have additional options that avoid downloading the whole file:

• S3 Select — Amazon S3 can run SQL-like queries directly on JSON or CSV files stored in S3, returning only the matching rows. You pay only for data scanned and transferred, which is far cheaper than downloading a 1GB file to extract 1,000 rows.
• BigQuery external tables — Google BigQuery can query JSONL files stored in Cloud Storage directly without importing them. Useful for one-off analysis.
• Azure Blob with Apache Spark — for very large files (multi-GB), reading JSON or JSONL with PySpark distributes the parsing across a cluster, making even terabyte-scale files manageable.
• Convert to Parquet — for repeated queries on the same large JSON dataset, converting to Apache Parquet format reduces file size by 5–10x and makes column-oriented queries 10–100x faster.

# Python: convert JSONL to Parquet using pandas
import pandas as pd

df = pd.read_json('large-data.jsonl', lines=True)
df.to_parquet('data.parquet', compression='snappy')

# Later: read back only the columns you need
df2 = pd.read_parquet('data.parquet', columns=['name', 'email', 'age'])

Related Articles

• How to Parse JSON in JavaScript, Python, Java, and PHP
• How to Convert JSON to CSV, XML, and YAML
• JSON Cheatsheet — syntax quick reference
• JSON Formatter — browser tool for JSON under 5MB