Recent benchmarks on an AWS instance with two 192-core AMD EPYC processors demonstrated dramatic speedups using memory-mapped files and multi-threaded execution. Standard single-core Python parsing typically maxes out at 0.28 GB/s, whereas fastsimdjson processed a 7.6 GB GitHub archive dataset in just 55 milliseconds at 138 GB/s. Performance proved especially strong when extracting targeted fields rather than aggregating simple counts. Across a suite of six test datasets, fastsimdjson consistently matched or outpaced alternative tools like msgspec, achieving a maximum speed of 277 GB/s on a Twitter dataset.
Key points
Python 3.14's free-threaded build allows concurrent multi-threaded execution without the…: Python 3.14's free-threaded build allows concurrent multi-threaded execution without the traditional Global Interpreter Lock bottleneck.
Fastsimdjson scaled up to 192 threads to reach a peak processing speed of 277 GB/s on a T…: Fastsimdjson scaled up to 192 threads to reach a peak processing speed of 277 GB/s on a Twitter dataset.
Process pool alternatives like msgspec degraded beyond 96 workers, while the threaded fas…: Process pool alternatives like msgspec degraded beyond 96 workers, while the threaded fastsimdjson approach scaled cleanly across all 192 cores.
Sources checked by our analysis pipeline: 1
14 November 2026 · 3 min
Many systems log their data as JSON, one record per line (NDJSON). GitHub, for example, publishes every public event on its site that way through GH Archive: about a gigabyte per hour. Suppose you want to know which repositories received the most stars over eight hours. That is 7.6 GB of JSON, 2.2 million events.
In Python, the obvious code reads each line with the json
module:
for line in open("events.ndjson", "rb"):
event = json.loads(line)
if event["type"] == "WatchEvent":
stars[event["repo"]["name"]] += 1
It runs at 0.28 GB/s: 27 seconds for this file. The orjson package is twice as fast. The msgspec package, which decodes only the fields you declare, is eight times as fast.
A server has many cores, but a Python process with the GIL runs Python code on one core at a time. The usual answer is a pool of processes: cut the file in pieces, give a piece to each process, and add up their counts. Each process returns a small table, so little has to come back.
Modern processor architectures still suffer from slow integer division, forcing major compilers to substitute constant divisions with sequences of multiplication and shifts. Standard toolchains often introduce extra corrective instructions to handle multiplier overflow during signed calculations. Computer scientists have now designed a streamlined mathematical approach using 64-bit operations that eliminates these corrective steps and lowers instruction latency.
European regulators are intensifying their scrutiny of big tech's mobile operating system permissions and personal data handling. The significant financial penalty follows findings that personal media was improperly acquired from mobile devices without proper disclosure. This major enforcement action highlights mounting legal challenges surrounding unauthorized data collection practices across widespread digital platforms.
Repurposing legacy consumer electronics often requires creative workarounds to bypass modern software incompatibilities. By pairing human ingenuity with artificial intelligence troubleshooting, an outdated e-reader was successfully adapted into a functional, book-themed timepiece. The project demonstrates how aging hardware can be saved from landfills through iterative problem-solving and automated stability fixes.