PostHog decided to replace its new DuckDB-based warehouse architecture after discovering that single-node engines present severe operational challenges in the cloud. Running the engine as a managed service made estimating CPU and memory requirements difficult, while spilling queries to disk was far too slow. The platform also hit major scaling limits with its DuckLake lakehouse extension during large-scale testing. Additionally, supporting standard Postgres network connections over port 5432 introduced heavy operational overhead and complicated load balancing.
Key points
Single-node database engines create unpredictable resource provisioning challenges when d…: Single-node database engines create unpredictable resource provisioning challenges when deployed as managed multi-tenant cloud services.
Disk spilling and synchronous Amazon S3 data fetching proved too slow and costly for dema…: Disk spilling and synchronous Amazon S3 data fetching proved too slow and costly for demanding customer analytical workloads.
Providing native Postgres protocol access introduces complex load balancing issues and hi…: Providing native Postgres protocol access introduces complex load balancing issues and high connection maintenance overhead compared to standard HTTP interfaces.
Sources checked by our analysis pipeline: 1
Contents
Five months ago, we rebuilt our warehouse on DuckDB instead of ClickHouse. We did it because a shared ClickHouse cluster couldn't handle hours-long modeling queries, and it lacked a cost-based query optimizer. DuckDB gave every customer their own instance, and tools like dbt could connect over Postgres. We're still fans.
But we underestimated how big and complex our customers' workloads would get. We didn't want them to have just a good experience, we wanted the best experience you can get when you pay someone else to run your warehouse. It should work at any scale, at a fair price.
So we spent the last three months re-architecting it, this time without DuckDB.
DuckDB is brilliant on one machine
We love DuckDB. It's probably the best single-node tool for analyzing heaps of data, and that node can be your laptop. It's robust and reliable. It almost always finishes your query, and if it runs out of memory, it spills to disk.
The same features that are great locally gave us problems in the cloud. Customers ran more complex queries than…
Development teams are actively scaling back feature sets to prioritize basic functions like FaceTime and lighting automation, leaving the product's release window in limbo between next year and 2028. Although the hardware has sat completed for about two years, critical software roadblocks are preventing the company from bringing its stationary tabletop assistant to market.
The multimedia technology company is expanding its local workforce by targeting emerging talent in the Colorado area. This recruitment drive offers recent engineering and computer science students an entry point into specialized development positions. The initiative strengthens the organization's regional engineering capabilities through dedicated entry-level professional roles.
A budget-friendly ThinkPad X250 acquired in 2018 continues to handle daily programming and travel tasks despite its aging specifications. Powered by an efficient Linux setup, the compact machine provides exceptional battery life alongside straightforward hardware maintainability. This longevity demonstrates that modular, older electronics can remain viable development tools while offering a sustainable alternative to costly modern hardware.