Artificial Intelligence1 min read

Mitra-v2: Scaling synthetic pretraining for tabular foundation models

By · Published by Everything Blog

In short

The system was developed using entirely synthetic prediction problems, entirely omitting customer and real-world data during its pretraining phase. Compared to the prior release, it scales its underlying training pipeline to accommodate tables with ten times more rows and triple the columns. During production inference, singular-value decomposition is employed to compress datasets that surpass its standard dimensional limits. Evaluations show it finished all tasks within a one-hour threshold while surpassing its predecessor across 32 of 33 shared datasets.

Key points

  • Deploying the model eliminates the standard requirement to build and tune specialized mac…: Deploying the model eliminates the standard requirement to build and tune specialized machine learning algorithms for distinct tabular datasets.
  • The architecture handles classification and regression directly on unseen tables by learn…: The architecture handles classification and regression directly on unseen tables by learning context from provided labeled rows.
  • The open-source model is released under an Apache-2.0 license to allow broad implementati…: The open-source model is released under an Apache-2.0 license to allow broad implementation for structured data analytics.

Sources checked by our analysis pipeline: 1

Much of the data that businesses run on lives in tables: the rows might represent customers, transactions, or interactions, and their “attributes” are listed in the columns. Predicting missing attributes — such as likelihood of customer retention or how much demand to expect next week — is one of the most common machine learning tasks in industry.

The standard tools for this job, such as gradient-boosted decision trees, produce models tailored to specific tables. Every new table means choosing an algorithm, engineering features, tuning hyperparameters, and setting up a processing pipeline. Tabular foundation models promise to end that cycle: one pretrained model reads the labeled rows of a previously unseen table and predicts the rest, with no training run in-between.

Last year we introduced Mitra, a tabular foundation model pretrained entirely on synthetic tables. It worked well on small and mid-sized tables but ran out of room on large ones, wide ones, and problems with many classes.

Today we're introducing Mitra-v2, the second generation of the model, for both classification and regression. We describe it in a technical report, and…

Artificial Intelligence