PRX Part 4: Our Data Strategy

Hugging Face pRX Part 4: Our Data Strategy

Building the dataset in Lance Existing captions and embeddings for exploration Profiling and exploring the data 3. Re-captioning everything with a VLM Choosing the captioner 4. Writing Mosaic Data Shards What MDS is, and why we use it over Lance Resolution + aspect-ratio bucketing 5. Data filtering 6. Deduplication What's next? References Welcome back! This is Part 4 of the PRX series. Parts 1 to 3 covered model architectures, training design, and a 24-hour speedrun. This time we’re pulling back the curtain on the part that quietly underpins all of it: the data. Of all the things that shaped PRX’s quality, the data pipeline was one of the least glamorous parts to build but nevertheless an important piece to get right. Here’s what we did, what we’d do differently, and a few things we only learned the slow way. In one sentence: we assemble training data from a mix of public and internal datasets, re-caption the images with a VLM, and turn the result into the streamable corpus we trained PRX on. At a high level, the data pipeline looks like this: The goal was to assemble a large, diverse dataset for pre-training. At this stage the model is learning how the world looks: the visual concepts, the objects and scenes, how things are composed and lit, and the sheer range of what images can contain. That is a problem of coverage and diversity, not of per-image perfection.