Rethinking Pretraining for Specialized Design Data: Evidence from the JONES-19 Cultural Design Dataset

A new study from arXiv (arXiv:2608.00135v1) evaluating Convolutional Neural Networks on the JONES-19 cultural design dataset finds that learning from scratch with repeated local sampling (multi-crop) can match the discriminative performance of ImageNet pretraining, challenging the reliance on massive general-purpose pretraining for specialized design data. The authors suggest that carefully curated smaller datasets capturing design principles may be more effective than large-scale data collection in specialized design domains.

arXiv:2608.00135v1 Announce Type: new Abstract: Design and architectural archives encode expert human knowledge in graphical formats, providing a critical testbed for design-inspired Machine Learning ML challenges absent with typical computer vision benchmarks. Building on JONES-19, a small-size image dataset based on The Grammar of Ornament London, 1857 , we evaluate the discriminative performance of Convolutional Neural Networks CNNs in two model training strategies: a ImageNet pretraining for domain-general "visual common sense," and b learning from scratch on the design data in JONES-19. We find that while domain-general priors improve discriminative performance, learning from scratch augmented with repeated local sampling multi-crop effectively recovers these gains. For highly structured design data, local design-driven representations provide sufficient foundation for learning, challenging a reliance on massive general-purpose pretraining. These findings suggest that in specialized design domains, careful curation of smaller high-quality datasets that capture empirical and formal design principles may prove more effective and informative on the nature of a particular design domain than prioritizing large-scale data collection.