MMDM: Measuring Massive Multitask Decision Making

A benchmark called MMDM (Measuring Massive Multitask Decision Making) is proposed to evaluate System-1 Decision Models, which make structured probabilistic decisions over query-specified options from a given state in a single forward pass rather than autoregressively generating reasoning trajectories or text. The work argues that recent evaluations span diverse tasks but often rely on existing benchmarks. No results, model names, or figures are provided in the available text.

System-1 Decision Models directly make structured probabilistic decisions over query-specified options given a state in a single forward pass, without autoregressively generating reasoning trajectories or textual responses. Recent evaluations cover diverse tasks but often rely on existing benchmarks