Knoware aspires to make the “emerging capabilities prefiguration” the prime methodology for forecasting, explaining, assessing, and validating (pre-)emerging capabilities in GPAI systems. Prefiguration is the detectable “proto-form” of a capability as evidence that the building blocks, partial behaviours, and enabling conditions are already present, so that the full capability is plausible through scaffolds (with modest scaling, tooling, or integration).
The goal of KnoWare is to shift GPAI capabilities understanding from reactive observation to proactive detection. KnoWare will utilize psychometrics-inspired practices and tests and explainable AI techniques to make gradual improvement visible rather than sudden, and avoid emergence illusions.
For evaluating GPAI models, KnoWare will develop a novel benchmarking approach, beyond narrow task-based performance and aggregated metrics, that considers prefigured capabilities and includes open tasks that measure not just “what the model can do” but also “whether the emerging capability is aligned with human intent” (risk-aligned governance). The approach will incorporate protocols for updating the tasks as new capabilities prefigure, also monitoring how quickly the benchmark saturates. PI: Jan. Proposal team: Ana Maria, Andrea