Marco Nurisso - Bound by semanticity: universal laws governing the generalization-identification tradeoff

Abstract

Intelligent systems must deploy internal representations that are simultaneously structured—to support broad generalization—and selective—to preserve input identity. We expose a fundamental limit on this tradeoff. For any model whose representational similarity between inputs decays with finite semantic resolution ϵ, we derive closed-form expressions that pin its probability of correct generalization pS and identification pI to a universal Pareto front independent of input space geometry. Extending the analysis to noisy, heterogeneous spaces and to n>2 inputs predicts a sharp 1/ncollapse of multi-input processing capacity and a non-monotonic optimum for pS. A minimal ReLU network trained end-to-end reproduces these laws: during learning a resolution boundary self-organizes and empirical (pS,pI) trajectories closely follow theoretical curves for linearly decaying similarity. Finally, we demonstrate that the same limits persist in two markedly more complex settings—a convolutional neural network and state-of-the-art vision–language models—confirming that finite-resolution similarity is a fundamental emergent informational constraint, not merely a toy-model artifact. Together, these results provide an exact theory of the generalization-identification trade-off and clarify how semantic resolution shapes the representational capacity of deep networks and brains alike.

Marco Nurisso is a Ph.D. student in Pure and Applied Mathematics at Politecnico di Torino and member of CENTAI Institute. His research interests lie in the field of applied topology and geometry and their applications to network science, artificial intelligence and cognitive science. His current research focuses on the topology of learning of neural networks and the processing limits induced by representational geometry.