Learning Algorithms from Natural Proofs

Marco Carmosino, Russell Impagliazzo, Valentine Kabanets, and Antonina Kolokolova

Abstract

Based on Håstad's (1986) circuit lower bounds, Linial, Mansour, and Nisan (1993) gave a quasipolytime learning algorithm for $AC^0$ (constant-depth circuits with AND, OR, and NOT gates), in the PAC model over the uniform distribution. It was an open question to get a learning algorithm of any kind for the class of $AC^0[p]$ circuits (constant-depth, with AND, OR, NOT, and MOD$_p$ gates for a prime $p$). Our main result is a quasipolytime learning algorithm for $AC^0[p]$ in the PAC model over the uniform distribution with membership queries. This algorithm is an application of a general connection we show to hold between natural proofs {in the sense of Razborov and Rudich (1997)) and learning algorithms. We argue that a natural proof of a circuit lower bound against any (sufficiently powerful) circuit class yields a learning algorithm for the same circuit class. As the lower bounds against $AC^0[p]$ by Razborov (1987) and Smolensky (1987) are natural, we obtain our learning algorithm for $AC^0[p]$.

Versions