(Submitted on 31 Jul 2017)
Few-shot learning is challenging for learning algorithms that learn each task in isolation and from scratch. In contrast, meta-learning learns from many related tasks a meta-learner that can learn a new task more accurate and faster with fewer examples, where the choice of meta-learners is crucial. In this paper, we develop an SGD-like, easily trainable meta-learner, called Meta-SGD, that can initialize and adapt any differentiable learner in just one step. Compared to the popular meta-learner LSTM, Meta-SGD is conceptually simple, easy to implement, and can be learned efficiently. Compared to the latest meta-learner MAML, Meta-SGD has a much higher capacity in learning to learn not just the learner initialization, but also the learner update direction and learning rate, all in a single meta-learning process. Meta-SGD shows highly competitive performance for few-shot learning on regression and classification.