Autonomous agents, learning and the future of AI: six questions to Piotr Mirowski (Google DeepMind)

INNS Big Data Conference website:

Piotr MirowskiPiotr Mirowski is a Research Scientist at Google DeepMind, the research lab focused on “solving intelligence”, and investigating new research directions such as deep reinforcement learning or systems neuroscience. Dr. Mirowski has a decade-long experience on machine learning at its highest level: after obtaining his Ph.D. in computer science (2011) at New York University under the supervision of Prof. Yann LeCun, he worked at Bell Labs and Microsoft Bing before joining DeepMind. Before that, he worked as a research engineer at Schlumberger Research. We interviewed him on his activities, the current rise of deep learning, and the new challenges to look forward in AI.

Piotr Mirowski will hold a tutorial on “Learning Sequences” on the 24th October – do not miss it if you attend the conference!

INNS BigData: Can you describe briefly your research interests and main activities?

Piotr: The common thread running through my research interests is learning sequences. It started when, in 2002 and freshly out of engineering school, I was working at Schlumberger on geological logs, classifying rocks deposited through sedimentary processes using a combination of neural networks and hidden Markov models. It continues today when I train robotic agents using deep reinforcement learning at DeepMind.

Google DeepMind's Deep Q-learning playing Atari Breakout
Google DeepMind’s Deep Q-learning playing Atari Breakout (Mnih et al., Nature, 2015)

Recurrent neural networks and non-convex optimization are my everyday tools now, even though, in the past, I have worked on data-poor applied problems that required a different approach (e.g. learning gene regulation networks from sequences of mRNA or electric load forecasting from hourly transformer measurements). Today, in my work at DeepMind, I am not only interested in solving a time series prediction problem, but also in discovering insight about a sequence by learning its possible representations in the hidden units of an RNN, revisiting with fresh eyes some of the work I did during my PhD with Yann LeCun at NYU.

INNS BigData: Since 2014, you work as a research scientist at Google DeepMind. What project(s) are you involved with?

Piotr: I work within the deep learning research team, which focuses on fundamental research in various aspects of representation learning. We work primarily on image and sound data (there are other teams at DeepMind who work on text or other type of structured or unstructured data). This research can include training generative models of images building upon variational auto-encoders or deep reinforcement learning for learning to control agents playing games.

Whilst I am still in the process of publishing my current work, I can mention some of the work published by my colleagues from the deep learning research team, which includes Pixel RNN, PixelCNN and WaveNet, which is their application to voice synthesis. Unsupervised Learning of 3D Structure from Images, Asynchronous Deep Reinforcement Learning which beats the previous Deep Q-Learning baseline for playing Atari games, or Progressive Neural Nets, which are a way to address the problem of continual learning across a variety of different tasks (without forgetting how to solve earlier tasks).

INNS BigData: DeepMind is on a quest to “solve intelligence”. After mastering deep learning, what is the next step according to your view?

The next steps […] are to learn autonomous agents solving increasingly complex tasks in increasingly complex environments

Piotr: I hope that not all of deep learning is considered as having been mastered yet! What may have been mastered are simple pattern recognition tasks with fully labelled supervision. The next steps, on which we are working at DeepMind, are to learn autonomous agents solving increasingly complex tasks in increasingly complex environments, where the only labelled supervision comes from very sparse rewards during reinforcement learning. The challenges now are to learn good representations about the environment, to make predictions about it, and to learn a fully differentiable memory to solve complex tasks in one-shot.

INNS BigData: Before DeepMind, you worked at Microsoft and the Bell Laboratories. Have you found comparable research environments?

Piotr: Bell Labs was defined, from the forties to the nineties, as the ideal corporate research environment. The management was very supportive of research carried out by individuals and had the courage of undertaking ambitious and risky projects. The funding model was simple and came directly from AT&T operations. For more details, I recommend reading The Idea Factory, a book that gives thrilling examples of achievements at Bell Labs: the first transatlantic cable, the transistor, the first telecommunication satellite Telstar or CCD cameras. What one could add to that book is the impact of research in statistical learning and pattern recognition at Bell Labs, which resulted in such methods as Support Vector Machines, Boosting and of course, Convolutional Networks and the beginning of Deep Learning.

Google DeepMind: Ground-breaking AlphaGo masters the game of Go
Google DeepMind: Ground-breaking AlphaGo masters the game of Go (David et al., Nature, 2016)

I worked at Bell Labs in Murray Hill, NJ, between 2011 and 2013, with Tin Kam Ho, and I tremendously benefitted from the research freedom and Tin’s support, the academic focus, and the inspiration from Bell Labs’ history. I then briefly worked in a product team at Microsoft Bing, and greatly enjoyed interacting with colleagues from Microsoft Research. I believe that today, several corporate research labs, including DeepMind, Microsoft Research and a few others have taken the relay of ambitious academic research in the tradition of Bell Labs.

What makes DeepMind stand out are focus and collaboration as well as academic rigour. Researchers are not only encouraged to conduct independent research but also to participate in regular, lab-wide discussions. As a result, most of the papers come from the joint effort of different teams. We all work towards the same goals (solving artificial general intelligence), starting from different perspectives.

INNS BigData: Back to deep learning, it has already revolutionized image and audio processing, and many agree that the next challenges await in the biomedical field. What should we expect in the near future?

Piotr: The analysis of medical data is definitely a field that could benefit from advances in deep learning, and DeepMind Health has just begun exploring whether ML methods could reduce the amount of time it takes to plan radiotherapy treatment for head and neck cancers. Another interesting field of application of research in AI is on text understanding, as illustrated by the recent push into conversational bots.

We all work towards the same goals (solving artificial general intelligence), starting from different perspectives

INNS BigData: More in general, which are going to be the biggest technological challenges in the future, according to your point of view?

Training deep learning models is expensive in terms of energy and time, especially when one considers doing all the possible hyper-parameter sweeps. While a few of my colleagues at DeepMind have successfully used machine learning to reduce the energy footprint of data servers, the models and problems keep growing in size and complexity. A solution could come from new hardware or from machine learning algorithm themselves.

INNS BigData: To close, what would you say to a student starting today his/her PhD in machine learning?

Piotr: The PhD will be your most intense and fulfilling educational experience!

  • Do not be shy of internships in corporate research labs, because they give you exposure to large-size datasets and to state-of-the-art computing facilities. Some of the best researchers that I have met were in such labs, and your internship advisor could give you ideas for your PhD.
  • Interacting with a co-advisor (whether it be from an academic or corporate research environment) will give you a second opinion on everything you do, potentially new projects and opportunities (as well as extra research duties).
  • If your PhD programme allows you to take classes, I would recommend brushing up on linear algebra, optimisation, statistics, and graphical models. If you are interested in AI, consider taking a class in computational neuroscience.
  • If you can, take advantage of the fact that you are at a pluridisciplinary university, and take (or audit) classes that inspire you. A PhD is also a unique opportunity for learning new things.
  • Something I wish I had done more during my studies: participating in hackathons and in big data competitions.

Further readings

Learn more about the activities of Dr. Mirowski on his blog and his Twitter account!

About the Author

Simone Scardapane is a post-doc fellow in La Sapienza, Rome. He is enthusiast about machine learning technologies, and he is helping as a publicity chair for the conference.