Sessions,
by Silver Beach
Weekly guided tours through foundational and cutting-edge research
Deep Learning Track
Neural Networks and the backpropagation algorithm were invented before the AI Winter.
In the last 10 years, they have been thrust back into the limelight as Deep Learning. What happened? What changed? This series explores deep learning in the context of major papers in the field which demonstrated breakthrough capabilities, and discusses connections to other fields and prior work.
We'll cover a new paper every week - you can sign up for all of them, or just drop into the ones you're fascinated in.
Potential Topics
- PAC Theory
- Compare original paper with Prof. Haghtalab's lecture notes
- Talk about how once things get really established the presentation can get cleaned up and simplified for textbooks -- most papers will be harder to parse than the textbook version
- Explain that empirical risk minimization actually does work well in iid settings, with limited hypothesis classes, etc
- Conv Nets
- Convolutions inspired by neurological studies and locally receptive fields
- AI as replicating human abilities vs doing engineerable thing
- AlexNet
- GPUs + Data Availability to make NNs take off
- Ideas can be too early, unrelated other things (like GPUs for graphics/videogames) can be really useful
- Deep Q Learning
- At first glance, it’s temporal differences + CNNs
- Why experience replay is super important
- GANs
- How do you generate data?
- Adversarial games
- AlphaGo
- Adversarial self-play (also mention TD-Gammon)
- Superhuman abilities with lots of data
- It was not obvious that Go was going to be doable any time soon
Details
A Session is an hour-long online discussion, in which you, your guide, and up to 11 other participants dissect and debate a particular research paper, as well as explore potential new directions that stem from that work.
In the week prior to the Session, your guide will provide instructions for preparatory work that could include reading some related work, watching a pre-recorded lecture, and writing down your thoughts about specific parts of the research being discussed.
Every Session participant will then actually read the paper and make annotations. We have an online tool that lets the entire group annotate papers collaboratively. Your annotations should be questions, comments, or ideas you had while reading the paper -- anything that will enhance your and your peers’ understanding of the ideas behind it.
Next, your guide will schedule an optional group ‘pre-session’ to answer any questions or clear any doubts that have prevented you or your peers from getting through the whole paper.
Your guide then goes through all the annotations before the Session. During the actual Session, your guide will moderate and direct discussion between you and the rest of the group, guiding you towards a collaborative understanding of the paper, the underlying research, the ideas behind it, and its context in and impact on the field as a whole.
At the end of the week, you will be asked to send in a list of your own ideas for research that could expand on that week’s topic, generate more theories, and further develop the field.
The topics and papers to be covered each week will be posted online in advance, so you can drop in for a single Session that interests you, book seats in advance for the Sessions you like, attend Sessions in different fields, decide to attend every Session in a given track for a year to get a deep understanding of the ideas underpinning a single field of research, or anything in between.
Each Session booking includes an optional 10 minute individual office hours slot with your guide, which can be scheduled at your mutual convenience anytime during the week
I am Aaron Tucker, a third year PhD student in Computer Science at Cornell University, where I am fortunate to be advised by Thorsten Joachims. I graduated from Harvard College in 2016, then worked as a software engineer doing antifraud for Sendwave. I previously interned at UC Berkeley’s Center for Human-Compatible AI researching inverse reinforcement learning, and was a summer fellow at the Centre for the Governance of AI at the University of Oxford.
With the synchronous sessions and reading you have to do, we suggest you commit around 5 hours/week for Sessions.
Each week costs $15 to participate in — this will include the session, the curated preparatory material, access to the shared annotations, and the pre-session, as well as an optional 10 minute personal office hours slot with your guide.
We don’t want affordability to be an obstacle for anyone who is genuinely curious and wants to attend Sessions. Scholarships are available, and each session will have at least three spots reserved for a scholar! Just one condition: as a scholar, you will have the responsibility to prepare notes from the Sessions you’re a part of. Eventually, we’ll make these notes freely available to the public, as a way of giving back to the scientific community!