Interesting Research Papers Presented By Meta AI At NeurIPS 2021
Meta AI researchers will be presenting a total of 83 papers at NeurIPS 2021. NeurIPS 2021 is the Thirty-fifth Conference on Neural Information Processing Systems held from December 6 to 14. The virtual conference boasts 2334 papers, 60 workshops, eight keynote speakers, and 15k+ attendees. In 2020, Meta AI (previously Facebook) presented 48 research papers at Neurips 2020. This article highlights the top 10 featured research publications by Meta at NeurIPS 2021. 1. Unsupervised Speech Recognition Authors: Alexei Baevski, Wei-Ning Hsu, Alexis Conneau, Michael Auli The research paper highlights the limitations of data labelling training for ML models. Therefore, the authors put forth in its place, wav2vec-U, short for wav2vec Unsupervised, a method to train speech recognition models without any labelled data. For this, the authors leverage self-supervised speech representations to segment unlabeled audio and learn a mapping from these representations to phonemes via adversarial training. Find the research paper here. 2. Bandits with Knapsacks beyond the Worst-Case Analysis Authors: Karthik Abinav Sankararaman, Aleksandrs Slivkins In this research, the authors present three original results beyond the worst-case scenario analysis of Bandit with Knapsacks (BwK). The results are built on the BwK algorithm from Agrawal and Devanur (2014), providing new analyses thereof. Firstly, the authors provide upper and lower bounds for a complete characterisation of logarithmic rates. Secondly, the authors consider the “simple regret” in BwK which tracks the algorithmic performance to prove that it is small in all but a few rounds. Finally, the authors provide a general “reduction” from BwK to leverage helpful structures and apply reduction to semi-bandits. Find the research paper here. 3. Volume Rendering of Neural Implicit Surfaces Authors: Lior Yariv, Jiatao Gu, Yoni Kasten, Yaron Lipman This research paper on computer vision and core machine learning improves geometrical representation and reconstruction in neural volume rendering. The authors take the approach of modelling the volume density as a function of the geometry to produce high-quality geometry reconstructions, outperforming relevant baselines. Find the research paper here. 4. Parameter Prediction for Unseen Deep Architectures Authors: Boris Knyazev, Michal Drozdzal, Graham Taylor, Adriana Romero Soriano The authors believe that deep learning has successfully automated the design of features in machine learning pipelines. However, the algorithms optimising neural network parameters remain largely hand-designed and computationally inefficient. Therefore, the authors study whether they can use deep learning to predict these parameters directly by exploiting the past knowledge of training other networks. Find the research paper here. 5. Learning Search Space Partition for Path Planning Authors: Kevin Yang, Tianjun Zhang, Chris Cummins, Brandon Cui, Benoit Steiner, Linnan Wang, Joseph E. Gonzalez, Dan Klein, Yuandong Tian The research paper develops a novel formal regret analysis for when and why an adaptive region partitioning scheme works. Furthermore, the authors propose a new path planning method, LaP3, which improves the function value estimation within each sub-region and uses a latent representation of the search space. Find the research paper here. 6. Antipodes of Label Differential Privacy: PATE and ALIBI Authors: Mani Malek, Ilya Mironov, Karthik Prasad, Igor Shilov, Florian Tramer The authors propose two novel approaches to preferential privacy ML where the trained model satisfies differential privacy with respect to labels of the training examples. The two approaches, the Laplace mechanism and the PATE framework, demonstrate their effectiveness on standard benchmarks. Find the research paper here. 7. NovelD: A Simple yet Effective Exploration Criterion Authors: Tianjun Zhang, Huazhe Xu, Xiaolong Wang, Yi Wu, Kurt Keutzer, Joseph E. Gonzalez, Yuandong Tian The research paper presents NovelD as an alternative exploration method to RND. The criterion called NovelD weighs every novel area approximately equally. The authors believe that the algorithm is very simple yet shows comparable performance or even outperforms multiple SOTA exploration methods in many hard exploration tasks. The researchers have also discovered that NovelD outperforms RND in many Atari Games. Find the research paper here. 8. Luna: Linear Unified Nested Attention Authors: Xuezhe Ma, Xiang Kong, Sinong Wang, Chunting Zhou, Jonathan May, Hao Ma, Luke Zettlemoyer The research paper proposes Luna, a linear unified nested attention mechanism that approximates softmax attention with two nested linear attention functions, yielding only linear (as opposed to quadratic) time and space complexity. The alternative to traditional attention mechanisms, Luna, introduces an additional sequence with a fixed length as input and an additional corresponding output, allowing Luna to perform attention operation linearly while storing adequate contextual information. Find the research paper here. 9. Interesting Object, Curious Agent: Learning Task-Agnostic Exploration Author: Simone Parisi, Victoria Dean, Deepak Pathak, Abhinav Gupta In this research paper on reinforcement learning, the authors propose a paradigm change in the formulation and evaluation of task-agnostic exploration. The authors suggest that the agent first learn to explore many environments without any extrinsic goal in a task-agnostic manner. Later on, the agent effectively transfers the learned exploration policy to explore new environments better when solving tasks. Find the research paper here. 10. DOBF: A Deobfuscation Pre-Training Objective for Programming Languages Authors: Baptiste Rozière, Marie-Anne Lachaux, Marc Szafraniec, Guillaume Lample In this research paper, the authors introduce a new pre-training objective, DOBF, that leverages the structural aspect of programming languages and pre-trains a model to recover the original version of obfuscated source code. The authors demonstrate how models pre-trained with DOBF significantly outperform existing approaches on multiple downstream tasks. The authors, during their research, also discovered that their pre-trained model could deobfuscate fully obfuscated source files, and suggest descriptive variable names. Find the research paper here.
The article explores the top ten featured research publications by Meta AI at NeurIPS 2021
The article explores the top ten featured research papers by Meta AI at NeurIPS 2021 on reinforcement learning, computer vision
["AI (Artificial Intelligence)", "Meta AI"]
["Meta AI", "machine learning", "TPU", "AI", "neural network", "ML", "computer vision", "RAG", "deep learning", "differential privacy", "AI (Artificial Intelligence)"]
["AI", "machine learning", "ML", "deep learning", "neural network", "computer vision", "Meta AI", "differential privacy", "RAG", "TPU"]
Abhishree is a budding tech journalist with a UGD in Political Science. In her free time, Abhishree can be found watching French new wave classic films and playing with dogs.