NeurIPS 2023 Highlights: Shaping the Next Wave of AI and ML

Editor Note: Pythonic employs a team of experts in engineering, AI, and machine learning. With their insights and expertise, Pythonic solutions simplify the complicated, applying cutting-edge AI to the title industry. This latest post highlights their knowledge and ongoing learning for the company and its clients.

Have you ever wondered what powers the rapid evolution of technology that shapes our world today? Could breakthroughs in artificial intelligence (AI) and machine learning (ML) be the key to unlocking even more AI advancements?

At Pythonic, the data science team recently attended NeurIPS 2023, which is one of the top artificial intelligence conferences in the field of AI. Here’s our summary of the most interesting papers presented to share with you the latest breakthroughs in this field. 

The two metrics of h5-index and h-median typically reflect the conference's influence in the academic community, with higher values indicating greater impact. NeurIPS consistently ranks as one of the premier conferences in the field, known for presenting cutting-edge research. [google scholar metrics page]

NeurIPS 2023 showcased an array of interesting research papers, each contributing valuable insights into machine learning and artificial intelligence. You can access a visualization of all papers online.

At NeurIPS 2023, discussions were concerned with several significant topics that reflect the cutting edge of AI and machine learning research. Large language models (LLMs) were the main topic, emphasizing efficient finetuning, problem-solving frameworks like the "Tree of Thoughts," and a critical examination of emergent abilities in LLMs, questioning whether these are genuine insights or artifacts of evaluation metrics​​. Reinforcement learning also emerged as a key theme, exploring innovative algorithms like Direct Preference Optimization for aligning models with human preferences​​. Neural network optimization was another critical area, focusing on challenges and strategies for scaling language models under data constraints and improving optimization techniques​​.

Here are our highlights:

Scaling Data-Constrained Language Models 

This research focuses on how to effectively grow language models when there's not enough unique text data to use for training—something that's becoming a concern as models get bigger and require more data. The authors conducted a wide range of experiments, exploring what happens when you keep using the same data over and over (up to 900 billion times) while also increasing the model's complexity (up to 9 billion different parameters to adjust during learning). They discovered that reusing data a few times doesn't really hurt the model's learning process, but after a certain point, just making the model or the dataset bigger doesn't help it get any smarter. They suggest a new rule of thumb for figuring out the best balance between the size of the model, how much data you have, and how many times you use that data. They also tried some creative solutions to get around the problem of not having enough data, like adding code snippets to the mix or taking away some common restrictions on what data could be used. Their findings and the data from their 400 experiments are shared openly for others to use and build upon. [paper]

Direct Preference Optimization: Your Language Model is Secretly a Reward Model

Language models (LMs) learn a lot on their own, but guiding them to do what we specifically want is not straightforward due to their non-deterministic/generative nature. Direct Preference Optimization (DPO) is a new, straightforward approach introduced to make these models follow human preferences more accurately. Unlike previous complex methods, such as Reinforcement Learning with Human Feedback (which, allegedly, OpenAI used to train ChatGPT), DPO simplifies the process by reparameterizing the reward model, allowing for the direct extraction of the optimal policy (a strategy for choosing actions) in a more straightforward manner. This process eliminates the need for reinforcement learning loops and simplifies the training process. DPO effectively adjusts the language model using a simple classification loss, making it stable, efficient, and less computationally intensive.

This makes training language models to align with human desires much simpler and more effective, achieving or even surpassing previous methods in terms of controlling the model's output to match what humans prefer. [paper]

Are Emergent Abilities of Large Language Models a Mirage?

Recent studies have highlighted that large language models seem to suddenly gain new abilities as they grow, abilities that weren't there in smaller versions. These abilities pop up unexpectedly and all at once, which is really fascinating but also unpredictable. However, this paper proposes an alternative explanation: maybe these new abilities seem to appear not because the models are actually getting smarter in a new way, but because of how researchers measure their performance. Depending on the type of measurement (or metric) used, a model might seem to gain new abilities suddenly or improve gradually. The authors did some experiments with big language models like InstructGPT/GPT-3 and found that changing the way they measured things could make these so-called new abilities appear or disappear. They suggest that what looks like a new ability might just be an illusion created by the way we're looking at the model's performance, not a real leap in what the model can do. This means we need to be careful about how we interpret these abilities and ensure we're not getting tricked by the metrics we choose. [paper]

Tree of Thoughts: Deliberate Problem Solving with Large Language Models

Large language models( LLMs) are skilled at various tasks but typically follow a linear approach (step-by-step manner), which might not suit problems requiring strategic thinking. To enhance their capabilities, authors have developed a new method called "Tree of Thoughts'' (ToT), which builds on a simpler method known as "Chain of Thought. ToT enables LLMs to think through problems more like a human might, considering different possibilities before making a decision. 

This method improves their ability to solve complex problems by allowing them to rethink or backtrack decisions. Tests showed that ToT greatly improves the models' effectiveness in tasks requiring planning, such as games or creative writing, showing significant improvements in solving mathematical tasks. While this new method may not be necessary for all tasks that LLMs currently excel at, it opens up possibilities for tackling more complex real-world problems in the future. Although implementing ToT can be more demanding on resources, its flexibility allows for a balance between cost and performance over time.  [paper]

Image Details: On the left, previous simple approaches for problem-solving with LLMs are shown, while on the right-hand side, each box represents a thought, which is processed with a Tree search algorithm. See more examples in the paper. Source: Figure 1 from 'Tree of Thoughts: Deliberate Problem Solving with Large Language Models' by S. Yao et al.

Test of Time Award: Word2Vec

This year, a groundbreaking research paper from a decade ago, "Distributed Representations of Words and Phrases and their Compositionality," by Tomas Mikolov and colleagues [paper], was honored with the Test of Time Award. This work, known for introducing the influential word2vec technique, has been a breakthrough in natural language processing (NLP), cited over 40,000 times since its publication at NeurIPS 2013. It showcased how learning from vast amounts of text without explicit structure could significantly advance our understanding and processing of human language, marking a pivotal moment in NLP history.

Image Details: Word2vec was a follow-up on an ICLR 2013 workshop paper. Both papers have been cited many times and have been a breakthrough in NLP. (Source: NeruIPS 2023 Test of Time Award presentations)

Here's a simplified breakdown of the key learnings from this talk and related research (from the presentation slides [source]):

1. The Importance of Semi-Supervised Learning: Learning from large text collections is crucial for understanding natural language, a notion that remains true today.
2. The Efficiency of Parallel, weakly-synchronized Computation: Fast, parallel processing is essential in machine learning, supported by specialized hardware that enables even large-scale operations to run smoothly. Asynchronous methods might see a resurgence in the future.
3. Focus compute where it matters: Focusing efforts on specific learning areas can lead to better performance. Simpler, more parallelizable models often outperform their complex counterparts, as seen in the preference for word2vec over RNNs and Transformers over LSTMs.
4. Advanced Tokenization Techniques: Powerful models have evolved tokenization, moving away from phrase-based vocabulary towards more granular approaches like SentencePiece and sub-word vocabularies.
5. Vector Representations of Language: Treating language as a sequence of dense vectors has proven to be more effective than anticipated, a principle that continues to hold strong in current NLP research.

These insights highlight the evolution of NLP technology, emphasizing the significance of scalable, efficient computational strategies and the ongoing exploration of how best to represent and process language.

Discussion

At the NeurIPS 2023 conference, over 3,500 papers were presented, each showcasing the boundaries of what's possible in AI, offering alternative perspectives and innovative solutions to complex challenges. These works each contribute to the knowledge that defines the evolving machine learning and artificial intelligence disciplines. 

If all this seems overwhelming and complex, take heart. At Pythonic, we're dedicated to seamlessly incorporating the latest technologies and cutting-edge research into our end-to-end solutions. This ensures our products remain on the cutting edge of innovation, freeing you from the burden of dealing with these complexities on your own.