Recent developments in machine learning research have the potential to create a lasting impact in academic research. From StreamingLLM, a framework that enables large language models (LLMs) to efficiently stream long interactions without fine-tuning, to Batch Calibration, a technique to mitigate the effects of biases in LLMs while recovering performance, to L2CEval, a comprehensive evaluation of the language-to-code generation capabilities of large language models, to TRGL, a new module-wise training technique that can improve accuracy of neural networks while using up to 60% less memory, to a network model of language competition in bilingual societies, to LMMs, specifically GPT-4V(ision), to \texttt{RAFA}, a principled framework to enable autonomous LLM agents to complete tasks with provable sample efficiency, to data filtering networks (DFN) to create high-quality datasets for machine learning, to a novel Transformer-based architecture tailored to tabular data and cross-table representation learning, the potential for these developments to create a lasting impact in academic research is clear.
In
This paper presents StreamingLLM, a framework that enables large language models (LLMs) to efficiently stream long interactions without fine-tuning. It introduces the concept of an attention sink, which allows LLMs to generalize to infinite sequence lengths. Experiments show that StreamingLLM can achieve up to 22.2x speedup in streaming settings, with potential to create a lasting impact in academic research.
CRAFT is a tool creation and retrieval framework for LLMs that enables them to solve complex tasks with specialized toolsets. It provides a plug-and-play approach to adapt off-the-shelf LLMs to unseen domains and modalities, resulting in substantial performance improvements. The potential for this approach to create a lasting impact in academic research is promising.
This paper presents Batch Calibration, a technique to mitigate the effects of biases in LLMs while recovering performance. It provides a unified view of existing calibration methods, and is zero-shot, inference-only, and incurs negligible additional costs. Results show state-of-the-art performance across multiple tasks, potentially creating a lasting impact in academic research.
L2CEval presents a comprehensive evaluation of the language-to-code generation capabilities of large language models, analyzing the factors that affect their performance. The evaluation framework and model outputs are released, providing a basis for further research and potential for lasting impact in academic research.
This paper presents a new module-wise training technique, TRGL, which can improve accuracy of neural networks while using up to 60% less memory. The technique is based on a regularization inspired by the minimizing movement scheme and has the potential to create a lasting impact in academic research by providing a more efficient and effective way to train neural networks.
This paper presents a network model of language competition in bilingual societies, which allows agents to adapt their local interactions in accordance with their language preference. The results of the simulations suggest that this freedom to agents can lead to linguistically segregated communities for small network sizes, and the extinction of one language for larger sizes. The findings of this work have the potential to create a lasting impact in academic research, by helping to understand the impact of speakers' preferences and choices in the complex language landscape of bilingual societies.
This paper explores the potential of LMMs, specifically GPT-4V(ision), to create a lasting impact in academic research. Through carefully designed qualitative samples, the paper demonstrates GPT-4V's ability to process interleaved multimodal inputs and its genericity in a variety of domains and tasks. It also introduces new human-computer interaction methods such as visual referring prompting. The paper concludes with discussions on potential applications and future research directions.
This paper proposes a principled framework, \texttt{RAFA}, to enable autonomous LLM agents to complete tasks with provable sample efficiency. It combines long-term reasoning and short-term acting to reduce uncertainty and maximize value functions, with theoretical analysis proving a $\sqrt{T}$ regret bound. This could have a lasting impact in academic research, as it could enable LLMs to be used in real-world applications.
This paper presents a new technique for data filtering networks (DFN) to create high-quality datasets for machine learning. The authors demonstrate that DFN-5B and DFN-2B datasets enable state-of-the-art models to achieve improved performance on a variety of tasks. The potential for these techniques to create a lasting impact in academic research is clear.
This paper presents a novel Transformer-based architecture tailored to tabular data and cross-table representation learning. Through careful scaling experiments, the authors demonstrate the potential for this architecture to create a lasting impact in academic research by achieving superior performance in both single-table and cross-table pretraining setups.