Recent developments in machine learning research have the potential to revolutionize the way we interact with technology. From Dynamic Sparsified Transformer Inference (DSTI) to Neural PG-RANK, researchers are pushing the boundaries of what is possible with machine learning. DSTI can reduce the computational cost of Transformer models by up to 60%, while FIRE improves the generalization of Transformers to longer contexts. Amortizing intractable inference in large language models allows for data-efficient adaptation of LLMs to tasks that require multi-step rationalization and tool use. Results from evaluating the performance of Large Language Models (LLMs) on benchmark biomedical tasks suggest that LLMs have the potential to create a lasting impact in academic research. Compressing and selectively augmenting retrieved documents can improve the performance of language models, while Neural PG-RANK provides a principled method for end-to-end training of retrieval models. LATS unifies reasoning, acting, and planning in LLMs, and a novel keyword augmented retrieval framework integrates a speech interface to enable quick and low cost information retrieval. Finally, a large-scale Korean text dataset for class
This paper presents a method, Dynamic Sparsified Transformer Inference (DSTI), to reduce the computational cost of Transformer models by exploiting their activation sparsity. DSTI can be applied to any Transformer-based architecture and has minimal impact on accuracy, resulting in a 60% reduction in inference cost. This could have a lasting impact on academic research, allowing for more efficient use of resources.
This paper presents a novel functional relative position encoding, FIRE, which improves the generalization of Transformers to longer contexts. Theoretically proven to represent popular relative position encodings, FIRE has been empirically shown to create a lasting impact in academic research by improving the performance of Transformers on longer inputs.
This paper presents a novel approach to amortizing intractable inference in large language models, allowing for data-efficient adaptation of LLMs to tasks that require multi-step rationalization and tool use. This could have a lasting impact in academic research, as it provides a powerful tool for exploring complex language tasks.
This paper evaluates the performance of Large Language Models (LLMs) on benchmark biomedical tasks. Results show that LLMs can outperform current state-of-the-art models in biomedical datasets with smaller training sets. This suggests that LLMs have the potential to create a lasting impact in academic research of the described techniques.
This paper presents a novel approach to improve the performance of language models by compressing and selectively augmenting retrieved documents. The proposed compressors reduce computational costs while still providing relevant information to the model, resulting in improved performance on language modeling and open domain question answering tasks.
This paper presents Neural PG-RANK, a novel training algorithm that optimizes language models for text retrieval tasks by leveraging policy gradient. This technique has the potential to create a lasting impact in academic research by providing a principled method for end-to-end training of retrieval models with little reliance on complex heuristics, and by effectively unifying the training objective with downstream decision-making quality.
This paper presents a novel approach to prevent overoptimization of reward models in academic research. It uses constrained reinforcement learning to prevent the agent from exceeding each reward model's threshold of usefulness, and an adaptive method to identify and optimize towards these points during a single run. This could create a lasting impact in academic research by improving evaluation performance and better aligning language models with human preferences.
This paper introduces LATS, a framework that unifies reasoning, acting, and planning in LLMs. It repurposes the latent strengths of LLMs for enhanced decision-making, and uses an environment for external feedback to provide a more deliberate and adaptive problem-solving mechanism. Experiments across diverse domains show the effectiveness and generality of LATS, with potential to create a lasting impact in academic research.
This paper presents a novel keyword augmented retrieval framework that integrates a speech interface to enable quick and low cost information retrieval from structured and unstructured data. The proposed framework has the potential to create a lasting impact in academic research by reducing inference time and cost, and providing a seamless interaction with language models.
This paper introduces a large-scale Korean text dataset for classifying biased speech in real-world online services. Leveraging state-of-the-art BERT-based language models, the proposed approach surpasses human-level accuracy across diverse classification tasks. This work has the potential to create a lasting impact in academic research by providing practical solutions for real-world hate speech and bias mitigation, contributing directly to the improvement of online community health.