Recent developments in machine learning research have the potential to revolutionize the field. From reducing the costs of training foundation models to providing improved accuracy and efficiency, the advances in this area are creating a lasting impact in academic research. In this newsletter, we present a selection of papers that explore the potential of machine learning research, from a new FP8 mixed-precision framework for training large language models to a novel dataset to evaluate the performance of large language models in handling controversial issues. We also explore the potential for language models to discern truth from falsehood in contradicting data, a multimodal deep learning approach to identify fake news in Malayalam, and an automated approach to fine-tune pre-trained language models for autonomous systems. Finally, we present a novel approach to improve the efficiency of incremental processors in NLP and a leader-follower bilevel framework that uses reinforcement learning to generate high-quality prompts for decision making with LLMs. All of these papers have the potential to create a lasting impact in academic research, and we look forward to seeing the potential breakthroughs that will come from them.
This paper presents a new FP8 mixed-precision framework for training large language models, which offers significant memory and speed improvements without compromising accuracy. This could have a lasting impact in academic research, as it reduces the costs of training foundation models and can be applied to other tasks such as instruction tuning and reinforcement learning.
This paper presents a Disentangled Graph-Text Learner (DGTL) model to enhance the reasoning and predicting capabilities of large language models (LLMs) for text-attributed graphs. DGTL incorporates graph structure information through tailored disentangled graph neural network layers, allowing LLMs to capture intricate relationships hidden in TAGs. Experiments show that DGTL outperforms state-of-the-art baselines and provides natural language explanations for predictions, creating a lasting impact in academic research of the described techniques.
This paper presents a novel dataset to evaluate the performance of large language models (LLMs) in handling controversial issues. The dataset provides human-annotated labels to measure the LLMs' stances on current debates, and the evaluation of the dataset reveals potential improvements in LLMs' comprehension of complex societal debates, creating a lasting impact in academic research.
This paper presents a leader-follower bilevel framework that uses reinforcement learning to generate high-quality prompts for decision making with LLMs. The framework is capable of learning to ask relevant questions and guide the learning of actions to be performed in an environment, leading to decisive, high-performing actions. The potential for this technique to create a lasting impact in academic research is significant.
This paper explores the potential for language models to discern truth from falsehood in contradicting data by modeling a "truthful persona" - a group of agents likely to produce truthful text. Results suggest that models can separate true and false statements and generalize truthfulness across agents, creating a lasting impact in academic research of the described techniques.
This paper presents a multimodal deep learning approach to identify fake news in Malayalam, a regional language in India. The proposed model has the potential to create a lasting impact in academic research, as it is the first to extract features from multiple modalities to classify fake news in the language. The model is expected to be more accurate than single-modality approaches, providing an effective way to detect and mitigate misinformation.
This paper presents a novel dataset, X-CLAIM, consisting of 7K real-world claims collected from multiple social media platforms in five Indian languages and English. The authors demonstrate the potential of training on multiple languages to identify claims in social media posts, and show that state-of-the-art encoder-only language models can achieve strong baselines. The findings of this paper have the potential to create a lasting impact in academic research of the described techniques, by providing a better understanding of how to identify claims in multilingual social media.
ArcheType is a novel framework for open-source column type annotation using large language models. It enables zero-shot classification performance on a wide range of tasks, and has the potential to create a lasting impact in academic research by providing improved accuracy and efficiency.
This paper presents an automated approach to fine-tune pre-trained language models for autonomous systems, reducing the cost of sourcing human feedback. The method synthesizes controllers from pre-trained models, verifiable against independently provided specifications. Results indicate an improvement in percentage of specifications satisfied, potentially creating a lasting impact in academic research of the described techniques.
This paper presents a novel approach to improve the efficiency of incremental processors in NLP by using regressions and skips in human reading eye-tracking data as signals to inform revision policies. The results suggest that this technique can create a lasting impact in academic research, as it can potentially serve as a useful predictor for revisions in various languages.