Recent developments in machine learning research have the potential to create a lasting impact in academic research and beyond. From training configurations for large language models to knowledge distillation methods and cognitively inspired components for social conversational agents, the possibilities are endless. In this newsletter, we will explore the potential of these recent developments and how they can be used to unlock the potential of machine learning research.
This paper presents a comprehensive ablation study of various training configurations for large language models, uncovering key recommendations for the most efficient training. The results of this study have the potential to create a lasting impact in academic research, enabling state-of-the-art training efficiency and utilization of up to 70.5% of model FLOPs. TencentLLMEval presents a comprehensive evaluation framework to assess the real-world capabilities of LLMs in following instructions on diverse tasks. It provides a standardized methodology to evaluate human alignment in LLMs, with a task tree, dataset, and evaluation processes. This framework has the potential to create a lasting impact in academic research by providing a reliable benchmark for the development of safe and human-aligned
This paper presents a comprehensive ablation study of various training configurations for large language models, uncovering key recommendations for the most efficient training. The results of this study have the potential to create a lasting impact in academic research, enabling state-of-the-art training efficiency and utilization of up to 70.5% of model FLOPs.
TencentLLMEval presents a comprehensive evaluation framework to assess the real-world capabilities of LLMs in following instructions on diverse tasks. It provides a standardized methodology to evaluate human alignment in LLMs, with a task tree, dataset, and evaluation processes. This framework has the potential to create a lasting impact in academic research by providing a reliable benchmark for the development of safe and human-aligned LLMs.
This paper presents a novel Knowledge Distillation method, IBKD, which leverages the Information Bottleneck principle to distill large pre-trained language models into smaller representations. This approach maximizes the mutual information between the teacher and student models, while reducing the risk of over-fitting, potentially creating a lasting impact in academic research of text representation techniques.
LLaVA-Plus is a general-purpose multimodal assistant that can use pre-trained vision and vision-language models to fulfill real-world tasks. It has been trained on multimodal instruction-following data to acquire the ability to use tools, and has been shown to outperform existing models. Its potential to create a lasting impact in academic research lies in its ability to directly ground image queries and actively engage throughout the entire human-AI interaction session.
Mirror is a universal framework for various information extraction tasks, which reorganizes IE problems into unified multi-slot tuples and uses a non-autoregressive graph decoding algorithm to extract all spans in a single step. It has the potential to create a lasting impact in academic research by providing a versatile and efficient solution to complex IE tasks, machine reading comprehension, and classification tasks.
This paper demonstrates that fine-tuning can be used to remove RLHF protections from GPT-4, a powerful LLM. With as few as 340 examples, attackers can successfully remove these protections with a 95% success rate. This has the potential to create a lasting impact in academic research, as it highlights the need for further research on LLM protections.
LCM-LoRA is a universal Stable-Diffusion acceleration module that can be used to accelerate text-to-image generative tasks with minimal inference steps and high-quality images. It is distilled from pre-trained latent diffusion models and can be plugged into various Stable-Diffusion fine-tuned models or LoRAs without training, providing a lasting impact in academic research of the described techniques.
This paper explores the potential of Generative Large Language Models (LLMs) to create and refine visualizations through conversational interfaces. Through a Wizard-of-Oz study and crowdsourced study, the authors found that LLMs can be used to support visual analysis, but need to be improved to support progressive refinements. The authors developed AI Threads, a multi-threaded analytic chatbot, to address this issue and demonstrated its potential with a dataset outside the LLM's training corpus. This research has the potential to create a lasting impact in academic research by providing a new way to visualize multidimensional datasets.
This paper presents a novel approach to mapping out the memorisation-generalisation continuum in Neural Machine Translation, which could have a lasting impact on academic research. It provides a resource to place 5M NMT datapoints on the continuum, and explores how surface-level characteristics and training signals are predictive of memorisation.
This paper presents a survey of cognitively inspired components for social conversational agents, which could potentially address both technical and social issues. These components, such as semantic and episodic memory, emotion, working memory, and the ability to learn, could create a lasting impact in academic research and lead to improved conversational quality and user experience.