NVIDIA Introduces ChatQA, GPT-4 Level Conversational QA Models
NVIDIA researchers recently introduced ChatQA, a family of conversational question answering (QA) models, aiming to achieve GPT-4 level accuracies. NVIDIA’s ChatQA introduces a range of models, ranging from 7B to 70B in size. Extensive evaluations across 10 conversational QA datasets reveal that the top-performing ChatQA-70B model not only outperforms GPT-3.5-turbo but also performs at par with GPT-4. What’s notable is that these achievements are attained without relying on any synthetic data from ChatGPT models. The team behind ChatQA proposes a two-stage instruction tuning method, significantly enhancing zero-shot conversational QA results from large language models (LLMs). To address retrieval in conversational QA, a dense retriever is fine-tuned on a multi-turn QA dataset, delivering comparable results to state-of-the-art query rewriting models with reduced deployment costs. NVIDIA showcases the effectiveness of fine-tuning a single-turn query retriever using their curated conversational QA data. This approach performs comparably to the state-of-the-art LLM-based query rewriting model, all achieved without the need for extra computational time and potential API costs associated with rewriting. NVIDIA’s ChatQA has demonstrated a remarkable advancement in handling scenarios where answers are elusive. The incorporation of a small number of “unanswerable” samples has proven to significantly enhance the model’s capabilities. Through an evaluation of unanswerable cases, it is evident that the leading model, ChatQA-70B, exhibits only a slight performance gap when compared to the formidable GPT-4. For more details, refer to the paper: ChatQA Paper. NVIDIA is not alone. Several foundational models have achieved the capabilities of GPT-4. Google is likely to launch Gemini Ultra at any moment. Meanwhile, Mistral CEO Arthur Mensch announced on French national radio that the company will unveil an open-source GPT-4-level model in 2024.
ChatQA-70B stands out by surpassing GPT-4 in average scores on 10 conversational QA datasets (54.14 vs. 53.90).
NVIDIA researchers recently introduced ChatQA, a family of conversational question answering (QA) models, aiming to achieve GPT-4 level accuracies.
[]
["Go", "ChatGPT", "API", "synthetic data", "AI", "GPT", "Aim", "llm_models:Gemini", "R", "llm_models:GPT"]
["AI", "ChatGPT", "Aim", "R", "Go", "API", "GPT", "synthetic data", "llm_models:GPT", "llm_models:Gemini"]
Siddharth is a media graduate who loves to explore tech through journalism and putting forward ideas worth pondering about in the era of artificial intelligence.