(4/4) We provide case studies to show that the COAT can be easily applied to draw causal insight from different datasets, like clinical notes, MRI images, and the NetCDF-format NOAA Reanalysis dataset in climate science.

(3/4) We provide:.- theoretical guarantees.- metrics on LLMs' ability.- empirical results. about identifying a set of Markov Blankets for the given target variables from unstructured data.

(2/4).Specifically, given the target variable Y , COAT aims to identify a Markov blanket to Y from raw observation and also produce the theoretical-guaranteed causal structure.

(1/4) 🎉Thrilled to share our work at #NeurIPS2024.Discovery of the Hidden World with Large Language Models.ArXiv: 2402.03941.We propose the first framework, Causal representatiOn AssistanT (COAT), for grounded causal results over LLM-proposed factors from unstructured data.

🧵8/8 Summary:. 1️⃣ New Problem: The problem of Noisy Rationales in chain-of-thought prompting. 2️⃣ New Dataset: The NoRa dataset to evaluate LLM robustness under noisy rationales. 3️⃣ New Method: CD-CoT, which enables LLMs to rectify noisy rationales using a single clean rationale.

🧵7/8 Evaluating the new method:. CD-CoT significantly improves performance across all tasks, achieving an average accuracy improvement of 17.8% compared to the base model. CD-CoT demonstrates strong resistance to high noise levels, particularly in mathematical reasoning.

🧵6/8 The new method:. We propose CD-CoT, a simple yet effective method that introduces external supervision signals. These signals are both practical and sufficient for denoising. CD-CoT uses a single clean rationale example to help rectify noisy rationales.

🧵5/8 The new dataset:. We constructed the NoRa dataset and conducted extensive experiments. Results show that the prevailing LLMs are highly vulnerable to noisy rationales. For example, the accuracy of GPT-3.5-Turbo can drop by up to 40.4% under noisy rationales.

🧵4/8 The new research problem:. While most existing studies focus on Noisy Questions, the impact of Noisy Rationales on LLM reasoning remains underexplored. This work defines Noisy Rationales as rationales that include irrelevant or inaccurate reasoning steps.

🧵3/8 Background:. Existing research highlights a significant limitation of LLMs in handling noisy inputs:.when questions contain irrelevant information or are slightly modified, models can be easily distracted, deviating from the correct reasoning process.

🧵2/8 Paper information:. Paper: Code: .Slides: . This work underscores the critical need for robust reasoning in LLMs, introducing Noisy Rationales as a novel challenge.

🧵1/8 Thrilled to share our research work: .Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales?  #NeurIPS2024