ArXiv Preprint
We present Reddit Health Online Talk (RedHOT), a corpus of 22,000 richly
annotated social media posts from Reddit spanning 24 health conditions.
Annotations include demarcations of spans corresponding to medical claims,
personal experiences, and questions. We collect additional granular annotations
on identified claims. Specifically, we mark snippets that describe patient
Populations, Interventions, and Outcomes (PIO elements) within these. Using
this corpus, we introduce the task of retrieving trustworthy evidence relevant
to a given claim made on social media. We propose a new method to automatically
derive (noisy) supervision for this task which we use to train a dense
retrieval model; this outperforms baseline models. Manual evaluation of
retrieval results performed by medical doctors indicate that while our system
performance is promising, there is considerable room for improvement. Collected
annotations (and scripts to assemble the dataset), are available at
https://github.com/sominw/redhot.
Somin Wadhwa, Vivek Khetan, Silvio Amir, Byron Wallace
2022-10-12
