EQA Universal Data FormatterQuestion anwering data comes from a variety of sources and is formatted in just as many ways. The EQA Universal Data Formatter is a Python tool for processing QA data in its proprietary format and converting it into a common schema. The schema is defined here.
Download the EQA Universal Data Formatter
EQA Universal EvaluatorA standardized evaluation script aids in establishing comparability between system results. We release the EQA Universal Evaluator as a method for computing standard evaluation measures on files in the EQA Universal Data Format. It also produces aggregate scores over multiple data sets (if provided). We include such meta-scores to help measure the general ability of each system, apart from any data-specific assumptions or biases.
Download the EQA Universal Evaluator