The paper “Offspring from Reproduction Problems: What Replication Failure Teaches Us” which was co-authored by NewsReader team members Antske Fokkens, Marieke van Erp and Piek Vossen was runner up for the best paper award at ACL 2013 last week.
In the paper, difficulties in reproducibility of prior research tools and experiments are addressed. Many researchers have come across cases where they had wanted to reuse an approach described in a research paper but found out that it was very difficult to reproduce the results or even didn’t succeed. In the paper, two use cases are presented, and it is argued that improving reproducibility is not only handy to promote reuse, but also to enable systematic testing of approaches to gain better insights into their workings.
In NewsReader, we aim to make our research results as reproducible as we can. All software developed in the project will be made available through our GitHub account and datasets will be published on this website. Please do not hesitate to contact us, if you have trouble using any of it.
The slides of the ACL presentation can be found on slideshare. There is also a recording of the presentation at ACL. The discussion board and website to promote reproducibility in NLP research can be found here. The paper is also mentioned at Lightside’s Top Ten Papers at ACL 2013.