Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER)

James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, Arpit Mittal, A. Wawer, Grzegorz Wojdyga, Justyna Sarzy ska-Wawer, Andreas Hanselowski, Hao Zhang, Zi-Le Li, Daniil Sorokin, Wei Fang, Moin Nadeem, M. Mohtarami, James R. Glass, Piotr Niewi, M. Pszona, M. Janicka, Alireza Mohammadshahi, R. Lebret, K. Aberer, R. Samuel, G. Bowman, Christopher Angeli, Christopher Potts, Manning, Rich Caruana, Kevin Clark, Minh-Thang Luong, Christopher D. Manning, Q. Le, R. Collobert, Jason Weston, A. Conneau, Douwe Kiela, H. Schwenk, L. Barrault, A. Bordes, Zi-Hang Dai, Zhi-Lin Yang, Yi-Ming Yang, Jaime G. Carbonell, R. Salakhutdinov, J. Devlin, M. Chang, Kenton Lee, Kristina Toutanova, Bill Dolan, Chris Brockett, Avinesh P.V.S, B. Schiller, Felix Caspelherr, Debanjan Chaudhuri, Christian M. Meyer, Iryna Gurevych, Kazuma Hashimoto, Cai-Ming Xiong, Y. Tsuruoka, R. Socher, Xiao-Dong Liu, Peng He, Wei-Zhu Chen, Jian Gao, Ye-Long Shen, Kevin Duh, Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan-Cheng Huang, Andrew Ng, Bryan McCann, James Bradbury, R. Baly, Preslav Nakov, Llus Mrquez, Alessandro Moschitti, Jialin Sinno, Qiang Pan, Yang, M. Peters, Mark Neumann, M. Iyyer, Matt Gardner, C. Clark, Luke S. Zettlemoyer, Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, P. Liang, A. Perelygin, Jean Y. Wu, J. Chuang, Anders Sgaard, Yoav Goldberg, A. Vaswani, Noam Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, Ukasz Kaiser, I. Polosukhin, Alex Wang, Amanpreet Singh, J. Michael, F. Hill, Omer Levy, Samuel Bowman, Adina Williams, Nikita Nangia, Brian Xu, Yu-Kun Zhu, R. Kiros, R. Zemel, R. Urtasun, A. Torralba, S. Fidler, V. Levenshtein, George Doddington, W. Liggett, Alvin Martin, Mark A. Przybocki, Douglas Reynolds, Takashi Miyazaki, N. Shimizu, A. Karpathy, Chris Manning, Peter Young, Alice Lai, M. Hodosh, J. Hockenmaier, Sanjeev Arora, Ying-Yu Liang, Teng-Yu Ma, Ankur Bapna, Orhan Firat, Daniel Cer, Yin-Fei Yang, Sheng-Yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St, Noah John, Mario Constant, Steve Guajardo-Cespedes, C. Yuan, Yun-hsuan Tar, B. Sung, Ray Strope, Kurzweil, G. Paul, T. Kwiatkowski, J. Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, D. Epstein, Matthew Kelcey, Andrew M. Dai, Slav Petrov, Gabor Angeli, M. Premkumar, Husam Ali, Yllias Chali, Sadid A. Hasan, Yanghoon Kim, Hwanhee Lee, Joongbo Shin, Kyomin Jung, M. Surdeanu, John Bauer, J. Finkel, Steven Bethard, D. Mcclosky, Yixin Nie, Hao-Nan Chen, Mohit Bansal, Takuma Yoneda, Jeff Mitchell, Johannes Welbl, Pontus Stenetorp, Sebastian Riedel, Filip Boltui, Joanna Najder, Sheng-Yi Kong, B. Tar, John C. Duchi, Elad Hazan, Y. Singer, Steffen Eger, Johannes Daxenberger, Martin Gleize, E. Shnarch, Leshem Choshen, Lena Dankin, Guy Moshkowich, R. Aharonov, N. Slonim, Ivan Habernal, Henning Wachsmuth, Benno Stein, S. Hochreiter, J. Schmidhuber, Xinyu Hua, Lu Wang, Yan-Kai Lin, S. Shen, Zhi-Yuan Liu, Huan-Bo Luan, Mao-Song Sun, S. Baccianella, Andrea Esuli, F. Sebastiani, A. Blei, Michael Ng, Jordan, Ge-Li Fei, Bing Liu, M. Hsu, M. Castellanos, Riddhiman Ghosh, Min-Qing Hu, Niklas Jakob, Wei-Qiang Jin, H. Hồ, R. Srihari, Kang Liu, Liam Heng, Yang Xu, Jun Liu, Zhao, Li-Heng Xu, Jun Zhao, T. Mikolov, I. Sutskever, Kai Chen, G. Corrado, J. Dean, S. Moghaddam, M. Ester, Aitor Garca Pablos, Montse Cuadros, German Rigau, Maria Pontiki, D. Galanis, Haris Papageorgiou, S. Manandhar, Ion Androutsopoulos, Ravikanth Repaka, Ranga Reddy Pallelra, Akshay Reddy Koppula, Venkata Subhash, Movva, Z. Toh, Jian Su, Wei Wang, Huan Xu, Xiao-Qiu Huang, Fan Zhang, Tuhin Chakrabarty, Tariq Alhindi, S. Muresan, Claudia Schulz, Christopher Hidey, Mona T. Diab, Jackson Luken, Nan-Jiang Jiang, Marie-Catherine de Marneffe, Wolfgang Otto, P. Ankur, Oscar Parikh, Dipanjan Tckstrm, Jakob Das, Uszkoreit, A. Trotman, Antti Puurula, Blake Burgess, Qiang Zhang, E. Yilmaz, Shang-Song Liang, Ehud Aharoni, Anatoly Polnarov, Tamar Lavee, Daniel Hershcovich, Ran Levy, Ruty Rinott, D. Gutfreund, Khalid Al-Khatib, Matthias Hagen, J. Khler, Martine Hanne, Aleksandrs Eckhoff, Berdicevskis, Karn Fort, Benot Sagot, Jungo Kasai, Sa-Sa Qian, Yunyao Gurajada, Lu-Lu Li, Popa, Gary King, Lang-Che Zeng, Jan-Christoph Klie, Michael Bugert, Beto Boullosa, Richard Eckart de Castilho, Ben Bogin, S. Gretz, Dong Lin, Frank Lee, Ou-Yu Xu, Xiang-Yong Lan, Ren, Jing Ma, Wei Gao, Shafiq R. Joty, Kam-Fai Wong, Tobias Mayer, Elena Cabrio, S. Villata, J. Pennington, Kashyap Popat, Subhabrata Mukherjee, Andrew Yates, G. Weikum, Avinesh P.V.S, Benjamin Httasch, Orkan Zyurt, Carsten Binnig, Nils Reimers, T. Beck, C. Perez, Mitesh M. Khapra, Tobias Kahse, Matthew Shardlow, R. Batista-Navarro, Paul Thompson, R. Nawaz, J. McNaught, Sophia Ananiadou, Carlos Alzate, Yu-Fang Hou, C. Stab, Tristan Miller, Pranav Rai, C. Stahlhut, Bill Adair, Cheng-Kai Li, Yang Jun, Cong Yu, C. F. Bond, Bella Depaulo, Leon Derczynski, Kalina Bontcheva, William Ferreira, J. George, Keane, Jennifer Hancock, Thompson Thom-Santelli, Ritchie, D. Benjamin, S. Horne, Adali, Sara Horne, Sibel Khedr, Pei-yun Hsueh, Prem Melville, Vikas Sindhwani, Tanushree Mitra, Eric Gilbert, Ndapandula Nakashole, Tom M. Mitchell, Jannik Strtgen, Martin Potthast, Johannes Kiesel, Kevin Reinartz, J. Bevendorff, Georg Rehm, J. Moreno-Schneider, Peter Bourgonje, M. Hany, Salaheldeen, Micah Nelson, Kai Shu, A. Sliva, Su Wang, Ji-Liang Tang, Huan Liu, M. Waldrop, William Yang, Wang, Xue-Zhi Wang, S. Baumgartner, F. Korn, X. Wang, C. Yu, S. Baumgartner, F. Korn, Christopher Malon, D. Rao, Paul McNamee, Mark Dredze, P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, Alessandro Bondielli, F. Marcelloni, Giovanni Luca Ciampaglia, Prashant Shiralkar, L. Rocha, J. Bollen, F. Menczer, A. Flammini, N. Conroy, V. Rubin, Y. Chen, S. Estévez-Velarde, Y. Gutiérrez, A. Montoyo, Alejandro Piad-Morffis, R. Munoz, Yudivián Almeida-Cruz, M. Honnibal, I. Montani, D. Korman, E. Mack, J. Jett, Allen H. Renear, S. Pad, Taeil Noh, A. Stern, R. Wang, R. Zanoli, T. Lukoianova, K. Shu, A. Sliva, S. Wang, J. Tang, H. Liu · 2019

• DOI
• Semantic Scholar

Abstract

We present the results of the second Fact Extraction and VERification (FEVER2.0) Shared Task. The task challenged participants to both build systems to verify factoid claims using evidence retrieved from Wikipedia and to generate adversarial attacks against other participant's systems. The shared task had three phases: building, breaking and fixing. There were 8 systems in the builder's round, three of which were new qualifying submissions for this shared task, and 5 adversaries generated instances designed to induce classification errors and one builder submitted a fixed system which had higher FEVER score and resilience than their first submission. All but one newly submitted systems attained FEVER scores higher than the best performing system from the first shared task and under adversarial evaluation, all systems exhibited losses in FEVER score. There was a great variety in adversarial attack types as well as the techniques used to generate the attacks, In this paper, we present the results of the shared task and a summary of the systems, highlighting commonalities and innovations among participating systems.

References

See the full reference list in the paper.