Publications

All Publications

  1. aug 2024
    “My Answer is C”: First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models

    Wang, Xinpeng and Ma, Bolei and Hu, Chengzhi and Weber-Genzel, Leon and Röttger, Paul and Kreuter, Frauke and Hovy, Dirk and Plank, Barbara

    Findings of the Association for Computational Linguistics ACL 2024


  2. aug 2024
    Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoning

    Mondorf, Philipp and Plank, Barbara

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)


  3. aug 2024
    VariErr NLI: Separating Annotation Error from Human Label Variation

    Weber-Genzel, Leon and Peng, Siyao and De Marneffe, Marie-Catherine and Plank, Barbara

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)


  4. aug 2024
    CLIMATELI: Evaluating Entity Linking on Climate Change Data

    Zhou, Shijia and Peng, Siyao and Plank, Barbara

    Proceedings of the 1st Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2024)


  5. jun 2024
    MaiNLP at SemEval-2024 Task 1: Analyzing Source Language Selection in Cross-Lingual Textual Relatedness

    Zhou, Shijia and Shan, Huangyan and Plank, Barbara and Litschko, Robert

    Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)


  6. may 2024
    MaiBaam: A Multi-Dialectal Bavarian Universal Dependency Treebank

    Blaschke, Verena and Kovačić, Barbara and Peng, Siyao and Schütze, Hinrich and Plank, Barbara

    Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)


  7. may 2024
    Sebastian, Basti, Wastl?! Recognizing Named Entities in Bavarian Dialectal Data

    Peng, Siyao and Sun, Zihang and Shan, Huangyan and Kolm, Marie and Blaschke, Verena and Artemova, Ekaterina and Plank, Barbara

    Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)


  8. may 2024
    Slot and Intent Detection Resources for Bavarian and Lithuanian: Assessing Translations vs Natural Queries to Digital Assistants

    Winkler, Miriam and Juozapaityte, Virginija and van der Goot, Rob and Plank, Barbara

    Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)


  9. apr 2024
    Look at the Text: Instruction-Tuned Language Models are More Robust Multiple Choice Selectors than You Think

    Wang, Xinpeng and Hu, Chengzhi and Ma, Bolei and Röttger, Paul and Plank, Barbara

    arXiv


  10. mar 2024
    EEVEE: An Easy Annotation Tool for Natural Language Processing

    Sorensen, Axel and Peng, Siyao and Plank, Barbara and Van Der Goot, Rob

    Proceedings of The 18th Linguistic Annotation Workshop (LAW-XVIII)


  11. mar 2024
    Donkii: Characterizing and Detecting Errors in Instruction-Tuning Datasets

    Weber, Leon and Litschko, Robert and Artemova, Ekaterina and Plank, Barbara

    Proceedings of The 18th Linguistic Annotation Workshop (LAW-XVIII)


  12. mar 2024
    Proceedings of the 1st Workshop on Uncertainty-Aware NLP (UncertaiNLP 2024)

    Vázquez, Raúl and Celikkanat, Hande and Ulmer, Dennis and Tiedemann, Jörg and Swayamdipta, Swabha and Aziz, Wilker and Plank, Barbara and Baan, Joris and de Marneffe, Marie-Catherine


  13. mar 2024
    Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?

    Baan, Joris and Fernández, Raquel and Plank, Barbara and Aziz, Wilker

    Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)


  14. mar 2024
    NNOSE: Nearest Neighbor Occupational Skill Extraction

    Zhang, Mike and Goot, Rob and Kan, Min-Yen and Plank, Barbara

    Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)


  15. mar 2024
    Exploring the Robustness of Task-oriented Dialogue Systems for Colloquial German Varieties

    Artemova, Ekaterina and Blaschke, Verena and Plank, Barbara

    Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)


  16. mar 2024
    Deep Learning-based Computational Job Market Analysis: A Survey on Skill Extraction and Classification from Job Postings

    Senger, Elena and Zhang, Mike and Goot, Rob and Plank, Barbara

    Proceedings of the First Workshop on Natural Language Processing for Human Resources (NLP4HR 2024)


  17. mar 2024
    Entity Linking in the Job Market Domain

    Zhang, Mike and Goot, Rob and Plank, Barbara

    Findings of the Association for Computational Linguistics: EACL 2024


  18. mar 2024
    Different Tastes of Entities: Investigating Human Label Variation in Named Entity Annotations

    Peng, Siyao and Sun, Zihang and Loftus, Sebastian and Plank, Barbara

    Proceedings of the Third Workshop on Understanding Implicit and Underspecified Language


  19. mar 2024
    More Labels or Cases? Assessing Label Variation in Natural Language Inference

    Gruber, Cornelia and Hechinger, Katharina and Assenmacher, Matthias and Kauermann, Göran and Plank, Barbara

    Proceedings of the Third Workshop on Understanding Implicit and Underspecified Language


  20. mar 2024
    Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark

    Mayhew, Stephen and Blevins, Terra and Liu, Shuheng and Šuppa, Marek and Gonen, Hila and Imperial, Joseph Marvin and Karlsson, Börje F. and Lin, Peiqin and Ljubešić, Nikola and Miranda, LJ and Plank, Barbara and Riabi, Arij and Pinter, Yuval

    arXiv


  21. mar 2024
    MaiBaam Annotation Guidelines

    Blaschke, Verena and Kovačić, Barbara and Peng, Siyao and Plank, Barbara

    arXiv


  22. feb 2024
    Through the Lens of Split Vote: Exploring Disagreement, Difficulty and Calibration in Legal Case Outcome Classification

    Xu, Shanshan and Santosh, T. Y. S. S and Ichim, Oana and Plank, Barbara and Grabmair, Matthias

    arXiv


  23. feb 2024
    What Do Dialect Speakers Want? A Survey of Attitudes Towards Language Technology for German Dialects

    Blaschke, Verena and Purschke, Christoph and Schütze, Hinrich and Plank, Barbara

    arXiv


  24. feb 2024
    The Science of Data Collection: Insights from Surveys can Improve Machine Learning Models

    Eckman, Stephanie and Plank, Barbara and Kreuter, Frauke

    arXiv


  25. dec 2023
    Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training

    Müller-Eberstein, Max and van der Goot, Rob and Plank, Barbara and Titov, Ivan

    Findings of the Association for Computational Linguistics: EMNLP 2023


  26. dec 2023
    What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability

    Giulianelli, Mario and Baan, Joris and Aziz, Wilker and Fernández, Raquel and Plank, Barbara

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing


  27. dec 2023
    From Dissonance to Insights: Dissecting Disagreements in Rationale Construction for Case Outcome Classification

    Xu, Shanshan and T.y.s.s, Santosh and Ichim, Oana and Risini, Isabella and Plank, Barbara and Grabmair, Matthias

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing


  28. dec 2023
    ACTOR: Active Learning with Annotator-specific Classification Heads to Embrace Human Label Variation

    Wang, Xinpeng and Plank, Barbara

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing


  29. dec 2023
    Establishing Trustworthiness: Rethinking Tasks and Model Evaluation

    Litschko, Robert and Müller-Eberstein, Max and van der Goot, Rob and Weber-Genzel, Leon and Plank, Barbara

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing


  30. dec 2023
    Evaluating Emotion Arcs Across Languages: Bridging the Global Divide in Sentiment Analysis

    Teodorescu, Daniela and Mohammad, Saif

    Findings of the Association for Computational Linguistics: EMNLP 2023


  31. dec 2023
    Language and Mental Health: Measures of Emotion Dynamics from Text as Linguistic Biosocial Markers

    Teodorescu, Daniela and Cheng, Tiffany and Fyshe, Alona and Mohammad, Saif

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing


  32. oct 2023
    LoHoRavens: A Long-Horizon Language-Conditioned Benchmark for Robotic Tabletop Manipulation

    Zhang, Shengqiang and Wicke, Philipp and Şenel, Lütfi Kerem and Figueredo, Luis and Naceri, Abdeldjallil and Haddadin, Sami and Plank, Barbara and Schuetze, Hinrich

    arXiv


  33. jul 2023
    ActiveAED: A Human in the Loop Improves Annotation Error Detection

    Weber, Leon and Plank, Barbara

    Findings of the Association for Computational Linguistics: ACL 2023


  34. jul 2023
    Silver Syntax Pre-training for Cross-Domain Relation Extraction

    Bassignana, Elisa and Ginter, Filip and Pyysalo, Sampo and van der Goot, Rob and Plank, Barbara

    Findings of the Association for Computational Linguistics: ACL 2023


  35. jul 2023
    Boosting Zero-shot Cross-lingual Retrieval by Training on Artificially Code-Switched Data

    Litschko, Robert and Artemova, Ekaterina and Plank, Barbara

    Findings of the Association for Computational Linguistics: ACL 2023


  36. jul 2023
    SemEval-2023 Task 11: Learning with Disagreements (LeWiDi)

    Leonardelli, Elisa and Abercrombie, Gavin and Almanea, Dina and Basile, Valerio and Fornaciari, Tommaso and Plank, Barbara and Rieser, Verena and Uma, Alexandra and Poesio, Massimo

    Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)


  37. jul 2023
    How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives

    Wang, Xinpeng and Weissweiler, Leonie and Schütze, Hinrich and Plank, Barbara

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)


  38. jul 2023
    ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain

    Zhang, Mike and van der Goot, Rob and Plank, Barbara

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)


  39. jul 2023
    Uncertainty in Natural Language Generation: From Theory to Applications

    Baan, Joris and Daheim, Nico and Ilia, Evgenia and Ulmer, Dennis and Li, Haau-Sing and Fernández, Raquel and Plank, Barbara and Sennrich, Rico and Zerva, Chrysoula and Aziz, Wilker

    arXiv


  40. may 2023
    A Survey of Corpora for Germanic Low-Resource Languages and Dialects

    Blaschke, Verena and Schuetze, Hinrich and Plank, Barbara

    Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)


  41. may 2023
    Low-resource Bilingual Dialect Lexicon Induction with Large Language Models

    Artemova, Ekaterina and Plank, Barbara

    Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)


  42. may 2023
    Multi-CrossRE A Multi-Lingual Multi-Domain Dataset for Relation Extraction

    Bassignana, Elisa and Ginter, Filip and Pyysalo, Sampo and van der Goot, Rob and Plank, Barbara

    Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)


  43. may 2023
    Findings of the VarDial Evaluation Campaign 2023

    Aepli, Noëmi and Çöltekin, Çağrı and Van Der Goot, Rob and Jauhiainen, Tommi and Kazzaz, Mourhaf and Ljubešić, Nikola and North, Kai and Plank, Barbara and Scherrer, Yves and Zampieri, Marcos

    Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2023)


  44. may 2023
    Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languages

    Blaschke, Verena and Schütze, Hinrich and Plank, Barbara

    Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2023)


  45. dec 2022
    CrossRE: A Cross-Domain Dataset for Relation Extraction

    Bassignana, Elisa and Plank, Barbara

    Findings of the Association for Computational Linguistics: EMNLP 2022


  46. dec 2022
    Experimental Standards for Deep Learning in Natural Language Processing Research

    Ulmer, Dennis and Bassignana, Elisa and Müller-Eberstein, Max and Varab, Daniel and Zhang, Mike and van der Goot, Rob and Hardmeier, Christian and Plank, Barbara

    Findings of the Association for Computational Linguistics: EMNLP 2022


  47. dec 2022
    On Language Spaces, Scales and Cross-Lingual Transfer of UD Parsers

    Samardžić, Tanja and Gutierrez-Vasques, Ximena and van der Goot, Rob and Müller-Eberstein, Max and Pelloni, Olga and Plank, Barbara

    Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL)


  48. dec 2022
    The “Problem” of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation

    Plank, Barbara

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing


  49. dec 2022
    Spectral Probing

    Müller-Eberstein, Max and van der Goot, Rob and Plank, Barbara

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing


  50. dec 2022
    Evidence > Intuition: Transferability Estimation for Encoder Selection

    Bassignana, Elisa and Müller-Eberstein, Max and Zhang, Mike and Plank, Barbara

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing


  51. dec 2022
    Stop Measuring Calibration When Humans Disagree

    Baan, Joris and Aziz, Wilker and Plank, Barbara and Fernandez, Raquel

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing


  52. oct 2022
    An Interdisciplinary Perspective on Evaluation and Experimental Design for Visual Text Analytics: Position Paper

    Kucher, Kostiantyn and Sultanum, Nicole and Daza, Angel and Simaki, Vasiliki and Skeppstedt, Maria and Plank, Barbara and Fekete, Jean-Daniel and Mahyar, Narges

    2022 IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV)


  53. sep 2022
    Skill Extraction from Job Postings using Weak Supervision

    Zhang, Mike and Jensen, Kristian Nørgaard and van der Goot, Rob and Plank, Barbara

    arXiv


  54. jul 2022
    SkillSpan: Hard and Soft Skill Extraction from English Job Postings

    Zhang, Mike and Jensen, Kristian and Sonniks, Sif and Plank, Barbara

    Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies


  55. jul 2022
    Sort by Structure: Language Model Ranking as Dependency Probing

    Müller-Eberstein, Max and van der Goot, Rob and Plank, Barbara

    Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies


  56. jul 2022
    Sliced at SemEval-2022 Task 11: Bigger, Better? Massively Multilingual LMs for Multilingual Complex NER on an Academic GPU Budget

    Plank, Barbara

    Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)


  57. jun 2022
    Fine-tuning vs From Scratch: Do Vision & Language Models Have Similar Capabilities on Out-of-Distribution Visual Question Answering?

    Jensen, Kristian Nørgaard and Plank, Barbara

    Proceedings of the Thirteenth Language Resources and Evaluation Conference


  58. jun 2022
    Frustratingly Easy Performance Improvements for Low-resource Setups: A Tale on BERT and Segment Embeddings

    van der Goot, Rob and Müller-Eberstein, Max and Plank, Barbara

    Proceedings of the Thirteenth Language Resources and Evaluation Conference


  59. jun 2022
    Kompetencer: Fine-grained Skill Classification in Danish Job Postings via Distant Supervision and Transfer Learning

    Zhang, Mike and Jensen, Kristian Nørgaard and Plank, Barbara

    Proceedings of the Thirteenth Language Resources and Evaluation Conference


  60. may 2022
    What Do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classification

    Bassignana, Elisa and Plank, Barbara

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop


  61. may 2022
    Probing for Labeled Dependency Trees

    Müller-Eberstein, Max and van der Goot, Rob and Plank, Barbara

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)