ARDUOUS 2024 : 8th International Workshop on Annotation of useR Data for UbiquitOUs Systems


Link: https://arduous.eu/


When September 24, 2024
Where INFORMATIK FESTIVAL, Wiesbaden, Germany.
Submission Deadline May 26, 2024
Notification Due June 24, 2024
Final Version Due June 30, 2024



Labelling user data is central to designing and evaluating modern machine-learning-based systems. Although the use of annotated user data in the lifecycle of machine learning and artificial intelligence systems is well understood and workflows have been established over the years, the current regulatory environment increasingly considers the potential risks associated with automated decision-making and data analysis, particularly with regard to protecting the rights of the individuals whose data is being processed by these systems. The current examination of risks (for instance, in the context of the development of the new EU AI Act) includes but is not limited to, the characterization of model bias, robustness, sustainability concerns, and identification of security and privacy issues such as malicious training data, data poisoning, or source data leakage. As the comprehensive monitoring of AI systems throughout their entire lifecycle gains importance among practitioners and system deployers, it is crucial to translate methodologies derived from academic research into practical policies and practices that can be readily implemented. Furthermore, the transparency of AI systems greatly benefits from the accessibility of ground truth data, enabling a concrete understanding and characterization of the performance of automated systems in real-world scenarios.



To address the above-mentioned problems, we invite academia and industry researchers and practitioners to participate in this year’s 8th ARDUOUS (Annotation of useR Data for UbiquitOUs Systems) workshop. We encourage submissions from people working in diverse areas of computer science, machine learning, artificial intelligence, data science and engineering, as well as those working in areas in which these methods are widely applied, such as digital health, logistics and the digital humanities, exploring the role of user and ground truth data, from methods of data collection and quality assurance through to the use of this data for exploring and validating machine learning models in the real world. Our expected outcomes are to develop a roadmap via interactive discussion within the workshop, which will be published and subsequently integrated into upcoming and existing research data management and analytics initiatives within and beyond Germany.



We invite you to submit full or short papers that offer new empirical or theoretical insights on the challenges and innovative solutions associated with the labelling of user data and the impact that labelling choices have on the user and the developed system. Position papers, as are papers that draw on experience or present empirical outcomes, or industrial papers are welcome.


This year topics of interest include


- experiences of the development, validation and sharing of data annotation protocols, both in academia and in industry
- ensuring compliance with GDPR (DSGVO) and AI EU Act when annotating data
- human-centred and human-in-the-loop approaches to designing and deploying AI systems
- characterising forms of annotation appropriate for specific sectors - for example, digital health, logistics, text mining for the digital humanities
- Low-resource annotation workflows
- Annotation metrics: inter-indexer consistency, similarity, bias and subjectivity



Further general annotation-related topics include but are not limited to



- methods and intelligent tools for annotating data
- methods for standardisation and normalisation in annotation practices
- influence of interface on annotation
- processes of and best practices in annotating data
- methods towards automation of the annotation process
- improving and evaluating the quality of annotations
- beyond the labels: ontologies for semantic annotation of user data
- high-quality and re-usable annotation for publicly available datasets
- impact of annotation on a system's performance
- building classifier models that are capable of dealing with multiple (noisy) annotations and/or making use of taxonomies/ontologies
- the potential value of incorporating modelling of the annotators into predictive models
- evaluating the efficacy of transfer learning via existing annotated datasets
- handling semantic and temporal shift and drift in the applications of annotated datasets



Submission guidelines:


Format:
For your submission you should use one of the following dedicated templates:

- (.doc format) https://gi.de/fileadmin/GI/Hauptseite/Service/Publikationen/LNI/LNI-authorsinstructions-englisch.doc
- (LaTex) https://gi.de/service/publikationen/lni


All accepted papers will be published in the GI-Edition Lecture Notes in Informatics (LNI) under a Creative Commons BY-SA 4.0 license which allows re-publication of any part of the paper.

The paper length should be proportional to their contribution. The absolute minimum page limit is 3 pages and the maximum limit is 12 pages. Generally the paper length should not exceed the following guideline page limits including references:


Full paper: 12 pages
Short paper: 8 pages
Position paper: 3 to 5 pages
Industry papers: 3 to 5 pages

Submission: through the EasyChair submission system https://easychair.org/conferences/?conf=arduous2024 .

Review process: the review process will be single blind

Registration: Each accepted workshop paper requires a registration. It is mandatory that at least one author registers and presents the paper during the workshop (virtually or in-person).
The registration site can be found at https://informatik2024.gi.de/registration.html .



Review process: the review process will be single blind

Registration: Each accepted workshop paper requires a registration. It is mandatory that at least one author registers and presents the paper during the workshop (virtually or in-person).
The registration site can be found at https://informatik2024.gi.de/registration.html .



Important dates:
Submission deadline: May 26, 2024

Notification: June 24, 2024

Camera ready version: June 30, 2024

Workshop: September 24, 2024

The 8th International Workshop on Annotation of useR Data for UbiquitOUs Systems is held as part of the INFORMATIK FESTIVAL 2024 in Wiesbaden, Germany and virtually.

General Workshop Chairs:

Kristina Yordanova, University of Greifswald, DE
Emma Tonkin, University of Bristol, UK
Gregory Tourte, University of Oxford, UK

Organising Committee:

Teodor Stoev, University of Greifswald, DE
Dipendra Yadav, University of Greifswald, DE
Fernando Moya Rueda, Motion Miners GmBh, Technical University of Dortmund, DE
Nilah Nair, Technical University of Dortmund, DE