Masterarbeit
Designing an Experiment Research Environment for Chatbots - Comparing Chatbot Interfaces
Completion
2023/12
Research Area
Students
Anjali Gupta
Advisers
Description
Chatbots should engage users in conversations, offer assistance, information, or entertainment. More and more companies provide chatbots as one of their main contact to users, which means in turn that also more people are interacting with chatbots. The research on chatbots is manifold, from LLM training, dialogue creation, to user interfaces and dark patterns. One problem in the research on chatbots is that the interactions between human and chatbot are hard to observe "in the wild", as their dynamic nature makes it hard to detect any changes in their implementation and interactions are hard to direct. When research on chatbots is conducted in several fields, it is also not feasible to implement one chatbot per experiment, as this creates a high effort. A solution for this would be to create an experiment environment in which chatbots for specific use cases can be easily implemented and adapted.
As a contact for users, chatbots are often included as a pop-up in a corner of the web interface. Since the rise of ChatGPT, full-screen chatbots are also becoming more popular again. This raises scientific interest about influences on the effectiveness and user experience which may be caused by these different interfaces.
This thesis researches on how to build an experiment research environment to investigate full-screen and small-screen chatbot interfaces, evaluate their effectiveness, user experience, and impact on user engagement across domains. For this, an environment for scientific research is needed, which will offer researchers a basis to conduct experiments and automate some tasks in the environment itself. This environment should be adaptable to other experiment settings, include a database for different use cases, and include not only an immersive environment with a homepage and chatbot interface for the participants, but also a control interface for the experimenter, which can be used for preparation and post-processing.
The objective of this thesis is the creation of a solution or the combination of existing approaches to solve the above described problem of an experiment research environment to compare chatbot interfaces. This includes the analysis of the state of the art of experiment environments, chatbot interfaces and technologies, the research on human-computer-interaction concerning these topics, and other relevant literature. From this analysis, a possible solution should be conceptualized and demonstrated by creating a prototypical implementation of an experiment research environment, as well as a suitable evaluation based on the user experience of this prototype and its compliance with requirements which were extracted through the literature research.
