Web Engineering Seminar (SS 2025)

Questions:

• Chatbot Interaction Patterns are used in the graphical interface of chatbots and provide the users with known interaction elements which can make the interaction easier. Explain the term Chatbot Interaction Patterns and show some examples.
• Provide an overview of the State of the Art about graphical interface elements in chatbots, where they were already part in the literature and in which different ways they are described in the the literature.
• You will receive access to a dataset of chatbots and chatbot interactions, as well as an empty codebook with chatbot interaction patterns. Make yourself familiar with these pattern and understand how they are used.
• Each student must analyse the complete dataset accordingly to the provided labels (patterns and metadata). The dataset mostly includes textual descriptions, screenshots, and video recordings of chatbot interactions. Detailed instructions will be provided in a meeting with the advisor.
• Additionally to the provided datasets, search for 15 new chatbots in the domain of e-commerce (customer service, retail, etc.). Each student should conduct a conversation with all newly found chatbots, analyse them accordingly to the codebook and record their interaction. For any uncertainties in this process, there will be regular meetings with the advisors.
• Present your analysis results of the pattern distribution with diagrams, tables, and visual examples.

Literature:

• Own research
• Unpublished research (will be provided)
• Traubinger, V., Gaedke, M. (2024). Interaction Design Patterns of Web Chatbots. In: Stefanidis, K., Systä, K., Matera, M., Heil, S., Kondylakis, H., Quintarelli, E. (eds) Web Engineering. ICWE 2024. Lecture Notes in Computer Science, vol 14629. Springer, Cham. https://doi.org/10.1007/978-3-031-62362-2_22
• Tidwell, J., Brewer, C., Valencia, A.: Designing Interfaces: Patterns for Effective Interaction Design. O’Reilly, Beijing [China]; North Sebastopol, CA, third edition edn. (2020)
• Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N. Patel. 2018. Evaluating and Informing the Design of Chatbots. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS ’18). Association for Computing Machinery, New York, NY, USA, 895–906. https://doi.org/10.1145/3196709.3196735
• Lorenz Cuno Klopfenstein, Saverio Delpriori, Silvia Malatini, and Alessandro Bogliolo. 2017. The Rise of Bots: A Survey of Conversational Interfaces, Patterns, and Paradigms. In Proceedings of the 2017 Conference on Designing Interactive Systems (Edinburgh, United Kingdom) (DIS ’17). Association for Computing Machinery, New York, NY, USA, 555–565. https://doi.org/10.1145/3064663.3064672
• Ulrich Gnewuch, Stefan Morana, Marc Adam, and Alexander Maedche. 2018. “The Chatbot is typing ...” – The Role of Typing Indicators in Human-Chatbot Interaction. SIGHCI 2018 Proceedings (dec 2018). https://aisel.aisnet.org/sighci2018/14

Questions:

• Chatbot Interaction Patterns are used in the graphical interface of chatbots and provide the users with known interaction elements which can make the interaction easier. Explain the term Chatbot Interaction Patterns and show some examples.
• You will receive access to a dataset of chatbots and chatbot interactions, as well as an empty codebook with chatbot interaction patterns. Make yourself familiar with these pattern and understand how they are used.
• Each student must analyse the complete dataset accordingly to the provided labels (patterns and metadata). The dataset mostly includes textual descriptions, screenshots, and video recordings of chatbot interactions. Detailed instructions will be provided in a meeting with the advisor.
• Additionally to the provided datasets, search for 15 new chatbots in the domain of e-commerce (customer service, retail, etc.). Each student should conduct a conversation with all newly found chatbots, analyse them accordingly to the codebook and record their interaction. For any uncertainties in this process, there will be regular meetings with the advisors.
• Present a statistical analysis of your results like frequency and distribution of patterns, dependencies of patterns (i.e. which ones depend on others, etc) and more. Choose sensible statistical methods for this and also use them not only on the complete datasets but also on each dataset on its own. Use diagrams, tables, and visual examples in your presentation.

Literature:

• Own research
• Unpublished research (will be provided)
• Traubinger, V., Gaedke, M. (2024). Interaction Design Patterns of Web Chatbots. In: Stefanidis, K., Systä, K., Matera, M., Heil, S., Kondylakis, H., Quintarelli, E. (eds) Web Engineering. ICWE 2024. Lecture Notes in Computer Science, vol 14629. Springer, Cham. https://doi.org/10.1007/978-3-031-62362-2_22
• Lorenz Cuno Klopfenstein, Saverio Delpriori, Silvia Malatini, and Alessandro Bogliolo. 2017. The Rise of Bots: A Survey of Conversational Interfaces, Patterns, and Paradigms. In Proceedings of the 2017 Conference on Designing Interactive Systems (Edinburgh, United Kingdom) (DIS ’17). Association for Computing Machinery, New York, NY, USA, 555–565. https://doi.org/10.1145/3064663.3064672

Questions:

• What are chatbots, which technical properties do they have and how can they be developed?
• Search for a meaningful list of websites which can be used for data scraping. Take a random sample of at least a 100 websites and visually check them for chatbots. Look at their source code and the implementation of the chatbot. Check if the chatbot was created by using a chatbot development platform. Note down your findings.
• Search and use at least two tools for web crawling and use them on this list of urls to find websites with chatbots - a third one will be given to you. Do this first on your manually checked list and compare the results. Then see if you have to refine the use of the tools and test them on the larger sample.
• Evaluate the results of the scraping processes and compare them. Present your findings, for example: Which tool is the most accurate? Could you improve it and how? How did the precisions change from the first iterations to the last ones?
• In a next step, use a different tool to find out which chatbot development platforms were used in the creation of the chatbot. You might have to check the code again from a random sample of found chatbots for this and refine that tool. Present your results: Which platforms did you find and how often were they used?

Literature:

• Own research
• scrapers
• Dagkoulis, I., & Moussiades, L. (2022, November). A Comparative Evaluation of Chatbot Development Platforms. In Proceedings of the 26th Pan-Hellenic Conference on Informatics (pp. 322-328).
• S. Pérez-Soler, S. Juárez-Puerta, E. Guerra and J. de Lara, "Choosing a Chatbot Development Tool," in IEEE Software, vol. 38, no. 4, pp. 94-103, July-Aug. 2021, doi: 10.1109/MS.2020.3030198.
• Adamopoulou, Eleni, and Lefteris Moussiades. "An overview of chatbot technology." IFIP international conference on artificial intelligence applications and innovations. Springer, Cham, 2020.

Questions:

• What are data infrastructures, and how do they organize data in distributed systems? Present at least two to three types and categorize them.
• How are knowledge graphs applied in web engineering or distributed systems? Provide real-world examples.
• What are the strengths and weaknesses of using knowledge graphs in distributed systems?
• How do different implementations of knowledge graphs compare in terms of performance and scalability?

Literature:

• Own research
• Obraczka, D., et al. "Construction of Knowledge Graphs: Current State and Challenges." Information, 2024, doi: 10.3390/info15080509.
• Heiko Paulheim, et al. "A Knowledge Graph Perspective on Knowledge Engineering." Machine Learning and Knowledge Extraction, 2022, doi: 10.1007/s42979-022-01429-x.
• Bradley P. Allen, et al. "Evaluating Class Membership Relations in Knowledge Graphs Using Large Language Models." arXiv, 2024.

Questions:

• Conduct a systematic literature review on how knowledge graphs are used for data integration in web engineering.
• How do knowledge graphs compare to traditional data management methods regarding integration, performance, and scalability?
• What are the latest trends in knowledge graph research, including AI integration?
• What challenges do knowledge graphs face in real-time data integration, such as handling data heterogeneity and scalability?
• How can knowledge graphs be optimized for better performance in web engineering applications?

Literature:

• Own research
• Färber, M. "Applications of Knowledge Graphs in Industry." IEEE Access, 2019, doi: 10.1109/ACCESS.2019.2933496.
• Paul Groth, et al. "Knowledge Graphs and their Role in the Knowledge Engineering of the 21st Century." Dagstuhl Reports, 2022.
• Ji, S.; Pan, S.; Cambria, E.; Marttinen, P.; Philip, S.Y. "A survey on knowledge graphs: Representation, acquisition, and applications." IEEE Transactions on Neural Networks and Learning Systems, 2021, doi: 10.1109/TNNLS.2021.3070843.

Questions:

• What are knowledge graphs? Describe what they are and why they are important
• How do we assess the quality of knowledge graphs? Discuss various metrics used for evaluation.
• What are the methods for validating knowledge graphs? Explain different approaches to ensure their accuracy and reliability.
• What challenges exist in knowledge graph evaluation and validation? Identify issues faced in maintaining high-quality knowledge graphs.

Literature:

• Own research
• Huaman, J. and Kärle, E., “Knowledge Graph Validation,” 2020 IEEE International Conference on Data Mining Workshops, DOI: 10.1109/ICDMW51313.2020.00114. Available: Semantic Scholar
• Schuitema, S. and Vreeken, J., “Knowledge Graph Completeness: A Systematic Literature Review,” HAL Archives, 2021. Available: HAL
• Schlichtkrull, M., et al., “Modeling the Future of Knowledge Graphs: Towards a Framework for Knowledge Graph Evaluation,” Wageningen University, 2022. Available: Wageningen
• Zhang, C., et al., “A Survey on Knowledge Graphs: Representation, Acquisition, and Applications,” arXiv, 2020. Available: arXiv

Questions:

• What is trustworthy AI? What are the characteristics that need to exist in AI to be trustworthy? How to measure the trustworthiness of AI?
• Why is trustworthy AI recommended when it is used in the healthcare domain rather than standard AI?
• Demonstrate the differences between trustworthy and untrustworthy AI by showing an example of both AI’s in chatbots.

Literature:

• https://dl.acm.org/doi/full/10.1145/3555803
• https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7133476/
• https://link.springer.com/article/10.1007/s44204-023-00063-5
• https://www.sciencedirect.com/science/article/pii/S1532046420302835
• https://dl.acm.org/doi/full/10.1145/3546872
• https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10651407/

Questions:

• What is machine unlearning? And in which domains can it be applied?
• What algorithms are used in machine unlearning? Explain the differences between Machine learning and Machine unlearning.
• Find a tutorial related to machine unlearning implementation and demonstrate it.

Literature:

• https://arxiv.org/abs/1912.03817
• https://arxiv.org/abs/2108.11577
• https://arxiv.org/abs/2311.02240
• https://arxiv.org/abs/2103.03279
• https://ieeexplore.ieee.org/document/10097553

Questions:

• What is a Brain-Computer Interface (BCI) and how does it function?
• In which areas or applications can BCI technology be implemented?
• Design a demonstration using BCI technology for user interaction or cognitive monitoring. (The necessary hardware can be provided by the supervisor.

Literature:

• Own research

Questions:

• What guidelines and principles exist to safeguard Good Research Practice?
• How can these guidelines and principles be integrated into the research process?
• What is scientific misconduct/scientific malpractice?
• What is considered "high-quality research"? What are indicators thereof?

Literature:

• Own Research
• The European Code of Conduct for Research Integrity http://www.allea.org/wp-content/uploads/2017/03/ALLEA-European-Code-of-Conduct-for-Research-Integrity-2017-1.pdf
• Guidelines for Safeguarding Good Research Practice https://www.dfg.de/download/pdf/foerderung/rechtliche_rahmenbedingungen/gute_wissenschaftliche_praxis/kodex_gwp_en.pdf
• https://erc.europa.eu/sites/default/files/document/file/ERC_info_document-Open_Research_Data_and_Data_Management_Plans.pdf

Questions:

• What is design science research? What are the objectives of design science research?
• Which are activities? How research is conducted? How results are evaluated?
• In which research areas of computer science this methodology is more practical?
• Using design science research produce a viable and simplified artifact in the form of a construct, a model, a method and demonstrate activities

Literature:

• Own Research
• Johannesson Perjons (2021), An Introduction to Design Science, https://link.springer.com/book/10.1007/978-3-030-78132-3

Questions:

• Which browser-compatible inference engines support running transformer-based LLMs locally?
• How does WebAssembly compare to pure JavaScript in terms of inference speed and resource usage?
• What are the main memory constraints when loading and running large models in the browser, and how can quantization mitigate these?
• To what extent can WebGPU or WebGL be leveraged for hardware-accelerated LLM inference, and what are the current browser support limitations?
• How can we benchmark end-to-end latency and throughput of client-side inference across different devices (desktop vs. mobile)?

Literature:

• Own Research

Questions:

• What is the current state of Web Engineering research? To answer this question, systematically analyze all publications of the 2 venues listed under literature as detailed below. Your primary information used should be the title, authors/affiliations, keywords, and abstract.
• For each publication, capture title, authors/affiliations, keywords, and abstract, venue, year, (for conference papers) name of track/workshop, (for journal articles) volume number, (for journal articles) issue number, (for journal articles) name of issue, page numbers in proceedings/issue, length of the publication, current number of citations of the publication, URL of online resource.
• Based on your raw data collection, analyze the following aspects: 1. What are the main topics of research interest and in which areas of the Web Engineering field, along with the number of publications belonging to them? 2. What authors are publishing in these venues, from which affiliations, from which countries, along with the number of publications for each of these? 3. Which are the most cited articles (relative to their age), which topics/areas receive the most citations, which authors/affiliations/countries receive the most citations? 4. Considering the time dimension, are there any visible trends for aspects 1-3 over the 5 years considered?
• Visualize your data and insights and provide the raw data in re-usable form (CSV).

Literature:

• Venue 1: ICWE Proceedings of last 5 complete years (2019-2023)
• Venue 2: JWE Journals Issues of last 5 complete years (2019-2023)
• For citation counts use: Google Scholar
• Tool for analysis and inspiration for your data visualization: https://www.connectedpapers.com/

Questions:

• Which technologies, frameworks, patterns, paradigms, best-pratices and guidelines are currently used by web applications?
• To answer the question, conduct a survey gathering publicly available information from web repositories and identification through semi-automatic analysis.
• Start by identifying relevant UI Testing patterns, paradigms, best-pratices and guidelines in published literature (and potentially gray literature).
• Then, write supporting scripts and crawl public repositories (GitHub, Gitlab, Bitbucket, etc.) to identify applications using UI Tests.
• Per each application, identify the aspects from question one and keep track of the source of information (e.g. the URL of the corresponding source code), the type of application (domain), the application technology (e.g. programming language, web frameworks).
• Summarize your findings across all surveyed applications using statistical methods and visualizations and show if there are differences in the UI Testing aspects across the different application technologies.

Literature:

• Orient your methodology on the following paper: http://www.pautasso.org/biblio-pdf/apiace-icwe2023-versioning.pdf
• You own research is required to identify relevant UI Testing technologies, frameworks, patterns, paradigms, best-pratices and guidelines

Questions:

• What are quantitative methods? Which experimental designs are used with them? What are post-task assessments? What evaluation goals are typically addressed using them? What is the role of experimentation and quantitative methods in Software Engineering, Web Engineering, User Studies and Research?
• Provide a systematic overview on common scales for quantitative experiments with human users. What are latent variables? What is a construct? What does construct validity mean? What does construct reliability mean? What are semantic differentials? What are items? What are Likert scales?
• Create guidelines for computer science students how to conduct quantitative experimentation with human participants. Why is it important to use validated scales instead of inventing your own? Why does the visual representation of questionnaires play a role? How to process/further analyze empirical quantitative data e.g. to test hypotheses, find correlations etc.

Literature:

• https://katalog.bibliothek.tu-chemnitz.de/Search/Results?lookfor=Quantitative+method&s=
• https://germanupa.de/wissen/fragebogenmatrix
• Maximilian Speicher: Seven heuristics for identifying proper UX instruments and metrics. ACM Interactions Blog 2023
• C. Wohin et al.: Experimentation in Software Engineering. Springer
• J. Cresswell et al: Research design: qualitative, quantitative, and mixed methods approaches. Sage 2023
• Barnum et al: Usability testing and research. Longmont 2005.
• SUS, UEQ(-S), NASA TLX, SEQ, UMUX(-LITE),
• https://katalog.bibliothek.tu-chemnitz.de/Search/Results?lookfor=Quantitative+method&s=
• https://germanupa.de/wissen/fragebogenmatrix
• Own Research

Questions:

• How does the property graph data model define nodes, edges, properties, and labels?
• Compare Property Graphs to Knowledge Graphs.
• How can Knowledge Graphs be translated into Property Graphs?

Literature:

• Own research

Questions:

• What are the objectives of SHACL and how are they achieved?
• In which real-world scenarios and use cases is SHACL employed?
• Compare the Shapes Constraint Language to the Shape Expressions Language.

Literature:

• https://www.w3.org/TR/shacl/
• Own research

Questions:

• What are Virtual Knowledge Graphs and how can they be implemented?
• In which real-world scenarios and use cases are Virtual Knowledge Graphs employed?
• What are the trade-offs in query performance and architecture when using virtual knowledge graphs compared to conventional Knowledge Graphs?

Literature:

• Own research