The resource must develop a structured mapping of the state of the art on Chatbot/LLM validation methodologies in contexts informed by specific documentation (manuals, procedures, knowledge base), with particular attention to the technical-industrial domain. The student will take care of the following activities:

· Collection and classification of approaches/metrics for evaluating chatbots (automatic, human, hybrid)

· Focus on key concepts: accuracy vs relevance, semantic similarity, robustness to ambiguous/out-of-scope queries, aspects of reliability/confidence

· Implement automatic comparison tools between chatbot responses and original content (ground truth) through semantic similarity and retrieval accuracy techniques.

· Perform a comparative analysis (pros/cons, data prerequisites, costs, repeatability)

Produce a final report containing a proposal for a 'validation framework' applicable to the business context.

• Three-year degree in progress (Computer Engineering/Mathematics/AI/Data Science or related)
• Interest in NLP/LLM and evaluation of AI systems
• Ability to read and summarize scientific articles (EN)
• Basic knowledge of Python, embeddings, NLP metrics
• Familiarity with RAG/knowledge base concepts