“What are they not telling me?” Learning machine learning: Understanding the challenges for novices

Machine Learning (ML) is increasingly accessible to users with limited knowledge of its theoretical foundations. However, misapplying it can lead to negative consequences. This paper reports on a qualitative study designed to reveal challenges that novices encounter when learning about basic ML concepts and building their first models. Twenty participants were introduced to fundamental ML concepts for classification through an interactive tutorial involving an off-the-shelf GUI application, built their own ML model for a shape gesture dataset, and participated in a semi-structured interview. A thematic analysis revealed insights into these challenges, particularly around problem selection and multi-dimensionality, but also around what constitutes ML, algorithm selection, cross-validation, and interpreting visualizations. Despite these and other misconceptions, participants reflected on good model building practices, discussing that algorithm selection might require knowledge and context and that input features may introduce bias. We discuss the findings’ implications for the design of ML tools for novices.