talk: Validation of the Cybersecurity Curriculum Assessment

Joint work with Alan T. Sherman, Linda Oliva, Peter Peterson, Shan Huang, and Enis Golaszewski. This work earned the designation of one of the best papers in the Computing Education Research Track at the 2023 SIGCSE Technical Symposium.

Since 2015, the Cybersecurity Assessment Tools (CATS) project has been developing two research instruments to measure how well students learn cybersecurity concepts: the Cybersecurity Concept Inventory (CCI) and the Cybersecurity Curriculum Assessment (CCA). The CCI measures student conceptual knowledge after a first course in cybersecurity, whereas the CCA measures student knowledge after a full curriculum on cybersecurity. In this talk, we will discuss our development process of both assessment tools. We will discuss how we identified the critical subset of cybersecurity topics to include on the assessments and how we constructed the items on the assessment. We will then focus on our administration of the CCA to 193 students from seven colleges and universities across the United States. We present qualitative and statistical evidence for why the CCA can be validly used for measuring student knowledge of cybersecurity. In particular, we will show how items on the CCA identify common student misconceptions and how the CCA is the most informative assessment tool developed by computing education researchers.

Dr. Geoffrey L. Herman is the Severns Teaching Associate Professor in the Department of Computer Science at the University of Illinois at Urbana-Champaign. He was the 2020 recipient of the IEEE Education Society Mac Van Valkenburg Early Career Teaching Award and the 2022 recipient of the Scott H. Fisher Computer Science Teaching Award. He helped found the Grainger College of Engineering's Strategic Instructional Innovations Program, which has been empowering faculty innovation in teaching for over ten years and has provided seed funding that has led to several millions of dollars in external grant funding and hundreds of research papers. His research focuses on how students learn engineering and computing concepts and studying processes for creating systemic change in how engineering and computer science are taught in college settings. His research on student misconceptions in programming was awarded the best paper in the first 50 years of the ACM Special Interest Group, Computer Science Education. Email: glherman@illinois.edu.

Host: Alan T. Sherman, sherman@umbc.edu. Support for this event was provided in part by the National Science Foundation under SFS grant DGE-1753681. The UMBC Cyber Defense Lab meets biweekly Fridays 12-1pm.  All meetings are open to the public. Upcoming CDL meetings: February 17, Andrea Ferketich (UMBC), Security of cyber-physical systems in UMBC's ILSB; March 3, Enis Goleszewski (UMBC), Channel binding in FIDO should not be optional. May 5, CSEE Research Day (Library 7th floor)