DISCUSSION AND CONCLUSIONS
A significant amount of work by the cybersecurity community has gone into providing the rationale for using gamified simulations and live activities such as hackathons and capture the flag competitions in cybersecurity education, but there have been few, if any, studies that have systematically compared these modes. Given that we conducted this at one educational institution, we classify this research as exploratory. However, we do provide strong evidence that modes of education and activities are significant in learning outcomes in cybersecurity, and that none of these modes are optimal in isolation. Furthermore, we have reviewed and summarized cybersecurity educational best practices, which should help to inform cybersecurity pedagogy.
Within the cybersecurity space, industry-accepted certification schemes, such as the CISSP and associated programs (e.g., Dulaney, 2009; Tipton & Henry, 2007) already provide interested parties with a wealth of information related to information security. As an example, the “Official (ISC)2 Guide to the CISSP CBK” contains nearly 1000 pages of study material. Despite this wealth of information, in practice, organizations typically suffer penetrations and compromises due to poor user behavior or incorrectly managed systems. It is often the case that the system fails not because of ignorance on the part of the defender, but because basic but well-known steps were not taken (Workman, et al., 2008). There remains a significant knowing-doing gap, as evidenced by rampant cybersecurity breaches that have recently taken place.
In our study, one of our core goals was, therefore, to suggest how to change the behavior of participants, moving them toward actions that enhance cybersecurity. In the cybersecurity space, improving awareness of the principles of information assurance and moderating behaviors is often more important than presenting an overwhelming amount of information that is not put in to practice. Beyond that, getting practitioners to habituate affirmative behaviors using best practice methods is clearly beneficial. Furthermore, as actual preventative steps change quickly, care must be taken to produce learning materials that are actionable, but that have a reasonable period of applicability before obsolescence.
Consequently, we sought to understand the actual state of the art in cybersecurity education and gain insight into neglected areas, and the approximate level of awareness and technical understanding of the issues. We wished to ensure that our curriculum was both complete and focused, aimed at changing core behaviors that would immediately bolster the stability of cyber infrastructure. In essence, we derived from our research that our educational philosophical approach should be: (1) to stimulate change in a reasonable number of behaviors, rather than to educate broadly that create no lasting benefits, and (2) imbue and reinforce learning through “live-fire” practice with realistic simulations.
Next, our goal from the research was to determine ways for “doing better”. Motivated students who understand the importance and applicability of the materials presented to them learn better. To this end, we suggest to change the traditional learning approach from the present-memorize-test model to a show-test-practice-assess model. Moreover, we introduced how incremental chunks of knowledge situated in real-world contexts, that is, gamified, may instill a sense of emotional and cognitive investment in the scenarios by the learner. With regard to commercial, civil, and governmental organizations, regardless of the size or sophistication of the entity, a program that clearly but concisely communicates and experientially situates real threats posed to cyber infrastructure will help engage participants and aid knowledge retention and implementation. Most importantly, we argue that this approach will produce responsive actors who will apply their knowledge when it most counts.
Limitations and lessons learned from our study include the notions that the quantity of cybersecurity information available in books or articles, or online from researchers, companies, user groups, and blogs provide a virtual “firehose” of warnings and advice related to cybersecurity. Indeed, perhaps the largest problem is the overwhelming and untargeted raft of information available. Cybersecurity risks surround us, but there is little understanding on the part of users, technologists and managers that links a particular behavior to an undesirable outcome. For example, users who infect their machines often have no idea of the source of infection, or the choices that led to it; they simply know something has gone wrong. This low-quality feedback mechanism has jaded users at all levels, and led to a laissez-faire approach to cybersecurity. Users know better, but threats are abstract, distant, and omnipresent, all at the same time, and this accounts for why people may know better but don’t do better (Workman, et al., 2008). We aimed to carefully articulate a pedagogical approach with material that can be been personalized or will allow customization that can be optimized for the learner using mixed-modes.
What our research also tells us is that electronic infrastructure is critical to the smooth and safe operation of all aspects of everyday operations. Attackers are well motivated, and do not approach problems the way most people typically expect, and smooth running is critical to businesses and individuals. Education should tie cybersecurity threats back to the system, using real examples, and illustrate how defenders should not “stovepipe” threats. Finally, it is important to realize that seemingly small behavioral changes by users, and how attackers can leverage small errors in operations, compromises many kinds and areas of systems that form the threat matrix and vectors to be considered in cybersecurity education.
In summary, in contrast to the materials that focus exclusively on managing cybersecurity or the more technical aspects of cybersecurity within an ecosystem, training materials at this level are challenging due to the massive range of environments we must consider – ranging from small companies to large corporations, and government infrastructure. It is tempting to provide a simple list of technical topics in a checklist, but doing so is actually a prime example of the wrong approach. Although topics such as secure remote access, patch management, change management, and the intersection of physical and cybersecurity are suitable for checklists, they simply fail to ignite behavioral change that is so needed in cybersecurity responsiveness. Immersion in an environment via simulations and live activities appear to us to be critical to applied learning performance.