The measurement crisis behind cyber awareness 

Article content

Without accurate measurement, organizations cannot calibrate their training to focus on real world threats. 

Article content

Why vendor benchmarks are a dead end 

Article content

To solve this, Johnson says the industry must abandon its attachment to vendor-supplied benchmarks. Benchmarks are seductive because they offer a sense of competition. Where do we stand compared to peers.  

Article content

But they are only point-in-time snapshots and do nothing to help an organization understand its own progress. 

Article content

Instead, Johnson advocates for a scientific approach rooted in baselining. Rather than comparing one organization to another, he proposes a model that compares an organization to itself over time.  

Article content

It begins with a baseline established through an initial assessment. That baseline is then reset on a consistent rhythm monthly or quarterly. Each reset serves two purposes – it defines a new starting point and reveals the incremental impact of what happened since the last measurement. 

Article content

This allows security leaders to track whether their program is strengthening or stagnating.  

Article content

Over time, it builds undeniable evidence of what works and what does not. The transformation is profound. Security awareness stops being a compliance chore and becomes a measurable component of risk reduction. 

Article content

Article content

Focus on the few who drive most of the risk 

Article content

One of Johnson’s most pragmatic insights comes from segmentation analysis based on the Pareto principle. In every organization, three groups emerge from phishing simulations: a majority who never click, a large middle group he calls learners who occasionally fail but improve and a very small subset of movers – repeat offenders whose decisions repeatedly expose the organization to risk.  

Article content

With proper measurement, a company can direct its most focused interventions to the group that drives a disproportionate percentage of incidents.  

Article content

Meanwhile, the top performers can be leveled up into a strength more likely to report real attacks faster than security teams can detect them. 

Article content

The benefit is operational as much as it is behavioral. Security teams overwhelmed by training obligations can finally allocate their resources based on evidence rather than one-size-fits-all mandates. 

Article content

The threat landscape is evolving faster than training models 

Article content

Johnson notes that phishing attacks themselves have transformed dramatically. The earliest phishing attempts were crude plain text messages asking users to reset a BlackBerry ID. Today, they are fully functioning replicas of brand sites with professional design, tight copywriting and sophisticated obfuscation of malicious elements.  

Article content

Article content

Many assume that artificial intelligence is responsible for this leap. Johnson says, the real turning point arrived earlier with the rise of phishing as a service (PHAAS). The PHAAS ecosystem supplies attackers with ready-made kits that include templates, hosting, credential capture infrastructure and automated delivery.  

Article content

It democratized cybercrime and reduced both the technical and financial barriers to entry. AI has since accelerated this evolution even further. 

Article content

The takeaway is stark: if the threat landscape has evolved dramatically, training models built on legacy assumptions cannot hope to keep pace without measurement that reflects current realities. 

Article content

The skill cyber professionals need most 

Article content

Although the briefing focuses heavily on measurement, Johnson also offers advice for professionals working across security disciplines.  

Article content

The single most important skill, he says, is communication. Cyber professionals must learn to adjust their language to their audience. 

Article content

With leaders, the conversation must connect directly to business outcomes. With peers it can be technical and acronym rich. With end users it must be stripped of jargon and shaped around clear guidance.