Do phishing simulations actually serve a purpose?

Are there more effective and simpler solutions?

The purpose of this article is to analyze certain aspects of phishing simulation campaigns and to raise questions about their impact and effectiveness. The intention is to spark a debate that helps organizations in designing their cybersecurity awareness campaigns to achieve the highest possible level of protection in terms of user behavior.

What is a phishing simulation?

A phishing simulation is a tool many organizations use to diagnose the degree of exposure of their employees to malicious phishing messages and to educate users on the prevention of these types of risks.

Real malicious messages often arrive via email, attempting to deceive users. The objectives of cybercriminals sending phishing messages usually involve obtaining payment information, acquiring login credentials, or gathering personal information for criminal purposes that pose serious harm to all types of organizations.

Phishing simulation campaigns replicate behaviors characteristic of real phishing attacks such as social engineering techniques, use of links or attachments or redirection to fake websites or replicas of real websites.

These campaigns are usually conducted using controlled and harmless phishing simulation platforms, allowing for the measurement of user reactions and providing relevant statistics such as the number of users who read the phishing simulation message, open the link, or download an attachment.

In some cases, users who have performed some incorrect action in response to the phishing simulation message are invited to receive training that will help them learn how to detect future real malicious messages that may arrive in their email inboxes.

What happens to users who do not open the spoofed phishing message?

Interpreting instances in which users open simulated phishing messages and their links is clear. But how can one understand the cases where users have not opened or interacted with them?

It’s unrealistic to assume that all users who didn’t fall for the simulated trap made a conscious decision to ignore, delete, or report it. Since it’s a subjective action, user behaviors towards the hundreds or thousands of messages they receive daily depend on many variables, which we analyze below:

That the message reaches all users of the organization. Preparing phishing simulations is extremely complex. For example, filter systems must be configured to let the phishing simulation message pass through and reach each user’s inbox. Even so, corporate email systems often have systems in place that can indicate that a message is suspicious.

The workload and number of messages to be read. Some users may not open a phishing simulation message for reasons as simple as already having a large quantity of work to do that day, or many messages to read and respond to. Those users might not have seen the message and therefore would not have had the chance to determine which action is the correct one.

Users may be more or less likely to open certain messages depending on the device used to access their inbox (i.e. a mobile device or a desktop computer).

Individual subjectivity. A single simulated phishing message can spark varying levels of interest in each individual, depending on their personal situation, interests, or hobbies. Thus, it’s possible that the reason for not opening a simulated phishing link could be due to a lack of interest in the content of the message.

Taking all of these variables into account, can we really interpret that a user who doesn’t fall for the phishing simulation does so because they are able to analyze the message? If that’s the case, can we infer that they won’t be a victim of real attacks?

Or is it more realistic to consider that an undetermined number of the users targeted in the simulation never saw the message, or did not read it for one or more of the reasons mentioned above?

Other drawbacks of phishing simulations

In addition to the uncertainty arising from the inability to clearly diagnose cases in which a large group of users have not interacted with the simulated phishing message, there are other drawbacks to consider before launching phishing simulation campaigns.

Technical complexity. Any phishing simulation campaign is subject to significant complexities. On one hand, there is the process of creating all the elements of the simulation, such as the message and a landing webpage, as well as configuring the simulation platform. Another complex aspect is that corporate networks and external systems are increasingly better equipped to prevent dubious messages from reaching users. Therefore, there is a high level of complexity involved in ensuring that all filters and firewalls allow the campaign messages through.

High Cost. Closely associated with the technical complexity is the issue of high cost. Whether an external service is contracted or the simulation is carried out with internal resources, the cost of these simulations is usually in the thousands of Dollars or Euros.

Negative employee reactions. Another significant consideration is the impact a phishing simulation campaign may have on the staff as a whole, as well as on specific staff members. Depending on how the simulation processes are contextualized, individual or collective reactions, be they positive or negative, can play an important role in the effectiveness of the campaigns.

Are there alternatives to phishing simulations?

Given all these questions and potential drawbacks, it is worth asking whether there are more effective alternatives to test users and raise awareness at the same time.

Here are the criteria that an alternative system should meet to surpass the effectiveness of a phishing simulation:

• Ensure that you can raise awareness among the entire workforce.
• Ensure it is not dependent on workload
• Ensure it is not affected by individual subjectivity.
• Optimize the balance between cost, complexity, and effectiveness.
• Make it a positive experience for users.

In the article on gamified simulators we analyze, in detail, one of the alternatives many organizations consider a more effective method for raising cybersecurity awareness.