Extremists often use subtle forms of coding rather than explicit statements. These may include tentative allusions, fashion choices, symbols, or references to cultural products such as music. Such indirect communication can make extremist ideas less immediately recognizable and may complicate efforts to establish intent or enforce relevant laws. From the perspective of online content moderation, the problem is particularly difficult when publicly accessible material uses deliberately ambiguous or obscure language.
Extremist users may also employ irony, religious or ethnic othering, and coded references rather than direct expressions of hatred. They can combine symbols, images, and cultural references in ways that are meaningful primarily to insiders. Even when moderators recognize these signals, determining whether a particular post constitutes genuine extremist advocacy, satire, provocation, or merely an ambiguous cultural reference can be difficult.
As a result, content removal or account bans are not always straightforward, particularly when apparently humorous posts combine violent imagery with messages that may appeal to younger or less informed audiences. Extremist communication can also involve positive reinforcement rather than openly negative propaganda. For example, users may repeatedly promote particular identities, narratives, or symbols, while organized discussion pages and comment threads gradually reinforce those ideas through repetition and social validation.
Nevertheless, not every ambiguous or controversial expression can, or should, be prohibited. Mainstream cultural references that have been removed from their original context may be provocative without necessarily constituting extremist communication. The greater concern arises when individual posts form part of a cumulative pattern in which hateful rhetoric, dangerous ideas, and collective hostility reinforce one another. Consequently, effective moderation requires attention not only to individual viral contributions but also to the broader context, patterns of interaction, and cumulative development of online discussions.
Thorsten Koch, MA, PgDip
January 2020
