Scholars argue that hate speech on social media can lead to violence and must be regulated.
In Myanmar, at least 10,000 civilians in Muslim villages were slaughtered following years of local Facebook posts featuring anti-Muslim rhetoric and threats of violence. After the genocide, Facebook defended its failure to remove these posts by citing the lack of user flagging.
As social media companies take an increasingly active role in regulating hate speech, a fierce debate rages: Are companies removing the right posts? The answer is “no,” according to the authors of a recent paper.
Arguing that social media companies moderate posts without sufficiently considering context, the authors, Richard Wilson and Molly Land of the University of Connecticut School of Law, advocate an empirically based, context-specific approach to hate speech regulation.
Wilson and Land present empirical evidence correlating hate speech posts and violent acts. They point to the simultaneous rise of social media and populist politics, with social media platforms creating echo chambers that deepen political polarization.
One study cited by Wilson and Land shows a statistically significant correlation between anti-immigrant and anti-Muslim tweets and physical attacks on Muslims and immigrants in Germany and the United States in 2016.
Although the empirical evidence indicates only correlation and not necessarily causation, Wilson and Land conclude that hate speech likely contributes to physical violence. In addition, they note that online disinformation campaigns have purportedly undermined democratic elections and influenced the outcome of the Brexit vote and 2016 U.S. presidential election.
Wilson and Land acknowledge the dangers of overregulating online speech, which can potentially silence minority opinions. For instance, in the 1830s, southern states justified banning abolitionist speech based on its ability to incite violence and rebellion. More recently, the Chinese government imprisoned a Christian pastor for saying the Chinese Communist Party’s ideology was “incompatible with the Christian faith.” The governor of South Dakota proposed legislation banning “incitement to riot” to restrain protests by environmental activists at the Keystone XL pipeline.
Such instances of overregulation may present additional concerns when private entities, beholden to shareholders and not to the public, police themselves. Currently, social media platforms are not forthcoming about their self-regulation methods, and their use of “black box” artificial intelligence compounds the lack of transparency.
Despite these dangers, regulation of hate speech in the United States rests almost entirely in private actors’ hands because the First Amendment constrains government regulation of speech. Under current case law, federal and state governments can only regulate speech that represents a “true threat” or incitement to imminent lawless acts, leaving most offensive speech outside the government’s regulatory authority. In response to calls for greater regulation of hate speech, the private sector has stepped up. In recent years, social media companies have rapidly increased their own regulation of online speech. Of note, no social media platform has reversed a rule regulating hate speech after implementing it.
Currently, social media companies use various means to regulate speech. One mechanism, called “user flagging,” relies on users to report content they consider inappropriate. Another method involves algorithms that classify potentially problematic content. Once posts are tagged by either user flagging or automation, human content moderators take over the process. For example, at Facebook, tens of thousands of moderators rapidly review posts flagged by users, relying on a 10,000-word guidebook covering harmful behavior, sensitive content, and legal violations. If uncertain about a post, moderators can escalate a post to senior moderators and eventually to lawyers or policymakers at headquarters.
Wilson and Land, however, say that this approach fails to consider context. Context often defines speech. A context-free approach can limit important speech about uncomfortable topics while permitting seemingly benign speech with “corrosive environmental effects.”
For example, broad rules may lead a social media company to remove a discussion about racial slurs that is obviously intended for the public interest. Conversely, the same rules may allow language that, although harmless in some parts of the world, would likely incite violence where a post was made. Such regulations can be simultaneously overbroad and underinclusive.
Although most platform policies acknowledge the relevance of context, Wilson and Land say that current approaches fall short. For example, Twitter’s approach looks to the text of a post but does not integrate external information or geographically localized meanings. Artificial intelligence (AI) has an increasing role in identifying hate speech, but it cannot reliably distinguish between extremist speech and parody, or between hate speech and legitimate discussions of sensitive topics. Likewise, human moderators must review posts quickly and may not understand local customs, resulting in insufficient consideration of context.
User flagging can fail too. After the Myanmar genocide, for instance, Facebook explained that users had not flagged enough inciting posts for the platform to take action. Coordinated campaigns can also manipulate the flagging system to silence certain perspectives, and groups that are frequently attacked by hate speech may experience “flagging fatigue,” Wilson and Land warn.
Wilson and Land offer five recommendations for improving context-based regulation. First, they suggest greater care in the use of AI and automation. Companies should disclose the accuracy of algorithms and design systems informed by political and cultural context.
Second, the authors encourage moving from a one-size-fits-all model to multiple models appropriate for different countries, with country specialists informing programmers.
Third, Wilson and Land propose maintaining a list of country-specific slurs and having content moderators consider each country’s political and cultural context. The categories of protected persons on companies’ lists should be flexible and consider coordinated attacks occurring in a particular region.
Fourth, Wilson and Land suggest running pilot projects in at-risk countries, such as where international agencies have identified risks of genocide, and then closely scrutinizing posts by political leaders.
Finally, Wilson and Land advocate reorganizing social media companies to adopt an artisanal model, in which employees around the world moderate local content. Compared to AI and centralized workers far removed from the circumstances of the speech, local moderators would be culturally and linguistically close enough to the sources of the posts to detect coded hate speech adequately.
Wilson and Land warn that until companies apply context-specific regulation, the platforms will remain prone to accusations of responsibility for real-world violence.
