The Purple Light: The A.I. Surveillance Tool DHS Uses to Detect “Sentiment and Emotion”

Saturday, August 26, 2023

The A.I. Surveillance Tool DHS Uses to Detect “Sentiment and Emotion”

By Joseph Fox

Posted on August 26, 2023

Customs and Border Protection (CBP), part of the Department of Homeland Security, has bought millions of dollars worth of software from a company that uses artificial intelligence to detect “sentiment and emotion” in online posts, according to a cache of documents obtained by 404 Media.

CBP told 404 Media it is using technology to analyze open source information related to inbound and outbound travelers who the agency believes may threaten public safety, national security, or lawful trade and travel. In this case, the specific company called Fivecast also offers “AI-enabled” object recognition in images and video, and detection of “risk terms and phrases” across multiple languages, according to one of the documents.

Marketing materials promote the software’s ability to provide targeted data collection from big social platforms like Facebook and Reddit, but also specifically names smaller communities like 4chan, 8kun, and Gab. To demonstrate its functionality, Fivecast promotional materials explain how the software was able to track social media posts and related Persons-of-Interest starting with just “basic bio details” from a New York Times Magazine article about members of the far-right paramilitary Boogaloo movement. 404 Media also obtained leaked audio of a Fivecast employee explaining how the tool could be used against trafficking networks or propaganda operations.

The news signals CBP’s continued use of artificial intelligence in its monitoring of travelers and targets, which can include U.S. citizens. In May, I revealed CBP’s use of another AI tool to screen travelers which could link peoples’ social media posts to their Social Security number and location data. This latest news shows that CBP has deployed multiple AI-powered systems, and provides insight into what exactly these tools claim to be capable of while raising questions about their accuracy and utility.

“CBP should not be secretly buying and deploying tools that rely on junk science to scrutinize people’s social media posts, claim to analyze their emotions, and identify purported ‘risks,’” Patrick Toomey, deputy director of the ACLU’s National Security Project, told 404 Media in an email.

404 Media obtained the documents through Freedom of Information Act requests with CBP and other U.S. law enforcement agencies.

One document obtained by 404 Media marked “commercial in confidence” is an overview of Fivecast’s “ONYX” product. In it Fivecast says its product can be used to target individuals or groups, single posts, or events. As well as collecting from social media platforms big and small, Fivecast users can also upload their own “bulk” data, the document says. Fivecast says its tool has been built “in consultation” with Five Eyes law enforcement and intelligence agencies, those being agencies from the U.S., United Kingdom, Canada, Australia, and New Zealand. Specifically on building “person-of-interest” networks, the tool “is optimized for this exact requirement.”

Related to the emotion and sentiment detection, charts contained in the Fivecast document include emotions such as “anger,” “disgust,” “fear,” “joy,” “sadness,” and “surprise” over time. One chart shows peaks of anger and disgust throughout an early 2020 timeframe of a target, for example.

The document also includes a case study of how ONYX could be used against a specific network. In the example, Fivecast examined the Boogaloo movement, but Fivecast stresses that “our intent here is not to focus on a specific issue but to demonstrate how quickly Fivecast ONYX can discover, collect and analyze Risks from a single online starting point.”

That process starts with the user inputting Boogaloo phrases such as “civil war 2.” The user then selects a discovered social media account and deployed what Fivecast calls its “‘Full’ collection capability,” which “collects all available content on a social media platform for a given account.” From there, the tool also maps out the target’s network of connections, according to the document.

Lawson Ferguson, a tradecraft advisor at Fivecast, previously showed an audience at a summit how the tool could be used against trafficking networks or propaganda operations. “These are just examples of the kind of data that one can gather with an OSINT tool like ours,” he said. Jack Poulson, from transparency organization Tech Inquiry, shared audio of the talk with 404 Media.

Ferguson said users “can train the system to recognize certain concepts and types of images.” In one example, Ferguson said a coworker spent “a huge amount of time” training Fivecast’s system to recognize the concept of the drug oxycontin. This included analyzing “pictures of pills; pictures of pills in hands.”

Fivecast did not respond to a request for comment.

CBP’s contracts for Fivecast software have stretched into the millions of dollars, according to public procurement records and internal CBP documents obtained by 404 Media. CBP spent nearly $350,000 in August 2019; more than $650,000 in September 2020; $260,000 in August 2021; close to $950,000 in September 2021; and finally almost $1.17 million in September 2022.

CBP told 404 Media in a statement that “The Department of Homeland Security is committed to protecting individuals’ privacy, civil rights, and civil liberties. DHS uses various forms of technology to execute its mission, including tools to support investigations related to threats to infrastructure, illegal trafficking on the dark web, cross-border transnational crime, and terrorism. DHS leverages this technology in ways that are consistent with its authorities and the law.”

In the context of why CBP needs to buy Fivecast’s software, the internal CBP documents point to several specific parts of the agency. They are the Office of Field Operations (OFO), the main bulk of CBP which enforces border security; the National Targeting Center (NTC) based out of Virginia which aims to catch travelers and cargo that the agency believes threaten the country’s security; the Counter Network Division (CND) which is part of the NTC; and finally the Publicly Available Information Group (PAIG), which focuses on data such as location information according to other documents I’ve obtained previously.

Yahoo News reported in 2021 that the CND has gathered information on a range of journalists. The Office of the Inspector General made a criminal referral for an official who worked with CND for their role in the monitoring, but they were not charged. A supervisor of that division previously told investigators that at CND “We are pushing the limits and so there is no norm, there is no guidelines, we are the ones making the guidelines.”

“The public knows far too little about CBP’s Counter Network Division, but what we do know paints a disturbing picture of an agency with few rules and access to an ocean of sensitive personal data about Americans,” Toomey from ACLU added. “The potential for abuse is immense.”