Mental Models of Intelligence Collectors and Analysts for Characterizing Adversarial Threats

Project Details

Abstract:

James L. Regens (University of Oklahoma Health Sciences Center, OUHSC) and his colleagues Carl J. Jensen, III (University of Mississippi), David N. Edger (3CI Consulting), and David Cid (Memorial Institute for the Prevention of Terrorism, MIPT) will examine factors that can impact the potential effectiveness of an intelligence-led policing (ILP) approach to counterterrorism efforts in the United States.

In reaction to al-Qa’ida attacks targeting New York City and Washington, DC on 11 September 2001, intelligence-led policing (ILP) has become a key element in law enforcement efforts in the US to counter terrorism. Implicit in the application of ILP to counterterrorism (CT) is the assumption that the approach is robust enough to generate timely, credible intelligence sufficient to identify and disrupt terrorist planning even though several constraints are inherent in terrorism: (1) terrorist entities are clandestine, closed systems; (2) terrorism campaigns and their associated adversarial behaviors are dynamic over time; and (3) the spatial domain is fluid because the operations of terrorist groups can transcend political boundaries. The US legal system imposes additional constraints on information collection and archiving which affect police intelligence operations including CT.

Despite those limiting factors, with its focus on prevention rather than reaction, ILP has been embraced as a way for police to ‘get ahead of the curve’ by generating actionable intelligence to guide counterterrorism operations that preempt rather than respond to potential terrorist incidents. Past research and the team’s professional experience supports the premise that situational awareness of real world phenomena within specified temporal and spatial domains (e.g., perception of environmental elements) forms a common intelligence picture of potential adversarial threats. Presumably, law enforcement personnel will encounter a variety of situations that provide information about suspicious activities that are precursors of terrorist attacks. Accurate terrorism threat characterization, therefore, is a necessary first step for ILP to be an effective counterterrorism tool. In essence, situational  awareness  provides  the  framework  which  law  enforcement  officers  functioning  as  intelligence collectors utilize in order to identify and report information about suspicious activities based on either direct observation or provided by sources that may be indicative of terrorism-centric behaviors. Analysts similarly apply situational awareness to identify indicators of terrorism and determine whether the weight-of-evidence merits issuing warnings.

Primary Findings:

During the first year of this project, the research team completed multiple tasks: compliance approvals, study design finalization including access to public information sources (i.e., collectors and analysts) in order to collect unclassified information about terrorism scenarios for qualitative and statistical analysis, operational definitions and quantifiable indicators for scenario development and programming for web-based situational awareness assessment tool, database schema, and data collection. A systematic literature review literature was conducted and is an ongoing activity.

The research team drew on its subject matter expertise and experience in law enforcement, intelligence, and counterterrorism to generate 28 scenarios illustrative of ‘real-world’ contexts for this research. The scenarios taken as a group include inherently non-suspicious behaviors, generic suspicious behaviors, non-terrorist criminal behaviors, and potentially terrorist-centric behaviors. The potentially terrorist-centric behaviors span the spectrum of terrorist activities including recruiting, funding, weapons acquisition, materials acquisition, expertise acquisition, eliciting information, surveillance, testing security, and deploying assets operationally. Each individual scenario has from one to six components. Using a web-based situational analysis assessment tool designed by our research team, police officers draw on their experience and training to individually score each component iteratively on an 11-point Likert-type scale ranging from not suspicious (‘0’) to highly suspicious (‘10’). This allows the officers to differentiate variation in terms of the likelihood that activities embedded in specific contexts are highly suspicious. Intelligence analysts rate each suspicious component on two 11-point Likert-type scales; first as an indicator of terrorism ranging from not likely (‘0’) to highly likely (‘10’) and second as a warning ranging from not likely (‘0’) to highly likely (‘10’) respectively. The resultant scaling produces quantitative data for empirical analysis. Multi- component scenarios are dynamic; that is, each component is rated sequentially prior to proceeding to the next component and/or scenario. However, it is possible to simultaneously review but not change the original rating for all prior components within a multi-component scenario before scoring new components. This feature emulates the concept of the ability to recall earlier knowledge and place new information in the context of cascading events.

The research team also determined the optimal number of public sources (i.e., collectors and analysts) from which to collect unclassified information based on statistical significance, statistical power and sample size considerations. Two crucial study design questions were addressed: (1) sample size needed to assure a given probability of detecting a statistically non-stochastic significant effect of a given magnitude if one truly exists and (2) likelihood that a statistically significant effect of a given magnitude can be identified if it really is there. The first question was addressed by sample size calculations and the second by power calculations. For sample size and statistical power calculations, we specified the desired values for the probabilities of type I and type II errors in order to calculate sample size typical of studies published in medical and psychology journals in order to be conservative with regard to erroneously accepting false findings. We opted for the conventional α level of ρ =0.05 used by most researchers in the behavioral sciences. Given the exploratory nature of our research, the research team set the β level at 0.20 in order to obtain a reasonable amount of power (i.e., power = .80) thereby minimizing the likelihood of type II error.  Data have been collected from a broad cross-section of US police officers and intelligence analysts subsequent to obtaining a determination from the OUHSC Institutional Review Board that this research is exempt (IRB #15744).

Peer-reviewed journal articles produced to date from this project:

  • James L. Regens, Carl J. Jensen III and David N. Edger. “Situational Awareness as a Cornerstone of Terrorism Threat Characterization.” Intelligence Analysis (in press).
  • Carl J. Jensen III, N. Griffin and James L. Regens. “Intelligence-Led Policing as a Tool for Countering the Terrorism Threat.” Homeland Security Review (in press).
  • Frederic Lemieux and James L. Regens. 2012. “Assessing Terrorist Risks: Developing an Algorithm-Based Model for Law Enforcement.” Pakistan Journal of Criminology 3 (March): 33-49.
Methodology:

This project is developing a methodology to elucidate the mental models of law enforcement personnel acting as intelligence collectors and intelligence analysts for adversarial threat characterization. The general methodology is applicable to Federal, state, and local agencies. Delineating the mental models used by line officers acting as collectors and those used by law enforcement intelligence analysts can help inform ways to improve situational awareness so that bias in both collection (e.g., reporting) and analysis is reduced.

Timeframe

Project Period:
to