3.1.1. Transformer encoder input embedding generation

Consider E1 again. As shown in Figures 1 and 2, the model first tokenizesFootnote ⁹ the input sequence into individual tokens. Then it maps the tokens into input IDs according to their position in the vocabulary containing all possible words in the training corpus. At this stage, both ‘the’s have the same input ID since they occupy the same position in the vocabulary. Then, the model takes the input IDs and maps them into vectors of size 512Footnote ¹⁰ through an embedding layer which acts as a look-up table to get the initialized input embeddings (Vaswani et al., Reference Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin2017). These corresponding embeddings can be loaded from other previously pre-trained models or initialized randomly,Footnote ¹¹ and the embedding layer is often the first trainable layer in a transformer-based language model. Notice that at this stage, the same word is always mapped to the same embedding without any contextual information from surrounding words, similar to static word embedding models.

Figure 2.

Input embedding generation to the transformer encoder. numbers are for illustration purposes only.

Figure 2 Long description

Step-by-step diagram using the sentence ‘Stop the evil yellow invasion from the Chinese’ to illustrate how input tokens are converted into input IDs, token embeddings, and positional encodings. The token embeddings and positional encodings are added together to produce an 8-by-512 encoder input embedding matrix. Numbers are illustrative.

The next part of the embedding layer is the calculation of positional encodings, which are also of length 512 for each token and used by the model to keep track of their positions in the sequence and how distant they are from each other.Footnote ¹² The token embeddings shown on the top in Figure 2 and positional encodings shown in the middle are summed together to construct the input embeddings to a transformer encoder illustrated in Figure 1.

3.1.2. The self-attention mechanism

The core of the transformer encoder is the multi-head self-attention layer. Before covering multi-head self-attention, I first discuss the (single-head scaled dot-product) self-attention mechanism without introducing trainable parameters to build intuition.

Consider E1 and E2 again. The mere syntactic arrangement fails to capture the polysemous nature of the word ‘yellow.’ For a nuanced and context-rich representation of the word in E1, it is necessary to establish a semantic linkage between ‘yellow’ and contextual markers such as ‘evil’ or ‘Chinese.’ In the parlance of transformers, the word ‘yellow’ should pay attention to other surrounding words, creating connections that vary in strength based on context. The key advantage of transformers is their ability to not only estimate word positions in a semantic space, but also the relative importance of a given word in shaping the meaning of other words (Vaswani et al., Reference Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin2017). This is achieved through the incorporation of the scaled dot-product self-attention mechanism.

To disentangle the mechanism mathematically, imagine I want to calculate the contextualized embeddings incorporating self-attention scores for E1, as illustrated in Figure 3. The self-attention mechanism takes in an input embedding matrix $X$ (i.e., the encoder input embedding matrix shown in Figure 2 and in the middle of Figure 1) and outputs a contextualized embedding matrix ${X^{\text{*}}}$. ${X^{\text{*}}}$ is generated using the self-attentionFootnote ¹³ score matrix $A$ through:Footnote ¹⁴

(1)\begin{equation}{X^{\text{*}}} = {\text{Self - Attention Score}}\left( X \right) \cdot X = A \cdot X = softmax\left( {\frac{{X \cdot {X^T}}}{{\sqrt d }}} \right) \cdot X\end{equation}

Figure 3.

Calculation of the self-attention mechanism. Numbers are for illustration purposes only.

Figure 3 Long description

Diagram illustrating the calculation of self-attention. An encoder input embedding matrix is multiplied by its transpose, scaled, and normalized with a softmax function to produce an 8-by-8 self-attention score matrix. This matrix is then multiplied by the original input embedding matrix to produce a contextualized embedding matrix. Example tables show token-to-token attention scores and resulting contextualized embeddings.

where:

\begin{equation*}\begin{array}{*{20}{c}} {X = {\text{the encoder input embedding matrix, }}} \\ {d = {\text{dimensionality of the model (see footnote 10}}{\text{.)}}} \end{array}\end{equation*}

For E1, the input sequence length is 8 and the dimension $d$ of the model and input embedding is 512,Footnote ¹⁵ which is the dimension of the encoder in the original paper that introduced transformers (Vaswani et al., Reference Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin2017). As shown in Figure 3, the model first conducts dot-product matrix multiplication of $X$ and the transpose of $X$, and divide it by a scaling factor, usually the square root of the input embedding dimension. The resulting matrix is then passed through a softmax function for normalization, resulting in an 8 by 8 matrix where all the rows sum up to 1. The values in this resulting matrix, as shown in matrix $A$ in Figure 3, are the self-attention scores, where they represent the strengths of the relationships between one token and another. If a particular token is not relevant to the understanding of the target token, the self-attention mechanism may assign a very low score to that position, effectively ignoring its contribution. Higher self-attention scores correspond to stronger relationships between two tokens. Intuitively, I can think of the word ‘yellow’ consisting of 0.01 Stop $ + $ 0.03 the $ + $ 0.15 evil $ + $ 0.33 yellow $ + $ 0.25 invasion $ + $ 0.04 from $ + $ 0.02 the $ + $ 0.17 Chinese, which sums to 1. Different from the input embedding generation, the same token ‘the’ can have different self-attention scores when they appear multiple times in the input sequence.

To obtain the contextualized embedding matrix ${X^{\text{*}}}$ (i.e., the yellow matrix in Figure 3) from the self-attention mechanism, the model conducts another dot-product matrix multiplication of $A$ and $X$. Each row in the resulting matrix ${X^{\text{*}}}$ contains the contextualized embedding of the initial input sequence, where the self-attention scores and positional encodings have been incorporated into the resulting embeddings.Footnote ¹⁶

At this stage, the calculation of self-attention requires no trainable parameters. Up to now the interaction between tokens has been driven by their own embeddings. To add trainable parameters to the model, I add updatable weight matrices. Although in the simplified formulation of self-attention based on Equation 1 the model uses the same input embedding matrix $X$ three times, in practice, each instance of $X$ is multiplied by a distinct weight matrix ${W_Q}$, ${W_K}$, and ${W_V}$, as illustrated in Figure 4. These three weight matrices learn task-relevant linear projections of the same input sequence, each emphasizing different aspects of how tokens relate to one another. By separating the transformations this way, the model can capture multiple dimensions of contextual meaning and generate more expressive attention patterns. This operation yields the contextualized embedding matrix ${X^{\text{*}}}$ as the output.Footnote ¹⁷ In practical transformer implementations, the self-attention mechanism therefore operates not directly on the original embeddings, but on these learned projections.Footnote ¹⁸

Figure 4.

Self-attention with trainable weight matrices ${W_Q}$, ${W_K}$, and ${W_V}$.

Figure 4 Long description

Diagram showing how trainable query, key, and value weight matrices are added to the self-attention mechanism. The input embedding matrix is projected into query, key, and value representations; query and key representations are used to calculate normalized self-attention scores, which are then applied to the value representation to produce a contextualized embedding matrix.

3.1.3. The multi-head self-attention layer

The entire process described so far is everything that happens within a module called an attention head. When using only one set of weight matrices ${W_Q}$, ${W_K}$, and ${W_V}$, it is called single-head attention. Often times, single-head attention is not enough to capture complex relationships among tokens in the same sequence. Therefore, multi-head attention is used to make the model attentive to different aspects of the tokens.

The only architectural difference for multi-head attention is that the model maintains one set of weight matrices for each attention head and then concatenates the resulting contextualized representations from all heads afterwards. Consider a transformer encoder with two attention headsFootnote ¹⁹ as shown in Figure 5. Notice that the encoder input embedding matrix is split into two smaller sub-matrices ${X_1}$ and ${X_2}$ to use as the input for each attention head. The input embedding is typically split across the embedding dimension (columns) instead of the sequence length dimension (rows).Footnote ²⁰ Such approach allows separate sections of the input embedding to learn different aspects of the meanings of each token as it relates to other tokens in the sequence, which allows the transformer encoder to capture richer interpretations of the sequence.Footnote ²¹

Figure 5.

Calculation of the multi-head self-attention with two attention heads.

Figure 5 Long description

Diagram illustrating multi-head self-attention with two attention heads. The encoder input embedding matrix is split into two submatrices, each processed through separate query, key, and value weight matrices. The two attention heads produce separate contextualized embedding matrices, which are concatenated and multiplied by an output weight matrix to produce the final multi-head contextualized embedding matrix.

Within each attention head, the model performs the same self-attention operation as shown earlier in Figure 3, producing one contextualized embedding matrix per head. Then it concatenates the output matrices, i.e., the contextualized embedding matrices, from each attention head to obtain ${X_{1\& 2}}^{\text{*}}$. To learn the optimal way to combine different output matrices from all attention heads, the model adds another trainable weight matrix ${W_O}$ before obtaining the final multi-head embedding matrix ${X^{\text{*}}}$ which contains the contextualized embedding of the input text after the multi-head self-attention mechanism. The matrix ${W_O}$ allows the model to capture more complex interactions across different heads.

During training, all weight matrices are initialized with small random values and updated iteratively through backpropagation. In this process, the model first generates predictions based on its current parameters and compares them to the true labelsFootnote ²² to compute a loss function that quantifies prediction error. The model then computes the gradient and uses this information to adjust the parameters in the direction that reduces the loss. This gradient-based optimization is repeated across many iterations and batches of data, gradually refining the weight matrices to minimize overall error. Once training converges, these learned parameters encode how strongly each token attends to others across multiple representational subspaces, and they are subsequently retained for downstream inference.

3.1.4. Feed-forward neural network layer

The multi-head self-attention mechanism is the core innovation of the transformer encoder. Following Figure 1, the next componentFootnote ²³ is a feed-forward neural network (FNN).Footnote ²⁴ The FNNFootnote ²⁵ receives the output from the multi-head self-attention layer, denoted as ${X^{\text{*}}}$ in Figure 5, and processes it through dimensionality expansion and reduction to produce the encoder’s final output (Lin and Lucas, Reference Lin, Lucas, Box-steffensmeier, Christenson and Sinclair-chapman2024).

While the multi-head self-attention mechanism enables the model to handle long-range dependencies within the data, the FNN enhances the model’s capacity to learn complex and non-linear relationships.Footnote ²⁶ This capacity is crucial for classification tasks requiring language comprehension, where the meaning of a word often relies on its context within a sentence or even across a paragraph.

3.2.1. Domain definition

Before turning to the implementation of the PFP, it is useful to clarify what a domain is in this context. In NLP research, a domain refers to a collection of texts that share systematic linguistic regularities—topical, stylistic, or community-specific—that differ from the general corpora on which base models such as BERT are originally pre-trained. These differences may arise from topical specialization (e.g., legislative debates or policy documents that employ distinctive jargon), stylistic variation (e.g., the informal, sarcastic tone of social-media discourse), or community-specific lexicons that encode culture-bound meanings (e.g., ethnic slurs or political slogans).

Distinguishing among these forms of domain variation is important because they pose distinct representational challenges for language models. Polysemous variation, as in the word ‘left,’ represents a linguistic-semantic challenge: the same surface form can express multiple, unrelated meanings. Resolving such ambiguity depends primarily on the model’s ability to use local syntactic and semantic context—a strength already present in contextual encoders through mechanisms such as multi-head self-attention. In contrast, socio-cultural or community-specific variation, exemplified by the phrase ‘yellow invasion,’ reflects a cultural-pragmatic challenge. Its pejorative sense depends on historically and socially embedded usage that may be under-represented in the general pre-training data. Addressing this type of variation requires additional exposure to in-domain texts in which such expressions appear, allowing the model to learn their contextual associations and pragmatic connotations.

Domain-adaptive pre-training (DAPT) directly addresses this latter problem by continuing pre-training on a large, unlabeled corpus drawn from the target domain before task-specific fine-tuning. Through this intermediate adaptation step, the model updates its internal representations to reflect domain-specific linguistic patterns without discarding the broad semantic knowledge acquired during initial pre-training. In short, DAPT allows the model to better align its representational space with the statistical properties of the target corpus, thereby improving downstream classification performance when the new data differ markedly from the general-domain text.

In later sections, I return to this distinction to show that when the target domain is only moderately distinct from the general pre-training corpus—as in the case of political discourse on Twitter—the empirical gains from DAPT remain limited, underscoring the importance of matching method to domain specificity.

3.2.2. Pre-training stage

To contextualize the following discussion, I use the classic BERTFootnote ²⁷ as an example of encoder-only models (Devlin et al., Reference Devlin, Chang, Lee and Toutanova2019). Encoder-only models exploit transfer learning (Zhuang et al., Reference Zhuang, Zhiyuan, Duan, Xi, Zhu, Zhu, Xiong and He2021).Footnote ²⁸ In the context of classification tasks, such as toxic language detection, researchers usually start to train neural networks by randomly initializing the weights from a specified random seed. Intuitively, as the training process begins, these weights are continually updated to perform the task with fewer errors, a process often referred to as optimization. During the pre-training stage, the model is trained using self-supervised learning, which allows the use of enormous amounts of data without the need for manual labels. For example, for BERT, the self-supervised pre-training task consists of predicting missing words from sentences (i.e., Masked Language Modeling) from a large internet corpus of unlabelled text including the entire Wikipedia and Book Corpus.Footnote ²⁹

The pre-training process of a BERT-like encoder-only model is shown in Parts 1 and 2 in Figure 6. The input tokens (along with a special < CLS > tokenFootnote ³⁰ denoting the start of the sequence specific for BERT-like models) are converted to token embeddings through the embedding layer as described in Section 3.1.1 and Figure 2. These embeddings are passed through a series of transformer encoders to create a set of output embeddings as illustrated in Section 3.1.3 and Figure 5. The first encoder receives the output from the embedding layer as the input, while the subsequent encoders receive the output from the previous encoder as their input. A small fraction of the input tokens is randomly replaced with a generic < mask > token.Footnote ³¹ In pre-training, the goal is to predict the missing token from the associated output embedding.Footnote ³² The model then calculates the loss from each of its predictions to update the weights for all learnable layers as highlighted in shaded boxes in Figure 6 through back-propagation by calculating the gradients or derivatives of the loss with respect to the weights (Devlin et al., Reference Devlin, Chang, Lee and Toutanova2019).

Figure 6.

Pre-training and fine-tuning for BERT-like encoder-only models. Shaded boxes contain trainable parameters.

Figure 6 Long description

Large workflow diagram showing the pre-training and fine-tuning process for a BERT-like encoder-only model. Part 1 shows input text passing through an embedding layer and a stack of transformer encoders. Part 2 shows pre-training through masked token prediction and cross-entropy loss. Part 3 shows fine-tuning for toxicity classification, where the model predicts toxic or non-toxic labels and updates trainable parameters through backpropagation. Shaded boxes indicate trainable parameters.

In the PFP, researchers can optionally conduct DAPT from a publicly pre-trained model to improve classification performance by repeating the pre-training process (Gururangan et al., Reference Gururangan, Marasović, Swayamdipta, Lo, Beltagy, Downey and NA2020), such as training with masked language modeling on domain-specific data, to update trainable parameters in both Parts 1 and 2 shown in Figure 6. Part 1 is then saved as the domain-adapted model to use later for fine-tuning tasks with Part 3. However, for researchers with limited computational resources, fine-tuning on a good quality labeled dataset could also achieve similar performance as discussed in the following section.

3.2.3. Fine-tuning stage

Considering the substantial parameterization inherent to language models,Footnote ³³ pre-training or DAPT can incur significant computational expenses and necessitate extensive datasets. Rather than conducting the pre-training ourselves, political science researchers can leverage pre-trained models previously developed and made available by companies or research institutions worldwide.Footnote ³⁴

Transformer-based pre-trained language models like BERT perform well on general-purpose language tasks but may struggle with domain-specific tasks like toxic language classification. Subsequently, scholars can fine-tune the pre-trained models to accommodate diverse downstream tasks using task-specific data, thereby capitalizing on the general linguistic knowledge established during the pre-training stage (Barbieri et al., Reference Barbieri, Camacho-Collados, Neves and Espinosa-Snke2020; Chalkidis et al., Reference Chalkidis, Fergadiotis, Malakasiotis, Aletras and Androutsopoulos2020; Lee et al., Reference Lee, Yoon, Kim, Kim, Kim, So and Kang2020; Caselli et al., Reference Caselli, Basile, Mitrović and Granitzer2021). In the fine-tuning stage, the model parameters are updated to specialize the model to a particular task.

As illustrated in Parts 1 and 3 in Figure 6, to fine-tuneFootnote ³⁵ a pre-trained model like BERT, scholars can append a task specific neural network on top of the encoder-only model and train the entire model with the a small annotated dataset. Instead of comparing the predicted token with the original masked token in pre-training, during fine-tuning for classification, the model compares the predicted label with the true label to compute the loss and update the weights through back-propagation as shown in Part 3 in Figure 6. Learned linguistic features such as syntactic and semantic roles from the pre-trained model are largely preserved in the fine-tuned models (Merchant et al., Reference Merchant, Rahimtoroghi, Pavlick and Tenney2020).

Compared to pre-training, fine-tuning is a computationally inexpensive approach that requires considerably smaller volumes of data while affording adaptability to specific tasks, and can take just a few minutes to complete. In Section 4, I apply the PFP approach to a toxic language classification task to demonstrate its data efficiency, training efficiency, and performance enhancement capabilities. I show that models trained with PFP outperform the traditional text classification methods reviewed in Section 2 using an annotated sample Twitter dataset of conversation threads following tweets from US Senators one month before and after the 2020 Presidential Election.

4.2.1. Out-of-sample performance

Figure 8 illustrates the balanced accuracy scores on the out-of-sample test dataset for all models listed in Table 1. Detailed evaluation results for each model category (i.e., off-the-shelf models, fine-tuned PFP models, and baseline models) are presented in Tables A1, A2, and A3 in Appendix 6. Across all metrics, the fine-tuned PFP models exhibit significantly improved performance relative to the baseline models consistently. Among these, Model 10 achieves the highest balanced accuracy score of 0.8860, highlighting its effectiveness for toxic language classification. Figure 9 shows the confusion matrix for Model 10. Confusion matrices for Models 14-17 are provided in Appendix 6.

Figure 8.

Balanced accuracy scores of all models.

Figure 8 Long description

Scatterplot showing balanced accuracy for Models 1–17, grouped into off-the-shelf PFP models, fine-tuned PFP models, and baseline models. Fine-tuned PFP models generally perform best, with Model 10 achieving the highest balanced accuracy at 0.886. Model 12, which freezes the encoder parameters, performs substantially worse than the other fine-tuned models. Baseline models show lower and more variable balanced accuracy.

Figure 9.

Confusion matrix for model 10.

Figure 9 Long description

Confusion matrix for Model 10 on the test set. The model correctly classifies 152 non-toxic tweets and 97 toxic tweets. It misclassifies 13 non-toxic tweets as toxic and 17 toxic tweets as non-toxic, indicating strong overall classification performance with relatively few false positives and false negatives.

As shown in Table 1, fine-tuning times for the models varied, with most models requiring only a few minutes. Details are included in Appendix 6. The longest fine-tuning duration was recorded by Model 11, taking 3 minutes and 46 seconds. In contrast, Model 12, which involved updating only the task-specific layer rather than performing full fine-tuning, required just 32 seconds for training, but displayed a notable reduction in performance. It is the worst-performing model, and should not be used for research in general. This underscores the importance of updating the parameters of the encoders during fine-tuning to maximize model performance for a given task. Even without additional training, certain off-the-shelf models (Models 1, 2, 4, and 5) achieved balanced accuracy scores exceeding that of the best-performing baseline model (Model 16). The high performance among off-the-shelf models may, in part, be due to the alignment between toxicity definitions in the annotated OffensEval and AbusEval dataset, which were used to fine-tune these models, and the definition applied within my annotated Twitter dataset. This highlights the importance of validating off-the-shelf models to ensure they align well with the specific requirements of the target dataset.

Interestingly, the process of DAPT prior to fine-tuning, which required a substantial 46 hours of computational time, did not produce any measurable improvement in classification accuracy compared to models that were simply fine-tuned. It is important to note that the Twitter corpus used in this study represents a moderately domain-shifted context rather than a highly specialized linguistic environment. Political discourse on social media differs from the formal, encyclopedic style of the corpora on which BERT was originally pre-trained, yet it does not feature entirely novel or domain-exclusive vocabulary, such as medical or legal documents. Consequently, the limited additional benefit from DAPT observed in Model 13 is theoretically consistent with expectations. This finding is informative for applied researchers: it suggests that while DAPT can substantially improve performance in domains with distinct lexicons—such as medical or legal texts—it may offer only marginal returns for general political communication. The empirical illustration thus serves as a baseline case, demonstrating that DAPT is not always necessary and that task-specific fine-tuning alone can often achieve strong performance at much lower computational cost.

4.2.2. Content validity

Beyond evaluating classification metrics, I also assess the content validity of PFP models in comparison to the baseline models. To this end, I examine instances where Model 10, the best-performing PFP model, classified tweets as toxic with highest probabilities, while baseline models yielded divergent classifications. Table 2 highlights examples of such classifications. In many cases, the baseline models failed to detect toxicity in tweets where neutral words, such as ‘crying’ were used with negative or derogatory intent. This finding suggests that the baseline models struggle with subtle contextual cues that indicate toxicity, while Model 10’s more nuanced understanding of context allows for improved identification.

Table 2.

Examples of disagreed tweets in the test set. Probabilities are for the predicted label

Table 2 Long description

Table showing examples of test-set tweets where Model 10 and baseline models disagree. The columns include the tweet text, true label, Model 10 prediction and confidence, and predictions from the dictionary-based model, bag-of-words model, fastText model, and Google Perspective API. The examples illustrate that Model 10 often identifies contextual toxicity with high confidence, while baseline models produce inconsistent classifications.

To enhance interpretability of the models’ classification decisions, I employed the integrated gradients (IG) algorithm to calculate attribution scores for each token when classifying the input tweet as toxic or not (Sundararajan et al., Reference Sundararajan, Taly and Yan2017). The IG method quantifies how much each input token contributes to the model’s prediction by computing the gradients of the output with respect to the token embeddings, and then integrating these gradients along a path from a neutral baseline input (e.g., an empty or zero vector) to the actual input. Tokens with higher positive attribution scores increase the model’s confidence in the predicted class (e.g., toxicity), whereas tokens with negative scores suppress that prediction.

Figure 10 provides a representative example comparing E1 and E2 both containing the word ‘yellow,’ showing how words such as ‘evil’ contribute positively to toxicity detection, while neutral words receive near-zero or negative attribution. Table A4 in Appendix 7 visualizes token-level attributions for additional examples. This interpretability approach allows for an intuitive understanding of how the model weighs contextual cues when making predictions—revealing that transformer-based models capture nuanced linguistic relationships that simpler approaches cannot. More comprehensive details regarding the implementation and interpretive value of integrated gradients in this context are provided in Appendix 7.

Figure 10.

Token-level attribution scores with integrated gradients algorithm.

Figure 10 Long description

Two-panel visualization of token-level attribution scores for two example sentences containing the word ‘yellow.’ Panel A shows a toxic sentence in which terms such as ‘evil’ and ‘yellow’ contribute to the model’s toxicity prediction. Panel B shows a non-toxic sentence about flowers, where the same word appears in a neutral context. The figure illustrates how the model uses surrounding context to assign different meanings to the same word.

4.2.3. Proportion validity

I further analyze the aggregate classification outcomes by comparing the proportion of tweets labeled as toxic and non-toxic by human annotators versus the models. Since the human-annotated dataset is a random sample from the full dataset, it provides a strong approximation of the true proportion of toxic and non-toxic tweets. While the models align closely in ranking senators based on the volume of toxic tweets in their conversation threads, notable discrepancies emerge in the absolute counts and proportions. Figure 11 presents the proportion of toxic and non-toxic tweets classified across different models, with the dotted line indicating the proportions identified in the human-annotated random sample. Notably, Model 15 classifies more than twice the proportion of toxic tweets compared to the human benchmark, whereas Model 14 substantially underestimates toxicity. Model 10 yields the closest approximation to the human-annotated proportion, followed by Model 16.

Figure 11.

Bar-plots of Proportion of Toxic and Non-Toxic Tweets Labeled by Different Models.

Figure 11 Long description

Two bar plots comparing the proportions of tweets labeled toxic and non-toxic by Models 10, 14, 15, and 16. A red dashed line marks the human-annotated benchmark of 41.2 percent toxic tweets and 58.8 percent non-toxic tweets. Model 10 most closely matches the human benchmark, Model 14 substantially underestimates toxicity, and Model 15 substantially overestimates toxicity.

These discrepancies persist when disaggregating results by gender and party affiliation. As shown in Figure 12, Model 15 continues to classify a substantially higher proportion of tweets as toxic, whereas Model 14 reports significantly lower proportions compared to the human-annotated random sample. Among the baseline models, Model 16 most closely aligns with Model 10 and the human benchmark in overall classification proportions, where others demonstrate considerable misalignment. This analysis highlights Model 10’s superior performance in maintaining both contextual sensitivity and reliability in toxicity detection.

Figure 12.

Proportion of toxic tweets labeled by different models.

Figure 12 Long description

Two-panel bar plot showing the proportion of tweets labeled toxic by Models 10, 14, 15, and 16, disaggregated by senator gender and party. A red dashed line marks the 41.2 percent human-annotated benchmark. Across both panels, Model 15 labels a very high proportion of tweets as toxic, Model 14 labels a much lower proportion as toxic, and Model 10 remains closest to the human benchmark while preserving subgroup differences.

4.2.4. Predictive validity

To demonstrate the potential usefulness of the PFP in political science and check predictive validity of the toxicity measure, I analyze how Twitter user-base responded to tweets from Senators of the ${116^{th}}$ U.S. Congress and explore whether factors correlate to toxicity within conversation threads in theoretically expected directions.

In recent years, the U.S. political landscape has become increasingly polarized, leading to a heightened level of incivility in political discourse, especially in online spaces (Druckman et al., Reference Druckman, Peterson and Slothuus2013; Iyengar and Westwood, Reference Iyengar and Westwood2015; Rogowski and Sutherland, Reference Rogowski and Sutherland2016; Abramowitz and Webster, Reference Abramowitz and Webster2018; Iyengar et al., Reference Iyengar, Lelkes, Levendusky, Malhotra and Westwood2019). Research on partisan polarization suggests that political divisions between Republicans and Democrats have intensified, with individuals becoming more entrenched in their ideological camps (Iyengar and Westwood, Reference Iyengar and Westwood2015). Negative partisanship has fueled hostile rhetoric, particularly against high-profile political figures (Abramowitz and Webster, Reference Abramowitz and Webster2018). When politicians post toxic content, it can provoke out-partisan responses that mirror or escalate such toxicity. Social media platforms further amplify negative content through engagement-driven algorithms, where posts expressing toxicity or hostility are often more widely circulated (Steve et al., Reference Steve, Van Bavel and van der Linden2021; Huszár et al., Reference Huszár, Ktena, O’brien, Belli, Schlaikjer and Hardt2022). As a result, some scholars have argued that toxic tweets by politicians are more likely to generate toxic responses and be displayed widely, creating a self-reinforcing cycle where toxicity is both rewarded and amplified (Munger, Reference Munger2017; Steve et al., Reference Steve, Van Bavel and van der Linden2021; Huszár et al., Reference Huszár, Ktena, O’brien, Belli, Schlaikjer and Hardt2022). Therefore, I propose the following hypothesis:

To validate this theory, each tweet is classified as either containing toxic content ( $ = 1$) or not ( $ = 0$). Using these classification results, I calculate a Toxicity score for each parent tweet, representing the proportion of toxic tweets within the conversation threads following that tweet. The Toxicity score ranges from 0 to 1, where 0 indicates the absence of toxic tweets in the threads and 1 indicates that all tweets in the threads are toxic. The main explanatory variable ParentToxicity measures whether the parent tweet is toxic or not, where 1 indicates toxicity and 0 otherwise. To test this hypothesis, I additionally control for whether the parent tweet mentions the two presidential candidates (Donald Trump or Joe Biden), specific policy issues: (economy, COVID, abortion, crime or climate), and whether the parent tweet references other Senators from either the same or opposing party.Footnote ³⁸ I also control for Senator-specific fixed effects.

To assess the effect of toxic parent tweets on toxicity levels in subsequent conversation threads, I implement a fixed effects model. For each Senator $i$:

(2)\begin{equation}\begin{gathered} Toxicit{y_i}{ = \,{\alpha _i}}\, + \,{\beta _1}\,ParentToxicit{y_i} \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{+ \,{\beta _2}} MentionTrum{p_i}\, + \,{\beta _3}MentionBide{n_i}\, + \,\,{\beta _4}MentionEconom{y_i} \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{+ {\beta _5}MentionCovi{d_i}} + \,{\beta _6}MentionAbortio{n_i}\, + \,{\beta _7}MentionCrim{e_i} \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{{ + \beta }_8}MentionClimat{e_i}\, + \,{\beta _9}MentionCopartisa{n_i}\, + \,{\beta _{10}}MentionOutpartisa{n_i} \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{ + { \in _i}} \hfill \\ \end{gathered} \end{equation}

with Senator-specific intercept ${\alpha _i}$ and error term ${ \in _i}$. The regression analysis was implemented in R using the plm package.

The results from Model 10, summarized in Table 3, show a positive and statistically significant coefficient for the main explanatory variable ParentToxicity at 0.0767, suggesting that, for a given Senator, posting toxic tweets correlates with increased probability of toxic content for each subsequent tweet in the conversation threads. Importantly, while the direction of this relationship is consistent across models, the magnitude varies substantially. Model 15, for instance, reports a considerably higher coefficient of 0.3002, leading to massive differences in interpretation using BOW. Such high coefficient could be attributed to its systematic over-estimation of toxicity, as evidenced in Section 4.2.3. Conversely, classification results from Model 14 produced a lower coefficient, which may reflect its observed tendency to under-count toxicity levels.

Table 3.

Results from fixed effects model with senator-clustered standard errors

Table 3 Long description

Regression table showing fixed effects models predicting toxicity in conversation threads using four different toxicity measures. The key predictor, ParentToxicity, is positive and statistically significant for Model 10, the dictionary-based model, and the bag-of-words model, but not statistically significant for the fastText model. The table reports coefficients, senator-clustered standard errors, observations, R-squared values, adjusted R-squared values, and F statistics for each model.

Note: Full regression table including other control variables is provided in Appendix 8. Additionally, parent tweets that mention Trump, the economy, and COVID are associated with increased toxicity in conversation threads, whereas tweets mentioning co-partisan Senators correspond with lower toxicity levels in the conversation threads.

Despite the closest alignment of Model 16 to Model 10 on an aggregate level shown in Section 4.2.3, Model 16 fails to capture a statistically significant relationship between ParentToxicity and toxicity in subsequent threads. For further details, Table A5 in Appendix 8 provides a complete table of regression coefficients. Additionally, parent tweets that mention Trump, the economy, and COVID are also associated with increased toxicity in conversation threads, whereas tweets mentioning co-partisan Senators correspond with lower toxicity levels in the conversation threads. Collectively, these findings underscore the robustness and reliability of PFP models in measuring toxicity and capturing meaningful relationships in downstream analysis, particularly for nuanced patterns in toxic language within political discourse.