Data sources, preparation and feature engineering

While parliamentary voting data is increasingly available, it is often not readily accessible in a format that encourages genuine exploration. Existing sources are often limited to static presentations, require technical skills to analyze or focus on predefined metrics such as group loyalty. This motivated the development of the web app. The merged dataset used in this study combines roll-call voting records with metadata on MEPs, covering the period from 2004 to 2022. It draws on two publicly available sources: VoteWatch (Hix, Frantescu and Hagemann Reference Hix, Frantescu and Hagemann2022) and Parltrack (Parltrack Community 2025).

VoteWatch Europe, active from 2009 until its closure in 2022, provided structured records of plenary votes in the European Parliament. The platform made available vote-level data for each MEP, including decisions categorized as Yes, No, Abstain, and different forms of Not Present. This dataset spans four legislative terms and comprises over 36,000 roll-call votes.

Parltrack supplements this with biographical and institutional metadata, including nationality, gender, political group affiliation, committee memberships, and parliamentary activity such as authored reports and motions. Combining these sources allows researchers to get a more detailed insight into legislative life.

To harmonize the data, all voting records were merged together in a dataset, where each observation represents the voting decision of a specific MEP in a specific roll-call. Group names were standardized to ensure comparability across legislative periods. The same applies to inconsistencies in metadata, for example, variations in national party labels or political group codes. Missing values occur mainly for biographical attributes. Values in variables such as place and date of birth were added manually where possible. Committee activity and authored reports, retrieved through the Parltrack API, were retained even where some entries were missing, as the absence of recorded activity often reflects actual parliamentary behavior rather than data gaps. Due to turnover and resignations, the number of entries per term exceeds the number of official seats. Abstentions and non-participation were recorded as distinct categories, preserving the difference between deliberate dissent and absence. No MEPs were excluded at this stage.

In addition to the raw vote data, several variables were generated to support interpretation and the exploratory analysis. These include the following:

• • Attendance Index: proportion of roll-call votes attended by each MEP.

• • Loyalty Index: share of votes cast in line with the majority position of the MEP’s European Political Group (EPG).

• • Activity Index: calculated as the (weighted) average of an MEP’s recorded activity counts over all available activity dimensions, ignoring missing values:

(1) $A{I_i} = {{\sum\nolimits_{k = 1}^K {{w_k}\cdot{a_{ik}}} } \over {\sum\nolimits_{k = 1}^K {{w_k}\cdot1\left( {{a_{ik}} \ne {\rm{NA}}} \right)} }}$

where ${a_{ik}}$ is the raw count of activity $k$ for MEP $i$ , ${w_k}$ is the weight assigned to activity $k$ (defaulting to $1$ for all $k$ ), and the denominator sums only over non-missing entries.

• • Winning Index: frequency with which an MEP voted on the winning side in the plenary.

• • Domain-specific voting scores: every roll-call is tagged with an OEIL (European Legislative Observatory) policy label (eg economy, civil liberties, environment, foreign affairs). Votes are coded as $ +\! 1$ (yes), $ - \!1$ (no) or 0 (abstain/absent) and averaged within each domain, optionally weighted by vote salience. Vote salience here indicates whether the calculation includes all roll-calls or is restricted to final votes only. The resulting value (rescaled to $\left[ {0,1} \right]$ ) summarizes how consistently an MEP supports legislation in that policy area.

These indicators were calculated as illustrative examples and are not intended as definitive measures. Rather than prescribing a specific weighting or aggregation method, options inside the application allow users to explore and interpret these features flexibly. The full data processing workflow has been implemented in R, with all scripts and documentation available via GitHub.Footnote ¹

ParliamentLab application

ParliamentLab aims to offer low-threshold access to the underlying data in order to make it usable for a broader audience, especially for those interested in generating their own questions rather than testing existing ones.

The application has been developed in R using the Shiny framework (Chang, Cheng, Allaire et al. Reference Chang, Cheng, Allaire, Xie and McPherson2021; R Core Team 2024), which enables an intuitive user interface and dynamic visualizations based on user input. The app is fully self-contained and can be run locally or visited online via ParliamentLab.eu. The application runs directly in modern web browsers without further installations.

We deliberately avoided a fixed outcome and instead built a pipeline process where users can compare, interpret, and adjust freely. Thus, the application is not tied to a specific research question. Instead, it is intended as a tool to support exploratory approaches – for example, identifying unexpected coalitions, examining national deviations or looking into individual MEP behavior across legislative terms. Users make active choices about how data is handled: how strictly to filter it, whether to remove outliers, how to define specific features or metrics. A key aspect of the tool is showing the immediate consequences of these choices. Changing a filter setting or tweaking how a feature is calculated visibly alters the resulting graphs and tables.

While research questions long for objective answers, the process of finding them yields decisions made by the researcher. ParliamentLab is designed to demonstrate the impact of these decisions.

Functionality overview

The application mirrors the structure of a classical research process. Each tab represents a distinct step in the analysis pipeline, allowing users to follow a coherent path from data selection to interpretation. As a starting point, a legislative period has to be selected. This choice defines the scope of all subsequent data operations. The user can then narrow the dataset by applying their own rules regarding the handling of missing data or the identification of outliers. These selections are carried over to all views in the application.

In the Data Preparation and Feature Engineering tab (Figure A3), users are introduced to the structure of the dataset. Key variables such as attendance, loyalty, or voting alignment are precomputed and can be included or excluded from the analysis. Some features, like binned attendance categories or topic-specific indices, can also be adjusted manually. These steps reflect the kind of early-stage decisions that researchers often make before moving to statistical modeling. The user-specified pre-processed data set can be easily downloaded for subsequent use.

The Exploratory Analysis tab (Figure A4) provides tools for visualizing patterns and distributions. Users can examine feature values across political groups or countries, display selected variables as boxplots or histograms, and inspect profiles of individual MEPs. A special component is the interactive parliament plot (Figure 1), which shows MEPs in a semicircular layout inspired by the actual seating order. Clicking on a member reveals detailed biographical data and summary statistics, supporting focused comparisons or outlier detection.

Figure 1.

Interactive parliament plot in the Exploratory Analysis tab.

Note: MEPs are arranged in a semicircle reflecting the actual seating order of the European Parliament. Members aged between 30 and 55 are highlighted in color; all others are shown in grey. Clicking on a member reveals a tooltip with biographical data and summary statistics.

Finally, the Clustering tab (Figure A5) allows users to run dimensionality reduction techniques (eg UMAP (McInnes, Healy and Melville Reference McInnes, Healy and Melville2020) or W-NOMINATE (Poole and Rosenthal Reference Poole and Rosenthal2000)) and apply clustering algorithms such as k-means (Jain Reference Jain2010), PAM (Kaufman and Rousseeuw Reference Kaufman and Rousseeuw2009), or HDBSCAN (McInnes, Healy and Astels Reference McInnes, Healy and Astels2017). The outputs are shown as two- or three-dimensional scatterplots, where users can explore how the resulting data-driven grouping relates to the known party affiliation. Furthermore, multiple evaluation metrics, including the Silhouette Score (Rousseeuw Reference Rousseeuw1987), Davies–Bouldin Index (DBI) (Davies and Bouldin Reference Davies and Bouldin1979), and the Caliński–Harabasz Index (CHI) (Caliński and Harabasz Reference Caliński and Harabasz1974), are provided in a separate tab to assess clustering quality. Interactive visualization of the results of the clustering process can be found in the Results tab (Figure A6).

Example: unsupervised clustering of voting behavior

To illustrate one possible use case, we briefly highlight the clustering module of ParliamentLab. This feature allows users to group MEPs based on their voting patterns without relying on formal party affiliations or ideological labels. The resulting latent coalitions are referred to as ‘silent parties’: blocs of legislators who vote alike but rarely present themselves as a unified actor in public debate. The clusters are visualized in different ways, eg a radar chart on major policy areas (see Figure 3).

Figure 2.

Clustering workflow for the 6th European Parliament (2004–2009) based on all roll-call votes.

Figure 3.

Cluster analysis of MEP voting behavior.

Note: W-NOMINATE scores along the left–right axis, with MEPs color-coded by data-driven cluster (left). Radar plot showing the average policy positions of each cluster across six issue areas (right).

Users can choose the number of clusters and the algorithm (eg $k$ -means, PAM, or HDBSCAN), as well as the dimensionality reduction technique used for visualization (eg UMAP or W-NOMINATE). The resulting plot shows each MEP as a point in two-dimensional space, colored by cluster membership.

Consider the voting data from the 6th European Parliament (2004–2009) exemplarily: applying UMAP and HDBSCAN identified five well-separated clusters. These clusters partially align with official party groups but also highlight unexpected voting coalitions. For instance, members of the Green and Left factions form a single cohesive cluster (Cluster 1 in Figure 2(b)), suggesting overlapping preferences, particularly on environmental and civil rights issues. Meanwhile, a shared cluster of MEPs from the European People’s Party (EPP) and the eurosceptic IND/DEM group reveals unexpected alignment on key economic votes, echoing broader trends of cross-cutting issue alignments and ideological complexity in modern legislatures (Dalton Reference Dalton2018). Silhouette Scores and other evaluation metrics confirm that these clusters were moderately well-defined, with HDBSCAN outperforming $k$ -means in handling heterogeneous densities and detecting outliers.

In another example using data from the 9th Parliament (2019–2022), we identified seven distinct clusters, again reflecting the seven official European Political Groups. Notably, several liberal and Green MEPs were grouped with more conservative factions. By adjusting parameters like the minimum number of points per cluster or selecting only final votes, users can explore how voting alliances shift under different assumptions.

Beyond visualization, ParliamentLab lets users compare data-driven clusters with the official European Parliament groups, making it easy to study intra-party cohesion, cross-party alliances, and ideological drift. By expanding any cluster in the overview table, one can immediately spot MEPs whose voting behavior systematically diverges from their declared affiliations (see Figure 4).

Figure 4.

Interactive overview of all data-driven clusters.

Note: The table summarizes key metrics for each cluster (size, age, experience, loyalty, activity, etc.). Expanding a row reveals two widgets: distribution of official parliamentary groups within the cluster (left) and the five most active MEPs based on the activity index (right).

The clustering feature is not intended to produce definitive classifications but to offer a complementary view on parliamentary behavior. It allows users to experiment with alternative groupings and compare them to formal affiliations, potentially raising new questions about coalition-building. As with any unsupervised method, results should be interpreted with care: cluster structures depend on methodological choices such as the selected algorithm, parameter settings, and preprocessing steps. Rather than offering final answers, the feature is designed to support exploration and hypothesis generation, especially in political contexts where alliances are often fluid and situational.