Do politicians keep their promises?

Can artificial intelligence be used to check whether political parties deliver on their campaign promises? A team of researchers from the Valencian Research Institute for Artificial Intelligence (VRAIN) at the Universitat Politècnica de València (UPV) has developed a tool capable of comparing the commitments set out in election manifestos with the initiatives, speeches and actions of political representatives that occur later.

The study, entitled Following the Promise: Retrieval-Augmented Analysis of Parliamentary Action, proposes a new application of artificial intelligence to the analysis of institutional activity. It aims to simplify a traditionally complex task: identifying politicians' promises, linking them to their parliamentary actions, and determining the extent to which the two are consistent.

The research is led by Amalia Pons and Antoni Mestre, researchers at UPV-VRAIN, with the participation of Vicente Botti, Manoli Albert and Vicente Pelechano. The team outlines how artificial intelligence can serve as a support tool for journalists, researchers, parliamentary advisers, and citizens interested in learning what happens to electoral commitments after elections.

Artificial intelligence that compares promises and actions

Unlike conventional AI systems, which generate responses based on information learnt during their training, the tool developed by the Valencian researchers directly consults the election manifestos to identify the pledges related to each parliamentary action. Based on this information, the system analyses the objective pursued by the initiative, its scope and the mechanisms used to carry it out.

The tool can identify actions that are consistent with electoral pledges, partially consistent or contradictory, and recognise cases where there is insufficient evidence to draw a conclusion. Furthermore, it does not merely provide a classification: it identifies the specific passages in the electoral manifesto that underpin its analysis and offers an explanation that can be reviewed and verified.

"It is not a question of artificial intelligence deciding whether a politician keeps or breaks their promises, but rather of providing a tool capable of identifying election pledges, comparing them with documented facts and explaining any possible contradictions. Political judgement must remain the responsibility of people," explains Amalia Pons.

Promising results in initial tests

To assess how the system works, the researchers analysed various cases of parliamentary activity. They enlisted the participation of five MPs and two political advisers, who evaluated both the tool's conclusions and the relevance of the documentary evidence used.

The results show that artificial intelligence successfully identified consistency between parliamentary actions and election pledges in nearly four out of five cases, significantly improving on the results obtained using conventional artificial intelligence systems.

The research also examined one of the main problems with these technologies: the generation of claims unsupported by the sources consulted. The system developed by the Valencian team significantly reduced this type of error thanks to a design that requires its responses to be based on verifiable electoral documents.

Participants particularly appreciated the tool's ability to justify its conclusions and present complex information in an understandable way.

"One of the main challenges in analysing political consistency is that promises are not always specific commitments. In many cases, they are general statements, long-term objectives or proposals subject to conditions. That is why we have designed a system that can also recognise when the available information does not allow a conclusion to be reached," says Antoni Mestre.

The researchers emphasise that these are preliminary results that will need to be tested with a larger number of cases and across different parliamentary contexts.

From speech analysis to accountability

The research opens new possibilities for applying artificial intelligence to monitoring political commitments and analysing large volumes of institutional documentation.

The tool enables the study of parliamentary initiatives, amendments, contributions to debates and voting records, linking them to the corresponding election manifestos. It also incorporates features to explore a party's commitments by topic, explain parliamentary documents in simpler language and compare election manifestos from different election cycles.

These capabilities could facilitate the work of the media and organisations dedicated to transparency, by enabling them to identify relevant commitments and locate the sources needed to verify specific political actions.

"Parliaments generate an enormous amount of public information, but the fact that this information is available does not necessarily mean it is accessible to citizens. Artificial intelligence can help us transform scattered documents into information that is understandable, verifiable and useful for democratic debate," adds Mestre.

The work forms part of research conducted at UPV-VRAIN on artificial intelligence, natural language processing, and intelligent systems applied to the analysis of complex information, and has received support from research projects funded by the Valencian Regional Government and the Spanish Government.