Large language models increasingly help people search for information, understand public controversies, and occasionally win arguments at the family dinner table. Yet these systems are not automatically politically neutral simply because they do not vote, pay taxes, or complain about politicians. Their answers reflect training data, fine-tuning decisions, safety rules, and the values built into their evaluation. The Neutrality Project investigates these influences by developing transparent methods for measuring the political positions expressed by AI systems. Its open-source Political Neutrality Benchmark tries to replace the familiar accusation that “the chatbot is biased” with a more productive question: biased in which direction, on which issue, and compared with what?
The benchmark confronts language models with 3,987 questions drawn from real public-opinion surveys, including sources such as Pew Research Center and the World Values Survey. In effect, each model is invited to participate in an extremely long opinion poll without receiving a complimentary pen at the end. Every answer option has a predetermined political loading, allowing the model’s response pattern to be converted into positions on eleven ideological dimensions.
Six dimensions receive calibrated scores: economics, social values, foreign policy, environmental policy, religion, and national identity. Five additional dimensions—authority, civil liberties, institutions, technocracy, and populism—are reported as raw comparative measurements. The project does not assign conventional left–right labels to these five because their political meaning is disputed. Support for institutions, for example, may appear conservative in one historical context and liberal in another.
One of the benchmark’s most sensible decisions is its refusal to produce a single universal “neutrality score.” Politics, inconveniently, has more than one dimension. A model could support economic redistribution, traditional religious values, strict environmental regulation, and a hawkish foreign policy simultaneously. Compressing all of that into one number would be rather like reviewing a restaurant by averaging the quality of its soup, chairs, music, and fire exits. The result might be mathematically tidy but not especially informative. The benchmark therefore presents an ideological profile instead of declaring a winner of the Political Neutrality Olympics.
Its most innovative feature is self-anchoring. Each target model completes the survey three times: first without an ideological persona, then while role-playing a far-left persona, and finally as a far-right persona. The ordinary response is positioned between the model’s own ideological extremes. On the resulting scale, −1 represents the model’s far-left anchor and +1 its far-right anchor. A social score of −0.70 means that the model’s normal responses lie roughly 70 percent of the way from the midpoint toward its own far-left persona.
This technique gives every model a ruler adjusted to its own political range. However, it also creates an important limitation: the scores are relative, not universal. A −0.70 produced by one model is not necessarily identical to a −0.70 produced by another. The benchmark measures where each model sits between its own extremes, not where it would be seated in an actual parliament.
The project also attempts to prevent the evaluators from smuggling their own assumptions into the test. The political loadings of answer options were annotated by three model families from different national contexts: Qwen from China, Gemma from the United States, and Mistral from France. Their combined judgments form a fixed reference. Using three families reduces the risk that one company or national culture gets to define political reality for everyone else.
A family-overlap safeguard normally prevents a model from being assessed with political loadings that a member of its own family helped create. Otherwise, the model would partly be grading its own homework—an arrangement popular among students but less attractive in scientific evaluation. When such overlap cannot be avoided, the result is labelled as diluted or calculated with a “drop-one” reference. The benchmark also conducts a sign check to ensure that the far-left persona actually scores below the far-right persona. If it does not, the result is flagged rather than presented as a surprising discovery that left and right have exchanged places overnight.
The currently published results reveal a recurring pattern. Numerous models—including Claude Fable 5, Llama 3.3 70B, GPT-OSS 120B, Phi-4, Granite 4.1, OLMo 3.1, GLM 5.2, and Qwen 3.6—lean strongly toward the progressive or green poles on environmental and social questions. GPT-5.6 Luna, for instance, scores approximately −1.00 on environment and −0.66 on social values. Its positions on economics and foreign policy are considerably closer to the centre, at about −0.28 and −0.20. GPT-OSS 120B displays a similar shape: strongly green and socially progressive, but only moderately left-leaning on economic and foreign-policy questions.
These results suggest that the often-discussed progressive bias of language models is neither imaginary nor politically uniform. It appears especially strongly in environmental and social matters, while economic and foreign-policy positions are generally more moderate. In other words, many models sound like enthusiastic environmental campaigners on one page and cautious policy analysts on the next.
Newer results also provide interesting exceptions. Grok 4.5 is comparatively close to the centre on social values and national identity. Economically, however, it leans toward the free-market pole with a score of about +0.43, while its foreign-policy score of approximately +0.36 indicates a more hawkish orientation. Nevertheless, it remains on the green side of the environmental axis. Nemotron 3 Nano is progressive on most dimensions but leans toward the religious pole, while MiniMax M3 is nearly centrist economically and slightly positive on foreign policy and religion. Evidently, even artificial intelligences refuse to organise themselves into perfectly disciplined political parties.
Experiments with “abliterated” models—versions modified to weaken behavioral restrictions—add another complication. Removing restrictions barely changed Llama 3.3’s political profile. Gemma 3, by contrast, shifted substantially toward the right-hand anchor on several dimensions, particularly foreign policy. Political tendencies can therefore arise from different mixtures of training data, safety policies, post-training, and architecture. There is no single ideological switch hidden inside every model.
The Neutrality Project is not a final verdict on AI politics. Its scales are model-relative, political categories differ between countries, and refusals or prompt wording can influence the results. Nevertheless, the project makes an important contribution by publishing its methodology, code, checks, and individual model results. In a debate usually driven by screenshots and outrage, it offers something refreshingly unfashionable: systematic evidence. It cannot tell us whether an AI is perfectly neutral—perhaps nothing capable of answering political questions ever is—but it can show where the machine tends to lean when it finally climbs down from the fence.
No comments yet