Written by Lydia Clove, 11th grade student
Posted with permission.
The AllSides Bias Checker was launched in 2024. In alignment with the 7 Lenses for Evaluating AI for Civic Education we asked classrooms across the country to give us feedback on the AI-enhanced Bias Checker, because students should learn to be active analyzers of AI content, not just passive consumers. Students were asked to check an article in the Bias Checker and then answer the following questions:
- AllSides says “The AI-powered tool has been carefully calibrated to provide ratings and analysis that reflect the average judgment of all Americans across the political spectrum, not just a panel of insiders or biased AI.” Write down one way it lived up to this description and one way it didn’t.
- How is this tool useful in the classroom? How is this tool useful in your personal life?
Their feedback was sent to the AllSides CTO and Development team and this thoughtful response was shared with you. Since this essay was written in September AllSides has released subsequent upgrades to the Bias Checker to address the issues identified by this student. This essay was written by an 11th grade student at Star Valley High School as part of an assignment in a U.S. History class.
I checked several articles each from two news source curators to see how the tool would handle more neutral media. The first source was Tangle, and the second was 1440. Both make a point of handling often politically charged media with an independent or unbiased lens. Tangle, written primarily by Isaac Saul, focuses on showing how both ends of the political spectrum are reacting to a specific topic, and then gives an independent and nonpartisan take from Isaac or one of the editors, who all have moderate views and consider the arguments from both sides. It also includes a poll on the day’s subject for readers to share what they think about the subject, with the results included in the following day’s article. 1440, written by a team, focuses on only representing the facts without letting bias or motive twist or color them. It covers multiple news stories each day and has several repeating sections on topics such as sports, science and technology, and business and politics in each article.
I checked multiple articles from Tangle, and several came back with a left lean, several with a right lean, and several as neutral. Each rating seemed mostly on point, though I would have rated a few of them slightly more neutral. In the detailed analysis section, I noticed that several times, the bias checker was picking up on biases from quoted statements, which were included to show what partisan writers were saying. It also sometimes flagged statements from the author that were clearly opinions as misleading and improperly stated as fact. [For example:]
One of my fundamental personal beliefs is that everyone who can should exercise their right to vote — not by law, but by choice.
Opinion Statements Presented as Fact
The author presents their personal belief as a universal truth, which could be interpreted as an opinion statement presented as fact.
It is clear that this statement is a personal belief. It is explicitly written. It is not portrayed as a fact. The bias checker also put a list of objects on the list of details that show the bias of the article. I disagree with the analysis of the statement, because this is a simple list of facts - what items the second person to attempt to assassinate former president Trump had with them. If anything from that list was alarming, it was the rifle, not the tiles used as armor or the camera.
A rifle with a scope, two backpacks containing ceramic tile, and a GoPro camera were recovered from the bushes where Routh was said to be hiding.
Word Choice
The detailed description of the items found, especially 'ceramic tile' and 'GoPro camera,' could be interpreted as an attempt to paint a vivid and alarming picture of the suspect's intentions.
The list certainly wasn’t written in a way intended to cause alarm. It is meant to inform readers what the attempted assassin had with him. There is not a way to significantly alter the wording of this list.
I also checked multiple articles from 1440. They all had a neutral bias rating, which I agree with. 1440 simply lists the facts and lets you form your own opinions. However, the bias checker cited an advertisement as a bias in the detailed analysis.
We know that these days, it feels nearly impossible to find quality, affordable meat and seafood at the grocery store.
Opinion Statements Presented as Fact
The statement 'it feels nearly impossible to find quality, affordable meat and seafood at the grocery store' is subjective and presented as a fact, which could be interpreted as the author's opinion.
This was an ad. It wasn’t a political ad. It was a sponsored advertisement for an online grocery store. It seems like this is an area where the AI detection could be fine tuned and improved.
Allsides does have a disclaimer at the start of the detailed analysis section: “These may not always represent legitimate bias concerns,” but some of the items in the section just didn’t show biases. There were quite a few items in the detailed analysis section that did have a hint of bias, though. If I could add or suggest a feature for the tool, I would maybe add a rating for how biased the individual statements are in the detailed analysis.
I can easily see how it could be useful to check an article for bias and have a list of some of the statements that most clearly show that bias, especially in English or Social Studies classes. It can sometimes be difficult to tell what your own biases are, and the bias checker tool is useful in pointing out both how biased it is and picking out specific statements to support that claim.
As with all AI tools, it is important to not become reliant on it to think for you, but the bias checker is a good tool to show you where your own biases lie and help you know what you need to expose yourself to to break out of a bubble of self confirming media that just reinforces beliefs you already have, instead of challenging them and making you think.
