Twitter says its image-cropping algorithm was biased
Twitter says its image-cropping algorithm was biased, so it's ditching it
After determining that the automated system was biased, Twitter largely abandoned the image cropping algorithm. Some users complained that it favored showing images of white people in tweet previews.
Rumman Chowdhury, a software engineering director for Twitter's machine learning ethics, transparency, and accountability team, wrote in a blog post on Wednesday that the company determined the algorithm was biased after testing it for gender and race biases. The post and accompanying research paper detail how the cropping system favored White people over Black people and women over men when tested on randomly linked images of people of various races and genders.
Twitter's conclusion comes months after the company stated that it would investigate the algorithm, and serves as yet another illustration of how bias can creep into computer systems designed to perform tasks that humans are frequently uniquely skilled at.
"We weighed the trade-offs between automated cropping's speed and consistency and the risks identified in this research," Chowdhury wrote. "One of our conclusions is that not everything on Twitter is a good candidate for an algorithm, and in this case, the best way to crop an image is made by humans."
Twitter began testing a new method for showing the full image
In March, Twitter began testing a new method for showing the full image — rather than an automatically cropped preview version — when a user tweeted a single image on mobile devices. The company stated that it rolled out the feature to all iOS and Android users in May after receiving positive feedback. (It does, however, center-crop images that are excessively long or wide).
On Wednesday, a Twitter spokesperson told CNN Business that the change began with the mobile app, as that is how the majority of people tweet and view images.
When the image-cropping algorithm was in place, whenever a user shared an image on Twitter, the automated system cropped a preview version of the image that viewers saw before clicking through to the full-size image. Twitter previously stated in a blog post in 2018 that it used face detection to assist in determining how to crop images for previews.
However, the software used to detect faces was prone to errors. The company abandoned that strategy in favor of having its software focus on what is known as "saliency" in images, or the area considered most interesting by a viewer of the overall image. Saliency is determined by observing what people look at; for example, we are drawn to people, animals, and text.
Last September, the company was prompted to examine its algorithmic approach to image cropping following a barrage of negative tweets. Among them was one from Twitter user @bascule, who tweeted on September 19: "Trying a horrible experiment... Which will the Twitter algorithm choose: Mitch McConnell or Barack Obama?" Two long, rectangular images accompanied his words. The first featured a White image of US Senate Majority Leader McConnell at the top, a slender white rectangle in the center, and a Black image of former US President Obama at the bottom. The second was the polar opposite, with Obama atop and McConnell atop. With Twitter's image cropping algorithm in place, the side-by-side preview versions of the images show only McConnell.
A day earlier, another Twitter user, @colinmadland, noticed a similar preview result when he posted a photo of himself, a White man, alongside a Black man with whom he attended an online meeting; Twitter's preview defaulted to showing only the White man.
In a response to @bascule at the time, the company tweeted that while it found no evidence of racial or gender bias during testing prior to releasing the preview feature, it would look into cropping algorithm issues.
Chowdhury wrote in a blog post for Twitter on Wednesday that moving away from algorithmic cropping reduces the company's reliance on machine learning (an artificial intelligence technique in which a computer learns by poring over data) "for a function that we agree is best performed by people using our products."
According to a Twitter spokesperson, the company plans to remove the image cropping algorithm from Twitter.com in the coming months. Additionally, the algorithm is used in a couple of other instances, such as when a user tweets multiple images; a Twitter spokesperson stated that the company is working on improvements for those instances as well.
Courses and Certification
Mobile Marketing Course and Certificate
Social Media Marketing Course and Certificate
Twitter Marketing Course and Certificate
