Gender and AI: Promise and Peril

Artificial Intelligence is intrinsically gender and racially biased but has the potential to promote fairness if its development is interdisciplinary, democratic, and inclusive. This was the theme of a recent two-day conference that gathered leading computer scientists, engineers, and ethicists at Harvard Radcliffe Institute—and left the audience with more hope than fear.

But panelists warned that fairness cannot be an afterthought and needs to be built into AI development and the datasets that feed AI from the start, otherwise it will replicate and amplify the same biases embedded in today’s historical data. The conference on Friday, April 11, showcased a range of projects aiming to do just that—from a mammography-based deep learning model to an algorithm designed to strip race-related signals from police reports.

Cosponsored by Harvard Radcliffe Institute and the Harvard Kennedy School Women and Public Policy Program, the Gender and AI conference kicked off the night before with a discussion on the intersection of art, artificial intelligence, and ethics featuring the artists Stephanie Dinkins and Anna Ridler.

Dinkins spoke about her process and what inspired her to create one of her most well-known projects, an interactive chatbot designed to convey the multigenerational story of her Black American family. She decided to pursue it after realizing that Bina48—a humanoid robot modeled after a Black woman—did not really represent blackness. “Because she (Bina48) was primarily created by white men ... her representation of blackness was, in my estimation, pretty flat,” Dinkins said, emphasizing the need for diversity in the development of artificial intelligence.

The process of designing an interactive chatbot that does represent blackness was complicated by the challenge of finding a language dataset that would not replicate harmful stereotypes surrounding Black people, with AI being only as good as the data we feed it.

“If we feed it very biased data, we are going to get very biased systems to come out,” said Fernanda Viégas, Sally Starling Seaver Professor at Harvard Radcliffe Institute and Gordon McKay Professor of Computer Science at the Harvard John A. Paulson School of Engineering and Applied Sciences.

Unable to find any unbiased data to feed its algorithm, Dinkins decided to create her own. Even though the dataset was “more loving and caring to the information that it was to hold,” she said, it still failed to capture the diverse perspectives and experiences of Black people around the world, as it was based primarily on her own viewpoint.

Dinkins’s struggles were revisited at the next session by computer scientists who have dedicated their careers to coding algorithms grounded in the principle of fairness while dealing with biased datasets that lack diversity.

“Why do we have to give you all of our data ... so you can represent us back to us, usually in an extremely biased and imperfect way?” said Catherine D’Ignazio, director of the Data + Feminism Lab at the Massachusetts Institute of Technology, referencing pushback from advocacy groups in developing countries in the southern hemisphere. Having different people at AI’s table is not only important for producing diverse algorithms but for making sure they are used in a way that benefits society, D’Ignazio said in a session titled “Inputs.”

D’Ignazio, who studies feminist approaches to technology, invited the audience to reflect on why there is not a healthier AI ecosystem in which all people can develop and run their own computing infrastructure that truly represents them. It’s important, she said, to organize resources in a way that gives groups full self-determination.

“What are the best use cases that people actually need right now to level the playing field?” she asked, adding that this type of thinking is absent in those large corporations developing and commodifying AI.

The next session, titled “Outputs,” featured five distinct AI projects that have tried to prioritize fairness and AI equity.

One of those projects was a mammography-based deep learning model developed by Regina Barzilay, a distinguished professor for AI and health at the MIT School of Engineering. “It is very important to develop healthcare tools that represent all of us and represent women,” Barzilay said.

Through her analysis, Barzilay found current FDA breast and lung cancer screening guidelines are inaccurate and desperately need to be changed. Her model analyzes 1.6 million mammograms from a diverse set of women across the US and around the world to close a lethal data gap in women’s health by more accurately predicting breast cancer risk through AI.

Current FDA guidelines require mammography providers to inform women with dense breast tissue that they may have an increased risk of breast cancer. Barzilay pointed out that around 40 percent of American women have dense breasts, yet the vast majority will not develop cancer, making density an unreliable predictor of the disease. “There are studies that, if [doctors] are looking at the image, some identify 8 percent as dense, and others found 80 percent,” said Barzilay.

These misleading guidelines result in women falling through the cracks and not being diagnosed on time. In all, Barzilay said doctors fail to identify more than 70 percent of women with breast cancer in early mammograms. Her mammography-based deep learning model offers more hope by using AI to study a larger audience and challenge outdated medical norms.

Barzilay is not alone. Frida Polli, a Harvard- and MIT-trained neuroscientist and CEO of the AI start-up Pymetrics, has developed a series of algorithms grounded in the principles of fairness and transparency, backed up by constant testing to help companies match talent with their ideal roles. Her work seeks to address biased hiring practices that result in only six women receiving callbacks for engineering jobs out of every 10 men with identical resumes.

“There is at least a decade of research showing that removing the bias from the human brain doesn’t work,” said Polli.

Although Polli and her team have not achieved perfect parity in hiring outcomes between men and women or among racial and ethnic groups, her models have sparked dramatic improvements. In cases where the previous callback ratio between Hispanic and white applicants was 8:10, her model increased it to 9:10.

But Polli’s desire to achieve fairness did not stop with computer science. She worked with lawyers and public policy experts who helped her advance a policy in New York City to require public disclosure of hiring algorithms’ impact ratios.

Sandra Wachter, Professor of Technology and Regulation at the University of Oxford, echoed the same collaborative and interdisciplinary spirit.

“Nobody could have developed that by themselves,” Wachter said in a later session referencing the artificial intelligence bias test she and her team developed. “This only worked because very different people were in the room, sometimes screaming at each other but coming out with ideas to find solutions.”

The development of AI poses real challenges, and solving them requires collaboration among ethicists, activists, engineers, and people from diverse backgrounds and perspectives.

But AI might also facilitate innovation to solve those problems, helping level the playing field by providing all people with “assistance to create whatever [their] vision is,” said Finale Doshi-Velez, the Herchel Smith Professor of Computer Science at the Harvard John A. Paulson School of Engineering and Applied Sciences. “We are in this moment when, perhaps, it is easier than ever to create stuff.”

The conference ended on a hopeful note with Megan Smith—current Walter Shorenstein Media and Democracy Fellow at Harvard Kennedy School and former US chief technology officer under Barack Obama—sharing a story from her time at Google that reflected the spirit of democratic collaboration when developing AI.

To address a lack of global data, she and her team launched a Wikipedia-like tool called MapMaker, which enabled people in countries like Pakistan to contribute directly, transforming blank maps into detailed maps developed not by machines but by the people who live there.

“We didn’t create these problems,” said Smith at the end of her presentation, “but let’s fix them.”

Click here to explore this article's original source for more.