Covert racism in LLMs

The article discusses a new paper from Allen AI, Stanford, and others, which reveals shocking results about the covert racism embedded in Language Learning Models (LLMs). The study found that while these systems rarely display overt racism, they exhibit significant covert racism, particularly when prompted with African American English. The study also found that the scale of the LLMs exacerbates the covert racism, and that trivial, dialectical choices in wording drive much of this phenomenon.

The author argues that the widespread adoption of LLMs could have massive consequences, particularly as they are used in consequential matters such as housing decisions, loan decisions, and crime proceedings. The author calls for an investigation by Congress, the EEOC, HUD, the FTC, and other organizations, and suggests that LLM manufacturers should recall their systems until they can find an adequate solution to this issue. The author has also started a petition on change.org to address this issue.

Key takeaways

A new paper from Allen AI, Stanford and others reveals shocking results about covert racism in Language Learning Models (LLMs).
The study found that while the systems rarely show overt racism, they display a high level of covert racism, especially when given an African American English prompt.
The widespread adoption of LLMs could have massive potential consequences, particularly as scale makes the covert racism worse.
The author suggests that LLM companies should recall their systems until they can find an adequate solution to this problem, and encourages Congress and other bodies to investigate.

Covert racism in LLMs

Key takeaways

Discussion (0)