Bias in AI is systematic unfairness in a model's behavior, usually inherited from skewed training data or historical patterns and expressed as worse…
Understand AI bias and its main source: the data.
Bias in AI is systematic unfairness: a model that consistently produces worse or skewed outcomes for certain groups. It is rarely put there on purpose. Models learn patterns from data, so if the data reflects historical inequities, under-represents a group, or carries biased labels, the model absorbs and can amplify that skew.
A hiring model trained on past hires learns who was hired before, including any historical discrimination. The model is doing its job — reproducing patterns in the data — which is exactly the problem when those patterns are unfair.
Recognize where bias enters and the harms it causes.
Bias creeps in at several stages. Historical bias: the world the data came from was already unequal. Representation bias: some groups are under-sampled, so the model performs worse for them. Measurement and label bias: the features or labels themselves are skewed or proxy for a protected attribute. Even deployment can introduce bias if a model is used on a population unlike its training data.
Harms fall into two broad kinds. Allocative harm is unfair distribution of opportunities or resources — a loan or job denied at higher rates to one group. Representational harm is demeaning or stereotyping representations — a model that associates certain roles or traits with a gender or ethnicity. Both matter, and a system can cause one without the other.
See why there is no single mathematical notion of fair.
Fairness can be formalized several ways, and they don't all agree. Demographic parity asks for positive outcomes at equal rates across groups. Equal opportunity asks for equal true-positive rates among those who actually qualify. Individual fairness asks that similar individuals be treated similarly.
A well-known result is that some of these definitions are mathematically impossible to satisfy at once when base rates differ between groups. So you cannot be 'fair' in every sense simultaneously — you must choose the definition that fits the context and its stakes, and be explicit about the trade-off.
Detect bias with metrics and reduce it at the right stage.
You can't fix what you don't measure. Evaluate the model separately for each relevant group — accuracy, error rates, and your chosen fairness metric — to reveal disparities a single aggregate score hides. Then mitigate at one of three stages: pre-processing (rebalance or clean the data), in-processing (add fairness constraints during training), or post-processing (adjust thresholds or outputs per group).
Bias is not a one-time fix. Audit before release and monitor in production, since data drift and new populations can reintroduce it. For generative models, also test for stereotyping and biased content, not just classification disparities.
Watch for: judging fairness by overall accuracy (which masks per-group gaps); dropping the protected attribute and assuming the model is now fair (proxies remain); optimizing a fairness metric without asking if it fits the context; and treating a single audit as permanent. Fairness is a sociotechnical choice, not just a number to minimize — involve domain and affected-community perspectives.
AI bias is systematic unfairness learned from skewed, unrepresentative, or historically biased data, causing allocative harms (unfair resources) or representational harms (stereotyping). Fairness has several formal definitions — demographic parity, equal opportunity, individual fairness — that can be mathematically incompatible, so you must choose one that fits the context. Measure per group (not just overall accuracy), mitigate in pre-, in-, or post-processing, and audit continuously, treating fairness as a sociotechnical choice.
You are auditing a loan-approval model. Choose which fairness definition fits and justify it, describe how you would measure disparities across groups, and explain why simply removing the applicant's race from the features would not make it fair.
What is bias in AI, and where does it usually come from?
Models learn patterns from data, so biased or skewed data leads the model to reproduce and often amplify that unfairness.
What is the difference between allocative and representational harm?
Allocative harm denies resources unfairly (a loan, a job); representational harm reinforces stereotypes — a system can cause either or both.
Why can't a model satisfy every fairness definition at once?
It is provably impossible to meet certain fairness criteria simultaneously under differing base rates, so teams must choose a context-appropriate definition.
What is a common mistake when addressing bias?
Aggregate accuracy masks disparities and removing the attribute leaves proxies; fairness needs per-group measurement and ongoing auditing.