An independent study reference written by Dr Phuc V. Nguyen. It is not official subject material — for assessment requirements always follow your subject outline and vUWS.
Why ethics belongs in analytics
Analytics converts records about people into decisions about people. Legality and ethics are different tests, because law is a floor written after the fact and it lags what the technology can do. The recurring failure is not malice but a defensible-looking design choice whose harm nobody costed: a proxy variable, an averaging assumption, a default that opts people in. An ethical dilemma is the moment two duties pull against each other, accuracy against privacy, personalisation against autonomy, speed against contestability. Recognising the dilemma while the design is still cheap to change is the skill.
Why it matters
Every model quietly answers a question nobody wrote down. Who bears the cost when it is wrong? A model that is right 95 per cent of the time is wrong about one person in twenty, and those people are real. Ethics is not a lecture bolted on the end of a project. It is the habit of asking, before you build, who is in the data, who is affected by the output, and who is allowed to argue back.
A team building a hardship-detection model finds a variable that lifts accuracy sharply: whether the customer changed their mobile number in the past year. What is the strongest reason to pause?
Worked examples
A government agency wants to detect welfare overpayment at scale by comparing what people reported fortnightly with the annual income their employers reported to the tax office.
Australia's Robodebt scheme averaged a person's annual income evenly across the year and raised a debt wherever that average exceeded what had been declared for a fortnight. The averaging assumption is false for anyone with casual, seasonal or irregular work, so the arithmetic manufactured debts that did not exist. The design then shifted the burden of disproof onto the individual, who had to find old payslips to rebut a computer. The Royal Commission found the scheme unlawful. No step in it required a bad actor. The whole failure sat inside one assumption a modeller could have flagged on the first day.
A platform wants to know whether the emotional tone of a feed changes what people post, so it alters the ranking for a large sample of accounts without telling them.
Kramer, Guillory and Hancock did this with roughly 689,000 accounts and published the result in 2014. The finding was real and the terms of service arguably covered it, so the study was probably lawful. It was still contested, because the participants had no meaningful chance to decline, some of them were plausibly vulnerable, and the journal ended up publishing an editorial expression of concern. The dilemma is genuine rather than a mistake: experimental validity requires subjects who do not know they are being studied, and respect for persons requires the opposite. Someone has to choose, in the open.
Common mistakes
- ✗If it is legal it is ethical. Law is a floor set after the fact and it lags what analytics can do. Plenty of harmful designs are lawful, and some later turn out not to have been lawful at all, which is an expensive thing to discover after deployment.
- ✗Ethics is somebody else's job, usually legal or compliance. A compliance review checks a finished system against existing rules. The choices that create the harm, which variable stands in for what, which setting is on by default, which errors are tolerable, were made by the analyst long before that review.
- ✗Ethical problems come from bad people. Most documented failures came from ordinary teams making locally reasonable choices under a deadline. The mechanism is a proxy, an assumption or a default, not a villain, which is why looking for villains finds nothing.
- ✗An accurate model is an ethical model. Accuracy is an average over a population. It says nothing about who absorbs the errors, whether the target variable measures what you claim it does, or whether the person on the receiving end can contest the result.
Revision bullets
- •Legality is a floor; ethics asks whether a lawful use is defensible
- •A dilemma is two duties in conflict, not a choice between good and bad
- •Harm usually enters through a proxy, an assumption or a default setting
- •Accuracy is an average and says nothing about who absorbs the errors
- •Robodebt: annual income averaged into fortnights, burden of proof reversed
- •The cheapest place to fix an ethical problem is before the build starts
Quick check
A team building a hardship-detection model finds a variable that lifts accuracy sharply: whether the customer changed their mobile number in the past year. What is the strongest reason to pause?
Which of these is an ethical dilemma rather than simply a bad decision?
Connected topics
More in Ethics and Governance
Sources
- Robodebt Royal Commission (2023)Royal Commission into the Robodebt Scheme, Report. Commonwealth of Australia, 2023.Documents how an averaging assumption inside an automated income-matching process produced unlawful debts and reversed the burden of proof.
- Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. "Experimental evidence of massive-scale emotional contagion through social networks." Proceedings of the National Academy of Sciences, 111(24), 8788-8790, 2014.The emotional contagion experiment; the paper later carried an editorial expression of concern about informed consent.
- Nissenbaum (2010)Nissenbaum, H. Privacy in Context: Technology, Policy, and the Integrity of Social Life. Stanford University Press, 2010.Source of contextual integrity, the idea that an information flow is judged against the norms of the context the information came from.
- Australia's AI Ethics PrinciplesDepartment of Industry, Science and Resources (Australian Government). Australia's AI Ethics Principles, part of Australia's Artificial Intelligence Ethics Framework, 2019.Eight voluntary principles including fairness, transparency and explainability, contestability and accountability.