this post was submitted on 15 Aug 2024
129 points (97.1% liked)
Data is Beautiful
2517 readers
15 users here now
Be respectful
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
In introductory stuff they just say discard outliers by eyeball, but obviously that's not very rigorous. You can do mathier versions, but it can be considered cooking the data if it changes your conclusion, and can get you in real trouble if you don't announce it.
I wasn't even talking about outliers in the data-breaking sense there, though. I really just meant that there must have been a lot of variation because that's not as big a difference as I was expecting, based on old stories. There's a few standard ways to measure that, actually an infinite series, but the two you often focus on for "outliers" in this sense are variance (literally just called that) and kurtosis. The higher order ones become increasingly nitpicky.