this post was submitted on 15 Aug 2024
129 points (97.1% liked)

Data is Beautiful

2517 readers
15 users here now

Be respectful

founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] CanadaPlus@lemmy.sdf.org 2 points 11 months ago* (last edited 11 months ago)

In introductory stuff they just say discard outliers by eyeball, but obviously that's not very rigorous. You can do mathier versions, but it can be considered cooking the data if it changes your conclusion, and can get you in real trouble if you don't announce it.

I wasn't even talking about outliers in the data-breaking sense there, though. I really just meant that there must have been a lot of variation because that's not as big a difference as I was expecting, based on old stories. There's a few standard ways to measure that, actually an infinite series, but the two you often focus on for "outliers" in this sense are variance (literally just called that) and kurtosis. The higher order ones become increasingly nitpicky.