During a recent data science lecture, my colleague Jose paused the class with a thought-provoking question for our instructor, Austin Lasseter.
Looking at our evaluation metrics, Jose noted how closely the calculation for Mean Absolute Error (MAE) resembles Standard Deviation (SD). He asked: "If standard deviation is such a universal measure of variability, why don't we just use standard deviation instead of Mean Absolute Error in regression models?"
It was a fantastic question, one that bridges the gap between data science theory, mathematical optimization, and financial risk management.
Outliers and Data Distributions: The Core Difference
The primary difference between standard deviation and mean absolute deviation comes down to how each metric treats extreme values (outliers).
Standard Deviation (RMSE) ---> Squares errors ---> Heavily penalizes large outliers Mean Absolute Error (MAE) ---> Takes absolute errors ---> Treats all errors linearly
1. Sensitivity to Outliers
- Standard Deviation (and RMSE): Because the calculation squares the deviations before averaging them, larger errors have a disproportionately heavy impact on the final number. A single massive outlier will inflate the standard deviation significantly.
- Mean Absolute Deviation (and MAE): Because it takes the absolute value rather than squaring, every error is weighted proportionally to its size. A mistake of 10 units is simply twice as bad as a mistake of 5 units.
2. Mathematical Differentiability
In data science, standard-deviation-based metrics (like Mean Squared Error or RMSE) are smooth and continuously differentiable everywhere. This makes them mathematically ideal for optimization algorithms like Gradient Descent. MAE, on the other hand, creates a "sharp corner" at zero error, making pure calculus-based optimization slightly more complex.
Why Finance Prefers Standard Deviation
In portfolio management, financial analysts often operate under the assumption that asset returns follow a roughly normal distribution.
More importantly, in finance, outliers represent severe market risks (such as market crashes or sudden liquidity spikes). Investors want a metric that heavily penalizes extreme outliers because large losses can destabilize a portfolio. Standard deviation highlights these tail risks far more effectively than MAD does.
Choosing the Right Metric in Data Science
Unlike financial models that assume stable or normally distributed returns, real-world data science datasets rarely follow clean, symmetrical distributions. You do not always know the underlying distribution of the data you are modeling.
When building a regression model:
- Use Standard-Deviation-Based Metrics (MSE/RMSE) when your dataset is clean, normally distributed, or when large errors are extremely costly and need to be heavily penalized.
- Use MAE (Mean Absolute Deviation) when your dataset contains heavy outliers or noise that you don't want dominating your model's evaluation.
Jose’s question highlighted a critical lesson: choosing an evaluation metric isn't just about applying a standard formula—it's about understanding the underlying nature of your data and the real-world cost of errors.