This is a short exposition of Clauset, A., Shalizi, C.R., Newman, M.E.J., 2009. Power-Law Distributions in Empirical Data. SIAM Rev. 51, 661–703. (arXiv). I studied this paper extensively for work I did on entropy analysis of diffusion processes and I believe it’s an important paper to study for anyone who has exposure to power laws. With power laws once again getting much attention it may be more important than ever.
The article was motivated by an influx of statistical physics papers in the 1990s and 2000s claiming to discover power laws all over the place. Often the articles would simply plot a histogram in log-log scale, observe a straight line, and declare it a power law. Power laws do appear in many varied places, but, as Clauset et al. demonstrate, there are other distributions that make straight lines when plotted on a log-log scale. For example, log-normally distributed data will also follow a straight line on a log-log plot. So will log-logistic distributions, certain Lévy distributions, and certain Weibull distributions, among others.
A straight line on a log-log plot isn’t sufficient to prove that data are power law distributed.
After seeing a straight line of a log-log histogram, researchers would then often perform a least-squares linear regression over the log-log plot to estimate the power law exponent. Clauset et al. explain that although this method dates back to the work of Pareto, Auerbach, and Zipf it has a severe pitfall: when applied on log-log scale data the errors of a least-squares linear regression are systematically biased and non-Gaussian.
Even when the data are power law distributed, least-squares fitting isn’t satisfactory for finding the scaling parameter.
It’s also common for researchers plotting log-log histograms to simply log the axes and do nothing with the bin widths, resulting in bin widths that vary rather than bin edges spaced evenly in log scale. The varying bin widths further compound the problem for least-squared regression, as table 3.1 in the article illustrates.
Clauset et al. take twenty-four instances of reported power laws and reanalyze the data. They provide a maximum likelihood estimator for robustly estimating a power law’s scaling parameter and fit other heavy-tailed distributions to each case for comparison. Some empirical power laws are confirmed but others are rejected for other distributions which turned out to be better fits.
The most important takeaways from the article are the points emphasized:
- A straight line on a log-log plot isn’t sufficient to prove that data are power law distributed.
- Even when the data are power law distributed, least-squares fitting isn’t satisfactory for finding the scaling parameter.
In their article, Clauset et al. explain how to rigorously confirm whether data really follow a power law and how to robustly estimate the scaling parameter if it does. Alstott, J., Bullmore, E., Plenz, D., 2014. powerlaw: A Python Package for Analysis of Heavy-Tailed Distributions. PLoS ONE 9, e85777. (arXiv) built on their work and published the powerlaw Python package, so the power law fitting and comparison methods established by Clauset et al. are easily accessible to everyone.
For more analysis of power laws, their properties, and some more empirical examples see also Newman, M.E.J., 2005. Power laws, Pareto distributions and Zipf’s law. Contemporary Physics 46, 323–351. (arXiv).