Posts

Showing posts from April, 2010

Fake data, part 2: The black swan distribution

Image
May 2017: This is from my other blog that's no longer online. The original comments are no longer available, but you are welcome to add more.In the previous part when I wrote about the logistic distribution, I observed that it fit the S&P 500 daily returns pretty well out to 5 standard deviations (much better than the normal distribution) but broke down in the tails beyond.That has been part of a weeks-long quest to find a distribution that fits, with the criterion that it must be closed-form; that is, we shouldn't need numerical methods to solve for the cumulative distribution or its inverse. Why do this? Because I want to generate an endless stream of artificial market data for analysis. Not only do I want my artificial market to exhibit behavior that is statistically similar a real-world market, but I also want closed-form expressions for ease of programming into Excel or whatever I'll use for analysis. The normal distribution fails on both counts.A normal distribut…

Fake data, part 1: The forgotten logistic distribution

Image
May 2017: This is from my other blog that's no longer online. The original comments are no longer available, but you are welcome to add more.Problem: We need to generate an endless stream of artificial market data, such that it maintains the statistical properties and behaviors of a real market. Everybody is familiar with the normal (Gaussian) distribution. The classic bell curve that underlies many tools in statistics. A normal distribution has a couple of glaring flaws, however:It requires numerical methods to calculate cumulative probability (area under the bell curve) and inverse of probability (i.e. for a known value of cumulative probability, what's the corresponding value of x?)It doesn't model many real-world situations well. Real-world distributions often have more outliers ("fatter tails" in the bell curve) than the normal distribution would suggest.Fortunately, there's a simple alternative: The logistic distribution. Like the normal distribution, i…