Eran Raviv - Page 15 of 16 - Modern statistics and econometrics with applications to financial markets

Better summary function in R

CodePosted on 06/10/2012

The summary function in R returns:


summary(x)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
   9.14   10.70   11.10   11.30   12.10   13.60

summary(x)

Min. 1st Qu. Median Mean 3rd Qu. Max.

9.14 10.70 11.10 11.30 12.10 13.60

For the univariate case I wrote what I consider to be a better summary function which returns:


usum(x) # For univariate Summary
Summary Statistics:
   min   med  mean   max    sd   skew    kurt
1 9.14 11.13 11.35 13.65 1.057 0.3028 -0.6389
------
NA's?:
No NA's in the series
------
Head                   Tail
13.65 13.55            13.08 13.13
------
Length           Class
207            numeric

usum(x) # For univariate Summary

Summary Statistics:

min med mean max sd skew kurt

1 9.14 11.13 11.35 13.65 1.057 0.3028 -0.6389

------

NA's?:

No NA's in the series

------

Head Tail

13.65 13.55 13.08 13.13

------

Length Class

207 numeric

Random books

Finance and Trading, MiscellaneousPosted on 06/03/2012

It seems like a very long while since my bachelor. Checking my bookshelf the other day I was thinking to flag some of those books which helped or inspired me along the way. Here they are in no particular order.

Forecasting the Eurozone Misery index

Blog, Finance and Trading, R, Statistics and EconometricsPosted on 05/23/2012

Is Miss Stagflation coming to visit?
The Misery index is the sum of inflation and unemployment rate. We would like them both to stay naturally low, and we are miserable when they are not. The index is currently floating in it’s record scratching levels. In this post I demonstrate the use of the nice FitAR package in R to fit an AR model and see what we can expect accordingly. Inflation and unemployment numbers concerning the Eurozone (17 countries) can be found here.
Have a look at the index over time:

Stock market Kurtosis over time

Finance and Trading, Risk, Statistics and EconometricsPosted on 05/12/2012

In the last decade we have observed an increase in computational power, information availability, speed of execution and stock market competition in general. One might think that, as a result, we are prone to larger shocks that occur faster than what was common in the past. I crunched some numbers and was surprised to see that this is not the case.

Kurtosis Interpretation

Blog, R, Statistics and EconometricsPosted on 05/07/2012

When you google “Kurtosis”, you encounter many formulas to help you calculate it, talk about how this measure is used to evaluate the “peakedness” of your data, maybe some other measures to help you do so, maybe all of a sudden a side step towards Skewness, and how both Skewness and Kurtosis are higher moments of the distribution. This is all very true, but maybe you just want to understand what does Kurtosis mean and how to interpret this measure. Similarly to the way you interpret standard deviation (the average distance from the average). Here I take a shot at giving a more intuitive interpretation.

Marriage is good for your income

Blog, Miscellaneous, R, Statistics and EconometricsPosted on 04/29/2012

For those of you who are into machine learning, here you can find a cool collection of databases to play around with your favorite algorithm. I choose one out of the available 200 and fit a logistic regression model. The idea is to see what kind of properties are common for those who earn above 50K a year. Our data is such that the “y” variable is binary. A value of 1 is given if the individual earns above 50K and 0 if below. We know many things about the individual. Level of education in years, age, is she married, where from, which sector is she working in, how many working hours per week, race, and more. We can fit logistic regression, which is quite standard for a binary dependent variable, and see which variables are important.

Backtesting trading strategies with R

Blog, Finance and Trading, RPosted on 04/21/2012

Few weeks back I gave a talk about Backtesting trading strategies with R, got a few requests for the slides so here they are:

Most profitable hedge fund style

Blog, Finance and Trading, RPosted on 04/21/2012

This is not an investment advice!!

Couple of weeks back, during amst-R-dam user group talk on backtesting trading strategies using R, I mentioned the most effective style for hedge funds is relative value statistical arbitrage, I read it somewhere. After the talk was over, I was not sure anymore if it was correct to say it and decided to check it.

Bootstrap example

Blog, R, Statistics and EconometricsPosted on 03/30/2012

Bootstrap your way into robust inference. Wow, that was fun to write..

Introduction
Say you made a simple regression, now you have your $\widehat{\beta}$ . You wish to know if it is significantly different from (say) zero. In general, people look at the statistic or p.value reported by their software of choice, (heRe). Thing is, this p.value calculation relies on the distribution of your dependent variable. Your software assumes normal distribution if not told differently, how so? for example, the (95%) confidence interval is $\widehat{\beta} \pm 1.96 \times sd( \widehat{\beta})$ , the 1.96 comes from the normal distribution.
It is advisable not to do that, the beauty in bootstrapping* is that it is distribution untroubled, it’s valid for dependent which is Gaussian, Cauchy, or whatever. You can defend yourself against misspecification, and\or use the tool for inference when the underlying distribution is unknown.

Europe most dangerous cities

Blog, RPosted on 03/15/2012

When I was searching for data about U.S prison population, for another post, I ran across eurostat, a nice source for data to play around with. I pooled some numbers, specifically homicides recorded by the police. A panel data for 36 cities over time, from 2000 to 2009. Lets see which are the cities that have problems in this area.

U.S. prison population

Blog, MiscellaneousPosted on 03/13/2012

I recently finished reading: [asa mytpl]0312343493[/asa]The author writes that 6.6% of U.S. American residents will find themselves at some point in their life incarcerated, about 20 million people. A big number on anyone’s scale. You can also find disturbing figures in Wikipedia: figure. Are these facts misleading? we need to account for population growth. The prison population should naturally rise even if the proportion of crime in the general population is constant, since the population itself is growing. Here I show that these facts are NOT misleading, and that the system is indeed not fulfilling its purpose.

Spurious Regression Illustrated

Blog, Statistics and EconometricsPosted on 03/04/2012

Spurious Regression problem dates back to Yule (1926): “Why Do We Sometimes Get Nonsense Correlations between Time-series?”. Lets see what is the problem, and how can we fix it. I am using Morgan Stanley (MS) symbol for illustration, pre-crisis time span. Take a look at the following figure, generated from the regression of MS on the S&P, actual prices of the stock, actual prices of the S&P, when we use actual prices we term it regression in levels, as in price levels, as oppose to log transformed or returns.

Live Rolling Correlation Plot

Blog, Finance and TradingPosted on 02/19/2012

Open source is amazing! I cannot even start to imagine the amount of work invested in R, in firefox browser (Mozilla), or Rstudio IDE, all of which are used extensively around the globe, free. Not free as in: free sample till you decide to upgrade, or: sure it’s free, just watch this one minute commercial every time you need to use it, but free, as in: we think it might make your life better, enjoy. Warms the heart, in direct opposite to the fabulous fabs out there, that instead of contributing to a better, safer society, set it back and get paid for it (see appendix). Character is also normally distributed I guess.

Piecewise Regression

Blog, R, Statistics and EconometricsPosted on 02/11/2012

A beta of a stock generally means its relation with the market, how many percent move we should expect from the stock when the market moves one percent.

Market, being a somewhat vague notion is approximated here, as usual, using the S&P 500. This aforementioned relation (henceforth, beta) is detrimental to many aspects of trading and risk management. It is already well established that volatility has different dynamics for rising markets and for declining market. Recently, I read few papers that suggest the same holds true for beta, specifically that the beta is not the same for rising markets and for declining markets. We anyway use regression for estimation of beta, so piecewise linear regression can fit right in for an investor/speculator who wishes to accommodate himself with this asymmetry.

Resistant Regression

Blog, R, Statistics and EconometricsPosted on 01/29/2012

It is a fact that on most days, not much is going on in the stock market. When we estimate the relation of a stock with the market, or the “beta” of a stock, we use all available daily returns. This might not be wise as some days are not really typical and contaminate our estimate. For example, Steve Jobs past away recently, AAPL moved quite a bit as a result. However, this is a distinct event that does not reflect on the relation with the market, but is company specific. Our aim is to exclude such observations, taking into consideration that we don’t want to lose too much information, not all large swings are irrelevant.