In regression analysis, a dummy variable (also known as indicator variable or just dummy) is one that takes a binary value (0 or 1) to indicate the absence or presence of some categorical effect that may be expected to shift the outcome.[1] In machine learning this is known as one-hot encoding.
Dummy variables are commonly used in regression analysis to represent categorical variables that have more than two levels, such as education level or occupation. In this case, multiple dummy variables would be created to represent each level of the variable, and only one dummy variable would take on a value of 1 for each observation. Dummy variables are useful because they allow the use of categorical variables in our analysis, which would otherwise be difficult to include due to their non-numeric nature. .
Function and use
editA dummy variable is applied to a binary variable--for example, sex may be encoded as a dummy variable by listing females as 0 and men as 1.
As with any addition of variables to a model, the addition of dummy variables will increase the within-sample model fit (coefficient of determination), but at a cost of fewer degrees of freedom and loss of generality of the model (out of sample model fit). Too many dummy variables result in a model that does not provide any general conclusions.
Dummy variables are useful in various cases. For example, in econometric time series analysis, dummy variables may be used to indicate the occurrence of wars, or major strikes. It could thus be thought of as a Boolean, i.e., a truth value represented as the numerical value 0 or 1 (as is sometimes done in computer programming).
Dummy variables may be extended to more complex cases. For example, seasonal effects may be captured by creating dummy variables for each of the seasons: D1=1 if the observation is for summer, and equals zero otherwise; D2=1 if and only if autumn, otherwise equals zero; D3=1 if and only if winter, otherwise equals zero; and D4=1 if and only if spring, otherwise equals zero. In the panel data fixed effects estimator dummies are created for each of the units in cross-sectional data (e.g. firms or countries) or periods in a pooled time-series.
In such regressions, either the constant term or one of the dummies has to be removed making this the base category against which the others are assessed. This is because if dummy variables for all categories were included, their sum would equal 1 for all observations, which is identical to and hence perfectly correlated with the vector-of-ones variable whose coefficient is the constant term; if the vector-of-ones variable were also present, this would result in perfect multicollinearity,[2] so that the matrix inversion in the estimation algorithm would be impossible. This is referred to as the dummy variable trap.
See also
edit- Binary regression – Statistical estimation method
- Chow test – Mathematical test proposed by Gregory Chow
- Hypothesis testing – Method of statistical inference
- Indicator function – Mathematical function characterizing set membership
- Linear discriminant function – Method used in statistics, pattern recognition, and other fields
- Multicollinearity – Linear dependency situation in a regression model
- One-hot – Bit-vector representation where only one bit can be set at a time
References
editFurther reading
edit- Asteriou, Dimitrios; Hall, S. G. (2015). "Dummy Variables". Applied Econometrics (3rd ed.). London: Palgrave Macmillan. pp. 209–230. ISBN 978-1-137-41546-2.
- Kooyman, Marius A. (1976). Dummy Variables in Econometrics. Tilburg: Tilburg University Press. ISBN 90-237-2919-6.
External links
edit
- Maathuis, Marloes (2007). "Chapter 7: Dummy variable regression" (PDF). Stat 423: Applied Regression and Analysis of Variance. Archived from the original (PDF) on December 16, 2011.
- Fox, John (2010). "Dummy-Variable Regression" (PDF).
- Baker, Samuel L. (2006). "Dummy Variables" (PDF). Archived from the original (PDF) on March 1, 2006.