Hostname: page-component-74656f9884-j7cmm Total loading time: 0 Render date: 2026-11-09T10:57:07.262Z Has data issue: false hasContentIssue false
  • English
  • Français

IDENTIFICATION IN DISCRETE MARKOV DECISION MODELS

Published online by Cambridge University Press:  23 September 2014

Sorawoot Srisuma
Sorawoot Srisuma*
Affiliation:
University of Surrey
*
*Address correspondence to School of Economics, University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom; e-mail: s.srisuma@surrey.ac.uk.
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

We derive conditions for the identification of the structural parameters in Markov decision model under the assumptions of Rust (1987, Econometrica 55, 999–1033) when the payoff function is parametrically specified. Identification in this class of dynamic problems is difficult to establish since the parameters of interest enter the value function nonlinearly, and the value function is only defined implicitly as a fixed point of some functional equation. We show it is sufficient to verify identification in the pseudomodel, which is more tractable as it is originally designed to reduce the computational burden in the estimation problem, for the identification of the data generating parameter of the underling model. Our results extend naturally to a class of dynamic discrete action games commonly used in empirical industrial organizations.

Information

Type
ARTICLES
Information
Econometric Theory , Volume 31 , Issue 3 , June 2015 , pp. 521 - 538
Copyright
Copyright © Cambridge University Press 2014 

Footnotes

I thank the editor, a co-editor, and anonymous referees for valuable suggestions that have greatly improved the paper. I also thank Xiaohong Chen, Oliver Linton, and Hashem Pesaran for helpful comments.

References

REFERENCES

Aguirregabiria, V. & Mira, P. (2002) Swapping nested fixed point algorithm: A class of estimators for discrete Markov decision models. Econometrica 70, 1519–1543.CrossRefGoogle Scholar
Aguirregabiria, V. & Mira, P. (2007) Sequential estimation of dynamic discrete games. Econometrica 75, 1–53.CrossRefGoogle Scholar
Aguirregabiria, V. & Mira, P. (2010) Dynamic discrete choice structural models: A survey. Journal of Econometrics 156, 38–67.CrossRefGoogle Scholar
Amemiya, T. (1985) Advanced Econometrics. Harvard University Press.Google Scholar
Bajari, P., Benkard, C.L., & Levin, J. (2007) Estimating dynamic models of imperfect competition. Econometrica 75, 1331–1370.CrossRefGoogle Scholar
Bajari, P., Chernozhukov, V., Hong, H., & Nekipelov, D. (2012) Semiparametric Estimation of a Dynamic Game of Incomplete Information. Working paper, University of Minnesota.Google Scholar
Bajari, P., Hong, H., & Nekipelov, D. (2013) Econometrics for Game Theory. Advances in Economics and Econometrics: Theory and Applications, 10th World Congress.Google Scholar
Benkard, C.L. & Berry, S. (2006) On the nonparametric identification of nonlinear simultaneous equations models: Comment on Brown (1983) and Roehrig (1988). Econometrica 74, 1429–1440.CrossRefGoogle Scholar
Brown, B.W. (1983) The identification problem in systems nonlinear in the variables. Econometrica 51, 175–196.CrossRefGoogle Scholar
Egesdal, M., Lai, Z., & Su, C. (2013) Estimating Dynamic Discrete-Choice Games of Incomplete Information. Working paper, University of Chicago Booth School of Business.CrossRefGoogle Scholar
Fisher, F.M. (1961) Identifiability criteria in nonlinear systems. Econometrica 29, 574–590.CrossRefGoogle Scholar
Fisher, F.M. (1965) Identifiability criteria in nonlinear systems: A further note. Econometrica 33, 197–205.CrossRefGoogle Scholar
Fisher, F.M. (1966) The Identification Problem in Economics. McGraw-Hill.Google Scholar
Heckman, J. & Navarro, S. (2007) Dynamic discrete choice and dynamic treatment effects. Journal of Econometrics 136, 341–396.CrossRefGoogle Scholar
Hong, H. & Shum, M. (2010) Pairwise-difference estimation of a dynamic optimization model. Review of Economic Studies 77, 273–304.CrossRefGoogle Scholar
Hotz, V. & Miller, R.A. (1993) Conditional choice probabilities and the estimation of dynamic models. Review of Economic Studies 60, 497–531.Google Scholar
Hotz, V., Miller, R.A., Sanders, S., & Smith, J. (1994) A simulation estimator for dynamic models of discrete choice. Review of Economic Studies 61, 265–289.Google Scholar
Komunjer, I. (2012) Global identification in nonlinear models with moment restrictions. Econometric Theory 28, 719–729.CrossRefGoogle Scholar
Kreyszig, E. (1989) Introduction to Functional Analysis with Applications. Wiley Classics Library.Google Scholar
Magnac, M. & Thesmar, D. (2002) Identifying dynamic discrete decision processes. Econometrica 70, 801–816.CrossRefGoogle Scholar
Maskin, E. & Tirole, J. (2001) Markov perfect equilibrium I: Observable actions. Journal of Economic Theory 100, 191–219.CrossRefGoogle Scholar
Matzkin, R.L. (1991) Semiparametric estimation of monotone and concave utility functions for polychotomous choice models. Econometrica 59, 1315–1327.CrossRefGoogle Scholar
Pakes, A., Ostrovsky, M., & Berry, S. (2007) Simple estimators for the parameters of discrete dynamic games (with entry/exit example). RAND Journal of Economics 38, 373–399.CrossRefGoogle Scholar
Pesendorfer, M. & Schmidt-Dengler, P. (2008) Asymptotic least squares estimator for dynamic games. Reviews of Economic Studies 75, 901–928.CrossRefGoogle Scholar
Roehrig, C.S. (1988) Conditions for identification in nonparametric and parametric models. Econometrica 56, 433–447.CrossRefGoogle Scholar
Rothenberg, T.J. (1971) Identification in parametric models. Econometrica 39, 577–591.CrossRefGoogle Scholar
Rust, J. (1987) Optimal replacement of GMC bus engines: An empirical model of Harold Zurcher. Econometrica 55, 999–1033.CrossRefGoogle Scholar
Rust, J. (1994) Structural estimation of Markov decision processes. In Engle, R.F. & McFadden, D. (eds.), Handbook of Econometrics, Vol. 4. North-Holland.Google Scholar
Rust, J. (1996) Numerical dynamic programming in economics. In Aumann, H.M., Kendrick, D.A., & Rust, J. (eds.), Handbook of Computational Economics, Vol. 1. Elsevier.Google Scholar
Sanches, F., Silva, D., & Srisuma, S. (2013) An Alternative Asymptotic Least Squares Estimator for Dynamic Games.Google Scholar
Srisuma, S. (2013) Minimum distance estimators for dynamic games. Quantitative Economics 4, 549–583.CrossRefGoogle Scholar
Srisuma, S. & Linton, O.B. (2012) Semiparametric estimation of Markov decision processes with continuous state space. Journal of Econometrics 166, 320–341.CrossRefGoogle Scholar
Wegge, L. (1965) Identifiability criteria for a system of equations as a whole. Australian Journal of Statistics 7, 67–77.CrossRefGoogle Scholar