» Download Article

Authors
Christian P. Robert

Identifier
10.3182/20060329-3-AU-2901.00004

Index Terms
adaptivity,Bayesian inference,MCMC algorithm,Monte Carlo techniques,missing variables,model choice,particle filter,population Monte Carlo

Abstract
The toolbox available in Bayesian Statistics has increased considerably in the past decade and it has opened new avenues for Bayesian inference, the first and foremost being Bayesian model choice. The MCMC and particle filter technologies have hugely increased the potential for Bayesian applications, in particular in missing variable models, as illustrated in this short tutorial. We will also mention a new direction in this field, namely the development of adaptive algorithms that avoid a lenghty tuning to fit the problem at hand by automatically modifying the parameters of the algorithm.

References

[1] C. Andrieu and A. Doucet. Simulated annealing
    for Bayesian estimation of hidden Markov models.
     IEEE Trans. Information Theory, 46(3):
    994-1004, 2000.
[2] C. Andrieu, E. Moulines, and P. Priouret. Stability
     of stochastic approximation under verifiable
    conditions. SIAM Journal on Control and Optimization
    , 44(1): 283-312, 2005.
[3] C. Andrieu and C.P. Robert. Controlled Markov
    chain Monte Carlo methods for optimal sampling.
     Technical Report 0125, Univ. Paris
    Dauphine, 2001.
[4] A. Beskos, O. Papaspiliopoulos, G.O. Roberts,
    and P. Fearnhead. Exact and computationally
     efficient likelihood-based estimation for discretely
     observed diffusion processes (with discussion).
     J. Royal Statist. Soc. Series B, 2006.
    (To appear.).
[5] S.P. Brooks, P. Giudici, and G.O. Roberts. Efficient
     construction of reversible jump Markov
    chain Monte Carlo proposal distributions (with
    discussion). J. Royal Statist. Soc. Series B, 65
    (1): 3-55, 2003.
[6] O. Cappé, A. Guillin, J.M. Marin, and C.P.
    Robert. Population Monte Carlo. J. Comput.
    Graph. Statist., 13(4): 907-929, 2005.
[7] O. Cappé, E. Moulines, and T. Rydén. Inference
    in Hidden Markov Models. Springer-Verlag,
    New York, 2005.
[8] O. Cappé, C.P. Robert, and T. Rydén. Reversible
    jump, birth-and-death, and more general continuous
     time MCMC samplers. J. Royal Statist.
    Soc. Series B, 65(3): 679-700, 2003.
[9] B.P. Carlin and S. Chib. Bayesian model choice
    through Markov chain Monte Carlo. J. Roy.
    Statist. Soc. (Ser. B), 57(3): 473-484, 1995.
[10] J.B. Carlin. Meta-analysis for 2 × 2 tables: a
     Bayesian approach. Statistics in Medicine, 11:
     141-159, 1992.
[11] J. Carpenter, P. Clifford, and P. Fernhead. Improved
      particle filter for nonlinear problems.
     IEE Proc.-Radar, Sonar Navig., 146: 2-4, 1999.
[12] G. Celeux, J.M. Marin, and C.P. Robert. Iterated
     importance sampling in missing data problems.
     Computational Statistics and Data Analysis,
     2006a. (To appear.).
[13] G. Celeux, J.M. Marin, C.P. Robert, and D.M.
     Titterington. Bayesian inference in k-nearest-neighbour
      classification models. Technical report,
      University Paris Dauphine, 2006b.
[14] M.H. Chen, Q.M. Shao, and J.G. Ibrahim.
     Monte Carlo Methods in Bayesian Computation
     . Springer-Verlag, New York, 2000.
[15] N. Chopin. A sequential particle filter method for
     static models. Biometrika, 89: 539-552, 2002.
[16] D. Crisan, P. Del Moral, and T. Lyons. Discrete
     filtering using branching and interacting particle
      systems. Markov Processes and Related
     Fields, 5(3): 293-318, 1999.
[17] P. Del Moral and A. Doucet. Sequential Monte
     Carlo samplers. Technical report, Dept. of
     Engineering, Cambridge Univ., 2003.
[18] P. Dellaportas, O. Papaspiliopoulos, and G.O.
     Roberts. Bayesian inference for non-Gaussian
     Ornstein-uhlenbeck stochastic volatility processes.
      J. Royal Statist. Soc. Series B, 66: 369-
     393, 2004.
[19] R. Douc, A. Guillin, J.-M. Marin, and C.P.
     Robert. Convergence of adaptive sampling
     schemes. Technical Report 2005-6, University
     Paris Dauphine, 2005.
[20] A. Doucet, N. de Freitas, and N. Gordon. Sequential
      Monte Carlo Methods in Practice. Springer-Verlag,
      New York, 2001.
[21] A. Doucet, S. Godsill, and C.P. Robert. Marginal
     maximum a posteriori estimation using Markov
     chain Monte Carlo. Statistics and Computing,
     12: 77-84, 2002.
[22] W.R. Gilks, G.O. Roberts, and S.K. Sahu. Adaptive
      Markov chain Monte Carlo. J. American
     Statist. Assoc., 93: 1045-1054, 1998.
[23] N. Gordon, J. Salmond, and A.F.M. Smith.
     A novel approach to non-linear/non-Gaussian
     Bayesian state estimation. IEEE Proceedings
     on Radar and Signal Processing, 140: 107-113,
     1993.
[24] P.J. Green. Reversible jump MCMC computation
     and Bayesian model determination. Biometrika,
     82(4): 711-732, 1995.
[25] H. Haario, E. Saksman, and J. Tamminen. Adaptive
      proposal distribution for random walk
     Metropolis algorithm. Computational Statistics
     , 14(3): 375-395, 1999.
[26] H. Haario, E. Saksman, and J. Tamminen. An
     adaptive Metropolis algorithm. Bernoulli, 7(2):
     223-242, 2001.
[27] C.C. Holmes and N.M. Adams. A probabilistic
     nearest-neighbour algorithm for statistical pattern
      recognition. J. Royal Statist. Soc. Series
     B, 64(2): 295-306, 2002.
[28] C.C. Holmes, D.G.T. Denison, B.K. Mallick, and
     A.F.M. Smith. Bayesian methods for nonlinear
     classification and regression. John Wiley, New
     York, 2002.
[29] Y. Iba. Population-based Monte Carlo algorithms.
     Trans. Japanese Soc. Artificial Intell., 16(2):
     279-286, 2000.
[30] G. Kitagawa. Monte Carlo filter and smoother for
     non-Gaussian non-linear state space models. J.
     Comput. Graph. Statist., 5: 1-25, 1996.
[31] J.S. Liu. Monte Carlo Strategies in Scientific
     Computing. Springer-Verlag, New York, 2001.
[32] J.S. Liu and R. Chen. Blind deconvolution via
     sequential imputations. J. American Statist.
     Assoc., 90: 567-576, 1995.
[33] J.M. Marin, K.L. Mengersen, and C.P. Robert.
     Bayesian modelling and inference on mixtures
     of distributions. In C.R. Rao and D. Dey,
     editors, Handbook of Statistics, volume 25. Elsevier,
      2005.
[34] J.M. Marin and C.P. Robert. The Bayesian Core.
     Springer-Verlag, New York, 2006. To appear.
[35] P. McCullagh and J. Nelder. Generalized Linear
     Models. Chapman and Hall, New York, 1989.
[36] X.L. Meng and W.H. Wong. Simulating ratios of
     normalizing constants via a simple identity: a
     theoretical exploration. Statist. Sinica, 6: 831-
     860, 1996.
[37] P. Müller. Simulation based optimal design.
     In J.M. Bernardo, J.O. Berger, A.P. Dawid,
     and A.F.M. Smith, editors, Bayesian Statistics
     6, pages 459-474, New York, 1999. Springer-Verlag.
     
[38] P. Müller, G. Parmigiani, C.P. Robert, and
     J. Rousseau. ptimal sample size for multiple
     testing: the case of gene expression microarrays.
     J. American Statist. Assoc., 99:990-1001, 2004.
[39] M.K. Pitt and N. Shephard. Filtering via simulation:
      Auxiliary particle filters. J. American
     Statist. Assoc., 94(446): 590-599, 1999.
[40] S. Richardson and P.J. Green. On Bayesian
     analysis of mixtures with an unknown number
     of components (with discussion). J. Royal
     Statist. Soc. Series B, 59: 731-792, 1997.
[41] C.P. Robert. The Bayesian Choice. Springer-Verlag,
      New York, second edition, 2001.
[42] C.P. Robert. Bayesian computational methods.
     In J. Gentle, W. Härdle, and Y. Mori, editors,
      Handbook of Computational Statistics.
     Springer-Verlag, New York, 2005.
[43] C.P. Robert and G. Casella. Monte Carlo Statistical
      Methods. Springer-Verlag, New York, second
     edition, 2004.
[44] C.P. Robert and J.T.G. Hwang. Maximum likelihood
      estimation under order constraints. J.
     American Statist. Assoc., 91: 167-173, 1996.
[45] D. Rubin. Using the SIR algorithm to simulate
     posterior distributions. In J.M. Bernardo, M.H.
     Degroot, D.V. Lindley, and A.M. Smith, editors,
      Bayesian Statistics 3: Proceedings of the
     Third Valencia International Meeting, June 1-
     5, 1987. Clarendon Press, 1988.
[46] D.B. Rubin. Multiple Imputation for Nonresponse
     in Surveys. John Wiley, New York, 1987.
[47] P. Stavropoulos and D.M. Titterington. Improved
     particle filters and smoothing. In A. Doucet,
     N. deFreitas, and N. Gordon, editors, Sequential
     MCMC in Practice. Springer-Verlag, New York,
     2001.
[48] A. Zellner. On assessing Prior Ddistributions
     and Bayesian Regression analysis with g-prior
     distribution regression using Bbayesian variable
      selection. In Bayesian inference and decision
      techniques: Essays in Honor of Bruno de
     Finetti, pages 233-243. North-Holland/Elsevier,
      1986.