Discovering the hidden structure of complex dynamic systems(17)

Dynamic Bayesian networks provide a compact and natural representation for complex dynamic systems. However, in many cases, there is no expert available from whom a model can be elicited. Learning provides an alternative approach for constructing models of

learned after only a single iteration of SEM. We can see that the algorithm does discover a few interesting correlations, such as one between TurnSignal and XdotSens (sensed lateral movement).

7 Discussion and ConclusionsIn this paper, we combine two lines of works. The rst deals with search techniques for learning in the presence of hidden variables Friedman 1997; Friedman et al. 1998]. The second deals with fast approximate inference in complex networks Boyen and Koller 1998b; Boyen and Koller 1999]. While approximate DBN inference has been playing a major role in parametric learning Boyen and Koller 1998a; Ghahramani and Jordan 1996a], this is the rst paper to deal with the issues involved in applying it to structure search. In particular, we had to deal with the computation of a large number of di erent statistics and to introduce methods for discovering hidden variables. Although we based our solution on the Boyen-Koller approximation, many of these ideas can be applied to other approximate inference methods, including the variational methods of Ghahramani and Jordan 1996a]. Clearly, our work only scratches the surface of the problem of discovering hidden variables. While our algorithm discovers correlations that involve temporal interactions, it is less apt at detecting atemporal correlations as we saw in the stock market data. On the other extreme, our algorithm does not support the discovery of truly long-range dependencies and aggregate in uences from variables evolving at di erent speeds. Our current method for discovering hidden in uences requires that the time scale of the interaction matches the time scale of the model. If there is a hidden variable evolving much more slowly than the observables, then our algorithm would not nd it. This problem can be addressed by explicitly searching for violations of the Markov property at widely varying time granularities. Speci cally, applying our algorithm on a data sequence subsampled by a factor of k would exhibit interactions with a time constant of the order of k. We believe this issue to be of crucial importance when learning from real-world data. In real systems, observable variables are typically in uenced by hidden processes with widely di ering time scales, which furthermore are not always related to the sampling rate of the observations.

References

Boyen, X. and D. Koller (1998a). Approximate learning of dynamic models. In NIPS 11. Boyen, X. and D. Koller (1998b). Tractable inference for complex stochastic processes. In Proc. UAI. Boyen, X. and D. Koller (1999). Exploiting the architec-

ture of dynamic systems. In Proc. AAAI. Buntine, W. (1991). Theory re nement on Bayesian networks. In Proc. UAI, pp. 52{60. Chickering, D., D. Geiger, and D. Heckerman (1995). Learning Bayesian networks: search methods and experimental results. In Proc. AI& Stats, pp. 112{128. Dagum, P. and A. Galper (1993). Forecast

ing sleep apnea with dynamic network models. In Proc. UAI, pp. 64{71. Dempster, A., N. Laird, and D. Rubin (1977). Maximum-likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B39, 1{38. Forbes, J., T. Huang, K. Kanazawa, and S. Russell (1995). The BATmobile: Towards a Bayesian automated taxi. In Proc. IJCAI. Friedman, N. (1997). Learning belief networks in the presence of missing values and hidden variables. In Proc. ICML. Friedman, N. (1998). The Bayesian structural EM algorithm. In Proc. UAI, pp. 129{138. Friedman, N., K. Murphy, and S. Russell (1998). Learning the structure of dynamic probabilistic networks. In Proc. UAI, pp. 139{147. Friedman, N., D. Peer, and I. Nachman (1999). Learning Bayesian network structure from massive datasets: The\sparse candidate" algorithm. In Proc. UAI. Ghahramani, Z. and M. Jordan (1996a). Factorial hidden Markov models. In Proc. NIPS. Ghahramani, Z. and M. Jordan (1996b). Factorial hidden Markov models. In Proc. NIPS. Heckerman, D. (1999). A tutorial on learning with Bayesian networks. In M. I. Jordan (Ed.), Learning in Graphical Models. MIT Press. Heckerman, D., D. Geiger, and D. M. Chickering (1995). Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197{243. Jensen, F., S. Lauritzen, and K. Olesen (1990). Bayesian updating in recursive graphical models by local computations. Computational Statistical Quarterly 4. Lauritzen, S. and D. Spiegelhalter (1988). Local computations with probabilities on graphical structures and their application to expert systems. J. Roy. Stat. Soc. B 50. Lauritzen, S. L. (1995). The EM algorithm for graphical association models with missing data. Computational Statistics and Data Analysis 19, 191{201. Rabiner, L. and B. Juang (1986, January). An introduction to hidden Markov models. IEEE Acoustics, Speech& Signal Processing . Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics 6, 461{464. Shenoy, P. P. and G. R. Shafer (1990). Axioms for probability and belief-function propagation. In Proc. UAI, pp. 169{198. Spirtes, H., C. Glymour, and R. Scheines (1993). Causation, Prediction, and Search. Springer-Verlag. Weigend, A. and N. Gershenfeld (1990). Time-series competition. In Proc. Nonlinear Modeling and Forecasting, Volume XII. Santa Fe institute: AddisonWesley.

搜索“diyifanwen.net”或“第一范文网”即可找到本站免费阅读全部范文。收藏本站方便下次阅读，第一范文网，提供最新人文社科Discovering the hidden structure of complex dynamic systems(17)全文阅读和word下载服务。