×

Inductive process modeling. (English) Zbl 1470.68088

Summary: In this paper, we pose a novel research problem for machine learning that involves constructing a process model from continuous data. We claim that casting learned knowledge in terms of processes with associated equations is desirable for scientific and engineering domains, where such notations are commonly used. We also argue that existing induction methods are not well suited to this task, although some techniques hold partial solutions. In response, we describe an approach to learning process models from time-series data and illustrate its behavior in three domains. In closing, we describe open issues in process model induction and encourage other researchers to tackle this important problem.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
00A71 General theory of mathematical modeling
92D40 Ecology
93B30 System identification
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Arrigo, K. R., Worthen, D. L., & Robinson, D. H. (2003). A coupled ocean-ecosystem model of the Ross Sea: 2. Iron regulation of phytoplankton taxonomic variability and primary production. Journal of Geophysical Research, 108, 3231. · doi:10.1029/2001JC000856
[2] Asgharbeygi, N., Bay, S., Langley, P., & Arrigo, K. R. (2006). Inductive revision of quantitative process models. Ecological Modelling, 194, 70–79. · doi:10.1016/j.ecolmodel.2005.10.008
[3] Åström, K. J., & Eykhoff, P. (1971). System identification–a survey. Automatica, 7, 123–167. · Zbl 0219.93004 · doi:10.1016/0005-1098(71)90059-8
[4] Bay, S. D., Shrager, J., Pohorille, A., & Langley, P. (2002). Revising regulatory networks: from expression data to linear causal models. Journal of Biomedical Informatics, 35, 289–297. · doi:10.1016/S1532-0464(03)00031-5
[5] Bechtel, W., & Abrahamsen, A. (2005). Explanation: a mechanistic alternative. Studies in History and Philosophy of the Biological and Biomedical Sciences, 36, 421–441. · doi:10.1016/j.shpsc.2005.03.010
[6] Berryman, A. A. (1992). The origins and evolution of predator–prey theory. Ecology, 73, 1530–1535. · doi:10.2307/1940005
[7] Box, G., Jenkins, G. M., & Reinsel, G. (1994). Time series analysis: forecasting & control (3rd ed.). Englewood Cliffs: Prentice Hall. · Zbl 0858.62072
[8] Bradley, E., Easley, M., & Stolle, R. (2001). Reasoning about nonlinear system identification. Artificial Intelligence, 133, 139–188. · Zbl 0984.68158 · doi:10.1016/S0004-3702(01)00143-6
[9] Bridewell, W., Sánchez, J. N., Langley, P., & Billman, D. (2006). An interactive environment for the modeling and discovery of scientific knowledge. International Journal of Human–Computer Studies, 64, 1099–1114. · Zbl 05136387 · doi:10.1016/j.ijhcs.2006.06.006
[10] Bunch, D., Gay, D., & Welsch, R. (1993). Algorithm 717: subroutines for maximum likelihood and quasi-likelihood estimation of parameters in nonlinear regression models. ACM Transactions on Mathematical Software, 19, 109–130. · Zbl 0889.65149 · doi:10.1145/151271.151279
[11] Cohen, S., & Hindmarsh, A. (1996). CVODE, a stiff/nonstiff ODE solver in C. Computers in Physics, 10, 138–143.
[12] Dennis, J. E. Jr., Gay, D. M., & Welsch, R. E. (1981). An adaptive nonlinear least-squares algorithm. ACM Transactions on Mathematical Software, 7, 348–368. · Zbl 0464.65040 · doi:10.1145/355958.355965
[13] Dietterich, T. G. (1990). Exploratory research in machine learning. Machine Learning, 5, 5–9.
[14] Domingos, P. (1997). Knowledge acquisition from examples via multiple models. In Proceedings of the fourteenth international conference on machine learning (pp. 98–106). Nashville: Kaufmann.
[15] Džeroski, S., & Todorovski, L. (1993). Discovering dynamics. In Proceedings of the tenth international conference on machine learning (pp. 97–103). Amherst: Kaufmann.
[16] Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York City: Chapman & Hall. · Zbl 0835.62038
[17] Forbus, K. D. (1984). Qualitative process theory. Artificial Intelligence, 24, 85–168. · doi:10.1016/0004-3702(84)90038-9
[18] Forbus, K. D., & Falkenhainer, B. (1990). Self-explanatory simulations: an integration of qualitative and quantitative knowledge. In Proceedings of the eighth national conference on artificial intelligence (pp. 380–387). Boston: AAAI Press.
[19] Garrett, S., Coghill, G. M., Srinivasan, A., & King, R. D. (2007). Learning qualitative models of physical and biological systems. In S. D. Džeroski & L. Todorovski (Eds.), Computational discovery of scientific knowledge. Berlin: Springer.
[20] Gay, D. M. (1983). Algorithm 611: Subroutines for unconstrained minimization using a model/trust-region approach. ACM Transactions on Mathematical Software, 9, 503–524. · Zbl 0519.65039 · doi:10.1145/356056.356066
[21] Ghahramani, Z. (1998). Learning dynamic Bayesian networks. In C. L. Giles & M. Gori (Eds.), Adaptive processing of sequences and data structures. Berlin: Springer.
[22] Ghosh, R., & Tomlin, C. J. (2001). Lateral inhibition through delta-notch signaling: a piecewise affine hybrid model. In Proceedings of the fourth international workshop on hybrid systems: computation and control (pp. 232–246). Springer: Rome. · Zbl 0997.92010
[23] Glennan, S. (2002). Rethinking mechanistic explanation. Philosophy of Science, 69, S342–S353. · doi:10.1086/341857
[24] Glymour, C., Scheines, R., Spirtes, P., & Kelly, K. (1987). Discovering causal structure: artificial intelligence, philosophy of science, and statistical modeling. San Diego: Academic Press. · Zbl 0778.68004
[25] Härdle, W., Horowitz, J., & Kreiss, J. (2003). Bootstrap methods for time series. International Statistical Review, 70, 435–459. · Zbl 1114.62347
[26] Hempel, C. G., & Oppenheim, P. (1948). Studies in the logic of explanation. Philosophy of Science, 15, 135–175. · doi:10.1086/286983
[27] Holling, C. S. (1959). The components of predation as revealed by a study of small-mammal predation of the European pine sawfly. Canadian Entomologist, 91, 293–320. · doi:10.4039/Ent91293-5
[28] Iwasaki, Y., & Simon, H. A. (1994). Causality and model abstraction. Artificial Intelligence, 67, 143–194. · Zbl 0942.68711 · doi:10.1016/0004-3702(94)90014-0
[29] Jost, C., & Ellner, S. (2000). Testing for predator dependence in predator–prey dynamics: a non-parametric approach. Proceedings of the Royal Society of London B, 267, 1611–1620. · doi:10.1098/rspb.2000.1186
[30] Langley, P. (1981). Data-driven discovery of physical laws. Cognitive Science, 5, 31–54. · doi:10.1111/j.1551-6708.1981.tb00869.x
[31] Langley, P., Simon, H. A., Bradshaw, G. L., & \.Zytkow, J. M. (1987). Scientific discovery: computational explorations of the creative processes. Cambridge: MIT Press.
[32] Lavrač, N. L., & Džeroski, S. D. (1994). Inductive logic programming: techniques and applications. New York City: Ellis Horwood.
[33] Machamer, P., Darden, L., & Craver, C. F. (2000). Thinking about mechanisms. Philosophy of Science, 67, 1–25. · doi:10.1086/392759
[34] Martin, J. H., Gordon, R. M., & Fitzwater, S. E. (1991). The case for iron. Limnology and Oceanography, 36, 1793–1802. · doi:10.4319/lo.1991.36.8.1793
[35] Murray, J. D. (2004). Mathematical biology, I: an introduction (3rd ed.). Berlin: Springer.
[36] Needoba, J. A., & Harrison, P. J. (2004). Influence of low light and a light: dark cycle on NO3 uptake, intracellular NO3 and nitrogen isotope fractionation by marine phytoplankton. Journal of Phycology, 40, 505–516. · doi:10.1111/j.1529-8817.2004.03171.x
[37] Olson, R. J., Sosik, H. M., Chekalyuk, A. M., & Shalapyonok, A. (2000). Effects of iron enrichment on phytoplankton in the Southern Ocean during late summer: active fluorescence and flow cytometric analyses. Deep-Sea Research Part II-Topical Studies in Oceanography, 47, 3181–3200. · doi:10.1016/S0967-0645(00)00064-3
[38] Ourston, D., & Mooney, R. J. (1990). Changing the rules: a comprehensive approach to theory refinement. In Proceedings of the eighth national conference on artificial intelligence (pp. 815–820). Boston: AAAI Press.
[39] Pazzani, M. J., Mani, S., & Shankle, W. R. (2001). Acceptance by medical experts of rules generated by machine learning. Methods of Information in Medicine, 40, 380–385.
[40] Poritz, A. (1988). Hidden Markov models: a guided tour. In Proceedings of the international conference on acoustic, speech and signal processing (pp. 7–13). New York City: IEEE Press.
[41] Schwabacher, M., & Langley, P. (2001). Discovering communicable scientific knowledge from spatio-temporal data. In Proceedings of the eighteenth international conference on machine learning (pp. 489–496). Williamstown: Kaufmann.
[42] Simon, H. A. (1954). Spurious correlation: a causal interpretation. Journal of the American Statistical Association, 49, 467–479. · Zbl 0057.11501 · doi:10.2307/2281124
[43] Todorovski, L. (2003). Using domain knowledge for automated modeling of dynamic systems with equation discovery. Doctoral dissertation, Faculty of Computer and Information Science, University of Ljubljana. Ljubljana, Slovenia.
[44] Todorovski, L., & Džeroski, S. (1997). Declarative bias in equation discovery. In Proceedings of the fourteenth international conference on machine learning (pp. 376–384). Nashville: Kaufmann.
[45] Veilleux, B. G. (1979). An analysis of predatory interaction between paramecium and didinium. Journal of Animal Ecology, 48, 787–803. · doi:10.2307/4195
[46] Washio, T., Motoda, H., & Niwa, Y. (2000). Enhancing the plausibility of law equation discovery. In Proceedings of the seventeenth international conference on machine learning (pp. 1127–1134). Stanford: Kaufmann. · Zbl 1105.68399
[47] Williams, R., & Zipser, D. (1989). A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1, 270–280. · doi:10.1162/neco.1989.1.2.270
[48] Woodward, J. (2002). What is a mechanism? A counterfactual account. Philosophy of Science, 69, S366–S377. · doi:10.1086/341859
[49] Zheng, J., Vankataramanan, L., & Sigworth, F. J. (2001). Hidden Markov model analysis of intermediate gating steps associated with the pore gate of Shaker potassium channels. Journal of General Physiology, 118, 547–562. · doi:10.1085/jgp.118.5.547
[50] \.Zytkow, J. M., Zhu, J., & Hussam, A. (1990). Automated discovery in a chemistry laboratory. In Proceedings of the eighth national conference on artificial intelligence (pp. 889–894). Boston: AAAI Press.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.