Papers of general interest that survey incremental approaches, at least in part:

Fisher, D., & Pazzani, M. (1991). "Computational Models of Concept Learning," in Fisher, D., Pazzani, M., & Langley, P. (eds.), Concept Formation: Experience and Knowledge in Unsupervised Learning. San Mateo, CA: Morgan Kaufmann, 3--44.

Fisher, D., & Schlimmer, J. (1988). "Models of Incremental Concept Learning," Technical Report 88-05, Department of Computer Science, Vanderbilt University, Nashville, TN.

Our ID4 system for incremental decision tree induction was influential:

Schlimmer, J., & Fisher, D. (1986). "A Case Study of Incremental Concept Formation," Proceedings of the Fifth National Conference on Artificial Intelligence, Philadelphia, PA: Morgan Kaufmann, 496--501.

Cobweb and its descendents -- incremental conceptual clustering or concept formation.

Fisher, D. (1987) "Knowledge Acquisition Via Incremental Conceptual Clustering," Machine Learning, 2, 139--172. Reprinted in J. Shavlik & T. Dietterich (eds.), Readings in Machine Learning, 267--283, Morgan Kaufmann, 1990.

Gennari, J., Langley, P., & Fisher, D. (1989). "Models of Incremental Concept Formation," Artificial Intelligence, 40, 11--62.

My latest installment in the Cobweb family of systems introduced a novel form of interative optimization that redistributed many objects simultaneously within a clustering by taking advantage of a clustering's hierarchical organization. I call the strategy hierarchical redistribution, and it was inspired by incremental clustering operations of merging and by macro learning in problem-solving work and is a good scaleup strategy for data mining:

Fisher, D. (1995). "Optimization and Simplification of Hierarchical Clusterings," First International Conference on Knowledge Discovery in Databases, Montreal, Canada: AAAI Press, 118--123.

Fisher, D. (1996). "Iterative Optimization and Simplification of Hierarchical Clusterings," Journal of Artificial Intelligence Research, 4, 147--179.

The papers above also describe methods of identifying optimal frontiers, one per attribute, that are optimal for purposes of predicting each attribute. The method of frontier identification is adapted from resampling-based pruning methods od supervised decision tree induction, and an incremental variant described in:

Fisher, D. (1989). "Noise-Tolerant Conceptual Clustering" Proceedings of the International Joint Conference on Artificial Intelligence, Detroit, MI: Morgan Kaufmann, 825--830.