ASYMPTOTIC PROPERTIES OF DISTRIBUTED AND COMMUNICATING STOCHASTIC-APPROXIMATION ALGORITHMS

The asymptotic properties of extensions of the type of distributed or decentralized stochastic approximation proposed by J. N. Tsitsiklis are developed. Such algorithms have numerous potential applications in decentralized estimation, detection and adaptive control, or in decentralized Monte Carlo simulation for system optimization. The structure involves several isolated processors (recursive algorithms) that communicate to each other asynchronously and at random intervals. The asymptotic (small gain) properties are derived. In many applications, the dynamical terms are merely indicator functions, or have other types of discontinuities. The 'typical' such case is also treated, as is the case where there is noise in the communication. The linear stochastic differential equation satisfied by the (interpolated) asymptotic normalized error sequence is derived, and issued to compare alternative algorithms and communication strategies. Weak convergence methods provide the basic tools.