<p>Just wondering.</p>
<p>MCMC = Markov Chain Monte Carlo.</p>
<p>It is a probability algorithm.</p>
<p>Just wondering.</p>
<p>MCMC = Markov Chain Monte Carlo.</p>
<p>It is a probability algorithm.</p>
<p>MCMC is a tool used in Operations Research. It has fallen out of favor for a number of reasons. First, if the TPM (transition probability matrix) is stationary, you can solve the problem as an MDP (Markov Decision Problem) using a dynamic programming technique that derives from the “Bellman Equation”. (You can google all this stuff for a good description.) This avoids the need to do a bazillion runs of an MCMC simulation. </p>
<p>I sense, however, that you’re wondering if real-world problems can be solved with these academic techniques. No. Normally the TPM is not stationary, and even when it is stationary, it’s unkown. For the unknown problem, there is a class of solutions called “Partially Observed Markov Decision Models” or PODMs. These are used in a variety of systems design situations. All together, they would be taught in a course called ‘dynamic programming’. </p>
<p>A more advanced application of MCMC is used in Bayesian statistics, and in ‘data mining’ techniques. But, that’s beyond the scope of this course.</p>
<p>Yes, MCMC is very popular in statistical signal processing. It has been /is used in several applications ranging from blind equalization to target tracking.</p>
<p>What about use of spectral analysis for nonstationary models.</p>
<p>[VTboy] I believe you might find papers on that topic if you go to the IEEE Xplore database and search for authors like M. Davy, S. Godsill, W.J. Fitzgerald, and P. J. Rayner . </p>
<p>Personally, I don’t follow MCMC papers very closely as I am more interested in sequential Monte Carlo methods (aka particle filtering) instead (which have the advantage of being on-line/recursive Bayesian estimation algoritms as opposed to MCMC).</p>