Talk:Markov process

From Wikipedia, the free encyclopedia

Contents

[edit] Article needs major overhaul

I will just point out some inconsistencies and mistakes in this article:

A Markov process is, as translations from Russian state, "a process without after-effect" This sounds as if the word 'Markov' translates to 'without after-effect'. In fact Markov is the name of a Russian mathematician. It should be made clear to which original text the translation refers to.

From the standard definitions of drift and diffusion coefficients this means that those coefficients may depend at most on a single point (x, t) and on no history at all. Some author seems to have mixed the concepts of 'Ito-process' and a 'Markov-process'. There is no 'drift and diffusion coefficient' of a general Markov process. The whole first paragraph is very confusing and seems to be more about Ito-processes than about Markov-processes

The Markov property is stated twice with different definitions. --128.130.51.57 (talk) 13:58, 26 March 2008 (UTC)

[edit] Merge

This article should be merged with the article Markov chain. As far as I can tell, this article doesn't actually say anything that the Markov chain article doesn't say. linas 15:56, 4 September 2005 (UTC)

I agree that the content may be related, but I think having a separate entry for Markov processes is still useful for the layperson. (Anonymous user) 23:37, 22 September 2005 (PST).
?? I don't get the sense that this aricle is somehow "easier to understand" than the other one. linas 16:18, 23 September 2005 (UTC)
Laypersons looking for a Markov process will be redirected to Markov chain when the redirect is in place --Anthony Liekens 14:43, 10 October 2005 (UTC)
I think it should be merged with Markov property not with markov chain. Markov property
the contents of this article discusses the Markov property, but it's title is more related to a Markov chain, not the Markov property --Anthony Liekens 14:43, 10 October 2005 (UTC)
Agree, just putting up a redirect to Markov chain would be sufficient. By the way, there's a lot of work to be done in Markov chain itself! --Anthony Liekens 14:43, 10 October 2005 (UTC)
It makes sense to fold Process into the Property entry, then have a link called Process. But I would leave the Chain entry alone unless someone has a clear idea how to integrate Process, Property, Chain, and perhaps HMM, MDP, and POMDP. --Randy Crawford 20:55, 10 October 2005 (UTC)
At the very least, somewhere there needs to be either an entry or subsection for 'Markov process.' Any textbook on Stochastic Processes usually has an entire chapter devoted to 'Markov processes'. Furthermore, not all Markov processes are Markov chains (e.g. a process that is not stationary or homogeneous in time); redirecting the entry for the 'process' to an entry for one particular class of that process would only confuse and bias the reader. If it is to be redirected, it ought to be to folded into Markov property and not Markov chain. --H. C. Hodges 15:52, 22 October 2005 (PST).
Markov property seems a nicer article than Markov process, but since the Markov property is a property of processes, an article titled Markov process and also covering the Markov property seems best. Then, Markov property could redirect there. Merging this with Markov chain would be a mistake, for the reasons given by H. C. Hodges. Ben Cairns 13:39, 23 October 2005 (UTC).

I have clarified the second paragraph slightly. LS

[edit] Introductory sentence

It would be good to have an introductory sentence that introduces the subject to a layperson. For example, see [1]. Jim 18:51, 24 March 2006 (UTC)

[edit] Continuous Time

For mathematicians, usually Markov process means continuous time, whereas Markov chain means discrete time. Markov processes are often used in chemistry and biology, and their properties are different than Markov chains -- would be nice to have a separate entry. Leboudec 16:24, 15 May 2006 (UTC)

Greetings Leboudec. Are you Jean-Yves Le Boudec, author of "Fluid Approximations of Continuous Time Markov Chains"? I've generally heard continuous time Markov processes on discrete state spaces described as Markov chains. LachlanA 02:21, 30 July 2007 (UTC)

[edit] Continuous Value

Markov chain is only for countable set of states. I thought Markov process might contain continuous valued version. (c.f. Mathworld [2]) Memming 13:15, 24 September 2006 (UTC)

[edit] Markov chains, processes, etc

Are not there too many pages about the same thing? M. chain, M.property, continuous time M.process, etc. I propose the following: 1) merge M. process with M. property; this will include the def. both for discrete and cont. time 2) then there will be a link to M. chain for properties specific for discrete time, and to cont.-time M. proc. - for properties specific for cont. time 3) To the latter (cont. time) one should add geometrical properties such as the Bakry-Emery condition. Convergence to equilibrium should also appear somewhere - maybe, in the first page (M. process?)

What do you think? Sodin 14:38, 1 March 2007 (UTC)

Please excuse me asking yet another question here instead of helping directly with yours, Sodin. Can the term 'Markov Chain' also relate to a continuous time process if it has a discrete state space? If so, the MC page would need to deal with both continuous and discrete time. Trog23 17:10, 3 March 2007 (UTC)
I think a chain is with discrete time and any state space. The division between discrete and continuous state space is a bit obscure: the state space can be any, say, topological space (whereas the time should be either Z or Z_+ or R or R_+). Sodin 01:06, 4 March 2007 (UTC)
OK, thanks. Your definition agrees with the present text on both the 'Markov process' and 'Markov Chain' pages, and with the Oxford Users' Guide to Mathematics (ISBN 0-19-85073-1 section 6.4.2 p 1039) which defines a MC in terms of a discrete time parameter. However Norris ('Markov Chains', Cambridge University Press, ISBN 0-521-63396-6) states that “it is usual to refer to [a MP with a discrete state space] as a Markov Chain”, which is in agreement with Michael Hardy's recent comment on the Continuous-Time MP talk page. So, there seems to be some inconsistency here. Surely a discrete sequence of values would be generated if either the time parameter or the state space (or both) were discrete, and it should be reasonable to describe either case as a 'chain'? Regards Trog23 17:55, 5 March 2007 (UTC)

I agree that merging the content from Markov property into this page would be suitable. LachlanA 02:25, 30 July 2007 (UTC)

Yes, one of us has to get to work and clean up the Markov sections. I suggest one entry called Markov Processes. Under this heading we say that "chain" refers to discrete time (see Meyn & Tweedie 2005, [1]). Most of the chapter heading will deal with discrete time, but one subsection can be entitled continuous time. This will contain a link to a wiki diffusions entry.

Spmeyn (talk) 15:25, 11 December 2007 (UTC)

[edit] Merge all, separate sections for each

Why not merge all pages but maintain them in separate sections on the Markov Chain page? —Preceding unsigned comment added by 128.200.138.240 (talk) 18:17, 8 October 2007 (UTC)

  • Suppport merges. way too many... Dicklyon (talk) 02:11, 12 December 2007 (UTC)
  • Support merge. Too many, and 50% of this article is a wandering digression into other processes Professor marginalia (talk) 19:45, 2 January 2008 (UTC)

[edit] markov processes that are not dynamical systems?

In the back of my mind, I tend to draw this (possibly incorrect) mental model of a Markov process as some function on a measure-preserving dynamical system, but where the underlying time evolution is not made explicit (or is not explicitly known). I think my mental model is correct (?) ... but it occurs to me: are there *any* examples of Markov processes that cannot be formulated as functions on a measure-preserving dynamical system? If so, can examples of these be given? linas (talk) 00:15, 10 April 2008 (UTC)