Talk:Markov chain

Convergence speed to the stationary distribution

I am studying the Markov chain now and I found some possible typo.

> π(k) approaches to π as k → ∞

I believe it should be "π(k) approaches a1*π as k → ∞".

> In other words, π = ui ← xPP...P = xPk as k → ∞.

It should be π = u1 instead of ui. Here, u1 represents the eigenvector of P corresponding to lambda1=1.

— Preceding unsigned comment added by Jgdyjb (talk • contribs) 05:02, 15 August 2023 (UTC)

Thanks for this. I believe you are correct and that I've fixed it here. In future, when you find a problem you need to fix it yourself as generally no-one else will. — Bilorv (talk) 09:33, 21 August 2023 (UTC)

This can be far more comprehensible for students.

Students may need to understand how to apply Markov Chain computations to the following example. A shepherd has a flock of sheep, three pastures and two sheep dogs, to graze seven days a week. He grazes the pastures on a rotating basis, working one dog at a time on alternating days. Say one is black and one is white. My student needs to compute when will the shepherd will work the black sheep dog in pasture 1 on a Thursday/all Thursday occurrences. I can make the chain more interesting by adding variables, and apply it to retail sales, cell phone tracking and re-stocking needs, for a mathematical application.

My interest in this discussion derives from an appreciation of exactly how good the IDEA encryption algorithm is, understanding that it derives from substitution on a password dependent Markov chain with an astronomical period/periodicity.

While it may be theoretically vulnerable, it is exceedingly difficult to cryptanalyze, and it should be understood by ENTRY LEVEL USERS to be very adequate, even for life and death uses. Formfollows (talk) 04:45, 3 October 2023 (UTC)

Awful article

Scientific articles are Wikipedia's Achilles heel, but this is truly terrible. I'm trying to understand section A non-Markov example and I'm giving up. That's just not the way to do it. The counter-example should be simple and clear, not so complicated and obscure. And if you do such complicated examples, you need to add drawings.

It is also questionable description in Applications. Statistics? It is all statistics. Economics and finance? Ok, but next Social sciences includes economics. Finance could be separated from economics and then social sciences combined with economics. Pawel.jamiolkowski (talk) 18:32, 9 June 2024 (UTC)

Hi @Pawel.jamiolkowski,

For your feedback to be useful, you have to be more specific. Is your problem with the stochastic process itself, or with the way it is presented? How do you think that a drawing could help here?

Regarding your comment on the section Applications: listing "statistics" is perfectly relevant; please have a look into the difference between probability theory and statistics to see why the study of Markov chains is not a subfield of statistics. I think grouping economics and finance is also relevant here, but I agree that something more specific than social science could be used (especially considering that the meaning of the word is a bit vague and has changed over time). However to me these are pretty minor issues.

Malparti (talk) 08:31, 10 June 2024 (UTC)

First of all, A non-Markov example is the sub-section of Examples, right? Reader reads examples which are brief description without details. And it's okay. And then we asking: Ok, so what process won't be Markov chain? The reply is below. However instead seeing a similarly short description, we see loooong piece of writing.

It should be consistent with the previous part. Short decription for intuitive understanding.

I don't know what drawings could be included here, because I don't understand this.

And secondly, why is a separate article with the same title? I guess to deepen the topic. There should actually be such a counter-example there (although better written anyway - first an intuitive description, then a deeper explanation). Pawel.jamiolkowski (talk) 13:49, 10 June 2024 (UTC)

Hi again @Pawel.jamiolkowski,

The subsection is actually rather short; what makes it look long is in part the explanation of how to build a Markov chain out of this non-Markovian process (which is somewhat irrelevant — here we want to understand what makes a process non-Markovian; not how to make non-Markovian processes Markovian by augmenting the state-space...). My guess is that this is a recent addition; articles on basic and widely used math tools have a natural tendency to grow because they attract a lot of readers from different backgrounds and many people want to add something — so they have to be trimmed down once in a while. I will probably remove the last paragraph of the subsection, but there are other problems I would like to fix first.

Another problem might be the use of the terms "dime" and "nickel" and "quarter", which lengthens the text and make is look more complex than it is to non-American readers. I guess that could be improved, too.

Other than that, I think this example is about as simple at it can get without getting something entirely trivial, and I think the level of detail is more-or-less OK. So I think it would be helpful if you could invest time to understand it and then explain what you found hard to understand, and how the description could be improved.

Best, Malparti (talk) 22:27, 10 June 2024 (UTC)

You have almost the same example, but simple and clear here: Markov property

Do you see? It should look like that. 3 balls, colors etc.

But generally for grasping idea, at the beginning it should just explain that the essence of difference lies in difference between draw with replacement and draw without it. It's obvious that if we have constant number of objects and draw without replacement, the future depends on draw, because each draw corresponds with each ball / coin, 1:1. As the number of objects decreases, it is obvious that the probability of guessing increases. Everyone can understand it without any problem. Pawel.jamiolkowski (talk) 23:47, 10 June 2024 (UTC)