Stumbling Intuition #1: The Monty Hall Problem

[LegionOnomaMoi]

Maybe I can explain this more easily without the math.

The question that matters is does Monty want me to win the car, or not. (We are presuming that I want to win the car.)

There's quite a bit more than this. And although it is important (although it's perhaps not the best way to make clear the relevant assumptions), it doesn't necessarily make things clearer.
Assume, for example, that Monty doesn't want you to win the car but that there are 1,000 doors, 999 of which have no prizes behind them. Assume, again, that you get to pick one of the doors first. Here again, Monty knows which door has the prize behind it and doesn't want to reveal it, and here again Monty will leave you with two options. But this time, to leave you with only two options, Monty has to open 998 non-prize doors.

Originally, you have a 1/1000 chance of guessing correctly. Monty leaves you with two choices just as in the original. But this time, if Monty didn't care whether or not you one or didn't know where the prize was or just started picking doors (other than your original choice) at random, almost every single time he'd open the prize door due to random chance. However, so long as he knows which door has the prize and so long as he isn't supposed to reveal the prize, he must open 998 non-prize doors.

You started out with a 1/1000 chance of guessing correctly. Monty has opened 998 non-prize doors and had to follow two constraints: he couldn't pick your door and he couldn't pick a prize door. There's a 1/1000 chance that these are the same door. So the chances are overwhelmingly high that Monty had to avoid picking not only your door, but also the door with the prize. Hence, even though only two doors remain, you should switch.

 

[Heyo]

This is a bit too smart for me. And I don't grasp it enough to ask a question. I doubt I would ever understand it. But I enjoyed reading you geeking out over it. So that's nice.

That's how mathematicians tackle problems. They don't solve the problem but all problems that can come up which are similar.

The shortest way to explain Monty Hall is to be a pessimist. Assume you chose wrong. When you chose your door there was a 2/3 chance that your choice was wrong. That doesn't change when Monty opens the door. But one of the other chances has been eliminated, so the remaining has to have a 2/3 chance of being the car.

 

[Shadow Wolf]

That's how mathematicians tackle problems. They don't solve the problem but all problems that can come up which are similar.

The shortest way to explain Monty Hall is to be a pessimist. Assume you chose wrong. When you chose your door there was a 2/3 chance that your choice was wrong. That doesn't change when Monty opens the door. But one of the other chances has been eliminated, so the remaining has to have a 2/3 chance of being the car.

I wonder if this idea of the house deliberately kicking the odds in favor of the player drives some unconscious suspicion and doubt behind this, and it is indeed very much a very strong kick to achieve this in my head (not a literal kick of something, but just a kick).

 

[PureX]

There's quite a bit more than this. And although it is important (although it's perhaps not the best way to make clear the relevant assumptions), it doesn't necessarily make things clearer.
Assume, for example, that Monty doesn't want you to win the car but that there are 1,000 doors, 999 of which have no prizes behind them. Assume, again, that you get to pick one of the doors first. Here again, Monty knows which door has the prize behind it and doesn't want to reveal it, and here again Monty will leave you with two options. But this time, to leave you with only two options, Monty has to open 998 non-prize doors.

Originally, you have a 1/1000 chance of guessing correctly. Monty leaves you with two choices just as in the original. But this time, if Monty didn't care whether or not you one or didn't know where the prize was or just started picking doors (other than your original choice) at random, almost every single time he'd open the prize door due to random chance. However, so long as he knows which door has the prize and so long as he isn't supposed to reveal the prize, he must open 998 non-prize doors.

You started out with a 1/1000 chance of guessing correctly. Monty has opened 998 non-prize doors and had to follow two constraints: he couldn't pick your door and he couldn't pick a prize door. There's a 1/1000 chance that these are the same door. So the chances are overwhelmingly high that Monty had to avoid picking not only your door, but also the door with the prize. Hence, even though only two doors remain, you should switch.

It still all depends on whether Monty wants me to win the car, or he doesn't, and whether or not he has to refuse to open the first door I choose no matter what's behind it, or he is choosing not to open it. (If Monty does not know where the car is, the odds are 50/50 to me in the end. But this is unlikely because the game would be "diminished" 1/3rd of the time when Monty choose to open the door with the car.) But we don't know the answers to these questions. All we can do is surmise what those answers would likely be, logically, from the information that we have. You are not looking at the problem with the element of time, or human nature: the fact that the information comes to us sequentially via Monty's behavior. And yet, this is really all the information that we get.

The fact that Monty did not open the first door I chose, immediately, tells me that the result of my choice contradicts his preference. And the fact that I was originally only being given a 1 in 3 chance at winning the car tells me that his preference was that I don't win the car. So I would stick with my original door choice even as he tries to get me to change it, as it would be logically more likely that Monty does not want me to win the car, and so would not open that door when I first chose it.

However, if the first part of the game is just theater, and Monty was always going to disregard my first door choice, and then open a door with a goat behind it, then none of that "information" was ever relevant, and I was always facing a 50/50 proposition with a little pointless theater to try and entertain the audience (and some obsessed mathematicians).

If I choose to extrapolate probability from the theatrics ...

... I was given less than even odds to begin with, and Monty did not open the door I chose, when I chose it. Both actions presumably intending that I don't win the car, but that together tell me where the car is likely to be.

If I ignore the theatrics then I am left with an even probability of either result regardless of which door I choose. So I would stay with my first choice because it is reasonable relative to both scenarios.

 

[Debater Slayer]

This is true, but highly misleading. It's trivially true because "the notion of looking at different events in terms of their likelihood or probability" is at the heart of frequentist inference and statistical inference more generally, not just Bayesian inference.
Also, this problem isn't about Bayesian inference, or inference more generally (not in the usual sense of the term, at least not in the type of context where Bayesian vs. non-Bayesian would apply). t's about conditional probability.
One way to sort of illustrate this is by seeing it in terms of how one might distinguish statistics and probability. In statistics, where inference is one of the central goals (perhaps THE central), one uses samples from some population of interest to make inferences about the population in general. In probability, one starts with the probability space (a triple consisting of a set Ω, a sigma field of subsets of Ω, and a probability measure). In problems like Monty Hall, this is extremely important because one can partition the space of possible outcomes in a way that allows the use of Bayes' theorem (or other methods that are similarly made possible for the same reason). Bayes' theorem is not Bayesian inference. Bayes' theorem is a part of straightforward, elementary probability theory. The Bayesian inference comes in when one extends this to interpreting uncertainties and conditioning on e.g., parameters that a frequentists wouldn't consider to be random variables in order to make inferences from samples from the total space (the population).

In other words, one knows in this problem what all the possible outcomes are and the probabilities associated with "elementary events" are given. It's not inferential, but deductive reasoning that one needs. The difficulties lie in the fact that it is usually quite a bit harder to reason conditionally, or at least it is usually quite a bit harder to be tripped up with problems involving conditional probabilities.


Again, there's nothing "Bayesian" about this. A logician, a probabilist, a frequentist statistician, or indeed many people without much knowledge of probability theory or statistics may think about things this way. The differences come into play in how Bayesians interpret probability itself, as well as the elements of the formalism underlying statistics. So, for example, in textbook statistics (which unfortunately is almost always presented in classical frequentist terms without any mention that there exists alternatives or even that this is an interpretation or philosophy), one might learn about hypothesis testing. The basic example usually concerns a null hypothesis and an alternative. One assumes the null hypothesis to be true and calculates the probability of seeing the data one has or data more extreme given the null hypothesis. If the probability is lower then a selected alpha level, then one rejects the null (sometimes in favor of the alternative). But the hypothesis itself is either true or false in frequentist statistics. One cannot interpret hypothesis tests in this approach as in anyway telling one about the probability that a hypothesis is true or is false. One can only speak of how probable data are given the null hypothesis (or some hypothesis, not necessarily a null).
Bayesians, however, don't think of probabilities in the same way. They don't construct sample distributions in the same way. And thus they have no problem (at least not philosophically; computationally, it can be a nightmare) with treating hypotheses or parameters as random variables.

Thanks for the detailed post. So far, I'm familiar with Bayesian statistics and hypothesis testing for some applications in computer science and machine learning, but I haven't specifically sought out textbooks about the deeper theories or philosophies as my field typically only requires that at a research level (if even then). Probability and statistics in data science also go hand in hand so much that you practically can't use one without the other in a lot of cases. Most courses I have seen teach both, not just one or the other.

While this is largely simplified, I like the clarity of this page on Bayesian inference from Brown University. I'm not sure about other fields, but it enables one to understand and apply a few key concepts to a lot of situations that arise in data science:

Bayesian Inference

I'm sure reading longer, theory-heavy textbooks would provide much more depth and explore the underlying philosophies in greater detail, but I'm planning to cross that bridge when or if I get to it (hopefully if I end up working in research at some point).

 

[Debater Slayer]

Someone posted this problem many years ago on this site and I ended up in a big argument with almost everyone else here at the time, about it.

The fact is the "math" is wrong. Well, it's not the mathematics itself that is wrong, but the way it's being applied, here. The eliminated door is of no consequence in the real world, scenario, probability speaking. It only appears that way on paper because the probability equations when written down on paper have no apparent time signature before or after which information vanishes (as in a musical score, for example). So the mathematician sees the equation as a singular whole, and develops his solution from that body of information.

But the actual scenario that the equation is supposed to be representing is an 'event' taking place in time, and that time signature is crucial to determining a reasonably likely result because information vanishes before and after the ever-moving 'time = zero' moment in the scenario.
The information being presumed here, for example ...

... does not exist in the real time that the scenario is taking place in. It exists in this abstract "eternal mind-scape" of the page. And as a consequence, the results being assumed here are fiction. They are fiction that so long as all the components remain on the page, will support themselves and each other logically. But when lifted off the page and applied to the real world scenario that it's supposed to be representing, become mythical.

I expect most of you will argue with this just as most did in the past. But so be it. Math isn't music and music isn't math. And real probabilities change constantly, in time and via relative circumstance. like music. The necessary information is always vanishing into the future and past of that ever advancing 'now' moment.

I'm not trying to start a fight. I'm just saying that what makes sense laid out on paper as an abstract equation often does not accurately represent the probabilities that occur in real time.

The math on this is certainly not wrong, as it is proven to be correct both with mathematical proofs and simulations. It also does take into account chronological order of each event, since the step where Monty opens the door affects the odds that follow it.

However, it's important to keep in mind that, at the end of the day, we're talking about probabilities that are less than 1 (i.e., not certain). Even if you had a 95% chance of winning something, you would still have a 5% chance of losing it. Switching the door would double your odds of winning, but you could still lose. There's still a 1/3 chance of losing when you switch. The presence of this chance is why knowing the probabilities only improves your odds in this case but doesn't guarantee a win.

 

[PureX]

The math on this is certainly not wrong, as it is proven to be correct both with mathematical proofs and simulations. It also does take into account chronological order of each event, since the step where Monty opens the door affects the odds that follow it.

However, it's important to keep in mind that, at the end of the day, we're talking about probabilities that are less than 1 (i.e., not certain). Even if you had a 95% chance of winning something, you would still have a 5% chance of losing it. Switching the door would double your odds of winning, but you could still lose. There's still a 1/3 chance of losing when you switch. The presence of this chance is why knowing the probabilities only improves your odds in this case but doesn't guarantee a win.

the math is not wrong. But the perspective from which it is being formulated, is. From "god's perspective" the probabilities are as indicated. Because the determination is being made based on a view that sees behind the doors, and into Monty's (or the show producer's) mind. But that isn't the view afforded to the contestant. To him, the theater is either relevant to the odds, or it's not. (50/50) If he deems it relevant, then he has to use logic to try and determine what those events are indicating. If the theater is not relevant, then all that matters is the choice that produces a result. And the probability of that choice being the car is 50/50 after the theatrics.

The mathematicians are not considering the fact that the theater before the actual choice is irrelevant to the actual choice facing the contestant.

Are we calculating the odd to the mathematician? Or to the contestant?

 

[9-10ths_Penguin]

I half get it.

What's losing me is the "why" is it not 50-50.

Wouldn't it only be 2/3 -1/3 if the original goat door was left in the set (door reclosed)? If removed from the set, as a whole, then you're left with one goat and one car?

Why would the goat remaining make a difference?

Regardless of whether the door with the revealed goat is removed from play or is kept in play but you're certain that there's a goat behind it, you aren't going to pick it.

The key for me to wrap my head around it is to realize that in the problem, Monty Hall has given extra information about the doors you didn't pick and no extra information about the door you initially picked. This is enough to make the probabilities non-equal between the remaining doors.

 

[Mock Turtle]

Interesting to see some of the comments to the original article:

Game Show Problem | Marilyn vos Savant

And as to this from the Wiki:

Pigeons repeatedly exposed to the problem show that they rapidly learn to always switch, unlike humans.

 

[9-10ths_Penguin]

The mathematicians are not considering the fact that the theater before the actual choice is irrelevant to the actual choice facing the contestant.

No, you're missing the fact that it is relevant. That's where the extra information comes from.

The door to reveal wasn't chosen at random. It was chosen by the show host, who knows what's behind each door and made sure not to reveal the car.

You aren't bothering to make the inferences from this that you could, which is why you're getting the wrong answer.

Are we calculating the odd to the mathematician? Or to the contestant?

They're the same.

 

[PureX]

Why would the goat remaining make a difference?

Regardless of whether the door with the revealed goat is removed from play or is kept in play but you're certain that there's a goat behind it, you aren't going to pick it.

The key for me to wrap my head around it is to realize that in the problem, Monty Hall has given extra information about the doors you didn't pick and no extra information about the door you initially picked. This is enough to make the probabilities non-equal between the remaining doors.

Not when that "extra information" doesn't correlate, though. It's just theater. Romoving one door from a blind scenario doesn't make the scenario any less blind. It simply limited the options. I still don't know anything more about the door Monty didn't pick than I know about the door I did pick. Because they could still be either door. The only way for me to know anything would be to extrapolate motive from the theatrics.

The odds being found on paper don't leave the paper. (Or the computer simulation.) They are only there because we are imposing a correlation that can only be imposed via the abstract properties of mathematics.

 

[muhammad_isa]

It still all depends on whether Monty wants me to win the car, or he doesn't, and whether or not he has to refuse to open the first door I choose no matter what's behind it, or he is choosing not to open it..

That is not what is described in the initial problem.
Monty always opens a second door.

However, if the first part of the game is just theater, and Monty was always going to disregard my first door choice, and then open a door with a goat behind it, then none of that "information" was ever relevant, and I was always facing a 50/50 proposition with a little pointless theater to try and entertain the audience (and some obsessed mathematicians).

I don't understand.
It is better to switch, because we have new information provided by Monty when he opens his door..
The probability the car is behind the one you chose is 1/3.
The probability the car is behind the one you didn't chose is now 2/3, since one is eliminated.
Which door is better?

 

[The Hammer]

Why would the goat remaining make a difference?

It's the difference between leaving it in the set or not. If you pull a card out of a deck of cards, but then put the card back, the odds of drawing stay the same. But if it is removed totally from the deck after drawing, than the probability of the deck changes.

 

[LegionOnomaMoi]

Probability and statistics in data science also go hand in hand so much that you practically can't use one without the other in a lot of cases.

Statistics requires probability (unless one somehow eschews any mention of random variables from statistical analysis). The reverse is not true. In fact, probabilists have so little contact so much of theoretical statistics (let alone actual statistics in practice) that a great deal is unrecognizable to many. Probabilists are experts in measure theory, among other things, a topic most statisticians rarely touch on in their training and that basically cannot come up in practice. That said, there is often more overlap when it comes to certain kinds of stochastic models or stochastic differential equations or other probability models. Ironically, though, the overlap is often less with applied statistics and more with e.g., Brownian motion and diffusion processes or the interplay between functional analysis (e.g., (conditional) expectations as linear functionals in Hilbert spaces), geometry and topology (e.g., convexity, polytopes, etc), and anaysis of the type one might find in physics or in other kinds of modeling in the applied sciences.

The problem I have with casting this puzzle in terms of Bayesian inference has to do both with the fact that it misses most of the Bayesian approach in practical terms, and outright ignores the central tenet of Bayesian inference the Baeysian interpretation of probability.
Bayesians have received a great deal of criticism (mostly unjustly) because they define probabilities in terms of beliefs or something similarly subjective and allow for a choice of priors. In many cases, this is great: it allows one to incorporate our actual knowledge to avoid e.g., base rate fallacies among other things. And it is particularly valuable in machine learning and similar approaches to data analysis where one can perform many, many thousands of trials or tests or “experiments” in which the algorithm starts out with some prior distribution and updates using some rule, weights, etc., to (hopefully) converge to some optimal “solution” (whatever that means in the particular context).
It most certainly doesn’t mean adopting the frequentist approach by assigning 1/3 probabilities because “they are all equally likely” at first. No, they aren’t equally likely or probable from a Bayesian perspective, which is behind both the problems and benefits of priors. Prior distributions aren’t given, they are subjective. You can decide to make them whatever you want.
So it still works, and it can be illuminating if you're learning about Bayesian analysis, machine learning, statistical computing, etc. But I am not sure how much clearer it makes the problem.

 

[LegionOnomaMoi]

The shortest way to explain Monty Hall is to be a pessimist. Assume you chose wrong. When you chose your door there was a 2/3 chance that your choice was wrong. That doesn't change when Monty opens the door. But one of the other chances has been eliminated, so the remaining has to have a 2/3 chance of being the car.

Excellent! And perfectly correct, of course (although I would perhaps make sure to add that we require Monty to 1) know where the prize is and 2) only open a non-prize door)
To recast the problem without using Bayes' theorem, we can get it in about 2 lines:
Let Door_i be the probability that the prize is behind door_i=1,2,3 and Win be the event that the, well, we win. Then we can rewrite the probability of winning using the law of total probability, partitioning the sample space into three mutually exclusive and collectively exhaustive outcomes and expanding the event Win in terms of its probability conditioned on these outcomes:
Line 1: P(Win)=P(Win|Door_1)*1/3+P(Win|Door_2)*1/3+P(Win|Door_3)*1/3
Assume, without loss of generality, that we select the first door (and therefore, if the prize is behind door 1, then P(Door_1)=1 and 0 for Door_2 and Door_3). Now assume we try the switching tactic. We can write all the probabilities in one line:
Line 2: P(Win)= 0* 1/2 + 1*1/3 + 1*1/3
(In words, if we switch and the prize is behind the first door, then P(Win|Door_1)=0 or the probability that we win given that the prize is behind the first door is 0, because we switched, but if the prize is behind either of the other doors, then we will win because Monty will open the non-winning door).
Last time I checked, 1/3+1/3=2/3.

 

[LegionOnomaMoi]

It still all depends on whether Monty wants me to win the car

You can replace Monty with a robot or even an algorithm and play this game with the given probabilities and a mindless machine incapable of wants. You can even replace the God's eye view you describe elsewhere and just decide that you want to win and figure out an optimal strategy given the odds.

It works exactly the same way. So long as Machine Monty has to
1) Open one of the remaining two doors
2) open a non-prize door
&
3) if there are two options (because your first guess was correct) Machine Monty has to pick from the two available doors at random with equal probability

And you write the code for the game to start with equal probabilities for selecting the winning "door" (1/3, 1/3, 1/3), then you can take whatever kind of approach you wish, but if you want to win, then you can show mathematically AND with the assumption that probabilities are subjective that switching increases your chances of winning substantially.

 

[PureX]

That is not what is described in the initial problem.
Monty always opens a second door.

Does he? I don't recall that being expressed in the scenario given. And does the contestant know this? If so, then he also knows he has no chance at all of winning the car on the first guess. Which completely screws up the math equation being discussed, here. The odds are no longer 1 in 3, they are 0 in 3.In fact, they were always 0 in 3 so long as the doors remained unopened.

I don't understand.
It is better to switch, because we have new information provided by Monty when he opens his door..

"We" (contestants) don't have any new information, the mathematician does. And the reason he has it is because he can correlate information looking at the problem "from above" that the contestant cannot correlate from within the scenario. The mathematician imagines the scenario as an abstraction through which he can see behind all the doors. So he can correlate revealed information with unrevealed information. But in the actual world there is no such correlation available. And therefor no such informational advantage. Unless one extrapolates motive from the events.

The probability the car is behind the one you chose is 1/3.

It doesn't matter what the probability of the car being behind that door was, because there was no probability at all that the contestant could win it. And that is the probability we are seeking, is it not? This is what I mean when I say that the probability changes from the perspective of the mathematician to the reality of the actual contestant. In the scenario given, the contestant has zero probability of winning the car no matter what door he chooses, initially. The mathematician is ignoring this essential fact because he is looking at the scenario from "above".

The probability the car is behind the one you didn't chose is now 2/3, since one is eliminated.

Only to the mathematician that is imagining that he is seeing the scenario from "above". To the actual contestant in front of the doors, there are now two doors, each with an equal probability that there will be a car, or a goat, behind it. The mathematician's odds are fictional (abstract/ideological), while the contestant's odd are actual.

Which door is better?

Neither is better unless the contestant attributes motive to the proceedings. Then choosing the same door that he chose the first time is probably the better option.

 

[PureX]

You can replace Monty with a robot or even an algorithm and play this game with the given probabilities and a mindless machine incapable of wants. You can even replace the God's eye view you describe elsewhere and just decide that you want to win and figure out an optimal strategy given the odds.

It works exactly the same way. So long as Machine Monty has to
1) Open one of the remaining two doors
2) open a non-prize door
&
3) if there are two options (because your first guess was correct) Machine Monty has to pick from the two available doors at random with equal probability

And you write the code for the game to start with equal probabilities for selecting the winning "door" (1/3, 1/3, 1/3), then you can take whatever kind of approach you wish, but if you want to win, then you can show mathematically AND with the assumption that probabilities are subjective that switching increases your chances of winning substantially.

What you are overlooking is that picking the "right door" and winning the car are two different goals. If we are assessing the probability of "picking the right door" then the odds were 1/3. But if we are assessing the chances of winning the car then the odds when choosing among the three doors was zero. Not 1/3. Because no matter what door the contestant picked he was not going to win the car. That was all just a bit of theater.

So if the contestant is trying to win the car, that bit of theater with the three doors was an irrelevant distraction, as it imparted no information at all regarding what's behind the two remaining doors before him. (Unless he ascribes motive to that bit of theater and considers that information perrtinent.)

 

[muhammad_isa]

Does he? I don't recall that being expressed in the scenario given.

The OP..
Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?

It doesn't matter what the probability of the car being behind that door was, because there was no probability at all that the contestant could win it..

There is. He doesn't have to switch from his first choice, if he doesn't want to.
The probability that he chooses the right door initially is 1/3.

In the scenario given, the contestant has zero probability of winning the car no matter what door he chooses, initially.

How do you figure that?

To the actual contestant in front of the doors, there are now two doors, each with an equal probability that there will be a car, or a goat, behind it. The mathematician's odds are fictional (abstract/ideological),,

No .. I know it might seem like that
That's the whole point of the paradox.

The elimination of a "goat" door from a choice of 2 changes the odds.
Monty doesn't eliminate the first choice, but one of the other two.
In doing so, it changes the odds of the switch being right to 2/3.
The car can only be behind one of 3 doors, so there are only 3 scenarios.

The odds are not 1/2, because Monty does not reveal the first choice, whatever is behind it.

 

[PureX]

The OP..
Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?


There is. He doesn't have to switch from his first choice, if he doesn't want to.
The probability that he chooses the right door initially is 1/3.


How do you figure that?

You can't win a car behind a door that does not open. There is no probability of his winning the car when the contestant is presented with the choice of the three doors because whatever door he picks, it is not going to be opened and he is not getting a car, or a goat. Choosing the "right door" in this instance is irrelevant to his winning a car. Are we seeking the odds of his choosing the "right door"? or are we seeking the odds of his winning a car? Because they are not the same result, and therefor do not manifest the same odds.

No .. I know it might seem like that
That's the whole point of the paradox.

It's not a paradox. It's a matter of different perspectives. Mathematics is a system of ideological abstraction that we can use to help us cognate reality more fully and clearly. But because it is an abstraction, it can sometimes create an illusion of clarity or universality that is actually misleading. And this an example of it.