1
   

Thinking Fast and Slow probability question

 
 
Reply Mon 6 Feb, 2012 09:29 am
In his book (Thinking Fast and Slow) Kahneman states: For example, if you believe that 3% of graduate students are enrolled in computer science (the base rate), and you also believe that the description of Tom W is 4 times more likely for a graduate student in that field than in other fields, then Bayes’s rule says you must believe that the probability that Tom W is a computer scientist is now 11%. If the base rate had been 80%, the new degree of belief would be 94.1%.

I would appreciate it if someone could show the math here. Not being able to do it is driving me crazy.
  • Topic Stats
  • Top Replies
  • Link to this Topic
Type: Question • Score: 1 • Views: 4,899 • Replies: 12
No top replies

 
maxdancona
 
  1  
Reply Mon 6 Feb, 2012 06:50 pm
@wardjames,
BM.

I am looking at this problem. It is poorly worded I am trying to figure out what the second part means.

I think there is a missing piece of information here needed for Baye's law, namely the percentage of students who are "like Tom W'.

Let me look a little more.

maxdancona
 
  1  
Reply Mon 6 Feb, 2012 07:01 pm
@wardjames,
Yep, unless I am really misunderstanding the question then there is not enough information here to solve it.

Bayes law say P(A|B) = P(B|A) * P(A)/P(B)

A is Computer science
B is TomLike (i.e. someone with the "description of Tom W")

So P(A|B) is read "the probability of Computer Science given TomLike". In other words this is the chance that someone who is TomLike is in Computer science.

P(B|A) is the probability that someone who is in Computer Science is TomLike. We have to figure out what this is from that strangely worded second part.

P(A) is the chance that someone is in computer science. This is clearly 3%.

P(B) is the chance that someone is TomLike. This is the problem, there is apparently no way to figure this out even if we can what the heck they mean for by that strangly worded phrase for P(B|A).

I don't think this can be solved.

markr
 
  1  
Reply Tue 7 Feb, 2012 12:59 am
@wardjames,
Found this in the back of the book:

"For the hypothesis that Tom W is a computer scientist, the prior odds that correspond to a base rate of 3% are (.03/.97 = .031). Assuming a likelihood ratio of 4 (the description is 4 time as likely if Tom W is a computer scientist than if he is not), the posterior odds are 4 x .031 = 12.4. From these odds you can compute that the posterior probability of Tom W being a computer scientist is now 11% (because 12.4/112.4 = .11)."

By the way, I found this by googling:

Thinking Fast and Slow computer scientist is now 11%

and found it in Google Books.
maxdancona
 
  1  
Reply Tue 7 Feb, 2012 07:26 am
@markr,
Markr,

I think this math is simply wrong, and I think I can prove it.

Let's assume 10000 graduate students of which 300 are Computer scientists (that makes 3%).

Let's say 4 people who are Computer Scientists are TomWLike and 1 is not TomWLike (that makes it 4 times as likely that Tom is a computer scientist than not).

Now I think I have met all the inputs to the problem (base rate and likihood ratio), where the heck do you get 11%?

It is possible that I am just misunderstanding the question, but if you want to defend this reasoning then please do it with real numbers.




wardjames
 
  1  
Reply Tue 7 Feb, 2012 03:32 pm
@markr,

I see the math and it works. But then the author says change the base rate to 80 % and the new answer is 94.1. Given his formula I can't make that work.
maxdancona
 
  1  
Reply Tue 7 Feb, 2012 07:57 pm
@wardjames,
Wardjames, can you show me how the math works with a real example (i.e. 10,000 students)?
0 Replies
 
markr
 
  1  
Reply Tue 7 Feb, 2012 10:10 pm
@maxdancona,
I'm not defending anything. I merely posted what I found in the back of the book.
wardjames
 
  1  
Reply Wed 8 Feb, 2012 03:02 pm
@markr,
I'm not sure what a "real numbers" scenario means. But too my surprise (I'm not a math person) I used an Excel spreadsheet and got both sets of numbers to work:

.03 (prior probability) * 4 (conditional probability) = .12 .97 (prior probability) * 1 (conditional probability) = .97.

.12/(.12+.97) = 11


.08 * 4 = 3.2 .2 * 1 = .2

3.2/(3.2 + .2) = .941
maxdancona
 
  2  
Reply Wed 8 Feb, 2012 04:43 pm
@wardjames,
OK sure, so you can get numbers that the author gave you to match numbers that the author got. But what does it mean? You probably did the same calculation that the author did, but that doesn't mean the calculation isn't bogus.

And it certainly doesn't help me, or anyone else, understand the calculation.

I gave you Bayes theorem. It would help if someone could explain why the calculation you did has anything to do with Bayes. I admit that it might, but I have done these calculations before and I don't see it.

0 Replies
 
markr
 
  1  
Reply Wed 8 Feb, 2012 08:01 pm
@maxdancona,
I haven't digested this, but it references Bayes' Theorem in forms that are different than the "standard" form you provided. It seems to use language that will explain the usage in the stated problem.

http://plato.stanford.edu/entries/bayes-theorem/

jonh1
 
  1  
Reply Thu 7 May, 2015 01:30 pm
@markr,
If you use this form of the theorem it works

Another form of Bayes' Theorem that is generally encountered when looking at two competing statements or hypotheses is:

P(A |B) = P(B | A)*P(A)/{(B | A)* P(A) + (B |-A)P(-A)}


Where
A is the probability someone is a computer science student: in the first example its 0.03 then 0.8 in the second example

B is the probablility someone has Tom W's personality: this is unknown and doesn't need to be known (read on)

-A is the probability someone is NOT a computer science student: first 0.97, then 0.2

P((B | A) is the probablility a computer science student has Tom W's personality. This is technically unknown though we do know it is 4 times higher for this group of students than other students, so we call it 4X

P(B | -A) s the probability a student of something other than computing has Tom W's personality. This is unknown though we do know it is 4 times lower than the above (so call it X)

We want to know P(A |B)

If you plug in the numbers then the X's cancel on top and bottom and the equation works out to give the answers Kahneman gives.
0 Replies
 
louissogukim
 
  1  
Reply Sun 4 Oct, 2015 02:27 am
@maxdancona,
I think what Kanhman tired to solve Tom W like problem is misundersttod.

The posterior probability is estimated at 0.42 % from simply Bayes Rule.
( the porbability that the graduate student who took a compuer science would be a computer scientist like Tom W is 0.42 %).
S. G. Kim
0 Replies
 
 

Related Topics

 
  1. Forums
  2. » Thinking Fast and Slow probability question
Copyright © 2024 MadLab, LLC :: Terms of Service :: Privacy Policy :: Page generated in 0.03 seconds on 05/06/2024 at 03:48:44