Data Analysis #12: Chi-squared and Control Charts

[samvara]

Last seminar. Handed in assignment of d00m all prettily printed out and bound.

χ² (Chi-squared - pronounced 'ki' as in kite) Test for Two Proportions: Basic Idea

• Compares observed frequencies to expected frequencies if H_o is true.
• The closer the observed frequencies are to the expected frequencies, the more likely H_o is true.
  • Measured by squared differences relative to expected frequency (as per goodness of fit)
  • Sum of relative squared differences is the test statistic (as per goodness of fit)

Evaluation/Perception: Contingency Table.

• Consider comparing people's perception of the fairness of two different evaluation methods. We have the observed frequencies and their respective totals.
  • H_o: there is no relationship between methods (Null hypothesis)
  • H₁: there is a relationship between methods (Alternative hypothesis)

	Evaluation	Method 1	Evaluation	Method 2
Perception	1: Observed	1: Expected	2: Observed	2:Expected	Total
Fair	63	54.6 (70%)	49	57.4 (70%)	112 (70%)
Unfair	15	23.4 (30%)	33	24.6 (30%)	48 (30%)
Total	78	78	82	82	160

• Analysis: We can calculate expected frequencies by using the Totals.
  • Step 1: Total Fair = 112/160 * 100 = 70%
  • Step 2: Method 1 Fair = 70% of 78 = 78 * 70/100 = 54.6

The χ² Test Statistic

• χ² = ∑ (O - E)²/E


• ∑ means 'Sum' or 'add them all up' and in this case refers to all cells.
• O: is the observed frequency in the cell
• E: is the expected frequency in the cell

Computing χ² - the above calculation broken down into steps

Observed	Expected	O - E	(O - E)2	(O - E)2/E
63	54.6	8.4	70.56	1.293
49	57.4	-8.4	70.56	1.229
15	23.4	-8.4	70.56	30.15
33	24.6	8.4	70.56	2.868
-	-	-	Total	8.405

We then sneak off to read the Critical Values of χ² table which, in the 6th Ed of Levine is on page 769. In this table you match up α (alpha) with df (degrees of freedom) and read off the corresponding probability.

• Let α = 0.01 (confidence level of 99%)
• df = (number of rows - 1)(number of columns minus 1) = (2 - 1) x (2 - 1) = 1
  • just to be confusing I wedged the calculations into the Contingency Table and the rows are actually 2 (Fair & Unfair) and the columns are 2 as well (Method 1 & Method 2)

Table says Critical Value is = 6.635, i.e. for H_o to be accepted, probability must be below 6.635; anything over that means we reject the Null Hypothesis. As χ² is an exciting 8.405 we reject the Null Hypothesis and declare that there is significant evidence to support a relationship between the perception and evaluation methods. Method 1 is perceived to be fairer.

Now if we'd just run the darn thing through a stats program we could have done this a LOT faster.

PHStat Output

Level of Significance	0.01
Number of Rows	2
Number of Columns	2
Degrees of Freedom	1
Critical Value	6.635
Chi-Square
Test Statistic	8.405
P-Value	0.004

Analysis: We have options; use either one but not both :p

1. P-Value is less than Level of Significance = reject the null hypothesis
2. Critical Value is less than Test Statistic = reject the null hypothesis

Comments

[fred_mouse]

*excuse me while I find my quibble hat*

specifically:
- this is a Pearson's chi-squared test for goodness of fit (although I've seen other terms for it as well - YMMV depending on area).
- Chi-squared is a type of distribution (see http://en.wikipedia.org/wiki/Chi-square_distribution for specifics).
- you can't accept a null hypothesis. Anyone who tells you you can doesn't know what they are talking about. You can merely fail to reject it (or similar terminology that translates to "you can't prove a negative")

more randomly:
- An alternative method that is arguably better is the Fisher's exact test, which is more computationally intensive, and thus better suited to being done using the grunt of a computer, whereas the above test can be done so very easily by hand.
- you don't mention any of the restrictions of this particular test. For example, expected values need to be over 5 in each cell (for counts). I have a feeling that there is something to do with zeroes as well, but can't remember if that is just expecteds as well, or if it goes wibbly when the observed countss are zero.