14.5hypoth

In previous parts of this topic we have been finding estimates for the mean of a population using samples. The last part we are now moving onto is asking a different question. We want to know if there has been a change in the mean. For example if you are testing a new drug you will have a sample of people using the new drug and a control sample using either a placebo or an old drug. When we compare the two samples we want to know if there has been a increase in effectiveness **and** the likelihood that it is due to the **drug and not random chance**.
 * Hypothesis Testing **

Setting up Hypotheses for Testing If you think of a courtroom, prosecutors want to show that a person is guilty **but** we start under the assumption that the person is innocent. We do the same in statistics, we start with the assumption that the mean **has not** changed and it is simply variance in the sample. This is called the null hypothesis (H 0 ) **.** If we can show that the null hypothesis does not make sense then we have evidence for our alternative hypothesis (H 1 ), that is, the mean has changed**.**

For example if we return to our population of year 12 students and find a sample with a mean of 103.6. Knowing the population average for an IQ test is 100 we might want to test is it likely that year 12 students have higher IQs than the general population. So we write down our null and alternative hypotheses: The null hypothesis will state that there is no change in the mean, that is, H 0 : μ = 100 The alternative hypothesis will say that the sample (our year 12 students) is drawn from a population with a hgiher mean, that is H1: μ > 100 Note that in this case we have a **directional hypothesis,** that is we think the year 12 mean is higher than the general population.

P values To test if our null hypothesis is true or not we need a measure of statistical significance, that is, how likely is it that the sample mean differed by random chance. So for our year 12 students math \text{p value}=Pr( \bar{X} \geq 103.6 \mid \mu =100) math
 * The p value is the probability of observing a value of the sample mean as extreme or more extreme than the one observed, assuming the null hypothesis is true.**

Now using our central limit theorem where: math \sigma_{\overline{x}}=\dfrac{\sigma}{ \sqrt{n}} math so given the sd of an IQ test is 15 and a sample size of 100 we get sigma of 1.5 math \text{p value}=Pr( \bar{X} \geq 103.6 \mid \mu =100) \\ \text{p value}=Pr(Z \geq \dfrac{103.6 - 100}{1.5}) \\ \text{p value}=Pr(Z \geq 2.4) \\ \text{p value}= 0.0082 \\ math This may or may not be evidence to overturn our cnull hypothesis, the necessary level depends on our **significance level, α. ** If our p value is less than α we reject the null hypothesis, if it is greater, we accept the null hypothesis.

Typically significance levels of 0.05 (5%) are used however 1% and 0.001% levels are used as well. In particle phyiscs the 5 sigma level, Pr(Z>5), is often used, that is there is a 0.00001% chance of it being sample variance.

Using this process for a sample from a normally distributed population is called a **z** test.

One and Two tailed Tests The test we have done above was a one tailed test as we had a direction in mind. That is we only got the right hand tail of the normal distribution. If we do not know the direction the mean changed we use a **two-tailed test**. That is either the sample mean is significantly more **or** less that the population mean. the p value for a two tailed test is **twice** the p value of a one tailed test.

Typically the two-tailed test is a more conservative test as it requires a greater deviation from the population mean to be considered statistically significant.
 * Always use a two tailed test unless you have a VERY good reason for thinking you have a direction.**

Errors in Hypothesis Testing There are two types of errors we can have (shown below), type 1 and type 2. So if this was a courtroom, a type 1 error would be to convict an innocent person, a type 2 error would be to let a guilty person go free. That is: Type 1: reject null hypothesis when is true Type 2: do not reject null hypothesis when it is false. You need to know these two types of errors and be able to state them for a given situation.
 * || Actual || Situation ||
 * Our decision || H 0 true || H 0 not true ||
 * Reject H 0 || **Type 1 error** || Correct decision ||
 * Do not reject H 0 || Correct Decision || **Type 2 error** ||

We should note that as the significance level required is lower (1% as opposed to 5%) the chance of a type 1 error decreases (from 5% to 1%) but the chance of a type 2 error increases.

Testing on your CAS From the menu go to **statistics** then **calc** then **test.**

We then want to select a **one sample Z test,** and check **variable (not list).**

From above population mean is 100, population standard deviation is 15, sample mean is 103.6 sand sample size is 100. We also did a directional hypothesis so we choose > test.

Note that we get the same value as before

We can also do a non-directional test and we get the result below:

Here we can see that the p value is twice the one directional value **and** that we would **not reject** the null hypothesis at the null hypothesis at the 1% level using this test. This is why it is important to know when to use a one or two-tailed test.

Lastly we should note that as the sample size increases, our p value decreases as the distribution of sample means becomes more narrow (lower standard deviation).