Side-projects/Hypthesis-testing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we set up null and alternative hypothesis, we need to use data to determine which hypothesis is most likely to be true. Two ways:\n",
    "\n",
    "- Confidence intervals: 1. Simulate sampling distribution of our statistics (bootstraping). 2. See which hypothesis is consistent with what we observed in the sampling distribution. \n",
    "- Common way: 1. Simulate what we belive to be possible under the null. 2. See if our data is consistent with that."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We want to prove: Average height of all individuals in the coffee dataset is greater than 70."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$H_0: \\mu \\leq 70$\n",
    "\n",
    "$H_1: \\mu > 70$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "df = pd.read_csv('data/coffee_dataset.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>age</th>\n",
       "      <th>drinks_coffee</th>\n",
       "      <th>height</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>4509</td>\n",
       "      <td>&lt;21</td>\n",
       "      <td>False</td>\n",
       "      <td>64.538179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1864</td>\n",
       "      <td>&gt;=21</td>\n",
       "      <td>True</td>\n",
       "      <td>65.824249</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>2060</td>\n",
       "      <td>&lt;21</td>\n",
       "      <td>False</td>\n",
       "      <td>71.319854</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>7875</td>\n",
       "      <td>&gt;=21</td>\n",
       "      <td>True</td>\n",
       "      <td>68.569404</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>6254</td>\n",
       "      <td>&lt;21</td>\n",
       "      <td>True</td>\n",
       "      <td>64.020226</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id   age  drinks_coffee     height\n",
       "0     4509   <21          False  64.538179\n",
       "1     1864  >=21           True  65.824249\n",
       "2     2060   <21          False  71.319854\n",
       "3     7875  >=21           True  68.569404\n",
       "4     6254   <21           True  64.020226"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Approach 1: Confidence intervals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_df = df.sample(150)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "means = []\n",
    "for _ in range(10000):\n",
    "    bootsample = sample_df.sample(150, replace=True)\n",
    "    means.append(bootsample.height.mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "low, high = np.percentile(means, 2.5), np.percentile(means, 97.5) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.lines.Line2D at 0x1b0417a3d48>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD4CAYAAADsKpHdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAASYUlEQVR4nO3dfZBddX3H8fenROjUJ4IEGkNokIm2+EcjroilWpQaQpxpdEZa6FQzFCfWgY5Y22mwM4XRYYZaH1qmloqSAtZK8almJBUjtXWcESShEQgRWTGVNRGiqNg6o0W//eP+Vi/L3ee7e+/G92vmzj33e37nnO89ObufPec+JFWFJEm/MOgGJEnDwUCQJAEGgiSpMRAkSYCBIElqlg26gakce+yxtWbNmkG3cfjavbtz//znD7YPaSY8Xmds9+7d36qqFbNdLsP8ttORkZHatWvXoNs4fCWd+yE+BqSf8nidsSS7q2pktst5yUiSBBgIkqTGQJAkAQaCJKkxECRJgIEgSWoMBEkSYCBIkhoDQZIEDPlXV0jDbM3Wmwey3f1XvmIg29XhzzMESRJgIEiSGgNBkgQYCJKkxkCQJAEGgiSpMRAkSYCBIElqDARJEmAgSJIaA0GSBBgIkqRm2kBIsjrJZ5PsS7I3yRtb/fIk30iyp902di1zaZLRJPclOburvqHVRpNsXZinJEmai5l82+ljwJur6s4kTwV2J9nZ5r27qt7RPTjJKcB5wHOBZwKfSfLsNvs9wMuBMeCOJNur6t5+PBFJ0vxMGwhVdRA42Ka/n2QfsGqKRTYBN1bVD4GvJRkFTmvzRqvqAYAkN7axBoIkDYFZvYaQZA3wPOD2Vro4yV1JtiVZ3mqrgAe7FhtrtcnqE7exJcmuJLsOHTo0m/YkSfMw40BI8hTgo8AlVfUocDVwMrCOzhnEO8eH9li8pqg/vlB1TVWNVNXIihUrZtqeJGmeZvQ/piV5Ep0w+GBVfQygqh7qmv8+4JPt4RiwumvxE4ADbXqyuiRpwGbyLqMA1wL7qupdXfWVXcNeBdzTprcD5yU5KslJwFrgi8AdwNokJyU5ks4Lz9v78zQkSfM1kzOEM4DXAHcn2dNqbwHOT7KOzmWf/cDrAapqb5Kb6LxY/BhwUVX9GCDJxcAtwBHAtqra28fnIkmah5m8y+jz9L7+v2OKZa4AruhR3zHVcpKkwfGTypIkwECQJDUGgiQJMBAkSY2BIEkCDARJUmMgSJIAA0GS1Mzou4ykYbVm682DbkE6bHiGIEkCDARJUmMgSJIAA0GS1BgIkiTAQJAkNQaCJAkwECRJjYEgSQIMBElSYyBIkgADQZLUGAiSJMBAkCQ1BoIkCTAQJEmNgSBJAgwESVJjIEiSAANBktRMGwhJVif5bJJ9SfYmeWOrH5NkZ5L72/3yVk+Sq5KMJrkryald69rcxt+fZPPCPS1J0mzN5AzhMeDNVfVrwOnARUlOAbYCt1bVWuDW9hjgHGBtu20BroZOgACXAS8ETgMuGw8RSdLgTRsIVXWwqu5s098H9gGrgE3A9W3Y9cAr2/Qm4IbquA04OslK4GxgZ1U9UlXfAXYCG/r6bCRJc7ZsNoOTrAGeB9wOHF9VB6ETGkmOa8NWAQ92LTbWapPVJ25jC50zC0488cTZtCf9XFiz9eaBbXv/la8Y2La18Gb8onKSpwAfBS6pqkenGtqjVlPUH1+ouqaqRqpqZMWKFTNtT5I0TzMKhCRPohMGH6yqj7XyQ+1SEO3+4VYfA1Z3LX4CcGCKuiRpCMzkXUYBrgX2VdW7umZtB8bfKbQZ+ERX/bXt3UanA99rl5ZuAdYnWd5eTF7fapKkITCT1xDOAF4D3J1kT6u9BbgSuCnJhcDXgXPbvB3ARmAU+AFwAUBVPZLkbcAdbdxbq+qRvjwLSdK8TRsIVfV5el//Bzirx/gCLppkXduAbbNpUJK0OPyksiQJMBAkSY2BIEkCDARJUmMgSJIAA0GS1BgIkiTAQJAkNQaCJAkwECRJjYEgSQIMBElSYyBIkgADQZLUGAiSJMBAkCQ1BoIkCTAQJEmNgSBJAgwESVJjIEiSAANBktQYCJIkwECQJDUGgiQJMBAkSY2BIEkCDARJUmMgSJKAGQRCkm1JHk5yT1ft8iTfSLKn3TZ2zbs0yWiS+5Kc3VXf0GqjSbb2/6lIkuZjJmcI1wEbetTfXVXr2m0HQJJTgPOA57Zl/j7JEUmOAN4DnAOcApzfxkqShsSy6QZU1eeSrJnh+jYBN1bVD4GvJRkFTmvzRqvqAYAkN7ax9866Y0nSgpjPawgXJ7mrXVJa3mqrgAe7xoy12mR1SdKQmGsgXA2cDKwDDgLvbPX0GFtT1J8gyZYku5LsOnTo0BzbkyTN1pwCoaoeqqofV9VPgPfxs8tCY8DqrqEnAAemqPda9zVVNVJVIytWrJhLe5KkOZhTICRZ2fXwVcD4O5C2A+clOSrJScBa4IvAHcDaJCclOZLOC8/b5962JKnfpn1ROcmHgDOBY5OMAZcBZyZZR+eyz37g9QBVtTfJTXReLH4MuKiqftzWczFwC3AEsK2q9vb92UiS5mwm7zI6v0f52inGXwFc0aO+A9gxq+4kSYvGTypLkgADQZLUGAiSJMBAkCQ1BoIkCTAQJEmNgSBJAgwESVJjIEiSAANBktQYCJIkwECQJDUGgiQJMBAkSc20X38tzcSarTcPugVJ8+QZgiQJMBAkSY2BIEkCDARJUmMgSJIAA0GS1BgIkiTAQJAkNQaCJAkwECRJjYEgSQIMBElSYyBIkgADQZLUTBsISbYleTjJPV21Y5LsTHJ/u1/e6klyVZLRJHclObVrmc1t/P1JNi/M05EkzdVMzhCuAzZMqG0Fbq2qtcCt7THAOcDadtsCXA2dAAEuA14InAZcNh4ikqThMG0gVNXngEcmlDcB17fp64FXdtVvqI7bgKOTrATOBnZW1SNV9R1gJ08MGUnSAM31NYTjq+ogQLs/rtVXAQ92jRtrtcnqkqQh0e8XldOjVlPUn7iCZEuSXUl2HTp0qK/NSZImN9dAeKhdCqLdP9zqY8DqrnEnAAemqD9BVV1TVSNVNbJixYo5tidJmq25BsJ2YPydQpuBT3TVX9vebXQ68L12SekWYH2S5e3F5PWtJkkaEsumG5DkQ8CZwLFJxui8W+hK4KYkFwJfB85tw3cAG4FR4AfABQBV9UiStwF3tHFvraqJL1RLkgZo2kCoqvMnmXVWj7EFXDTJerYB22bVnSRp0fhJZUkSYCBIkpppLxlJ0rg1W28eyHb3X/mKgWz3541nCJIkwECQJDUGgiQJMBAkSY2BIEkCDARJUmMgSJIAA0GS1BgIkiTAQJAkNQaCJAkwECRJjYEgSQIMBElSYyBIkgADQZLUGAiSJMBAkCQ1BoIkCTAQJEmNgSBJAgwESVJjIEiSAANBktQYCJIkwECQJDUGgiQJmGcgJNmf5O4ke5LsarVjkuxMcn+7X97qSXJVktEkdyU5tR9PQJLUH/04Q3hpVa2rqpH2eCtwa1WtBW5tjwHOAda22xbg6j5sW5LUJwtxyWgTcH2bvh54ZVf9huq4DTg6ycoF2L4kaQ7mGwgFfDrJ7iRbWu34qjoI0O6Pa/VVwINdy4612uMk2ZJkV5Jdhw4dmmd7kqSZWjbP5c+oqgNJjgN2JvnyFGPTo1ZPKFRdA1wDMDIy8oT5kqSFMa8zhKo60O4fBj4OnAY8NH4pqN0/3IaPAau7Fj8BODCf7UuS+mfOgZDkyUmeOj4NrAfuAbYDm9uwzcAn2vR24LXt3UanA98bv7QkSRq8+VwyOh74eJLx9fxzVX0qyR3ATUkuBL4OnNvG7wA2AqPAD4AL5rFtSVKfzTkQquoB4Nd71L8NnNWjXsBFc92eJGlh+UllSRIw/3cZacis2XrzjMfun8Mykg5fniFIkgADQZLUGAiSJMBAkCQ1BoIkCTAQJEmNgSBJAgwESVJjIEiSAANBktQYCJIkwECQJDV+uZ2kobdm680D+zLG/Ve+YlG3N0ieIUiSAANBktQYCJIkwECQJDUGgiQJMBAkSY2BIEkCDARJUmMgSJIAP6m8IBb7k5SS1A+eIUiSAANBktQYCJIkwECQJDWL/qJykg3A3wJHAO+vqisXuwdJmqlBvUlkEF+7vaiBkOQI4D3Ay4Ex4I4k26vq3oXYnu/2kaSZW+xLRqcBo1X1QFX9CLgR2LTIPUiSekhVLd7GklcDG6rqde3xa4AXVtXFXWO2AFvaw+cA9y1QO8cC31qgdS+EpdTvUuoVlla/S6lXWFr9LqVeYep+f6WqVsx2hYv9GkJ61B6XSFV1DXDNgjeS7KqqkYXeTr8spX6XUq+wtPpdSr3C0up3KfUKC9PvYl8yGgNWdz0+ATiwyD1IknpY7EC4A1ib5KQkRwLnAdsXuQdJUg+Lesmoqh5LcjFwC523nW6rqr2L2UOXBb8s1WdLqd+l1CssrX6XUq+wtPpdSr3CAvS7qC8qS5KGl59UliQBBoIkqTlsAiHJ0Uk+kuTLSfYleVGr/3GS+5LsTfL2Hss9J8mertujSS5p8y5P8o2ueRsXqtck/9K1nf1J9kyy7Ib2fEaTbO2qn5Tk9iT3t3Ud2Y9e59NvktVJPtuW2ZvkjV3zhnHf7k9ydxu3q6t+TJKdbd/uTLK8H73Op98hOm7XJbltfJ8lOW2SZTe3/Xd/ks1d9ee3fT6a5Kokvd6avqj9tjFfaMfsXUl+r2vedUm+1rVv1w2y17bsj7v62d5Vn/3vhKo6LG7A9cDr2vSRwNHAS4HPAEe1+nHTrOMI4Jt0PtQBcDnwp4vR64T57wT+cpL+vgo8qy33JeCUNu8m4Lw2/Q/AG4ag35XAqW36qcBXuvodqn3b5u0Hju1RfzuwtU1vBf5qGPodhuMW+DRwTqttBP6jx3LHAA+0++Vtenmb90XgRXQ+o/Rv4+sacL/PBta26WcCB8f/XYDrgFcPy75t8/5nkvqsfyccFmcISZ4GvAS4FqCqflRV3wXeAFxZVT9s9YenWdVZwFer6r8H0Ov4/AC/C3yox+I9v/qjLfMy4CNt3PXAKwfdb1UdrKo72/T3gX3Aqn701e9ep7GJzj6FIdm3EwzyuC3gaW3Y0+n9uaKzgZ1V9UhVfQfYCWxIshJ4WlV9oTq/tW5g4ffttP1W1Veq6v42fQB4GJj1p34Xo9cp1jmn3wmHRSDQ+Yv5EPCPSf4ryfuTPJlO0r+4nTb9Z5IXTLOe83jiD9/F7bRxW58uFUzW67gXAw+NH5ATrAIe7Ho81mrPAL5bVY9NqPfDfPr9qSRrgOcBt3eVh2nfQucH8NNJdqfzFSrjjq+qg9AJOeC4PvTaj37HDfK4vQT46yQPAu8ALu2x7GTH7ao2PbHeD/Pp96faZZoj6ZyZj7ui7dt3JzlqCHr9xXZJ6bYk47/05/Q74XAJhGXAqcDVVfU84H/pnNovo3OKejrwZ8BNk12jbNfXfgf4cFf5auBkYB2d08Z3LmCv485n8r8IJ/vqj2m/EmQe5tMvAEmeAnwUuKSqHm3lYdu3AGdU1anAOcBFSV7Sh56m0o99O+jj9g3Am6pqNfAm2l+5E9vsURvUcTuTfgFoZzAfAC6oqp+08qXArwIvoHMJ7M+HoNcTq/MVFr8P/E2Sk5nrvu33tbBB3IBfBvZ3PX4xcDPwKeDMrvpXgRWTrGMT8OkptrEGuGehem3Ty4CHgBMmWfZFwC1djy9tt9D5kqtlvcYNqt825kl0Poj4J8O8b3us63LadXg6X7C4sk2vBO4bhn07DMct8D1+9nmmAI/2WPZ84L1dj9/baiuBL082blD9tnlPA+4Ezp1iG2cCnxx0rxPWdR3w6rn+TjgszhCq6pvAg0me00pnAfcC/0rnOhpJnk3n1G+ybwd8wl9j7S+Eca8C7lnAXgF+m84PyFjPhSf56o/q/It/ls6BALAZ+MR8e51vv+1s7FpgX1W9a8K8odq3SZ6c5Knj08D6rp6209mnMCT7tsugj9sDwG+12suAXpe3bgHWJ1neLl+tp/PL6SDw/SSnt2PltSz8vp223/az9XHghqr68IR5K9t96FyTH+i+bfv0qDZ9LHAGcO+cfyf0I42H4Ubn9HgXcBedIFhOJwD+ic4/2p3Ay9rYZwI7upb9JeDbwNMnrPMDwN1tndtpfyUuRK9d6f5HE8ZO7HUjnXfrfBX4i676s+i8Y2OUzuWDoxZy386kX+A36Zym3gXsabeNw7hv2/77UrvtnbBvnwHcSucH8lbgmEHv22E5btu/8e62324Hnt/GjtD5HxHHl/3DdmyO0rkEQ9e4e9rx/He0v4gH2S/wB8D/dR2ze4B1bd6/t317D53fLU8ZcK+/0fr5Uru/sGuds/6d4FdXSJKAw+dFZUnSPBkIkiTAQJAkNQaCJAkwECRJjYEgSQIMBElS8/+6dkM1PMaLhgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(means);\n",
    "plt.axvline(x=low, color='r', linewidth=2);\n",
    "plt.axvline(x=high, color='r', linewidth=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above case, our interval was entirely below 70, which would suggest the null (the population mean is less than 70) is actually true."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Approach 2: Common"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Simulate from the closest value to the alternative that is still in the null space.\n",
    "\n",
    "We can use the std of the sampling distribution to determine what is the samplling distribution would look like if it is from the null hypothesis. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "means = []\n",
    "for _ in range(10000):\n",
    "    bootsample = sample_df.sample(150, replace=True)\n",
    "    means.append(bootsample.height.mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.2492166963157652"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.std(means)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The sampling distribution of mean would follow a normal distribution by central limit theorem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "null_vals = np.random.normal(70,  np.std(means), 10000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([   5.,   39.,  259., 1077., 2366., 3006., 2179.,  837.,  207.,\n",
       "          25.]),\n",
       " array([68.95832132, 69.15122862, 69.34413592, 69.53704321, 69.72995051,\n",
       "        69.92285781, 70.1157651 , 70.3086724 , 70.5015797 , 70.694487  ,\n",
       "        70.88739429]),\n",
       " <a list of 10 Patch objects>)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAT7UlEQVR4nO3dfaxk9X3f8ffHYEhjO2ExC1ov6yx2NmmwKi9ki3GpWifUPFkKWDLtUjVe2VQbVSDZUip1SSPhxkXCVW1aqw4JDhtj1zUhtimbQEq2hChKVR4WZ3lYMOHyUHO9G1gH29i1QgP59o/53Xi4zH3Ye+fOvfB7v6TRnPme35nzPWfv/dwzZ87MpqqQJPXhdavdgCRpcgx9SeqIoS9JHTH0Jakjhr4kdeTo1W5gPieccEJt3rx5tduQpFeV++6771tVtX7UvDUd+ps3b2bfvn2r3YYkvaok+T9zzfP0jiR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SerIgqGf5EeS3JPk/iQHkvy7Vj8lyd1JHkvyO0mOafVj2+OpNn/z0HNd0eqPJjl3pTZKkjTaYo70XwB+vqreCWwFzktyJvAJ4Jqq2gJ8G7i0jb8U+HZV/SRwTRtHklOB7cA7gPOAX09y1Dg3RpI0vwVDvwa+3x6+vt0K+Hngy61+A3BRm76wPabNPztJWv3Gqnqhqp4EpoAzxrIVkqRFWdQnctsR+X3ATwKfAR4HvlNVL7Yh08DGNr0ReBqgql5M8l3gza1+19DTDi8zvK6dwE6At771rUe4OdLkbN5166qs96mr37cq69Vrw6LeyK2ql6pqK3Ayg6Pznxk1rN1njnlz1Wev67qq2lZV29avH/nVEZKkJTqiq3eq6jvAHwNnAsclmXmlcDJwsE1PA5sA2vwfB54bro9YRpI0AYu5emd9kuPa9N8B/gnwCHAn8IE2bAdwS5ve0x7T5v9RDf4j3j3A9nZ1zynAFuCecW2IJGlhizmnvwG4oZ3Xfx1wU1X9fpKHgRuT/Hvgz4Dr2/jrgS8kmWJwhL8doKoOJLkJeBh4Ebisql4a7+ZIkuazYOhX1QPAaSPqTzDi6puq+ivg4jme6yrgqiNvU5I0Dn4iV5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6smDoJ9mU5M4kjyQ5kOQjrf6xJN9Msr/dLhha5ookU0keTXLuUP28VptKsmtlNkmSNJejFzHmReCXq+prSd4E3Jdkb5t3TVX9x+HBSU4FtgPvAN4C/M8kP9VmfwZ4LzAN3JtkT1U9PI4NkSQtbMHQr6pDwKE2/b0kjwAb51nkQuDGqnoBeDLJFHBGmzdVVU8AJLmxjTX0JWlCjuicfpLNwGnA3a10eZIHkuxOsq7VNgJPDy023Wpz1SVJE7Lo0E/yRuArwEer6nngWuDtwFYGrwQ+OTN0xOI1T332enYm2Zdk3+HDhxfbniRpERYV+klezyDwv1hVXwWoqmeq6qWq+hvgs/zwFM40sGlo8ZOBg/PUX6aqrquqbVW1bf369Ue6PZKkeSzm6p0A1wOPVNWnhuobhoa9H3ioTe8Btic5NskpwBbgHuBeYEuSU5Icw+DN3j3j2QxJ0mIs5uqds4BfBB5Msr/VfgW4JMlWBqdongJ+CaCqDiS5icEbtC8Cl1XVSwBJLgduB44CdlfVgTFuiyRpAYu5eudPGX0+/rZ5lrkKuGpE/bb5lpMkrSw/kStJHTH0Jakjhr4kdcTQl6SOLObqHWnN2rzr1tVuQXpV8Uhfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR3xC9ekV5nV/JK5p65+36qtW+Phkb4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpIwuGfpJNSe5M8kiSA0k+0urHJ9mb5LF2v67Vk+TTSaaSPJDk9KHn2tHGP5Zkx8ptliRplMUc6b8I/HJV/QxwJnBZklOBXcAdVbUFuKM9Bjgf2NJuO4FrYfBHArgSeBdwBnDlzB8KSdJkLBj6VXWoqr7Wpr8HPAJsBC4EbmjDbgAuatMXAp+vgbuA45JsAM4F9lbVc1X1bWAvcN5Yt0aSNK8jOqefZDNwGnA3cFJVHYLBHwbgxDZsI/D00GLTrTZXffY6dibZl2Tf4cOHj6Q9SdICFh36Sd4IfAX4aFU9P9/QEbWap/7yQtV1VbWtqratX79+se1JkhZhUaGf5PUMAv+LVfXVVn6mnbah3T/b6tPApqHFTwYOzlOXJE3IYq7eCXA98EhVfWpo1h5g5gqcHcAtQ/UPtqt4zgS+207/3A6ck2RdewP3nFaTJE3IYr5P/yzgF4EHk+xvtV8BrgZuSnIp8A3g4jbvNuACYAr4AfAhgKp6LsnHgXvbuF+rqufGshWSpEVZMPSr6k8ZfT4e4OwR4wu4bI7n2g3sPpIGJUnj4ydyJakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0JakjC4Z+kt1Jnk3y0FDtY0m+mWR/u10wNO+KJFNJHk1y7lD9vFabSrJr/JsiSVrIYo70PwecN6J+TVVtbbfbAJKcCmwH3tGW+fUkRyU5CvgMcD5wKnBJGytJmqCjFxpQVX+SZPMin+9C4MaqegF4MskUcEabN1VVTwAkubGNffiIO5YkLdlyzulfnuSBdvpnXattBJ4eGjPdanPVXyHJziT7kuw7fPjwMtqTJM221NC/Fng7sBU4BHyy1TNibM1Tf2Wx6rqq2lZV29avX7/E9iRJoyx4emeUqnpmZjrJZ4Hfbw+ngU1DQ08GDrbpueqSpAlZ0pF+kg1DD98PzFzZswfYnuTYJKcAW4B7gHuBLUlOSXIMgzd79yy9bUnSUix4pJ/kS8B7gBOSTANXAu9JspXBKZqngF8CqKoDSW5i8Abti8BlVfVSe57LgduBo4DdVXVg7FsjSZrXYq7euWRE+fp5xl8FXDWifhtw2xF1J0kaKz+RK0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6sjRq92AXhs277p1tVuQtAge6UtSRxYM/SS7kzyb5KGh2vFJ9iZ5rN2va/Uk+XSSqSQPJDl9aJkdbfxjSXaszOZIkuazmCP9zwHnzartAu6oqi3AHe0xwPnAlnbbCVwLgz8SwJXAu4AzgCtn/lBIkiZnwdCvqj8BnptVvhC4oU3fAFw0VP98DdwFHJdkA3AusLeqnquqbwN7eeUfEknSClvqOf2TquoQQLs/sdU3Ak8PjZtutbnqr5BkZ5J9SfYdPnx4ie1JkkYZ9xu5GVGreeqvLFZdV1Xbqmrb+vXrx9qcJPVuqaH/TDttQ7t/ttWngU1D404GDs5TlyRN0FJDfw8wcwXODuCWofoH21U8ZwLfbad/bgfOSbKuvYF7TqtJkiZowQ9nJfkS8B7ghCTTDK7CuRq4KcmlwDeAi9vw24ALgCngB8CHAKrquSQfB+5t436tqma/OSxJWmELhn5VXTLHrLNHjC3gsjmeZzew+4i6kySNlZ/IlaSOGPqS1BFDX5I6YuhLUkcMfUnqiN+nL2nRVuv/TXjq6vetynpfizzSl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjqyrNBP8lSSB5PsT7Kv1Y5PsjfJY+1+XasnyaeTTCV5IMnp49gASdLijeNI/+eqamtVbWuPdwF3VNUW4I72GOB8YEu77QSuHcO6JUlHYCVO71wI3NCmbwAuGqp/vgbuAo5LsmEF1i9JmsNyQ7+AP0xyX5KdrXZSVR0CaPcntvpG4OmhZadb7WWS7EyyL8m+w4cPL7M9SdKwo5e5/FlVdTDJicDeJF+fZ2xG1OoVharrgOsAtm3b9or5kqSlW9aRflUdbPfPAjcDZwDPzJy2affPtuHTwKahxU8GDi5n/ZKkI7Pk0E/yhiRvmpkGzgEeAvYAO9qwHcAtbXoP8MF2Fc+ZwHdnTgNJkiZjOad3TgJuTjLzPP+tqv5HknuBm5JcCnwDuLiNvw24AJgCfgB8aBnrliQtwZJDv6qeAN45ov6XwNkj6gVcttT1SZKWz0/kSlJHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHVnu9+lrjdm869bVbkHSGuaRviR1xNCXpI4Y+pLUEc/pS1rzVvO9qqeuft+qrXsleKQvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSMT/xqGJOcB/xk4Cvitqrp60j2sNL/eWNJaNdHQT3IU8BngvcA0cG+SPVX18CT7kKTFWq2DuJX6zp9JH+mfAUxV1RMASW4ELgRWJPQ94pakl5t06G8Enh56PA28a3hAkp3Azvbw+0keXYE+TgC+tQLPOy5rub+13Bus7f7sbenWcn8r0ls+sazFf2KuGZMO/Yyo1cseVF0HXLeiTST7qmrbSq5jOdZyf2u5N1jb/dnb0q3l/tZyb6NM+uqdaWDT0OOTgYMT7kGSujXp0L8X2JLklCTHANuBPRPuQZK6NdHTO1X1YpLLgdsZXLK5u6oOTLKHZkVPH43BWu5vLfcGa7s/e1u6tdzfWu7tFVJVC4+SJL0m+IlcSeqIoS9JHXlNhH6S45J8OcnXkzyS5N1J3pnkfyd5MMnvJfmxOZY9L8mjSaaS7Bqqn5Lk7iSPJfmd9sbzxHpLsinJnW2ZA0k+MjTvY0m+mWR/u12wlN6W019b9qk2Zn+SfUP145Psbftub5J1k+wtyU8P7Zv9SZ5P8tE2byz7bq51LHbbk+xoYx5LsmOo/rNt26aSfDrJqMucV6y3JFvb/j2Q5IEk/2xo3ueSPDn0vFsn2Vtb/qWhZfcM1cf1+7qcffdzs5b9qyQXtXnL3ndjU1Wv+htwA/Av2/QxwHEMrhT6x632YeDjI5Y7CngceFtb7n7g1DbvJmB7m/4N4F9NuLcNwOlt+k3Anw/19jHgX6/mvmvzngJOGFH/D8CuNr0L+MSke5v1b/wXwE+Me9+NWsdith04Hnii3a9r0+vavHuAdzP4TMsfAOdPuLefAra06bcAh4Dj2uPPAR9Yrf3W5n1/jvpYfl+X29+sf+PngB9diX23rO1a7QbG8A/zY8CTtDelh+rP88M3qjcBD49Y9t3A7UOPr2i3MPiE3dGjxk2itxHPdQvw3jY9luBabn/MHfqPAhva9Abg0dXad8A5wP8aejyWfTfXOhaz7cAlwG8OPf7NVtsAfH2ucZPobcTy9/PDPwJjDa6l9MaI0B/X7+s49x2DbxX44tDjse675dxeC6d33gYcBn47yZ8l+a0kbwAeAn6hjbmYl38obMaor4XYCLwZ+E5VvTirPsne/laSzcBpwN1D5cvby+/dSz19Mob+CvjDJPdl8PUZM06qqkMA7f7EVehtxnbgS7Nq49h3c61jMds+18/dxjY9uz7J3v5WkjMYvMJ6fKh8Vdt31yQ5dhV6+5Ek+5LcNXPqhPH9vo6jv1HLzhjnvluy10LoHw2cDlxbVacB/5fBy68PA5cluY/B6ZH/N2LZub4WYsGvi5hAb4MGkzcCXwE+WlXPt/K1wNuBrQxefn9yCb2No7+zqup04Pw2/h8tsY+V6I12XvcXgN8dKo9r3823jgUXG1Eb58/dYCVL621m2Q3AF4APVdXftPIVwN8F/j6D0xf/ZhV6e2sNvvLgnwP/KcnbGfN+W2Z/M/vu7zH4PNKMse275XothP40MF1VM0fBX2ZwLvzrVXVOVf0sg7+4j8+x7KivhfgWcFySo2fVJ9kbSV7PIPC/WFVfnalX1TNV9VL7Zfwsg28vXYpl9VdVB9v9s8DNQ308037wZ34Bnp10b835wNeq6pmhnse17+Zax2K2fa6fu+k2Pbs+yd7I4M3xW4Ffraq7ZupVdagGXgB+m+XtuyX1NvQz9wTwxwxeAY/r93XZ/TX/FLi5qv56qO9x7rtledWHflX9BfB0kp9upbOBh5OcCJDkdcCvMnhzZ7aRXwtRg5NwdwIfaON2MDinPrHekgS4Hnikqj41a96GoYfvZ3DK44gts783JHnTzDSD858zfexhsM9gFfbdkEuY9RJ7XPtunnUsZttvB85Jsq6dXjqHwTnoQ8D3kpzZ/v0/OMfyK9Zb+z24Gfh8Vf3urHkzoRfgIpa375bS27qZ0yJJTgDOYvCezlh+X5fb3zzLjnvfLc9qv6kwjhuDl+r7gAeA/87gioiPMLji5c+Bq/nhm39vAW4bWvaCNuZx4N8O1d/G4EqKKQYv8Y6dZG/AP2TwEvUBYH+7XdDmfQF4sM3bQ3uDacL9vY3Bm3z3Awdm7bs3A3cAj7X741fh3/VHgb8EfnzWc45z371iHXNtO7CNwf8UNzPuw+1na4rBKRSGxj3Ufh7/y8z2Tao34F8Afz30M7cf2Nrm/VHbdw8B/xV444R7+wdt/fe3+0vH/fs6hn/XzcA3gdfNes6x7Ltx3PwaBknqyKv+9I4kafEMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktSR/w+wKWvvbB5AcgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(null_vals)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "67.6372260338609"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_df.height.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since our null is that the population mean is less than or equal to 70, we do have evidence to support this claim with our sample mean of approximately 67. This would suggest not rejecting our one-sided null alternative.\n",
    "\n",
    "If we had a null where we asked if the population mean was equal to 70, then we would be able to reject this null in favor of an alternative that suggested the population mean was actually different from 70. You will see more of this idea in the upcoming content."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compute p-value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Simulate the values of your statistic that are possible from the null.\n",
    "- Calculate the value of the statistic you actually obtained in your data.\n",
    "- Compare your statistic to the values from the null.\n",
    "- Calculate the proportion of null values that are considered extreme based on your alternative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_mean = sample_df.height.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(null_vals > sample_mean).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a new null and alternative hypothesis:\n",
    "\n",
    "$H_0: \\mu \\geq 70$\n",
    "\n",
    "$H_1: \\mu < 70$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(null_vals < sample_mean).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a new null and alternative hypothesis:\n",
    "\n",
    "$H_0: \\mu = 70$\n",
    "\n",
    "$H_1: \\mu \\neq 70$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "null_mean = 70\n",
    "(null_vals < sample_mean).mean() + (null_vals > null_mean + null_mean - sample_mean).mean()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}