Quiz/m7/spam_rf.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fit a Random Forest\n",
    "In this exercise, you'll train a Random Forest classifier to predict whether or not a text message is \"spam\". In order to train the classifier, you'll use [a dataset of SMS messages labeled as \"spam\" and \"ham\" (not spam)](https://archive.ics.uci.edu/ml/datasets/SMS+Spam+Collection). The predictions will be based on the counts of each word in the text message. Before using a Random Forest, see how well a simple Decision Tree model performs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy score:  0.9655419956927495\n",
      "Precision score:  0.8663101604278075\n",
      "Recall score:  0.8756756756756757\n",
      "F1 score:  0.8709677419354839\n"
     ]
    }
   ],
   "source": [
    "# Import our libraries\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score\n",
    "\n",
    "\n",
    "# Read in our dataset\n",
    "df = pd.read_table('SMSSpamCollection.dms',\n",
    "                   sep='\\t', \n",
    "                   header=None, \n",
    "                   names=['label', 'sms_message'])\n",
    "\n",
    "# Fix our response value\n",
    "df['label'] = df.label.map({'ham':0, 'spam':1})\n",
    "\n",
    "# Split our dataset into training and testing data\n",
    "X_train, X_test, y_train, y_test = train_test_split(df['sms_message'], \n",
    "                                                    df['label'], \n",
    "                                                    random_state=1)\n",
    "\n",
    "# Instantiate the CountVectorizer method\n",
    "count_vector = CountVectorizer()\n",
    "\n",
    "# Fit the training data and then return the matrix\n",
    "training_data = count_vector.fit_transform(X_train)\n",
    "\n",
    "# Transform testing data and return the matrix. Note we are not fitting the testing data into the CountVectorizer()\n",
    "testing_data = count_vector.transform(X_test)\n",
    "\n",
    "# Instantiate our model\n",
    "decision_tree = DecisionTreeClassifier()\n",
    "\n",
    "# Fit our model to the training data\n",
    "decision_tree.fit(training_data, y_train)\n",
    "\n",
    "# Predict on the test data\n",
    "predictions = decision_tree.predict(testing_data)\n",
    "\n",
    "# Score our model\n",
    "print('Accuracy score: ', format(accuracy_score(y_test, predictions)))\n",
    "print('Precision score: ', format(precision_score(y_test, predictions)))\n",
    "print('Recall score: ', format(recall_score(y_test, predictions)))\n",
    "print('F1 score: ', format(f1_score(y_test, predictions)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The simple Decision Tree appears to have worked reasonably well, but there is room for improvement. Notice that in order to train and test the model, we took the following steps:\n",
    "\n",
    "1. **Import** the model\n",
    "2. **Instantiate** the model\n",
    "3. **Fit** the model on training data\n",
    "4. **Test** the model on testing data\n",
    "5. **Score** the model by comparing the predictions to the true values\n",
    "\n",
    "We'll do the same steps for the Random Forest model—but this time, you fill in the appropriate code!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Step 1:** First `import` the [RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html) module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the Random Forest Classifier\n",
    "\n",
    "from sklearn.ensemble import RandomForestClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Step 2:** Then, `instantiate` the classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiate a RandomForestClassifier with\n",
    "# 200 weak learners (n_estimators) and everything else as default values\n",
    "rf = RandomForestClassifier(n_estimators=200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Step 3:** Now, `fit` (train) the model with `training_data` and `y_train`. This may take a little time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, ccp_alpha=0.0, class_weight=None,\n",
       "                       criterion='gini', max_depth=None, max_features='auto',\n",
       "                       max_leaf_nodes=None, max_samples=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, n_estimators=200,\n",
       "                       n_jobs=None, oob_score=False, random_state=None,\n",
       "                       verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fit the RandomForestClassifier model\n",
    "rf.fit(training_data, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Step 4:** Use `predict` to test the model on previously unseen data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Call model.predict() to test the model on the test data\n",
    "rf_predictions = rf.predict(testing_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Step 5:** Score the predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Random Forest scores:\n",
      "Accuracy score:  0.9827709978463748\n",
      "Precision score:  1.0\n",
      "Recall score:  0.8702702702702703\n",
      "F1 score:  0.930635838150289\n"
     ]
    }
   ],
   "source": [
    "# Calculate the accuracy, precision, recall, and F1 scores\n",
    "print('Random Forest scores:')\n",
    "print('Accuracy score: ', format(accuracy_score(y_test, rf_predictions)))\n",
    "print('Precision score: ', format(precision_score(y_test, rf_predictions)))\n",
    "print('Recall score: ', format(recall_score(y_test, rf_predictions)))\n",
    "print('F1 score: ', format(f1_score(y_test, rf_predictions)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interesting! It looks like the Random Forest outperformed the simple Decision Tree in all metrics except recall. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}