0303-klue-ml-ensemble-cv-output.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "🌱 인프런 📚 모두의 한국어 텍스트 분석과 자연어처리 with 파이썬 🐍 https://inf.run/FX4TP\n",
    "\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/corazzon/python-text-analysis/blob/main/0303-klue-ml-ensemble-cv-output.ipynb)\n",
    "\n",
    "\n",
    "## 연합뉴스 타이틀 주제 분류\n",
    "* 데이터셋 출처 : \n",
    "    * [뉴스 토픽 분류 AI 경진대회 - DACON](https://dacon.io/competitions/official/235747/overview/description)\n",
    "\n",
    "## 기초 분류 모델 만들기\n",
    "\n",
    "* 데이터 로드\n",
    "* 데이터 전처리\n",
    "* 단어 벡터화(BOW, TF-IDF)\n",
    "* 분류기 설정하기\n",
    "* 분류기로 학습시키기\n",
    "* 학습의 정확도 보기\n",
    "* 테스트 데이터 예측하기\n",
    "* 실제 데이터와 예측결과의 차이를 보기"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 라이브러리 로드"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 데이터 분석을 위한 pandas, 수치계산을 위한 numpy, 시각화를 위한 seaborn, matplotlib 을 로드합니다.\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 시각화를 위한 폰트 설정"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import koreanize_matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: title={'center': '한글'}>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa4AAACNCAYAAADvup62AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAgMklEQVR4nO3deVxU9f4/8NcsMAM4A8kmCC6Esq+aW65pqZhbi1nKvVqmGYnevH67mlfTq9lPy19x08S8dX8/aeOX2WKGS5hrWgmaYqCIJIssgsDAMPvn9wcysjMIM+cceD8fj3kgZz4z854PyGvO+XzO54gYYwyEEEKIQIi5LoAQQgjpCAouQgghgkLBRQghRFAouAghhAgKBRchhBBBoeAihBAiKBRchBBCBIWCixBCiKBQcBFCCBEUCi5CrESpVEIkEplvYrEYlZWV5vtPnTqFwMDAZo+Li4tr9LiWbk5OTjhy5Igt3w4hvEHBRYiVFBcXQ6VSmW/V1dVwdnY236/RaKDRaJo97v3334der2/zNnbsWGRkZNjy7RDCG1KuCyCkO8nJycGDDz7YbrvNmzdj2LBhLd4nEokglbb9X1MsFkOhUNxXjYQIHQUXIV3Iz88PGo0G7a1dbW9vj9TU1Pt+naKiIvj4+Nz34wkRMgouQrqYTCYDAFy9ehXbt29HRkYGvLy8MH/+fMyYMaPTz6/T6ZCZmYmoqKhOPxchQkRjXIRYQXp6OiIiIvD999/Dzc0NOTk5mDlzJjZv3tysrcFggMFgQGBgYLuTMkQiEWQyGdRqNTw9PSESiTBlyhQO3iEh3BHR9bgI6XpPPfUUtFot9u/fbx6v2rNnD+Li4lBRUQEHBwccPXoUjz76qPkxiYmJeP755zv8WmKxGGIxfQYlPQcdKiTECtRqNaZMmdJokkVMTIx5VqCDgwMAoH///sjNzeWoSkKEiT6mEWIFixcvxttvv43U1FTU1tYiOzsbS5cuxfz586FUKtt87MqVKyGXy9u8eXh44Ny5czZ6N4TwCwUXIVYwa9YsbN26FfHx8VAoFBgzZgwCAwORmJjY7mPfeecd8zlerd3CwsJw8eJFG7wTQviHDhUSYiVz587F3LlzYTQaIZFIuvS5RSJRu1PuCemuaI+LECvZtWsXVq1a1WpoSSSSFu/74osv8MADD7R5qPCXX35BeHi4td8CIbxEe1yEWElRURHy8vJavX/8+PE4f/58s+1nz57Fs88+i507d1qzPEIEi4KLECsRi8XQ6XQwGAyttunVqxcYYxCJROZt9edrEUJaRocKCbGSqKgopKSkwM7Ors3b0qVLGz1u6NCh+M9//gOpVNrmbfLkyRy9M0K4RScgE0IIERTa4yKEECIoFFyEEEIEhYKLEEKIoFBwEUIIERTOp8ObTCYUFhZCoVDQFGBCCOnBGGNQqVTw9vZu84oHnAdXYWEhfH19uS6DEEIIT+Tl5bV5hW/Og0uhUACoK7S9VbMJIYR0X1VVVfD19TXnQms4D676w4NKpZKCixBCBCj3dg0u5ldgZmTfLnm+9oaNOA8uQgghwnUsswTLP0+HWmeEl7MDhg3sbfXXpOAihBDSYSYTw45j2dh+9CoYA6L7uaC/q6NNXpuCixBCSIeoNHqsTL6Iw1eKAQDzhvfD+ukhsJfa5gwrCi5CCCEWyy6pxpK9v+F6aQ3sJWL8a1YInnmon01roOAihBBikcMZRXg1+SKqtQb0UcqxK3YIIn1dbF4HBRchhJA2mUwM7x69ioTUbADAsIG9seO5aLgrZJzUQ8FFCCGkVZW1eqz4PB3HskoBAAsfHoA1MUGwk3C3YiAFFyGEkBZlFamwZO9vyC1TQyYV460nwzA7qvUVLWyFgosQQkgzB34vxP98+TvUOiP6ujggMXYIQvs6c10WAAouQgghDRiMJmw7nIXE4zkAgNH+bkh4Ngq9new5ruweCi5CCCEAgDs1Oiz7LB2nsm8DAJaM88OqxwIg5XA8qyUUXIQQQnC5oBIvJZ1H/p1aONhJsO3pcDwe7s11WS2i4CKEkB5uf3o+/rHvErQGE/q7OiIxdggC+/B30fNOB1dSUhLi4+PRr9+9M6dlMhnOnDkDiUTS2acnhBBiJXqjCW8e/AMfn84FAIwPcMd7z0TB2dGO28La0engMhgMiImJQVJSUlfUQwghxAZuV2sR90kazt0oBwAse8QfKyYNhkTM/yvR06FCQgjpYS7mVeClpPO4ValBL5kU78yJwOSQPlyXZTEKLkII6UGSf83D2m8uQ2cwwc/dCbtjh8Dfo+0rDvONzYNLq9VCq9Wav6+qqrJ1CYRwQqM3olSlRYlKg+IqLUqqNChRaVFcpUVptRYDXR3xyiODOFv/jXRvOoMJGw9kIOnsTQDAo8Ge2D4nAgo5v8ezWtLp4BKJRDhx4gRGjx6NsrIy+Pv7Y82aNRg5cmSL7bds2YINGzZ09mUJ4Q2N3oiSKi2KVZq6r3cDqT6Y6oOqslbf5vOcAPBVegFefXQwYkf05925M0S4Sqo0WPpJGs7/eQciEfC3SYPxygR/iAUwntUSEWOMdeYJampqYDQaoVQqwRjDwYMHsWDBApw5cwaDBg1q1r6lPS5fX19UVlZCqeTv9EvS86h1hsZBpGq4l3Tvq0pjsPg5ZVIxPJQyeCrk8FDK4HH36wOO9vj03E1cKqgEAAT2UWDDjBAM93O11tsjPcT5P8uxNCkNJSotFHIpEuZGYUKgB9dltaiqqgrOzs7t5kGng6slixYtQnh4OOLj49tta2mhhHSVGq2hUfCUqhruJdXtOZVWaaHSWh5IcjsxPJVyeChk8Lj71fy9Qg7PuyGldJBCJGr5U67RxPD5rzex7VAWKtR1e2ezIr2xJiYIHkp5l7x30nMwxvDJuZvY8F0G9EaGAE8FEmOHYICbE9eltcrSPLDKGJfRaIRUSvM+iO0wxlCtNTQPoyotiu/uKdVvq9EZLX5eR3sJPJVyuCtkjcKoUUgpZVDIWg8kS0nEIswb3h8xoV7YdjgLn/1yE19fKMSRK8VYMWkwFjw8gNNLSRDh0OiNWPfNZST/lg8AmBbmha1PhcNJ1j3+Lnd6j6ugoACenp7moNq3bx9eeeUVpKWlwcvLq93H0x4XaQtjDFUaA0rrJzSYx5Hu/bt+DKlWb3kg9ZJJ7wZP4z2ihofvPJVy9OLwP/ql/Er885vLuJBXAQDw9+iFjTNCMMrfjbOaCP8VVtRiadJ5XMyvhFgEvDYlEIvH+nX6g5Ut2GyPKyUlBdu2bYNMVjcTKiAgAKmpqRaFFum5GGOoqjU0n9DQJIxKVBpo9CaLn1chlzY+PNdgz8izwWE8IXzyDPNxxldLR+HL8/l4KyUT2SXVeG7POUwL98LaaUHwcnbgukTCM2dzyhD3SRrKanRwcbTDv5+NwphB7lyX1eWsMsbVEbTH1b0wxlCh1jebwNBoHOluKOkMlgeSUi6tOzzXZM/Is8lXB/vuucxYpVqP7UeysPfsnzAxwMFOgmUT/fHC6IGQSbvneyaWY4zh49O52HzwDxhNDMFeSiTGDoFvb0euS+sQTidndAQFlzCYTAx31Drz7Lrm40h1X0tVWuiMlgeSi6OdeczIvZUJDR5KGeR29McZAK4UVmH9t5fxa+4dAICfmxPWzwjBuMHd71M1sUytzojVX/2Ory8UAqib0LPliXBBfoij4CIWMZkYytW6xuceVWkb7TGVVGlQWq2F3mj5r8oDjnbNwujehIa6QHJXUCDdD8YY9qcX4M2DmbhdXXdqyeQQT6ydFiy4T9ikc/LK1Viy9zyu3KqCRCzC6zFBWPjwAEGMZ7WEgquHM5oYymq05vGihhMaiqu0KFXVhVKpSguDyfJfAVcn+3vjRvVh1GRCg3svGeylNPvN2lQaPd49eg3/PZMLo4lBJhUjboI/Fo/1ow8EPcDJa6VY9lk6KtR6uDrZ4/3nojHyQWGf90fB1U0ZjCaU1eiaTWioD6P6cLpdrYPRwkASiQBXp/oguneYzr3JnpIbBRIvXS1WYd03l3E2p26V7/6ujlg/PRiPBHpyXBmxBsYYEk/kYGtKJkwMCPdxxq75Q+DtIvzJOhRcAmMwmnC7WtdoQkNLywaVVWth6Q6SWAS49moSRg3Hju6Gkmsvezo/SOAYY/ju91vY/P0VFFfVHT6cGOiBddOD0d+Vvyecko6p0RrwP/t+x/e/3wIAPD3EB/+aFdpt9rApuHhCbzQ1mVF3bxypuMHU77IaHSz9SUjEIrj1sm8xjBpOaHB1sqf17nqYGq0BCanX8J+TN2AwMdhLxXhprB+WjvcX5GA9uSf3dg2W7D2PrGIV7CQirJsegvnD+wl2PKslFFxWpjXUr/TdZP26qsaz7spqdBY/p0QsMo8dNQujRoEkE8TF3gh3skuq8ca3GTiVfRsA0NfFAeumB+OxYM9u9YeupziWWYLln6ejSmOAu0KGD+ZFY+iA3lyX1eUouO5Tw0tPNBxHqh87qt97uqNue6XvhuwkIrj3angybINFVhuMI/V2tBfsas2EfxhjSLlchH8duILCSg0AYOxgd7wxPRh+7r04ro5YwmRi2HEsG9uPXgVjQHQ/F3wwfwg8u+nalRRcTTS99ERJkyWELL30REP2EnHdGnaNVvtuvsjqAxRIhENqnQE7j13H7hM50BlNsJOIsGiMH5Y94g9He/6vINJTqTR6rEy+iMNXigEA84b3w/rpId16glSPCi6VRo8/bqlandBQUqVBVQcuPWEvFTdbUNW9yXlIngo5XBzt6LALEYwbt2uw4bsM/JRVCgDwcpbj9WlBmBbmRb/HPJNdUo0le3/D9dIa2EvE2DQrFHMe8uW6LKuzeXB9+OGHSEhIgFgshre3N/bs2YO+fft2WaFtOZdThmd2n223nUwqbjWM7p2PJIOzAwUS6Z4YYzj6Rwk2HshAXnktAOBhf1dsmBEiuMu3d1eHM4rwavJFVGsN8HKW44P5QxDp68J1WTZh08uaHDp0CLt378apU6fg7OyM5ORkPPHEEzh37lxXPH27vJwd0N/VEZ4KOdwbHLZrOg1cKe/8pScIETKRSIRHgz0xZpAbdh2/jg9+uo7T2WWY8u5JPD96IOInDuJ0RfyezGRiePfoVSSkZgMAhg/sjR3zouHWS8ZxZfzTJXtcTzzxBF588UVMnTrVvG3UqFHYuXMnIiMj23ws3yZnENKT5JWrsfHAFRy5O47ioZDh9WlBmBHhTR/ybKiyVo8Vn6fj2N3DuAsfHoA1MUE97vxKmx4qdHZ2RmFhIZyc7p3ouHr1avTu3RurVq3qkkIJIdZzLKsEG77NQG6ZGgAwbGBvbJwZgsA+9H/S2rKKVFiy9zfklqkhk4rx1pNhmB3lw3VZnLA0Dzod59XV1ZBKpY1CCwB8fX2Rk5PTrL1Wq0VVVVWjGyGEWxMCPHDob2OxanIA5HZi/HKjHNMSTuGNbzM6NNOWdMyB3wsxe+dp5Jap0dfFAfuWjuqxodURnQ6uiooKyOXNzymQy+VQq9XNtm/ZsgXOzs7mm69v958pQ4gQyKQSxE3wx48rxyMmrA+MJob/nsnFxHd+wpfn82HqwGLMpG0GowlbfvgDr3yaDrXOiNH+bvhu2WiE9nXmujRB6HRwyWQyaDSaZttra2vh4NB80cfVq1ejsrLSfMvLy+tsCYSQLtTXxQE75w3B3heGwc/dCberdfj7/7uIp3adweWCSq7LE7w7NTos+PhXJB6vOyK1ZJwf/rvwIfR2sue4MuHodHC5ubmhtrYW1dXVjbbn5eXBx6f5Lq9MJoNSqWx0I4Twz5hB7khZPharpwbC0V6CtJsVmPH+Kaz9+hIq1JYvZUbuuVxQienvn8Kp7NtwsJPg/eeisHpqEK0p2kGd7i2RSIThw4fjxIkTjbYfP34co0aN6uzTE0I4ZC8VY8m4B5G6cjxmRHjDxICkszcx4e2f8NkvN+nwYQfsT8/Hkx+cQf6dWvR3dcT+uFF4PNyb67IEqUtiPj4+HuvWrTNPtEhOTkZNTQ3Gjx/fFU9PCOFYH2c5Ep6NwmcvjsBgz164o9Zj9VeXMHvnaVzMq+C6PF7TG03Y8F0G/vbFRWgNJowPcMe3caNpxmYndNnKGQkJCUhMTIRYLEafPn2we/duDBw4sN3H0XR4QoRFbzTh//78J949chUqrQEiETD3IV+smhxI4zRN3K7WIu6TNJy7UXeRz2WP+GPFpMF0dYdW9Ki1Cgkhtlei0uCtg5n4Kr0AAODsYIe/Tw7Ac8P60R9mABfzKvBS0nncqtSgl0yKd+ZEYHJIH67L4jUKLkKITfyaW45132Tgj1t1QwUh3kpsnBmKIf0f4Lgy7iT/moe131yGzmCCn7sTdscOhb8HXUqmPRRchBCbMRhN+PSXm3j7UJb5SgxPRvvgH1MD4a7oOWvt6QwmbDyQgaSzNwEAjwV74p05EVDI7TiuTBgouAghNne7WottKVn44re68zMVcilefXQwYkf07/ZTvkuqNFj6SRrO/3kHIhHw6qTBiJvgT9fi6wAKLkIIZ9Jv3sG6bzJw6e4Jy4F9FNgwIwTD/Vw5rsw6zv9ZjqVJaShRaaGQS5EwNwoTAj24LktwKLgIIZwymhi++DUPWw9lokJdt97hrEhvrI4J6jaXnmeM4ZNzN7HhuwzojQwBngokxg7BADen9h9MmqHgIoTwwp0aHd4+nIVPf7kJxgAnewlWTBqMBQ8PEPRlOzR6I9Z9cxnJv+UDAKaFeWHrU+FwouuZ3TcKLkIIr1zKr8Q/v7mMC3dPWPb36IWNM0Iwyt+N28LuQ2FFLZYmncfF/EqIRcBrUwKxeKwfXcOskyi4CCG8YzIxfJmWj//1QybKaurWO5wW7oXXY4Lg7dJ8UW4+OptThrhP0lBWo4OLox3+/WwUxgxy57qsboGCixDCW5VqPbYfycLes3/CxAAHOwmWTfTHC6MHQiaVcF1eixhj+Ph0LjYf/ANGE0OwlxKJsUPg29uR69K6DQouQgjvXSmswvpvL+PX3DsAAD83J6yfEYJxg/m1B1OrM2L1V7/j6wuFAOommWx5IhwO9vwMWaGyWXAlJSUhPj4e/fr1M2+TyWQ4c+YMJJL2f6gUXIT0bIwxfH2hAG8ezESpSgug7sTdfz4ezIu9mbxyNZbsPY8rt6ogEYvwekwQFj48gMazrMDSPOj09BeDwYCYmBgkJSV19qkIIT2QSCTC7CgfTAryxHtHr+HjM7k4fKUYx6+WIm6CPxaP9YPcjps9m5PXSrHss3RUqPVwdbLH+89FY+SD3fNcNCER7lxUQki3opDbYe3jwfhh+RiM8OsNrcGE7Ueu4rH/fQI//lFs01oYY9h1/Dr++tEvqFDrEe7jjO+WjabQ4gkKLkIIrwz2VOCzF0fg389GoY9Sjpvlarzwf37DC//9FX+W1Vj99Wu0BrzyWTre+iETJgbMGeqD5CUjBTPrsSew+ZlyWq0WWq3W/H39xScJIaSeSCTC9AhvPBLogYTUa/jo1A38mFmCk9m38dJYPywd72+ViRG5t2uwZO95ZBWrYCcRYf30EMwb3o/Gs3imQ3tcly9fRmRkpPm2b98+iEQinDhxAqNHj0ZQUBCmT5+On3/+udXn2LJlC5ydnc03X1/fTr8JQkj35CSTYvXUIPywfCzGDHKDzmBCQmo2Jm0/jpTLRejKSdHHMksw4/1TyCpWwV0hw+eLR2D+iP4UWjzU6VmFNTU1MBqNUCqVYIzh4MGDWLBgAc6cOYNBgwY1a9/SHpevry/NKiSEtIkxhkMZRfjXgT9QUFELABg72B1vTA+Gn/v9X+vKZGLYcSwb249eBWPAkP4P4IN50fDoJuspCgmn53EtWrQI4eHhiI+Pb7ctTYcnhHRErc6IHceysftEDnRGE+wkIiwa44dlj/jD0b5jox8qjR4rky/i8JW6yR/zR/TDusdDYC+l4X8uWJoHVvnpGI1GSKW00CQhpOs52Evw98kBOPy3sZgQ4A69keGDn65j4jvHceD3QosPH2aXVGPWjtM4fKUY9hIxtj4Zjk2zwii0BKDTP6GCggIYDAbz9/v27UNKSgpmz57d2acmhJBWDXBzwkcLHsKevwyFb28H3KrU4JVP0zFvzzlcK1a1+djDGUWYteM0rpfWwMtZjuSXRmLOQzTeLhSd3i1KSUnBtm3bIJPVXZ47ICAAqamp8PLy6nRxhBDSFpFIhEnBnhg9yA27jl/HBz9dx5nrZZj63kksfHgA4icOgkJuZ25vMjG8e/QqElKzAQDDB/bGjnnRcOsl4+otkPtAaxUSQrqNvHI1Nh64giN3x6w8FDKsiQnCzEhvVGkMWPF5Oo5llQIAFj48AGtiggR9TbDuhhbZJYT0WMeySrDh2wzklqkBAMMG9EaJSoPcMjVkUjHeejIMs6N8OK6SNEXBRQjp0bQGI/acvIF/p16DRm8CAPR1cUBi7BCE9nXmuDrSEpstsksIIXwkk0oQN8Efs6L6YvvhqzCYTFg/PQS9ney5Lo10EgUXIaRb6+vigHfmRHBdBulCnAdX/ZFKWrOQEEJ6tvocaG8Ei/PgUqnqzregNQsJIYQAdbng7Nz6OCTnkzNMJhMKCwuhUCg6tZhl/ZqHeXl5gpjkQfVaF9VrXVSvdfXUehljUKlU8Pb2hljc+mkKnO9xicVi+Ph03bRUpVIpiB90ParXuqhe66J6rasn1tvWnlY9OvOOEEKIoFBwEUIIEZRuE1wymQzr1683r5nId1SvdVG91kX1WhfV2zbOJ2cQQgghHdFt9rgIIYT0DBRchBBCBIWCixBCiKAIKrg+/PBDhIWFISIiAlOnTkVBQUGrbVUqFebPn4/Q0FCEhIRg48aNFl/Su6t0pN5JkybB398fkZGR5tvGjRttWG2djz76CDKZDLm5uW2240P/ApbXy3X/Hjx4EBMnTkR4eDhCQ0Px0ksvQa1Wt9qe6/7taL1c929CQgLCw8MRERGBwMBAxMbG8vrvQ0fr5bp/G8rMzIRMJsOGDRtabaPX67F8+XKEhIQgJCQEy5Ytg06n67oimECkpKSwoUOHsoqKCsYYY1988QUbNmxYq+3nzp3LNm/ezBhjTKvVspkzZ7IdO3bYpFbGOl7vuHHj2JEjR2xVXovWrl3LpkyZwjw9Pdm1a9fabMt1/zLWsXq57t/jx4+z/Px8xhhjer2ePffcc2zlypWttue6fztaL9f9m5OTw2praxljdfWuXbuWRUZGttqe6/7taL1c929Djz32GJsyZQp7/fXXW23zj3/8gy1ZsoQZjUZmNBpZXFwcW7VqVZfVIJjgmj17Njt48GCjbSNHjmTp6enN2paVlTFfX19mMBjM27KyslhERISVq7ynI/Uyxv0vptFoZDt27GAGg4H179+/zSDgQ/92pF7GuO/fptLT01lYWFiL9/Ghf5tqq17G+Ne/RqORKZVKVlBQ0Ow+PvZvW/Uyxp/+/fLLL1lsbCxbv359q8FlNBpZ37592Z07d8zbKisrmbe3d6M+7wzBHCr88ccfMXbs2Ebbxo0bhyNHjjRr+9NPP2HEiBGQSCTmbYMHD0ZJSQlKSkqsXivQsXr5QCwW4+WXX27UZ63hQ/92pF4+Ki8vh1wub/E+PvRvU23Vy0dqtRoikQiurq7N7uNj/7ZVL1+o1WqsW7cOb731VpvtLly4AG9vb7i4uJi3KZVK9OvXD2lpaV1SiyCCq7q6GlKpFE5OTo22+/r6Iicnp1n7wsLCFleb9/HxwY0bN6xWZ72O1is0XPdvd7Br1y785S9/afE+PvZvW/XyTUZGBp555plWT4jlW/+2Vy9fvPnmm5g3bx68vb3bbNda/3bl3z9BBFdFRUWLn/bkcnmLA8Ydbd/V7uf1RSIR1qxZg+joaERERGDFihUoLy+3dqn3hev+vR986t9Dhw7hwoULePHFF1u8n2/92169AD/6d9WqVejTpw9CQ0Ph7e2N5cuXt9iOL/1rab0A9/17/fp17Nu3D6+++mq7bW3Rv4IILplMBo1G02x7bW0tHBwcOt2+q93P6ycnJ+Pnn39GWloaTp48CYPBgLlz51q71PvCdf/eD770b15eHhYvXoxPP/201U/XfOpfS+oF+NG/27ZtQ1FREW7fvg25XI6FCxe22I4v/WtpvQD3/bt8+XJs2rTJosPFtuhfQQSXm5sbamtrUV1d3Wh7Xl5ei5dE8fHxwc2bN5ttb619V+tovQDg7u5uPuauVCrx7rvv4uTJk6isrLR6vR3Fdf/eDz70b01NDWbNmoVNmzZh6NChrbbjS/9aWi/Aj/6t5+rqivfeew/79+9v8fX50r/12qsX4LZ/U1JSoFar8eSTT1rU3hb9K4jgEolEGD58OE6cONFo+/HjxzFq1Khm7UeOHInTp0/DaDSat2VlZcHe3t4mv5gdrbclJpMJYrGYl5MPuO7frmDr/jUajZg7dy6mTp2K2NjYNtvyoX87Um9LuP791Wq10Ol0jfqwHh/6t6m26m2JLfv3xo0byM/Pb3QO2a5du7Bnzx4MHTq02eG/yMhIXLt2DRUVFeZtVVVVyMzMRHR0dNcU1SVzE23gq6++YkOGDGGVlZWMsbrzosLCwpjRaGyx/YwZM8znaeh0OjZz5ky2detW3tabm5tr/ndlZSV7/vnn2TPPPGOTWpuyZHo51/3bkCX1ct2/cXFx7Omnn2Ymk8mi9lz3b0fr5bJ/tVoty8vLM39/584d9vTTT7N58+a1+hgu+/d+6uX697eptqbDM8ZYfHy8+Twuk8nE4uLi2Msvv9xlry+Y4GKMsffee48FBwez0NBQNmnSJJaTk8MYq/vFmzVrFrt165a5bVlZGZszZw4LDg5mgYGB7LXXXms1NPhQ7+OPP86CgoJYREQEi4iIYBs2bGBqtdqm9dYbNGhQo/8ofO3fepbUy2X/lpeXMwAsICDA/PoREREsMjKSFRUV8a5/76deLvs3Pz+fRUdHm+uNiopi27ZtYzqdjjHGv9/f+6mXT38fGGNs06ZN7I033jB//9e//pVdunTJ/H1tbS1bsmQJCwwMZIGBgWzRokVdWi9d1oQQQoigCGKMixBCCKlHwUUIIURQKLgIIYQICgUXIYQQQaHgIoQQIigUXIQQQgSFgosQQoigUHARQggRFAouQgghgkLBRQghRFAouAghhAjK/wenssinTPd82AAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 500x100 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.Series([1, 3, 5, -7, 9]).plot(title=\"한글\", figsize=(5, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 데이터 로드"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((45654, 3), (9131, 2))"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 학습, 예측 데이터셋을 불러옵니다.\n",
    "train = pd.read_csv(\"data/klue/train_data.csv\")\n",
    "test = pd.read_csv(\"data/klue/test_data.csv\")\n",
    "train.shape, test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>topic</th>\n",
       "      <th>topic_idx</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>IT과학</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>경제</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>사회</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>생활문화</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>세계</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>스포츠</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>정치</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  topic  topic_idx\n",
       "0  IT과학          0\n",
       "1    경제          1\n",
       "2    사회          2\n",
       "3  생활문화          3\n",
       "4    세계          4\n",
       "5   스포츠          5\n",
       "6    정치          6"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 토픽을 불러옵니다.\n",
    "topic = pd.read_csv(\"data/klue/topic_dict.csv\")\n",
    "topic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 전처리를 위한 데이터 병합\n",
    "* 학습, 예측 세트를 똑같이 전처리 해주기 위해 각각 같은 방법을 적용해 줄 수도 있지만\n",
    "* 두 데이터를 병합해서 전처리를 해주고 다시 나눠주는 방법을 알아보기 위해 병합 후 나눠보겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## concat\n",
    "<img src=\"https://pandas.pydata.org/docs/_images/08_concat_row.svg\" width=\"500\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(54785, 3)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 전처리를 위해 데이터 병합\n",
    "df = pd.concat([train, test])\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## 정답값 빈도수\n",
    "* train 에서 제공하는 정답값의 빈도수"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    IT과학\n",
       "1      경제\n",
       "2      사회\n",
       "3    생활문화\n",
       "4      세계\n",
       "5     스포츠\n",
       "6      정치\n",
       "Name: topic, dtype: object"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "topic['topic']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "topic\n",
       "IT과학    4824\n",
       "경제      6222\n",
       "사회      7362\n",
       "생활문화    5933\n",
       "세계      7629\n",
       "스포츠     6933\n",
       "정치      6751\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 정답값 빈도수를 확인합니다.\n",
    "topic_count = train[\"topic_idx\"].value_counts()\n",
    "topic_count = topic_count.sort_index()\n",
    "topic_count.index = topic['topic']\n",
    "topic_count"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkUAAAEnCAYAAACnqjcNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAw5UlEQVR4nO3df3yO9f////u57XRuVhul1Gqk15ZhGGELr/xIxF5+lJTfPimUYpR+SImsDJX8Kr1DSIVCXpWaX40oJUVG60VZ+VWJ2Snm3I/z+f3Dxfl1to2NnTu3c7fr5XJc6nw+n8dxPJ607d5xPI9jFmOMEQAAQAXn5+0CAAAAygJCEQAAgAhFAAAAkghFAAAAkghFAAAAkghFAAAAkghFAAAAkghFAAAAkghFAAAAkghFADzI6XQqMDBQFovFbbNarXryySfdxrZr106LFi1ya5s5c6YqVaqUb3+LxaLJkye77fvuu+9esJ5NmzYpIiIiX/sXX3yRrz0qKkpffPFFvrG9evWSv79/gTWdu82ePfuC9RQkIyNDVapUUUZGxkXtD+DiBXi7AAC+y8/PT3a7XU6n0609Pj5eR44ccWs7ffq0Tp8+7da2bds2DR06VElJSfmOXalSJbd9s7OzL1hPbm6ucnNz87Xn5eXlaz99+rQcDke+sZs3b9aKFSvUqVOn854rIODivr2+9dZbyszM1MGDB1W1atWLOgaAi0MoAuBR54YXSfrpp5+UkpKiTz/99IL7GmMUGBiowMBAT5VXbE6nUyEhIRcdes7nu+++07hx4zR48GD1799fKSkpCgkJKfHzACgYt88AlKpnnnlGderUUbt27VxXaAq6elNc9913nywWi66//vrzjvv111/z3epq06bNJZ//Ui1fvly33367EhMT9cYbbyg2NlYtWrTQzz//7O3SgAqDUASg1Lz22mtatmyZXn/9dZ0+fVpBQUGyWq2yWq3avHnzJR177ty5ysnJ0W+//XbecTVr1pQxxm37/PPPL+ncl2L37t3q37+/hgwZotmzZ2v48OGSpNdff109evRQTEyMnn32Wf35559eqxGoKAhFAErFsmXLNHLkSIWEhGj9+vUKCgpSdna2K5i0aNEi3z5+fn7KysqSdOZWWkZGhtLS0pScnKxZs2Zp6NChrrVJfn5+CggIkJ9f+fm29vLLLysuLk5XXnmlfvzxR/Xo0cOtf+zYsdq0aZO++eYbhYeH69lnn/VSpUDFwJoiAB7ldDo1adIkTZgwQXPnzlXLli31n//8R+vWrdPkyZMVFxdX6L4tWrTQoEGDNH36dElScHCwrr76aoWHh6tWrVpq0KCBgoKCilyLxWJRTk6OnE6nW3g6ffq0LBZLkY5xdvH4hW75nX1C7Xz69++vgQMHnndBdcOGDZWcnKwDBw7o1KlTRaoRwMUhFAHwmC1btmjo0KGy2+36/PPPFRsbK0n65ptvNH78eLVp00Y7d+4s8DF5Sbr//vvVp08fORwOBQUF5Vu0XVw1atTQyZMn5e/v79ZusVjUtWvXIh2jRYsW6tatm4wx5x03ZswYJSYmnnfMVVddVaRzSrrgWikAl678XGcGUO4sX75c99xzj3bu3OkKRJJUuXJlTZo0SX/88UehgeiswMBAbd269bxXXQYOHKiYmJgL1lOrVi0dP34835oip9OpFStWFGlO7733npxOp2vfjz76SLVr1853zAsFojvuuOOC7zoqaIuKiipSnQCKjytFADzm3BcsGmP0/vvv66233tKuXbv0559/qmrVqoqMjFTv3r01YsQINWrUKN8xcnNzdfvtt2vfvn264YYbCjzPwoULlZ2dXaRgVFasWrUq3/ubJOnkyZOqUqWKDhw4oOrVq+frL09rpoDyhlAEoFQMHjxYa9as0bhx49SmTRtdddVVysjI0FdffaXJkyfL6XQW+AbpsxwOR76XO55VULg4H6fTqQYNGmj9+vW6+uqri7VvSfHz8ysw4Jy9tefv7++RdyEBKBxfcQA87vfff9fcuXP1008/KTIy0tVeuXJl3X333erSpYtq166tZcuWqW/fvgUe40K3jXr37l3kepxOp3bt2nXehcvz5s3TzTff7Pqcl5dX4Dqis4GssIXXRVlwDaBsIBQBKBVn305dEH9/f1mt1vPuf77bZ61bt77E6vJr27at2+fg4OACf+3HWYXVf/b9QwDKPkIRAI+75pprdN9996ldu3ZKTEzUv//9b1155ZXKzMzUN998o8mTJ+vyyy9X9+7d8+179irLiRMnCr19VpxH6s895t9//13oMaUzt7jOPvF2vnEAfIPFXOi5UgAoAcYYvfPOO5o7d65SU1N19OhRhYSEKDIyUn379tWgQYNUuXLlAveNjo7Wrl27Cj125cqVtWrVKrVq1arI9cTGxuqbb74575iAgAA5HI5SXdyclZWlyy67TH/88YeqVatWaucFQCgC4CX/fIEiAHgboQgAAEC8vBEAAEASoQgAAEASoQgAAEASj+SXCqfTqUOHDunyyy/nJW4AABSDMUYnTpxQWFiYxx/OIBSVgkOHDik8PNzbZQAAUG7t379f119/vUfPQSgqBZdffrmkM3+hISEhXq4GAIDyw263Kzw83PWz1JMIRaXg7C2zkJAQQhEAABehNJafsNAaAABAhCIAAABJhCIAAABJhCIAAABJhCIAAABJhCIAAABJPJJfqqKfS5afrbK3ywAAwCPSk+K9XcIl4UoRAACAKngouv322/XTTz9p3bp1GjRoUJHHr169ukjjAQBA+VFhQtGoUaM0d+5ct7acnBy3TZImTJigqKgo19a4cWP98MMPbuOzs7Nd4wEAgG+oMKEoKyurSK8If/bZZ5WWlubarr32Wu3fv78UKgQAAN5UYULRX3/9papVqxZ7v7S0NCUkJCg6Olpbt271QGUAAKAsqDChaN++fQoMDCzWPp9//rkiIyO1d+9epaamqmnTph6qDgAAeFuFeCT/9OnTSk1N1YYNG9SxY0fFxsbq5MmT2rdvX6H7GGM0evRovfDCC8U+n8PhkMPhcH222+0XVTcAACg9FeJK0YoVKxQTE6MlS5YoKytLX3/99QWv/DzxxBOqU6eObrvttmKfb+LEiQoNDXVt4eHhl1I+AAAoBT4fiowxSkpK0ujRo9WzZ08lJiZecHxiYqK2bt2q1157TS+//LKio6OLtaZo9OjRyszMdG0s1AYAoOzz+dtnY8eOVc2aNdW5c2e1bdtWTZo00W233aa2bdvmG2uM0Z133il/f399/PHHCgoK0mOPPabHHntMktS6desindNms8lms5XkNAAAgIf59JWi7du367///a8WLlwoSQoODtZ7772nDRs2FDjeYrFo2LBhWrZsmS677LLSLBUAAHiZT18piomJ0datW1WpUiW3tpiYmEL3uZg1RAAAoPzz6StFktwC0T9ZrVZZrdYiHysgIKBY4wEAQPnh01eKLmTNmjWSpAMHDhQp7Kxdu1aSlJ6eTjgCAMDHWIwxxttF+Dq73X7m0fwRS+Vnq+ztcgAA8Ij0pPgSP+bZn6GZmZkKCQkp8eOfq0JfKSptqeM7ePwvFAAAXByfX1MEAABQFIQiAAAAEYoAAAAkEYoAAAAkEYoAAAAkEYoAAAAkEYoAAAAkEYoAAAAkEYoAAAAkEYoAAAAkEYoAAAAkEYoAAAAkEYoAAAAkEYoAAAAkEYoAAAAkEYoAAAAkSQHeLqAiiX4uWX62yt4uAwCAQqUnxXu7BK/hShEAAIAIRdq8ebPuvPNOt7bDhw8rMjJSERERioiIUJs2bdz6b7/9du3YsaM0ywQAAB5WYW6fORwOhYWF6ejRo27tOTk5ysnJcWu79tprtWfPnkKPVdA+AACgfKswoejo0aOqVKnSBce1atVKBw8eLLDvpptu0qpVq0q6NAAAUAZUmFD0xx9/qGbNmhcct2HDhlKoBgAAlDUVZk3Rhg0bFBgYWKSxe/fuVefOnRUdHa2GDRvq3nvv1YEDBzxcIQAA8KYKE4o+/fRT7dixo8C1QikpKYqKilL37t0lSXfffbeGDx+u1NRU7dixQ/fee6969+5d5HM5HA7Z7Xa3DQAAlG0VIhRt27ZNe/fu1dy5czVy5Mh8/a1bt1ZaWpqWLVsmY4wOHTqk1q1bu/rbt29/3oXX/zRx4kSFhoa6tvDw8JKYBgAA8CCfD0XGGD3++OMaN26c7rrrLgUGBmrGjBmFjrdYLBo8eLA6dOighQsXav78+erYsWOBYaowo0ePVmZmpmvbv39/SUwFAAB4kM8vtH7++ecVGhqqfv36SZLmzp2rli1b6sorryz0llhiYqI2b96sCRMmyGaz6dVXX9XNN99c5HPabDbZbLYSqR8AAJQOnw5FP/74o5KTk5WcnOxqCw0NVXJysj7//PPz7tuiRQs1a9ZMxhhddtllWr16tfbs2VOkJ9gAAED549OhqE6dOvryyy/ztYeFhalPnz752j/55BM99thj8vPzk7+/v6xWq4KCgvTDDz+oRo0aioyMVK1atUqjdAAAUMp8OhQVV3x8vOLjK+4vwgMAoCLz+YXWF2K1WmW1Wj2+DwAAKNsq/JWiFi1aqEWLFsXaZ82aNR6qBgAAeEuFD0WlKXV8B4WEhHi7DAAAUIAKf/sMAABAIhQBAABIIhQBAABIIhQBAABIIhQBAABIIhQBAABIIhQBAABIIhQBAABIIhQBAABIIhQBAABIIhQBAABIIhQBAABIIhQBAABIIhQBAABIIhQBAABIIhQBAABIkgK8XUBFEv1csvxslb1dBgCgnEpPivd2CT6NK0UAAAAiFAEAAEgiFJ3XU089pYiICNe2cOFCb5cEAAA8pFyvKTp16pTq1aunffv2Fdifm5urJk2aKDMzU8eOHVN4eLirLz09XdWrV1dERISSk5ML3D8pKUlJSUkeqR0AAJQt5ToUffzxxzp8+LAOHTqksLCwfP0BAQHavn271q5dq9mzZ+uDDz5w9bVr106JiYmKi4vLt19SUpLmz5+fr91isahq1apavHixatSoUaJzAQAA3lVub5+dPHlSY8aM0YMPPqiHHnqoRI/91FNPKS0tzW3btWuXOnbsKIfDoerVq5fo+QAAgPeVy1CUk5OjgQMHqkuXLnr11VdVpUoVjRw50mPn27lzp2699Vbt3btX69evl81mO+94h8Mhu93utgEAgLKt3IWiffv2qVOnTgoODtaUKVMkSfPmzVNGRoY6deqkX3/9tcTOtXv3bt1///2Ki4vT4cOHNWrUKIWEhFxwv4kTJyo0NNS1nbuWCQAAlE3lKhTNmjVLrVu3Vu/evTVv3jz5+Z0p39/fX/Pnz9eAAQPUsWNHvfTSS5KkTZs2KTo6WkOGDFFKSoqio6MVGRmp6Ohofffdd+rXr5/q16+v9PR01zn+/PNPvf7662rTpo0GDBigdu3aKSMjQ7Nnz9bcuXMVHR2tQYMGafPmzYXWOXr0aGVmZrq2/fv3e/TPBQAAXDqLMcZ4u4ii+uOPPxQaGqrAwMBCxzidTp04cUKhoaEF9gcGBur06dOF7r9kyRL99NNP6tGjh+rUqZOvPzc3V5s3b1ZISIgaNWpUpLrtdvuZK0YjlvJGawDARauIb7Q++zM0MzOzSHdrLkW5evqsKAuc/fz8Cg1ERXHvvfeetz8gIECtWrW66OMDAICyqVzdPgMAAPCUcnOlaNKkSXrrrbeKPP7XX39VzZo187XfcMMNioqKytfes2dPBQUFFescPXv21Lhx44o8HgAAlF3lak1RecWaIgBASWBNkWfXFHH7DAAAQOXo9pkvSB3fweMpFwAAXByuFAEAAIhQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIOkiQtEzzzxTaN/o0aMvqRgAAABvKXYo+uSTTwrtS05OvqRiAAAAvKXYoejEiROF9uXk5FxSMQAAAN5S7FD0r3/9S5s2bcrXnpKSoipVqpRETQAAAKUuoLg7TJ8+XXfccYcSEhJ06623Kjc3V6tXr9asWbO0evVqT9QIAADgccW+UlS7dm1t3LhRu3fvVv/+/TVw4ED99ttv+vLLL1W/fn1P1AgAAOBxFmOM8XYRvs5utys0NFThI5bKz1bZ2+UAAMqQ9KR4b5dQpp39GZqZmamQkBCPnqtIt8+2bdumm2++WZK0adMmZWdnFziuUqVKatmyZclVBwAAUEqKFIrefvttVyiaMGFCoaHIZrPps88+K7nqSknjxo310Ucf6brrrvPIeAAAUPYVKRS9+uqrrn8vb+8iWrlypZ588knXZ2OMDh48qO+//16RkZGSpOzsbLfXCbzzzjuaMGGC6/P06dPVvn171+d/jgcAAOVfsZ8+O9exY8cUFBSkoKCgkqqnxHXt2lVdu3Z1fd6xY4f69u2rWrVqFbpPnz591KdPn9IoDwAAlBHFfvrs5MmTGj58uKpUqaKbbrpJ1apVU61atfTKK694or4SdfToUQ0aNEjz58/X1KlTFRUVpaioKO3Zs8c1Ztq0aYqOjnZt9erVU2hoqH799VcvVg4AADyt2KHooYce0rFjx/Tjjz/qr7/+0smTJ/XJJ59o/fr1evrppz1RY4nYt2+fOnXqpGPHjik1NVWPP/640tLSlJaW5rqNJkkJCQlKTU11bcuWLVPVqlUVFhbmxeoBAICnFTsUffXVV3r77bd17bXXutrq1q2rZcuWadGiRSVaXEnIy8vT7NmzFR8fr1mzZmnTpk1asGCBOnbsqN27d593X2OMHn74YY0dO1ZWq7XI53Q4HLLb7W4bAAAo24odikJDQ2WxWPK122w2XX755SVSVElxOp1q1qyZUlNT9cUXX6hJkya65pprtG7dOvXp0+e8v8dNkkaNGqVdu3YVe33RxIkTFRoa6trCw8MvZRoAAKAUFDsUtWjRQjNnzszXPnXqVLVp06ZEiiopfn5+Wrt2rWbOnKkrr7zS1W6xWNS3b1/FxsZKkuLi4nTZZZe5+nNzczVs2DD9/PPPeuSRR9SjRw9lZWUV+byjR49WZmama9u/f3/JTQoAAHhEsZ8+czgcGj16tGbOnKmGDRsqJydHO3bs0OHDh9W9e3cNHDhQVqtVb7zxhifqLbaqVatKOnMr7I033tD8+fNlt9vldDpltVrVuXNnJSUlqVq1apKkLVu2aMSIEWrevLmWL18uPz8/zZgxQy1atNCcOXPUuHHjC57TZrPJZrN5dF4AAKBkFTsUDRgwQL169TrvGH9//4suyFNGjx6tH3/8UUuXLlWNGjUkScePH9eMGTPUqlUr7dixQ8YYJSUlacaMGWratKlr32HDhqlJkyaFvrQSAACUf8UORbfccosn6vC4jz/+WAsWLHAFIkmqUqWKnn32Wb3zzjv65ZdfdNNNN+nDDz8scP/yOm8AAFA0xV5TJElr1qxRmzZtVK1aNVWvXl133HGHUlJSSri0kvWf//xHkyZN0u+//+5qO3HihF5++WVVqlRJN954oxerAwAA3lbsULR48WIlJCQoISFBaWlpSk1N1ZAhQ/TQQw8VepWlLJg4caJuvfVWdevWTdHR0apbt65atmypv/76S59//rkCAop+0cxqtRZrPAAAKPssxhhTnB0aNmyo//73v6pZs6Zb+759+9S9e3d99913JVqgL7Db7WcezR+xVH62yt4uBwBQhqQnxXu7hDLt7M/QzMxMhYSEePRcxb7ckZOTky8QSVKtWrXkcDhKpChflTq+g8f/QgEAwMUp9u0zp9OpY8eO5Ws/cuSIinnRCQAAoMwodigaMmSIOnfurB9++MHVtn37dnXp0kXDhg0r0eIAAABKS7Fvn40cOVI2m02dO3dWRkaGJCksLEyjRo3SAw88UOIFAgAAlIZih6LJkyfriSee0NChQ2W32+Xn5+f6FRmdOnXSqlWrSrxIAAAAT7uoR/LPCgkJcQWiU6dOae/evSVXGQAAQCkq8pWimTNnatKkSTpy5IjbW6GlM79XLDMzk9tnAACg3Crye4pOnjypI0eOKD4+Pt8tMn9/f1WtWlXBwcEeKbK8K813LAAA4EvK5HuKgoODFRwcrAEDBhT4niIAAIDyrNhvtEbxcaUIAICLU5o/Qy/qF8ICAAD4GkIRAACACEUAAACSCEUAAACSCEUAAACSCEUAAACSCEUAAACSLuIXwuLiRT+XLD9bZW+XAQD4h/SkeG+XgDKAK0UAAAAiFEmSbr/9du3YseOC49q3b6+ffvpJq1ev1qBBg0qhMgAAUFp8PhRt2bJFUVFRblvVqlW1cOFC15icnBzl5ORIkuLj493GNm3aVEeOHJEkZWdnKycnx/VPAADgO3x+TVFcXJzS0tLc2po3b65nn31WkydPliTt27fP1ffJJ5+4/v3kyZOqVq2aKlWqVDrFAgAAr/H5UPRPCxYsUFhYmL788ktXW+vWrQsc+9lnnykuLk6hoaGlVB0AAPCWChWKVq5cqQkTJiglJaVI4xctWqS+fft6tigAAFAmVIhQdOLECb344otasWKFPv74Y61bt05Tpkxx9Z97++ysb7/9VsnJyXr33XfVsmVLHT9+vMBxBXE4HHI4HK7Pdrv90icBAAA8yucXWi9ZskR169ZVVlaWtm3bpqioKA0YMECpqamurWnTpm77ZGdna9iwYapdu7aWLFmiTZs2FTiuMBMnTlRoaKhrCw8P98TUAABACfL5UFS/fn19/fXXevXVVxUcHFzgmDp16ritG0pISFBcXJzWrVunl156STt37izWOUePHq3MzEzXtn///kuaAwAA8Dyfv31Wt25d178vXrxYr7/+uo4cOSJjjCwWi9q2basxY8bo+uuvl9PpVEJCgg4fPqwPPvhAAQEBWrRokXr06KGVK1cW+Zw2m002m80T0wEAAB7i86HorNdee03vvvuu5s6dq9q1a0uSTp06pbffflutW7fW1q1btWHDBuXk5GjZsmXy9/eXJMXExOjdd99VSEiIN8sHAAAeVmFC0UcffaTHH3/cFYgkqXLlyhoyZIhWrFihrVu3qlu3burWrVu+fRs3blyKlQIAAG/w+TVFZ3Xu3FnTpk1ze4LM4XBo4cKFSk9PV7NmzS54DKvVKqvV6skyAQCAl1SYK0VDhw5VSEiIBgwYoGPHjsnpdMrPz0+tWrXSunXrVKVKlQseY82aNZKk9PR0whEAAD7GYowx3i7C19nt9jOP5o9YKj9bZW+XAwD4h/SkeG+XgEKc/RmamZnp8fW9FeZKUVmQOr4DC7YBACijKsyaIgAAgPMhFAEAAIhQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIEkK8HYBFUn0c8nys1X2dhkA4HPSk+K9XQJ8AFeKAAAAVAZCkTEmX1uNGjUkSZs3b1bv3r1d7dnZ2WrRooXi4uIUFxen6dOnu/o2btyoHj16lEhNM2fOVEREhGubMWOGq2/z5s266667SuQ8AACg7PD67bM5c+Zo9+7dmjp1qqvt1KlTkqScnBxlZ2dLku6++26lp6e77btw4UItXLhQPXv2VExMjHJycvIdf9KkSZo6daqqVKmSry8zM1MDBw7UCy+84Nb+yCOP6JFHHimw3nNrAgAAvsProWjnzp266aabLjjugw8+kNPp1Jdffqm9e/cqKChIsbGxuuGGGyRJa9euLXC/ffv26aWXXlLfvn3z9X344Yf64IMPXJ+//vpr9enTp9AannrqKUVERFywVgAAUP54NRQ5HA5t2bJF3377re6//34FBQVJko4fP66oqCidOnVKzZo1c41t06aNoqKiVL9+fR0+fFjTp09Xx44d9cwzz5RIPbGxsdq7d+95x6SkpJTIuQAAQNni1VD05JNPavz48Tpy5Ih69eqlpUuXqlKlSqpSpYrS0tKUkpKimTNnSjoTlA4ePKg1a9YoODhYkhQXF6dnnnnGFYrWrVunqKgoNWzYUEuWLJEkhYWFadSoUUpMTMx3/szMTN1///352pcsWaLJkycrOztbAQEB6tWrl5544glP/TEAAIAywCuhyBijsWPHyuFwqGPHjpKk/fv3q0WLFvryyy8L3Kd69eqaMmWKunfvrpMnT8pqteqqq67S3LlzXWNuu+02ffjhh277jR07VmPHji1ybd9//70SExO1du1aVa9eXadPn1b//v01b948DRw4sEjHcDgccjgcrs92u73I5wcAAN7hlVD0+++/Kzc3V7NmzXK1jRkzRr1795bVanXdRjvr0KFD+uGHHxQSEqIRI0Zo6tSpioyMVHR0tN5//3399ttvateunds+kyZN0ltvvVXkmnr27Klx48bpf//7n+Li4lS9enVJUmBgoNq1a6fU1FTX2JSUFEVFRalevXpatmxZvmNNnDhR48ePL/K5AQCA91lMQc/Ee5nT6ZSfn59++uknffTRR+rUqZNWr14tSbJYLKpUqZKCgoIUHBysatWq6brrrpO/v782b96s/v37n/fYN9xwg7Zs2aJrrrmmwP6MjAw1a9ZM/+///T/FxsYqPT1dU6ZM0dKlS9WwYUOlpKTopZde0scff1zoOQq6UhQeHq7wEUt5eSMAeAAvb/RddrtdoaGhyszMVEhIiEfP5fWnz5xOp9566y0tWrRIf/31l3Jzc+Xn56eoqCgNHjxYo0aNkiTVrVtXb7zxht58880Cj+Pv768HHnggX3tmZqZCQ0OLXE/VqlX1zTff6O2339bdd9+tESNGaO3atQoPDy/yMWw2m2w2W5HHAwAA7/N6KHr00UeVlpamN9980/W4uzFGmzdv1kMPPaSDBw+61vIMGTJEQ4YMKfA4a9eu1bRp0zRo0CC39o4dO2rOnDmqW7euJMnPz08Wi+W8NVWtWlXDhw/X9OnT9cADD8hiseirr75SWlqarrrqqkudMgAAKIO8Hoo++eQTvf32227v/7FYLGrZsqWef/55zZkzxxWKPv/8cz399NMFvqTRbrerW7du+dqdTqecTqfr886dO11Pr/3TtGnT3K5EWSwW3XXXXapevbpq1Kihf/3rX7riiisudqoAAKAM83ooio+PV1JSkqZPn+769R6S9O233+qFF17Q0KFDXW0bN25U165d9dRTTxX5+P+8KlRYIJKkhIQEJSQknPd4vKcIAADf5PVQ9PLLL2vOnDnq06ePjh8/rry8PElSZGSkxo8fr/j4/3/xXGRkpB5//HEtXry4wGNZrVZt3brVrS06OlpdunRR5coFL3C2WCxav359kW+LWa1WWa3WIo0FAADlR5l8+szXnF05z9NnAOAZPH3muyrU02cVSer4Dh7/CwUAABfHz9sFAAAAlAWEIgAAABGKAAAAJBGKAAAAJBGKAAAAJBGKAAAAJBGKAAAAJBGKAAAAJBGKAAAAJBGKAAAAJBGKAAAAJBGKAAAAJBGKAAAAJBGKAAAAJBGKAAAAJBGKAAAAJEkB3i6gIol+Lll+tsreLgOAj0lPivd2CYBP4EoRAACAKngoeuedd/Too4/max83bpwiIiJcW1JSkts+I0eOLM0yAQBAKSiXt8/at2+vli1bavny5ZKkY8eO6e+//1aNGjUkSe3atdNLL72kfv36aevWrW77+vv7a9SoUbrvvvuUk5Oj7OzsfMcfN26cxo0bV+C5c3JylJOTU7ITAgAAXlcuQ1F2draaN2+usWPHSpLmz5+vtWvXatGiRW7j3n777Xz7rlq1SgsWLNB9992Xr2/x4sWFhiFJ6tOnj8LDwy+teAAAUCaVy1B0KWw2m/z9/Qvs69mzp3r27Knc3FytWrVKe/fu1XXXXacuXbooKChI0pkABgAAfE+FW1Nkt9sVEhJSaP+JEyfUvHlzbdq0SWFhYfrf//6nZs2a6eDBg64x7733nqKjo1lbBACAD6kQV4pee+01BQYGauDAgTp48KDCwsIKHZucnKzGjRtr8uTJrjar1aoFCxbo6aefliT16tVLM2fOLPQYDodDDofD9dlut5fALAAAgCdViCtFf/75p44dOyZJSk1NVe3atV1977zzjqKiojR8+HBJUq1atbR9+3adPHlS0pmF1V9//bUiIiKKfL6JEycqNDTUtbEOCQCAsq9CXCk61/PPP+92+6xPnz5uV31uvvlmPfzww2rTpo2sVqtOnz6te++9V/fcc0+RzzF69Gi3R/3tdjvBCACAMq7ChaKrr75aR48e1Y4dO2SMKXBMv3791K9fPxljZLFY3Po6d+6s1q1bn/ccNptNNputpEoGAAClwGdD0bhx4/Txxx9Lkg4ePKi8vDzNnz9f/v7+uuaaa1S7dm3VrVu3wH3/7//+T6+88kqBfZmZmRozZoweeeQRj9UOAABKn8+GoqeeekrDhg2TJFWqVEnBwcHy83NfQlXY4/WDBw/W4MGDC+ybP3++tm3bVqK1AgAA7yuXoahSpUqqVKlSoZ8lKTAwUIGBgR45f2G33QAAQPlVLkPR6tWr3T737t1bvXv3LvZxrFarrFZrsfYJCAhQQEC5/GMDAADnYTFc9igWY4ycTmehb8UuiN1uP/No/oil8rNV9mB1ACqi9KR4b5cAeMzZn6GZmZnnfflySeCSRzFZLJZiBaJzpY7v4PG/UAAAcHEqxMsbAQAALoRQBAAAIEIRAACAJEIRAACAJEIRAACAJEIRAACAJB7JLxVnXwVlt9u9XAkAAOXL2Z+dpfFaRUJRKTh69KgkKTw83MuVAABQPh09elShoaEePQehqBRcccUVkqTffvvN43+h3mS32xUeHq79+/f79EsqmadvqSjzlCrOXJmnb8nMzFSNGjVcP0s9iVBUCvz8zizdCg0N9en/cM8KCQlhnj6EefqeijJX5ulbzv4s9eg5PH4GAACAcoBQBAAAIEJRqbDZbHruuedks9m8XYpHMU/fwjx9T0WZK/P0LaU5T4spjWfcAAAAyjiuFAEAAIhQBAAAIIlQBAAAIIlQVCrefPNN1a9fXw0bNlTHjh118OBBb5dUZPPmzZPNZlN6erpb+48//qhWrVopJiZGjRo10vLly936c3JylJCQoHr16qlevXoaNmyYsrOz3casXLlSjRo1UkxMjG699Vbt2rXL09PJZ9WqVbrtttvUoEEDRUdH68EHH9SpU6dc/b4wz+nTp6tBgwZq2LChoqKi1K9fP7f/Bn1hjv+UlpYmm82m8ePHu9oOHz6s+Ph4NWzYUPXr19fs2bPd9jHGKDExUfXq1VN0dLR69eqV71fzbN68WbGxsYqJiVFsbKy++OKLUpnPuRYtWqQrrrhCMTExri02NlZ5eXmSfGeekpSVlaXnnntON998sxo1aqQ6depo/fr1rn5fmGteXp7r/OduISEh+vTTTyX5zteo3W7X8OHD1bBhQ8XExKhFixZau3atq79MzNPAoz777DPTpEkTc/z4cWOMMUuWLDHNmjXzclVF88wzz5g77rjDVK9e3ezZs8fVnpWVZSIjI01KSooxxpjDhw+bm266yezYscM15qmnnjJDhgwxeXl5Ji8vzzz88MPm8ccfd/WnpqaaiIgIc/DgQWOMMV988YWJiIgwp06dKqXZnbFhwwZz4MABY4wxOTk5pnfv3uaxxx4zxvjOPH/55ReTlZVljDkzx2eeecbExMQYY3xnjv/Uvn17c8cdd5gxY8a42m655RazaNEiY4wxdrvdxMbGmk8++cTVP3v2bBMfH29Onz5tjDFm0qRJpkePHq7+P/74w9SoUcPs2rXLGGNMWlqaqVmzpjl8+HBpTMnlrbfeMn369Cm031fmmZOTY1q1amXGjRvnqtXpdJqcnBzXGF+Z6z9lZWWZsLAwk5GR4VNfox06dDAvvviiycvLM8YY8+2335prr73WpKenl5l5Eoo87M477zSrVq1ya7vlllvM999/752CiigvL8/MmjXL5Obmmpo1a7qFopUrV5p77rnHbfwbb7xhEhISXPted911JiMjw9WfmZlpwsLCTG5urjHGmJEjR5rXXnvN7Ri9evUyK1as8Mh8iur777839evXN8b47jzz8vJMSEiIOXjwoE/O8YMPPjD9+vUzzz33nCsU7dixI9//jCQnJ5uuXbu6Pjdq1Mjs3r3b9TkvL8/UqFHD/PXXX8YYY6ZNm2aeeOIJt2OMHj3aTJ061TMTKcT5QpEvzXPevHmmS5cuhfb70lz/af78+aZfv37GGN/6PmS1Wl0XCM6Kj483y5YtKzPz5PaZh61bt0633nqrW1urVq20Zs0aL1VUNH5+fho6dKj8/f3z9a1du1atWrVyazt3Ttu3b1dYWJiqVKni6g8JCVGNGjX03XffFekY3nLs2DEFBgZK8t15njp1ShaLRVdeeaXPzfHUqVMaO3askpKS3NoLqvHf//631q9fL2OMjh49qoMHD6pOnTqufj8/PzVv3tx1u6YszbMwvjTPxYsXa8iQIYX2+9Jc/+mNN97QoEGDJPnW96G4uDi98sorrs8bN27Ul19+qWbNmpWZeRKKPOjvv/9WQECAgoOD3drDw8P1yy+/eKmqS3fo0CGFh4e7tZ07p4L6izKmLPy5zJ49W/3795fkm/PctWuX7r33XteL0Hxtji+++KL69OmjsLAwt/aCagwKClJgYKD+/PNPHT58WNdff32+45XVeRbGl+a5Y8cOBQUFqXv37mrQoIHatm2rzz77zNXvS3M9186dO5WRkaF///vfknzr+9CCBQu0ZMkSdejQQcOHD9ddd92lRYsW6frrry8z8yQUedDx48ddVx3OFRgY6LaYt7wpaF6BgYE6ffq0jDFFmndhx/Dmn0tycrK2b9/u+j80X5rn448/rmuuuUbR0dEKCwtTQkLCeesrj3P8+eeftWzZMj366KP5+i40j/I0T4vFoo0bN6ply5aqU6eOOnfurK+++qrQGs+tszzN8+jRo0pMTNQLL7ygH374Qa+++qoGDx6slJSUQus8t9byNNdzzZ49Ww888IDrsy99jdasWVMPP/yw1q1bpxkzZqh9+/Zq2rTpeWss7XkSijzIZrPp9OnT+dqzsrIUFBTkhYpKRkHzysrKks1mk8ViKdK8CzuGt/5c9u/fr8GDB+vdd991vUrel+Y5ZcoU/f777/rrr78UGBio++6777z1lcc5JiQkKDExscBvnBeaR3ma5913363U1FRt2rRJu3fv1oMPPqguXbpoz549PjVPPz8/PfHEE4qKipIkNWjQQCNHjtS8efMKrfPcWsvTXM86efKk3n//fQ0YMMDV5ktfo3379tXChQu1du1a/fzzz7JarWrQoIEOHDhQZuZJKPKgatWqKSsrS3///bdb+/79+wu8rFteXH/99frtt9/c2s6dU0H9RRnjrT+XkydPqlu3bkpMTFSTJk1c7b42T0m68sorNW3aNK1YsUKZmZk+M8fPPvtMp06dUvfu3QvsL6jGs1+bV199dbmZpyQFBwcrJCRE0pmrRvHx8eratas+/fRTn5rn1VdfrZtuusmtLSIiQkeOHJHkW3+nZy1evFht27ZVtWrVXG2+8jW6d+9erVq1SmvXrlXr1q114403asGCBerQoYNee+21MjNPQpEHWSwWxcbGauPGjW7tGzZsUPPmzb1U1aVr3ry5NmzY4NZ27pxiYmK0Z88eHT9+3NVvt9uVlpamxo0bF+kYpSUvL089e/ZUx44d1a9fP7c+X5rnuRwOh7Kzs5WXl+czc9y3b58OHDjg9p6X2bNna86cOWrSpEmBNW7cuFFNmzaVn5+frr32Wl122WXavXu3q9/pdGrTpk2ueZSFeRYmLy9PAQEBPjXPpk2baufOnW5te/bsUUREhKSC6yyvcz1r9uzZrtv3Z/nK16jdbldYWJhCQ0Pd2uvXr6+MjIyyM88iP6eGi7J8+XJz8803m8zMTGPMmfcU1a9f3/WehvLgn4/k//3336ZGjRpu75OIiIgwW7ZscY0ZPny4630STqfTPPzww2bo0KGu/q1bt5obb7zR9T6JTZs2mfDwcHPixIlSmtUZDz/8sOnRo4dxOp35+nxhng6Hw+zfv9/1OSMjw/To0cP1SLcvzLEw5z6S73Q6TUxMTL532ixdutQ1/pVXXjHx8fHG4XAYY86806ZTp06u/v3795uwsDC3d9pcd911Jj09vbSmZIwx5sCBA27v6vnggw/MNddcYw4dOuRT81yzZo2pV6+e651Bu3fvNjVr1jQ//vijMca3/k6NMWbbtm3mxhtvzPe9yFe+RnNzc02zZs3MK6+84vr5t3fvXlO7dm2zadOmMjNPQlEpmDZtmqlbt66Jjo427dq1M7/88ou3SyqWyMjIfN8ktm/fbpo3b24aNGhgoqOjzTvvvOPWn5WVZYYMGWKioqJMVFSUeeCBB/K9QGvx4sWmfv36pkGDBiYuLs589913Hp/LuY4dO2Ykmdq1a5uGDRu6tpiYGPP7778bY8r/PA8cOGAaN27smmOjRo3MlClTTHZ2tmtMeZ9jYRITE824ceNcn9PT00379u1NdHS0qVOnjnn55ZfdxjudTjNmzBgTFRVl6tata7p3726OHDniNmbdunWmcePGpkGDBqZRo0Zm9erVpTKXc82ZM8fUrl3bNGjQwDRo0MD06NHD7V08vjJPY4x58803TWRkpKldu7Zp3Lix+fTTT936fWmuDz30kJkyZUqBfb7yNXrkyBEzePBgU79+fRMTE2OaN29uVq5c6eovC/O0GGNM0a8rAQAA+CbWFAEAAIhQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIIlQBAAAIEn6/wDFBicrGyVe/AAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 600x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 정답값 빈도수를 시각화 합니다.\n",
    "topic_count.sort_values().plot(kind=\"barh\", figsize=(6, 3), title=\"정답 빈도 수\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 문자 전처리"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 정규표현식\n",
    "import re\n",
    "\n",
    "def preprocessing(text):\n",
    "    # 개행문자 제거\n",
    "    text = re.sub('\\\\\\\\n', ' ', text)\n",
    "    # 특수문자 제거\n",
    "    # 특수문자나 이모티콘 등은 때로는 의미를 갖기도 하지만 여기에서는 제거했습니다.\n",
    "    # text = re.sub('[?.,;:|\\)*~`’!^\\-_+<>@\\#$%&-=#}※]', '', text)\n",
    "    # 한글, 영문, 숫자만 남기고 모두 제거하도록 합니다.\n",
    "    #text = re.sub('[^가-힣ㄱ-ㅎㅏ-ㅣa-zA-Z0-9]', ' ', text)\n",
    "    # 한글, 영문만 남기고 모두 제거하도록 합니다.\n",
    "    text = re.sub('[^가-힣ㄱ-ㅎㅏ-ㅣa-zA-Z]', ' ', text)\n",
    "    # 중복으로 생성된 공백값을 제거합니다.\n",
    "    text = re.sub('[\\s]+', ' ', text)\n",
    "    # 영문자를 소문자로 만듭니다.\n",
    "    text = text.lower()\n",
    "    return text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# map을 통해 전처리 일괄 적용\n",
    "df[\"title\"] = df[\"title\"].map(preprocessing)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 불용어 제거"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 불용어 제거\n",
    "def remove_stopwords(text):\n",
    "    tokens = text.split(' ')\n",
    "    stops = [ '합니다', '하는', '할', '하고', '한다', \n",
    "             '그리고', '입니다', '그', '등', '이런', '및','제', '더']\n",
    "    meaningful_words = [w for w in tokens if not w in stops]\n",
    "    return ' '.join(meaningful_words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"title\"] = df[\"title\"].map(remove_stopwords)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 학습, 예측 데이터셋 분리\n",
    "* 전처리가 되었다면 concat 으로 합쳐주었던 데이터프레임을 다시 학습과 예측 데이터셋으로 분리합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'topic_idx'"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 정답값으로 사용할 topic_idx 를 변수에 담아 재사용 합니다.\n",
    "label_name = \"topic_idx\"\n",
    "label_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((45654, 3), (9131, 2))"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.shape, test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((45654, 3), (9131, 2))"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# label_name 을 기준으로 학습, 예측 세트를 분리합니다.\n",
    "# label_name 이 있다면 학습 세트, 없다면 예측 세트가 됩니다.\n",
    "df_train = df[df[label_name].notnull()].copy()\n",
    "df_test = df.loc[df[label_name].isnull(), [\"index\", \"title\"]].copy()\n",
    "df_train.shape, df_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        4\n",
       "1        4\n",
       "2        4\n",
       "3        4\n",
       "4        4\n",
       "        ..\n",
       "45649    1\n",
       "45650    2\n",
       "45651    1\n",
       "45652    2\n",
       "45653    2\n",
       "Name: topic_idx, Length: 45654, dtype: int64"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 기출문제의 정답을 만들어 줍니다.\n",
    "# 잘 나뉘었는지 정닶 값의 빈도수를 확인해 봅니다.\n",
    "y_train = df_train[label_name].astype(int)\n",
    "y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 테스트의 정답은 모두 결측치 입니다.\n",
    "# Dacon 사이트에 예측 값을 제출해야 맞고 틀린 갯수를 알 수 있습니다.\n",
    "# y_test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 벡터화\n",
    "* 머신러닝이나 딥러닝 알고리즘은 문자를 이해할 수 없습니다. 내부에서는 수치 계산이 이루어지기 때문에 문자를 숫자로 변경해 주어야 합니다.\n",
    "\n",
    "\n",
    "### TF-IDF(Term Frequency - Inverse Document Frequency)\n",
    "\n",
    "정보 검색과 텍스트 마이닝에서 이용하는 가중치로, 여러 문서로 이루어진 문서군이 있을 때 어떤 단어가 특정 문서 내에서 얼마나 중요한 것인지를 나타내는 통계적 수치이다. 문서의 핵심어를 추출하거나, 검색 엔진에서 검색 결과의 순위를 결정하거나, 문서들 사이의 비슷한 정도를 구하는 등의 용도로 사용할 수 있다.\n",
    "\n",
    "TF(단어 빈도, term frequency)는 특정한 단어가 문서 내에 얼마나 자주 등장하는지를 나타내는 값으로, 이 값이 높을수록 문서에서 중요하다고 생각할 수 있다. 하지만 단어 자체가 문서군 내에서 자주 사용 되는 경우, 이것은 그 단어가 흔하게 등장한다는 것을 의미한다. 이것을 DF(문서 빈도, document frequency)라고 하며, 이 값의 역수를 IDF(역문서 빈도, inverse document frequency)라고 한다. TF-IDF는 TF와 IDF를 곱한 값이다.\n",
    "\n",
    "IDF 값은 문서군의 성격에 따라 결정된다. 예를 들어 '원자'라는 낱말은 일반적인 문서들 사이에서는 잘 나오지 않기 때문에 IDF 값이 높아지고 문서의 핵심어가 될 수 있지만, 원자에 대한 문서를 모아놓은 문서군의 경우 이 낱말은 상투어가 되어 각 문서들을 세분화하여 구분할 수 있는 다른 낱말들이 높은 가중치를 얻게 된다.\n",
    "\n",
    "\\begin{equation*}\n",
    "\\text{tfidf}(w, d) = \\text{tf} \\times (\\log\\big(\\frac{N + 1}{N_w + 1}\\big) + 1)\n",
    "\\end{equation*}\n",
    "\n",
    "\n",
    "* 출처 : [tf-idf - 위키백과, 우리 모두의 백과사전](https://ko.wikipedia.org/wiki/Tf-idf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {color: black;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>TfidfVectorizer(max_features=2000)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">TfidfVectorizer</label><div class=\"sk-toggleable__content\"><pre>TfidfVectorizer(max_features=2000)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "TfidfVectorizer(max_features=2000)"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TfidfVectorizer 로 벡터화 합니다.\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "# fit 으로 변환할 어휘를 학습합니다.\n",
    "\n",
    "tfidfvect = TfidfVectorizer(max_features=2000)\n",
    "tfidfvect.fit(df_train[\"title\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<45654x2000 sparse matrix of type '<class 'numpy.float64'>'\n",
       " \twith 127949 stored elements in Compressed Sparse Row format>,\n",
       " <9131x2000 sparse matrix of type '<class 'numpy.float64'>'\n",
       " \twith 25576 stored elements in Compressed Sparse Row format>)"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# transform\n",
    "X_train = tfidfvect.transform(df_train[\"title\"])\n",
    "X_test = tfidfvect.transform(df_test[\"title\"])\n",
    "X_train, X_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['afc', 'ag', 'ai', ..., '희망', '희생자', '힐러리'], dtype=object)"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 단어 사전 feature_names\n",
    "feature_names = tfidfvect.get_feature_names_out()\n",
    "feature_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "afc      15.177806\n",
       "ag       21.725468\n",
       "ai      117.674345\n",
       "ceo      21.679064\n",
       "cj       17.911939\n",
       "           ...    \n",
       "흑자전환     16.867666\n",
       "흥국생명     29.266299\n",
       "희망       27.503375\n",
       "희생자      17.099391\n",
       "힐러리      56.208043\n",
       "Length: 2000, dtype: float64"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# np.sum 으로 위에서 구한 TF-IDF 값을 단어별로 더합니다. axis=0 으로 합니다. \n",
    "\n",
    "df_train_vocab = pd.DataFrame(X_train.toarray(), columns=feature_names)\n",
    "df_train_vocab.sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "대통령    635.826447\n",
       "신간     515.872746\n",
       "게시판    359.763017\n",
       "억원     352.378423\n",
       "출시     336.948266\n",
       "감독     277.461546\n",
       "트럼프    276.234327\n",
       "이란     262.322975\n",
       "개발     251.091199\n",
       "개최     246.356875\n",
       "한국     243.032916\n",
       "kt     242.074555\n",
       "분기     234.710982\n",
       "만에     196.365893\n",
       "정부     193.406648\n",
       "김정은    190.230796\n",
       "올해     190.204747\n",
       "코로나    189.042454\n",
       "그래픽    186.746914\n",
       "월드컵    185.458872\n",
       "dtype: float64"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TF-IDF 합계 값이 높은 상위 20개 어휘\n",
    "df_train_vocab.sum().nlargest(20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: >"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAz0AAAF4CAYAAAB+YpeDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABH9ElEQVR4nO3deZyO9eL/8fc92z1jxlhjMJMlZRmMyXaaL6aS40RyflEphcoWiRLZsiW7TnVOTI5T2c5p+ZKEhIQoSmLGQclwTJY4zIZZmPv6/eE7V3PPgmHc1+2a1/PxuB/c13XNfb/v/X5f13V/LodhGIYAAAAAwKZ8rA4AAAAAADcSpQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANian9UBisvlcunYsWMqW7asHA6H1XEAAAAAWMQwDKWnp6t69ery8Sl6e85NV3qOHTumiIgIq2MAAAAA8BJJSUkKDw8vcv5NV3rKli0r6dINCw0NtTgNAAAAAKukpaUpIiLC7AhFuelKT+4ubaGhoZQeAAAAAFf82QsDGQAAAACwNUoPAAAAAFuj9AAAAACwNUoPAAAAAFuj9AAAAACwNUoPAAAAAFuj9AAAAACwNUoPAAAAAFuj9AAAAACwNUoPAAAAAFuj9AAAAACwNUoPAAAAAFvzszrAjVRr5KoSu6zD0zqV2GUBAAAA8By29AAAAACwNUoPAAAAAFuj9AAAAACwNUoPAAAAAFuj9AAAAACwNUoPAAAAAFuj9AAAAACwNUoPAAAAAFuj9AAAAACwNUoPAAAAAFuj9AAAAACwtWKXnoyMDI0fP17NmjVTdHS0GjRooA0bNpjzjx8/rk6dOikqKkqNGzdWXFyc298bhqHJkycrMjJSjRo10mOPPaa0tLTrvyUAAAAAUIhilZ6LFy/q/vvvl4+Pj7755hv9+OOP2rt3r9q2bWsu07VrVz3++OPavXu3vvnmG73//vtavXq1OX/evHnatm2bdu7cqT179ig6Olp9+vQpuVsEAAAAAHkUq/QsWrRI5cqV0/jx4+V0OiVJDodDfn5+kqT4+Hjl5OSoR48ekqSyZctq0qRJmjdvnnkZ77zzjmbOnGn+/UsvvaTt27fr9OnTJXKDAAAAACCvYpWeDz74QP379y9y/vr16xUbG+s2rU2bNtqwYYMMw9Dp06d19OhRNWjQ4PcAPj6KiYlx20Uur6ysLKWlpbmdAAAAAOBqFav07N69W0FBQeratauaNGmie++9V2vWrDHnHzt2TBEREW5/ExQUpMDAQJ08eVLHjx9XeHh4gcuNiIhQYmJiodc5depUlStXzjzlv3wAAAAAuJxilZ7Tp09r8uTJeu211xQfH6833nhD/fr108aNGyVJKSkpCgwMLPB3gYGBOn/+/BXnF2bUqFFKTU01T0lJScWJDAAAAKCUK1bp8fHx0YgRI1S/fn1JUpMmTfTCCy/o3XfflSQ5nU5lZmYW+LuMjAwFBQVdcX5hnE6nQkND3U4AAAAAcLWKVXqqVKmiO+64w21a3bp1derUKUlSeHi4jhw54jY/IyNDZ8+eVZUqVQqdL0lJSUmF7vYGAAAAANerWKWnRYsWSkhIcJt24MAB1a1bV5IUExOjTZs2uc3fvHmzWrRoIR8fH1WrVk0hISHau3evOd/lcmnLli2KiYm51tsAAAAAAEUqVukZOHCgRo8erRMnTkiS9u3bp7feekuDBg2SJLVt21YXLlzQkiVLJEnp6ekaP368Bg8ebF7G888/rxEjRig7O1uSNGvWLEVFRalOnTolcoMAAAAAIC+/4ix83333aejQoWrbtq18fHwUHBysuLg48zc+DodDy5cvV79+/TRt2jTl5OSoT58+evjhh83LGDp0qE6fPq2oqCj5+PioQYMGWrBgQcneKgAAAAD4Pw7DMAyrQxRHWlqaypUrp9TU1CsOalBr5KoSu97D0zqV2GUBAAAAuH5X2w2KtXsbAAAAANxsKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDWKD0AAAAAbI3SAwAAAMDW/KwOUBrVGrmqRC/v8LROJXp5AAAAgJ2wpQcAAACArVF6AAAAANhasUrP4sWLVbFiRTVt2tQ8tWrVSjk5OZKk48ePq1OnToqKilLjxo0VFxfn9veGYWjy5MmKjIxUo0aN9NhjjyktLa3kbg0AAAAA5FOs0nPx4kV17NhRu3btMk/bt2+Xr6+vJKlr1656/PHHtXv3bn3zzTd6//33tXr1avPv582bp23btmnnzp3as2ePoqOj1adPn5K9RQAAAACQR4nt3hYfH6+cnBz16NFDklS2bFlNmjRJ8+bNM5d55513NHPmTDmdTknSSy+9pO3bt+v06dMlFQMAAAAA3JRY6Vm/fr1iY2PdprVp00YbNmyQYRg6ffq0jh49qgYNGvx+5T4+iomJ0YYNG0oqBgAAAAC4KbEhq48dO6aaNWu6TQsKClJgYKBOnjypU6dOKTw8vMDfRUREKDExscjLzcrKUlZWlnme3wABAAAAKI5ibelxOBzavHmzWrdurQYNGqhz58769ttvJUkpKSkKDAws8DeBgYE6f/78FecXZerUqSpXrpx5ioiIKE5kAAAAAKVcsUpPt27dtGfPHm3ZskV79+7VgAED9OCDD+rAgQNyOp3KzMws8DcZGRkKCgq64vyijBo1SqmpqeYpKSmpOJEBAAAAlHLF2r0tODjY/L/D4VCnTp3UpUsXff755woPD9eRI0fcls/IyNDZs2dVpUoVGYZRYL4kJSUlKSoqqsjrdDqd5sAHAAAAAFBc1z2QQU5Ojvz8/BQTE6NNmza5zdu8ebNatGghHx8fVatWTSEhIdq7d6853+VyacuWLYqJibneGAAAAABQqGKVnqNHj+rixYvm+aVLl2rNmjX6f//v/6lt27a6cOGClixZIklKT0/X+PHjNXjwYHP5559/XiNGjFB2drYkadasWYqKilKdOnVK4rYAAAAAQAHF2r1tzZo1bsfZqVevnjZs2KBq1apJkpYvX65+/fpp2rRpysnJUZ8+ffTwww+bfz906FCdPn1aUVFR8vHxUYMGDbRgwYISvDkAAAAA4M5hGIZhdYjiSEtLU7ly5ZSamqrQ0NDLLltr5KoSu97D0zqV2GWVZC6pZLMBAAAAN4ur7QYldnBSAAAAAPBGlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtnZdpWf//v1yOp2aOHGiOe348ePq1KmToqKi1LhxY8XFxbn9jWEYmjx5siIjI9WoUSM99thjSktLu54YAAAAAFCk6yo9Q4YM0b333qsLFy6Y07p27arHH39cu3fv1jfffKP3339fq1evNufPmzdP27Zt086dO7Vnzx5FR0erT58+1xMDAAAAAIp0zaVn6dKlqlq1qlq1amVOi4+PV05Ojnr06CFJKlu2rCZNmqR58+aZy7zzzjuaOXOmnE6nJOmll17S9u3bdfr06WuNAgAAAABFuqbSc/78eY0bN07Tpk1zm75+/XrFxsa6TWvTpo02bNggwzB0+vRpHT16VA0aNPg9gI+PYmJitGHDhmuJAgAAAACX5XctfzRlyhT16NFD1atXd5t+7Ngx1axZ021aUFCQAgMDdfLkSZ06dUrh4eEFLi8iIkKJiYmFXldWVpaysrLM8/z+BwAAAEBxFHtLz8GDB7V06VK9+OKLBealpKQoMDCwwPTAwECdP3/+ivMLM3XqVJUrV848RUREFDcyAAAAgFKs2KVnyJAhmjx5cqHlxel0KjMzs8D0jIwMBQUFXXF+YUaNGqXU1FTzlJSUVNzIAAAAAEqxYu3etmbNGp0/f15du3YtdH54eLiOHDniNi0jI0Nnz55VlSpVZBhGgfmSlJSUpKioqEIv0+l0moMeAAAAAEBxFav0HDp0SL/++quaNm1qTjtx4oSkS4Vo9uzZGj58uNvfbN68WS1atJCPj4+qVaumkJAQ7d27Vw0bNpQkuVwubdmyRVOmTLnOmwIAAAAABRVr97Znn31WP//8s3bt2mWeBgwYoD59+mjHjh1q27atLly4oCVLlkiS0tPTNX78eA0ePNi8jOeff14jRoxQdna2JGnWrFmKiopSnTp1SvBmAQAAAMAl1zR6W17+/v5yOBySJIfDoeXLl6tfv36aNm2acnJy1KdPHz388MPm8kOHDtXp06cVFRUlHx8fNWjQQAsWLLjeGAAAAABQKIdhGIbVIYojLS1N5cqVU2pqqkJDQy+7bK2Rq0rseg9P61Ril1WSuaSSzQYAAADcLK62G1zTwUkBAAAA4GZB6QEAAABga5QeAAAAALZG6QEAAABga5QeAAAAALZG6QEAAABga5QeAAAAALZG6QEAAABga5QeAAAAALZG6QEAAABga5QeAAAAALbmZ3UAeJdaI1eV6OUdntapRC8PAAAAKC629AAAAACwNUoPAAAAAFuj9AAAAACwNX7Tg5tGSf7eiN8aAQAAlB5s6QEAAABga5QeAAAAALZG6QEAAABga5QeAAAAALZG6QEAAABga5QeAAAAALbGkNVACWA4bQAAAO/Flh4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrlB4AAAAAtkbpAQAAAGBrxS49b731lpo0aaKoqCjVr19fTz75pI4ePWrO37dvn2JjY9W0aVNFR0dr2bJlbn9/4cIFDRkyRJGRkYqMjNTgwYOVnZ19/bcEAAAAAApR7NLTuXNnfffdd9q9e7f27NmjWrVq6YEHHpAkZWZmqkuXLpo0aZJ27dqlzz//XKNGjVJ8fLz59+PGjVNWVpYSEhKUkJAgwzA0duzYkrtFAAAAAJBHsUtP7dq1FRgYKEny8/PTxIkTlZiYqGPHjmnt2rWKjo5WbGysJCksLEzDhg3Tu+++K0lyuVxatGiRpk2bJh8fH/n4+GjKlClasmSJcnJySvBmAQAAAMAl1/2bnvPnz8vhcKhSpUpav369WXhyxcbGat26dZKkXbt2qXr16ipfvrw5PzQ0VLfeeqt27txZ6OVnZWUpLS3N7QQAAAAAV+u6Ss+///1vPfrooxo/frycTqeOHTumiIgIt2UiIiKUmJgoSYXOz79MflOnTlW5cuXMU2F/DwAAAABFuabSM3z4cIWFhalRo0aqXr26hgwZIklKSUkxd33LFRgYqMzMTBmGUej83GXOnz9f6HWNGjVKqamp5ikpKelaIgMAAAAopa6p9MycOVMnTpzQf//7XwUGBuqpp56SJDmdTmVmZrotm5GRIafTKYfDUej83GWCgoIKvS6n06nQ0FC3EwAAAABcrevava1SpUp688039cknnyg1NVXh4eE6cuSI2zJJSUkKDw+XpELn518GAAAAAErSdQ9kkJWVpezsbOXk5CgmJkabNm1ym79p0ybFxMRIkpo2baoDBw4oJSXFnJ+Wlqb9+/frzjvvvN4oAAAAAFBAsUpPdna2fv31V/N8SkqKevXqpW7duqlixYrq1q2btm/fbhafEydOaNasWRo0aJAkKSgoSL169dLIkSPlcrlkGIZGjx6tHj16qEyZMiV4swAAAADgEr/iLHzq1Cl16dJF586dU2BgoHx8fPT444+bAxkEBwdrxYoVGjhwoM6ePSuXy6WJEyeqVatW5mVMnz5dQ4cOVWRkpCSpdevWeuutt0rwJgEAAADA74pVemrUqKEffvjhsstERUVp69atRc4PDAxUXFxcca4WAAAAAK7Zdf+mBwAAAAC8GaUHAAAAgK0Va/c2ADeXWiNXlejlHZ7WqUQvDwAAwBMoPQAsQSEDAACewu5tAAAAAGyN0gMAAADA1ig9AAAAAGyN0gMAAADA1ig9AAAAAGyN0dsAIJ+SHFmOUeUAALAeW3oAAAAA2BqlBwAAAICtUXoAAAAA2BqlBwAAAICtMZABANxEGGQBAIDiY0sPAAAAAFuj9AAAAACwNXZvAwBct5Lc7U5i1zsAQMmi9AAAbI1CBgCg9AAAYBEGpgAAz+A3PQAAAABsjdIDAAAAwNbYvQ0AABTArncA7IQtPQAAAABsjdIDAAAAwNYoPQAAAABsjdIDAAAAwNYoPQAAAABsjdIDAAAAwNYoPQAAAABsjdIDAAAAwNYoPQAAAABsjdIDAAAAwNYoPQAAAABsrdilZ/Xq1WrXrp2aNGmiRo0aacCAATp//rw5f9++fYqNjVXTpk0VHR2tZcuWuf39hQsXNGTIEEVGRioyMlKDBw9Wdnb29d8SAAAAAChEsUtPSEiIFi5cqPj4eO3atUvp6ekaN26cJCkzM1NdunTRpEmTtGvXLn3++ecaNWqU4uPjzb8fN26csrKylJCQoISEBBmGobFjx5bcLQIAAACAPIpdetq2basaNWpIkvz8/DR8+HCtXbtWkrR27VpFR0crNjZWkhQWFqZhw4bp3XfflSS5XC4tWrRI06ZNk4+Pj3x8fDRlyhQtWbJEOTk5JXWbAAAAAMB03b/pOXPmjAIDAyVJ69evNwtPrtjYWK1bt06StGvXLlWvXl3ly5c354eGhurWW2/Vzp07C738rKwspaWluZ0AAAAA4Gpdd+mJi4tTz549JUnHjh1TRESE2/yIiAglJiYWOT//MvlNnTpV5cqVM0+F/T0AAAAAFOW6Ss8XX3yhXbt2qW/fvpKklJQUc6tPrsDAQGVmZsowjELn5y6TdzCEvEaNGqXU1FTzlJSUdD2RAQAAAJQyftf6h0lJSerXr5+WLl0qp9MpSXI6ncrMzHRbLiMjQ06nUw6Ho9D5ucsEBQUVej1Op9O8fAAAULrVGrmqRC/v8LROJXp5ALzTNZWec+fO6c9//rMmT56s5s2bm9PDw8N15MgRt2WTkpIUHh5e5Pz8ywAAANyMKGSA9yr27m05OTnq3r277r//fj355JNu82JiYrRp0ya3aZs2bVJMTIwkqWnTpjpw4IBSUlLM+Wlpadq/f7/uvPPOa4gPAAAAAJdX7C09Q4YMUVBQkF599dUC87p166Zx48Zp06ZNio2N1YkTJzRr1iwtXrxYkhQUFKRevXpp5MiRmjNnjhwOh0aPHq0ePXqoTJky139rAAAAUEBJboUq6S1Q3pqNLXf2UqzSk5ycrLffflv16tVTdHS0Od3hcGjNmjWqWrWqVqxYoYEDB+rs2bNyuVyaOHGiWrVqZS47ffp0DR06VJGRkZKk1q1b66233iqhmwMAAADYG4Ws+IpVeipUqCDDMC67TFRUlLZu3Vrk/MDAQMXFxRXnagEAAADcBLx1y911H6cHAAAAALwZpQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArVF6AAAAANgapQcAAACArV1z6Xn33XfldDp1+PBht+n79u1TbGysmjZtqujoaC1btsxt/oULFzRkyBBFRkYqMjJSgwcPVnZ29rXGAAAAAIDLuqbS88orr+jjjz9WhQoVdPHiRXN6ZmamunTpokmTJmnXrl36/PPPNWrUKMXHx5vLjBs3TllZWUpISFBCQoIMw9DYsWOv/5YAAAAAQCGKXXpcLpeqVaumlStXKjAw0G3e2rVrFR0drdjYWElSWFiYhg0bpnfffdf820WLFmnatGny8fGRj4+PpkyZoiVLlignJ6cEbg4AAAAAuCt26fHx8dHAgQPl6+tbYN769evNwpMrNjZW69atkyTt2rVL1atXV/ny5c35oaGhuvXWW7Vz585Cry8rK0tpaWluJwAAAAC4WiU6kMGxY8cUERHhNi0iIkKJiYlFzs+/TH5Tp05VuXLlzFNhfw8AAAAARSnR0pOSklJgl7fAwEBlZmbKMIxC5+cuc/78+UIvc9SoUUpNTTVPSUlJJRkZAAAAgM35leSFOZ1OZWZmuk3LyMiQ0+mUw+EodH7uMkFBQUVeptPpLMmYAAAAAEqREt3SEx4eriNHjrhNS0pKUnh4eJHz8y8DAAAAACWpREtPTEyMNm3a5DZt06ZNiomJkSQ1bdpUBw4cUEpKijk/LS1N+/fv15133lmSUQAAAABAUgmXnm7dumn79u1m8Tlx4oRmzZqlQYMGSZKCgoLUq1cvjRw5Ui6XS4ZhaPTo0erRo4fKlClTklEAAAAAQNJ1/qYnICBA/v7+5vng4GCtWLFCAwcO1NmzZ+VyuTRx4kS1atXKXGb69OkaOnSoIiMjJUmtW7fWW2+9dT0xAAAAAKBI11V6fv755wLToqKitHXr1iL/JjAwUHFxcddztQAAAABw1Up09zYAAAAA8DaUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2RukBAAAAYGuUHgAAAAC2Zlnp+fvf/67GjRsrKipK999/v44ePWpVFAAAAAA2Zknp+eKLLzRv3jxt2bJFu3fv1lNPPaWHHnrIiigAAAAAbM6S0vPOO+9o0qRJKleunCTpkUceka+vr3bt2mVFHAAAAAA25mfFlX755ZdatGiR27TY2FitW7dOTZs2dZuelZWlrKws83xqaqokKS0t7YrX48o6f/1h/8/VXN/VKslcEtmuRUnmkrw3W2l5PCXvzcZz7dqQrfh4rl0bshUfz7VrQ7biu5pcucsYhnHZ5RzGlZYoYWfPnlXNmjV1+vRpt+lz5sxRQkKC5s6d6zZ9woQJmjhxoicjAgAAALiJJCUlKTw8vMj5Ht/Sk5KSosDAwALTAwMDdf58wWY4atQovfjii+Z5l8ulM2fOqFKlSnI4HNeVJS0tTREREUpKSlJoaOh1XVZJI9u18dZs3ppLItu18tZs3ppLItu18tZs3ppLItu18tZs3ppLItu1KslshmEoPT1d1atXv+xyHi89TqdTmZmZBaZnZGQoKCio0OWdTqfbtPLly5doptDQUK97MuQi27Xx1mzemksi27Xy1mzemksi27Xy1mzemksi27Xy1mzemksi27UqqWy54wRcjscHMqhcubIyMjJ09uxZt+lX2iQFAAAAANfC46XH4XCoVatW2rx5s9v0TZs2KSYmxtNxAAAAANicJUNWP//88xo3bpw52sJHH32kc+fO6e677/ZoDqfTqfHjxxfYfc4bkO3aeGs2b80lke1aeWs2b80lke1aeWs2b80lke1aeWs2b80lke1aWZHN46O35Xrrrbf0zjvvyMfHR2FhYZo3b55q165tRRQAAAAANmZZ6QEAAAAAT7Bk9zYAAAAA8BRKDwAAAABbo/QAAAAAsDVKDwAAAABbo/QAAAAAsDVKjxd5+OGHrY5QJG/Ohus3adIkqyNc0c2Q0Sq8Pq/PzfbcSkxMtDoCrsPN9nxD4XjfvfkwZLUXadiwofbu3Vvk/K+//lpt2rTxYKLfeXM2b5GQkKALFy5c9fL+/v5q3LjxDUx09a70+FrhyJEjuvXWW83z3pCxX79+mjdvnqUZCuMN902u48ePF+t14Ofnp+rVq9/ARFfmTfdfrkceeUQfffRRofPatGmjr7/+2iM5bsbH09t54/NNulTGxo0bZ9n19+vXTxcvXrzq5f39/fXOO+/cwESX5y2P4+W+e9SvX18XLlxQYmKicr/uO51ORUZGWpotJCREt912m3bv3u02vUqVKgoPD79hefxu2CV7kcs9IXIf/PT0dG3dulXNmjXTLbfc4pFcDz30kNsLPCkpSQ8++KDbMsOGDVNsbKwkadCgQYqPj/eabH379lXnzp09nu1qy0XdunUVGhqqv/3tb3ruuedueK5hw4YV68tBQECAvvjiixuYyN1LL72kWbNmmed79eqlBQsWuC0zduxY7du3z3xznDhxokeK2f3336/PP//cbVrnzp3d3hC9Yf3Mli1bLjs/ISHBY0U272s07+uzbt26ev311/Xiiy/q4MGDat++vZ577jk1a9ZMP/zwww3P9cADD+jChQuXfbwcDoekS49pQECAR3Llevnll/XTTz+pfPnyql69um6//XZlZ2e7LTNy5Ei318GAAQPUsWNHj+Q7e/asTp48qb179+rQoUMF7sfDhw/r6NGjHskiXd3jmZenH09v/aJ89913y+VymecdDofat2+vsWPHesV7WWE++OCDQkvP6dOnValSpRt+/d27d3f7DDUMQ/379y9yRZO/v/8Nz5Qr/3ciqfDvRbn8/f21dOlST0Qr8ruHw+HQa6+9ppEjR8rlcsnP79JX/pCQEH322Wcez7Zjxw61aNFChmGodu3aevXVVzVs2DB9//33atmypVwul/bs2aP//ve/NyxPqdjS88c//lEXLlzQjh071Lx5c7c7PiQkRIsWLVLLli1Vr1497dmzR2vXrtXtt99+w3N98803V/yS3KhRI/PNpnHjxkpISLjhuaSry1avXj2FhYVJ8my23MczP4fDIcMwzBfQyy+/rA4dOqhJkyYeK2TeLP9aqbznc/8fERGh+fPnm8vcddddCg0NveHZ6tatq3/961/mlwHDMPTEE0/owIEDReb3hPxfqJYtW6aHHnrIbZmoqCgNGTJEkjz6XCvqNVq+fHlFRUWpTp06+vvf/65hw4Zp165dHn2NXsnGjRsVFham+vXre/y6w8PDtWTJEmVkZOjkyZNKTEzU/Pnz9euvv5rL1KxZU++++655PjIy0nyvu9HmzJmjmTNn6vjx46pWrZrbPF9fX1WuXFlDhgzRY4895pE83m7dunUFSuvl+Pv7649//OMNTHTJ/v37dfHiRT300ENavny5MjMz9dRTT2n37t1q0KCB9u3bd8MzXM64ceO0Zs0aPfPMM+rfv78kFZkrKiqqwBp5T/GWrSlX850oL39/f8XExNzARFfPW977i8qR93l3o7OWii09a9eulXTpC8lXX32lxo0ba8OGDeb82bNnq3fv3ho9erRWrFihWbNmeWRNUHFfELlrRz0hJiZGqampKleunDntl19+0aJFizRx4kRLs+U+nvktWrRITz75pBo0aOD2+JaCXn9V8t8Phd0vISEh6tChg6cimX777TeNGTPGLdNvv/3m8Rz55a7lztWpU6cCy9SoUcP8vyefa1d6/wgJCVG7du3c1jZbKS4uTm3btlXDhg2VmJiof//735aUnrJly5pbz3N9+OGHbudDQ0PVrl07T8YyDRw4UAMHDlR0dLR+/PFHSzLcTHJ337ka/v7+ioiIuMGJLsl9bjudTjVs2FCS93wWffTRR9q2bZtmzJihkSNHKjo6Wi1btizyc9xT7yE1a9ZURkaG27Tk5GRVqVJFhmG4bSEODg7W4cOHPZJLuvz77bZt23TixAn9+c9/9lie/FJSUlS+fHnz/IYNG5SSklJgJZ03yvu8u9HfJUtF6clV1BvOhg0b9Pbbb0u69CXnlVde8VimkydPKiQkRGXKlPHYdV6t2NhY7dq1yzxftWpVj+1Hfi1WrlxZ6Avck4Xscr744gvNmDFDX375pSXXn/9+8Jb7RZJq1apVoMx6w++ditp1YeHChQoODlbXrl3dpnvyPn3zzTeVk5PjNq1WrVpe8yE3ZswYBQQE6JVXXpGPj4/mzJmj7t27S7r05WbVqlWW5CrsMfLG18aIESMKnZ6VlaVRo0bp9ddf91iWb775RkeOHFFkZKRXvC7z6t27d4Hd73bt2qXo6OgCn/kBAQEef//1hudSfgsWLNBf//pX1atXT1OnTtX777+vli1bFvkdyVO34T//+Y/5//j4eDVp0sQj13u9srOz9cknn1haeu6++26372t+fn7atm2bV3we/O1vf1NkZGSRxdGTKwNKVel58803zX93796tTz75RBMmTNCxY8fMH075+Ph4dM1ow4YN5XK51KZNG40fP1533nmnx677Sn7++We1bNnSPG8Yxg3d17I43njjDc2fP18VKlTQ0KFD1bVrV4WHh+vgwYMF1uJ6g7lz52rOnDlatmyZ1VFMP/30k8qUKWPuX2ulq/kiapX09HQlJyfL19fX3KoTEBCgH3/8sUDp8XSunJwcbdu2Tb6+vmrRooUGDRrkFR9yc+bM0aeffqrmzZvrj3/8ox5++GE1b97cXBNZrVo1JSUlWRsyj5SUFHPFk2EYHtm9+Urmz5+vPXv26LXXXjOnxcfH64knnlC3bt08muXpp59W69at9cYbb+jEiRPq37+/XnjhBQUGBno0R2EKKzGNGzfW5s2bLUhTUFFf6PKu6DQMQ4GBgUpOTvZIpsTERNWrV0+S1KpVK40aNcoj11sc3bt31969e5Wenq7Zs2dr//79io2NVf/+/eXj410DD99xxx366aefLM3wyy+/qG3btub5c+fOqW7duhYm+t2kSZPUsGFDVaxYUWfPnlVISIhlWUpN6Xn11Vd16NAh7dixQyNGjNCRI0e0YcMGTZgwQcHBwcrIyFDZsmUlXdpv2lOqVq2q+Ph4ffLJJ3r00Uf1zDPPaOTIkR67/suJiIjQv/71L/O8YRge2Rf6SmbPnq1///vfWrdunZKTk9W3b1+FhoaqWrVqOn78uOLi4izLlpCQIKfTqWrVqqls2bLavHmzJkyYoLCwMH399ddum5+tdscdd5j70ebufoGCGjRooAoVKigtLU3Dhw/Xc889p9q1a1u2pSLX2LFjJV1aiePv76+BAwfqf//3fy3NlOuvf/2rHA6H3n//fY0ZM0YDBw7UL7/8ohMnTujdd99Venq6Tp48aXVMU7ly5XTs2DHzvDesYV6zZo0GDRqkbt26acGCBZo7d67mz5+v+fPnq3Xr1h7N4u/vb/7W7+jRo5o9e7aaNGmif/7zn2revLlHsxTl5MmT+vDDD5WQkKBff/1VgwcPVkxMjB555BGPfqZfrfPnz1t23XmLWJkyZXT27FnLsuTXu3dvvf/++2bGTp06qWvXrnr22Wf13nvv6ZdfftHs2bMtTukuLCxMp06dsjRD9erVNXfuXPN8amqqRo8ebWGi31WtWlWLFy9Wy5Yt1b59e3355Zdupd+TKzhLTemZN2+e3njjDf31r3/ViBEjzK0C0qUvfz/++KPatm2rU6dOebSF+vr6ytfXV926dVO7du10//33S1KhxcfT+wM7nU7ddtttbtOCgoJ04cKFAqOmeDLbokWL9N133ykgIEDVqlXTO++8o/Hjx+vBBx/Uli1bFBQUpIoVK6pRo0Yey5Rr4MCBOnXqlM6cOaOMjAydP39eI0eOdFtb6y28ZUuKdGl46kceecQ8bxiG12wJKFu2rBISErR+/Xpz7XH16tUt/5ArLqse79dee027d+/W5s2b1aFDB/n6+io0NFTp6emW5CmMN70Wcvn7+2vevHmaM2eOatWqpZiYGG3fvt3td5aekvf+qVGjhl5//XU99thjevzxxzV37lzLfv+Ua926derRo4ceeOABtWnTRvfff79+++03ffzxx5o0aZJWr15t2dZsb3xu5d2b5dy5c+YKX2/w3Xffmf//5ZdfVKtWLXOgmDZt2igqKsqqaJdVnME0boTAwEC3YaizsrJ05swZCxP9zuFwKCQkRB07dlSVKlU0ePBg/eMf/9DFixe1e/duZWZm6scff1ROTs4N/1woNaWnbNmy6tq1q7km1MfHx/zxY8+ePTVs2DC98sormj9/vp5++mlLMlaoUEGffvqp7rrrLsXGxuquu+7SHXfcoRMnTsgwDNWsWdOjec6cOeM2gpFhGDp37pzOnj2rChUq6O6779bOnTs9vntUTk6OAgICzPO1a9fWiRMnVKFCBS1dulS7du1Sy5YtLSk9eX/zlJSUpLVr1+q9995TixYtFBcXp2bNmnk8U67k5GQtWbJEDodDWVlZXrWLQMWKFc0RhKRLz7WdO3damOh3uV9a8n55CQkJKfCDW2+QN2Puioj//ve/uvfeez26a2qZMmXc7p+//e1v+tOf/qQnn3xSL7/8siRp8eLFHsuTl7f8mLwwW7ZsKfDlqX79+ho0aJDee+89ffXVVwoNDVVAQIDHt/bk16JFC33++efq0KGDPvvsMzVo0MCyLKNHj9aqVavUokULt+kDBgzQv/71L40YMUIff/yxRzMdPHhQOTk5ys7O1oEDB5SVleU1BahWrVrau3evGjZsqG3btnnN1rr8/Pz83H63mJOT47Wv34CAAF28eNEcFtrTTp48qSlTppjnXS6X+R6cfxRIKwQFBSkjI0OTJ09Ws2bNtG3bNkVEROill15SzZo19eKLL0rSDf95QqkpPYVxOp3Kzs7W3XffrcGDBysuLk733HOPR0tP/hdw1apV9fbbb2vw4MHasWOHfv75Z49lyS8rK0t79uxxy5idnW0O4btx40ZLclWtWlW7d+821/h8+umnatWqlUJCQvT444+bu/307dtXFy9etGyNfEREhJ555hk988wzWrlypbp3766RI0fqmWeesSTPyJEjtXXrVnMUnNz7KS+rdr8rV65cgbXF3ji4R668X+pfffVVt5UDnvL2228rJydHmzZtkp+fny5evKjTp0+b83N3Td2xY4eOHz/u0QNG/vDDD25fgmvVqqV7771XS5cuNY9ibtWXl6vZRfdGHhzvcl599dUi1xjXqVPH/F2q0+nUmjVrPJarqMfqtttuU1xcnJ544gnt2LHDsi/1qampBQpPrgceeEAzZ870cCKpR48eysnJUdmyZfX444/L4XCYx3qyuvw8+eSTGjhwoIYPH67x48cXOA5Odna2Nm3aJMMwZBiGR7fK3nHHHZIu3Ue1atVSSkqKxo8fr8aNG2vx4sV69NFHPZYlv6lTpxY6UqDL5VJ6erqlu1H6+PgoMDDQ7bWaWxiLGvHWUwzDkNPpVEZGhnx8fDRt2jS98sorWrdunb766iuPZik1pSf3ieByuTRx4kQZhqHU1FSzlffs2VM9e/b0eK7CRvu4//779eabb2rfvn2Wrj0LCwsrMELQt99+a/kb9syZM/Xoo4+qdevWOnv2rA4dOqQvvvhCCQkJbl8YHnjgAeXk5Gjw4MEWpv09yx/+8Ad16tRJd955p6Kjoz2eIXcXgcLkvj6+/fZbT8Vxs2PHjgLT8v82y6ovybnX6+fnpx9++EELFy5UZmam+YEyYsQIDRw4UA6HwyPHNMqVnJysnJwcs/wnJydr8uTJ5vzcLZ3Vq1f3aOHJtXLlSrfzQ4YMcRtdyKrHM7c4XM7q1as9kKQgTx6suDiGDx9e5Lz77rtP999/v06cOGHZGuUmTZpo9OjRGjt2rNvKkmPHjun5559Xly5dPJ5p27ZtRc6zemvFk08+qV9++UUzZ87U4MGDzQGUcj/bk5OT9fe//91cQda7d2+PZVu+fLmk3++jTz75RPPnz9c333yjnj17WjpQS3BwcJHHCFywYIGl340qV65sbi3JNWfOHIvSuHvvvfckSc8//7wkqUOHDh49gLEbo5T49NNPDcMwjN27dxuvv/66MWvWLGPp0qUWp/Juu3fvLjBt3bp1xsWLFy1I4y41NdVYtWqVsWHDBiM7O9swDMPYsmWLMWLECIuTFTRx4kTz/95w3+Xq1q2b+f8ZM2ZYmOTqTJ8+3ZLrnTNnjmEYhnHu3Dlj1KhRxnPPPWcMHjzYWLZsmSV57OLjjz+2OoLJx8fH6gg3pVdffdXqCIZhGEZaWprxzDPPGOXLlzeaNm1qtG7d2rjjjjuMypUrG+PHjzdycnKsjujGqveyK6lfv77VEUyffPKJ+X9veZ55sxMnThSYtmvXLguSeDeHYXjpDpKlQEJCwhUPqla/fn1zzdXkyZML3SUJlyQnJ+vXX3/1muNIJCQk6LbbblPz5s294ojS+XnLka5xbfr162fuano1/P39PXLQZW/NdTmXey1s2rTJK4fB9wZNmjRRfHy827TTp0+rUqVKluTJysrSY489pjFjxqh8+fKqXbu2Zb9dPH78eJGf7yEhIapYsaJcLpcSExNVp04dr/iN5YwZM4o8PpSVCnueeatJkyZp3LhxVse46Xjqfiv1pcfKJ+gf//jHy5Yeh8OhKVOm6A9/+IMkz77wr6aQ5eXv7++xsnG5bOHh4XI6nUpMTDQ3jwcHB5vHJPCEzMxMBQYGasCAARo+fLg6d+6svXv3au7cudq4caNiY2M1cOBAtWnTxqMHe+3atavb/fbVV1/pnnvukSQ999xz2rdvnzZs2GDu0lCvXj3NmDHjhue63ONZsWJFlS9fXocOHTJzVa9eXVWrVr3huaTLf3EpjJ+fn8d2JVu3bp3b7pyGYWjQoEFF7tLg7+/vkSHnvTVXroceeqhAKcv7WpAuDXyzZMkSSawcGDBggNtoX7Vq1TKHwm3cuLESEhLclo+KitLu3bs9mjGvKz1eX3/9tdq0aXPDczRr1sw8aOovv/yi22+/3fxMuuuuuzRt2jS1bdtWOTk5cjqd+uqrr1ShQoUbnsvbTZgwQdu3b9fAgQPVuXNnSb9/9xk9erQOHjyoDh066Omnn/b4Z2hRzp07p+DgYEnWvF/cjCua8vPU/VbqS8/N9IFW2AfMjXKlQpZfQECAx/ZHz5vt+++/N48k7XA41L17dy1YsEA+Pj7m77Vq1Kjh0ZGibr/9di1cuFDTp0/XBx98oDvvvFN79+5VdHS0Jk+erFGjRik+Pt6jj6d06ajqRT2mDRs2VPv27TV9+nQFBgbK5XKpd+/ebkfIvlHyPp7x8fGqX7++AgIC5HA41Lx5c23dulUul8vMtX//fv322283PJd06YtLcYYiDQgIsG5fZXnv+5k35brc6yBXYGCgWrVqJenScZpyj2dVGi1fvlw5OTkaNmyYZs+erRdeeEFHjhyRVPiKOE+/r+UtsYZhaOPGjW4FVrq0l0TusZes2GpQ2HNo7NixCgwM1NixYzV79mydOXPGY4c28NatUO+9954WL16sMWPGaOrUqZo7d67q1q1rPma33Xab3nrrLY0fP147duzw+HOtd+/e5uPYsWNHjR8/XtKl30/u2bNHkjXvF96+oqlXr15asGCB27Q//elPboOxeOx+8/wedZ43duxYwzAMY/Hixea0QYMGGYbx+z6sGRkZRrdu3YyHH37YyMzM9HzIfC5cuFBgWuPGjS1I4t0K2wfZqv2Sc59fYWFhxoMPPmh07NjRMAzDaNCggWEYhhEdHe32b6NGjSxIWbT8eazId9999xlHjx51m5b/8fS2+80qt956q3HLLbe4nfz8/ApMq1y5slGzZs1Snyu/Z5999qqWy339lna5nz95X3+FfSZ5+nNq69atxsaNGy97SktLM5e34v0j9z1s3759xnfffWcYhmE0adLESE1NNQzDMNLT042mTZt6LM+dd95pNG7c2GjUqJERGBho/r9Ro0ZG3759jdOnTxuRkZFG/fr1jaioKOPMmTMeyXXPPfcYP/30k2EYl36PMnnyZMMwfn9OWf0ZWrNmTeOnn34y4uPjjaioKHN63s8ob3m/8JYchlH4e0L+z3VP5S0Vo7d9+umnevXVVzV79mz16NFD0qW1fdLvo5W88cYbqlChgnx9ffXmm29aul/rRx99pFWrVhVoxiiosNFSrDwIY48ePRQWFqYKFSp45cErU1JSFBwcXODgslLB+83qUfpyeWsuq3liK9y18NZc+eUdcn/x4sU6cOCAHnzwQUuPpXWzyT0kgGEYCgwM9OjohbliYmKKtbwn3z8OHjyo2rVrm4cpGDZsmHr06KEWLVooOzvbvL9CQkKKtWfF9cq7NbpBgwYFtnyNHTtW3bt3N7dCzZo1yyNboY4ePWoOWR0VFaW33377hl9ncQQFBZn58h4/yOrPpJo1axY4blxycrKqVKniNs0wDAUHB+vw4cMeTFc4q+4z638550HGZfbkW7VqlSZNmqSJEyfq008/9WAqd1u3btW4ceP06quvWpbhanzxxReWHoXb5XJpxIgROnTokDp37qzNmzdblqUwHTt2NI+OvH//fvn7++unn36yONWlXe9uueUWde/eXQcOHLA6TgHTp09X1apVdeDAAU2YMEGS9cO7StLhw4c1YsQIxcbGqmHDhmrXrp2mTp3q0QN+FubgwYNat26dkpOTLc2Rn7fmkqQXXnjB7fyzzz6rDz74QFWqVNHTTz+tL7/80qJkN5+DBw+qVatWatWqlXnIByter0uXLtXrr7/udvrkk088niO/pk2bKjo6Wu3atTOHun/88cclqcAxXaweyGD//v36/vvvJUmfffaZObxw//79PTaEe/4De65evdqrjteW94u61UUnr//85z86efKk2+nChQvm/3/44Qft379fp06d8orCY6VSVXou5+jRowoLC1OVKlV07Ngxj11vSkqKXC6Xzp07pylTpujZZ5/VihUrdOutt3osQ3HNnTtXL730UoHjqHjStGnTtHPnTn377bcaMWKEhg4dqq1bt0ryji/J99xzj44ePSpJqlevni5cuGCuIbJSxYoV9euvv6pdu3a6++67PX6U8qL07dtX9957r+Li4uTr66sqVap4TZFdt26d2rRpo4oVK2rmzJlauXKlJk6cqNTUVN15553av3+/Zdkee+wxzZ49W02bNlVUVJQWLVrk9qNzchWU97eHycnJ+uSTT7R8+XINGjRIixcv1qxZsyxMd3O54447lJiYqEOHDpnHV7Hiy+Czzz6r9PR085SWlqb+/ft7PEd+NWrU0KJFizRgwABNnjzZPGCwdGmwjNzPiN9++838IbynHDx4UC6Xy20rVO6KMCu3QuXVvn17nT9/3pLrLkze7xZ5///TTz+pTJkyCgoKsiKWpEsrgpOTk3XmzBmlpqa6zVu6dKk++OADi5J5l1Kxe1thdu/erTJlyqhWrVqS3DdVevJLc8uWLXXq1CldvHhR1atX17fffquKFSua8z/77DNzBJj8T+QbLSEhQU6nU9WqVVPZsmW1efNmTZgwQWFhYfr6669Vvnx5j+bJa9GiRdq4caM5ildcXJxee+01S7fSPf3005IufejfcsstOn78uHk+77+5rPhy4Ovrq5CQEPXt21f33Xef7rvvPlWsWNHSrXaStH79ev3v//6vunXrJkkqV66c5VtRco0ePVpr1651O1BwnTp11Lp1a7Vr104jR440v/B5Wnp6ur777jtJ0i+//KJZs2bpL3/5iz766CPVrVvXkkzenCsvwzB04sQJVa9e3VzDXKdOHY8NkmEH3rK2u2rVquaPynN9+OGHFqX5nY+Pj5o0aaLVq1erU6dOOnr0qLnLUa9evfTcc89p3Lhxmj59up544gmPZmvatKnq1Kmj999/32u2QoWHh+vAgQO6/fbbFR8frzp16lx2eU8//4q6vjvuuMP8EX7Dhg09GcnUvn17HThwQL6+vjp//ryGDRtm/kyjcePGWrp0qSW5vE2pLT1RUVHauXOn+QTNu1k1/wv+Rvr555/lcrm0ZcsW/eMf/9Af/vAHLVmyRC1atJB06ajm2dnZcjgcGjhwoMdySdLAgQN16tQpnTlzRhkZGTp//rxGjhzpsRFmLufcuXNuwxY3adJEiYmJFiaSXnrpJUm/l+bKlSvr9OnT5vzc6efOndMXX3yhtLQ0z4fMo3bt2vrwww/1yCOPKD4+XmXLllWHDh3kdDrlcrk8usUzKChIzZo1U2BgYIF5Pj4+2rZtmwICAuRyuZSenu6xXNKlIcjzFp682rdvr2HDhnk0T155P4Tr1q2ruLg4rVy5Uu3bt9eKFSssO2aVt+aSfn8dOhwO3X777Tp16pTi4+PVpEkTLVy4UHfddZcSExPVrl07ORwOtxVipdE///lPXbx4USkpKVq4cGGhuywePnxY58+fl2EYlqyZL85vO63YE6BSpUr617/+pc6dO+vrr79WhQoVNGDAAB0+fFg9e/ZU586dNWjQII9myt0K1bdvXyUnJ2vLli3mvNytUDVq1PDoVqiePXuqf//+GjVqlKZPn665c+e6zc997FJTU7V27VpLP0O9bVe348ePm6Mqbty4UcuWLTPn1atXTwcPHrQqmlcptaUnv+rVq+vkyZPy8fHx2HE2cvn4+Kht27Zq27atNm7cqMcee0zvvfee2rRpY+lY6nnHv09KStLatWv13nvvqUWLFoqLi7P0B78VKlTQr7/+qvDwcEnSjz/+qNq1a0uSxx+//CZOnCjp0oFl8/5ANHeIyN69e2vBggXq1auXx7Pl/8Bv3ry5unXrpmnTpmnp0qX6+eefzWWqVavm8XyF6dmzp0aPHm0OS96+fXuPXn+ZMmX0448/Kjo6usC8zz77TLfffrtH8+RV2Be4Bx54QLfccov+/Oc/69tvvy3wY9bSnEuShg8fbv7fz89P8+fPV8eOHVW+fHlVqFBBy5cvV6VKlbRt2zY5HA5Lt2h7g++++045OTnq0qWLvv/+e3PlTl4vvPCCTpw4IYfDoXvvvdeClJcXFhZmlrLczwlPyPs6qF27tp577jmNHz9eb731lqRLu2lPmzbNY3ny8satUD169NDBgwf1xhtvaPDgwQXeW3M/Q5944gm9//77euqppzySK1fexzMjI0OHDh0q1iENPCV/CStbtqyluwmeO3fObaW0y+WyLo9HxoizWHh4uLFy5UqjTp06xqpVq4yPP/7YaN68uWEYvw+TN2PGDKN///7GwIEDjRkzZlgZ1/jhhx+MWrVqGWfPnrU0R1E+++wzo27dusb8+fMty7Bw4UKjZcuWxsqVK40PP/zQqF+/vrF161bL8hTm5MmTRnp6umVDaBdm2LBhBaadOnXK8uHQc++jqKgo45VXXjHGjx9v1KhRw9JMuTZv3mxUr17dGDp0qLF06VLjyy+/ND766COjb9++Rv369Y0jR45Ylu1y71V///vfr3pY5pLmrbnyyvu6zMnJMU6dOmVhmpuTtwwfX7VqVWP27NnmadasWUaVKlWsjmV8+umnbudzcnKMZs2aGSkpKRYl+l3eIYITExONyMhIt6GpX375ZaNRo0bGqFGjrIjnxlueZw899JD5/27duhmNGjUymjRpYrz88svmdKuGis57vZs2bXJ7jz179qzRokULK2IZhnHpfssdEj331LVrV7dlPHW/lYqDk06ePFl79+41W7qPj4/+9Kc/6cknnzQPmpeRkaEnn3xSvr6+WrhwoZxOp6WZ827F8Eb//e9/1alTJ8XFxRW6BtwTli5daj5Wzz//vFq3bm1JjiuZMWOGpUOg3wxWrFihBx98UFu3btX27dslXdpS1rFjR4uTXXLq1CktWbJEP/zwg5KTk1W5cmW1adNGjz76qEJCQqyOVySXy2X5qFCF8YZcW7Zs8dr3jJvFrFmzCt3y42kLFy7UoUOH3KbVrFlTvXv3tibQZRj/t8XaavkPBhkXF6e9e/eaW6G8ibc8z66GVQdibteunTnyZFJSku666y6lpaXJMAz5+vqqR48eXjcEeF6eOjhpqSg9l1Paj7Z9PXJycjz6+yeUbpMmTdK4ceOsjgEAN73cFU25XC6XWrZsqS+//FLlypWzMNnNLTg4WOfOnbM6RqG8+TPUUyuHS33pYS184fr166eLFy9e9fL+/v6W/v4I9mfVGjQAKA28ZSvUzcybV6Rb8RmalJRU5JDnoaGhqly5so4ePary5csrODhYy5Yt00MPPXTD8pSKgQwu9wXeMAxzqOFcfIGXHn744WL9QM/f3/8GpoHdPfHEE1q8eLEk6bXXXtOYMWP0pz/9SWvWrDGXKeXrZwCgRCQkJBT5RbROnToyDEOHDx82S1C1atUUFhbm4ZTe54knntCePXsUEBCgsmXLqkqVKoqIiFCzZs306KOPSrJuJLcxY8Zo9erV8vf3V9myZVWpUiXdcsstqlatmiIiIszDs3ha165dC32u/fzzz+rdu7dq166tv/3tb3K5XDpy5IgmTJhA6ble3bt3d7vTDcNQ//79NW/evEKX5wu8VKtWras+IJm/v7+lI1jh5rdz507z/x9++KHGjBmj//znP27LsAYSAK7fsGHDCv18dzgcGjp0qGbMmCGXyyWn0ynDMLRv3z6OXyVp69at2rp1q1wul2JjYzVp0iT95z//KXQYd09btmyZVq9eLV9fX507d04pKSk6c+aMjh49qp9//lmrVq2yJFfu8dry2r9/vx555BG9/fbbGj58uD788EP17dtX0o1fuVkqSk9hQ2gGBwerQ4cOFqS5OTz33HMFtvR8//33atmyZYEnZUBAgNtRzoHiYisOAHjG2rVrLzt/zJgxSkhIMM9beVwtbxIYGGgeEsPX11f/8z//o//5n/+xONXvrjQcu1UHTs3vL3/5i2XXXSpKT82aNZWRkeE2LTk5ucBxIgzDUHBwsA4fPuzBdN6psBLToEEDbdiwwYI0sDu24gCAd8j/fsz78yV57wdfX1+vGIUyl7c+RhkZGRowYIC2bdumHj16aNy4cfL391ebNm0syVMqSk/+3WTySk1N1ZEjR1iTUYg9e/Zo165dqlOnjpo3b+61LyoAAHD1li5dKn9/f7Vv315BQUFWx7kp5N0jwel06sKFC/r666+1efPmAvNxyZgxYxQZGamZM2dq8ODBWrRokQICAvT6669L8nxZ846K6gHbtm0rdPrRo0f1yiuveDjNzaF37976/PPPNW7cONWoUUNnzpwpcCwEoCTwYQEAntOnTx/Nnz9ftWrVsnR3o5tJ3i/ofn5+unDhgoKDg1WpUiVVqlRJlStXtjCdd1q/fr2GDx+uKlWqaOrUqfroo4+UkZGhzZs36+uvv1ZOTo5H85SKLT3SpRHc4uPjC0yvW7euDh48aEEi73fu3DktWbJE0qVNlHPmzNE999yjl19+Wc8++6zF6WAnbEUEAM8JCwvTihUrdOLECT3xxBP64YcfzBE0cWV+fn66ePGi7rrrLt11113m9NI+8m9+DofD/HyvUaOGkpOTFRwcrJkzZ8rhcJi/kcpd5kZ/Fyg1W3qKWpMcEBDAWuYi5H3yBQUFadiwYfr+++/1z3/+k61jKFHnz59XYmKiDh48qOzsbCUmJiorK8vqWABgS7mf72FhYVq7dq2Sk5M1YcIEc962bdu0c+dO7dixQ+np6RYm9R55vys6HA65XC4L09wcypQpo6SkJEnS5s2b1aRJE1WtWlWfffaZVqxYodDQUDkcDoWHhysiIuKGP9dKzZaey7XH4hyEszQprAzecsstWr9+vTp06KD58+erT58+FiSD3TRt2lRdunSRdGkI9C5duigqKsriVABgfz4+Plq8eLFatGihRx55RD169NDo0aPN7wCFjYBbGrVo0cL8/7333utVv4Wy6jg8V/LKK6+offv2atu2rb755hutXLlSM2bMMOePGzdOQUFBWrlypZKTkxUaGnpD85Sq0vPVV1/J19fXbbrL5WItRhFyx03Pz+l06sMPP9QDDzygnj17KiAgwMPJYDfLly+/4jJskQWAkpH//bRChQoaM2aMXnnlFS1dulQjRoywKJn3Wrhwofn/1157rdBlrPqcWr169RWXsSJbx44dVa9ePe3Zs0dTpkwp8LunkJAQ8/8VK1a84XlKTenp3bu3/vKXvxR40B0Oh/r162dRKu/24osvFjmvatWq+v777z2YBqXdU089ZXUEALCFwgYv6NWrl9vxeVB83vw5ZVW22267Tbfddpt53srv3A6D1acAAAAAbKzUDGQAAAAAoHSi9AAAAACwNUoPAAAAAFuj9AAAAACwNUoPAAAAAFuj9AAAAACwNUoPAAAAAFv7/xKgfbF4m40JAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1000x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 위에서 구한 빈도수를 그래프로 그립니다.\n",
    "\n",
    "df_train_vocab.sum().nlargest(20).plot(kind=\"bar\", figsize=(10, 4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 모델 학습\n",
    "\n",
    "\n",
    "\n",
    "<img src=\"https://i.imgur.com/rKOYbeX.png\">\n",
    "\n",
    "전체 과정\n",
    "* feature_names : 학습(훈련), 예측에 사용할 단어 사전 입니다.\n",
    "* label_name : 정답값\n",
    "\n",
    "* X_train : feature_names 에 해당되는 컬럼만 train에서 가져옵니다.\n",
    "    * 학습(훈련)에 사용할 데이터셋 예) 시험의 기출문제\n",
    "\n",
    "* X_test : feature_names 에 해당되는 컬럼만 test에서 가져옵니다.\n",
    "    * 예측에 사용할 데이터셋 예) 실전 시험문제\n",
    "\n",
    "* y_train : label_name 에 해당 되는 컬럼만 train에서 가져옵니다.\n",
    "    * 학습(훈련)에 사용할 정답 값 예) 기출문제의 정답\n",
    "\n",
    "* model : 학습, 예측에 사용할 머신러닝 알고리즘\n",
    "\n",
    "* model.fit(X_train, y_train) : 학습(훈련), 기출문제와 정답을 가지고 학습(훈련)하는 과정과 유사합니다.\n",
    "\n",
    "* model.predict(X_test) :  예측, 실제 시험을 보는 과정과 유사합니다. => 문제를 풀어서 정답을 구합니다.\n",
    "\n",
    "* score\n",
    "    * 시험을 봤다면 몇 문제를 맞고 틀렸는지 채점해 봅니다.\n",
    "* metric\n",
    "    * 점수를 채점하는 공식입니다. (예를 들어 학교에서 중간고사를 봤다면 전체 평균을 계산해 줍니다.)\n",
    "\n",
    "\n",
    "### RandomForestClassifier\n",
    "\n",
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/c/c7/Randomforests_ensemble.gif\">\n",
    "\n",
    "### DecisionTree와 Randomforest의 Parameter\n",
    "* n_estimators\n",
    "    * DecisionTree 는 트리를 하나만 만들기 때문에 없음\n",
    "    * 결정트리의 개수, 많을 수록 성능이 좋아지나 오래걸리며 샘플의 수에 따라 트리를 만들 수 있는 개수가 제한적\n",
    "* max_depth\n",
    "    * 트리의 최대 깊이\n",
    "* max_features\t\n",
    "    * 피처의 개수(int) 혹은 비율(float)\n",
    "    * sqrt 또는 auto : 전체 피처 중 √(피처개수) 만큼 사용\n",
    "    * log : 전체 피처 중 log2(전체 피처 개수) 만큼 \n",
    "* min_samples_split\n",
    "    * 노드를 분할하기 위한 최소 샘플 데이터수\n",
    "* min_samples_leaf\n",
    "    * 리프노드가 되기 위해 필요한 최소 샘플 데이터수\n",
    "* max_leaf_nodes\n",
    "    * 리프노드의 최대 개수\n",
    "* n_jobs\n",
    "    * 사용할 CPU 코어의 수, -1로 설정 시 사용할 수 있는 모든 CPU코어를 사용\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-4 {color: black;}#sk-container-id-4 pre{padding: 0;}#sk-container-id-4 div.sk-toggleable {background-color: white;}#sk-container-id-4 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-4 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-4 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-4 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-4 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-4 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-4 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-4 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-4 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-4 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-4 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-4 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-4 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-4 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-4 div.sk-item {position: relative;z-index: 1;}#sk-container-id-4 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-4 div.sk-item::before, #sk-container-id-4 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-4 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-4 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-4 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-4 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-4 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-4 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-4 div.sk-label-container {text-align: center;}#sk-container-id-4 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-4 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-4\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>RandomForestClassifier(n_jobs=-1)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" checked><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomForestClassifier</label><div class=\"sk-toggleable__content\"><pre>RandomForestClassifier(n_jobs=-1)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "RandomForestClassifier(n_jobs=-1)"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# RandomForestClassifier 를 불러옵니다.\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "model = RandomForestClassifier(n_estimators=100, n_jobs=-1)\n",
    "model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "# n_estimators=100\n",
    "# help(RandomForestClassifier)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 교차 검증\n",
    "\n",
    "<img src=\"https://i.imgur.com/x9M8zpI.png\" width=\"600\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done   3 out of   3 | elapsed:   38.4s finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([2, 1, 4, 1, 4, 4, 4, 4, 6, 4])"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# cross_val_predict 로 교차 검증을 합니다.\n",
    "from sklearn.model_selection import cross_val_predict\n",
    "\n",
    "y_valid_pred = cross_val_predict(model, X_train, y_train, cv=3, n_jobs=-1, verbose=2)\n",
    "y_valid_pred[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 교차 검증 정확도"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 1, 4, ..., 2, 3, 2])"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 교차 검증 결과 \n",
    "y_valid_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7049765628422482"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 교차 검증 결과 점수를 구해봅니다.\n",
    "# 모의고사를 보고 채점해 보는 과정과 유사합니다.\n",
    "valid_accuracy = (y_train == y_valid_pred).mean()\n",
    "valid_accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>topic_idx</th>\n",
       "      <th>valid_predict_idx</th>\n",
       "      <th>accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   topic_idx  valid_predict_idx  accuracy\n",
       "0          4                  2     False\n",
       "1          4                  1     False\n",
       "2          4                  4      True\n",
       "3          4                  1     False\n",
       "4          4                  4      True"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# df_accuracy 로 정답여부 컬럼을 만듦\n",
    "df_accuracy = y_train.to_frame()\n",
    "df_accuracy[\"valid_predict_idx\"] = y_valid_pred\n",
    "df_accuracy[\"accuracy\"] = (y_train == y_valid_pred)\n",
    "df_accuracy.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style type=\"text/css\">\n",
       "#T_b3835_row0_col0 {\n",
       "  background-color: #4e9ac6;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_b3835_row0_col1, #T_b3835_row5_col3 {\n",
       "  background-color: #f8f1f8;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row0_col2 {\n",
       "  background-color: #f6eff7;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row0_col3, #T_b3835_row6_col4 {\n",
       "  background-color: #f7f0f7;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row0_col4, #T_b3835_row1_col5, #T_b3835_row1_col6, #T_b3835_row3_col5, #T_b3835_row5_col0, #T_b3835_row6_col1 {\n",
       "  background-color: #fef6fa;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row0_col5, #T_b3835_row0_col6, #T_b3835_row6_col0, #T_b3835_row6_col5 {\n",
       "  background-color: #fff7fb;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row1_col0, #T_b3835_row1_col2 {\n",
       "  background-color: #f0eaf4;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row1_col1 {\n",
       "  background-color: #2383ba;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_b3835_row1_col3, #T_b3835_row2_col4 {\n",
       "  background-color: #f5eff6;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row1_col4 {\n",
       "  background-color: #fbf3f9;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row2_col0, #T_b3835_row4_col3 {\n",
       "  background-color: #f5eef6;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row2_col1, #T_b3835_row3_col2 {\n",
       "  background-color: #f2ecf5;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row2_col2 {\n",
       "  background-color: #2987bc;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_b3835_row2_col3 {\n",
       "  background-color: #d9d8ea;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row2_col5, #T_b3835_row3_col1, #T_b3835_row4_col5 {\n",
       "  background-color: #fdf5fa;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row2_col6, #T_b3835_row4_col2 {\n",
       "  background-color: #f3edf5;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row3_col0 {\n",
       "  background-color: #fbf4f9;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row3_col3 {\n",
       "  background-color: #056ba7;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_b3835_row3_col4, #T_b3835_row4_col0, #T_b3835_row5_col2, #T_b3835_row5_col4 {\n",
       "  background-color: #fcf4fa;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row3_col6, #T_b3835_row5_col1, #T_b3835_row5_col6 {\n",
       "  background-color: #fef6fb;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row4_col1 {\n",
       "  background-color: #f9f2f8;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row4_col4 {\n",
       "  background-color: #034e7b;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_b3835_row4_col6 {\n",
       "  background-color: #f1ebf5;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row5_col5 {\n",
       "  background-color: #023858;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_b3835_row6_col2, #T_b3835_row6_col3 {\n",
       "  background-color: #f4edf6;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_b3835_row6_col6 {\n",
       "  background-color: #045c90;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "</style>\n",
       "<table id=\"T_b3835\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"index_name level0\" >valid_predict_idx</th>\n",
       "      <th id=\"T_b3835_level0_col0\" class=\"col_heading level0 col0\" >0</th>\n",
       "      <th id=\"T_b3835_level0_col1\" class=\"col_heading level0 col1\" >1</th>\n",
       "      <th id=\"T_b3835_level0_col2\" class=\"col_heading level0 col2\" >2</th>\n",
       "      <th id=\"T_b3835_level0_col3\" class=\"col_heading level0 col3\" >3</th>\n",
       "      <th id=\"T_b3835_level0_col4\" class=\"col_heading level0 col4\" >4</th>\n",
       "      <th id=\"T_b3835_level0_col5\" class=\"col_heading level0 col5\" >5</th>\n",
       "      <th id=\"T_b3835_level0_col6\" class=\"col_heading level0 col6\" >6</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th class=\"index_name level0\" >topic_idx</th>\n",
       "      <th class=\"blank col0\" >&nbsp;</th>\n",
       "      <th class=\"blank col1\" >&nbsp;</th>\n",
       "      <th class=\"blank col2\" >&nbsp;</th>\n",
       "      <th class=\"blank col3\" >&nbsp;</th>\n",
       "      <th class=\"blank col4\" >&nbsp;</th>\n",
       "      <th class=\"blank col5\" >&nbsp;</th>\n",
       "      <th class=\"blank col6\" >&nbsp;</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_b3835_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "      <td id=\"T_b3835_row0_col0\" class=\"data row0 col0\" >3416</td>\n",
       "      <td id=\"T_b3835_row0_col1\" class=\"data row0 col1\" >342</td>\n",
       "      <td id=\"T_b3835_row0_col2\" class=\"data row0 col2\" >410</td>\n",
       "      <td id=\"T_b3835_row0_col3\" class=\"data row0 col3\" >386</td>\n",
       "      <td id=\"T_b3835_row0_col4\" class=\"data row0 col4\" >134</td>\n",
       "      <td id=\"T_b3835_row0_col5\" class=\"data row0 col5\" >67</td>\n",
       "      <td id=\"T_b3835_row0_col6\" class=\"data row0 col6\" >69</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_b3835_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
       "      <td id=\"T_b3835_row1_col0\" class=\"data row1 col0\" >674</td>\n",
       "      <td id=\"T_b3835_row1_col1\" class=\"data row1 col1\" >3974</td>\n",
       "      <td id=\"T_b3835_row1_col2\" class=\"data row1 col2\" >663</td>\n",
       "      <td id=\"T_b3835_row1_col3\" class=\"data row1 col3\" >437</td>\n",
       "      <td id=\"T_b3835_row1_col4\" class=\"data row1 col4\" >238</td>\n",
       "      <td id=\"T_b3835_row1_col5\" class=\"data row1 col5\" >117</td>\n",
       "      <td id=\"T_b3835_row1_col6\" class=\"data row1 col6\" >119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_b3835_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
       "      <td id=\"T_b3835_row2_col0\" class=\"data row2 col0\" >473</td>\n",
       "      <td id=\"T_b3835_row2_col1\" class=\"data row2 col1\" >578</td>\n",
       "      <td id=\"T_b3835_row2_col2\" class=\"data row2 col2\" >3880</td>\n",
       "      <td id=\"T_b3835_row2_col3\" class=\"data row2 col3\" >1296</td>\n",
       "      <td id=\"T_b3835_row2_col4\" class=\"data row2 col4\" >440</td>\n",
       "      <td id=\"T_b3835_row2_col5\" class=\"data row2 col5\" >158</td>\n",
       "      <td id=\"T_b3835_row2_col6\" class=\"data row2 col6\" >537</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_b3835_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
       "      <td id=\"T_b3835_row3_col0\" class=\"data row3 col0\" >215</td>\n",
       "      <td id=\"T_b3835_row3_col1\" class=\"data row3 col1\" >164</td>\n",
       "      <td id=\"T_b3835_row3_col2\" class=\"data row3 col2\" >565</td>\n",
       "      <td id=\"T_b3835_row3_col3\" class=\"data row3 col3\" >4595</td>\n",
       "      <td id=\"T_b3835_row3_col4\" class=\"data row3 col4\" >182</td>\n",
       "      <td id=\"T_b3835_row3_col5\" class=\"data row3 col5\" >121</td>\n",
       "      <td id=\"T_b3835_row3_col6\" class=\"data row3 col6\" >91</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_b3835_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
       "      <td id=\"T_b3835_row4_col0\" class=\"data row4 col0\" >186</td>\n",
       "      <td id=\"T_b3835_row4_col1\" class=\"data row4 col1\" >300</td>\n",
       "      <td id=\"T_b3835_row4_col2\" class=\"data row4 col2\" >530</td>\n",
       "      <td id=\"T_b3835_row4_col3\" class=\"data row4 col3\" >462</td>\n",
       "      <td id=\"T_b3835_row4_col4\" class=\"data row4 col4\" >5390</td>\n",
       "      <td id=\"T_b3835_row4_col5\" class=\"data row4 col5\" >172</td>\n",
       "      <td id=\"T_b3835_row4_col6\" class=\"data row4 col6\" >589</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_b3835_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
       "      <td id=\"T_b3835_row5_col0\" class=\"data row5 col0\" >128</td>\n",
       "      <td id=\"T_b3835_row5_col1\" class=\"data row5 col1\" >101</td>\n",
       "      <td id=\"T_b3835_row5_col2\" class=\"data row5 col2\" >186</td>\n",
       "      <td id=\"T_b3835_row5_col3\" class=\"data row5 col3\" >348</td>\n",
       "      <td id=\"T_b3835_row5_col4\" class=\"data row5 col4\" >190</td>\n",
       "      <td id=\"T_b3835_row5_col5\" class=\"data row5 col5\" >5870</td>\n",
       "      <td id=\"T_b3835_row5_col6\" class=\"data row5 col6\" >110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_b3835_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
       "      <td id=\"T_b3835_row6_col0\" class=\"data row6 col0\" >83</td>\n",
       "      <td id=\"T_b3835_row6_col1\" class=\"data row6 col1\" >133</td>\n",
       "      <td id=\"T_b3835_row6_col2\" class=\"data row6 col2\" >508</td>\n",
       "      <td id=\"T_b3835_row6_col3\" class=\"data row6 col3\" >505</td>\n",
       "      <td id=\"T_b3835_row6_col4\" class=\"data row6 col4\" >390</td>\n",
       "      <td id=\"T_b3835_row6_col5\" class=\"data row6 col5\" >72</td>\n",
       "      <td id=\"T_b3835_row6_col6\" class=\"data row6 col6\" >5060</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x13a784310>"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.crosstab(df_accuracy[\"topic_idx\"], df_accuracy[\"valid_predict_idx\"]).style.background_gradient(axis=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "topic\n",
       "스포츠     0.846675\n",
       "생활문화    0.774482\n",
       "정치      0.749519\n",
       "IT과학    0.708126\n",
       "세계      0.706515\n",
       "경제      0.638701\n",
       "사회      0.527031\n",
       "Name: accuracy, dtype: float64"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# groupby 로 분류별 정확도를 구합니다.\n",
    "df_group_accuracy = df_accuracy.groupby(\"topic_idx\")[\"accuracy\"].mean()\n",
    "df_group_accuracy.index = topic[\"topic\"]\n",
    "df_group_accuracy.sort_values(ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 학습"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-5 {color: black;}#sk-container-id-5 pre{padding: 0;}#sk-container-id-5 div.sk-toggleable {background-color: white;}#sk-container-id-5 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-5 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-5 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-5 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-5 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-5 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-5 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-5 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-5 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-5 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-5 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-5 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-5 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-5 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-5 div.sk-item {position: relative;z-index: 1;}#sk-container-id-5 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-5 div.sk-item::before, #sk-container-id-5 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-5 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-5 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-5 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-5 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-5 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-5 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-5 div.sk-label-container {text-align: center;}#sk-container-id-5 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-5 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-5\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>RandomForestClassifier(n_jobs=-1)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" checked><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomForestClassifier</label><div class=\"sk-toggleable__content\"><pre>RandomForestClassifier(n_jobs=-1)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "RandomForestClassifier(n_jobs=-1)"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# fit 으로 학습시킵니다.\n",
    "model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: >"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0oAAAFFCAYAAADSEx0RAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABbkElEQVR4nO3deXxM1/8/8Ndkm0RCgiBIKpbaiTSCqIrWVkuppflQSy0fWwitILEkiFpiK12EUMTSqtprqZ2QltoSlNKWfJKKJZqQPZLM/f3hN/ebuTOJzMydCfF6Ph7zkLlznXPuMnfO+55zz1EIgiCAiIiIiIiIRBalXQAiIiIiIqKXDQMlIiIiIiIiCQZKREREREREEgyUiIiIiIiIJBgoERERERERSTBQIiIiIiIikmCgREREREREJGFV2gUwB5VKhaSkJJQvXx4KhaK0i0NERERERKVEEASkp6ejRo0asLAout3otQiUkpKS4ObmVtrFICIiIiKil0RiYiJcXV2L/Py1CJTKly8P4PnOqFChQimXhoiIiIiISktaWhrc3NzEGKEor0WgpO5uV6FCBQZKRERERET0wkdyOJgDERERERGRBAMlIiIiIiIiCQZKREREREREEgyUiIiIiIiIJBgoERERERERSTBQIiIiIiIikmCgREREREREJMFAiYiIiIiISOK1mHC2OO7BB/T+P/GLepigJERERERE9LJgixIREREREZEEAyUiIiIiIiIJBkpEREREREQSDJSIiIiIiIgkGCgRERERERFJMFAiIiIiIiKSYKBEREREREQkYVCgtHbtWjRr1gweHh7o1q0b7t27V+S66enpGDx4MJo2bYomTZogLCwMgiCInx88eBAdO3ZE8+bN0bRpU4wdOxZZWVkaady8eRO+vr5o0aIFPD09sWvXLkOKTUREREREVCJ6B0qHDx9GZGQkzp49i7i4OAwfPhx9+/Ytcv3Ro0ejcePGuH79Oq5cuYLLly8jIiJC/NzBwQGbNm3C1atXERsbi/T0dISGhoqf5+TkoHfv3ggLC0NsbCwOHTqE6dOn4+rVq/oWnYiIiIiIqET0DpTWrFmDsLAwODo6AgD8/PxgaWmJ2NhYrXVTUlIQExODoKAgAICNjQ0WL16MyMhIcZ327dujZs2aAAArKytMnToVR44cET8/cuQIPD094evrCwBwcXFBYGAg1q9fr2/RiYiIiIiISkTvQOn48eNo3769xjJfX18cPXpUa91Tp06hTZs2sLS0FJfVr18fjx49wqNHj3Smn5KSAltbW/H9sWPHxCDpRfkRERERERHJQa9AKSMjA1ZWVrC3t9dY7ubmhjt37mitn5SUBDc3N63lrq6uuHv3rs48Vq9ejaFDhxabRlH5qeXm5iItLU3jRUREREREVFJ6BUpPnjzRaO1Rs7W11RqAwZD1Dx8+jNjYWIwaNarYNGxtbZGTk6MxKERhCxcuhKOjo/jSFawREREREREVRa9ASalUIicnR2t5dnY27OzsjFo/MTERo0ePxnfffQelUllsGtnZ2VAqlVAoFDrLOX36dDx9+lR8JSYmlmj7iIiIiIiIAMBKn5WdnZ2RnZ2NjIwMODg4iMsTExPh6uqqtb6rqyt+++03reXS9TMzM/Hhhx/i888/R8uWLbXSSEhIKPb/SymVSo1gi4iIiIiISB96tSgpFAq0bt0a0dHRGstPnz6Ntm3baq3v4+ODmJgYFBQUiMtu3boFGxsbMdApKCjAgAED0K1bNwwZMkQrjbZt2+L06dMlyo+IiIiIiEgOeo96N3HiRISGhooDJGzfvh2ZmZno0KGD1rru7u7w9vZGeHg4ACAvLw9BQUEICAgQ15k0aRLs7Owwb948nfn1798f58+fF4OlBw8eYOnSpRg/fry+RSciIiIiIioRvbreAUCfPn2QmJgIHx8fWFhYwMXFBXv37oWFhQXy8vLg5+eHiIgIuLi4AAA2bNiAcePGoUmTJlCpVOjduzcCAwMBAKmpqfjmm2/QoEEDeHp6inkoFAr8/PPPqFatGuzt7bFv3z74+/sjIyMDKpUKc+fORevWrWXaBURERERERJoUQlFDx5UhaWlpcHR0xNOnT1GhQgWNz9yDD+idXvyiHnIVjYiIiIiIzKi42KAwvbveERERERERlXUMlIiIiIiIiCQYKBEREREREUkwUCIiIiIiIpJgoERERERERCTBQImIiIiIiEiCgRIREREREZEEAyUiIiIiIiIJBkpEREREREQSDJSIiIiIiIgkGCgRERERERFJMFAiIiIiIiKSYKBEREREREQkwUCJiIiIiIhIgoESERERERGRBAMlIiIiIiIiCQZKREREREREEgyUiIiIiIiIJBgoERERERERSTBQIiIiIiIikmCgREREREREJMFAiYiIiIiISIKBEhERERERkQQDJSIiIiIiIgkGSkRERERERBIMlIiIiIiIiCQYKBEREREREUlYlXYBXhfuwQf0/j/xi3qYoCRERERERPQibFEiIiIiIiKSYKBEREREREQkwUCJiIiIiIhIgoESERERERGRBAMlIiIiIiIiCQZKREREREREEgyUiIiIiIiIJBgoERERERERSTBQIiIiIiIikmCgREREREREJMFAiYiIiIiISIKBEhERERERkYRBgdLatWvRrFkzeHh4oFu3brh3716R66anp2Pw4MFo2rQpmjRpgrCwMAiCoLXe+vXroVQqER8fr/WZlZUVWrRoofE6ePCgIUUnIiIiIiJ6ISt9/8Phw4cRGRmJs2fPwtHREdu3b0ffvn1x/vx5neuPHj0azZo1w5YtW/Ds2TP4+fkhIiIC/v7+4johISG4ePEiKlasiPz8fK00CgoKcPHiRVhZ6V1cIiIiIiIivendorRmzRqEhYXB0dERAODn5wdLS0vExsZqrZuSkoKYmBgEBQUBAGxsbLB48WJERkaK66hUKlSvXh379++Hra2tgZtBREREREQkH70DpePHj6N9+/Yay3x9fXH06FGtdU+dOoU2bdrA0tJSXFa/fn08evQIjx49el4ACwv4+/trrENERERERFSa9OrLlpGRASsrK9jb22ssd3Nzw7Vr17TWT0pKgpubm9ZyV1dX3L17F1WrVtWzuCWTm5uL3Nxc8X1aWppJ8iEiIiIiorJJrxalJ0+e6OweZ2tri6ysLKPXL87777+PZs2aoXXr1lixYgVUKlWR6y5cuBCOjo7iS1ewRkREREREVBS9WpSUSiVycnK0lmdnZ8POzk7n+qmpqSVevyj379+Hi4sLACA+Ph5Dhw5FVlYWZsyYoXP96dOnY/LkyeL7tLQ0BktERERERFRierUoOTs7Izs7GxkZGRrLExMT4erqqrW+q6srEhIStJYXtX5R1EESALi7u2PBggXYsWNHkesrlUpUqFBB40VERERERFRSegVKCoUCrVu3RnR0tMby06dPo23btlrr+/j4ICYmBgUFBeKyW7duwcbGRq9ASaqgoIBDhRMRERERkcnoPerdxIkTERoaKg6QsH37dmRmZqJDhw5a67q7u8Pb2xvh4eEAgLy8PAQFBSEgIKDE+WVlZSE5OVl8Hx8fjylTpmDEiBH6Fp2IiIiIiKhE9G6W6dOnDxITE+Hj4wMLCwu4uLhg7969sLCwQF5enjihrLq73IYNGzBu3Dg0adIEKpUKvXv3RmBgoM60bWxsYG1trbEsNTUVPXv2RF5eHqysrGBnZ4dJkyZh8ODBBmwuERERERHRiykEQRBKuxCmlpaWBkdHRzx9+lTreSX34AN6pxe/qIfe/8dc+RARERERUdGKiw0K07vrHRERERERUVnHQImIiIiIiEiCgRIREREREZEEAyUiIiIiIiIJBkpEREREREQSDJSIiIiIiIgkGCgRERERERFJMFAiIiIiIiKSYKBEREREREQkwUCJiIiIiIhIgoESERERERGRBAMlIiIiIiIiCQZKREREREREEgyUiIiIiIiIJBgoERERERERSTBQIiIiIiIikmCgREREREREJMFAiYiIiIiISIKBEhERERERkQQDJSIiIiIiIgkGSkRERERERBIMlIiIiIiIiCQYKBEREREREUkwUCIiIiIiIpKwKu0CkLzcgw/o/X/iF/UwQUmIiIiIiF5dbFEiIiIiIiKSYKBEREREREQkwUCJiIiIiIhIgoESERERERGRBAMlIiIiIiIiCQZKREREREREEgyUiIiIiIiIJBgoERERERERSTBQIiIiIiIikrAq7QLQq8k9+IBe68cv6mGikhARERERyY8tSkRERERERBIMlIiIiIiIiCQYKBEREREREUkwUCIiIiIiIpJgoERERERERCRhUKC0du1aNGvWDB4eHujWrRvu3btX5Lrp6ekYPHgwmjZtiiZNmiAsLAyCIGitt379eiiVSsTHx2t9dvPmTfj6+qJFixbw9PTErl27DCk2ERERERFRiegdKB0+fBiRkZE4e/Ys4uLiMHz4cPTt27fI9UePHo3GjRvj+vXruHLlCi5fvoyIiAiNdUJCQvDjjz+iYsWKyM/P1/gsJycHvXv3RlhYGGJjY3Ho0CFMnz4dV69e1bfoREREREREJaJ3oLRmzRqEhYXB0dERAODn5wdLS0vExsZqrZuSkoKYmBgEBQUBAGxsbLB48WJERkaK66hUKlSvXh379++Hra2tVhpHjhyBp6cnfH19AQAuLi4IDAzE+vXr9S06ERERERFRiegdKB0/fhzt27fXWObr64ujR49qrXvq1Cm0adMGlpaW4rL69evj0aNHePTo0fMCWFjA399fY53Cjh07JgZJL8pPLTc3F2lpaRovIiIiIiKikrLSZ+WMjAxYWVnB3t5eY7mbmxuuXbumtX5SUhLc3Ny0lru6uuLu3buoWrXqC/NMSkpC586dtfK7c+dOkf9n4cKFmDt37gvTppebe/ABvf9P/KIeJigJEREREb1u9GpRevLkic7ucba2tsjKyjJ6/ZLmaWtri5ycHJ2DQgDA9OnT8fTpU/GVmJhYoryIiIiIiIgAPVuUlEolcnJytJZnZ2fDzs5O5/qpqaklXr+keWZnZ0OpVEKhUBT5f5RKZYnSJyIiIiIiktKrRcnZ2RnZ2dnIyMjQWJ6YmAhXV1et9V1dXZGQkKC1vKj1ddGVhj7/n4iIiIiISF96BUoKhQKtW7dGdHS0xvLTp0+jbdu2Wuv7+PggJiYGBQUF4rJbt27BxsamxIFO27Ztcfr06RLlR0REREREJAe9R72bOHEiQkNDxZHktm/fjszMTHTo0EFrXXd3d3h7eyM8PBwAkJeXh6CgIAQEBJQ4v/79++P8+fNisPTgwQMsXboU48eP17foREREREREJaLXM0oA0KdPHyQmJsLHxwcWFhZwcXHB3r17YWFhgby8PPj5+SEiIgIuLi4AgA0bNmDcuHFo0qQJVCoVevfujcDAQJ1p29jYwNraWmOZvb099u3bB39/f2RkZEClUmHu3Llo3bq1AZtLRERERET0YnoHSsDzVqWJEydqLbe2tsbu3bs1llWqVAk//PBDidK9ffu2zuUeHh6IiYnRv6BEREREREQG0LvrHRERERERUVnHQImIiIiIiEiCgRIREREREZEEAyUiIiIiIiIJBkpEREREREQSDJSIiIiIiIgkGCgRERERERFJMFAiIiIiIiKSYKBEREREREQkYVXaBSAqbe7BB/T+P/GLepigJERERET0smCLEhERERERkQQDJSIiIiIiIgkGSkRERERERBIMlIiIiIiIiCQYKBEREREREUkwUCIiIiIiIpJgoERERERERCTBQImIiIiIiEiCgRIREREREZGEVWkXgOh14R58QO//E7+ohwlKQkREREQvwhYlIiIiIiIiCbYoEZUxbLkiIiIiMh4DJSIyCAMyIiIiKsvY9Y6IiIiIiEiCgRIREREREZEEAyUiIiIiIiIJPqNERC81fZ+F4nNQREREJAcGSkT02jPXwBQcAIOIiOjVwa53REREREREEgyUiIiIiIiIJBgoERERERERSTBQIiIiIiIikmCgREREREREJMFAiYiIiIiISIKBEhERERERkQTnUSIiKmM4XxMREZHx2KJEREREREQkwRYlIiIyCFuuiIioLGOLEhERERERkQQDJSIiIiIiIgmDAqW1a9eiWbNm8PDwQLdu3XDv3r0i101PT8fgwYPRtGlTNGnSBGFhYRAEQa/0rKys0KJFC43XwYMHDSk6ERERERHRC+n9jNLhw4cRGRmJs2fPwtHREdu3b0ffvn1x/vx5neuPHj0azZo1w5YtW/Ds2TP4+fkhIiIC/v7+JU6voKAAFy9ehJUVH6kiIiIiIiLT07tFac2aNQgLC4OjoyMAwM/PD5aWloiNjdVaNyUlBTExMQgKCgIA2NjYYPHixYiMjDQoPSIiIiIiInPQu4nm+PHj2Lx5s8YyX19fHD16FC1atNBYfurUKbRp0waWlpbisvr16+PRo0d49OgRqlatqld6JZWbm4vc3FzxfVpamkHpEBFR6ePoekREVBr0alHKyMiAlZUV7O3tNZa7ubnhzp07WusnJSXBzc1Na7mrqyvu3r2rd3oltXDhQjg6OoovXWUgIiIiIiIqil4tSk+ePIGtra3WcltbW2RlZem9vj7pvf/++3j48CHKlSuHgQMHYuLEibCw0B3nTZ8+HZMnTxbfp6WlMVgiIqJi6dtyxVYrIqKyTa9ASalUIicnR2t5dnY27OzsdK6fmppa5PolTe/+/ftwcXEBAMTHx2Po0KHIysrCjBkziiynUqks8XYRERGZA7sREhG9OvTqeufs7Izs7GxkZGRoLE9MTISrq6vW+q6urkhISNBarl6/pOmpgyQAcHd3x4IFC7Bjxw59ik5ERERERFRierUoKRQKtG7dGtHR0ejevbu4/PTp05g/f77W+j4+PggMDERBQYE4oMOtW7dgY2MjBkL6pKdWUFDAocKJiIiKwJYrIiLj6R1tTJw4EaGhoWjXrh0qVKiA7du3IzMzEx06dNBa193dHd7e3ggPD8eMGTOQl5eHoKAgBAQElDi9rKwsZGZmokqVKgCed72bMmUKRo4cadgWExERkSwYkBFRWaZ3oNSnTx8kJibCx8cHFhYWcHFxwd69e2FhYYG8vDxxQll1d7kNGzZg3LhxaNKkCVQqFXr37o3AwMASpQcAqamp6NmzJ/Ly8mBlZQU7OztMmjQJgwcPlmkXEBER0cvMXAEZAz8iKsyg/msTJ07ExIkTtZZbW1tj9+7dGssqVaqEH374waD0AKBmzZq4cuWKIcUkIiIieulwhEWiVwMf9CEiIiIqY9g6RmQ8BkpEREREZBAGZFSW6TU8OBERERER0euALUpERERE9FLjgB5UGtiiREREREREJMEWJSIiIiIiM2LL1auBgRIRERERURnEoeiNw0CJiIiIiIgMUpafH+MzSkRERERERBIMlIiIiIiIiCQYKBEREREREUkwUCIiIiIiIpJgoERERERERCTBQImIiIiIiEiCgRIREREREZEEAyUiIiIiIiIJBkpEREREREQSDJSIiIiIiIgkGCgRERERERFJMFAiIiIiIiKSYKBEREREREQkwUCJiIiIiIhIgoESERERERGRBAMlIiIiIiIiCQZKREREREREEgyUiIiIiIiIJBgoERERERERSTBQIiIiIiIikmCgREREREREJMFAiYiIiIiISIKBEhERERERkQQDJSIiIiIiIgkGSkRERERERBIMlIiIiIiIiCQYKBEREREREUkwUCIiIiIiIpJgoERERERERCTBQImIiIiIiEiCgRIREREREZEEAyUiIiIiIiIJgwKltWvXolmzZvDw8EC3bt1w7969ItdNT0/H4MGD0bRpUzRp0gRhYWEQBEGv9G7evAlfX1+0aNECnp6e2LVrlyHFJiIiIiIiKhG9A6XDhw8jMjISZ8+eRVxcHIYPH46+ffsWuf7o0aPRuHFjXL9+HVeuXMHly5cRERFR4vRycnLQu3dvhIWFITY2FocOHcL06dNx9epVfYtORERERERUInoHSmvWrEFYWBgcHR0BAH5+frC0tERsbKzWuikpKYiJiUFQUBAAwMbGBosXL0ZkZGSJ0zty5Ag8PT3h6+sLAHBxcUFgYCDWr1+vb9GJiIiIiIhKxErf/3D8+HFs3rxZY5mvry+OHj2KFi1aaCw/deoU2rRpA0tLS3FZ/fr18ejRIzx69AhVq1Z9YXrHjh0Tg6TCn69cubLIMubm5iI3N1d8//TpUwBAWlqa1rqq3KziN1gHXem8yOueT1naFubzcudTlraF+bzc+ZSlbWE+L3c+ZWlbmM/LnU9Z2pbi8lEvlz4OpEXQQ3p6ulCpUiWt5d98840wduxYreVfffWVMHnyZK3l3t7ewrlz50qUXr9+/YR9+/ZpfJ6ZmSnY2toWWc7Zs2cLAPjiiy+++OKLL7744osvvnS+EhMTi4199GpRevLkCWxtbbWW29raIitLO8p70folSU/XOra2tsjJyYEgCFAoFFr/f/r06Zg8ebL4XqVSISUlBZUrV9a5vi5paWlwc3NDYmIiKlSoUKL/oy9z5MF8mI+58mA+zMdceTAf5mOuPJgP8zFXHszHvPkIgoD09HTUqFGj2PX0CpSUSiVycnK0lmdnZ8POzk7n+qmpqUWuX5L0dK2TnZ0NpVJZZNCjVCqhVCo1ljk5ORW5XcWpUKGCSQ+uufJgPszHXHkwH+ZjrjyYD/MxVx7Mh/mYKw/mY7581OMjFEevwRycnZ2RnZ2NjIwMjeWJiYlwdXXVWt/V1RUJCQlay9XrlyQ9XWkUlR8REREREZEc9AqUFAoFWrdujejoaI3lp0+fRtu2bbXW9/HxQUxMDAoKCsRlt27dgo2NDVxdXUuUXtu2bXH69OkS5UdERERERCQHvYcHnzhxIkJDQ8XRIrZv347MzEx06NBBa113d3d4e3sjPDwcAJCXl4egoCAEBASUOL3+/fvj/PnzYrD04MEDLF26FOPHj9e36HpRKpWYPXu2Vhe+Vy0P5sN8zJUH82E+5sqD+TAfc+XBfJiPufJgPi9nPgpBeNG4eNq+/PJLrFmzBhYWFnBxcUFkZCRq166NvLw8+Pn5ISIiAi4uLgCez6U0btw4XL9+HSqVCr1798aCBQtgYWHxwvTU4uLi4O/vj4yMDKhUKkyfPh0ff/yxDJtPRERERESkzaBAiYiIiIiIqCzTu+sdERERERFRWcdAiYiIiIiISIKBEhERERERkQQDJSIiIiIiIgkGSkRERERERBIMlErBRx99ZPY8w8LCzJ6nOZTV7ZJDaZxn9Poy9/n2+eefmzU/Kpmyek0uq9uldufOndIuQonxt80wr9N+k/P7yuHBS0Hjxo1x48aNIj8/c+YM3nnnHbPmaYzRo0cjMjLSJGlLJSQk4I033hDfG7td165dQ15eXonXt7a2RrNmzQzOz5xMdcyL2mcODg6oW7cu4uLiNJZXrVoVrq6uspYhLCwMoaGhsqR1//59vc4BKysr1KhRQ+98SnKuvfnmmyhfvjyA54HArFmz9MrDXNuiiymvMWpbt25F165d4ezsjObNm+Pq1asmzQ+Q91wrTbGxsYiKikJ8fDwUCgXq16+P0aNHo06dOrLmY47zwFx++eUXWFpaonXr1jq3KysrC+XKldM73Y8//hjfffcdAKBevXp49uyZ1jqCIMDW1hZ//vmnYYXXwc/PD9u3b9f52TvvvIMzZ87onWZJrmsNGzYU99OcOXMwZ84cvfMprCydYwAwbtw4REREiO8DAwMREhICJycnWfMxd50AeD4RbJMmTZCeno6YmBh4eXmhSpUqspdBSs5ttZIllVeUuQ5u3759kZ+fL75PTExEr169NNYJDAyEr68vAGD8+PEGVQCmTJmCpUuXiu8/+eQTREVFaawza9Ys3Lx5E+r4eO7cuUZX/M+ePVvs59euXTMoj27duuHQoUMayz744AONirixcX5gYKBeFUsbGxscPnxY73xKGpDVq1cPFSpUwNdff40JEybonU/hc63weVavXj0sX74ckydPxt9//43OnTtjwoQJ8PLywqVLl/TKo/A+u3jxIry9vSEIAmrXro158+YhMDAQFy5cQKtWraBSqXD9+nU8fvxY720pzrZt23RWXv/9919UrlxZr7R69uyJvLy8Ep9LNjY2eu8z4MXnmkKhwIwZM9CpUycAwPbt2/UOlMy1LdJrGqD7uqZmbW2NnTt36p2P9MZIeHg4+vbtC8D4735xcnJycPr0aXTt2rXIc80QxV0H3nzzTeTn5yM+Ph6CIEChUKB69eri5O3G2LZtGyZMmIBx48bhk08+gSAIuHDhAtq1a4dt27ahffv2BqU7Y8YM3LlzB5UqVULdunXRsmVLva6nJWXOGwBJSUni//3xxx/Ro0cPALrPt7Zt2yI2NlbvPJYsWSL+HR0dXeS2WVtb6522LhkZGXj06BFu3LiBu3fvam1LfHw87t27Z1DaJbmuLViwAG3atAEA7Nq1y6BAqajfNgDo378/hg4dipCQEOzevRsfffQRZs+erXceQOncbIqJidF4f+HCBWRlZckSKBW337y8vDB79myj95v6HLh48SJatmypUS9wcHDA5s2b0apVKzRo0ADXr1/HkSNH8Oabbxq8Tf/9739x8eJFWFlZoVy5cnB2doarqyvq16+PESNGoFy5crL+PrzWLUpdunQxy8H95ZdfXvjFa9q0qVjBa9asGa5du6Z3PtIIuvB79d9ubm5Yt26duI6Pjw8qVKigVz6jR4/WqCTt2rVLrLyoeXh4YNKkSQBg8J3fevXq4fvvvxdPeEEQMHjwYI07bK/KnSX1uSalUCjESkurVq0QFBSErl27GrzPijrXnJyc4OHhgTp16mDt2rUIDAxEbGysweeaWlH/v1GjRrh582ax65RUaGgofv75Z4wcORJjxozRSr8wDw8PrRatV5Wx+82USnJNK8za2hpt27bVO5/q1avjvffew9atW3Hjxg1MmTIFBw8eBGD4daUk9uzZg927dyMqKkrjXNuwYQOioqLQrVs3BAUF6Z1ucdeB6dOnY+7cuVCpVFAqlRAEATdv3sTDhw+N3h5PT09s2rRJ64ZVXFwchg4davB3xs3NDVu2bMHTp09x584dXLx4Ebt370ZmZqbRZS7My8vLLDcAAMDZ2RmTJk1CSEgIvLy8EBMTA1tbW52/NXJ8R1UqFTZu3IjLly8jKytL47OCggKtm52GWLVqFZYsWYL79++jevXqGp9ZWlqK2zxw4ECj83oRQ/dZcdec2rVr49ixY9i5cyeWL1+OGTNmoGfPnhg+fLje+ZjrXCtcj9qzZw8+/PBD8bODBw+iQ4cOGq2V1tbWWLNmjd75FLffqlSpgvPnz2PHjh1G7zfg/67J0mO8bNky5ObmYsaMGdi3bx8OHDhg0Lao1alTB9HR0RAEARkZGXj8+DGSkpLwxx9/YNKkSXBycmKLklyOHDkC4PnBPXnyJJo1a4YTJ06Iny9btgzDhg0TD+7SpUsNOrj6VhAUCoXeeQDad7x0fdEdHBzQtWtXg9JXU9+1VlPfcSusZs2axZajJB4+fIiZM2dq/H85Kg2lQX2uSW3evBlDhgxBo0aNNM49Q/fZi841BwcHdOzYESqVyqD0S6rwOWzo+Qw8b1U5d+4cFi9ejODgYHh6eqJVq1ZFpmnodj158kTn3bvbt29j48aNWLBggUHplkR0dDQeP36sdbPBmP1masWdZ+fOncODBw80fvgNZWdnh7S0NCxZsgQPHz7E4MGDjU6zJL755hvMmDEDwP8dh4sXL2Lp0qWIiIjA559/jpo1a+pdnqKuA2qTJ0/WqGDI1c3XwsJCZ1oeHh6wsDD8UWUHBwexJ4Rao0aNDE6vKIYGPYYoX748zp49ix07dsDZ2Rm2trZFrivHd3TChAmIj49Hz549ER4ejqCgIPz+++/Yv3+/Rg8RY/j7+8Pf3x+enp64cuWKLGkaytB99qLftg0bNmDTpk2oXbs2vvjiCwwaNMigCr+5zrXC9ShpHUpXncrQ1kVd+23Lli2oVKkSfH19MWbMGFn2G1B0veXEiRP45ptvADzf7pCQEIPSV1MqlbJ35y/Oax0oqZnj4D569AgODg4G9WcuKekFyFQVraK612zatAn29vbo16+fLOVwd3fXqliY+/mgw4cPY/HixTh+/LhJ0t+/f79WBRkwfJ+tXLkSBQUFGsvc3d115mGMr7/+Gk2aNCnyx0uuhuqoqCh89dVXaNCgARYuXIiNGzeiVatWRaZv6H7r0KGDzi40VatW1eoWYYxff/0Vb7zxhngjISMjA6NHj8a2bdtky+OXX35BQkICmjRpUirP0z179gy7d++WJVBycHDAxo0b4eXlBQcHB4SHhxtfwBf44YcfYGNjg3fffVdj+ZYtWxAaGor27dtjyZIlmDJliuyBm6mu4U5OTjpbE27evKl3j4LCdJXvZQ7wS8LOzg5ffvklWrZsiUWLFpk8v7NnzyI2NhYWFhb4+uuv4e/vDwAYMmQIwsPDZX0Af9q0aTqX5+bmYvr06Vi+fLlB6e7atQvx8fEay0zxu7Nz507873//E98rFAq89dZb8PX1xePHj1G7dm0AwBtvvIHU1FRZ85ZbUfUoU9i6dSsGDRokvre3t8eZM2fQvXt3WffbypUrxX/j4uKwe/duzJkzB0lJSWJgY2FhYfSNWnNfYzjqHbQPrrr/rJwHt3HjxnB1dUXv3r1x+fJlo8tcErdu3UK5cuVgZ2cne9rp6elISEjQ6NdsY2Mj692q0v4RjoiIwJQpU7B69WpZ0luxYgWaNm2Kd955R3xew9XVFX///bfWXVlDpaenIz09HUePHsWJEyeQnp6O8ePHy5J2YWFhYZg7dy6Sk5ORkZEhe/pqd+7cQYMGDQAArVu3Ntl35/Hjx6hfv77Wy9vbG//8849s+XTv3h1eXl4YN24cHj58iA8++AATJkxAixYtZMtjxIgROHbsGEaNGgV3d3csXLgQOTk5sqX/IvXr18etW7dkS69y5cqYNWsWateuDUtLS3G5IAgoV66c+LKzs0PFihWNymv37t2YN2+ezu5OsbGxYvDk4eFh1ChhW7ZsgY+PD9q3b499+/YZnE5xkpKSkJCQgISEBIwfPx5du3ZFVFQUrl+/jtjYWKxcuRL9+vUz+u6ulPS4yHVs/vrrL1y8eNEs53KDBg3QsWNHeHh4mDwvhUIhtuqVL18e9+/fBwC0adNG9pHo1q1bh5kzZ2osu3r1Kry9vY16Hmb06NHib096ejrS0tLEgE9OY8aM0cjn8ePHmDhxIgB56wa//PILtm3bZrauz4cOHcLs2bPh7++PefPmyXpzDgDmz5+v8b5u3bpiYCvXfps3bx62bNmCxYsX47333kPFihXFXjL29vbIzs4W1y18HZdDdHQ0QkNDxZfc17TXvkVp3rx5uHv3Li5evIhp06YhISEBJ06cwJw5c8SDqx6FypiDW61aNVy9ehW7d+/Gf/7zH4wcORLBwcFybYZO9evXF/vWN27cWNa0GzVqhIoVKyItLQ1Tp07FhAkTULt2bRw4cEDWfMzh2rVrUCqVqF69OsqXL4/o6GjMmTMHLi4uOHPmjCwPVC5btgy///47jh49itTUVIwaNQoVKlRA9erVcf/+fdmCMfXD/ytXroS1tTX8/f2xY8cOWdIurFq1atiyZQtatWqFzp074/jx4xqtpXJdfAu3HJUrV85kQZmTk5POVkNBEAx+0F0XFxcXnDt3DnPmzIG7uzv8/f0NGrSjONbW1uJziPfu3cOyZcvQvHlzfPfdd2jZsqWseeni4uKC5ORkWdMcMWIEvvjiC8THx8Pd3R3A83NM+kyHMTw9PXH79m3ExcXB2dlZ6/OUlBSN5YZ2WYuKisL69euxYsUKZGZmYtKkSbC1tUWXLl1kfQB52LBhyM/PF9OsXLkyNm7ciI0bN4rrVKtWDUuXLhUHEJGD3MdFrVevXnBxccHNmzfRtWtXzJo1C/Xq1ZM9H7Xx48dj69ataNeuncnyAABbW1txEBofHx+sX78eM2fORHJysuwDlvz8888YP348+vfvj6ioKERERGDdunVYt26dUdtZpUoVrUEADBm8xZB8du/eDeD5oAqZmZlivc2Y+tqIESPQrl07rFixAg8ePMCYMWPw2WefFdsN0xB3795Fz5494eLigi5duqBly5a4d+8ePv30U9jZ2WHfvn2y1D+k51HNmjWRlJQEQL79FhkZiRUrVuCrr77CtGnTxJvAwPO66JUrV9C+fXskJyfDwcHBuA2SKFeunN6DOOnjtQ+UzHVwLS0tYWlpif79+6Njx47o1q0bAOgMluS6OJqy9aV8+fK4du0ajh07hujoaABAjRo1ZK0gJSQkwM/PT3wvCAISExNlS1/N398fycnJSElJQXZ2NrKyshAcHKx1F8YYmzdvxm+//QYbGxtUr14da9aswezZs9GrVy+cPXsWdnZ2qFSpEpo2bSpbniVh6DmiUCjg4OCA7t27o2rVqggICMC3336L/Px8xMXFIScnB1euXEFBQQHS09MNLl/hVtzMzEzxpoXcLCwsUK1aNZ2fWVlZIT8/H1ZWxl8uFQoFHB0d8cUXX6B9+/YIDg5GQECAWPk/efIkUlNTIQgCnj59anAeajVr1sTy5csxcOBAfPzxx4iIiEDHjh2N3o4X0TXcsTEsLCwwYcIEfPPNNxojhsnpl19+wbJly9CnTx+cPn0alSpV0vi8UqVK+Pfff8XRTw29Tn/xxRc4dOiQ2A1uw4YNCAkJQZcuXYzbAIkXPQ/1qrG0tMSJEyeQnZ2NevXqITo6GlOmTDFJywUAdOzYEePHj0deXp5so8/por5ZW7lyZUybNg3t27fH9u3b8ejRIyxcuFDWvKytrREZGYlVq1bB3d0dbdu2xfnz5+Ho6GhUui/q/fHrr7+KAyQY83tQ3O9V9+7d8eWXX2L69OlYvXo1OnfubHA+5rrZNHbsWAQEBGDs2LEay0NCQjBz5kxMnToVa9euNTof6X5zcnISR6KVa7+VL18e/fr1E2/MWlhYiM9gDR06VBzyfN26dRgxYoQRW6OtZcuWWsfFmMEipF77QKk0Dm7FihWxd+9e+Pj4wNfXFz4+Pqhfvz4ePHgAQRBQq1Ytg9JNTU3F1q1boVAokJuba9RDui+i/uIV/gI6ODhoNK8aq1KlSuIoZ8Dziokpul4VnjsiMTERR44cwYYNG+Dt7Y3Vq1fDy8vL6DwKCgpgY2Mjvq9duzYePHiAihUrYufOnYiNjUWrVq1MEigVPkbqyt3jx4/x3nvvGTVst52dHbKzs/H555/Dy8sL586dg5ubG6ZMmYJatWph8uTJAGBUt0J3d3fcuHEDjRs3xrlz50zWIpKTkyPOa1KYIAjIzc1FZmam0ZUJqT59+sDW1hZdu3bFb7/9BkdHR5w4cQL//PMPFAqFrBVAb29vHDp0CF27dsVPP/1kkgfuC7OxsZEluCwcjAwdOhReXl4mC5Ts7Owwa9YsVKtWDQMGDNAKNDw9PREdHY1+/frh+vXrYnCrr9TUVI1nhTw8PMRuMBYWFjh69CiUSiVUKpXBwXJZpL6O2dnZoUqVKjh58iQGDhyIu3fvynpOqM85CwsLdOjQASdPntQIYo8fP46CggKjK/1qhdOuUaMG/vjjD8TFxaFq1apwc3MzOv2zZ89q3bho2LAhxo8fjw0bNuDkyZOoUKECbGxsTNZ6tmTJEuTl5UGhUJhs0tPg4GB06dIFS5cuRb169QyaxkPNXDebrl27VmQ5Z82aJWuX7MIsLS3FLqxy7jcppVKJZ8+eoUOHDggICMDq1avx7rvvyh4omdprHyjpYoqDK737WK1aNXzzzTcICAjAxYsXcfv2bWOLjeDgYMTExIhzcOiag0XuCcwKK1eunBgozZs3D+vXrzcqPUdHR60LkikHwwCeD3c7cuRIjBw5Evv378eAAQMQHByMkSNHGpVutWrVEBcXJ/Z537t3L1q3bg0HBwd8/PHH4rEaNWoU8vPzDW6Z++abb1BQUIDTp0+LLSH//vuv+Pn3338P4PkIXvfv3zd4/gdBEKBUKpGdnQ0LCwssWrQIISEhOHr0KE6ePGlQmroMGTIE/v7+mDp1KmbPnq01sfGzZ89w+vRpCIJgVMUlPz8f586d09lKkJOTozVAhqGk6Xfr1g1xcXEICAjApk2bMG/ePNnzUKtbty5Wr16NwYMH4+LFi0a3OC9cuFDnsLMqlQrp6emy9EMfNWqU+Le9vT2aNGmCc+fOiXOymMKoUaOwd+9e7N27F7179xb359ChQzFkyBBUrVoV8+fPx7BhwwxK39HREf/884/4/GtcXJwYdI0cORJfffWVmKcxD3wXNZFpYcZOaiodHMJcrKysULFiRezbtw+9e/fGF198gc8++0yWtAvf3e/QoQOio6PFYKagoADr1q1Dfn4+FAqFwedAcaysrGS5Oac2b968Is+DOnXqiM9oK5VK/PzzzwblkZycjLCwMPG9IAgaI9Tu2rXLoHSlHj9+jGXLlonXrtzcXPE64+TkhN9++w0PHz4ssneAMUxxs8nOzk7nICvA82fM5ZqYNSUlRWOgjsJdcuXab+r0VCoV5s6dK/aKUN8sGzp0KIYOHWrEVvwf9XPL5vLaB0rmOri6RoDq1q0bVq5ciZs3b8rypVPPW6SLejt//fVXo/MpnJ6VlRUuXbqETZs2aVQop02bBn9/fygUCoNHVbp48aLWMumzPKacBqxnz55o06YNevTogbfeeguenp4Gp7VkyRL85z//Qbt27ZCRkYG7d+/i8OHDuHbtmsaPWM+ePVFQUICAgACD8klNTUVBQYEYkKWmpuLzzz8XP1e3WNWoUcOoSfI2bNgAAOKDtF27djXJsKpDhgzBX3/9hSVLliAgIABvvfUWgP+745eamoq1a9eKNwcMrbjY29vjyy+/1PmZnK2YhZ8PUZs6dSq6du2KtLQ0o0YgK5xeUTp16oRu3brhwYMHRldy7e3ti5wTKCoqSpauv+rzS23BggWoWrUqANN+9z/77DMsWbIEvXv3Fpd5enoiMDAQM2bMQNeuXQ0e8W7y5Mno06cPZs+ejezsbMyePRurVq0C8Hx7pdtsqOImMi3MmG5lRT3XZwqF01XfzLSxscF3332HVq1a4b333pNl8IXCv6OdOnUSb84JggBLS0vxZtOrQs4WgqJERkZqDTxR1PXUGMuXLxcfjQCet/pJp26QI0gy182mqVOnon379ggPD0f79u1RsWJFpKSk4Oeff8bs2bNl6XYHAIsXL8bdu3fF9wqFQqsuZex+U49IOnPmTBw/flycH8wUPZvUz6UVR9brkPCa27t3ryAIghAXFycsX75cWLp0qbBz585SLpU8+vfvL/69ePFiWdNetWqVIAiCkJmZKUyfPl2YMGGCEBAQIOzatUvWfF4kPDzcJOnOnTtX/Ds/P1+WNJ8+fSocOHBAOHHihPDs2TNBEATh7NmzwrRp02RJ/3XSsGFDWdOLi4sr8rOjR48KBQUFsuZXGubNm1faRZDVkiVLTJq++lom97kmCIKwY8cOoVevXkL//v2FU6dOyZ6+LvHx8cL27duFqKgojdf3338vaz6muiYXPn9HjhwpZGdni+/37t0rDBgwwCT5qplqu8hwhes4coqKiir285kzZwpJSUmy5HXo0CGhd+/egqurq2Bvby/UqlVLGDRokHDhwgVZ0n9dyfl9VQiCCW/LEYDn/VBfdGevYcOG4p2rzz//XGe3OX3JOTPx6+LatWuoW7cuWrZsaZZ9l5qain/++Ue2+W4Kz/ZdEobO9l0SYWFhCA0NNUnaixcvLnJOEFMx5faYg3rW9MLUo22Zyqu+z4DSOdfktmHDBnzzzTfo1KmT1nQRNjY2mD59umx57dmzB97e3hqTjptDQUGB0d09S/JbXZi1tXWpzFX2qjhy5AiqV69usn3EOo5hzHldljuv4r6jNWrUgIODA+7evSu2KFWoUMHg50nVGCgVQc6D26VLl2IvvgqFAgsWLBD73euq0JREv379NPI5efKkOO/HhAkTcPPmTZw4cULsptSgQQMsXrxY73zu37+v14+JlZWV3t28ivsyVKpUCU5OTuKXQaFQoEaNGkY1Hefk5MDW1hZjx47F1KlT8cEHH+DGjRuIiIjAqVOn4OvrC39/f7zzzjsagz/IsT2urq5QKpW4c+eO+OW2t7c3qB/u0aNHNbryCYKA8ePHi117pKytrWUfbUvtVf8RW758OUaNGiWOtGfs9pSkEmZjYyN2jwwNDdXo918SY8eO1Rgp0N3dHTNmzADwfLJm6bwgHh4eiIuL0yuPF1EPNQsYv8+Ku9Y4ODigUqVKUKlUuHPnDurUqWNUN4+SHJ+mTZuKg7LIdUPLHDw8PHDu3DmTzKknNX36dHz00UdiN1k5mCuAedFvtZSNjY1Zura97G7cuAFLS0ut36zw8HA0atRIlslV+/btq3ETUBAEnDp1SmtiaDVra2uDhigv7lxzdHRE3bp18fTpU1y4cAGtWrUyuLt0aQbl5vxtljuvrl27atRxLly4AG9vbygUCvTt2xc7duyAg4OD2KX4zTffNKieWxgDpSKUZiVPV4WmJH755Zciv3iNGzdG586dER4eDltbW6hUKgwbNkxjluuS8vLy0mv4XxsbG72fXyn8g3X16lU0bNgQNjY2UCgUaNmyJWJiYqBSqcRt+eOPPzQeHtXXm2++iU2bNiE8PBzbtm3DW2+9hRs3bsDT0xOff/45pk+fjqtXrxp8bApvj/oCqw7yBgwYgKioKFhYWIjPxtWsWRNbtmwxeHsKM8W5/Mknn2hNyvn+++9rPAzcqFEjcR4vQ5mzolxYQkICunfvjuvXr4vLjN0eaSXs4sWL8Pb2RlZWFv788080b94cdnZ2OHjwIADDjtuePXtQUFCAwMBALFu2DJ999hkSEhIA6L4BY+j5DDyfp0e9P7p37y7Ob9K0aVNxvxm7z7y8vMRhhaV8fHywaNEitG/fHgUFBVAqlTh58qTBE5uqj4/6uAiCgLi4OLRo0UL8rn711Vdo0qQJAMNuaJVWi2+TJk3w+++/G51Ocd59912cPHlSI1D66quv8N1336FLly6YO3euwWlLK0fPnj3DzZs3i3wmiQGMeYWHh8POzg4TJ07UGO1SHSh16dIFgwcPhqWlJaKiogyaj6i4+o0u1tbWaNu2rd756LoOqP9u1KgRZs6cCV9fX9SrVw//+9//cPz4cYOe9yx8Tl+8eFFjRNfCeasZek4HBQXh1q1bcHJyQo0aNfDmm29i/vz5+Ouvv8R1goODcfPmTTG/sWPHonv37iXOIyQkBPPmzcPWrVsxaNAgAM9vzn/99dfib0BOTg6GDBkChUKBzZs3Q6lU6r0tukh/Y4z5TSuSbJ34XkGzZs0SBEEQtmzZIi4bP368IAj/1y89Oztb6N+/v/DRRx8JOTk5spchLy9Pa1mzZs1kz0cQBKFp06bFvn9ZderUSbh3757GMulzA4Zui/rYu7i4CL169RK6d+8uCIIgNGrUSBAEQfD09NT4V459puuZB7meg3jjjTeEKlWqaLysrKy0ljk7Owu1atUyOB9d56h0G9T70BhvvfWW0KxZM6Fp06aCra2t+HfTpk2FUaNGCf/++6/QpEkToWHDhoKHh4eQkpJidJ6CIAg9evQQfvjhB41lcmxPYepz6c8//xQ6deqk9bkx54T6+BQ+X3UdM2OuNbVq1RJu3bolXL16VfDw8BCXFy633PtMaubMmeKzK0uXLhVmzJhhdJqF91mHDh1KtF5JHTlyRNi/f3+JX4cPHzZoG6RmzJghzJkzR7bnLXVRH+vg4GDh0qVLQnR0tODp6SlER0cLnTt31vidNdb9+/eLPTZkXosWLRJWrlwpXL9+XXBychIaNmwopKSkCIsWLRL27t0rLFq0SPjss8+EKVOmCPPnzy/t4pZI4e934b8DAgKENWvWCILw/FmmCRMmyJqXrvfGqFmzpnDq1Cnh0KFDQlRUlDB79myhZs2aGuu88cYbwrFjx8TX/fv39cpD/TuiricV/lt9XVi4cKEwatQoYezYsUY/P/TPP/8IN27cEHJzc7V+Y0xRr32tR73bu3cv5s2bh2XLlolR8C+//ALg/0bVWrFiBSpWrAhLS0usXLlS1r7q27dvx4EDB7TuzBvjyZMnsLe31zmSkXSEFlNOSGtqcm3L/PnzMWjQILi4uKBixYqyTphblBdNzmcMQ1oI5WKK86lwS2SjRo207uDPmjULAwYMwKxZs7Bs2TIsXbrU6ImCJ0+ejCpVqmhMdiwndZc6dXeUSpUq4b333tNaT+79qR52Xvj/Q0IbO8KenZ0d6tevDwAaw6eb+rpy9+5d/PLLLxg0aBB++uknsSvsmDFj8M477xh9/AuXv7htMWQ73d3dS3xX3NraGm+++abeeegSHByMjh07IiIiAvXq1dOY083GxsbgYaGLIggCNm/ejDlz5uCdd97BkiVLMGXKFPF31ljSfZ+Xl4f8/HyTdy08fPgwFi9erHO0v9fVvn37oFAoIAgCwsLC8OOPP+L3339HREQErKysIAgC9u/fjz179sDKygrvv/++2B1YTtu2bYMgCBg4cKAs6RV1HThx4oTYlWvAgAFYunSprHnpem+M8uXLa81l+MMPP2i8r1ChgixzQwnFdFA7cOAAfvzxR1hYWKBPnz5G1aW9vb1Rt25d/PnnnxAEAYmJieKcY926dcP9+/fFstjZ2Rncy0DttQ6U1Mx1cAuLiYlBaGio7LOnv/nmm8jLy8P777+PefPmyfZDq0t8fDxWrVqF8+fPIzk5GdWrV0enTp0watQoODs7y5ZPeHg4qlWrhj///BNbt27FnDlzTDIEbffu3cUhyf/44w9YW1trVCiMpVKpEBwcjLt37+KDDz4QhwaV299//407d+6gZcuWRl8gXkZ//PEH0tPT4e3tLWtF+cGDBxg7diwqVqxo9Bxgxfn6668xZ84chISEYM6cOfjjjz9MNgljYX///Tdat24NQRDg4eGBPXv2GPU9KmlAYazr16+jbt26YiU4JCQEPj4+AJ53wVIHfA4ODnp1zdFFpVIhNTUVb7/9NhwcHPDkyROj0pOaMGGCVrflwl1xC5Oz+9iIESPQt29ffPjhh1rBhDHDg+uiPheuXLkiDt3s4eFh9FyBKpVK7FpbpUoVfPHFF+JnW7duRUpKijjJtSlERERg1apVss0JVFYEBQVh+PDhAIA7d+6gU6dO8PDwwLBhw8QKenJysjhoTGpqquxl+Prrr7F27VrZvi+bN2/WmJOv8HdTfaMJeP4dLfxMqCHS0tJw//59DB8+HI8ePULNmjWRmZmJvLw8Wb6bJbkxa46b5vfu3YOLiwsAICkpyai0qlWrhjNnziAvLw/btm1Dp06dsGTJEvTq1QtxcXHo3bu3OLBL48aN8e233xqVHwOlF5Dr4D558gQVKlRAdnY2Vq5ciW3btmHfvn1444035CoqgOd3py9duoTvv/8eHTp0wIoVK0xSCTt69ChGjBiB8ePHY8mSJXB2dkZSUhL279+Pt956C0eOHEHDhg2NymPUqFH4+++/Ua9ePURGRqJq1aqIjo6WaQu0vfvuu9i7dy+A5xOa3bx506i5k6QWLVqEy5cv49dff0V6ejomTZqEr776Cm+//basgd/AgQNRqVIl3Lx5E05OTuKdXFPMZ2AOf//9N2rXri1O+hsYGIhBgwbB29tbtopynTp1xECp8MR8curXrx927twpHuvBgwejXr16GDFiBBYsWIBnz57JdjdUl/r162v13TbmB1JaeVC7desWypUrB0EQULt2bYPTV2vbti0qVqyIRYsWwd7eHnfu3MHmzZsBQGuUM2PP8QULFqBWrVoIDw9HQUEBpk6dKk48KwddFblGjRrhxIkTsqRflNu3b+PHH380aR7q456fn49+/fohLS0NlSpVki39Vq1aiTeyVCoVdu3ahRYtWgB43lKn7g1irGvXrkGpVKJ69eooX748oqOjMWfOHLi4uODMmTMmnbT9Vae+DlSuXBmPHz8WlxcOJuT8rfv777/x6aefws7ODmfOnJFlHjrg+VxajRo1gp+fH1asWKExN530mmnMNefq1avo0aMHPvjgA3Tt2hVVqlTBvXv3kJqaiqZNm+Lo0aOy1xGB5/XRwvODmeKGelxcHMqVKyeOOFe414Gx50CdOnUAPL/JM2TIEHTs2BHdu3eHg4MD7t+/b9BgaMVhoCRhqoPbqlUrJCcnIz8/HzVq1MCvv/6q8SPy008/iQ8tP3361OB8LC0t4eDggFGjRqFTp07o1KkTKlWqJEuzamEzZszAkSNHNCbKrVOnDtq1a4eOHTsiODgYe/bsMSqPY8eOYceOHejfvz+A5yPOFL74ymHEiBEAnl/8qlSpgvv374vvC/+rZkzFcvPmzTh16pQ4Ot/q1asxf/58MTiTS3p6On777TcAwF9//YWlS5fiiy++wPbt21GvXj1Z8zKHFi1aoE6dOti4caM4sfHHH38MQL6K8u+//46ffvoJs2bNgpWVldGj5OiifuBUoVAgOTkZf/75J3bs2AEAqFevHv773/+aNFCS+65hUenVr19f3NbGjRsbnU/NmjVx8OBBDB8+HLdu3cKlS5fEvMuXL4979+6hZs2aePjwoTjanqGioqJw+vRpcZTOXbt2oUuXLrIFSsDzFrLY2FjUqVMHLVu2NMvdXEtLS2RkZMDBwcFkeTRo0AA3btwQhxp3cnJCamqqbMPP5+bmin9bWVlh37594qiQdevWRWJioiz5+Pv7Izk5GSkpKcjOzkZWVhaCg4ON7tL5OpEe98KBUuE6lb6cnZ2hVCpRq1Yt2NraIi4uDt9++y0+/PBDY4qrpVq1aoiJiUFUVBQ6dOiA/fv3a5Thr7/+Qr169RAfH29Ur42AgABERESgZ8+eGsuHDBmCiIgITJ06VaubnBwcHR01bvw3b95c9jw8PDxw+fJl8TdAPcAHoP27ra/8/Hz07NkTffr0wfDhw1GjRg3s3bvXZAO4MFCSMNXBvX37NlQqFc6ePYtvv/0Wbdq0wdatW+Ht7Q0A2L9/P549ewaFQgF/f3/jNuL/q127Nn744Qf4+fnh6tWrKF++PLp27QqlUgmVSmVUC1lOTo5GkFRY586dERgYaHDaanZ2dvDy8tI5Qo6FhQXOnTsnNn0XbibXx5QpUwD8XxDs7OyMf//9V/xcvTwzMxOHDx9GWlqaQfmo0yg8hHnz5s21ZjOXQ+GKV7169bB69Wrs378fnTt3xr59+165eT9q1qyJzZs3Y9SoUUhNTcXZs2fFz+SqKNvZ2cHPzw89evTAyJEjMWrUKNlmRZcSBAHp6eka3VOdnZ2Rnp6OgoIC/PTTTygoKDDoXPvuu++Qn5+PJ0+eYNOmTTq7ucTHxyMrKwuCICArK8uobVEzZTc8hUKB2rVr4/Dhw/joo49w5MgRDBs2DMDz0RcnTJiA0NBQhIeHY/DgwUbllZOTozGVgbobjNr3338v3tAytAvRsGHD0KBBAzx8+BBxcXGwtLTE3bt3ZWl9K8qECRPQpUsXTJgwQewhoaZUKvH222/LnqenpyfOnTuHHj164Pfffzd+LhPJeVW4wi3nTbTC0z8kJibiyJEj2LBhA7y9vbF69Wp4eXnJkk9Z8emnn4o3d2vWrIkrV64gLi4OrVu3FtepWLEiUlJSxL8NlZSUhH///RcPHz7E6dOnYW1tjalTpyIrK0u8eSYH9bn2ySefwMnJCR9++CEuXboEOzs7jB07FiNGjMDUqVOxcuVKjBkzxuB8/vrrL60gSW3EiBFYsWKFwWkXpzSeT69RowYePXoECwsLvaeLkfrzzz+xZcsWbNy4Ee3atcO+fftQq1YtjB49usipUIzBQOkF5Dy4FhYWaN++Pdq3b49Tp05h4MCB2LBhA9555x3ZJv2Utnq1bNkS/fv3x6JFi7Bz507cvn1bXMeQIS3VypUrhytXrujsmvbTTz+Z9NkoABg6dChmzJghDtvbuXNno9JTD13bsGFDjWZb9RxDw4YNQ1RUFD755BOD86hYsSL++ecfuLq6Anjeh19dOTL23CpMV8tnz549UaVKFXz44Yf49ddfUbVqVaPyyMzM1AjyVCqVbJVuKQsLCzRv3hwHDx5Ejx49cO/ePbH8cleU7e3tsWXLFnTv3h2bNm3C0KFD5dgEAP93XBQKBdzd3fHw4UP8+uuv8PHxwcKFC9G5c2dkZ2fjp59+giAICAgI0DuP3377DQUFBejduzcuXLgg3ggo7LPPPsODBw+gUCh0DiKh7/YAQHZ2Nu7evavXtAH6UiqV+P777/H222+jWbNm8PLywtixYxEfH4+hQ4figw8+wPjx443KQ/od/eeff+Do6Ch+fuHCBeTl5UGhUCAoKMigPDIzM7F161YAz/fbqlWr8O677yIoKAjjxo0zqvxF2bhxI5RKpc7g39bWFocOHTIoTfXxfvbsmcbzFAqFAkOGDMGnn34KJycnzJ8/X9bvEqB541KpVCIjI0PW9AHAzc0NI0eOxMiRI7F//34MGDAAwcHBYjdgev5saHh4OBQKBUJCQvDee++hUqVKOHfuHNavXw+FQoFevXph4cKFUCgURQYGJWFjY4Pq1aujevXqaNGiBSZNmoRLly5h2rRp2LlzJzZt2mR0q7JU7969ce7cOcycORPLly/HgAEDkJKSgoiICPTr18+oAM3JyQk3btzQ2fIeExMjS7c7UzzPLZWamooDBw4gLS0NBw8eRFZWllbDQq9evRAaGgpLS0uj59VSKpV466238NZbb+Hnn39G9+7dcfr0adjZ2Ym9aWQl+zh6rxBXV1dh//79Qp06dYQDBw4IP/74o9CyZUtBEP5vSMPFixcLY8aMEfz9/YXFixfLmv+lS5cEd3d3ISMjQ7Y0AwMDtZYlJyfLPuR4dHS0UKNGDeHTTz8Vdu7cKRw/flzYvn27MGrUKKFhw4ZCQkKC0Xmohxn28PAQQkJCdA5rKbdHjx4J6enpsg3XXdimTZuEVq1aCfv37xd++OEHoWHDhkJMTIzs+RR3nq5du1YYN26c0Xn07dtXHKZb/erXr5/GOnINDV04nTt37ghNmjTRGAY8KChIaNq0qTB9+nRZ8lPn4+7uLuTm5uoshyGuXbsmCIIgVKxYUSgoKBCuXLkitGjRQqhWrZowaNAgITMz06j0X0TuYVP79u0r/t2/f3+hadOmQvPmzYWgoCBxuRzngPS7eOHCBaFt27ZGp6vL5s2bhZYtWwp79+4Vdu/eLbz11lvCtm3bZM1D1z559OiR0K5dO3HKilfBjBkzhClTpghTpkwRpk2bJhw8eFAQhP8bHlwQBGHFihVCmzZthNmzZxudX4MGDYSNGzcKUVFRQlRUlPDGG2+If69evdrkQ9ELwvPf0latWgmXL182eV6vEvXw4IIgCJmZmcKzZ8/E5Xv37hUyMzOFjz/+WBg4cKDJrnNz5swR3n77bVnSkl5z0tPTBTc3N72HzX6RnTt3CtWqVROWLl0qXLhwQfjrr7+E3377TQgLCxNq1aolXLlyxeg8Jk6cqLVM+l3p1q2bUXnMmzdPGDhwoDBgwABhwIABwscffyxs2rRJI6+srCyhX79+gp+fn9FT7UjruV9//bWwYMECo9Iszms94eznn3+OGzduiBG3hYUF3n//fQwZMkSc7DE7OxtDhgyBpaUlNm3aJNskWWqF716+apKTk7F161ZcunQJqampcHZ2xjvvvIP//Oc/svSF37dvH3r16oWYmBicP38ewPMWH30mQjPU4sWLZR0KXk1910upVGLixIlo166d7Hm8SOHRo0xJjglndaWzevVq3LhxA19++aXRaRdn1apV+Oijj1ClShUA8k3c6+Ligvv375u9+8PSpUt1tjCZkhz77NixY+jUqZPGsuHDh2Ps2LEa3Xvksm/fPmzYsAEKhQIjR45Ejx49ZE2/qO9Fbm4uunbtisGDB+O///2vrHmaU+EJZ+W0adMmXLx4UecdcgsLC3Tq1AkffPCBrHnqUlBQYPQzFmXN1atXYWVlpdUyop5w1tgWhJL6448/jB5ECgDOnz+vdW3Zvn07mjdvLkv6hV2/fh3r1q0T61GVK1dGu3btMGbMGJMM5ACYZhL6oshVDyhVJgvBXnGmaFEg+c2dO7e0i0BFMHZSObW9e/dqvC8oKBC8vLyEJ0+eyJJ+SZUrV86k6ZfFc9mU++xV3V/Lli0r8rMHDx4ILVu21GjJfNXs379fSEpKKu1i0EvA19dXuHHjRmkXgyQsLCzMlpdc9YDS9Fq3KBXHVC0KJC9z3hmh5xITE4schrtChQpwdnbGvXv34OTkBHt7e+zatQt9+/aVtQzC/382zZxMcWcsNzdXbKUui+eyKe8mlsX9VRaFhYUhNDS0tItBZvLTTz+hZ8+eUCgUOr+jWVlZ4tDUVDqKu3aePn1aa4La191rHSiNHj0a+fn5Oj/TVRGztraWbdAFerHBgwdjy5YtAID58+dj5syZeP/99zVmkS8TzbqvmFatWukMlG7fvo1hw4ahdu3a+Prrr6FSqZCQkIDmzZsbPK/BtWvXigzK6tSpA0EQEB8fL35fq1evrjWqV0kMHjwY169fh42NDcqXL4+qVavCzc0NXl5e+M9//gPA+Ir5hAkTcO3aNfHh5smTJ6NFixaIjY0F8Oqdy+bYZyEhIZg3bx62bt2KQYMGAXi+H7/++mtxf82aNQs3b94Uz4H169drDMDwMvvoo49MPr9RacjMzBQfqmdA+/pIS0tD8+bNER8fD0D3sS98zSPT69u3r1Y99+TJk3j33XfF9+XLlxcHmDHk+1pcXVqXV60u/VqPejdgwACNSpggCBgzZgwiIyN1ri/3DOZUvMuXL4t///DDD5g5cyb+97//aaxTGsNcvu50jSrzxx9/wM/PD998840498OoUaMAGDfqTmBgoM5ASaFQ4NNPP8XixYuhUqmgVCohCAJu3ryJhw8f6p1PTEwMYmJioFKp4Ovri7CwMPzvf/+TdRb5Q4cOYd++fcjNzcWIESMwefJkjSGOVSoVkpOTxf1laWkp2xw0pmCOfbZ3717MmzcPy5YtEwMl9cSi6u9+VFQU1q1bJy4rX768bPmb2u+//17s52fOnME777xjptLob9iwYWJw3717d8yePRsA0Lp1a1y/fh2AeUbdopfD2rVr4efnV+w6xsyjRPqbMmWK1m+odPqWwlOwGPJ9Let16dc6UNI1NK69vT26du1aCqUhKf7Avjq++OILk6R75MiRYj+fOXMmrl27Jr43dI4oW1tbcYh2S0tLvP3227LPLVOuXDk0adIEgOYEjGp3795F69atxfPe2toat2/flrUMcjLHPlMr7lrg4ODwylyzpXd3ExMTtR50Dw0NRcuWLQEA48ePl32WeTmdOnUKR44cQW5uLoYMGSIGSoUrw7yZ9XqIj4/HmjVrXjg8M88H82rbti2A5xMpl2SOIUOOT1mvS7/WgVKtWrWQnZ2tsSw1NVVrjhlBEGBvby82J5N58IL6csrOzsbYsWNx7tw5DBo0CKGhobC2ti6VO9/Sc8TQc6bw/7O0tDTJyIAvKlvdunVfqa535thnZY307q6uibkLz0H3st8ssrOzQ/369QEwOHqd3bp1CwMGDMCqVavg5ORU2sUhHU6dOiX+vWXLFvz555/o1auXLBMol/W69GsdKEm7cRX29OlTJCQkGHyHmqismjlzJpo0aYIlS5YgICAAmzdvho2NDZYvXw5A/krSzp07YW1tjc6dO8POzk7WtNUKV0iVSiXy8vJw5swZREdHa30uRx66vGqVS3Pss7JGfXe3pF72c6Jw+V72spJpTJ06FV9++SVWrVqFTp06ITo6Gj169IBCoXhlpz4pSz777DONHh/jxo1DYmIiunXrhhEjRmD58uXo2LGjUXmU9br0a3/779y5czqX37t3DyEhIWYuDRXGitbL6dixY5g6dSqqVq2KhQsXYvv27cjOzkZ0dDTOnDkjex/0//73v1i3bh3c3d1N1sWvcCXPysoKeXl5sLe3R+XKlVG5cmU4OzubJN9XGfeZ4c6cOYPBgwdj0KBBOH36dGkXx2CFr9GF/7516xbKlStnshsb9PJYvHgxduzYgXnz5uHs2bNo37490tPTkZaWVtpFIwCHDx8W/05NTcXu3buxZ88ejB8/Hlu2bMHSpUtlyacs16Vf6xYl4PloHbr6gNerVw9///13KZSI1HiH8uWkUCjEY1OzZk2kpqbC3t4eS5YsgUKhEJ9bUa9j7HF0cXHBvn378ODBAwwePBiXLl0SR0M0BSsrK+Tn58PHxwc+Pj7icmNH6XnRfoiPj9docbC0tMSZM2eMytNcTLXPyqKDBw9i6tSpmDlzpvjQ85IlS8wyWarcijqn69evL3YjlU5CSmWLQqHABx98gHr16qFbt264cOGCOEk3vTwEQcCDBw9Qo0YNWFk9r/rXqVPHoMGPdCnLdenXPlAqqtXCxsaGLRqlLCsrC3fu3IEgCHj27Bnu3LmD3Nzc0i7Wa69cuXJITEyEm5sboqOj0bx5c1hbW2PlypUAgGnTpondLtzc3MSLsqHUlTEXFxccOXIEH3zwAebMmYM5c+ZAoVDg3LlzsLGxgUqlQnp6ukF5FP6uKxQKnYMtGKtwHjk5Obh9+7ZGv+7q1atj7dq14ntLS0vZyyAnc+yz1NRUHDx4EGlpaTh48CCysrJe+eegwsLCsGPHDjRq1AgA4OXlhaFDh76SgVJh7Ib3emvUqBECAwMREhKC1atXa3x24sQJ5OfnQxAEg6/RZBj1dVqhUODNN99EcnIyrl69iubNm2PTpk3w8fHBnTt30LFjRygUCoN7hJTluvRrHygVd0HXZ1x4kl+LFi3Qu3dvAM9HAOvduzc8PDxKuVQUEhKCzp07o3379vjll1+wf/9+LF68WPw8NDQUdnZ22L9/P1JTU1GhQgXZ8rawsMCWLVvg7e0NPz8/DBo0CDNmzBAvxLpG3ykJb29v8e/33nvPJF2GWrRoIf7t5eWFQYMGaeSrVCrFUfFeBebYZ6NHj8aWLVvQqlUrbN68GRYWFggICNBY51V7eDwxMVEMkgCgYcOGePz4sc51X/YKRuHyZWdn4+7du3j27FkplohK06hRo1C3bl0sW7ZMnEdLEASsXbsW+fn5UCgUGDZsWOkW8jUzdepU8W8rKyusW7cO3bt3h5OTEypWrIg9e/agcuXKOHfuHBQKhcHX07Jcl2agpFDg5MmTWndvjbk7TfLYs2fPC9d52SsSZVH37t3RoEEDXL9+HQsWLNB6FsXBwUH8u1KlSkbnJz3GFStWxMyZMxESEoKdO3di2rRpRuexadMm8e/58+eXqBz62rx5s/j3999/L3v65maOfVZc33Z12r/++qtReZhb+fLl8ejRI3FEqOTkZCiVSvFzFxcXZGVlQRAE1K5du7SKWSKFu9W1aNECvXr1goWFhdaQ5/R6sLW1xfr168X6lHoSaF3XOzKPESNGAPi/62XXrl2RkJCAlJQUjd/uatWqGZVPWa5Lv/aB0rBhw/DFF19o/aArFAqMHj26lEpFJTV8+PDSLsJrqW7duqhbt6743pTfFV0DOHzyySca8yeZg6nPNfUPWlliyn32qu4vf39/DBw4EIsWLQIABAcHY9KkSeLnDx48KK2i6W3nzp3i3z/++GMploReFoXnzuHv88ujcLduCwsL2Qfb0VWXVrcyvep1aYXwqt3GJCIieoVFRkZiw4YN4g25stwdyd7eHpmZmaVdDCIys7CwMISGhpZ2MYzGQImIiIhkkZCQgDfeeEN836hRo1dqImUikkfjxo1x48YNreX//vsvKleuXAolMsyrPXwQERERlYpu3bppLZOO3scR8IjKvpCQEAwdOhRDhw7F7NmzART9jKqhgy6Vltf+GSUiIiJzuHbtGvLy8kq8vrW19Us9o/2ff/6JCxcuiBUiQRCQlZVVyqUiInP79ttvsXr1aqhUKowfPx5z584tcl1TTCVhSgyUiIiIzCAwMFCvQMnGxgaHDx82YYmM8/DhQ3Hi3MLLiOj14ujoKI52OWPGjGLXfdVamRkoERERmcGRI0dKuwiycnd319qml7kFjIhMo6jgJzk5WRwmvkqVKmYulTwYKBEREZHedFWOXrW7xURkvKKeR2rdujUEQYClpSWuXbtmkonJTY2DORAREb0EDh8+jI4dO5Z2MYiIZHHnzh3cvXsXf/311ysZJAFsUSIiIip1ERERWLVqFXbt2lXaRSmxhIQE+Pn5ie8FQUBiYmIploiISkNZbklmoERERGQm165dg1KpRPXq1VG+fHlER0djzpw5cHFxwZkzZ+Dk5FTaRSyxSpUqYcyYMeJ7QRBw+fLlUiwREZWG1NRUfP/991CpVHjy5InW53/88QdUKtUrOTImAyUiIiIz8ff3R3JyMlJSUpCdnY2srCwEBwdj/vz5pV00vTk6Omp1FSxXrlwplYaISstnn32G6OhoAMC4ceM0PktLS8PIkSOhUqmgUCheuXmUFEJRT2ARERGRySQmJuLIkSPYsGEDcnNzsXr1anh5eZV2sUqsoKAAlpaWGstiYmLw9ttvi+8bNWqEmzdvmrtoRFTKysp3n4M5EBERlQI3NzeMHDkSZ8+exezZszFgwAB8++23pV2sEpMGSQA0giQAGD58uLmKQ0QvkbLy3WeLEhER0Uvg8ePH6NGjB1avXg1PT8/SLg4R0WuPgRIREVEpGT16NCIjI8X3urqzERFR6WDXOyIiolJy9uxZjfcMkoiIXh4c9Y6IiMgMRo8ejfz8fI1lSUlJGDFihM71ra2tsWbNGnMUjYiIdGCgREREZAY9e/ZEXl6exrIePXoUub61tbWpi0RERMXgM0pEREREREQSbFEiIiIys0OHDuHcuXNITk5G9erV8d5772kNrU1ERKWLLUpERERmcvfuXfTs2RMuLi7o0qULqlSpgnv37mHfvn2ws7PDvn374OTkVNrFJCIiMFAiIiIym65du6JPnz4YO3as1mczZ87Eo0ePsHbt2lIoGRERSTFQIiIiMpMaNWogKSlJ52fZ2dlo0aIFbt26ZeZSERGRLpxHiYiIyEzs7Oxw//59nZ/dunULVapUMXOJiIioKAyUiIiIzGTq1Klo3749du3ahcePH6OgoADJycnYvHkz+vbti7lz55Z2EYmI6P9j1zsiIiIz+vnnn7F69WpcunQJT548QeXKldGuXTt8+umnaNmyZWkXj4iI/j8GSkRERERERBLsekdERERERCTBQImIiIiIiEiCgRIREREREZEEAyUiIiIiIiIJBkpEREREREQSDJSIiIiIiIgkGCgRERERERFJ/D8sixatpX0FkAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1000x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 피처 중요도 구하기\n",
    "fi = pd.Series(model.feature_importances_)\n",
    "fi.index = feature_names\n",
    "fi.nlargest(30).plot(kind=\"bar\", figsize=(10, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 예측"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 3, 1, 2, 3, 3, 5, 3, 4, 4])"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 실전 시험과 유사합니다. 정답 없이 문제만 넣고 풀어서 예측합니다.\n",
    "# predict로 예측합니다. \n",
    "y_predict = model.predict(X_test)\n",
    "y_predict[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 정답값 로드"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>topic_idx</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>45654</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>45655</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>45656</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>45657</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>45658</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   index  topic_idx\n",
       "0  45654          0\n",
       "1  45655          0\n",
       "2  45656          0\n",
       "3  45657          0\n",
       "4  45658          0"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# sample_submission 파일을 불러옵니다.\n",
    "# Dacon 에 제출할 답안지 양식입니다.\n",
    "submit = pd.read_csv(\"data/klue/sample_submission.csv\")\n",
    "submit.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>topic_idx</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>45654</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>45655</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>45656</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>45657</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>45658</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9126</th>\n",
       "      <td>54780</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9127</th>\n",
       "      <td>54781</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9128</th>\n",
       "      <td>54782</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9129</th>\n",
       "      <td>54783</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9130</th>\n",
       "      <td>54784</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>9131 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      index  topic_idx\n",
       "0     45654          2\n",
       "1     45655          3\n",
       "2     45656          1\n",
       "3     45657          2\n",
       "4     45658          3\n",
       "...     ...        ...\n",
       "9126  54780          3\n",
       "9127  54781          3\n",
       "9128  54782          4\n",
       "9129  54783          2\n",
       "9130  54784          2\n",
       "\n",
       "[9131 rows x 2 columns]"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 정답값 측정을 위해 y_test 변수에 할당\n",
    "submit[\"topic_idx\"] = y_predict\n",
    "submit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'data/klue/submit_0.70498.csv'"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "file_name = f\"data/klue/submit_{valid_accuracy:.5f}.csv\"\n",
    "file_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 파일로 저장합니다.\n",
    "submit.to_csv(file_name, index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>topic_idx</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>45654</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>45655</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>45656</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>45657</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>45658</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9126</th>\n",
       "      <td>54780</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9127</th>\n",
       "      <td>54781</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9128</th>\n",
       "      <td>54782</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9129</th>\n",
       "      <td>54783</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9130</th>\n",
       "      <td>54784</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>9131 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      index  topic_idx\n",
       "0     45654          2\n",
       "1     45655          3\n",
       "2     45656          1\n",
       "3     45657          2\n",
       "4     45658          3\n",
       "...     ...        ...\n",
       "9126  54780          3\n",
       "9127  54781          3\n",
       "9128  54782          4\n",
       "9129  54783          2\n",
       "9130  54784          2\n",
       "\n",
       "[9131 rows x 2 columns]"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 잘 저장이 되었는지 확인합니다.\n",
    "pd.read_csv(file_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* dacon에 제출해서 리더보드 확인하기 : https://dacon.io/competitions/official/235747/mysubmission\n",
    "\n",
    "\n",
    "## 여러 방법을 통해 예측비율을 높여보세요. \n",
    "* 아래 항목 외의 기법을 사용해도 됩니다.\n",
    " * 전처리 하기\n",
    " * 불용어 처리\n",
    " * BOW, TF-IDF의 파라메터 변경\n",
    " * 분류기의 파라메터 변경\n",
    " * 분류기 변경"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "320px"
   },
   "toc_section_display": false,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}