Vander_Plus_5_ML_04.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bba35913",
   "metadata": {},
   "source": [
    "# Проектирование признаков\n",
    "\n",
    "Далеко не всегда данные находятся в правильном формате ``[`` ``n_samples``, ``n_features`` ``]``.\n",
    "\n",
    "Потому одним из важнейших этапов использования ML на практике становится ``проектирование признаков (feature engineering)``, т.е. преобразование всей касающейса задачи информации в числа, пригодные для построения матрицы признаков.\n",
    "\n",
    "Рассмотрим несколько часто встречающихся примеров задач проектирования признаков:\n",
    "\n",
    "- признаки для представления ``категориальных данных (categorical data)``\n",
    "- признаки для представления ``текста``\n",
    "- признаки для представления ``изображений``\n",
    "\n",
    "И использование ``производных признаков (derived features)`` для повышения сложности модели и заполнения отсутствующих данных. Этот процесс часто называют ``векторизацией``, т.к. он включает преобразование данных в произвольной форме в аккуратные вектроы."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "267cbd97",
   "metadata": {},
   "source": [
    "## Категориальные признаки\n",
    "\n",
    "``категориальные данные (categorical data)`` - один из распространенных типов нечисловых данных.\n",
    "\n",
    "Например мы анализируем какие-то данные с ценами на жилье, и помимо числовых признаков, таких как цена, площадь, в них так же существуют нечисловые - микрорайон (neighborhood)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9995e93a",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = [\n",
    "    {'price': 850000, 'rooms': 4, 'neighborhood': 'Queen Anne'},\n",
    "    {'price': 700000, 'rooms': 3, 'neighborhood': 'Fremont'},\n",
    "    {'price': 650000, 'rooms': 3, 'neighborhood': 'Wallingford'},\n",
    "    {'price': 600000, 'rooms': 2, 'neighborhood': 'Fremont'}\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b1c5e14f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Queen Anne': 1, 'Fremont': 2, 'Wallingford': 3}"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Может показаться, что кодировать эти данные следует как словарь\n",
    "{'Queen Anne': 1, 'Fremont': 2, 'Wallingford': 3}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "778ae4e2",
   "metadata": {},
   "source": [
    "Но модели SKL исходят из допущения, что числовые признаки отражают алгебраические величины.\n",
    "Следовательно подобное отображение будет подразумевать:\n",
    "\n",
    "Queen Anne < Fremont < Wallingford\n",
    "\n",
    "или даже\n",
    "\n",
    "Wallingford - Queen Anne = Fremont\n",
    "\n",
    "что не имеет смысла."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "257c684b",
   "metadata": {},
   "source": [
    "Испытанным методом для такого случая будет ``прямое кодирование (one-hot)`` означающее создание дополнительных столбцов индикаторов наличия/отсутствия категории с помощью значений 1 или 0.\n",
    "\n",
    "При наличии данных в виде списка словарей для этой цели можно воспользоваться утилитой ``DictVectorizer`` библиотеки SKL."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "133f9732",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction import DictVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "86484b08",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[     0,      1,      0, 850000,      4],\n",
       "       [     1,      0,      0, 700000,      3],\n",
       "       [     0,      0,      1, 650000,      3],\n",
       "       [     1,      0,      0, 600000,      2]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vec = DictVectorizer(sparse=False, dtype=int)\n",
    "vec.fit_transform(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "927d3b3b",
   "metadata": {},
   "source": [
    "Столбец *neighborhood* превратился в 3 отдельных столбца.\n",
    "\n",
    "Чтобы узнать что означает каждый столбец можно посметреть названия признаков:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f185f4ec",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['neighborhood=Fremont',\n",
       " 'neighborhood=Queen Anne',\n",
       " 'neighborhood=Wallingford',\n",
       " 'price',\n",
       " 'rooms']"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vec.get_feature_names()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74728a63",
   "metadata": {},
   "source": [
    "У такого подхода есть очивидный недостатое: если кол-во категорий велико - то размер набора данных может значительно вырости.\n",
    "\n",
    "Однако, поскольку кодирование данных состоит в основном из 0, эффективным решением будет ``разрежанный формат`` вывода:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "049f2e92",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<4x5 sparse matrix of type '<class 'numpy.int32'>'\n",
       "\twith 12 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# sparse - разрежанный\n",
    "vec = DictVectorizer(sparse=True, dtype=int)\n",
    "vec.fit_transform(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "9fad815c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 1)\t1\n",
      "  (0, 3)\t850000\n",
      "  (0, 4)\t4\n",
      "  (1, 0)\t1\n",
      "  (1, 3)\t700000\n",
      "  (1, 4)\t3\n",
      "  (2, 2)\t1\n",
      "  (2, 3)\t650000\n",
      "  (2, 4)\t3\n",
      "  (3, 0)\t1\n",
      "  (3, 3)\t600000\n",
      "  (3, 4)\t2\n"
     ]
    }
   ],
   "source": [
    "print(vec.fit_transform(data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2fe1cf0e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "1c2e6af6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>price</th>\n",
       "      <th>rooms</th>\n",
       "      <th>neighborhood</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>850000</td>\n",
       "      <td>4</td>\n",
       "      <td>Queen Anne</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>700000</td>\n",
       "      <td>3</td>\n",
       "      <td>Fremont</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>650000</td>\n",
       "      <td>3</td>\n",
       "      <td>Wallingford</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>600000</td>\n",
       "      <td>2</td>\n",
       "      <td>Fremont</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    price  rooms neighborhood\n",
       "0  850000      4   Queen Anne\n",
       "1  700000      3      Fremont\n",
       "2  650000      3  Wallingford\n",
       "3  600000      2      Fremont"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a6e8c4e",
   "metadata": {},
   "source": [
    "Многие оцениватели SKL допускают передачу им подобных разрежанных данных при обучении и оценке моделей.\n",
    "\n",
    "Для поддержки подобного кодирования существуют 2 утилиты:\n",
    "- ``sklearn.preprocessing.OneHotEncoder``\n",
    "- ``sklearn.feature_extraction.FeatureHasher``"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07f46b08",
   "metadata": {},
   "source": [
    "## Текстовые признаки\n",
    "\n",
    "Часто требуется преобразовать текст в набор репрезентативных числовых значений.\n",
    "\n",
    "Один из простейщих методов - по *количеству слов*: для каждого фрагмента текста подсчитывается количество вхождений в него каждого из слов, после чего результаты помещаются в таблицу."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "46b16b8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "sample = ['problem of evil',\n",
    "          'evil queen',\n",
    "          'horizon problem']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "92617736",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<3x5 sparse matrix of type '<class 'numpy.int64'>'\n",
       "\twith 7 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "\n",
    "vec = CountVectorizer()\n",
    "X = vec.fit_transform(sample)\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "c33948a7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>evil</th>\n",
       "      <th>horizon</th>\n",
       "      <th>of</th>\n",
       "      <th>problem</th>\n",
       "      <th>queen</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   evil  horizon  of  problem  queen\n",
       "0     1        0   1        1      0\n",
       "1     1        0   0        0      1\n",
       "2     0        1   0        1      0"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(X.toarray(), columns=vec.get_feature_names())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1dd693cd",
   "metadata": {},
   "source": [
    "У такого подхода есть проболемма: подсчет кол-ва слов ведет к большим значениям для признков соответствующих часто встечающимся словам. Для некоторых алгоритмов классификации это можето оказаться субпродуктивным.\n",
    "\n",
    "Один из подходов к решению этой проблеммы: \n",
    "- ``частота термина, обратная частоте документа (term frequency-inverse document frequency)`` или ``TF–IDF``\n",
    "\n",
    "При таком подходе слова получают вес с учетом частоты их появления во всех документах."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "ece4e805",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>evil</th>\n",
       "      <th>horizon</th>\n",
       "      <th>of</th>\n",
       "      <th>problem</th>\n",
       "      <th>queen</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.517856</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.680919</td>\n",
       "      <td>0.517856</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.605349</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.795961</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.795961</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.605349</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       evil   horizon        of   problem     queen\n",
       "0  0.517856  0.000000  0.680919  0.517856  0.000000\n",
       "1  0.605349  0.000000  0.000000  0.000000  0.795961\n",
       "2  0.000000  0.795961  0.000000  0.605349  0.000000"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "vec = TfidfVectorizer()\n",
    "X = vec.fit_transform(sample)\n",
    "pd.DataFrame(X.toarray(), columns=vec.get_feature_names())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a79ab4c",
   "metadata": {},
   "source": [
    "## Признаки для изображений\n",
    "\n",
    "Простейший подход - использовать значения интенсивности пикселей. Но он может оказаться неоптимальным для прикладных задач.\n",
    "\n",
    "Существует библиотека, специализирующийся на анализе изображений ``Scikit-Image``  https://scikit-image.org"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c2f583f",
   "metadata": {},
   "source": [
    "### Производные признаки\n",
    "\n",
    "Ещё один **удобный тип признаков - выведенные математически из каких-либо входных признаков** (``тут мы типа создаем аналитические/расчетные столбцы (доп признаки)``.\n",
    "\n",
    "**Ранее мы создавали полиномиальные признаки из входных данных.**\n",
    "\n",
    "Можно преобразовать, например, линейную регрессию в полиномиальную **не меняя модель, а меняя (вычисляя) признаки**\n",
    "\n",
    "Из предыдущей главы:\n",
    "\n",
    "------------------------------------------------------------------------------------------------------------------------------\n",
    "Рассмотрим пример перекрестной проверки для расчета кривой проверки для класса моделей. Используем ``модель полиномиальной регрессии (polynomial regression model)``: это обобщенная линейная модель с параметризованной степенью многочлена.\n",
    "\n",
    "1я степень $a$ и $b$\n",
    "$$\n",
    "y = ax + b\n",
    "$$\n",
    "\n",
    "3я степень $a, b, c, d$\n",
    "$$\n",
    "y = ax^3 + bx^2 + cx + d\n",
    "$$\n",
    "\n",
    "Это можно обобщить на любое кол-во полиномиальных признаков.\n",
    "\n",
    "В библиотеке SKL реализовать это можно с помощью **простой линейной регрессии в сочетании с полиномиальным препроцессором.**\n",
    "\n",
    "Воспользуемся ``конвейером (pipeline)`` для соединения этих операций в единую цепочку.\n",
    "\n",
    "Как это выглядело:\n",
    "\n",
    "------------------------------------------------------------------------------------------------------------------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "a831f344",
   "metadata": {},
   "outputs": [],
   "source": [
    "# from sklearn.preprocessing import PolynomialFeatures\n",
    "# from sklearn.linear_model import LinearRegression\n",
    "# from sklearn.pipeline import make_pipeline\n",
    "\n",
    "# def PolynomialRegression(degree=2, **kwargs):\n",
    "#     return make_pipeline(PolynomialFeatures(degree),\n",
    "#                          LinearRegression(**kwargs))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d43d432",
   "metadata": {},
   "source": [
    "Такой метод известен как ``регрессия по комбинации базисных ф-ий (basis function regression)``.\n",
    "\n",
    "Например, очеведно, что следующие данные нельзя адекватно описать с помощью линии:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "87ca29f3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x1570b045160>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAOUElEQVR4nO3dUWid93nH8d9vsrqeJimC+dBGcjbRG8GW0MgIk2IIWdpVyRoyEXqRQjvWG2+jjJYNlakXLbvKhaCk28WGSbq1NGnXpbIYIYka6ELpRT2OLa/K6mh0xaWWsvlkQ02yHlZFeXahI1fWjnzek5z3vI+s7weEj973td6HP0dfy+95j+2IEAAgr1+pegAAwPURagBIjlADQHKEGgCSI9QAkNyRMr7o0aNHY3x8vIwvDQA3pHPnzr0SEfVO+0oJ9fj4uBqNRhlfGgBuSLZ/st8+Ln0AQHKEGgCSI9QAkByhBoDkCDUAJNf1rg/bE5L+ftem90n6fEQ8WtZQAHCQLC6vaX5pVesbLY2O1DQ7PaGZybG+ff2uoY6IVUl3SpLtIUlrks70bQIAOMAWl9c0t7Ci1uaWJGlto6W5hRVJ6luse7308UFJ/x4R+97vBwCHyfzS6tVI72htbml+abVv5+g11A9L+nqnHbZP2W7YbjSbzbc/GQAcAOsbrZ62vxWFQ237HZIelPQPnfZHxOmImIqIqXq947sgAeCGMzpS62n7W9HLT9T3SzofEf/Zt7MDwAE3Oz2h2vDQNdtqw0OanZ7o2zl6+bc+PqZ9LnsAwGG184JhpXd9SJLtd0n6HUl/2LczA8ANYmZyrK9h3qtQqCPi55J+rbQpAAD74p2JAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHKEGgCSI9QAkByhBoDkCDUAJEeoASA5Qg0AyRFqAEiOUANAcoQaAJIj1ACQHKEGgOQINQAkR6gBIDlCDQDJEWoASI5QA0ByhBoAkisUatsjtp+y/ZLti7Y/UPZgAIBtRwoe9yVJz0XER22/Q9K7SpwJALBL11DbfrekuyX9gSRFxC8k/aLcsQAAO4pc+nifpKakv7W9bPsx2zftPcj2KdsN241ms9n3QQHgsCoS6iOSjkv664iYlPQ/kv5870ERcToipiJiql6v93lMADi8ioT6sqTLEXG2/flT2g43AGAAuoY6Iv5D0k9tT7Q3fVDSD0udCgBwVdG7Pv5E0hPtOz5+LOmT5Y0EANitUKgj4oKkqXJHAQB0wjsTASA5Qg0AyRFqAEiOUANAcoQaAJIj1ACQHKEGgOQINQAkR6gBIDlCDQDJEWoASI5QA0ByhBoAkiPUAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHKEGgCSI9QAkByhBoDkCDUAJHekyEG2L0l6TdKWpDciYqrMoQAAv1Qo1G2/HRGvlDYJAKAjLn0AQHJFQx2Svm37nO1TnQ6wfcp2w3aj2Wz2b0IAOOSKhvpkRByXdL+kT9m+e+8BEXE6IqYiYqper/d1SAA4zAqFOiLW279ekXRG0okyhwIA/FLXUNu+yfYtO48lfVjSi2UPBgDYVuSuj/dIOmN75/gnI+K5UqcCAFzVNdQR8WNJ7x/ALACADrg9DwCSI9QAkByhBoDkCDUAJEeoASA5Qg0AyRFqAEiOUANAcoQaAJIj1ACQHKEGgOQINQAkR6gBIDlCDQDJEWoASI5QA0ByhBoAkiPUAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHJHih5oe0hSQ9JaRDzQ70EWl9c0v7Sq9Y2WRkdqmp2e0MzkWL9PAwAHTuFQS/q0pIuS3t3vIRaX1zS3sKLW5pYkaW2jpbmFFUki1gAOvUKXPmwfk/QRSY+VMcT80urVSO9obW5pfmm1jNMBwIFS9Br1o5I+K+nN/Q6wfcp2w3aj2Wz2NMT6Rqun7QBwmHQNte0HJF2JiHPXOy4iTkfEVERM1ev1noYYHan1tB0ADpMiP1GflPSg7UuSviHpXttf6+cQs9MTqg0PXbOtNjyk2emJfp4GAA6krqGOiLmIOBYR45IelvSdiPh4P4eYmRzTIw/dobGRmixpbKSmRx66gxcSAUC93fVRqpnJMcIMAB30FOqIeEHSC6VMAgDoiHcmAkByhBoAkiPUAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHKEGgCSI9QAkByhBoDkCDUAJEeoASA5Qg0AyRFqAEiOUANAcoQaAJIj1ACQHKEGgOQINQAkR6gBIDlCDQDJEWoASO5ItwNsv1PSdyX9avv4pyLiC2UPBqA6i8trml9a1fpGS6MjNc1OT2hmcqzqsQ6trqGW9L+S7o2I120PS/qe7Wcj4vslzwagAovLa5pbWFFrc0uStLbR0tzCiiQR64p0vfQR215vfzrc/ohSpwJQmfml1auR3tHa3NL80mpFE6HQNWrbQ7YvSLoi6fmIONvhmFO2G7YbzWazz2MCGJT1jVZP21G+QqGOiK2IuFPSMUknbN/e4ZjTETEVEVP1er3PYwIYlNGRWk/bUb6e7vqIiA1JL0i6r4xhAFRvdnpCteGha7bVhoc0Oz1R0UToGmrbddsj7cc1SR+S9FLJcwGoyMzkmB556A6NjdRkSWMjNT3y0B28kFihInd93CrpK7aHtB32b0bE0+WOBaBKM5NjhDmRrqGOiB9ImhzALACADnhnIgAkR6gBIDlCDQDJEWoASI5QA0ByhBoAkiPUAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHKEGgCSI9QAkByhBoDkCDUAJEeoASA5Qg0AyRFqAEiOUANAcoQaAJIj1ACQHKEGgOSOdDvA9m2SvirpvZLelHQ6Ir5U9mC4vsXlNc0vrWp9o6XRkZpmpyc0MzlW9VgAStA11JLekPRnEXHe9i2Sztl+PiJ+WPJs2Mfi8prmFlbU2tySJK1ttDS3sCJJxBq4AXW99BERL0fE+fbj1yRdlEQNKjS/tHo10jtam1uaX1qtaCIAZerpGrXtcUmTks522HfKdsN2o9ls9mk8dLK+0eppO4CDrXCobd8s6VuSPhMRr+7dHxGnI2IqIqbq9Xo/Z8QeoyO1nrYDONgKhdr2sLYj/URELJQ7ErqZnZ5QbXjomm214SHNTk9UNBGAMhW568OSHpd0MSK+WP5I6GbnBUPu+gAOhyJ3fZyU9AlJK7YvtLd9LiKeKW0qdDUzOUaYgUOia6gj4nuSPIBZAAAd8M5EAEiOUANAcoQaAJIj1ACQHKEGgOQINQAkR6gBIDlCDQDJEWoASI5QA0ByhBoAkiPUAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHKEGgCSI9QAkByhBoDkCDUAJEeoASA5Qg0AyR3pdoDtL0t6QNKViLi9/JGA/ltcXtP80qrWN1oaHalpdnpCM5NjVY8FFFLkJ+q/k3RfyXMApVlcXtPcworWNloKSWsbLc0trGhxea3q0YBCuoY6Ir4r6b8HMAtQivmlVbU2t67Z1trc0vzSakUTAb3p2zVq26dsN2w3ms1mv74s8Latb7R62g5k07dQR8TpiJiKiKl6vd6vLwu8baMjtZ62A9lw1wdueLPTE6oND12zrTY8pNnpiYomAnrT9a4P4KDbubuDuz5wUBW5Pe/rku6RdNT2ZUlfiIjHyx4M6KeZyTHCjAOra6gj4mODGAQA0BnXqAEgOUINAMkRagBIjlADQHKOiP5/Ubsp6Sdv8bcflfRKH8fpF+bqDXP1hrl6cyPO9RsR0fHdgqWE+u2w3YiIqarn2Iu5esNcvWGu3hy2ubj0AQDJEWoASC5jqE9XPcA+mKs3zNUb5urNoZor3TVqAMC1Mv5EDQDYhVADQHKVhNr2l21fsf3iPvtt+y9t/8j2D2wfTzLXPbZ/ZvtC++PzA5rrNtv/ZPui7X+1/ekOxwx8zQrONfA1s/1O2/9s+1/ac/1Fh2OqWK8ic1XyHGufe8j2su2nO+yr5HuywFxVfU9esr3SPmejw/7+rldEDPxD0t2Sjkt6cZ/9vyvpWUmWdJeks0nmukfS0xWs162Sjrcf3yLp3yT9ZtVrVnCuga9Zew1ubj8elnRW0l0J1qvIXJU8x9rn/lNJT3Y6f1XfkwXmqup78pKko9fZ39f1quQn6uj+H+b+nqSvxrbvSxqxfWuCuSoRES9HxPn249ckXZS09x9XHviaFZxr4Npr8Hr70+H2x95XzatYryJzVcL2MUkfkfTYPodU8j1ZYK6s+rpeWa9Rj0n66a7PLytBANo+0P6r67O2f2vQJ7c9LmlS2z+N7Vbpml1nLqmCNWv/dfmCpCuSno+IFOtVYC6pmufYo5I+K+nNffZX9fx6VNefS6pmvULSt22fs32qw/6+rlfWULvDtgw/eZzX9vvx3y/pryQtDvLktm+W9C1Jn4mIV/fu7vBbBrJmXeaqZM0iYisi7pR0TNIJ27fvOaSS9Sow18DXy/YDkq5ExLnrHdZhW6nrVXCuqr4nT0bEcUn3S/qU7bv37O/remUN9WVJt+36/Jik9YpmuSoiXt35q2tEPCNp2PbRQZzb9rC2Y/hERCx0OKSSNes2V5Vr1j7nhqQXJN23Z1elz7H95qpovU5KetD2JUnfkHSv7a/tOaaK9eo6V1XPr4hYb/96RdIZSSf2HNLX9coa6n+U9PvtV07vkvSziHi56qFsv9e2249PaHv9/msA57WkxyVdjIgv7nPYwNesyFxVrJntuu2R9uOapA9JemnPYVWsV9e5qliviJiLiGMRMS7pYUnfiYiP7zls4OtVZK6Knl832b5l57GkD0vae6dYX9erkv+F3B3+w1xtv7CiiPgbSc9o+1XTH0n6uaRPJpnro5L+2PYbklqSHo72S7wlOynpE5JW2tc3Jelzkn5912xVrFmRuapYs1slfcX2kLa/cb8ZEU/b/qNdc1WxXkXmquo59v8kWK8ic1WxXu+RdKb958MRSU9GxHNlrhdvIQeA5LJe+gAAtBFqAEiOUANAcoQaAJIj1ACQHKEGgOQINQAk93+yP1SHKCjRUwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "x = np.array([1, 2, 3, 4, 5])\n",
    "y = np.array([4, 2, 1, 3, 7])\n",
    "plt.scatter(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e169ccd",
   "metadata": {},
   "source": [
    "Тем не менее мы можем подобрать разделяющую линию для этих данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "0359990c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1570ba62310>]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAW+UlEQVR4nO3de3BU93nG8edFCCNssGwkrY2MLLBBXtmdWFQmdjAErFWwY49DO5lp3CZt46SkndzcpLghkybTP/pHh5lM0k6nHSZJmzSXNk0w0/EkIZZsYjuNHQvjBEcXX/FFYK0ACwwsIKS3f+yuELJk7dq7Or9dfT8zDKs9B+3rH6vHh7Pn2TV3FwAgXHOiHgAA8NYIagAIHEENAIEjqAEgcAQ1AARubjG+aU1NjTc2NhbjWwNAWdqzZ88hd6+dbFtRgrqxsVFdXV3F+NYAUJbM7KWptnHqAwACR1ADQOAIagAIHEENAIEjqAEgcNNe9WFmTZL+e9xdyyV92d2/VqyhAKCU7Nzbr227+nRgKKUl1VXasrFJm1rqC/b9pw1qd++TdL0kmVmFpH5J9xVsAgAoYTv39mvrjn1KDY9IkvqHUtq6Y58kFSys8z310SbpeXef8no/AJhNtu3qGwvprNTwiLbt6ivYY+Qb1B+S9IPJNpjZZjPrMrOuwcHBdz4ZAJSAA0OpvO5/O3IOajObJ+lOSf8z2XZ33+7ure7eWls7aQsSAMrOkuqqvO5/O/I5or5N0pPuPlCwRweAErdlY5OqKivOu6+qskJbNjYV7DHyea+PuzTFaQ8AmK2yLxhGetWHJJnZAkntkj5RsEcGgDKxqaW+oME8UU5B7e4nJS0u2hQAgCnRTASAwBHUABA4ghoAAkdQA0DgCGoACBxBDQCBI6gBIHAENQAEjqAGgMAR1AAQOIIaAAJHUANA4AhqAAgcQQ0AgSOoASBwBDUABI6gBoDAEdQAEDiCGgACR1ADQOAIagAIHEENAIEjqAEgcAQ1AAQup6A2s2oz+5GZ9ZpZj5ndVOzBAABpc3Pc7+uSfubuHzSzeZIWFHEmAMA40wa1mS2StE7Sn0uSu5+RdKa4YwEAsnI59bFc0qCkfzezvWb2DTO7cOJOZrbZzLrMrGtwcLDggwLAbJVLUM+VtErSv7p7i6QTkr4wcSd33+7ure7eWltbW+AxAWD2yiWoX5X0qrs/nvn6R0oHNwBgBkwb1O7+mqRXzKwpc1ebpO6iTgUAGJPrVR+flvS9zBUfL0j6aPFGAgCMl1NQu/tTklqLOwoAYDI0EwEgcAQ1AASOoAaAwBHUABA4ghoAAkdQA0DgCGoACBxBDQCBI6gBIHAENQAEjqAGgMAR1AAQOIIaAAJHUANA4AhqAAgcQQ0AgSOoASBwBDUABI6gBoDAEdQAEDiCGgACR1ADQOAIagAIHEENAIGbm8tOZrZf0huSRiSddffWYg4FADgnp6DO2ODuh4o2CQBgUpz6AIDA5RrULunnZrbHzDZPtoOZbTazLjPrGhwcLNyEADDL5RrUa9x9laTbJH3SzNZN3MHdt7t7q7u31tbWFnRIAJjNcgpqdz+Q+T0p6T5Jq4s5FADgnGmD2swuNLOF2duS3ifp6WIPBgBIy+Wqj5ik+8wsu//33f1nRZ0KADBm2qB29xckvWsGZgGAkuXuOnLijBZfdEHBv3c+11EDAMYZHhnVEy8e0QM9A+rsSWrUXY/cu0GZMxAFQ1ADQB6OnhzW7meS6uhJandfUm+cOqt5c+fo5qtr1Bav08ioa24FQQ0AM+qlwyf0QHf6qPnX+49oZNRVc9E83XbdZUrEY7p5RY0WzCtenBLUADDByKjrqVde1wPdSXX2DOjZ5HFJUlNsoT6xbrkSzTFdf0W15swp7JHzVAhqAJB04vRZPfLsIXX0DOjB3qSOnDijuXNM715+qe5a3aBEPKaGxQsimY2gBjBrHTyaUkdP+qj5/54/rDNnR7Vo/lxtuKZOiXhM61bW6uKqyqjHJKgBzB7urt8dOKYHugfU0TOg3x04Jkm6cvECfeTGK5WIx9TaeIkqK8J6vzqCGkBZOzU8ol89f1gdmUvoXjt2SmbS7zdcoi/cdo0S8TpdVXtRwS+pKySCGkDZOXT8tB7sTZ/SeOTZQzp5ZkQL5lVo3YpaJZpj2tBUW5RiSrEQ1ABKnrvr2eRxdfQMqKN7QHtfGZK7dPnF8/WHq+qViMd04/LFml9ZEfWobwtBDaAkZVuBHT1JdfQM6OUjJyVJv1d/se5pW6m2eJ2uXbIo6FMauSKoAZSMo6lh7e57cytwzVWL9Yn3LlfbNTFddvH8qMcsOIIaQNBeOnwifdTcPaAn9h/R2VHX4gvn6dZrL1OiOaa1RW4FhqC8/+sAlJypWoErYxfpL9YtVyIe0/VLq1UxQ63AEBDUACI3vhX4UG9ShzOtwNXLom8FhoCgBhCJg0dT6sy8EDi+Fbi+qU6J5pjeG0grMAQENYAZMb4V2Nk7oKf7z28FtsXrdEPjpcG1AkNAUAMomlPDI/rVC4fV0X1+K3BVwyX621vTrcCr68JuBYaAoAZQUFO1AteuqNHn4yu14Zo61ZRQKzAEBDWAd8Td9Vzy+NjHUT358utyly5blG4FtsVjuqmEW4EhIKgB5G2qVuB19Yv02bYVSsRjZdMKDAFBDSAn2VZgZ09SD01oBW5et1xt8TpdfnFV1GOWJYIawJTeqhXYFk+3Ai+8gBgpNlYYwJhsKzAbztlW4Iq6bCuwTtcvvWRWtQJDkHNQm1mFpC5J/e5+R6EH2bm3X9t29enAUEpLqqu0ZWOTNrXUF/phAEwwWSuwYo5pdeOl+qMblqq9OaYrF18Y9ZizWj5H1J+V1CNpUaGH2Lm3X1t37FNqeESS1D+U0tYd+ySJsAaKYLJW4ML5c7WhqU5t8TqtX1mnixfQCgxFTkFtZldIul3SP0j6XKGH2Larbyyks1LDI9q2q4+gBgog2wrs6El/VmC2Fdhw6QJ9+N1XKtFMKzBkuR5Rf03SvZIWTrWDmW2WtFmSGhoa8hriwFAqr/sBTG98K/DB3qQOHj3XCrz31ia1x2O0AkvEtEFtZndISrr7HjNbP9V+7r5d0nZJam1t9XyGWFJdpf5JQnlJNZf6APk4nGkFdoxrBVZVVmjdyhp9rp1WYKnK5Yh6jaQ7zez9kuZLWmRm33X3DxdqiC0bm847Ry1JVZUV2rKxqVAPAZSlbCswWzwZ3wr8g5Z6JZppBZaDaYPa3bdK2ipJmSPqvylkSEvnXjDkqg9gesMjo3pi/xF1dCfV2Tuglw6fawV+5pYVam+mFVhugrmOelNLPcEMTOFoali/eGZQHd0D2t2X1LFTZzWvYo7ec/VifXztcrVdU8epwjKWV1C7+25Ju4syCYDzvHz4ZOaNjgb06xfPtQLfd+1lStAKnFX4WwYCkW4FDqkjE87PDJxrBX587XK1N9MKnK0IaiBCJ89kWoGZS+jGtwL/7o4GJeJ1tAJBUAMz7bWjp9TZO6CO7gH9clwrcH1TnRK0AjEJghoosvGtwM6epPb1H5UkLb20Kt0KjNfphmW0AjE1ghoogmwrsDMTztlWYMvSat17a5MS8ZhW0ApEjghqoEAOj31WYFIPPzs41gpcu6JGf92+UrfQCsTbRFADb5O76/nB43qgO/1BrnsyrcDYogvSrcB4TDddRSsQ7xxBDeQh2wrMvkVothV47ZJ0KzARj+m6elqBKCyCGphGthXYmXljfVqBmGkENTCJlw+fHHvv5mwr8NKxVmCd1q6opRWIGcMzDZA0Oup66tUhdXSnwznbCryaViACQFBj1sq2Ajt70q3AQ8fPtQK/dPtSJeIxNdbQCkT0CGrMKrQCUYoIapS1bCswe5XG+Fbgn7y7Qe3xGK1ABI+gRtk5fXZEv3r+sDp70tc3HxjXCtyysUntzbQCUVoIapSFw8dP66G+9BvrP/LsoE6MawXeQysQJY6gRknKtgI7epLq6E5/VuBophX4gZZ6tdMKRBkhqFEyzo6M6on9r4+9sf7+ca3AT9MKRBkjqBG0Y6eG9Yu+QXX0DGh336COpoY1r2KObrpqsT5GKxCzBEGN4Lxy5Fwr8PEXzrUC25tjSsTrdPOKWl1EKxCzCM92RG58K7CzJ6m+gTcknWsFJuJ1ammgFYjZi6BGJE6eOatHnz2kjgmtwBsaL9GXbo/TCgTGIagxYwaOnRr7OKpfPndIp2kFAjkhqFE07q7ug8fU0Z1UZ++AfvvquVbgH9MKBHI2bVCb2XxJD0u6ILP/j9z9K8UeDKXp9NkRPfbCkcz5ZlqBpWrn3n5t29WnA0MpLamu0paNTdrUUh/1WLNWLkfUpyXd4u7HzaxS0qNm9lN3f6zIs6FEHDlxJvNZgQN6+JkJrcDESm24pk61C2kFloqde/u1dcc+pYZHJEn9Qylt3bFPkgjriEwb1O7uko5nvqzM/PJiDoWwpVuBJ8aKJ3teohVYTrbt6hsL6azU8Ii27eojqCOS0zlqM6uQtEfS1ZL+xd0fn2SfzZI2S1JDQ0MhZ0QAzo6Mquul18feWJ9WYPk6MJTK634UX05B7e4jkq43s2pJ95nZde7+9IR9tkvaLkmtra0ccZeBbCuws2dAD9EKnDWWVFepf5JQ5u86Onld9eHuQ2a2W9Ktkp6eZneUoKlagYl4TO3NtAJngy0bm847Ry1JVZUV2rKxKcKpZrdcrvqolTScCekqSQlJ/1j0yTAjsq3Azp4BdXTTCsS5Fwy56iMcuRwaXS7p25nz1HMk/dDd7y/uWCimbCuwsyepzt6kDh0/TSsQ59nUUk8wBySXqz5+K6llBmZBEQ0cOzX2iSePZluBF8zV+mtoBQKh42Rjmcq2ArOfFTixFZiIx3RD46WaN5dWIBA6grqMZFuB6fPNtAKBckFQl7gjJ87ood70UTOtQKA8EdQlJtsK7MxcQkcrECh/BHUJGN8K7OxN6sVDJySlW4GfumWF2mkFAmWNoA7UsVPDeviZQXV0v7kVePfNy2gFArMIQR2QV46czJzSSOqxFw7TCgQgiaCO1Oio6zevDo196knva+dagR9bu0zt8RitQAAE9UxLnRnRo88dGjvfTCsQwHQI6hkwVSvwvU21am+O0QoE8JYI6iJwd/UcfGPsjfV/M64VeNfqBrU30woEkDuCukBOnx3R4y8cSb9F6LhW4PWZVmAiHtPKGK1AAPkjqN+BbCuws3dAv+g71wq8mVYggAIiqPP0/ODxsY+jyrYC6xZeoDuvr1d7c53ec1UNrUAABUVQTyPbCsxe35xtBTZfnm4FJuJ1um7JxZrDJXQAioSgnsRUrcAbr1qsu9c06pZ4TPW0AgHMEII6Y3wr8PEXD2t4xHXJgkol4jEl4nVau5JWIIBozNrkmaoVeFXthbr75mVKxGNaRSsQQABmVVBP1QpsvTLdCmyLx7SMViCAwJR9UCePnVJnb1Id3W9uBSbiMa1vqlX1gnlRjwkAUyq7oJ6qFXjFJelWYCIe0+pltAIBlI6yCOrxrcDOnqT6h1Iyk951Ba1AAKWvZIP69RNn9FBf+rMCs63A+ZVztHZFrT7TdrU2XFOnuoXzox4TAN6xkgrqbCuwsyeprpeOnNcKTMTrtOZqWoEAys+0QW1mSyV9R9JlkkYlbXf3rxd7MGnqVmD88kX61IarlWiOzdpW4M69/dq2q08HhlJaUl2lLRubtKmlPuqxABRBLkfUZyV93t2fNLOFkvaY2QPu3l2MgbKtwM6epB7sTepoaliVFaabrqrRR9c0qo1WoHbu7dfWHfuUGh6RJPUPpbR1xz5JIqyBMjRtULv7QUkHM7ffMLMeSfWSChrUqTMj2vyfXXrshXOtwLZ4ndrjMVqBE2zb1TcW0lmp4RFt29VHUANlKK/0M7NGSS2SHp9k22ZJmyWpoaEh70Gq5lXogrkVunvNMiWaaQW+lQNDqbzuB1Dacg5qM7tI0o8l3ePuxyZud/ftkrZLUmtrq7+dYb7xZ61v54/NOkuqq9Q/SSgvmeWnhIBylVPrw8wqlQ7p77n7juKOhOls2dikqglXt1RVVmjLxqaIJgJQTLlc9WGSvimpx92/WvyRMJ3seWiu+gBmh1xOfayR9BFJ+8zsqcx9X3T3nxRtKkxrU0s9wQzMErlc9fGoJF7VA4CI8M5EABA4ghoAAkdQA0DgCGoACBxBDQCBI6gBIHAENQAEjqAGgMAR1AAQOIIaAAJHUANA4AhqAAgcQQ0AgSOoASBwBDUABI6gBoDAEdQAEDiCGgACR1ADQOAIagAIHEENAIEjqAEgcAQ1AASOoAaAwM2dbgcz+5akOyQl3f264o8EFN7Ovf3atqtPB4ZSWlJdpS0bm7SppT7qsYCc5HJE/R+Sbi3yHEDR7Nzbr6079ql/KCWX1D+U0tYd+7Rzb3/UowE5mTao3f1hSUdmYBagKLbt6lNqeOS8+1LDI9q2qy+iiYD8FOwctZltNrMuM+saHBws1LcF3rEDQ6m87gdCU7Cgdvft7t7q7q21tbWF+rbAO7akuiqv+4HQcNUHyt6WjU2qqqw4776qygpt2dgU0URAfqa96gModdmrO7jqA6Uql8vzfiBpvaQaM3tV0lfc/ZvFHgwopE0t9QQzSta0Qe3ud83EIACAyXGOGgACR1ADQOAIagAIHEENAIEzdy/8NzUblPTS2/zjNZIOFXCcQmGu/DBXfpgrP+U415XuPmlbsChB/U6YWZe7t0Y9x0TMlR/myg9z5We2zcWpDwAIHEENAIELMai3Rz3AFJgrP8yVH+bKz6yaK7hz1ACA84V4RA0AGIegBoDARRLUZvYtM0ua2dNTbDcz+ycze87MfmtmqwKZa72ZHTWzpzK/vjxDcy01s4fMrMfMfmdmn51knxlfsxznmvE1M7P5ZvZrM/tNZq6/n2SfKNYrl7kieY5lHrvCzPaa2f2TbIvkZzKHuaL6mdxvZvsyj9k1yfbCrpe7z/gvSeskrZL09BTb3y/pp5JM0o2SHg9krvWS7o9gvS6XtCpze6GkZyQ1R71mOc4142uWWYOLMrcrJT0u6cYA1iuXuSJ5jmUe+3OSvj/Z40f1M5nDXFH9TO6XVPMW2wu6XpEcUfv0H5j7AUnf8bTHJFWb2eUBzBUJdz/o7k9mbr8hqUfSxDdXnvE1y3GuGZdZg+OZLyszvya+ah7FeuUyVyTM7ApJt0v6xhS7RPIzmcNcoSroeoV6jrpe0ivjvn5VAQRAxk2Zf7r+1MyunekHN7NGSS1KH42NF+mavcVcUgRrlvnn8lOSkpIecPcg1iuHuaRonmNfk3SvpNEptkf1/Pqa3nouKZr1ckk/N7M9ZrZ5ku0FXa9Qg9omuS+EI48nle7jv0vSP0vaOZMPbmYXSfqxpHvc/djEzZP8kRlZs2nmimTN3H3E3a+XdIWk1WZ23YRdIlmvHOaa8fUyszskJd19z1vtNsl9RV2vHOeK6mdyjbuvknSbpE+a2boJ2wu6XqEG9auSlo77+gpJByKaZYy7H8v+09XdfyKp0sxqZuKxzaxS6TD8nrvvmGSXSNZsurmiXLPMYw5J2i3p1gmbIn2OTTVXROu1RtKdZrZf0n9JusXMvjthnyjWa9q5onp+ufuBzO9JSfdJWj1hl4KuV6hB/b+S/jTzyumNko66+8GohzKzy8zMMrdXK71+h2fgcU3SNyX1uPtXp9htxtcsl7miWDMzqzWz6sztKkkJSb0TdotivaadK4r1cvet7n6FuzdK+pCkB939wxN2m/H1ymWuiJ5fF5rZwuxtSe+TNPFKsYKuVySfQm6TfGCu0i+syN3/TdJPlH7V9DlJJyV9NJC5Pijpr8zsrKSUpA955iXeIlsj6SOS9mXOb0rSFyU1jJstijXLZa4o1uxySd82swqlf3B/6O73m9lfjpsrivXKZa6onmNvEsB65TJXFOsVk3Rf5v8PcyV9391/Vsz1okIOAIEL9dQHACCDoAaAwBHUABA4ghoAAkdQA0DgCGoACBxBDQCB+3+DAu6ycy6B8AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "X = x[:, np.newaxis]\n",
    "model = LinearRegression().fit(X, y)\n",
    "yfit = model.predict(X)\n",
    "plt.scatter(x, y)\n",
    "plt.plot(x, yfit)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a95f40f0",
   "metadata": {},
   "source": [
    "Очевидно, что для описания зависимости м/у **x** и **y** нам потребуется более сложная модель.\n",
    "\n",
    "Сделать это можно путем преобразования данных, добабви дополнительные столбцы признаков для увеличения гибкости модели. \n",
    "\n",
    "Добавить в данные полиномиальные признаки можно следующим образом:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "c146ce36",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  1.   1.   1.]\n",
      " [  2.   4.   8.]\n",
      " [  3.   9.  27.]\n",
      " [  4.  16.  64.]\n",
      " [  5.  25. 125.]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "poly = PolynomialFeatures(degree=3, include_bias=False)\n",
    "X2 = poly.fit_transform(X)\n",
    "print(X2)\n",
    "# матрица производных признаков:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a90b43",
   "metadata": {},
   "source": [
    "В матрице производных признаков первый столбец соответствует $x$, второй - $x^2$, а третий - $x^3$.\n",
    "\n",
    "//*вот только, что-то я не понял, это получается мы не точки берем, а пишем ура-я отрезков, тогда почему их 5?*//\n",
    "\n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html\n",
    "\n",
    "sklearn.preprocessing.PolynomialFeatures(degree=2, *, interaction_only=False, include_bias=True, order='C')\n",
    "\n",
    "\n",
    "Сгенерирует новую матрицу признаков, состоящую из всех полиномиальных комбинаций признаков со степенью, меньшей или равной указанной степени. Например, если входная выборка является двумерной и имеет форму ``[`` $a$, $b$ ``]``, то: \n",
    "- полиномиальные признаки степени 2 будут ``[`` $1$, $a$, $b$, $a^2$, $ab$, $b^2$ ``]``\n",
    "\n",
    "//Тогда почему их 3?*//"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f431fd96",
   "metadata": {},
   "source": [
    "###### анука:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "5c9ea56b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1],\n",
       "       [2, 3],\n",
       "       [4, 5]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = np.arange(6).reshape(3, 2)\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "c5e967f9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  0.,  1.,  0.,  0.,  1.],\n",
       "       [ 1.,  2.,  3.,  4.,  6.,  9.],\n",
       "       [ 1.,  4.,  5., 16., 20., 25.]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "poly = PolynomialFeatures(2)\n",
    "poly.fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "40cfcb91",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  0.,  1.,  0.],\n",
       "       [ 1.,  2.,  3.,  6.],\n",
       "       [ 1.,  4.,  5., 20.]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "poly = PolynomialFeatures(interaction_only=True)\n",
    "poly.fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "ec87ffe0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  1.,   0.,   1.,   0.,   0.,   1.,   0.,   0.,   0.,   1.],\n",
       "       [  1.,   2.,   3.,   4.,   6.,   9.,   8.,  12.,  18.,  27.],\n",
       "       [  1.,   4.,   5.,  16.,  20.,  25.,  64.,  80., 100., 125.]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "poly = PolynomialFeatures(3)\n",
    "poly.fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "8b44b5e3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  1.,   0.,   1.,   0.,   0.,   1.,   0.,   0.,   0.,   1.],\n",
       "       [  1.,   2.,   3.,   4.,   6.,   9.,   8.,  12.,  18.,  27.],\n",
       "       [  1.,   4.,   5.,  16.,  20.,  25.,  64.,  80., 100., 125.]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "poly = PolynomialFeatures(degree=3)\n",
    "poly.fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "34eed63b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0.,   1.,   0.,   0.,   1.,   0.,   0.,   0.,   1.],\n",
       "       [  2.,   3.,   4.,   6.,   9.,   8.,  12.,  18.,  27.],\n",
       "       [  4.,   5.,  16.,  20.,  25.,  64.,  80., 100., 125.]])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "poly = PolynomialFeatures(degree=3, include_bias=False)\n",
    "poly.fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "e73a7c25",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1570cf89f10>]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAW+UlEQVR4nO3de3BU93nG8edFCCNssGwkrY2MLLBBXtmdWFQmdjAErFWwY49DO5lp3CZt46SkndzcpLghkybTP/pHh5lM0k6nHSZJmzSXNk0w0/EkIZZsYjuNHQvjBEcXX/FFYK0ACwwsIKS3f+yuELJk7dq7Or9dfT8zDKs9B+3rH6vHh7Pn2TV3FwAgXHOiHgAA8NYIagAIHEENAIEjqAEgcAQ1AARubjG+aU1NjTc2NhbjWwNAWdqzZ88hd6+dbFtRgrqxsVFdXV3F+NYAUJbM7KWptnHqAwACR1ADQOAIagAIHEENAIEjqAEgcNNe9WFmTZL+e9xdyyV92d2/VqyhAKCU7Nzbr227+nRgKKUl1VXasrFJm1rqC/b9pw1qd++TdL0kmVmFpH5J9xVsAgAoYTv39mvrjn1KDY9IkvqHUtq6Y58kFSys8z310SbpeXef8no/AJhNtu3qGwvprNTwiLbt6ivYY+Qb1B+S9IPJNpjZZjPrMrOuwcHBdz4ZAJSAA0OpvO5/O3IOajObJ+lOSf8z2XZ33+7ure7eWls7aQsSAMrOkuqqvO5/O/I5or5N0pPuPlCwRweAErdlY5OqKivOu6+qskJbNjYV7DHyea+PuzTFaQ8AmK2yLxhGetWHJJnZAkntkj5RsEcGgDKxqaW+oME8UU5B7e4nJS0u2hQAgCnRTASAwBHUABA4ghoAAkdQA0DgCGoACBxBDQCBI6gBIHAENQAEjqAGgMAR1AAQOIIaAAJHUANA4AhqAAgcQQ0AgSOoASBwBDUABI6gBoDAEdQAEDiCGgACR1ADQOAIagAIHEENAIEjqAEgcAQ1AAQup6A2s2oz+5GZ9ZpZj5ndVOzBAABpc3Pc7+uSfubuHzSzeZIWFHEmAMA40wa1mS2StE7Sn0uSu5+RdKa4YwEAsnI59bFc0qCkfzezvWb2DTO7cOJOZrbZzLrMrGtwcLDggwLAbJVLUM+VtErSv7p7i6QTkr4wcSd33+7ure7eWltbW+AxAWD2yiWoX5X0qrs/nvn6R0oHNwBgBkwb1O7+mqRXzKwpc1ebpO6iTgUAGJPrVR+flvS9zBUfL0j6aPFGAgCMl1NQu/tTklqLOwoAYDI0EwEgcAQ1AASOoAaAwBHUABA4ghoAAkdQA0DgCGoACBxBDQCBI6gBIHAENQAEjqAGgMAR1AAQOIIaAAJHUANA4AhqAAgcQQ0AgSOoASBwBDUABI6gBoDAEdQAEDiCGgACR1ADQOAIagAIHEENAIGbm8tOZrZf0huSRiSddffWYg4FADgnp6DO2ODuh4o2CQBgUpz6AIDA5RrULunnZrbHzDZPtoOZbTazLjPrGhwcLNyEADDL5RrUa9x9laTbJH3SzNZN3MHdt7t7q7u31tbWFnRIAJjNcgpqdz+Q+T0p6T5Jq4s5FADgnGmD2swuNLOF2duS3ifp6WIPBgBIy+Wqj5ik+8wsu//33f1nRZ0KADBm2qB29xckvWsGZgGAkuXuOnLijBZfdEHBv3c+11EDAMYZHhnVEy8e0QM9A+rsSWrUXY/cu0GZMxAFQ1ADQB6OnhzW7meS6uhJandfUm+cOqt5c+fo5qtr1Bav08ioa24FQQ0AM+qlwyf0QHf6qPnX+49oZNRVc9E83XbdZUrEY7p5RY0WzCtenBLUADDByKjrqVde1wPdSXX2DOjZ5HFJUlNsoT6xbrkSzTFdf0W15swp7JHzVAhqAJB04vRZPfLsIXX0DOjB3qSOnDijuXNM715+qe5a3aBEPKaGxQsimY2gBjBrHTyaUkdP+qj5/54/rDNnR7Vo/lxtuKZOiXhM61bW6uKqyqjHJKgBzB7urt8dOKYHugfU0TOg3x04Jkm6cvECfeTGK5WIx9TaeIkqK8J6vzqCGkBZOzU8ol89f1gdmUvoXjt2SmbS7zdcoi/cdo0S8TpdVXtRwS+pKySCGkDZOXT8tB7sTZ/SeOTZQzp5ZkQL5lVo3YpaJZpj2tBUW5RiSrEQ1ABKnrvr2eRxdfQMqKN7QHtfGZK7dPnF8/WHq+qViMd04/LFml9ZEfWobwtBDaAkZVuBHT1JdfQM6OUjJyVJv1d/se5pW6m2eJ2uXbIo6FMauSKoAZSMo6lh7e57cytwzVWL9Yn3LlfbNTFddvH8qMcsOIIaQNBeOnwifdTcPaAn9h/R2VHX4gvn6dZrL1OiOaa1RW4FhqC8/+sAlJypWoErYxfpL9YtVyIe0/VLq1UxQ63AEBDUACI3vhX4UG9ShzOtwNXLom8FhoCgBhCJg0dT6sy8EDi+Fbi+qU6J5pjeG0grMAQENYAZMb4V2Nk7oKf7z28FtsXrdEPjpcG1AkNAUAMomlPDI/rVC4fV0X1+K3BVwyX621vTrcCr68JuBYaAoAZQUFO1AteuqNHn4yu14Zo61ZRQKzAEBDWAd8Td9Vzy+NjHUT358utyly5blG4FtsVjuqmEW4EhIKgB5G2qVuB19Yv02bYVSsRjZdMKDAFBDSAn2VZgZ09SD01oBW5et1xt8TpdfnFV1GOWJYIawJTeqhXYFk+3Ai+8gBgpNlYYwJhsKzAbztlW4Iq6bCuwTtcvvWRWtQJDkHNQm1mFpC5J/e5+R6EH2bm3X9t29enAUEpLqqu0ZWOTNrXUF/phAEwwWSuwYo5pdeOl+qMblqq9OaYrF18Y9ZizWj5H1J+V1CNpUaGH2Lm3X1t37FNqeESS1D+U0tYd+ySJsAaKYLJW4ML5c7WhqU5t8TqtX1mnixfQCgxFTkFtZldIul3SP0j6XKGH2Larbyyks1LDI9q2q4+gBgog2wrs6El/VmC2Fdhw6QJ9+N1XKtFMKzBkuR5Rf03SvZIWTrWDmW2WtFmSGhoa8hriwFAqr/sBTG98K/DB3qQOHj3XCrz31ia1x2O0AkvEtEFtZndISrr7HjNbP9V+7r5d0nZJam1t9XyGWFJdpf5JQnlJNZf6APk4nGkFdoxrBVZVVmjdyhp9rp1WYKnK5Yh6jaQ7zez9kuZLWmRm33X3DxdqiC0bm847Ry1JVZUV2rKxqVAPAZSlbCswWzwZ3wr8g5Z6JZppBZaDaYPa3bdK2ipJmSPqvylkSEvnXjDkqg9gesMjo3pi/xF1dCfV2Tuglw6fawV+5pYVam+mFVhugrmOelNLPcEMTOFoali/eGZQHd0D2t2X1LFTZzWvYo7ec/VifXztcrVdU8epwjKWV1C7+25Ju4syCYDzvHz4ZOaNjgb06xfPtQLfd+1lStAKnFX4WwYCkW4FDqkjE87PDJxrBX587XK1N9MKnK0IaiBCJ89kWoGZS+jGtwL/7o4GJeJ1tAJBUAMz7bWjp9TZO6CO7gH9clwrcH1TnRK0AjEJghoosvGtwM6epPb1H5UkLb20Kt0KjNfphmW0AjE1ghoogmwrsDMTztlWYMvSat17a5MS8ZhW0ApEjghqoEAOj31WYFIPPzs41gpcu6JGf92+UrfQCsTbRFADb5O76/nB43qgO/1BrnsyrcDYogvSrcB4TDddRSsQ7xxBDeQh2wrMvkVothV47ZJ0KzARj+m6elqBKCyCGphGthXYmXljfVqBmGkENTCJlw+fHHvv5mwr8NKxVmCd1q6opRWIGcMzDZA0Oup66tUhdXSnwznbCryaViACQFBj1sq2Ajt70q3AQ8fPtQK/dPtSJeIxNdbQCkT0CGrMKrQCUYoIapS1bCswe5XG+Fbgn7y7Qe3xGK1ABI+gRtk5fXZEv3r+sDp70tc3HxjXCtyysUntzbQCUVoIapSFw8dP66G+9BvrP/LsoE6MawXeQysQJY6gRknKtgI7epLq6E5/VuBophX4gZZ6tdMKRBkhqFEyzo6M6on9r4+9sf7+ca3AT9MKRBkjqBG0Y6eG9Yu+QXX0DGh336COpoY1r2KObrpqsT5GKxCzBEGN4Lxy5Fwr8PEXzrUC25tjSsTrdPOKWl1EKxCzCM92RG58K7CzJ6m+gTcknWsFJuJ1ammgFYjZi6BGJE6eOatHnz2kjgmtwBsaL9GXbo/TCgTGIagxYwaOnRr7OKpfPndIp2kFAjkhqFE07q7ug8fU0Z1UZ++AfvvquVbgH9MKBHI2bVCb2XxJD0u6ILP/j9z9K8UeDKXp9NkRPfbCkcz5ZlqBpWrn3n5t29WnA0MpLamu0paNTdrUUh/1WLNWLkfUpyXd4u7HzaxS0qNm9lN3f6zIs6FEHDlxJvNZgQN6+JkJrcDESm24pk61C2kFloqde/u1dcc+pYZHJEn9Qylt3bFPkgjriEwb1O7uko5nvqzM/PJiDoWwpVuBJ8aKJ3teohVYTrbt6hsL6azU8Ii27eojqCOS0zlqM6uQtEfS1ZL+xd0fn2SfzZI2S1JDQ0MhZ0QAzo6Mquul18feWJ9WYPk6MJTK634UX05B7e4jkq43s2pJ95nZde7+9IR9tkvaLkmtra0ccZeBbCuws2dAD9EKnDWWVFepf5JQ5u86Onld9eHuQ2a2W9Ktkp6eZneUoKlagYl4TO3NtAJngy0bm847Ry1JVZUV2rKxKcKpZrdcrvqolTScCekqSQlJ/1j0yTAjsq3Azp4BdXTTCsS5Fwy56iMcuRwaXS7p25nz1HMk/dDd7y/uWCimbCuwsyepzt6kDh0/TSsQ59nUUk8wBySXqz5+K6llBmZBEQ0cOzX2iSePZluBF8zV+mtoBQKh42Rjmcq2ArOfFTixFZiIx3RD46WaN5dWIBA6grqMZFuB6fPNtAKBckFQl7gjJ87ood70UTOtQKA8EdQlJtsK7MxcQkcrECh/BHUJGN8K7OxN6sVDJySlW4GfumWF2mkFAmWNoA7UsVPDeviZQXV0v7kVePfNy2gFArMIQR2QV46czJzSSOqxFw7TCgQgiaCO1Oio6zevDo196knva+dagR9bu0zt8RitQAAE9UxLnRnRo88dGjvfTCsQwHQI6hkwVSvwvU21am+O0QoE8JYI6iJwd/UcfGPsjfV/M64VeNfqBrU30woEkDuCukBOnx3R4y8cSb9F6LhW4PWZVmAiHtPKGK1AAPkjqN+BbCuws3dAv+g71wq8mVYggAIiqPP0/ODxsY+jyrYC6xZeoDuvr1d7c53ec1UNrUAABUVQTyPbCsxe35xtBTZfnm4FJuJ1um7JxZrDJXQAioSgnsRUrcAbr1qsu9c06pZ4TPW0AgHMEII6Y3wr8PEXD2t4xHXJgkol4jEl4nVau5JWIIBozNrkmaoVeFXthbr75mVKxGNaRSsQQABmVVBP1QpsvTLdCmyLx7SMViCAwJR9UCePnVJnb1Id3W9uBSbiMa1vqlX1gnlRjwkAUyq7oJ6qFXjFJelWYCIe0+pltAIBlI6yCOrxrcDOnqT6h1Iyk951Ba1AAKWvZIP69RNn9FBf+rMCs63A+ZVztHZFrT7TdrU2XFOnuoXzox4TAN6xkgrqbCuwsyeprpeOnNcKTMTrtOZqWoEAys+0QW1mSyV9R9JlkkYlbXf3rxd7MGnqVmD88kX61IarlWiOzdpW4M69/dq2q08HhlJaUl2lLRubtKmlPuqxABRBLkfUZyV93t2fNLOFkvaY2QPu3l2MgbKtwM6epB7sTepoaliVFaabrqrRR9c0qo1WoHbu7dfWHfuUGh6RJPUPpbR1xz5JIqyBMjRtULv7QUkHM7ffMLMeSfWSChrUqTMj2vyfXXrshXOtwLZ4ndrjMVqBE2zb1TcW0lmp4RFt29VHUANlKK/0M7NGSS2SHp9k22ZJmyWpoaEh70Gq5lXogrkVunvNMiWaaQW+lQNDqbzuB1Dacg5qM7tI0o8l3ePuxyZud/ftkrZLUmtrq7+dYb7xZ61v54/NOkuqq9Q/SSgvmeWnhIBylVPrw8wqlQ7p77n7juKOhOls2dikqglXt1RVVmjLxqaIJgJQTLlc9WGSvimpx92/WvyRMJ3seWiu+gBmh1xOfayR9BFJ+8zsqcx9X3T3nxRtKkxrU0s9wQzMErlc9fGoJF7VA4CI8M5EABA4ghoAAkdQA0DgCGoACBxBDQCBI6gBIHAENQAEjqAGgMAR1AAQOIIaAAJHUANA4AhqAAgcQQ0AgSOoASBwBDUABI6gBoDAEdQAEDiCGgACR1ADQOAIagAIHEENAIEjqAEgcAQ1AASOoAaAwM2dbgcz+5akOyQl3f264o8EFN7Ovf3atqtPB4ZSWlJdpS0bm7SppT7qsYCc5HJE/R+Sbi3yHEDR7Nzbr6079ql/KCWX1D+U0tYd+7Rzb3/UowE5mTao3f1hSUdmYBagKLbt6lNqeOS8+1LDI9q2qy+iiYD8FOwctZltNrMuM+saHBws1LcF3rEDQ6m87gdCU7Cgdvft7t7q7q21tbWF+rbAO7akuiqv+4HQcNUHyt6WjU2qqqw4776qygpt2dgU0URAfqa96gModdmrO7jqA6Uql8vzfiBpvaQaM3tV0lfc/ZvFHgwopE0t9QQzSta0Qe3ud83EIACAyXGOGgACR1ADQOAIagAIHEENAIEzdy/8NzUblPTS2/zjNZIOFXCcQmGu/DBXfpgrP+U415XuPmlbsChB/U6YWZe7t0Y9x0TMlR/myg9z5We2zcWpDwAIHEENAIELMai3Rz3AFJgrP8yVH+bKz6yaK7hz1ACA84V4RA0AGIegBoDARRLUZvYtM0ua2dNTbDcz+ycze87MfmtmqwKZa72ZHTWzpzK/vjxDcy01s4fMrMfMfmdmn51knxlfsxznmvE1M7P5ZvZrM/tNZq6/n2SfKNYrl7kieY5lHrvCzPaa2f2TbIvkZzKHuaL6mdxvZvsyj9k1yfbCrpe7z/gvSeskrZL09BTb3y/pp5JM0o2SHg9krvWS7o9gvS6XtCpze6GkZyQ1R71mOc4142uWWYOLMrcrJT0u6cYA1iuXuSJ5jmUe+3OSvj/Z40f1M5nDXFH9TO6XVPMW2wu6XpEcUfv0H5j7AUnf8bTHJFWb2eUBzBUJdz/o7k9mbr8hqUfSxDdXnvE1y3GuGZdZg+OZLyszvya+ah7FeuUyVyTM7ApJt0v6xhS7RPIzmcNcoSroeoV6jrpe0ivjvn5VAQRAxk2Zf7r+1MyunekHN7NGSS1KH42NF+mavcVcUgRrlvnn8lOSkpIecPcg1iuHuaRonmNfk3SvpNEptkf1/Pqa3nouKZr1ckk/N7M9ZrZ5ku0FXa9Qg9omuS+EI48nle7jv0vSP0vaOZMPbmYXSfqxpHvc/djEzZP8kRlZs2nmimTN3H3E3a+XdIWk1WZ23YRdIlmvHOaa8fUyszskJd19z1vtNsl9RV2vHOeK6mdyjbuvknSbpE+a2boJ2wu6XqEG9auSlo77+gpJByKaZYy7H8v+09XdfyKp0sxqZuKxzaxS6TD8nrvvmGSXSNZsurmiXLPMYw5J2i3p1gmbIn2OTTVXROu1RtKdZrZf0n9JusXMvjthnyjWa9q5onp+ufuBzO9JSfdJWj1hl4KuV6hB/b+S/jTzyumNko66+8GohzKzy8zMMrdXK71+h2fgcU3SNyX1uPtXp9htxtcsl7miWDMzqzWz6sztKkkJSb0TdotivaadK4r1cvet7n6FuzdK+pCkB939wxN2m/H1ymWuiJ5fF5rZwuxtSe+TNPFKsYKuVySfQm6TfGCu0i+syN3/TdJPlH7V9DlJJyV9NJC5Pijpr8zsrKSUpA955iXeIlsj6SOS9mXOb0rSFyU1jJstijXLZa4o1uxySd82swqlf3B/6O73m9lfjpsrivXKZa6onmNvEsB65TJXFOsVk3Rf5v8PcyV9391/Vsz1okIOAIEL9dQHACCDoAaAwBHUABA4ghoAAkdQA0DgCGoACBxBDQCB+3+DAu6ycy6B8AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = x[:, np.newaxis]\n",
    "model = LinearRegression().fit(X, y)\n",
    "yfit = model.predict(X)\n",
    "plt.scatter(x, y)\n",
    "plt.plot(x, yfit)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "e183730a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1],\n",
       "       [2],\n",
       "       [3],\n",
       "       [4],\n",
       "       [5]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "afa79a35",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  1.   1.   1.]\n",
      " [  2.   4.   8.]\n",
      " [  3.   9.  27.]\n",
      " [  4.  16.  64.]\n",
      " [  5.  25. 125.]]\n"
     ]
    }
   ],
   "source": [
    "poly = PolynomialFeatures(degree=3, include_bias=False)\n",
    "X2 = poly.fit_transform(X)\n",
    "print(X2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a79d81e0",
   "metadata": {},
   "source": [
    "###### Потому что это ``только для x``    ``не для y``"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "057fb60d",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = np.array([[0, 1], \n",
    "              [0, 2],\n",
    "              [0, 3],\n",
    "              [0, 4],\n",
    "              [0, 5]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "f100b119",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  0.,  1.,  0.,  0.,  1.],\n",
       "       [ 1.,  0.,  2.,  0.,  0.,  4.],\n",
       "       [ 1.,  0.,  3.,  0.,  0.,  9.],\n",
       "       [ 1.,  0.,  4.,  0.,  0., 16.],\n",
       "       [ 1.,  0.,  5.,  0.,  0., 25.]])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "poly = PolynomialFeatures(2)\n",
    "poly.fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e06ac7df",
   "metadata": {},
   "source": [
    "###### как-то так"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "a5446423",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x15705e61430>]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAW+UlEQVR4nO3de3BU93nG8edFCCNssGwkrY2MLLBBXtmdWFQmdjAErFWwY49DO5lp3CZt46SkndzcpLghkybTP/pHh5lM0k6nHSZJmzSXNk0w0/EkIZZsYjuNHQvjBEcXX/FFYK0ACwwsIKS3f+yuELJk7dq7Or9dfT8zDKs9B+3rH6vHh7Pn2TV3FwAgXHOiHgAA8NYIagAIHEENAIEjqAEgcAQ1AARubjG+aU1NjTc2NhbjWwNAWdqzZ88hd6+dbFtRgrqxsVFdXV3F+NYAUJbM7KWptnHqAwACR1ADQOAIagAIHEENAIEjqAEgcNNe9WFmTZL+e9xdyyV92d2/VqyhAKCU7Nzbr227+nRgKKUl1VXasrFJm1rqC/b9pw1qd++TdL0kmVmFpH5J9xVsAgAoYTv39mvrjn1KDY9IkvqHUtq6Y58kFSys8z310SbpeXef8no/AJhNtu3qGwvprNTwiLbt6ivYY+Qb1B+S9IPJNpjZZjPrMrOuwcHBdz4ZAJSAA0OpvO5/O3IOajObJ+lOSf8z2XZ33+7ure7eWls7aQsSAMrOkuqqvO5/O/I5or5N0pPuPlCwRweAErdlY5OqKivOu6+qskJbNjYV7DHyea+PuzTFaQ8AmK2yLxhGetWHJJnZAkntkj5RsEcGgDKxqaW+oME8UU5B7e4nJS0u2hQAgCnRTASAwBHUABA4ghoAAkdQA0DgCGoACBxBDQCBI6gBIHAENQAEjqAGgMAR1AAQOIIaAAJHUANA4AhqAAgcQQ0AgSOoASBwBDUABI6gBoDAEdQAEDiCGgACR1ADQOAIagAIHEENAIEjqAEgcAQ1AAQup6A2s2oz+5GZ9ZpZj5ndVOzBAABpc3Pc7+uSfubuHzSzeZIWFHEmAMA40wa1mS2StE7Sn0uSu5+RdKa4YwEAsnI59bFc0qCkfzezvWb2DTO7cOJOZrbZzLrMrGtwcLDggwLAbJVLUM+VtErSv7p7i6QTkr4wcSd33+7ure7eWltbW+AxAWD2yiWoX5X0qrs/nvn6R0oHNwBgBkwb1O7+mqRXzKwpc1ebpO6iTgUAGJPrVR+flvS9zBUfL0j6aPFGAgCMl1NQu/tTklqLOwoAYDI0EwEgcAQ1AASOoAaAwBHUABA4ghoAAkdQA0DgCGoACBxBDQCBI6gBIHAENQAEjqAGgMAR1AAQOIIaAAJHUANA4AhqAAgcQQ0AgSOoASBwBDUABI6gBoDAEdQAEDiCGgACR1ADQOAIagAIHEENAIGbm8tOZrZf0huSRiSddffWYg4FADgnp6DO2ODuh4o2CQBgUpz6AIDA5RrULunnZrbHzDZPtoOZbTazLjPrGhwcLNyEADDL5RrUa9x9laTbJH3SzNZN3MHdt7t7q7u31tbWFnRIAJjNcgpqdz+Q+T0p6T5Jq4s5FADgnGmD2swuNLOF2duS3ifp6WIPBgBIy+Wqj5ik+8wsu//33f1nRZ0KADBm2qB29xckvWsGZgGAkuXuOnLijBZfdEHBv3c+11EDAMYZHhnVEy8e0QM9A+rsSWrUXY/cu0GZMxAFQ1ADQB6OnhzW7meS6uhJandfUm+cOqt5c+fo5qtr1Bav08ioa24FQQ0AM+qlwyf0QHf6qPnX+49oZNRVc9E83XbdZUrEY7p5RY0WzCtenBLUADDByKjrqVde1wPdSXX2DOjZ5HFJUlNsoT6xbrkSzTFdf0W15swp7JHzVAhqAJB04vRZPfLsIXX0DOjB3qSOnDijuXNM715+qe5a3aBEPKaGxQsimY2gBjBrHTyaUkdP+qj5/54/rDNnR7Vo/lxtuKZOiXhM61bW6uKqyqjHJKgBzB7urt8dOKYHugfU0TOg3x04Jkm6cvECfeTGK5WIx9TaeIkqK8J6vzqCGkBZOzU8ol89f1gdmUvoXjt2SmbS7zdcoi/cdo0S8TpdVXtRwS+pKySCGkDZOXT8tB7sTZ/SeOTZQzp5ZkQL5lVo3YpaJZpj2tBUW5RiSrEQ1ABKnrvr2eRxdfQMqKN7QHtfGZK7dPnF8/WHq+qViMd04/LFml9ZEfWobwtBDaAkZVuBHT1JdfQM6OUjJyVJv1d/se5pW6m2eJ2uXbIo6FMauSKoAZSMo6lh7e57cytwzVWL9Yn3LlfbNTFddvH8qMcsOIIaQNBeOnwifdTcPaAn9h/R2VHX4gvn6dZrL1OiOaa1RW4FhqC8/+sAlJypWoErYxfpL9YtVyIe0/VLq1UxQ63AEBDUACI3vhX4UG9ShzOtwNXLom8FhoCgBhCJg0dT6sy8EDi+Fbi+qU6J5pjeG0grMAQENYAZMb4V2Nk7oKf7z28FtsXrdEPjpcG1AkNAUAMomlPDI/rVC4fV0X1+K3BVwyX621vTrcCr68JuBYaAoAZQUFO1AteuqNHn4yu14Zo61ZRQKzAEBDWAd8Td9Vzy+NjHUT358utyly5blG4FtsVjuqmEW4EhIKgB5G2qVuB19Yv02bYVSsRjZdMKDAFBDSAn2VZgZ09SD01oBW5et1xt8TpdfnFV1GOWJYIawJTeqhXYFk+3Ai+8gBgpNlYYwJhsKzAbztlW4Iq6bCuwTtcvvWRWtQJDkHNQm1mFpC5J/e5+R6EH2bm3X9t29enAUEpLqqu0ZWOTNrXUF/phAEwwWSuwYo5pdeOl+qMblqq9OaYrF18Y9ZizWj5H1J+V1CNpUaGH2Lm3X1t37FNqeESS1D+U0tYd+ySJsAaKYLJW4ML5c7WhqU5t8TqtX1mnixfQCgxFTkFtZldIul3SP0j6XKGH2Larbyyks1LDI9q2q4+gBgog2wrs6El/VmC2Fdhw6QJ9+N1XKtFMKzBkuR5Rf03SvZIWTrWDmW2WtFmSGhoa8hriwFAqr/sBTG98K/DB3qQOHj3XCrz31ia1x2O0AkvEtEFtZndISrr7HjNbP9V+7r5d0nZJam1t9XyGWFJdpf5JQnlJNZf6APk4nGkFdoxrBVZVVmjdyhp9rp1WYKnK5Yh6jaQ7zez9kuZLWmRm33X3DxdqiC0bm847Ry1JVZUV2rKxqVAPAZSlbCswWzwZ3wr8g5Z6JZppBZaDaYPa3bdK2ipJmSPqvylkSEvnXjDkqg9gesMjo3pi/xF1dCfV2Tuglw6fawV+5pYVam+mFVhugrmOelNLPcEMTOFoali/eGZQHd0D2t2X1LFTZzWvYo7ec/VifXztcrVdU8epwjKWV1C7+25Ju4syCYDzvHz4ZOaNjgb06xfPtQLfd+1lStAKnFX4WwYCkW4FDqkjE87PDJxrBX587XK1N9MKnK0IaiBCJ89kWoGZS+jGtwL/7o4GJeJ1tAJBUAMz7bWjp9TZO6CO7gH9clwrcH1TnRK0AjEJghoosvGtwM6epPb1H5UkLb20Kt0KjNfphmW0AjE1ghoogmwrsDMTztlWYMvSat17a5MS8ZhW0ApEjghqoEAOj31WYFIPPzs41gpcu6JGf92+UrfQCsTbRFADb5O76/nB43qgO/1BrnsyrcDYogvSrcB4TDddRSsQ7xxBDeQh2wrMvkVothV47ZJ0KzARj+m6elqBKCyCGphGthXYmXljfVqBmGkENTCJlw+fHHvv5mwr8NKxVmCd1q6opRWIGcMzDZA0Oup66tUhdXSnwznbCryaViACQFBj1sq2Ajt70q3AQ8fPtQK/dPtSJeIxNdbQCkT0CGrMKrQCUYoIapS1bCswe5XG+Fbgn7y7Qe3xGK1ABI+gRtk5fXZEv3r+sDp70tc3HxjXCtyysUntzbQCUVoIapSFw8dP66G+9BvrP/LsoE6MawXeQysQJY6gRknKtgI7epLq6E5/VuBophX4gZZ6tdMKRBkhqFEyzo6M6on9r4+9sf7+ca3AT9MKRBkjqBG0Y6eG9Yu+QXX0DGh336COpoY1r2KObrpqsT5GKxCzBEGN4Lxy5Fwr8PEXzrUC25tjSsTrdPOKWl1EKxCzCM92RG58K7CzJ6m+gTcknWsFJuJ1ammgFYjZi6BGJE6eOatHnz2kjgmtwBsaL9GXbo/TCgTGIagxYwaOnRr7OKpfPndIp2kFAjkhqFE07q7ug8fU0Z1UZ++AfvvquVbgH9MKBHI2bVCb2XxJD0u6ILP/j9z9K8UeDKXp9NkRPfbCkcz5ZlqBpWrn3n5t29WnA0MpLamu0paNTdrUUh/1WLNWLkfUpyXd4u7HzaxS0qNm9lN3f6zIs6FEHDlxJvNZgQN6+JkJrcDESm24pk61C2kFloqde/u1dcc+pYZHJEn9Qylt3bFPkgjriEwb1O7uko5nvqzM/PJiDoWwpVuBJ8aKJ3teohVYTrbt6hsL6azU8Ii27eojqCOS0zlqM6uQtEfS1ZL+xd0fn2SfzZI2S1JDQ0MhZ0QAzo6Mquul18feWJ9WYPk6MJTK634UX05B7e4jkq43s2pJ95nZde7+9IR9tkvaLkmtra0ccZeBbCuws2dAD9EKnDWWVFepf5JQ5u86Onld9eHuQ2a2W9Ktkp6eZneUoKlagYl4TO3NtAJngy0bm847Ry1JVZUV2rKxKcKpZrdcrvqolTScCekqSQlJ/1j0yTAjsq3Azp4BdXTTCsS5Fwy56iMcuRwaXS7p25nz1HMk/dDd7y/uWCimbCuwsyepzt6kDh0/TSsQ59nUUk8wBySXqz5+K6llBmZBEQ0cOzX2iSePZluBF8zV+mtoBQKh42Rjmcq2ArOfFTixFZiIx3RD46WaN5dWIBA6grqMZFuB6fPNtAKBckFQl7gjJ87ood70UTOtQKA8EdQlJtsK7MxcQkcrECh/BHUJGN8K7OxN6sVDJySlW4GfumWF2mkFAmWNoA7UsVPDeviZQXV0v7kVePfNy2gFArMIQR2QV46czJzSSOqxFw7TCgQgiaCO1Oio6zevDo196knva+dagR9bu0zt8RitQAAE9UxLnRnRo88dGjvfTCsQwHQI6hkwVSvwvU21am+O0QoE8JYI6iJwd/UcfGPsjfV/M64VeNfqBrU30woEkDuCukBOnx3R4y8cSb9F6LhW4PWZVmAiHtPKGK1AAPkjqN+BbCuws3dAv+g71wq8mVYggAIiqPP0/ODxsY+jyrYC6xZeoDuvr1d7c53ec1UNrUAABUVQTyPbCsxe35xtBTZfnm4FJuJ1um7JxZrDJXQAioSgnsRUrcAbr1qsu9c06pZ4TPW0AgHMEII6Y3wr8PEXD2t4xHXJgkol4jEl4nVau5JWIIBozNrkmaoVeFXthbr75mVKxGNaRSsQQABmVVBP1QpsvTLdCmyLx7SMViCAwJR9UCePnVJnb1Id3W9uBSbiMa1vqlX1gnlRjwkAUyq7oJ6qFXjFJelWYCIe0+pltAIBlI6yCOrxrcDOnqT6h1Iyk951Ba1AAKWvZIP69RNn9FBf+rMCs63A+ZVztHZFrT7TdrU2XFOnuoXzox4TAN6xkgrqbCuwsyeprpeOnNcKTMTrtOZqWoEAys+0QW1mSyV9R9JlkkYlbXf3rxd7MGnqVmD88kX61IarlWiOzdpW4M69/dq2q08HhlJaUl2lLRubtKmlPuqxABRBLkfUZyV93t2fNLOFkvaY2QPu3l2MgbKtwM6epB7sTepoaliVFaabrqrRR9c0qo1WoHbu7dfWHfuUGh6RJPUPpbR1xz5JIqyBMjRtULv7QUkHM7ffMLMeSfWSChrUqTMj2vyfXXrshXOtwLZ4ndrjMVqBE2zb1TcW0lmp4RFt29VHUANlKK/0M7NGSS2SHp9k22ZJmyWpoaEh70Gq5lXogrkVunvNMiWaaQW+lQNDqbzuB1Dacg5qM7tI0o8l3ePuxyZud/ftkrZLUmtrq7+dYb7xZ61v54/NOkuqq9Q/SSgvmeWnhIBylVPrw8wqlQ7p77n7juKOhOls2dikqglXt1RVVmjLxqaIJgJQTLlc9WGSvimpx92/WvyRMJ3seWiu+gBmh1xOfayR9BFJ+8zsqcx9X3T3nxRtKkxrU0s9wQzMErlc9fGoJF7VA4CI8M5EABA4ghoAAkdQA0DgCGoACBxBDQCBI6gBIHAENQAEjqAGgMAR1AAQOIIaAAJHUANA4AhqAAgcQQ0AgSOoASBwBDUABI6gBoDAEdQAEDiCGgACR1ADQOAIagAIHEENAIEjqAEgcAQ1AASOoAaAwM2dbgcz+5akOyQl3f264o8EFN7Ovf3atqtPB4ZSWlJdpS0bm7SppT7qsYCc5HJE/R+Sbi3yHEDR7Nzbr6079ql/KCWX1D+U0tYd+7Rzb3/UowE5mTao3f1hSUdmYBagKLbt6lNqeOS8+1LDI9q2qy+iiYD8FOwctZltNrMuM+saHBws1LcF3rEDQ6m87gdCU7Cgdvft7t7q7q21tbWF+rbAO7akuiqv+4HQcNUHyt6WjU2qqqw4776qygpt2dgU0URAfqa96gModdmrO7jqA6Uql8vzfiBpvaQaM3tV0lfc/ZvFHgwopE0t9QQzSta0Qe3ud83EIACAyXGOGgACR1ADQOAIagAIHEENAIEzdy/8NzUblPTS2/zjNZIOFXCcQmGu/DBXfpgrP+U415XuPmlbsChB/U6YWZe7t0Y9x0TMlR/myg9z5We2zcWpDwAIHEENAIELMai3Rz3AFJgrP8yVH+bKz6yaK7hz1ACA84V4RA0AGIegBoDARRLUZvYtM0ua2dNTbDcz+ycze87MfmtmqwKZa72ZHTWzpzK/vjxDcy01s4fMrMfMfmdmn51knxlfsxznmvE1M7P5ZvZrM/tNZq6/n2SfKNYrl7kieY5lHrvCzPaa2f2TbIvkZzKHuaL6mdxvZvsyj9k1yfbCrpe7z/gvSeskrZL09BTb3y/pp5JM0o2SHg9krvWS7o9gvS6XtCpze6GkZyQ1R71mOc4142uWWYOLMrcrJT0u6cYA1iuXuSJ5jmUe+3OSvj/Z40f1M5nDXFH9TO6XVPMW2wu6XpEcUfv0H5j7AUnf8bTHJFWb2eUBzBUJdz/o7k9mbr8hqUfSxDdXnvE1y3GuGZdZg+OZLyszvya+ah7FeuUyVyTM7ApJt0v6xhS7RPIzmcNcoSroeoV6jrpe0ivjvn5VAQRAxk2Zf7r+1MyunekHN7NGSS1KH42NF+mavcVcUgRrlvnn8lOSkpIecPcg1iuHuaRonmNfk3SvpNEptkf1/Pqa3nouKZr1ckk/N7M9ZrZ5ku0FXa9Qg9omuS+EI48nle7jv0vSP0vaOZMPbmYXSfqxpHvc/djEzZP8kRlZs2nmimTN3H3E3a+XdIWk1WZ23YRdIlmvHOaa8fUyszskJd19z1vtNsl9RV2vHOeK6mdyjbuvknSbpE+a2boJ2wu6XqEG9auSlo77+gpJByKaZYy7H8v+09XdfyKp0sxqZuKxzaxS6TD8nrvvmGSXSNZsurmiXLPMYw5J2i3p1gmbIn2OTTVXROu1RtKdZrZf0n9JusXMvjthnyjWa9q5onp+ufuBzO9JSfdJWj1hl4KuV6hB/b+S/jTzyumNko66+8GohzKzy8zMMrdXK71+h2fgcU3SNyX1uPtXp9htxtcsl7miWDMzqzWz6sztKkkJSb0TdotivaadK4r1cvet7n6FuzdK+pCkB939wxN2m/H1ymWuiJ5fF5rZwuxtSe+TNPFKsYKuVySfQm6TfGCu0i+syN3/TdJPlH7V9DlJJyV9NJC5Pijpr8zsrKSUpA955iXeIlsj6SOS9mXOb0rSFyU1jJstijXLZa4o1uxySd82swqlf3B/6O73m9lfjpsrivXKZa6onmNvEsB65TJXFOsVk3Rf5v8PcyV9391/Vsz1okIOAIEL9dQHACCDoAaAwBHUABA4ghoAAkdQA0DgCGoACBxBDQCB+3+DAu6ycy6B8AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = x[:, np.newaxis]\n",
    "model = LinearRegression().fit(X, y)\n",
    "yfit = model.predict(X)\n",
    "plt.scatter(x, y)\n",
    "plt.plot(x, yfit)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "26e267d2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  1.   1.   1.   1.]\n",
      " [  1.   2.   4.   8.]\n",
      " [  1.   3.   9.  27.]\n",
      " [  1.   4.  16.  64.]\n",
      " [  1.   5.  25. 125.]]\n"
     ]
    }
   ],
   "source": [
    "poly = PolynomialFeatures(degree=3, include_bias=True) # include_bias=True включать смещение\n",
    "X2 = poly.fit_transform(X)\n",
    "print(X2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "a398ba17",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  1.   1.   1.]\n",
      " [  2.   4.   8.]\n",
      " [  3.   9.  27.]\n",
      " [  4.  16.  64.]\n",
      " [  5.  25. 125.]]\n"
     ]
    }
   ],
   "source": [
    "poly = PolynomialFeatures(degree=3, include_bias=False)\n",
    "X2 = poly.fit_transform(X)\n",
    "print(X2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af45e099",
   "metadata": {},
   "source": [
    "Расчет линейной регрессии для таких **расширенных** входных данных позволяет получить намного лучшую аппроксимацию:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "64006107",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x1570e65ee80>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAma0lEQVR4nO3dd3xW5eH+8c+dPUkICTNA2EM2YQtuGaJFK+CoVCuibb9q1Tqwre1Xvy0utNUOxd06UdFWyyxuhUBYQUjCRpIwAiEhIfvJ/fsjkZ/SQJ7o8+Q843q/XnklOeeQc3GTXJyc55xzG2stIiLiu0KcDiAiIqenohYR8XEqahERH6eiFhHxcSpqEREfF+aNL5qcnGzT0tK88aVFRALSunXrDltrUxpb55WiTktLIzMz0xtfWkQkIBlj9p5qnU59iIj4OBW1iIiPU1GLiPg4FbWIiI9TUYuI+DgVtYjI95W1EB4fAL9LrH+ftdCjX77JojbG9DHGbPzG2zFjzC88mkJExF9lLYT3boGSfYCtf//eLR4t6yaL2lqba60dYq0dAgwHyoF3PJZARMSfrbwfair41DWAF2svpMaGQk1F/XIPae4NL+cBO621p7wwW0QkmNiSPFw2hPtrZ1FLKFeHrjyx3HhoH80t6iuA1zy0bxERv3eQZD5wDWS7TeWp8McJN64Ty9t7aB9uv5hojIkALgHePMX6OcaYTGNMZmFhoYfiiYj4tgeqZjK/djrpJpeJIWsBKLcRzKue7rF9NOeqj8nAemvtwcZWWmsXWGvTrbXpKSmNPldERCTgfBIxgSMkMDv031gMeXXJ3FMzm8xWF3hsH8059XElOu0hInLCwWOVVNXWEWoMN9XeDrX1y6PDQ5k3sY/H9uPWEbUxJga4AFjksT2LiPi5x1dsw2KZO6UvnRKjMUCnxGjmXTaQaUM7eWw/bh1RW2vLgTYe26uIiJ/bdrCUhZn7uHZsN2aP787s8d29ti/dmSgi8h3MW5xNbGQYN5/b0+v7UlGLiDTT5zsO82FuIf9zTk9ax0Z4fX8qahGRZqirs/xhcTadEqP58di0FtmnilpEpBne3ZjPloJj3DWpD1HhoS2yTxW1iIibKmtcPLosl4GdErh4UMcW26+KWkTETc9/vpuCkkrundKPkBBPPcmjaSpqERE3HCmr4m8f7uT8fm0Z06Nlr1ZWUYuIuOGJldspr3Fxz+S+Lb5vFbWISBN2FZbxSsZXzBzRmZ5t41t8/ypqEZEmPLw0l8iwEH5xfi9H9q+iFhE5jcw9RSzdcoAbz+pB2/goRzKoqEVETsFay+8XZ9M2PpLZ47s5lkNFLSJyCos3H2DDV8X88sI+xEQ0d0Isz1FRi4g0orq2joeW5tC3fTw/HJ7qaBYVtYhII/6xei9fFZVzz+S+hLbgzS2NUVGLiJykpKKGJz/YzvheyZzV2/mpBVXUIiIn+euHOyipqGHu5H4Y4+zRNKioRUS+ZV9ROS98vofLhqbSv2Mrp+MAKmoRkW95dHkuxsAvJ/Z2OsoJKmoRkQZZecX8c2MBs8d3o0NCtNNxTlBRi4jQcHPLv7NpExvBTWf1cDrOt6ioRUSAldmHyNhdxK3n9yI+KtzpON+iohaRoFfrqmPekmy6J8dy5cguTsf5LypqEQl6b2TuY2fhce6e3JfwUN+rRbcSGWMSjTFvGWNyjDHZxpgx3g4mItISyqpqeXzFNkaktebC/u2cjtMod58y8idgqbX2cmNMBBDjxUwiIi1mwcc7OVxWzTOz0n3i5pbGNFnUxphWwATgWgBrbTVQ7d1YIiLed6CkkgWf7mLqoA4M7dLa6Tin5M6pj+5AIfCCMWaDMeZZY0zsyRsZY+YYYzKNMZmFhYUeDyoi4mmPrcjFVWe5a2LLz4PYHO4UdRgwDPibtXYocBy45+SNrLULrLXp1tr0lBTnH2IiInI6OQeO8ea6PGaNSaNLG98+m+tOUecBedbajIbP36K+uEVE/Na8xTnER4Zx87k9nY7SpCaL2lp7ANhnjOnTsOg8YKtXU4mIeNGn2wv5eFshN5/bi8SYCKfjNMndqz5uBl5puOJjF3Cd9yKJiHiPq87yh8U5pLaOZtbYrk7HcYtbRW2t3QikezeKiIj3vbMhn+z9x3jiyqFEhoU6HcctvncLjoiIl1RUu5i/PJfBqQlcPKiD03HcpqIWkaDx/Oe72V9Syb1TfGPmFnepqEUkKBwuq+JvH+3k/H7tGNW9jdNxmkVFLSJB4YmV26mocXHPZN++uaUxKmoRCXg7C8t4JeMrrhzZmZ5t45yO02wqahEJeA8tySEqLIRbz/OdeRCbQ0UtIgFtze4ilm89yE/P7kFKfKTTcb4TFbWIBCxrLb9fnE37VlFcf2Z3p+N8ZypqEQlY72ftZ9O+Ym6/sDfREf5xc0tjVNQiEpCqal08vCyHvu3j+eGwVKfjfC8qahEJSP9YtZd9RRXcO6UfoSH+c3NLY1TUIhJwSsprePKDHYzvlcyE3v7/fHwVtYgEnD9/uJ1jlTXcO6Wf01E8QkUtIgFlX1E5L32xl8uHpdKvQyun43iEilpEAsrDy3IJCYE7LuzT9MZ+QkUtIgFj075i3ttUwOwzu9M+IcrpOB6johaRgPD1zS3JcRHcdHYPp+N4lIpaRALCiq0HWbO7iFvP701cpLuzDPoHFbWI+L0aVx0PLs2he0osV4zo7HQcj1NRi4jfe33tPnYVHmfu5H6EhwZerQXe30hEgkppZQ1/XLGNkd2SOL9fW6fjeEVgncgRkaDz9Me7OHK8muf9bB7E5tARtYj4rQMllTz72S4uHtyRwZ0TnY7jNSpqEfFb85fnUlcHd00MnJtbGuPWqQ9jzB6gFHABtdbadG+GEhFpytaCY7y1Po/ZZ3ajc1KM03G8qjnnqM+x1h72WhIRkWaYtySbVlHh/M85vZyO4nU69SEifufjbYV8uv0wN5/bk4SYcKfjeJ27RW2B5caYdcaYOY1tYIyZY4zJNMZkFhYWei6hiMg3uOos8xZn0zkpmmvGdHU6Totwt6jHWWuHAZOBnxtjJpy8gbV2gbU23VqbnpLi/w/qFhHf9Pb6PHIOlHLXxL5EhvnvPIjN4VZRW2sLGt4fAt4BRnozlIhIYyqqXcxfnsuQzolMHdTB6TgtpsmiNsbEGmPiv/4YuBD40tvBRERO9uynuzh4rIpfXRS4N7c0xp2rPtoB7zQMShjwqrV2qVdTiYicpLC0iqc+3smF/dsxIi3J6TgtqsmittbuAga3QBYRkVP608ptVNbWcffkvk5HaXG6PE9EfN6OQ2W8tmYfV4/qQo+UOKfjtDgVtYj4vAeX5BAdHsqt5wX+zS2NUVGLiE/L2HWE/2Qf5Kdn96BNXKTTcRyhohYRn1VXZ/nD4mw6JETxk3HdnI7jGBW1iPis97IK2JRXwh0X9iE6IjhubmmMilpEfFJVrYtHluXSr0MrLh3ayek4jtIMLyLik/7+xV7yjlbw8vWDCA0JnptbGqMjahHxOcXl1Tz5wXbO6p3Cmb2SnY7jOBW1iPicJz/YQVlVLXOnBN/NLY1RUYuIT/nqSDl/X7WHy4en0rd9K6fj+AQVtYj4lIeX5RAWEsLtFwT2PIjNoaIWEZ+x4aujvJ+1nxvGd6N9QpTTcXyGilpEfIK19Te3JMdFMOesHk7H8SkqahHxCcu3HmTtnqPcdkFv4iJ15fA3qahFxHE1rjoeXJJDz7ZxzEzv7HQcn6OiFhHHvbbmK3YfPs49k/oSFqpaOplGREQcdayyhj/+ZzujuydxXr+2TsfxSSpqEXHUUx/tpOh4Nb+a0j+o5kFsDhW1iDimoLiC5z7bzQ+GdGRgaoLTcXyWilpEHDN/+TashV9eqJtbTkdFLSKO2FJQwqINeVw3Lo3OSTFOx/FpKmoRaXHWWuYtziEhOpyfndPT6Tg+z3eKOmshPD4AfpdY/z5rodOJRMRLPt5WyGc7DnPzub1IiA53Oo7P843bf7IWwnu3QE1F/ecl++o/Bxg0w7lcIuJxrrr6o+mubWK4ZnRXp+P4BbePqI0xocaYDcaY9z2eYuX9UFPBu65x7LdJ9ctqKuqXi0hAeWvdPnIPlnLXxL5EhPnOL/W+rDmjdCuQ7Y0QtiSPozaO+2quZXr1feyta3tiuYgEjvLqWuYv38bQLolMGdje6Th+w62iNsakAhcBz3ojxEGSaW3KeCXiDxy30cyovo8ddR05iKbgEQkkz366m0OlVfxqSj/d3NIM7h5R/xG4C6g71QbGmDnGmExjTGZhYWGzQsyrnk65jWBgyG7eiLifOgzTq+/j7qprm/V1RMR3HSqt5KmPdzLpjPakpyU5HcevNFnUxpipwCFr7brTbWetXWCtTbfWpqekpDQrRGarC7inZjZ5dcn0NAU8Gf4klUTwiR3Kur1Hm/W1RMQ3/fE/26murePuyZoHsbncuepjHHCJMWYKEAW0Msa8bK39kadC3DmxD3MXVfOv6jNPLIsMCyE5Kpxrnsvg2VnpjO2p0yAi/mr7wVLeWLuPH43qQrfkWKfj+J0mj6ittXOttanW2jTgCuADT5Y0wLShnZh32UA6JUZjgE6J0Tz0w0H8+9Yz6dw6hmtfXMsHOQc9uUsRaUEPLskhJjyUW87r5XQUv+Qb11FTX9bThnb6r+WvzxnNj19Yw5y/r+NPVwzlokEdHEgnIt/Vqp1HWJlziLsm9aFNXKTTcfxSsy5itNZ+ZK2d6q0wjWkdG8Ers0cxtEsiN7+2njcz97Xk7kXke6irq58HsWNCFD8Z183pOH7LL642j48K56WfjGRcz2TufCuLv6/a43QkEXHDe1kFbM4v4ZcT+xAVHup0HL/lF0UNEBMRxrM/TueC/u24759b+NtHO52OJCKnUVnj4uGluZzRsRXThvz3aU1xn98UNUBkWCh/vXoYlwzuyENLc5i/PBdrrdOxRKQRL32xh/ziCu6d0o+QEN3c8n34zIuJ7goPDeHxmUOIiQjlyQ92cLzKxW+m6i4nEV9y9Hg1f/5wB2f3SWGcLq393vyuqAFCQwzzLhtIdEQoz3++m/LqWn5/6UBC9b+2iE944oPtHK+qZe7kfk5HCQh+WdQAxhjum9qfuMgwnvxgBxU1Lh6dPphwTTUv4qi9R47z8uq9zEjvTJ/28U7HCQh+W9RQX9Z3XNiHmIgwHlqaQ3m1iz9fNZTIML26LOKUh5fmEhYSwu0X9HY6SsAIiMPPn57dg/t/cAYrth5k9kuZVFS7nI4kEpTW7T3KvzfvZ86E7rRtFeV0nIAREEUNMGtMGo9cPojPdxzmx8+vobSyxulIIkHF2vqbW1LiI5kzobvTcQJKwBQ1wPT0zjx55TDWf3WUq5/N4OjxaqcjiQSNZVsOsG7vUW47vzexkX59VtXnBFRRA1w0qAMLZg0n50ApVyxYzaHSSqcjiQS86to6HlySQ6+2ccxIT3U6TsAJuKIGOLdvO168dgT7jpYz8+nV5BdXOB1JJKC9mrGXPUfKmTulL2G68srjAnZEx/ZM5h/Xj+JwWRUznlrFnsPHnY4kEpCOVdbwp5XbGdO9Def0aet0nIAUsEUNMLxra167YTQVNS6mP72KbQdLnY4kEnD+9tFOjpbXcK/mQfSagC5qgAGdEnhjzmgMMPPpVWzOK3E6kkjAyC+u4LnPdnPp0E4MTE1wOk7ACviiBujVLp43bxpDTEQYVz2zmsw9RU5HEgkI85flAnDHhbq5xZuCoqgBuraJ5c2bxpASH8k1z63hs+2HnY4k4te+zC/hnY35XDcujdTWMU7HCWhBU9QAHROjeePGMXRtE8NPXlzLf7ZqHkaR7+Lrm1sSo8P52dk9nY4T8IKqqAFS4iN5fc5o+nWI56aX1/HepgKnI4n4nY9yC/li5xFuOa8XCdHhTscJeEFX1ACJMRG8PHsUw7q25pbXN7BwreZhFHFXrauOeUuySWsTw9WjujodJygEZVFDwzyM141kfK8U7no7ixc/3+10JBG/8Na6PLYdLOPuSX2JCAvaCmlRQT3K0RGhPDNrOBPPaMfv3tvKXz7c4XQkEZ92vKqW+Su2MbxrayYNaO90nKAR1EUN9fMw/uWqYUwb0pFHluXyyLIczcMocgrPfLqLwtIq7p3SVze3tKAmH3FljIkCPgEiG7Z/y1r7W28Ha0lhoSE8NmMI0RFh/OXDnRyvcnHf1P6akFOC1rsb8nlkWS4FxRV0TIzmzol9GNujDQs+2cXkAe0Z3jXJ6YhBxZ1nEVYB51pry4wx4cBnxpgl1trVXs7WokJCDH+4dACxEaE8+1n9PIzzLhukeRgl6Ly7IZ+5izZTUVM/AUd+cQVzF21mSOdEqmvruHtSX4cTBp8mi9rWnwcoa/g0vOEtIM8NGGP41UX9iI0M408rt1Ne7eLxmUM0D6MElUeW5Z4o6a9V1LhYtesI145NIy051qFkwcutp3sbY0KBdUBP4C/W2oxGtpkDzAHo0qWLJzO2KGMMt13Qm9jIUP6wOIfKGhd/vmoYUeGah1GCQ8FpHgt8y3m9WjCJfM2tQ0VrrctaOwRIBUYaYwY0ss0Ca226tTY9JSXFwzFb3pwJPXhg2gD+k32I619aS3l1rdORRFpEx8ToRpe3igojKTaihdMINPOqD2ttMfARMMkbYXzNNaO7Mn/6YFbtPMKs59ZwTPMwShC4c2Ifok/6DdIAv5na35lA0nRRG2NSjDGJDR9HA+cDOV7O5TN+ODyVv1w1jE15xVz1zGqKNA+jBLhpQzvx9xF7WR11K4+F/QWA/+lVxPT0zg4nC17uHFF3AD40xmQBa4EV1tr3vRvLt0we2IEF16Sz/WAZM59exaFjmodRAljWQkZs/i2JtphHa2cywOzmtv13QdZCp5MFrSaL2lqbZa0daq0dZK0dYK29vyWC+Zpz+rblhetGkF9cwfSnV5F3tNzpSCLesfJ+6qoreax2OgUkc2/YK4TUlsPKoPzR9wm67qwZxvZI5uXZozh6vJoZT61iV2FZ039IxM/sO1rFVTX3ssA1lctCPmFs6Nb6FSV5zgYLYirqZhrWpTWvzRlNVW0dM55eTc6BY05HEvEIay2vZOxlUs1DfFnXjYfCFjA//Kn/v0FCqnPhgpyK+js4o2MCb9w4htAQuGLBarLyip2OJPK9FBRXMOv5NfzqnS8Z2i6MZbG/ZWbYR5x4nEd4NJx3n6MZg5mK+jvq2TaON28cS3xUGFc9k8Ga3ZqHUfyPtZY3M/cx8fFPyNxzlAemDeAft15Mpx/8DhI6A6b+/cVPwKAZTscNWsYbT4pLT0+3mZmZHv+6vmh/SQVXP5tBQXEFz8xKZ3wv/7/ZR4LDoWOVzF20mZU5hxiZlsQj0wfRtY1uD3eKMWadtTa9sXU6ov6eOiREs/DGMXRLjuP6FzNZvuWA05FETstayz835nPB45/w2Y7D/Pqifrw+Z7RK2oepqD0gOS6S128YTf+OrfjpK+v558Z8pyOJNOpwWRU/e2U9t76+kW7JsSy+dTyzx3fXI319nFsPZZKmJcSE8/LsUcx+aS2/eGMjFdUurhjpvw+nksCzZPN+fv3ul5RW1nLXpD7MGd+dMD0Z0i+oqD0oLjKMF68byU0vr+OeRZs5Xu3i+jO7OR1LglxxeTX3/XML/9pUwIBOrXh1+hD6tI93OpY0g4raw6LCQ1lwTTq3vr6BB97fSkV1LT8/p6emLRJHrMw+yD2LNnP0eDW3nd+bn53TQ89X90Mqai+ICAvhySuHctfbWTy6fBtlVS7untRHZS0t5lhlDfe/t5W31uXRt308L1w7ggGdEpyOJd+RitpLwkJDePTywUSHh/LUxzspr67ldxefoRdtxOs+2VbI3W9ncfBYJT8/pwe3nNeLyDBNfOHPVNReFBJi+L9pA4iNDGPBJ7sor3bx4GUD9QKOeEVZVS1/WJzNqxlf0SMllkU/G8eQzolOxxIPUFF7mTGGuZP7EhsRxuP/2UZFwzyMEWEqa/GcVTuPcOdbm8gvruCG8d2448I+mj4ugKioW4AxhlvP70VMRCi/X5xNRY2Lv16teRjl+6uodvHQ0hxe/GIPXdvEsPDGMYxIS3I6lniYiroF3TChOzGRofz63S+57oW1PPvjdGIj9U8g3826vUX88s0sdh8+zo/HdOXuyX2JidD3UyDSv2oLu3pUV2IiQvnlm1lc81wGL1w3koTocKdjiR+prHHx+IptPPPpLjokRPPqDaMY2yPZ6VjiRSpqB1w6NJXo8DBufm09Vy5YzT+uH0mbuEinY4kfyMor5o6Fm9h+qIwrR3bhVxf1I06/lQU8vaLlkEkD2vPMrHR2FpYxc8FqDpRoHkY5teraOuYvz+XSv35BaWUtL/1kJPMuG6iSDhIqaged3actL/1kJPuLK5jx9Cr2FWkeRvlvWwuO8YO/fM6TH+xg2pBOLLttAmf11uN0g4mK2mGju7fhlRtGU1JRw4ynV7FT8zBKgxpXHU+s3M4lf/6MwtIqnpmVzvwZg/WaRhBSUfuAIZ0TeX3OaGpcdcx8ehXZ+zUPY7DbdrCUy/76BY+t2MbkgR1YcdsELujfzulY4hAVtY/o16EVb9w4hvDQEK5YsJqN+4qdjiQOcNVZnvp4J1Of+Iz84gr+evUwnrxyKK1jI5yOJg5SUfuQHilxLLxxDAnR4Vz9zGoydh1xOpK0oF2FZUx/6gseXJLDOX1TWH7bBKYM7OB0LPEBTRa1MaazMeZDY0y2MWaLMebWlggWrDonxfDmTWPokBjNj19Yw8fbChvd7t0N+Yx78AO63fNvxj34Ae9u0Kwy/qquzvL8Z7uZ8sSn7DhUxh9nDuGpHw0nWZdsSgN3jqhrgTustf2A0cDPjTH9vRsruLVrFcUbc0bTPTmO2S+tZemX356H8d0N+cxdtJn84goskF9cwdxFm1XWfuirI+Vc+cxq7n9/K2O6t2HF7WcxbWgnPRJXvqXJorbW7rfWrm/4uBTIBjp5O1iwaxMXyWtzRjOgUwI/f3U972zIO7HukWW5VNS4vrV9RY2LR5bltnRM+Y6stby8ei+T/vQJWwqO8fAPB/H8tSNo1yrK6Wjig5p1tbwxJg0YCmQ0sm4OMAegSxfNFegJCdHhvHz9KGa/lMntCzdRUV3HVaO6UFBc0ej2p1ouvqWguIK7387i0+2HObNnMg9dPohOidFOxxIf5nZRG2PigLeBX1hr/+v6MWvtAmABQHp6uvVYwiAXGxnGC9eN4GevrOfedzZTXl1Lx8Ro8hsp5Y76Yfdp1lreXJfHA+9txWUtD0wbwI9GddFpDmmSW0VtjAmnvqRfsdYu8m4kOVlUeChP/Wg4t72xkf/7dzaTBrTnSFkVlbV1J7aJDg/lzol9HEwpp3PoWCVzF21mZc4hRnZL4tHLB9OlTYzTscRPNFnUpv6/++eAbGvtY96PJI2JCAvhT1cMISo8lLfX53Fxx1LuPfJr2lHIIZPCvmF3MmLoJKdjykmstfxrUwH3/XMLlTUu7pvan2vHpmlKNmkWd46oxwHXAJuNMRsblt1rrV3stVTSqLDQEB65fBCxpbv5+/Z4WoVezANhL9CeQtpv/i2ktYZBM5yOKQ0Ol1Xx63e+ZOmWAwztksj86YPpnhLndCzxQ00WtbX2M0D//fuIkBDD/5b8mpjQcTzluoS1dX0YG7KF0a5sRq54lCQVtU9Ysnk/v3r3S8oqa7lncl9uGN+dUB1Fy3ekZyT6IXMsj7vDXqebOcC/6sbyuuscXnRNgkLo/fjHjOrWhlHdkxjZLYm28brcqyUdPV7Nb/+1hX9tKmBgpwTmzxhM73bxTscSP6ei9kcJqZiSfcwM+4iZfES1DWWz7c7qiNGsSZjFovV5/GP1XgC6J8cyqnvSifLukKArQ7zlP1sPMvedzRw9Xs3tF/Tmp2f3IFwzzosHqKj90Xn3wXu3QE39JXoRxsXwiDyGXzyBnw8aSa2rji0Fx8jYfYSMXUW8n7Wf19bsA6BLUgyjuiUxqnsbRnVLonOSrjz4vkoqarj/va28vT6Pvu3jefG6EZzRMcHpWBJAjLWev+Q5PT3dZmZmevzryjdkLYSV90NJHiSk1pf3Kc5Pu+os2fuPkbG7iIxdR1izp4ji8hoAOiVGM7Jb0onyTmsTo+t6m+HjbYXc83YWh0qr+OlZPbjlvF5EhOkoWprPGLPOWpve6DoVdfCpq7NsP1R24og7Y/cRDpdVA9A2PvLE0fbo7kn0SIlTcTeirKqW3/87m9fWfEXPtnHMnz6YwZ0TnY4lfkxFLadlrWVn4fFvFffBY1UAtImN+NYRd5928UF/DfAXOw9z11tZ5BdXMGd8d267oDdR4aFOxxI/d7qi1jlqwRhDz7Zx9Gwbx9WjumKtZe+R8m8UdxFLGp7glxgTzoi0pIYj7jb069AqaC47K6+u5eGlubz4xR7S2sTw1k1jGN41yelYEgRU1PJfjDGkJceSlhzLzBH1D9jKO1p+4mg7Y3cRK7YeBCA+Moz0tNYnTpcM6JQQkFc6ZO4p4pdvbmLPkXKuHZvG3ZP6Eh2ho2hpGSpqcUtq6xhSh8fww+GpABwoqSRj9xFWN5T3h7n1ExzERIQyvGtrRjcU96DURL9+ca2yxsVjK7bxzKe76JQYzWs3jGZMjzZOx5Igo3PU4hGHSitZu/voidMluQdLAYgMC2FYl9YnruUe2iXRb87nbtpXzB1vbmLHoTKuGtWFe6f0Iy5SxzbiHXoxUVpc0fFq1uwuOlHc2QeOYS1EhIYwpHPiieIe1jWRmAjfKr/q2jqeWLmdv328k7bxkTz0w0FM6J3idCwJcCpqcVxJeQ1r9xSxZk/9tdxfFhzDVWcJCzEMSk1gZMOdk+ldWxMfFe5Yzi0FJdyxcBM5B0q5fHgqv5nan4Ro5/JI8FBRi88pq6olc0/RiZtwsvJKqK2zhBgY0Cmh/nLAbm0YkZZEQoz3i7LGVcffPtrJEyu30zo2gnmXDuT8/u28vl+Rr6moxeeVV9eyfm/xiVMlG/cVU+2qwxjo277ViRtwRnZrQ1JshEf3ve1gKXcs3MTm/BIuGdyR/73kDFp7eB8iTVFRi9+prHGxcV/xiUsC1391lMqa+hltereL88gTAl11lmc+3cVjy7cRFxXG76cNYPLADp78a4i4TUUtfq+6to6svGIydhexetcR1u09Snl1/Uzs3VNiT5wqcfcJgbsKy7jjzU1s+KqYSWe05/8uHUByXKS3/xoip6SiloBT46rjy/wSMnYXsWZ3EWt3F1FaVQuc4gmBDQ+xqivO54XwmTxcMZWoiAju/8EZXDK4o55nIo5TUUvA+/oJgat3HTlR3iUVDU8IjKljZHUG6Wzhn65xrLH9ODd0E/Mu7kW70dMdTi5ST0UtQaeuzpJ7sJSMXUfIWP4Ga6q6cIQE4innN2H/YHrox5jEznDbl05HFQH0UCYJQiEhhn4dWtGvQyuuXfEgNtKy03YkyZSSZOrvmqQkz9mQIm7y34cwiLgrIRVjoGdIwf8v6YblIv5ARS2B77z7IPykK0HCo+uXi/gBFbUEvkEz4OInIKEzYOrfX/zEKacuE/E1OkctwWHQDBWz+K0mi9oY8zwwFThkrR3g/UginvfuhnweWZZLQXEFHROjuXNiH6YN7eR0LBG3uHPq40VgkpdziHjNuxvymbtoM/nFFVggv7iCuYs28+6GfKejibilyaK21n4CFLVAFhGveGRZLhU1rm8tq6hx8ciyXIcSiTSPx15MNMbMMcZkGmMyCwsLPfVlRb63guKKZi0X8TUeK2pr7QJrbbq1Nj0lRbNhiO/omNj4Q5pOtVzE1+jyPAl4d07sQ/RJ8zRGh4dy58Q+DiUSaR5dnicB7+urO3TVh/grdy7Pew04G0g2xuQBv7XWPuftYCKeNG1oJxWz+K0mi9pae2VLBBERkcbpHLWIiI9TUYuI+DgVtYiIj1NRi4j4OK9MxWWMKQT2fsc/ngwc9mAcT1Gu5lGu5lGu5gnEXF2ttY3eLeiVov4+jDGZp5o3zEnK1TzK1TzK1TzBlkunPkREfJyKWkTEx/liUS9wOsApKFfzKFfzKFfzBFUunztHLSIi3+aLR9QiIvINKmoRER/nSFEbY543xhwyxnx5ivXGGPOEMWaHMSbLGDPMR3KdbYwpMcZsbHi7r4VydTbGfGiMyTbGbDHG3NrINi0+Zm7mavExM8ZEGWPWGGM2NeT630a2cWK83MnlyPdYw75DjTEbjDHvN7LOkZ9JN3I59TO5xxizuWGfmY2s9+x4WWtb/A2YAAwDvjzF+inAEsAAo4EMH8l1NvC+A+PVARjW8HE8sA3o7/SYuZmrxcesYQziGj4OBzKA0T4wXu7kcuR7rGHftwOvNrZ/p34m3cjl1M/kHiD5NOs9Ol6OHFHbpifM/QHwd1tvNZBojOngA7kcYa3db61d3/BxKZANnPxw5RYfMzdztbiGMShr+DS84e3kV82dGC93cjnCGJMKXAQ8e4pNHPmZdCOXr/LoePnqOepOwL5vfJ6HDxRAgzENv7ouMcac0dI7N8akAUOpPxr7JkfH7DS5wIExa/h1eSNwCFhhrfWJ8XIjFzjzPfZH4C6g7hTrnfr++iOnzwXOjJcFlhtj1hlj5jSy3qPj5atFbRpZ5gtHHuupvx9/MPAk8G5L7twYEwe8DfzCWnvs5NWN/JEWGbMmcjkyZtZal7V2CJAKjDTGDDhpE0fGy41cLT5expipwCFr7brTbdbIMq+Ol5u5nPqZHGetHQZMBn5ujJlw0nqPjpevFnUe0Pkbn6cCBQ5lOcFae+zrX12ttYuBcGNMckvs2xgTTn0ZvmKtXdTIJo6MWVO5nByzhn0WAx8Bk05a5ej32KlyOTRe44BLjDF7gNeBc40xL5+0jRPj1WQup76/rLUFDe8PAe8AI0/axKPj5atF/S9gVsMrp6OBEmvtfqdDGWPaG2NMw8cjqR+/Iy2wXwM8B2Rbax87xWYtPmbu5HJizIwxKcaYxIaPo4HzgZyTNnNivJrM5cR4WWvnWmtTrbVpwBXAB9baH520WYuPlzu5HPr+ijXGxH/9MXAhcPKVYh4dL0dmITeNTJhL/QsrWGufAhZT/6rpDqAcuM5Hcl0O/NQYUwtUAFfYhpd4vWwccA2wueH8JsC9QJdvZHNizNzJ5cSYdQBeMsaEUv+Du9Ba+74x5qZv5HJivNzJ5dT32H/xgfFyJ5cT49UOeKfh/4cw4FVr7VJvjpduIRcR8XG+eupDREQaqKhFRHycilpExMepqEVEfJyKWkTEx6moRUR8nIpaRMTH/T++Qm8lNW82QQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = LinearRegression().fit(X2, y)\n",
    "yfit = model.predict(X2)\n",
    "plt.scatter(x, y)\n",
    "plt.plot(x, yfit)\n",
    "plt.scatter(x, yfit)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "342ecbd3",
   "metadata": {},
   "source": [
    "Идея ``улучшения модели``  ``без изменения модели``, а **путем преобразования входных данных** является базовой для многих более продвинутых методов машинного обучения.\n",
    "//я такое уже делал оказывается 😊, сам того не подозревая, для аналитики//\n",
    "\n",
    "В общем случае **это путь к набору обладающих огромными возможностями методик**, известных под названием ``ядерные методы (kernel methods)``.\n",
    "\n",
    "//ну оно и понятно"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3503d2b1",
   "metadata": {},
   "source": [
    "## Внесение отсутствующих данных\n",
    "\n",
    "Еще одна часто встречающаяся задача в проектировании признаков - обработка отсутствующих данных. (которые NaN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "7534e419",
   "metadata": {},
   "outputs": [],
   "source": [
    "from numpy import nan\n",
    "X = np.array([[ nan, 0,   3  ],\n",
    "              [ 3,   7,   9  ],\n",
    "              [ 3,   5,   2  ],\n",
    "              [ 4,   nan, 6  ],\n",
    "              [ 8,   8,   1  ]])\n",
    "y = np.array([14, 16, -1,  8, -5])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40b24f8d",
   "metadata": {},
   "source": [
    "При использовании для подобных данных типичной модели машинного обучения ***необходимо сначала заменить отсутствующие данные каким-либо подходящим значением**.\n",
    "\n",
    "Это действие называется ``заполнением/вменением (imputation)`` пропущеных значений.\n",
    "\n",
    "Для реализации простого подхода на основе среднего значения/медианы/моды SKL предлагает класс ``SimpleImputer``"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "efce503e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[4.5, 0. , 3. ],\n",
       "       [3. , 7. , 9. ],\n",
       "       [3. , 5. , 2. ],\n",
       "       [4. , 5. , 6. ],\n",
       "       [8. , 8. , 1. ]])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.impute import SimpleImputer\n",
    "imp = SimpleImputer(strategy='mean')\n",
    "X2 = imp.fit_transform(X)\n",
    "X2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "48f46397",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([13.14869292, 14.3784627 , -1.15539732, 10.96606197, -5.33782027])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# а эти данные уже можно передать оценивателю\n",
    "model = LinearRegression().fit(X2, y)\n",
    "model.predict(X2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d427a3b5",
   "metadata": {},
   "source": [
    "## Конвейеры признаков\n",
    "\n",
    "Во всех предыдущих примерах может быстро надоесть выполнять преобразования вручную, особенно если нужно связать цепочкой несколько шагов. Например, нам может понадобиться следующий конвеер обработки:\n",
    "\n",
    "1. Внести вместо отсутствующих данных средние значения.\n",
    "2. Преобразовать признаки в квадратичные.\n",
    "3. Обучить модель линейной регрессии.\n",
    "\n",
    "Для организации потоковой обработки подобного конвейера библиотека SKL предоставляет объект конвейера, который можно использовать следующим образом:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "c2f10d6f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import make_pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "5ad9cac1",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = make_pipeline(SimpleImputer(strategy='mean'),\n",
    "                      PolynomialFeatures(degree=2),\n",
    "                      LinearRegression())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69b4d160",
   "metadata": {},
   "source": [
    "Этот конвейер выглядит и функционирует аналогично обычному объекту SKL, и выполняет все заданные шаги для любых входных данных."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "98e89f31",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[14 16 -1  8 -5]\n",
      "[14. 16. -1.  8. -5.]\n"
     ]
    }
   ],
   "source": [
    "# массив Х с пропущенными значениями\n",
    "model.fit(X, y)\n",
    "print(y)\n",
    "print(model.predict(X))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb59fb9b",
   "metadata": {},
   "source": [
    "Все шаги этой модели выполняются автоматически. Для простоты демонстрации мы применили модель к тем данным, на которых она была обучена, именно поэтому она смогла хорошо предсказать результат."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a31cf8f8",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}