probabilisitc_matrix_factorization_vi.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Probabilistc Matrix Factorization model with mean-field variational inference - pyTorch implementation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We train a Probabilistic Matrix Factorization model [1] using mean-field variational inference with automatic differentiation [2] in pyTorch. Our implementation works on dense representation and does not make a use of data sparsity but performs minibatching over data (matrix) rows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Mnih, Andriy, and Russ R. Salakhutdinov. \"Probabilistic matrix factorization.\" Advances in neural information processing systems. 2008.\n",
    "2. Kucukelbir, Alp, et al. \"Automatic differentiation variational inference.\" The Journal of Machine Learning Research 18.1 (2017): 430-474."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "import torch\n",
    "from torch.distributions import Normal\n",
    "from torch.nn.functional import softplus\n",
    "\n",
    "import logging\n",
    "logger = logging.getLogger(__name__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data minibatches"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create minibatches we split data along the first dimension:\n",
    "each batch consists of subset of rows.\n",
    "Apart from the data matrix we pass an associated data mask\n",
    "marking non-empty (and included in the training subset) cells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MaskedMatrix(torch.utils.data.Dataset):\n",
    "    \"\"\" Splits masked data along the first (rows) dimension. \"\"\"\n",
    "    \n",
    "    def __init__(self, x, mask):\n",
    "        self.x = x\n",
    "        self.mask = mask        \n",
    "\n",
    "    def __len__(self):\n",
    "        return self.x.shape[0]\n",
    "\n",
    "    def __getitem__(self, idx):        \n",
    "        if torch.is_tensor(idx):\n",
    "            idx = idx.tolist()        \n",
    "        return {\"rows\": idx, \"x\": self.x[idx], \"mask\": self.mask[idx]}     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we provide a complete implementation of mean-field automatic VI for PMF. Read through comments to find about details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class PMF_MFVI:\n",
    "    \"\"\" Toy implementation of Probabilistc Matrix Factorization \n",
    "        training using mean-field automatic VI \n",
    "        with minibatching over data rows.\n",
    "        \n",
    "        Probabilistc Matrix Factorization model: \n",
    "        X ~ N(ZW, sigma), W ~ prior_w(.), Z ~ prior_z(.)\n",
    "        where Y has shape (N, D), Z has shape (N, K) and W has shape(K, D).    \n",
    "        K is a latent dimension - size of latent representation.\n",
    "        By default we use \n",
    "        prior_z=Normal(0, 1.), prior_w = Normal(0, 1.) and sigma=1.\n",
    "        \n",
    "        We assume approximate (factorized) distributions q:\n",
    "        qz_ij = Normal(qz_ij_loc, qz_ij_scale)\n",
    "        qw_ij = Normal(qw_ij_loc, qw_ij_scale)\n",
    "        Internally we store 4 matrices: \n",
    "         - qz_loc, qz_scale - describing approximate posterior of Z\n",
    "         - qw_loc, qw_scale - describing approximate posterior of W        \n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self, N=1, K=1, D=1,  \n",
    "                 optimizer_class=torch.optim.Adam, lr=0.005,\n",
    "                 prior_z=Normal(0, 1.), prior_w = Normal(0, 1.), \n",
    "                 likelihood_class=lambda xhat: Normal(xhat, 1.)):        \n",
    "        self.prior_z = prior_z\n",
    "        self.prior_w = prior_w\n",
    "        self.likelihood_class = likelihood_class\n",
    "        \n",
    "        self.optimizer_class = optimizer_class\n",
    "        self.lr = lr\n",
    "        \n",
    "        self.N = N\n",
    "        self.K = K\n",
    "        self.D = D        \n",
    "        self.restart()\n",
    "\n",
    "    def restart(self):\n",
    "        \"\"\" Random initialization of approximate posteriors. \"\"\"\n",
    "        N, K, D = self.N, self.K, self.D # just for brevity\n",
    "        \n",
    "        self.qz_loc = torch.randn([N, K], requires_grad=True)\n",
    "        self.qz_scale = torch.randn([N, K], requires_grad=True)\n",
    "        self.qw_loc = torch.randn([K, D], requires_grad=True)\n",
    "        self.qw_scale = torch.randn([K, D], requires_grad=True)          \n",
    "        \n",
    "        trained_params = [self.qw_loc, self.qw_scale, self.qz_loc, self.qz_scale]\n",
    "        self.optimizer = self.optimizer_class(trained_params, lr=self.lr)\n",
    "        self._iteration = 0\n",
    "    \n",
    "    def sample_prior(self, nsamples_latent=1, nsamples_x=1):\n",
    "        \"\"\" Returns a sample (w, z, x) from prior distribution.\n",
    "        \n",
    "            Args:\n",
    "                nsamples_latent (int): How many samples of Z and W.\n",
    "                nsamples_x (int): How many samples of X for each latent sample.\n",
    "        \"\"\"        \n",
    "        # sample (r stands for reparametrized) latent\n",
    "        z = self.prior_w.rsample([nsamples_latent, self.N, self.K])\n",
    "        w = self.prior_z.rsample([nsamples_latent, self.K, self.D])\n",
    "        xhat = z.matmul(w)\n",
    "        \n",
    "        # sample (r stands for reparametrized) observed\n",
    "        lik = self.likelihood_class(xhat)\n",
    "        x = lik.rsample([nsamples_x])\n",
    "       \n",
    "        if nsamples_latent==1 and nsamples_x==1:\n",
    "            w, z, x = w[0], z[0], x[0,0]        \n",
    "        return w, z, x  \n",
    "\n",
    "    @property\n",
    "    def qw(self):\n",
    "        \"\"\" Retrieves approximating distribution of latent variable W. \"\"\"\n",
    "        return Normal(self.qw_loc, softplus(self.qw_scale))\n",
    "    \n",
    "    @property\n",
    "    def qz(self):\n",
    "        \"\"\" Retrieves approximating distribution of latent variable Z. \"\"\"\n",
    "        return Normal(self.qz_loc, softplus(self.qz_scale))            \n",
    "    \n",
    "    def sample(self, nsamples_latent=1, nsamples_x=1):  \n",
    "        \"\"\" Returns a tensor with samples from approximate predictive distribution.  \n",
    "            Samples are reparametrized, e.g., are differentiable w.r.t \n",
    "            approximate distribution parameters qz_loc, qz_scale, qw_loc and qw_scale.\n",
    "            \n",
    "            Args:\n",
    "                nsamples_latent (int): How many samples of Z and W.\n",
    "                nsamples_x (int): How many samples of X for each latent sample.\n",
    "        \"\"\"\n",
    "        # sample (r stands for reparametrized) latent\n",
    "        w = self.qw.rsample([nsamples_latent])\n",
    "        z = self.qz.rsample([nsamples_latent])\n",
    "        xhat = z.matmul(w)\n",
    "        \n",
    "        # sample (r stands for reparametrized) observed        \n",
    "        lik = self.likelihood_class(xhat)\n",
    "        x = lik.rsample([nsamples_x])\n",
    "\n",
    "        if nsamples_latent==1 and nsamples_x==1:\n",
    "            w, z, x = w[0], z[0], x[0,0]        \n",
    "        return x\n",
    "            \n",
    "    def model_log_prob(self, x, w, z, mask=None):\n",
    "        \"\"\" Joint log-probability of data x and parameters (w, z).\n",
    "        \n",
    "            Returns:\n",
    "                log(likelihood(x|w,z) * prior_w(w) * prior_z(z))                \n",
    "\n",
    "            Args:\n",
    "                mask (tensor): Boolean mask of shape like x, \n",
    "                               selecting non-empty cells.\n",
    "                               Default: None (use all data).\n",
    "        \"\"\"\n",
    "        if mask is None: \n",
    "            mask = torch.ones_like(x).type(torch.bool).to(x.device)\n",
    "        # since we are using mininbatches, the proprtion between data likelihood and \n",
    "        # priors need to be weighted accordingly (=like we used a whole data)        \n",
    "        minibatch_scaling = self.N/x.shape[0]  \n",
    "        \n",
    "        xhat = z.matmul(w)\n",
    "        likelihood = self.likelihood_class(xhat)\n",
    "        assert likelihood.loc.shape[1: ] == x.shape\n",
    "\n",
    "        # note rescaling of data likelihood and qz (the factor depending on number of rows)\n",
    "        return torch.masked_select(likelihood.log_prob(x), mask).sum()*minibatch_scaling \\\n",
    "                + self.prior_w.log_prob(w).sum() \\\n",
    "                + self.prior_z.log_prob(z).sum()*minibatch_scaling\n",
    "    \n",
    "    def elbo(self, x_minibatch, mask_minibatch, qw, qz,\n",
    "             latent_variables_nsamples=16): \n",
    "        \"\"\" Returns a MC estimate of ELBO \n",
    "            for given minibatch and approximations qw and qz. \"\"\"\n",
    "        \n",
    "        # reparametrized sampling from the approximation\n",
    "        w = qw.rsample([latent_variables_nsamples]) \n",
    "        z = qz.rsample([latent_variables_nsamples])\n",
    "\n",
    "        minibatch_scaling = self.N/x_minibatch.shape[0]                     \n",
    "        # MC estimate of the entropy term\n",
    "        entropy = -qw.log_prob(w).sum() -qz.log_prob(z).sum()*minibatch_scaling \n",
    "        # data and parameters joint log-probability\n",
    "        log_prob = self.model_log_prob(x_minibatch, w, z, mask_minibatch).sum()\n",
    "        # MC estimate of elbo\n",
    "        elbo = log_prob + entropy\n",
    "        elbo = elbo/latent_variables_nsamples\n",
    "        \n",
    "        return elbo   \n",
    "    \n",
    "    def fit(self, loader, nepochs=1, latent_variables_nsamples=16):\n",
    "        \"\"\" Runs requested number of epochs of data fitting. \"\"\"\n",
    "        \n",
    "        # check if dimensions match\n",
    "        if len(loader.dataset)!=self.N:\n",
    "            logger.warning(\"Reinitializing fit due to data length change!\")\n",
    "            self.N = len(loader.dataset)\n",
    "            self.restart()\n",
    "                    \n",
    "        for e in range(nepochs):\n",
    "          for minibatch in loader:  \n",
    "            \n",
    "            # check if dimensions match\n",
    "            assert minibatch[\"x\"].shape == minibatch[\"mask\"].shape                                    \n",
    "            if minibatch[\"x\"].shape[1]!=self.D:\n",
    "                logger.warning(\"Reinitializing fit due to data dimensionality change!\")\n",
    "                self.D = minibatch[\"x\"].shape[1]\n",
    "                self.restart()\n",
    "                \n",
    "            self._iteration += 1\n",
    "            self.optimizer.zero_grad()                                        \n",
    "\n",
    "            # obtain the current approximation\n",
    "            # (using current values of qz_loc, qz_scale, ...)\n",
    "            qw = Normal(self.qw_loc, softplus(self.qw_scale))\n",
    "            minibatch_rows = minibatch[\"rows\"]\n",
    "            qz = Normal(self.qz_loc[minibatch_rows, :], \n",
    "                        softplus(self.qz_scale[minibatch_rows, :] ))\n",
    "            \n",
    "            # use -ELBO as a loss\n",
    "            loss = -self.elbo(minibatch[\"x\"], minibatch[\"mask\"], \n",
    "                              qw, qz, latent_variables_nsamples)\n",
    "                        \n",
    "            # optimization step\n",
    "            logger.debug(\"%i. iteration: loss=%.3f\" % (self._iteration, loss))              \n",
    "            loss.backward(retain_graph=False)\n",
    "            self.optimizer.step() # update approximation parameters qz_loc, qz_scale, ...\n",
    "            \n",
    "          logger.info(\"%i. epoch / %i. iteration: loss=%.3f\" % \\\n",
    "                    (e+1, self._iteration, loss))                \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data generation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data is generated by sampling from prior distribution.\n",
    "Then, with probability nonmissing_p we choose 'non-empty' cells.\n",
    "Finally, the non-empty cells, according to probability training_p,\n",
    "we randomly split into training and test subset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_data(m=PMF_MFVI(123, 4, 11), \n",
    "                  nonmissing_p=0.05, training_p=0.5):\n",
    "    \"\"\" Samples artificial data from the model. \"\"\"\n",
    "    w_true, z_true, x = m.sample_prior()\n",
    "\n",
    "    # sample non-empty values, training & test mask\n",
    "    mask = np.random.choice([True, False], (m.N, m.D), \n",
    "                            p=[nonmissing_p, 1.-nonmissing_p])    \n",
    "    training_mask = np.random.choice([True, False], (m.N, m.D), \n",
    "                            p=[training_p, 1.-training_p])   \n",
    "    test_mask = ~training_mask & mask\n",
    "    training_mask = training_mask & mask\n",
    "    \n",
    "    test_mask = torch.tensor(test_mask) \n",
    "    training_mask = torch.tensor(training_mask)\n",
    "    \n",
    "    return w_true, z_true, x, training_mask, test_mask"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "    np.random.seed(125)\n",
    "    torch.manual_seed(125)\n",
    "    logging.basicConfig(level=logging.INFO, format='%(relativeCreated)6d %(message)s')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "    w_true, z_true, x, training_mask, test_mask = generate_data()\n",
    "    dataset = MaskedMatrix(x, training_mask)\n",
    "    loader = torch.utils.data.DataLoader(dataset, batch_size=16, shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train and evaluate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Note that for generated data latent dimension was K=4 whereas here we use K=2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "   904 Reinitializing fit due to data length change!\n",
      "   906 Reinitializing fit due to data dimensionality change!\n",
      "   951 1. epoch / 8. iteration: loss=624.236\n",
      "  1014 1. epoch / 16. iteration: loss=761.019\n",
      "  1056 1. epoch / 24. iteration: loss=190.928\n",
      "  1104 1. epoch / 32. iteration: loss=588.584\n",
      "  1143 1. epoch / 40. iteration: loss=875.210\n",
      "  1175 1. epoch / 48. iteration: loss=185.768\n",
      "  1223 1. epoch / 56. iteration: loss=471.640\n",
      "  1264 1. epoch / 64. iteration: loss=215.421\n",
      "  1293 1. epoch / 72. iteration: loss=174.616\n",
      "  1323 1. epoch / 80. iteration: loss=269.268\n",
      "  1359 1. epoch / 88. iteration: loss=478.387\n",
      "  1395 1. epoch / 96. iteration: loss=832.257\n",
      "  1432 1. epoch / 104. iteration: loss=419.271\n",
      "  1466 1. epoch / 112. iteration: loss=294.042\n",
      "  1504 1. epoch / 120. iteration: loss=479.445\n",
      "  1548 1. epoch / 128. iteration: loss=280.792\n",
      "  1587 1. epoch / 136. iteration: loss=547.537\n",
      "  1631 1. epoch / 144. iteration: loss=365.749\n",
      "  1659 1. epoch / 152. iteration: loss=203.954\n",
      "  1701 1. epoch / 160. iteration: loss=324.055\n",
      "  1736 1. epoch / 168. iteration: loss=489.391\n",
      "  1768 1. epoch / 176. iteration: loss=659.209\n",
      "  1805 1. epoch / 184. iteration: loss=412.465\n",
      "  1842 1. epoch / 192. iteration: loss=393.279\n",
      "  1886 1. epoch / 200. iteration: loss=368.519\n",
      "  1930 1. epoch / 208. iteration: loss=451.228\n",
      "  1960 1. epoch / 216. iteration: loss=646.506\n",
      "  1997 1. epoch / 224. iteration: loss=443.929\n",
      "  2031 1. epoch / 232. iteration: loss=604.719\n",
      "  2089 1. epoch / 240. iteration: loss=124.173\n",
      "  2165 1. epoch / 248. iteration: loss=236.606\n",
      "  2223 1. epoch / 256. iteration: loss=184.306\n",
      "  2281 1. epoch / 264. iteration: loss=234.921\n",
      "  2339 1. epoch / 272. iteration: loss=417.215\n",
      "  2394 1. epoch / 280. iteration: loss=97.192\n",
      "  2451 1. epoch / 288. iteration: loss=358.701\n",
      "  2490 1. epoch / 296. iteration: loss=323.294\n",
      "  2533 1. epoch / 304. iteration: loss=240.400\n",
      "  2573 1. epoch / 312. iteration: loss=173.196\n",
      "  2633 1. epoch / 320. iteration: loss=202.643\n",
      "  2689 1. epoch / 328. iteration: loss=362.067\n",
      "  2765 1. epoch / 336. iteration: loss=185.725\n",
      "  2838 1. epoch / 344. iteration: loss=363.499\n",
      "  2915 1. epoch / 352. iteration: loss=482.095\n",
      "  2977 1. epoch / 360. iteration: loss=466.933\n",
      "  3030 1. epoch / 368. iteration: loss=85.720\n",
      "  3081 1. epoch / 376. iteration: loss=382.862\n",
      "  3127 1. epoch / 384. iteration: loss=342.272\n",
      "  3184 1. epoch / 392. iteration: loss=225.847\n",
      "  3247 1. epoch / 400. iteration: loss=667.565\n",
      "  3305 1. epoch / 408. iteration: loss=213.312\n",
      "  3363 1. epoch / 416. iteration: loss=253.120\n",
      "  3425 1. epoch / 424. iteration: loss=138.603\n",
      "  3473 1. epoch / 432. iteration: loss=342.023\n",
      "  3530 1. epoch / 440. iteration: loss=323.973\n",
      "  3586 1. epoch / 448. iteration: loss=304.633\n",
      "  3637 1. epoch / 456. iteration: loss=252.636\n",
      "  3682 1. epoch / 464. iteration: loss=169.384\n",
      "  3734 1. epoch / 472. iteration: loss=129.736\n",
      "  3784 1. epoch / 480. iteration: loss=356.967\n",
      "  3839 1. epoch / 488. iteration: loss=348.193\n",
      "  3888 1. epoch / 496. iteration: loss=349.409\n",
      "  3947 1. epoch / 504. iteration: loss=146.337\n",
      "  4009 1. epoch / 512. iteration: loss=201.821\n",
      "  4071 1. epoch / 520. iteration: loss=239.477\n",
      "  4130 1. epoch / 528. iteration: loss=451.567\n",
      "  4180 1. epoch / 536. iteration: loss=331.365\n",
      "  4236 1. epoch / 544. iteration: loss=165.443\n",
      "  4303 1. epoch / 552. iteration: loss=155.487\n",
      "  4362 1. epoch / 560. iteration: loss=191.218\n",
      "  4425 1. epoch / 568. iteration: loss=280.761\n",
      "  4488 1. epoch / 576. iteration: loss=238.270\n",
      "  4533 1. epoch / 584. iteration: loss=138.283\n",
      "  4597 1. epoch / 592. iteration: loss=171.779\n",
      "  4660 1. epoch / 600. iteration: loss=214.442\n",
      "  4722 1. epoch / 608. iteration: loss=181.456\n",
      "  4781 1. epoch / 616. iteration: loss=142.382\n",
      "  4838 1. epoch / 624. iteration: loss=147.901\n",
      "  4907 1. epoch / 632. iteration: loss=437.949\n",
      "  4960 1. epoch / 640. iteration: loss=31.688\n",
      "  5006 1. epoch / 648. iteration: loss=376.605\n",
      "  5076 1. epoch / 656. iteration: loss=205.605\n",
      "  5136 1. epoch / 664. iteration: loss=177.971\n",
      "  5190 1. epoch / 672. iteration: loss=132.118\n",
      "  5252 1. epoch / 680. iteration: loss=339.347\n",
      "  5300 1. epoch / 688. iteration: loss=95.703\n",
      "  5352 1. epoch / 696. iteration: loss=292.001\n",
      "  5412 1. epoch / 704. iteration: loss=447.294\n",
      "  5493 1. epoch / 712. iteration: loss=53.955\n",
      "  5567 1. epoch / 720. iteration: loss=113.164\n",
      "  5615 1. epoch / 728. iteration: loss=171.035\n",
      "  5663 1. epoch / 736. iteration: loss=67.417\n",
      "  5707 1. epoch / 744. iteration: loss=536.333\n",
      "  5761 1. epoch / 752. iteration: loss=55.543\n",
      "  5818 1. epoch / 760. iteration: loss=118.773\n",
      "  5876 1. epoch / 768. iteration: loss=76.536\n",
      "  5923 1. epoch / 776. iteration: loss=98.720\n",
      "  5988 1. epoch / 784. iteration: loss=373.185\n",
      "  6042 1. epoch / 792. iteration: loss=217.584\n",
      "  6093 1. epoch / 800. iteration: loss=172.852\n"
     ]
    }
   ],
   "source": [
    "    m = PMF_MFVI(K=2) # mismatched model\n",
    "    test_rmses = []\n",
    "    for e in range(100):\n",
    "        m.fit(loader)\n",
    "        \n",
    "        # evaluate on the test subset\n",
    "        predictions = m.sample(100).mean([0,1]) # empirical expectations\n",
    "        test_rmse = torch.masked_select((predictions-x)**2, test_mask).mean().sqrt()\n",
    "        test_rmses.append(test_rmse)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot test RMSE vs epoch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9eXxcd33v/f7OPtJolyxZkm15i2Mn8Z7VCSRhS0pLoIVC2oal0JSnQEMLt5fy3JZ7u/deHri05QJhbVgvbQIEGgLZmgU7i+M4TuJ9t2zt+zqjmfk9f5xzRqPRmdFI1kiW9H2/XnpJc86Zmd/oSOdzvrsYY1AURVGUTDzzvQBFURTl0kQFQlEURXFFBUJRFEVxRQVCURRFcUUFQlEURXHFN98LmE2qq6tNU1PTfC9DURRlwfDiiy92GmNq3PYtKoFoampi7969870MRVGUBYOInMm2T11MiqIoiisqEIqiKIorKhCKoiiKKyoQiqIoiisqEIqiKIorKhCKoiiKKyoQiqIoiisqEDaxeJKXzvbwrV+d4ljbwHwvR1EUZd5ZVIVyMyEaT/C+bzzP/nO9jI4lAXjr5uV88Xe2z/PKFEVR5pclLxBBn5fycIDfuWYVO5sq+Nbu05zqGJrvZSmKosw7S14gAL58147Uz8+f6uaHe89hjEFE5nFViqIo84vGIDJYU1PMcCxBx0B0vpeiKIoyr6hAZNBUVQzAyU51MymKsrRRgchgdbUlEKdVIBRFWeKoQGRQXx4m4PVwqksFQlGUpY0KRAZej7CyqkgzmRRFWfKoQLjQVFXMabUgFEVZ4qhAuLCmppgzXcMkk2a+l6IoijJvqEC40FRVTDSepKV/dL6XoiiKMm+oQLjQVF0EoHEIRVGWNCoQLjiprprJpCjKUkYFwoXakhBhv1drIRRFWdKoQLjg8Qirqoo4pQKhKMoSRgUiC6uri9WCUBRlSaMCkYXV1cWc7R4mnkjO91IURVHmhYIJhIisEJEnROSQiLwmIve4HPO7InLA/totIlvS9t0mIkdE5LiIfKpQ68xGU3Ux8aShuWdkrt9aURTlkqCQFkQc+IQxZiNwHfAREdmUccwp4PXGmM3AXwP3AoiIF/gicDuwCbjT5bkFRTOZFEVZ6hRMIIwxLcaYffbPA8AhoCHjmN3GmB774bNAo/3zNcBxY8xJY0wM+AFwR6HW6oZ2dVUUZakzJzEIEWkCtgHP5Tjsg8DP7Z8bgHNp+5rJEJe0175bRPaKyN6Ojo6LX6xNVXGAkqBPM5kURVmyFFwgRCQC3A983BjTn+WYW7AE4r86m1wOc22MZIy51xiz0xizs6amZjaW7KyJFZVFGoNQFGXJUtCZ1CLixxKH7xpjHshyzGbga8Dtxpgue3MzsCLtsEbgQiHX6kZ5kZ/+kbG5fltFUZRLgkJmMQnwdeCQMeZzWY5ZCTwA3GWMOZq26wVgvYisFpEA8B7gwUKtNRtlYT99KhCKoixRCmlB7ALuAl4Rkf32tk8DKwGMMV8G/hKoAv6PpSfEbXdRXEQ+CvwC8ALfMMa8VsC1ulIa8tM/qgKhKMrSpGACYYx5BvdYQvoxHwI+lGXfQ8BDBVha3pQVqQWhKMrSRSupc1Aa8jE6liQaT8z3UhRFUeYcFYgclIX9APSPxOd5JYqiKHOPCkQOSm2ByHQzPXuyi5/sPz8fS1IURZkzVCBy4AhEZqD6a0+f4q9+enA+lqQoijJnqEDkoCyLBdEzHKNrKEbXYHQ+lqUoijInqEDkoDTkxCAyBGIoBsCx9sE5X5OiKMpcoQKRg/Eg9USB6B62BaJtYM7XpCiKMleoQOSgNGyVifSPjmcxJZIm5XJSC0JRlMWMCkQOgj4vIb9nQgyib2QMY7cNPKoWhKIoixgViCkoC/vpGx4XiG47/hAJ+jiuFoSiKIsYFYgpyOzH1GPHH3asqqBzMJYSDEVRlMWGCsQUZHZ0dQThmtWVgAaqFUVZvKhATEFpOMOCsAXiWlsgjqqbSVGURYoKxBRkWhA9djxiU32pFYdQC0JRlEWKCsQUlIZ8E5r19QzHCPk9FAV8rFsW4WibWhCKoixOVCCmoMx2MSWTVm5r91CMiqIAAJfVRrQWQlGURYsKxBSUhv0YA4Mxy4roSROI9ctK6ByMpuISiqIoiwkViClItfy2Yw/dwzEqi22BqI0AWlGtKMriRAViCsoyWn73Do9RkRKIEkArqhVFWZyoQEyB09HVyWTqHopRWWRtqy8LaUW1oiiLloIJhIisEJEnROSQiLwmIve4HHO5iOwRkaiIfDJj32kReUVE9ovI3kKtcyrSO7rGE0n6RsYot2MQImJnMqkFoSjK4sNXwNeOA58wxuwTkRLgRRF5xBiTPoqtG/hj4O1ZXuMWY0xnAdc4JamOriNxem0rwolBAKxbFuHpYx3zsjZFUZRCUjALwhjTYozZZ/88ABwCGjKOaTfGvACMubzEJUH6VDknW6kiTSBqS4N0DsZSabCKoiiLhTmJQYhIE7ANeG4aTzPAL0XkRRG5O8dr3y0ie0Vkb0fH7N/JFwd8eMQKUjtV1JVF4wJRWRwkkTST5lYriqIsdAouECISAe4HPm6M6Z/GU3cZY7YDtwMfEZHXuR1kjLnXGLPTGLOzpqZmFlY8EY9HKLXbbXSnLAh/an+VbU10TVELMRyL59yvKIpyqVFQgRARP5Y4fNcY88B0nmuMuWB/bwd+BFwz+yvMj9KQn/6RsVSr74oJFoT1c6623w++fIGtf/UIHQPRwi5UURRlFilkFpMAXwcOGWM+N83nFtuBbUSkGHgz8OrsrzI/yjItCBeB6Bp0FwhjDF996iSxeJKz3cOFX6yiKMosUcgspl3AXcArIrLf3vZpYCWAMebLIlIH7AVKgaSIfBzYBFQDP7I0Bh/wPWPMwwVca06sfkxxeoZihP1ewgFval9VJLcF8XJzH6+c7wOgc1AtCEVRFg4FEwhjzDOATHFMK9Dosqsf2FKIdc2E0rCP1v5ReobHJqS4wrgF4bifMrlvz2l8HiGeNCoQiqIsKLSSOg8cF1PPcIzyIv+EfUGfl0jQ5+pi6h6K8bMDLbxzh6WBnQPa1E9RlIWDCkQeOEHq7qHYJAsCrKym7qHJ1sEP954jFk/y+zeuprzIT8fg6FwsV1EUZVZQgciD0rCfaDxJa9/ohAC1Q2VxcFKaayJp+O5zZ7h2dSWX1ZZQHQmqBaEoyoJCBSIPnJbfrf2jrhZEVXFgUpD6qWMdnOse4a7rVwFQHQloDEJRlAWFCkQeOO02gCwWxGSB2H+2FxF406ZaAMuCUIFQFGUBoQKRBxMEotg/aX9VcYCuoRjGjPdjutA7wrKSIEGflRJrCYS6mBRFWTioQORBaWg8GzibBRGLJxmOJVLbLvSNUF8eTj2uKQkyGI0zknaMoijKpYwKRB6kWxBuMQi3dhsXekcnCkQkCGixnKIoCwcViDwonSIG4VRTO5lMxhjO947QkCYQ1SXWMR0qEIqiLBBUIPLAGTsK7haEIxpOLUTXUIxYPEl9WSh1TLVjQWjDPkVRFggqEHkQ8HkI+61gc2YlNUBVsXXxd6qpL/SOAExwMaUEQgPViqIsEArZrG9RURb2IwIhv3fSvsqMhn1uAuG4oTQGoSjKQkEFIk9Kwz68Hvfeg8UBLwGfJyUQ53utlhrpMYigz0tZ2K8CoSjKgkEFIk/Kwn4CPnePnIikaiHAsiDCfu8kd1R1JKBDgxRFWTCoQOTJH92yjkTCZN1fWRygxxaIlr4R6stD2PMsUmg1taIoCwkViDy5ZcOynPsr0yyI8xk1EA7VJUEOXpjOWG5FUZT5Q7OYZon0fkwXMmogHGoiQU1zVRRlwaACMUs4AhGNJ+gYiLK8zEUgSoIMROOMjmm7DUVRLn1UIGaJquIAg9E4Z7uGAagvD006plpTXRVFWUCoQMwSlXax3Cvn+wBcXUxOsZxmMimKshAomECIyAoReUJEDonIayJyj8sxl4vIHhGJisgnM/bdJiJHROS4iHyqUOucLZwWHI5AuAaptZpaUZQFRFaBEJE/S/v5XRn7/i6P144DnzDGbASuAz4iIpsyjukG/hj4bMbre4EvArcDm4A7XZ57SeFUSr9qC0RdmYuLqSR3R1dtBa4oyqVELgviPWk//3nGvtumemFjTIsxZp/98wBwCGjIOKbdGPMCMJbx9GuA48aYk8aYGPAD4I6p3nM+cSyI1y70Ux0JurbkSMUgbBfTWCLJ//nP43zgm89zzd8+ysa/fJifHbgwd4tWFEXJQS6BkCw/uz3OiYg0AduA5/J8SgNwLu1xMxnicqlRZQvEcCxBg0uAGqx2G6UhX8qCuP/FZv7nw0c43zvCjeuqWVVVxL88fnzCZDpFUZT5IpdAmCw/uz3OiohEgPuBjxtj8q0ScxMg1/cUkbtFZK+I7O3o6Mh3WbNOacif6tXkFn9wqC6xRo8aY7hvzxkuryvhFx9/HZ9791Y+dut6DrcO8OTR+fsciqIoDrkEYouI9IvIALDZ/tl5fFU+Ly4ifixx+K4x5oFprKsZWJH2uBFw9b0YY+41xuw0xuysqamZxlvMLh6PUGH3XsopEJEgHQNRXjzTw8GWfu66flWqJcfbttRTVxri3qdOzsmaFUVRcpFVIIwxXmNMqTGmxBjjs392Hk8eipCBWFe9rwOHjDGfm+a6XgDWi8hqEQlgxUMenOZrzDlOHCKXQNTY/Zju23OGkpCPt28d95wFfB4+sKuJ3Se6UsFuRVGU+SJXFlORbQE4jzeIyJ+IyDvyfO1dwF3ArSKy3/76NRH5sIh82H7NOhFpBv4U+G8i0iwipcaYOPBR4BdYwe0fGmNem+FnnDMcgcgWgwArUH2+d4Sfv9rCO3c0Uhyc2A7rzmtXUhL08RW1IhRFmWdyNet7GPggcExE1gF7gO8Cvy4i1xpjctYmGGOeYYpgtjGmFct95LbvIeChXM+/1HAmy+W0IEqCRONJAO66btWk/aUhP79z7Uq+9swp/uwtG1hRWVSYxSqKokxBrhhEhTHmmP3z+4DvG2M+hlWb8NaCr2wB4lgQbn2YHJxiuZvWV7OmJuJ6zAd2rcYYw7/tPee6X1EUZS7IN4vpVuARALsuIVnIRS1UrmwoZXV1cSrl1Y3GCssieP8NTVmPqSsLsbOpkkcOtc/2EhVFUfIml0AcEJHPisifAOuAXwKISPmcrGwB8u6rV/LEJ2/Gk2U0KcCudVX89KM38oaNtTlf600baznU0s+57uHZXqaiKEpe5BKIPwA6gSbgzcYY50q1iYzWGEr+iAhXNZZNedwbN1kC8tihtkIvSVEUxZWsQWpjzAjwDy7bdwO7C7koBVZXF7NuWYRHD7Xz/l2r53s5iqIsQbIKhIgcyPVEY8zm2V+Oks4bN9bytadP0jcyRll4ytITRVGUWSWXiykJJIBvA78N/EbGl1Jg3rRpGfGk0dYbiqLMC7kqqbcCdwIR4HvA3wJXAOeNMWfmZnlLm60rKqiOBHj0oMYhFEWZe3IODDLGHDbGfMYYsx34KXAf8CdzsjIFr0e49fJlPHGknbGEZhYrijK35BQIEWkQkU+IyDPA72GJw5fmZGUKYMUhBkbjPH+qe76XoijKEiNXL6YnsawGP/B+rGrq/wACIlI5J6tTuGl9DT6PsOdE13wvRVGUJUauXkyrsKqp/xC4O2272NvXFHBdik044KWuLMT53pH5XoqiKEuMXHUQTXO4DiUH9eVhFQhFUeacnDEIN+y2318txGIUd+rLQlxQgVAUZY7JFYPYLCK/FJFXReRvRKRWRO4HHgMOzt0SlfryMK19oySS2Se9futXp/j2s5p9rCjK7JHLgvgqVv3DbwEdwD7gJLDOGPP5OVibYlNfHiaeNHQORrMe84MXzvGVJ0/M4aoURVns5BKIoDHmW8aYI8aYL2BVVn/KGDM6R2tTbBrsAUS54hCdg1Gae0ZoH9DToyjK7JBLIEIisk1EtovIdmAQ2Jz2WJkjltsjTLPFIeKJJF1DMQD2nemds3UpirK4yZXm2gJ8Lu1xa9pjgzVESJkDnBGm2QSieziGscMTL53t4bYr6+ZqaYqiLGJypbneMpcLUbJTGvJTEvRxodfdfdQ5EEv9vO9sz1wtS1GURc6001yV+WF5efZUVyd4vWVFOQea+4jFF1ffpngiqZP1FGUeKJhAiMgKEXlCRA6JyGsico/LMSIi/yQix0XkQHpsQ0QSIrLf/nqwUOtcKNSXh7nQ5y4QHQOWQLx5Uy3ReJJDLf1zubSCc/++Zt7wuSfpGxmb76UoypKikBZEHPiEMWYjcB3wERHZlHHM7cB6++tuJjYCHDHGbLW/3lbAdS4I6svD2V1MtgXxliusMaWLzc10omOIWFytCEWZa6YUCBF5LJ9tmRhjWowx++yfB4BDQEPGYXcA9xmLZ4FyEVme18qXGA3lYbqHYoyOJSbt6xyMEvJ7WFsTYXlZiH1nF1cmU2ufJYzabkRR5pZcldQhu2trtYhUiEil/dUE1E/nTeznbAOey9jVAJxLe9zMuIiERGSviDwrIm/P8dp328ft7ehYvJPXlpdlT3XtHIxRHQkiImxfWcG+M4vLgnAEQtuNKMrcksuC+EPgReBy+7vz9RPgi/m+gYhEgPuBjxtjMp3j4vIUp5/ESmPMTuB3gP8tImvdXt8Yc68xZqcxZmdNTU2+y1pwjKe6TnYzdQxEqY4EAdi2spzzvSO09y+egrnWfhUIRZkPco0c/YIxZjXwSWPMGmPMavtrizHmX/J5cRHxY4nDd40xD7gc0gysSHvcCFyw39/5fhL4TywLZMnSkKMWonMwSk2JJRDbV1UAiycOYYxJE4jFI3qKshDIJ0jdKiIlACLy30TkgXwqqUVEgK8Dh4wxn8ty2IPAe+1spuuAPmNMi+3SCtqvUw3sYok3CKwtDSGCayZT5+C4BXFFfSkBr4enjnWy+3gnX3v6JL98rXWulztr9AyPpdJ2my8hC+JI6wDfe+7sfC9DUQpKrkpqh78wxvybiNwIvAX4LFa20bVTPG8XcBfwiojst7d9GlgJYIz5MvAQ8GvAcWAY+IB93EbgKyKSxBKxfzDGLGmBCPg81ESCkyyIRNLQPRSjJhIAIOjzcmVDKd977mzqAlYdCfLmKxZmdXWLLYhlYf8l5WL6/vNnuW/Pad599Qq8HjdPqaIsfPIRCCdt5q3Al4wxPxGR/z7Vk4wxz+AeY0g/xgAfcdm+G7gqj7UtKdxSXbuGoiQNKRcTwF/+xhXsPd3NhroSHj3YxrefPUMiaRbkhazNdi9tX1nOE0c6iMYTBH3eeV4V9AzHSBroHY5RFQlO/QRFWYDk42I6LyJfAX4beMh2/WgF9jzQUB6edBfttNmoTrtIbV1RzoduWsNN62tYUxNJXcgWIi12BtMOO7biZDTNN912c0Tnu6IsRvK50P828AvgNmNML1AJ/JeCrkpxpb48xIW+EYwZHxzkFMlVl7jfxVbZrqfOwYV5IWvrG8UjVhsRuHRqIXpswV2ov1dFyYcpBcIYMwy0Azfam+LAsUIuSnFneVmY0bEkPcPjLSecNhvVWdwcVcXW9q4cw4YuZVr6RqkpCbKysgiA8z2XiEAMWeega2hh/l4VJR/yqaT+DPBfgT+3N/mB7xRyUYo7bm2/HQuiJosFUVNiWxAL1BXS2j9KXVmYulSh4KXlYupSC0JZxOTjYnoH8DZgCFL1CSWFXJTijttkOafNRnHAPXDrWBCdAwvzTre1b5S60iBBn5eakslZXPPB6FiCEbvlSdcCFV5FyYd8BCJmZxsZABEpLuySlGw4k+VaJgjEeJsNN8rCfrweWbCukNb+UZaXWcKYq6PtXNKTFvBfqK47RcmHfATih3YWU7mI/AHwKPC1wi5LcaOqOEDQ56E5zQ+f3mbDDY9HqCoOLEhXyFA0zsBonNpSSxgbykOXRJA6PXNpIf5eFSVfpqyDMMZ8VkTeBPQDG4C/NMY8UvCVKZMQES5fXsr+c+PdWjsHo6ywA7jZqIoEU7GKhYTTYsNpVNhQHubxw+0YY7JaTHOBE6AOeD2a5qosavIJUv+jMeYRY8x/McZ80hjziIj841wsTpnMDWur2H+ul6FoHJjYZiMb1ZHAjNIxo/EE395zmmh8covxfPjI9/bxz4/NPOHNqXlwLIj6ciuLa74vyt22i2l1dTGdC9R1pyj5kI+L6U0u226f7YUo+bFrbTXxpOGF092T2mxkozoSnFEM4v4Xz/MXP3mNh1+dfi+neCLJL19r5Z8fPz7jwLIjEI4Fkauj7VzSYwvUutqIupiURU2ueRD/j4i8Amywx4E6X6eAA3O3RCWdHasqCHg97DnRlWqzka1IzqGqOJCquJ4OD+xrBmDPia4J24eice76+nP8/JWWrM891zPCWMIQSyT558dnZkU4Lqa6NBcTzH+xXPdQDBFYW11M38gYY4nFNQN8IZFMGu596oSOoy0QuSyI7wG/gdVx9TfSvnYYY35vDtamuBAOeNm2spzdJ7pSF/2aKVxMVZEgI2MJhmPxvN/nTNcQe8/04PMIuzME4smjHTx9rJN7frCfF053uz7/VOcgYLX9+OHeZk51DuX93g6tfaOUF/kJ+a0U3lwtz3Px+OG2Wb2A9AzHKAv7qbFdXz0ah5g3DrcO8HcPHebRg23zvZRFSa55EH3GmNPGmDuNMWfSvtyvCMqcccPaal690MeJDusiPJUFUW27oKbjDnlg33lE4IM3reZs9/CEedCPHWqnLOynsTLMH9y3l5P2OtI52WEJwv9652YCXg+ff+Ro3u/t0NI3Sp19EQYoL/IT9nunZUF0D8X4/W/t5TvPnpn2+2ejZ3iMyqIA1cULu43JYsBxnfaqBVEQtOneAuSGdVUYAz87cAHI3mbDwdnfkWcmUzJpeOClZnatrea3tjcCsOekZUUkkoYnjrRz84YavvX+a/CK8P5vvjCpHuBk5xDlRX7W15bwgV1NPPjyBQ61ZA4UzE1b/2jKvQRWFld9eWhaFoTTmuNE+2QRmyk9QzEqigOpLq6FCprf9fXnZmXmxEgsMe3f/XQxxkzoETZXODc96mIqDCoQC5AtjeWE/V6eOGzN4M7WZsOhysWCONzaz//6xWHXf+q9Z3o41z3Cb25vYP2yCNWRQCoOsf9cL91DMd6wsZaVVUV87X07ae4Z5l93n57wGic7BllTbdVU/uHr1lIS8vGhf93Lt/ecZnQsv6yoTAsCnJbn+QuEU1h3cgYurmx0D8WoKPJTaVsQhShCjCeSPH2skx+91HzRr/WVp05w+xee5r8/+Fpq+NJs8/Yv/op/fvx4QV47F04le/88CMTzp7r52tMn50UY5woViAVIwOfh6tWVxBLJnG02HBwLIv0u/9/2NvPFJ07Q7tKC44F9zRQFvNx2ZR0iwvVrq/nV8U6MMTx+uA2vR3j9emv+97aVFWyoK+WltNoMsFxMa2oiAJQV+fnae3dSUxLkL37yGrv+4XG+8uSJnBerWDxJ11B0ggUBVhzi/DSymJyq85Mdg7P2j9wzHKOiKJBy3RXCxeQ0ZNx/rpeR2MzSjB1eae4j6PPwrd2neddX9tDcM5zz+HgiydefOZV3zMoYw6GWgYJbKW50Oy6mOWxn3z4wyp/+3/389lf28Df/cYgLl0gL+kKgArFAuWFtFUDONhsOlSlf+bgYHG0bAJgUPB4dS/AfB1q4/crlFAV8qfdqH4hyomOIxw61c3VTBWVF/tRztjSW8cr5vtQFeDAap30gyurq8a4s166p4kd/dAM/uPs6rmgo4+9/fphf+6en2X2i03XN7QOjGMMkC6KhPEznYHRaVghA/2h8QhfcmWKMlVpcWRygNOTH55HURWo2cdxWYwnDi2cubr74kbYB3nxFHV/63e2cbB/k7V/cnfP39+zJbv76Zwf56csX8nr9gWicWCI5Lym/zu9prlxM+8728IbPPsnPDrRw0/pqYHEnKahALFDSBWIqQn4vJUHfhDvdY22WT/5M10SBePpYJwPROO/Y1jDpvf7txXMcbh3gDZfXTnjO5sZyeofHOGsHsk/ZAeq1NRPbdokI162p4r7fv4ZvvH8n0XiC3/nqc/z9Q4cmrbktI8XVobHSymRyguBTkX5352RWXQwjYwmi8SQVxQE8HqGiQG1M0q29PSfdRTQfBqNxmntGuLyuhNuvWs7/uOMKOgejqXPlxsGWPgD2nenNekw63fbnn4+iwbmOQfz8lRaiiSQPf/wmPnrLOgB6Z+HG41JFBWKBckV9GaUh35TxB4fqkmDKX9s3MpaqMTjVOfFCcdh2E2xbWZ7atrKyiIbyMN/81WkA3rBx2YTnbFlRBpBqAXLSvhA7LiY3br28lkf+5PXcdkUd39p9epK7ybnzzxSIXWurEYFH8kxrbOkdSRXanchTVHLh3LFWFllWWVXxzKrUp8I5V+VFfp49OfPEwSOtlqV4Wa3VgHlVlT1XI0cc51CL9Zx9Z/OzXJwYzHxUuM+1BdHaH2V5WYg1NREqbMu8Z56nNbYPjKbmwsw2KhALFK9H+MKd27jnDevzOt4qlrP+iI6nZfScznAxHe8YpKE8THFwvE2XFYeoIhZPsqa6eNKF/7LaEoI+DwearTvPkx1DiJAa8pONkN/L27bWE40nOZjhv3aqqDNdTMtKQ1y9qpKHchTppdPSN8rVTZX4PDKjWoxMnD5MzsWhOhIsqIvpLZvqeDmttcp0cVyJl9dZAuE2UySTgxesc3GsfTCvC69zF987PPdFg3MtEG19o6nWL+W2m3W+x/n+02PHeOPnnizIaxdMIERkhYg8ISKHROQ1EbnH5RgRkX8SkeN2lfb2tH3vE5Fj9tf7CrXOhcwtG5ZxZUNZXsdWRQKpO71j9kVj/bIIpzNcTMfaBlm3bPKdv+NmuvXyZZP2+b0erqgv5UCzY0EM0VgRThW45cKZNZ3pZz/Y0k9FkZ+ysH/Sc26/qo4jbQMThM6NRNLQ2j/KisowK6uKUq6vi8G5W6ws9tvfAwWZCdE1GEUEbruqjnhy5nGII60DFAe8qSLDZSUhvB7JKhDReIITHYNstUe87j83tZsp3XKYa398ulU8FzRc9HQAACAASURBVNlErf3jmXXlYceCmF8X0/mekdT5nW0KaUHEgU8YYzYC1wEfEZFNGcfcDqy3v+4GvgQgIpXAZ4BrgWuAz4hIRQHXuuipjgRTd3pH2wYJ+73sWlfNma7h1D9WImk40eEuEDdvWMb2leW8a+cK19ff3FjOq+f7iSeSnOocZE11dvdSOrWlIRrKw+xLuwAaY3jmWCe71lW7BuBvu7IOgIdfzW1FdAxESSQNy8vCrKkunh0LwhaICsfFFClQDGLIypS6xrZ+nDqU6XK4tZ/1tSV4PNbv0esR6kpDWftZHWsbJJ403HnNCkTgpTzcTOkCmUssHzvUxg9fODfNT5CdsUSSvpExigJexhImNcSpUBhj7AmHlkAEfB4iQd+8u5jO947QULHABMIY02KM2Wf/PAAcAhoyDrsDuM9YPIs1c2I58BbgEWNMtzGmB3gEuK1Qa10KVEWCdA/HiCeSHGsfYN2yCGtrihkZS6RSXc/3jBCNJ1nvIhCVxQEe+KNdbKhzHya4dUU5I2MJjrYNcqpjaEIG01TsWFXB3jPdKaE62jZI+0A0lSWSyfKyMDtWVfDQK7mbCDo1EPXlls/4VNcQyeTF3WU6d8uOQFRHggxG43lnVeVL16CVKVUc9LFlRTnPzkAgjDEcaR1IuZccrFRhdwvCcfVd3VTJhtoS9p2d2oJIF8hcYvnVp0/yL0/MXq2EY600VVl/axfrZvrJ/vN8e8/p7O83PEYsnpxU3T+fQWpjzIK1IFKISBOwDXguY1cDkH5L0Wxvy7bd7bXvFpG9IrK3o6Njtpa86KiOBDDG+iM/1jbI+toIq+x/LOfO+niH5XpysyCmYnOj5ep69FAbQ7HEpAymXOxYVUFbfzSVcfT0Mes83mjXWrhx+5V1HGzpnxRDSael1+kGG2Z1dTGxePKiJ9L1DMXwCJSGx11MMPsB2u6hGFX2a1+3ppIDzX0MTjMO0TEYpWd4bJKo56pGP9TST9jvZVVVMdtWVvDS2Z4pRbVrKIptoOQsGjzbNTyrLijHWllj/61d7IX63qdO8jf/cYj+UffXaXVJnKgoCsyrBdE3MsZQLEHjQrMgHEQkAtwPfNwYk1lJ45bAb3Jsn7zRmHuNMTuNMTtrarJfUJY6Tjrsqc4hWvtHuay2JHWX71xkndTXmQhEU1UxJSEfP37pPACr83QxweQ4xDPHO1lTXZzzruj2q5YD8PMcrchbHAvCFgjIPz02G93DMcqLAnjtK6JzEb8YN1Nb/yiJjItw11A0VQF//ZpqEknD3iyNEbPhZDBtqM0UiDCtfZPfE6wA9Ya6ErweYfvKcgZG46meX9noHoql7uKz/R6i8QQt/aMMROMzni/i9r4wni13MRZEImk43j5INJ7kZy+7uy6d1OvaDAvCLQYxV8F6Z7rkgrQgRMSPJQ7fNcY84HJIM5Du1G4ELuTYrswQ50LmuCrWL4tQXx7G7xVOd1mprsfbB6mOBCkvyj1fwg2PR9jcWJZqabFmGhbE5XUlFAW87DvTQzSe4LmT3dyYxb3k0FAeZsuK8pzZTBd6RykKeCkN+1JtPy42DtEzNEZFWpGg049ppu02Ogai3PQ/n+DBl89P2N41FKOq2Hrt7avK8XunH4dICcQkCyJMPGkmpUZaFdH9bKovtd/XEu6p0l07B2Osri7Gl2P2+bnuEZwYspMJdrGkLIjqi3cxne0eJmqnWv/7i+5xksz282BZEJlZTIdb+7niM7/IK8B/sTiuwgUXgxAruvh14JAx5nNZDnsQeK+dzXQd0GeMaQF+AbxZRCrs4PSb7W3KDHEuZE7l8mW11l3iisqilAVxvGPQNf6QL1sarcyXsN87KT01Fz6vh60rynnxTA/7zvQyMpbgxnW5BQLg166s45XzfRM6zabT0mfVQIgINSVBigPeixYIp4ra4WItiJfP9RKLJzlsX8zBanXROzyWep+igI+1NRFOtE9v7UdaB6iOBFPn3iHbXI0LfaP0j8bZuNwSiDXVxZSF/bw0RRyi27Z2chUNnu0eX/ts9a7qtosJV8+CQDhi+uubl7PvbK9rhlxr3ygisCyt9qiiyD/JvXikdYBYPMl9e07PeD35cn4BWxC7gLuAW0Vkv/31ayLyYRH5sH3MQ8BJ4DjwVeCPAOyW4n8NvGB//ZW2Gb84nJkR+872EvaPpz2urirmdNcQxhiOZ0lxzZfNtkA0VRensmbyZceqCg629PPLg614PcJ1dlptLpyU22x31hf6RlN5/yLCmprIRTftc/owOaQaIc7wonfgvFU74vyjw/hI06q0SYHLy0Ipl1m+HGmbHKCG7LUQTv3DJlsgRIRtK8tzWhBO65GqSJCqHCm/Z7rGRXy24jXO4KZUkPoiYhBOvcifveVyvB7h/n2TmyS29o1SVRzE7x2/bJYXBRgYjRNPcyk5rqj/ONByUWvKhwu9I4T8ngk3LbNJIbOYnjHGiDFmszFmq/31kDHmy8aYL9vHGGPMR4wxa40xVxlj9qY9/xvGmHX21zcLtc6lQmnYh88jxOJJ1tdGUhfwVbZAtPVHGYjGWV97ERaEXVE9HfeSw/ZVFSSShh88f46tK8opDU2uf8hkbU2E8iJ/Vt98ehU1WHeaF9tuo2d4ogURCfoI+DwzroV41RaI5nSBsF/LcTEB1JWFU0HSfEgmDUfbBlIV1OnUl1u/k0yBONTSjwgTRGX7ygqOtQ9mDdz2j8YZSxiqigN2KrW7UKa39pgtgei0U4FLQj48cpEWRNtAql7mlg01PLCveVKMxkpxnWiNOe7G9HkU7f1WDUs0nuTH+ye6Dmeb871WBtNU/dhmilZSLxFEJHVHmm4lrK4uYnQsya+Od07aN13qSkO8ceMy3rypduqDM9i+wvJ35+teAivusXNVBXtdishi8SQdg1GWl42b3quri2nuGZlxkNQYY8Ug0gRCRKieYT8mY0yq+jzd3eP0NkoXovqyEF1DsbzTac92DzM6lnS1IEpCfkpCPleBWFVZNKGKfvvKCoyB/VncTCkxiwRyFg2e7RpOifVs1Y1026nAHo9QGvZflEAcaxtIBfPfuaORtv4oTx2bmBXZ1j9KXelEV47zt5Aeh2gbiLKqsoirGsr4/vNnp13Ad6JjMO9sL6sGInfHgotBBWIJ4WQypd9VNtn+20cPWb2NLkYgRISvve9q7tjqmpGck7Iifyr+8brL8hMIgJ1NlZzsGJp059rWb3WDde6WwbJsjLHcHc+f6uaWz/4nn/jhyxPcA7kYiiWIJZITgtQAlZFA1jvnXLT1R+kcjFJTEqRjYLxDbefQZBeTExht78/vfQ5nCVA7uLVNP5gWoHbYuNx6fraqdedzVxYHqYoEUuKWyZnuYa5sKMPrkVl1MTkiWn4RAhGLJznZMZT6v7j18loqivz8+96JbiY3C8JJ6EjPZGrvH2VZaYj3XLOCw60D0wpWJ5OG3/7yHj79o1fyOr6QNRCgArGkqEoJxLgIOP7bJ492WHOW8+gOWyiuX1tFRZE/FezOh6ubLMsj04pwmv1lWhAAf/XTg7zn3j0MjMa5f18zH/3eS3kN0unJKJJzqCoOzuii57QmecsVlsXl3NE7wdeq4vQYhB03yDMOcaR1ABGyugwzBy8NRuOc6RpmY91EgagsDhD0ebLGP7pS7jDLxTTgUjSYTBrOdg+zurqYiiL/rLUm6RqKpn5HZWH/jMeOnuocIp40KTEN+DzcduVynjzakaoBGR1L0Ds8Nin5wrlZSL/jbx+IsqwkyNu21FMU8PKD5/OvHj/aPkDXUIzHDrVPGb8YiSXoGooVrAYCVCCWFM4M5fXLxu8q68vDBLwehmMJ1i2LFMyXmQ9/dtvl/PRjN+Lz5v9neWVDGQGfZ1IcoiWtitrBEYhnjnfyrh0rePK/3Mxf/vomHn6tlQ9/58Up3TepTq4ZAcGqyMw6ur56vg+vR3jjRksgnDhElx18TU83Xm5/jnzjEPvO9rCuJpKa6ZFJfXlogtg4w36cDCYHEWF5WSjrUBzHXeS4mGByjKFtYJRYPMnKyiIqiwOz1tzQCo5b73kxLqajqd5k4/8X21aWMxiNp5Ia3GogYPxmoTfDgqgtDVES8vO2LfU8+PIFBrLEcDJ54ZT1dxxLJHn4tdytZFIprmpBKLPB6upiqiPBCX9QVqqr9fhiUlxng0jQR+M0/alBn5etjeW8cHpqC6Ik5Oe/vXUjX33vTv7xnZspDvr4/RtX87fvuJLHD7fzPx8+kvO9nOyiikyBKA7MKIvpwPk+1i+LpNx6zj9811CMyrRiPBjvatuSh0DE4kleON2darDoRn15mN7hsVSX2CePdOD1CDubJrc8W14WTk3my8S52FcWB7Km/DoZTKuqiqzK41mog0gkDb0jY1TagfyysH/GY0ePtg3g9ciE5ArHinWsvGzt552Ork419WA0zlAskUqF/a0djYyMJVIxvql47lQ3daUh1lQX8+OXcpd+FboGAlQglhR3v34Nv/yT101KQXXcTBcTf5hPdjRV8NqFvgmjOVt6RygN+SYEXAE+dNMa3pQRRP/da1exfWU5R9pyj8x03AiVGS6m2tIQo2PJCRP7psIYw6vn+7iqoYy6UqvDqpPq6gRf0ykO+igN+WjNw8V0oLmX4ViC69dmj+U4NwmOpfXooTZ2rKpwLZJcXh7KKkxdQzFKgj6CPm/WosGzjkBUFk/oKnwx9AzHMGbcDVdeNHML4kjrAE1VRRO6D69bFqEo4OVlO36QGmCVYUFEglZ2oBODaM+wNNbZVd7pWWrZMMbw/Klurlldydu21vPsqa6cFmOhayBABWJJEfR5XfOlnUD1QhWIq5sqGEsYXm4eDwam10DkQ3q322w4F4FMC8KpOH5uGoN9WvpG6RyMcVVjGT6vh7rSUGpWdNdQ1PU8LS8L5zX/ePeJLkSsHk7ZqE8Vy43S3DPM4dYB3rTRPfusvixMW/+oazC/azBGZWR8eJKzLZ0z3UP4PEJ9ech2MV18DCLT3Vdmu5hm0vL7aNvApGC+1yNc2VDGy3aWmXOhrs2wIESsqYJOFlObnUTgWBDlRX5Cfk9elt/Z7mHaB6KWQGypxxj42YHsVsT53mG8HplQuDfbqEAoqd47l2cEKBcKO1ZaF8L0OIRTRZ0vVZHglHGEnqEYXo9QGppolWxuKCMS9GWdr+3GK3b9w1X2PI/GivAEF5PbKNnl5aG8YhC7T3RyRX1pzpYpDWnFco8fbgcmTwpMf9+kIdX1N530poLZigbPdA3TUBHG5/VQWRykd2TMtQ/UdOjMCOSXhf0kkmbaDQ1HxxKc6R6eEH9w2NJYxsGWfmLxJK39oxQHrNG9mVQU+VMupvYB6/wssy0IEaG+LJxXkeNzdvzhmtWVrKmJsLmxLGcdxfmeEepKQ9OK2U0XFQiF39zWwM/vuWmSf3WhUFbkZ0NtyYQ4REvvKMunZUFYwVO3zqU9QzG++atT/Hj/eSqKApMC+T6vh6ubKqbVK+mVZitA7QSFGyrC40FqFxcTONXUuQVidCzBvjO93JDDvQTWHa4zOOjRQ+2ukwId6sscd9Tk9+4cjKbiANmKBs92D6emC1YVO12FL86KSFkQkXGBgOkXyx1vH8QY93TgzY3lxOJJjrYN0NY/Sq3dtiWT8qJAmovJtiBKxwV+eXn2+RvpvHCqm4oif8ot9bYt9bx6vj9rinEh50A4qEAo+Lwe14rbhcTOpgr2ne0hkTS09Y/SNRSjfjoWRHGApGFSquS3fnWKa//uMf7HTw9SWRzgr++4wvX5N6yt5mTHUN5ZRq+c7+Oy2pKU37uxooi2/lFGYgn6RsYm1EA41JWG6RyM5iz0e/FMD7FEkuunaFXiuLWOtg3w7ImurNYDjGdQud0Fdw/FqLbXmq1o8EzXuEDMVnv0yS4m63s+AvGr452poHHmzO50nED1y829tPaNZu0vVlHkT3MxjRL2T7Q0ludpQTx/upudTZWpGOHbttQjAg9msSLO94zQWMD4A4B7DpyiLDCubqrku8+d5dq/eyzlfmiaxtCiSifAOjjR///wa600VIT54u9sn1RElo5zQd5zspN3bGvM+V7GGF453zfB599YHiZp4GCL5XqqymJBgHWXusK+4F7oHWEskUzN9th9ohOfR7i6KXv8waG+PMTjh9sZSxjekCX+YL2vbUFk3AU7fZjSf1+ZRYN9w2P0jYyxqmrcggA7TjH9gvsUjgg5aaYpCyKP3kf3/GA/nYNRbr18GUUBLwGvh6aqydlzKyrDVBT5OXCuj7b+KNeudv+dVhQF2Ddsxb/aB6IsKw1OsDTqy0K0D0QZSyQn9HFKp61/lDNdw9x13arUtmWlIa6sL+Mll0K7sYTl9iq0BaECoSwKXn9ZDTeuq6Y6EuDKhjKubCjjmjwukg5OjUjXUIz1advb+6Nsqi/NKQ5gNbgrC/vZfbxrSoE41z1C91CMKxvH54k7xU5O643KYvcYBFiuHkcg/vSH+3ntQj8P/D83sL62hN0nutiyopyIi688k/ryMC+c7qEs7GfnquwTfUtDPooC3klFev0jceJJM6FbbGbRoNODaWWlJWCOS2g2LIiysD91wc3XxdQ7HKNz0LrYP3+qm8Go1b3WzY8vImxuLGf/ud6Ui8mNcrvltzGW9VpbMvG45eVhjB3DyZZx9Lwdf8gU9uVloQmNDh1a+0ZJmsJmMIEKhLJIqCgO8J0PXTvj56dSNDPcI+0DUV6fR5aIxyNcv6aK3Se6MMbkLDh8yJ6l/bq0mRcNGQLh5mJyLAjHXRGLJ3npbC/ReJL3f/MFvvOhaznQ3Mcf3bx2yvXCeCbTLRtqcgY6nWK5TAvCCUanWztVkcAEn/kZu823Y0GMu5guLtU1PTgOVhwKphYIZ/jRh1+/lisbyvg//3mcK+rLsh6/pbGMJ49aPZlyuZjGEoahWIKOgSgbM24mUuetN3tbjBdOd1MU8HJFxnNrS0M879KMci5qIEBjEIoCuGfgDEbjDEbjk6pns3H92irO945wrju3v/nHL51n+8rylFsILDeOCKlUXTcXU11GsPi1C31E40n+6Oa1dA/F+K0v7SaRNFPGHxwcgcjlXko/NtOP3uVSWZ5ZNOjc/ToxCMcl1H2RxXLpE/fA6sUEUwuEI15rayLUlAT5zG9cwTt3ZLf4Nqe1fcn2d+B8pp6hmKsFkWqvniM+9fypbnasqpgk1MtKgvQOj02q8p+LGghQgVAUADs7iQmpruNFT/nlmd+QFofIxqGWfg63DvD2bRMbGgZ8HmpLQqmRqJlDfsDKEioJ+lKBcGdE6/tvaOKf79xG73CMgM/D9pXZ3UXp3HxZDe/Y1pAzQO3g1m4jvc2GQ1UkyOhYkuGYlW56tmuY6kgwVbDo93ooDflmxYJIF6aigBefR6bsx3SiY4iAz5P3nffmFePWRba0aaeaurlnxKqizvh7qUuzINw43j7A4dYB1y7GjihlTv9zLIjp1PrMBBUIRcEqjKosmhhgdfL+l5XkZ0GsWxahOhJk94ns6a4/3n8en0d4qz1TOx3nouWR8TviTKyqZuvi8OKZHlZUhllWGuKNm2r5/Lu38sk3XzahIjgXKyqL+Py7t2bt1zThfcusDKr0pobjLqb0GMR4ENoYw4HzfayunhgArooEL7phn5UKPP6+IpIqlsvF8fZB1lQXT2hjkotlJaGUMGRLA3cKJ52eTpk3FKUhP5GgL2uK8rf3nCHg9bhaMo7YOJXcDud7RqiOBPM+1zNFBUJRbKoiE1M026ZpQYgIN6wdj0NkkkwafvLSBV5/WY2rheAEqiuKAlkn8jmDg4wx7D3Tw85V40HNO7Y2cPfr8os/TJf68hDGTLxQuc2tcKyJzsEozxzv5FBL/6Sg/cVWUyeThp7h2CQ3XKZAfP/5s5zpmjhB8ETHIGun2TFgc6PVptyteBHGO7o6LdbdbiiWl4Umzd8Ay415/77zvHXzcte/CceCyCxSnIsaCFCBUJQUVcXBCf5zx6yvydOCALhpfTUdA9FUYDOdZ0910do/yh3b3OdlOP5ktwC1w/JSy9XT3DNCx0A01eaj0Cx3KZbrGopRErKK4xwca6J7KMa/PH6cutIQv7Vj4ue9WIHoHRkjaSZ31S0rGm/Yd6ZriD9/4BW+/OTJ1P7RsQTnuodThWj58v4bVvOxW9dltTqcivVsFgRYmUxuFsSPXjrPYDTOXdevmrTPei3rby/TgmjuGWaFCoSizB1uFkTI75nUWiMXd2xtoKmqiL/5j0OMZfQu+slLFygOeLP2PHI62eaaL1xXFqJzMJqq2s6Vnjqb1LsUy7m1BHHE7eevtvLcqW7uft0agr6JbpBcs6tz0Tc8xvH2AZ62J71lCmlZ2J9qu/2fR6xjnj817u473TVE0jBtC+L6tVV8/I2XZd3vuAOP2haE2w1FvctMcWMM39lzhisbStm2wn0GSkWRH79XUj2ewLKgzveOTLvz8UxQgVAUm+pIcEJH1rb+KLWl7u0VshHwefh/37qJ4+2DfPfZM6nto2MJHnqlhbdcWUc44O43dlwGVS41EA6Oq+fnr7QQCfrmrAI+NbAoLdU1s6gQxtd+/75mqooD3HnNykmvVVkcoGcoNq3Ges+f6mbrX/+SN37uKe75wX6ASRfIdBfTE0es/lInOoZSlqCTwTRdC2IqfF4PJSEfA9F41hsKK4YTm1AF//ypbo60DXDXdauy/o2JCMtKQqmECcAuujMFHRTkoAKhKDZVxQH6R+OpQGz7wOiMOmW+ceMyblxXzecfPZZKfbzr688xEI3zrh0rsj7P+YfP5WJyUl2fPtbJtpXleQdbLxan3Xj6XXBmLQJAOOClKODFGPjgTatdxbCyOEA8aegfyb+x3pNH2/GK8IX3bOU7H7yWR//09ezIsJ4cgRiJJdhzoovtK627cqcI7UT7ECLjg6NmEyfVdVmJ+w2FU+TY1jd+A3Lfs2coC/t525bcI3prS4O0DYwLhNP1d0ELhIh8Q0TaReTVLPsrRORHInJARJ4XkSvT9p0WkVdEZL+I7C3UGhUlncwq3/b+aKor53QQEf7i1zcxMDrGn/5wP2/9p2d49Xw/X3jP1pw1Cg3lYYI+T87URSejJp40ky6QhWZ5WXiiBZE20S2dqkiA0pBvQtuIzP3W8/NPdX3hdA9X1Jdyx9YGblxf7dqavjzsp390jN0nOonGk3zs1vUUBbw8Z7uZjncM0lAezmrBXQxOoDpbQkN9xsjYnqEYv3i1lXfuaJxyPbWloQkupnMpgVjYLqZvAbfl2P9pYL8xZjPwXuALGftvMcZsNcbsLND6FGUCjnvEcTM5s4Vnwoa6En732lU8caSD0rCPn3x0F3dszX2nGPJ7+dnHbuS9WQKWMDHVcs4FIi3F9mTHIJ2D7q0j/uCmNfzNO66iJOSeqjteLJdfHCIaT/DyuV52TtE6pTTsxxh48OULhP1eblhXxY5VFak5HSfaBws286Q8zYJwI7Ph4RNH2oknDW/bUj/la1sCkWZB2IWYc2FBFKzVhjHmKRFpynHIJuDv7WMPi0iTiNQaY9oKtSZFyUV1mgUx3SpqN/7stg1cVlfCO7Y15NUbCWD9FDEFJ6d+OBZna5bAZqFYXhZOtQL5/KPHCPu9vPvqyTGG917flPN10jOd8uHV8/1E40mudhmHmo7Tj+mXr7Wxa101QZ+Xa1dX8tlfHqVrMMrJzsG8q8yni2NBZBbJOTiWn2OBPXqojWUlwdQ8kFzUlAQZGI0zHItTFPDR3DNCTUnhayBgfmMQLwO/CSAi1wCrACdh2gC/FJEXReTuXC8iIneLyF4R2dvRMTm1UFHyJX1k5nSrqN0oCfm567pVeYtDvtSVhdhQV5r1Dr1Q1JeF6B6Ksf9cLz99+QIf2NVEzQwsrOk27HMGQe1YlduCcARiZCzBLZfXAHDtGksQfrz/AqNjyXmzIIoCPsrCflr6RojGEzx5pIM3bKzNWu+STqoWwnYzNfcOz4n1APPbrO8fgC+IyH7gFeAlwIla7TLGXBCRZcAjInLYGPOU24sYY+4F7gXYuXPnxY2pUpY0Kd/4YGzaVdRzyX+97XLCc3D3mIkzgOlT9x+gJOTj7ptmVpRXldY5Nx9eON3D6uriKcUofYLezRus9iGbG8sI+jx87zkro2ztLGcwOThus1w3FE7Dw2dPdjMUS/CmTVO3OEl/zbb+UZqqi2nuGZnQI6qQzJsFYYzpN8Z8wBizFSsGUQOcsvddsL+3Az8CrpmvdSpLh5Kgj4DXQ+dgbNpV1HPJmzbVcuP63BPjCoEzgOlw6wB/+Lo1qQ6q0yXktzKd8rEgkknDi2e686r3cCyIDbUlqdhI0Odl+8oKTtg9rgplQVQU2y6mHDcU9eXWTPFHD7ZZMZIppv45pIrlBqIkkoYLvSNzZkHMm0CISLmIOJL/IeApY0y/iBSLSIl9TDHwZsA1E0pRZhMRsYvloilzfjpV1Isdx4KoKg7wgV2rL+q18q2mPtk5SM/wWF4DkJw4wM22e8nh2jWVqf25ihAvhpWVRXhkvK25G067jUcPtfG6y6rzjiE43WHb+0dp6x+dsxoIKKCLSUS+D9wMVItIM/AZwA9gjPkysBG4T0QSwEHgg/ZTa4Ef2bnEPuB7xpiHC7VORUmnKmJV+bYPTL+KerFTX241rvvjN6xPdWedKflWUztzxndOEaAGawLb3//mVbxp08RK9WtXVwHHCmY9gDWw6lefujVVUOhGfXmYvhFrwt6fvil7ZXYmpWEfQZ+H9oFoam75ijlIcYXCZjHdOcX+PTBheJez/SSwpVDrUpRcVBUH6RqMEgn6pl1FvdgJ+rzs/tSts/I7qSwO0NI3yv5zvZzqHKQ8HOCWyyf75F843U1VcSDv4ja3yu1tK8sJ+jysW1a4qnNrqFLuu3onk0kEbnX5rLle20l1ncsiOdCJcooyAWciWjjgz/QIxwAACrxJREFUnXENxGJmtgSzOhLkiSMdvP2LvwLA5xEO/fVtk2Y27z3dw86miot635Dfy7c/eG1qaNF84QjIjpUVrp1bc1FbGrQFYm7mQDioQChKGs5EtIDPM+UcamXmfPjmtWyoK2FVVTEHL/Tz+UeP0po2axusrJ2z3cM5Cwfz5ZrV+c8nLxQrKq2L+puvmHqCXybLSkMcutBPc88wy+aoBgJUIBRlAs5EtOaeYW7eUDP1E5QZsbYmkko5LbZbTZzrGZ4gEM7EvLmuGC8UjRVFfO8Prp3R51lWEuQ/bQtirtxLoM36FGUCTo7+WMJcVBW1kj9OTyHHfeJwwu6+unH54rHkblhbPan9eT7UloYYiiU40jowJz2YHFQgFCWN9PkGl2INxGKkriyERyYLxOmuYepKQ3PmTrmUcf4Wu4ZiakEoynyR3p30UqyiXowEfB7qSkOpDB2Hs91DOesKlhK1aX+LK+Yw2K4CoShpVKkFMS80VhS5WhAqEBbpbefVglCUeSJ9AI5WUc8djRVhzqcJxFA0TsdAlFVVsz/cZyGSfrOiMQhFmSdCfi+RoE+rqOeYxoowrf2jxO053me7LXeTWhAWkaCPIjvby5kPPhfof4CiZODEIbSKeu5orCgikTS02LUQZ7qs5npNakEA49XUw7H4jLKgZooKhKJkUFsawqviMKc4fvXmnhFbICwLYqVaECmaqooYS8ztRAMVCEXJ4O9/8ypUHuaW8VqIYaCK013DVBYHKJ3joUiXMp991xbmeuCNCoSiZFCooTJKdjJrIc50aYprJtPt3zQbaJBaUZR5J+DzUFsaShOIYVbNc3M9RQVCUZRLhMaKMM09w0TjCS70jWiK6yWACoSiKJcETrHcue4RjNEU10sBFQhFUS4JnFqIkx1Wkz61IOYfFQhFUS4JGivCJJKGZ092A1ZapzK/qEAoinJJ4KS67j7RSSToozKt7YkyP6hAKIpySeAUyx1uHWBVVZFWsl8CFEwgROQbItIuIq9m2V8hIj8SkQMi8ryIXJm27zYROSIix0XkU4Vao6Iolw7Ly8I4mqAB6kuDQloQ3wJuy7H/08B+Y8xm4L3AFwBExAt8Ebgd2ATcKSKbCrhORVEuAZy5EKAB6kuFggmEMeYpoDvHIZuAx+xjDwNNIlILXAMcN8acNMbEgB8AdxRqnYqiXDo4biYNUF8azGcM4mXgNwFE5BpgFdAINADn0o5rtre5IiJ3i8heEdnb0dFRwOUqilJonED1ykq1IC4F5lMg/gGoEJH9wMeAl4A4uPZJy9qjyhhzrzFmpzFmZ01NTWFWqijKnJCyIKrVgrgUmLdmfcaYfuADAGKlK5yyv4qAFWmHNgIX5nyBiqLMOe/Y1oDPMx6LUOaXebMgRKRcRJxE5w8BT9mi8QKwXkRW2/vfAzw4X+tUFGXuWFMT4Z43rtcU10uEglkQIvJ94GagWkSagc8AfgBjzJeBjcB9IpIADgIftPfFReSjwC8AL/ANY8xrhVqnoiiK4k7BBMIYc+cU+/cA67Psewh4qBDrUhRFUfJDK6kVRVEUV1QgFEVRFFdUIBRFURRXVCAURVEUV1QgFEVRFFdUIBRFURRXxJisXSwWHCLSAZyZ4dOrgc5ZXM5CYCl+Zlian3spfmZYmp97up95lTHGtU/RohKIi0FE9hpjds73OuaSpfiZYWl+7qX4mWFpfu7Z/MzqYlIURVFcUYFQFEVRXFGBGOfe+V7APLAUPzMszc+9FD8zLM3PPWufWWMQiqIoiitqQSiKoiiuqEAoiqIorix5gRCR20TkiIgcF5FPzfd6CoWIrBCRJ0TkkIi8JiL32NsrReQRETlmf6+Y77XONiLiFZGXRORn9uPVIvKc/Zn/b9rgqkWDPZDr30XksH3Or1/s51pE/sT+235VRL4vIqHFeK5F5Bsi0i4ir6Ztcz23YvFP9vXtgIhsn857LWmBEBEv8EXgdmATcKeIbJrfVRWMOPAJY8xG4DrgI/Zn/RTwmDFmPfCY/XixcQ9wKO3xPwKftz9zD/awqkXGF4CHjTGXA1uwPv+iPdci0gD8MbDTGHMl1rCx97A4z/W3gNsytmU7t7djzd1ZD9wNfGk6b7SkBQK4BjhujDlpjIkBPwDumOc1FQRjTIsxZp/98wDWBaMB6/P+q33YvwJvn58VFgYRaQTeCnzNfizArcC/24csxs9cCrwO+DqAMSZmjOllkZ9rrAFoYRHxYc22b2ERnmtjzFNAd8bmbOf2DuA+Y/EsUC4iy/N9r6UuEA3AubTHzfa2RY2INAHbgOeAWmNMC1giAiybv5UVhP8N/BmQtB9XAb3GmLj9eDGe8zVAB/BN27X2NREpZhGfa2PMeeCzwFksYegDXmTxn2uHbOf2oq5xS10g3CajL+q8XxGJAPcDHzfG9M/3egqJiPw60G6MeTF9s8uhi+2c+4DtwJeMMduAIRaRO8kN2+d+B7AaqAeKsdwrmSy2cz0VF/X3vtQFohlYkfa4EbgwT2spOCLixxKH7xpjHrA3tzkmp/29fb7WVwB2AW8TkdNY7sNbsSyKctsNAYvznDcDzcaY5+zH/44lGIv5XL8ROGWM6TDGjAEPADew+M+1Q7Zze1HXuKUuEC8A6+1MhwBWUOvBeV5TQbB9718HDhljPpe260HgffbP7wN+MtdrKxTGmD83xjQaY5qwzu3jxpjfBZ4A3mkftqg+M4AxphU4JyIb7E1vAA6yiM81lmvpOhEpsv/Wnc+8qM91GtnO7YPAe+1spuuAPscVlQ9LvpJaRH4N667SC3zDGPO387ykgiAiNwJPA68w7o//NFYc4ofASqx/sncZYzIDYAseEbkZ+KQx5tdFZA2WRVEJvAT8njEmOp/rm21EZCtWYD4AnAQ+gHVDuGjPtYj8D+DdWBl7LwEfwvK3L6pzLSLfB27GauvdBnwG+DEu59YWy3/BynoaBj5gjNmb93stdYFQFEVR3FnqLiZFURQlCyoQiqIoiisqEIqiKIorKhCKoiiKKyoQiqIoiisqEIpyCSAiNzvdZhXlUkEFQlEURXFFBUJRpoGI/J6IPC8i+0XkK/asiUER+f9EZJ+IPCYiNfaxW0XkWbsP/4/SevSvE5FHReRl+zlr7ZePpM1w+K5d5KQo84YKhKLkiYhsxKrU3WWM2QokgN/Fagy3zxiz/f9v745VqwjCMAy/nwiiJGiVJoViGzAEa6vcgEUkIEguwMZOhKTxHgQtE7SQQOwFiwOpFMHKK0gvQgpB4m8xc8SEJS6Y5DTv051hdjhTLP/uwH4/MKF92QqwAzytqju0L9in42+AF1W1TMsLmkYfrABPaL1JbtOypKSZufzvKZK6VeAu8Kk/3F+lhaL9At72Oa+BvSTXgRtVNenj28BuknlgsareAVTVD4C+3seqOui/vwC3gP3z35Y0zAIhjRdgu6qeHRtMtk7MOy2/5rRjo78zgo7w/tSMecQkjfcBWEuyAH/6AN+k3UfTxNCHwH5VfQe+JbnXxx8Bk96D4yDJ/b7GlSTXLnQX0kg+oUgjVdXXJJvA+ySXgJ/AY1pDnqUkn2mdzNb7JRvAy14Apomq0IrFqyTP+xoPLnAb0mimuUr/KclhVc3N+n9IZ80jJknSIN8gJEmDfIOQJA2yQEiSBlkgJEmDLBCSpEEWCEnSoN/hne8xuRSXRAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "    import matplotlib.pyplot as plt\n",
    "    plt.figure()\n",
    "    plt.plot(test_rmses)\n",
    "    plt.xlabel(\"epoch\")\n",
    "    plt.ylabel(\"test RMSE\")\n",
    "    plt.show()    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}