Improvements to dev configurations

[Aveline-art]

Overview

As a developer we need to set up an environment friendly to developers to improve development speed and reduce the need to debugging our env. For this issue we will discuss, plan, and implement an environment that meets our needs as our development work evolve.

Action Items

• Discuss ways to implement our ideal dev env
• Implement it

Resources/Instructions

Our current (dev) environment is a fairly complex configuration. We have three docker containers as created by docker compose:

• pgdb (our postgresDB)
• webpack (which runs webpack in watch mode)
• django (which starts our server to host the site locally)

This setup however, leaves a lot to desire. For frontend work, it is adequate. It successfully compiles frontend code as we develop, and due to the existence of node_modules in our host, we can take advantage of Intellisense. As the project grows, however, this will not work. More specifically, backend work is hampered by the fact that python dependencies are installed within the container, meaning that and IDE's Intellisense and other QoL features cannot be leverage. Therefore, this issue will outline features of an ideal dev environment and come up with a plan to implement it.

Features of an ideal dev env

1. Leverages ide capabilities such as intellisense
2. Provide the same, isolated, env for all developers
3. Easy to add/upgrade dependencies
4. Simple and easy to run

Final note

In a perfect world we would be able to create the perfect env. However that might not be possible. For example, our current setup is mildly complex as it requires two dockerfiles and three containers to work. Also, to make dependencies easier to update, we sacrifice the ability to use docker to provide an isolated environment.

Developing inside a Container
Working with the Docker registry

[Aveline-art]

From @Enzyme3 over Slack:

I believe the reason for the error (and also why the workaround works) is because of the volume mount:

django:
    build: 
      context: ./app
      dockerfile: ../dev/django.dockerfile
    container_name: django
    command: python manage.py runserver 0.0.0.0:8000
    volumes:
      - ./app/:/code
    ports:
      - "8000:8000"
    env_file:
      - ./dev/dev.env
    depends_on:
      - pgdb

  webpack:
    build:
      context: ./app
      dockerfile: ../dev/webpack.dockerfile
    command: npm run watch
    env_file:
      - ./dev/dev.env
    volumes:
      - ./app/:/code

This stack overflow post explains what's happening, but the tl;dr is: during docker-compose build the Docker image runs npm install and installs the libs (such as webpack) in the container's /code WORKDIR. But afterwards during docker-compose up , the ./app directory from our local machine is mounted OVER the container's /code WORKDIR, essentially making it as if the build never happened. If you want to confirm this is happening, here are the steps:

• git clone a fresh copy of the code
• docker-compose build
• docker-compose up --> at this point, the webpack container will have died but the django and pgdb containers will still be running
• in a new terminal, docker ps and get the container id of the django container (because similar to the webpack container, it ALSO has the volume mount of ./app to /code
  • docker exec it /bin/sh
    • ls to confirm that the contents of the container
  • Now in your IDE, create a sample text file in ./CivicTechJobs/app (create this on your local filesystem, not in the container)
    • ls again to see that the new file you created shows up in the container, showing that the volume mount is active

This is also why the workaround "worked". It's because that command created the node_modules in your local FS and when it was mounted to the container, the container was able to run webpack

The quick fix is to remove the volume mounts from both webpack and django and then append COPY ./ . to the end of the django docker file. But if you start up the containers again and access http://localhost:8000/, you get this error: TemplateDoesNotExist at frontend/index.html This is because the docker-compose file currently creates 3X containers: pgdb, webpack, and django. The webpack container contains the webpack'd React code, but localhost:8000 points to the django container which does NOT have that webpack output. Funnily enough, the workaround had somehow hidden this issue as well, because the same ./app folder had been mounted to both the django and the webpack contianers.

So to confirm, are we going to have a separate container for React and a separate one for Django, or will we have one Django container that is responsible for serving up the webpack'd bundle? If it's the second option, we'll want to combine the webpack and django build into one process

[nooriaali9]

Closing this issue - since it is outdated/ not a priority anymore - based off of #552