You should add a code-of-conduct document to your project as early as possible.
Similar to how you want software documented to describe what behavior to expect from the code, the code-of-conduct documents the expected behavior from people participating in your project.
Yes, for most people and in most projects, these written rules are totally obvious and unnecessary to spell out because a majority of people has a natural sense for them. The document is still there and is useful to make it clear for newcomers and casual observers that your project is a nice place where people are expected to behave according to these standards. Having them spelled out removes the need for people to try to get a sense for unwritten rules and concepts that can be difficult to figure out until someone actually tries to break one of those invisible lines. Such "invisible lines" are also tricky to keep consistent and unbiased, especially over time.
Experience also says that it is way easier to set these rules early on and live by them from the start than it is to bring in code-of-conduct discussions many years later when there is a huge developer community and the arguments around specific phrases in such a document can be time and energy consuming. Time and energy you might rather spend on other things.