Mixing typing and non-typing information in annotations

[jstasiak]

First of all apologies if this is not the right project to post this question.

Sometimes one needs to annotate parameters and/or return values with not only types but some other information as well (an information that could be used at runtime).

The example below should show what I'm thinking about and what kind of API I have in mind:

# 1. mix (for the lack of a better name) is an imaginary construct that can combine
# type and non-type annotations
from library import some_marker
def function(d: mix[Dict[int, str], some_marker], i: int) -> str:
    # ...

# 2. get_type_hints is modified to only return type-related information (the first
# argument to mix for the sake of simplicity)
get_type_hints(function) == {'d': Dict[int, str], 'i': int, 'return': str}

# 3. get_nontype_annotations /for the lack of a better name/ function is introduced
get_nontype_annotations(function) == {'d': some_marker, 'i': None, 'return': None}

I apologise if there was a discussion about something like this already, I haven't found any. What do you think about this particular need? There's no simple way to achieve anything similar right now, is there?

[emmatyping]

Could you explain why you'd want to do something like this? This seems like something specific to your use case, so perhaps it would be better for you to implement this yourself?

[gvanrossum]

A simple way to achieve this without changing the way annotations work is to pass the runtime information with a decorator.

[jstasiak]

I'd want something like this because I'd like to keep the benefits of type annotations (mypy verifying them etc.) and be able to add some extra information without much hassle.

I am using a decorator-based approach @gvanrossum suggests, it looks like:

@extra_info('param', 'xxx')
def function(param: Dict[str, int]) -> int:
    ...

I have two issues with that - it puts the metadata in multiple places and requires one to repeat the name of the parameter - I was just wondering if mixing type and non-type annotations was ever discussed as that'd solve my problem.

[gvanrossum]

Yes, it was debated. There's a small section in PEP 484 under "Rejected Alternatives":

Despite all these options, proposals have been circulated to allow
type hints and other forms of annotations to coexist for individual
arguments. One proposal suggests that if an annotation for a given
argument is a dictionary literal, each key represents a different form
of annotation, and the key 'type' would be use for type hints.
The problem with this idea and its variants is that the notation
becomes very "noisy" and hard to read. Also, in most cases where
existing libraries use annotations, there would be little need to
combine them with type hints. So the simpler approach of selectively
disabling type hints appears sufficient.

[mitar]

I also needed that in the past. To combine semantic types with structural types. In fact one could do that:

def function(param: {'type': numpy.ndarray, 'semantic': 'image', 'width': 100, 'height': 100}) -> int:
    ...

The issue is that mypy does not know how to type-check this.

[gvanrossum]

It's basically less readable than the solution with decorators. In the future almost everyone will want to use types, but the kind of runtime annotations you're using here are always going to have pretty limited application.

[JukkaL]

If we had intersection types (#213) and protocols, you could include limited forms of extra information in annotations by declaring empty protocols that act like 'tags' that don't have any effect on static typing, but these could be used to encode information consumed by other tools.

Mypy has a plugin system that has some limited support for changing the syntax of types, and you might be able to use it to hack something together -- but it would be mypy-specific.

[ilevkivskyi]

In some cases one can just use a NewType with a descriptive name, but indeed supporting Intersection would be better.

[jstasiak]

Thanks, I'll keep an eye on the Intersection and protocol-related updates, this looks promising.