automatic script profiling without decorators

[ta946]

@Erotemic after having to write a module scraper, i got abit more understanding of ast and tried it out like you suggested.

i finally found a solution to automatically profile a script without magically injecting decorators in the script file!
Its so simple I actually feel bad after spending all that time fixing that hack i made

in kernprof.py replace line 220

execfile(script_file, ns, ns)

with

            import ast
            with open(script_file,'r') as f:
                script_text = f.read()
            tree = ast.parse(script_text, filename=script_file)

            class ASTProfile(ast.NodeTransformer):
                def visit_FunctionDef(self, node):
                    node.decorator_list.append(ast.Name(id='profile', ctx=ast.Load()))
                    return self.generic_visit(node)

            tree_profiled = ASTProfile().visit(tree)
            ast.fix_missing_locations(tree_profiled)

            exec(compile(tree_profiled, script_file, 'exec'), ns, ns)

so it parses the script file into an ast. For every function, it adds profile as a decorator, which line_profiler adds to the global space.
inserting profile at the beginning of decorator_list makes it work for methods

Haven't properly tested all edge cases, but seems to work for functions, methods, decorated functions and methods.

This is just a v0.1 that i would add more options

• add a commandline flag -a to choose automatic profiling.
• check if function already profiled with profile, then do not add to decorator list
• profile imported modules
• a check to see if @profile was already used in the file, if so then skip adding the decorator to the whole file. use case being you only want to profile only 1 function without having to change the cmd.
• pass a list of functions/classes/modules at commandline to either include or exclude from profiling

i already have solutions for most of these, but thought id get this past you first before spending time on those

[Erotemic]

I think something like this is a big enough improvement that it justifies inclusion in line-profiler. But - as I've likely mentioned - because I'm just the maintainer and not the designer, and this is a widely used package, I want to be very careful when it comes to adding new features.

I like the idea of passing a list of modules / classes / functions to be profiled, although I think it could get verbose quickly. I recommend having the ability to specify a root package, and recursively include it's members. I've written code to perform a similar operation in xdoctest (to collect modules to be tested), perhaps you can get some reuse out of it: https://github.com/Erotemic/xdoctest/blob/main/xdoctest/static_analysis.py#L77

I think a robust solution to this is going to require it's own submodule from which kernprof can import, and then make use of. I recommend developing this as a standalone application, and once that proof of concept is complete, we can work on integrating it into line-profiler / kernprof.

[ta946]

yea I understand. tbh my main motivation of adding this functionality is so that I can install line_profiler from pip and its up to date instead of having to maintain a fork with the changes, and ofcourse other people can benefit from it

From what i can tell, the profiler can only add what it can see in the script file. It can add a module, but to recursively add its modules that would require modifying line_profiler's add_module function to include modules aswell, which shouldnt be too hard
Unless you meant adding a root package by its file location, then i would have to figure out when its submodules are visible to the script. The recurssion issue would still need to be solved

I'll can try my hand at turning this into a submodule when i get the chance

[ta946]

@Erotemic revisting this again if you're still interested 😁
been testing the ast solution for a few months with no issues. but its by no means a comprehensive test

just added the option to automatically profile all imported modules, previously you would have to add them manually. It's not recursive to avoid blowing up the results and possibly slowing down execution
similar to how the functions are profiled, which is by adding profile to their decorator list using ast. modules are profiled by adding the call profile.add_module(module) for all imported immediately after they are imported

I couldn't figure out how to make it run as a standalone application as it requires kernprof's global namespace.
So ive just created a fork and added it as a submodule autoprofile thats imported and modifies the ast when -a command line arg is passed

you can check it out here if you'd like main...ta946:line_profiler:autoprofile

can look into including/excluding by name and the root package recursion if you think this looks promising

[Erotemic]

In this case you don't need to care about "bar" unless it is specified relative to "foo". If the user wanted to only profile "bar" they would specify --prof-mod="foo.bar", but if they wanted to specify everything in the pacakge foo they would do: --prof-mod=foo.

The mapping from module paths to names is straightforward. The mappings from module names to module paths should basically work exactly when python -c "import modname; print(modname)" works. You don't need to work around sys.path. It and the cwd define exactly what module names are allowed.

Consider this demo code:

import ubelt as ub
root = ub.Path.appdir('line_profiler/test/poc/').delete().ensuredir()
repo = (root / 'repo').ensuredir()
modpaths = {}
modpaths['script'] = (root / 'repo/script.py').touch()
modpaths['foo'] = (root / 'repo/foo').ensuredir()
modpaths['foo.__init__'] = (root / 'repo/foo/__init__.py').touch()
modpaths['foo.bar'] = (root / 'repo/foo/bar.py')
modpaths['foo.bar'].write_text('def asdf():\n    ...')
modpaths['foo.baz'] = (root / 'repo/foo/baz').ensuredir()
modpaths['foo.baz.__init__'] = (root / 'repo/foo/baz/__init__.py').touch()
modpaths['foo.baz.spam'] = (root / 'repo/foo/baz/spam.py').touch()
modpaths['foo.baz.spam'].write_text('def spamfunc():\n    ...')
modpaths['foo.baz.eggs'] = (root / 'repo/foo/baz/eggs.py').touch()
modpaths['foo.baz.eggs'].write_text('def eggfunc():\n    ...')

# Moving from module paths to module names is nearly unambiguous
# (the only catch is there **might** be another name that conflicts with it
#  but we can ignore that case in the code and just document for the user)
modnames = {}
modnames['script'] = ub.modpath_to_modname(modpaths['script'])
modnames['foo'] = ub.modpath_to_modname(modpaths['foo'])
modnames['foo.bar'] = ub.modpath_to_modname(modpaths['foo.bar'])
modnames['foo.baz'] = ub.modpath_to_modname(modpaths['foo.baz'])
modnames['foo.baz.spam'] = ub.modpath_to_modname(modpaths['foo.baz.spam'])
modnames['foo.baz.eggs'] = ub.modpath_to_modname(modpaths['foo.baz.eggs'])


# Moving from module names to paths depends heavilly on the current pythonpath
# But that's fine, the user needs to set that up if they want to run the code
# anyway. In this test explicitly pass the path because we *didnt* set it up in
# our python environment.
recon = {}
for modname in modnames.values():
    recon[modname] = ub.modname_to_modpath(modname, sys_path=[str(repo)])

modpaths_rel = ub.map_vals(lambda p: p.shrinkuser(), modpaths)
print('modpaths = {}'.format(ub.repr2(modpaths_rel, nl=1, align=':')))
print('modnames = {}'.format(ub.repr2(modnames, nl=1, align=':')))
print('recon = {}'.format(ub.repr2(recon, nl=1)))

### Getting the members of a package requires first getting the root path
### then it's just a matter of walking directories.
from xdoctest import static_analysis as static
members_from_path = {}
for modpath in modpaths.values():
    members_from_path[str(modpath.shrinkuser())] = [
        str(ub.Path(p).shrinkuser())
        for p in static.package_modpaths(modpath)
    ]

members_from_name = {}
for modname in modnames.values():
    modpath = ub.modname_to_modpath(modname, sys_path=[str(repo)])
    members_from_name[modname] = [
        ub.modpath_to_modname(p)
        for p in static.package_modpaths(modpath)
    ]

print('members_from_path = {}'.format(ub.repr2(members_from_path, nl=2)))
print('members_from_name = {}'.format(ub.repr2(members_from_name, nl=1)))

The result is:

modpaths = {
    'foo'             : Path('~/.cache/line_profiler/test/poc/repo/foo'),
    'foo.__init__'    : Path('~/.cache/line_profiler/test/poc/repo/foo/__init__.py'),
    'foo.bar'         : Path('~/.cache/line_profiler/test/poc/repo/foo/bar.py'),
    'foo.baz'         : Path('~/.cache/line_profiler/test/poc/repo/foo/baz'),
    'foo.baz.__init__': Path('~/.cache/line_profiler/test/poc/repo/foo/baz/__init__.py'),
    'foo.baz.eggs'    : Path('~/.cache/line_profiler/test/poc/repo/foo/baz/eggs.py'),
    'foo.baz.spam'    : Path('~/.cache/line_profiler/test/poc/repo/foo/baz/spam.py'),
    'script'          : Path('~/.cache/line_profiler/test/poc/repo/script.py'),
}
modnames = {
    'foo'         : 'foo',
    'foo.bar'     : 'foo.bar',
    'foo.baz'     : 'foo.baz',
    'foo.baz.eggs': 'foo.baz.eggs',
    'foo.baz.spam': 'foo.baz.spam',
    'script'      : 'script',
}
recon = {
    'foo': '/home/joncrall/.cache/line_profiler/test/poc/repo/foo',
    'foo.bar': '/home/joncrall/.cache/line_profiler/test/poc/repo/foo/bar.py',
    'foo.baz': '/home/joncrall/.cache/line_profiler/test/poc/repo/foo/baz',
    'foo.baz.eggs': '/home/joncrall/.cache/line_profiler/test/poc/repo/foo/baz/eggs.py',
    'foo.baz.spam': '/home/joncrall/.cache/line_profiler/test/poc/repo/foo/baz/spam.py',
    'script': '/home/joncrall/.cache/line_profiler/test/poc/repo/script.py',
}
members_from_path = {
    '~/.cache/line_profiler/test/poc/repo/foo': [
        '~/.cache/line_profiler/test/poc/repo/foo/bar.py',
        '~/.cache/line_profiler/test/poc/repo/foo/baz/eggs.py',
        '~/.cache/line_profiler/test/poc/repo/foo/baz/spam.py',
    ],
    '~/.cache/line_profiler/test/poc/repo/foo/__init__.py': [
        '~/.cache/line_profiler/test/poc/repo/foo/__init__.py',
    ],
    '~/.cache/line_profiler/test/poc/repo/foo/bar.py': [
        '~/.cache/line_profiler/test/poc/repo/foo/bar.py',
    ],
    '~/.cache/line_profiler/test/poc/repo/foo/baz': [
        '~/.cache/line_profiler/test/poc/repo/foo/baz/eggs.py',
        '~/.cache/line_profiler/test/poc/repo/foo/baz/spam.py',
    ],
    '~/.cache/line_profiler/test/poc/repo/foo/baz/__init__.py': [
        '~/.cache/line_profiler/test/poc/repo/foo/baz/__init__.py',
    ],
    '~/.cache/line_profiler/test/poc/repo/foo/baz/eggs.py': [
        '~/.cache/line_profiler/test/poc/repo/foo/baz/eggs.py',
    ],
    '~/.cache/line_profiler/test/poc/repo/foo/baz/spam.py': [
        '~/.cache/line_profiler/test/poc/repo/foo/baz/spam.py',
    ],
    '~/.cache/line_profiler/test/poc/repo/script.py': [
        '~/.cache/line_profiler/test/poc/repo/script.py',
    ],
}
members_from_name = {
    'foo': ['foo.bar', 'foo.baz.eggs', 'foo.baz.spam'],
    'foo.bar': ['foo.bar'],
    'foo.baz': ['foo.baz.eggs', 'foo.baz.spam'],
    'foo.baz.eggs': ['foo.baz.eggs'],
    'foo.baz.spam': ['foo.baz.spam'],
    'script': ['script'],
}

[Erotemic]

Also it is possible to import each module from the given path (This does requires that sys.path has no conflicts, but if that is the case, you have other problems):

details = []
for modpath in modpaths.values():
    modname = ub.modpath_to_modname(modpath)
    module = ub.import_module_from_path(modpath, index=0)
    details.append({
        'modpath': modpath,
        'modname': modname,
        'module': module
    })
print('details = {}'.format(ub.repr2(details, nl=2)))

Results in:

details = [
    {
        'modname': 'script',
        'modpath': Path('/home/joncrall/.cache/line_profiler/test/poc/repo/script.py'),
        'module': <module 'script' from '/home/joncrall/.cache/line_profiler/test/poc/repo/script.py'>,
    },
    {
        'modname': 'foo',
        'modpath': Path('/home/joncrall/.cache/line_profiler/test/poc/repo/foo'),
        'module': <module 'foo' from '/home/joncrall/.cache/line_profiler/test/poc/repo/foo/__init__.py'>,
    },
    {
        'modname': 'foo',
        'modpath': Path('/home/joncrall/.cache/line_profiler/test/poc/repo/foo/__init__.py'),
        'module': <module 'foo' from '/home/joncrall/.cache/line_profiler/test/poc/repo/foo/__init__.py'>,
    },
    {
        'modname': 'foo.bar',
        'modpath': Path('/home/joncrall/.cache/line_profiler/test/poc/repo/foo/bar.py'),
        'module': <module 'foo.bar' from '/home/joncrall/.cache/line_profiler/test/poc/repo/foo/bar.py'>,
    },
    {
        'modname': 'foo.baz',
        'modpath': Path('/home/joncrall/.cache/line_profiler/test/poc/repo/foo/baz'),
        'module': <module 'foo.baz' from '/home/joncrall/.cache/line_profiler/test/poc/repo/foo/baz/__init__.py'>,
    },
    {
        'modname': 'foo.baz',
        'modpath': Path('/home/joncrall/.cache/line_profiler/test/poc/repo/foo/baz/__init__.py'),
        'module': <module 'foo.baz' from '/home/joncrall/.cache/line_profiler/test/poc/repo/foo/baz/__init__.py'>,
    },
    {
        'modname': 'foo.baz.spam',
        'modpath': Path('/home/joncrall/.cache/line_profiler/test/poc/repo/foo/baz/spam.py'),
        'module': <module 'foo.baz.spam' from '/home/joncrall/.cache/line_profiler/test/poc/repo/foo/baz/spam.py'>,
    },
    {
        'modname': 'foo.baz.eggs',
        'modpath': Path('/home/joncrall/.cache/line_profiler/test/poc/repo/foo/baz/eggs.py'),
        'module': <module 'foo.baz.eggs' from '/home/joncrall/.cache/line_profiler/test/poc/repo/foo/baz/eggs.py'>,
    },
]

[ta946]

i created the demo you mentioned and used ubelt's modname_to_modpath & modpath_to_modname and xdoctest package_modpaths. they work like magic. went through the code and has alot more checks and considerations i didn't think about!

i also figured out how to get the modnames for all the different import patterns using ast.

so my next steps are:

1. run prof_mod in modname_to_modpath to get modpath (i do this to avoid having to figure out whether the input is modname or modpath, just convert all inputs to modpath makes it easier)
2. if no result, assume prof_mod is modpath
3. run modpath in package_modpaths to get submodules
4. run all modpaths in modpath_to_modname

this will convert all prof_mod inputs into modnames

now the problem i need to solve is mapping the modnames relative to the way the script.py imports them
ill share an example that hopefully explains what i mean

prof-mod converted to modnames
'foo.bar':                   'repo.foo.bar',
'foo.baz':                   'repo.foo.baz',
'foo.baz.eggs':              'repo.foo.baz.eggs',
'foo.baz.spam':              'repo.foo.baz.spam',
'pipinstall_module.foo.bar': 'pipinstall_module.foo.bar',
'asdf.qwer':                 'pythonpath_module.asdf.qwer',
'numpy.round':               None

/repo/script.py:
import foo
import foo.bar
import foo.baz
import foo.baz.eggs
import foo.baz.spam
import pipinstall_module.foo.bar
import asdf.qwer
from numpy import round

you can see that all the local modules in prof-mod eg: foo.bar gets converted to repo.foo.bar
but in script.py it needs to be in the format foo.bar, removing repo,
as that is the relative import path in script import foo.bar, which i will use to call profile.add_module(foo.bar)

i could probably just check the script directory and remove the repo. just wanted to know if you have a better solution

the other challenge is asdf.qwer. this is a module imported from PYTHONPATH.
the modname is pythonpath_module.asdf.qwer. but you would import it in script.py as import asdf.qwer

so again, need to know to remove pythonpath_module from pythonpath_module.asdf.qwer so i can call profile.add_module(asdf.qwer)

the challenge is differentiating between pipinstall_module and pythonpath_module
to know to remove pythonpath_module but not pipinstall_module

these happen in modpath_to_modname. any idea how to solve this?

also, numpy round does not resolve as round is not an immediate child of numpy

[Erotemic]

Just glancing at this. Not going to address everything here, but I'll start with obvious things and then come back to the rest.

• numpy.round does not resolve because it is not a module. It's a function. Actually, it's a reference to a function, which really exists in numpy.core.fromnumeric.round_ and it's even more complex than that because the actual function is a dynamically constructed function returned by array_function_dispatch.

• When you convert modname_to_modpath and there is no result, you should check that the module path actually exists. I suppose there is a case for archived modules that live inside zipfiles, but I think I'd like to ignore that. We can restrict the scope of this to only simple file-based modules that exist on disk.

• I'm not sure why foo.bar is converted to repo.foo.bar. The logic should stop when it sees a directory that doesn't contain an __init__.py, and if it does contain an __init__.py, then importing in a relative way should be disallowed anyway. It always needs to be specified from the root of a known module. (Note this is different than what was allowed in Python2. In Python2 you could be inside foo/baz and import spam, but in python 3 you must either do from foo.baz import spam or from . import spam where the . indicates that python should basically introspect the former statement).

What will be very important is coming up with tests that go through all of the possible cases. Doing so will serve to both write the tests necessary to integrate this into line_profiler as well as to provide an unambiguous reference point from which we can discuss issues and problems. See the ubelt tests for some example demo setups: https://github.com/Erotemic/ubelt/blob/main/tests/test_import.py See if you can encode your cases in something like that framework.

[Erotemic]

Also note, that all of the ubelt / xdoctest module parsing is entirely static. It doesn't require importing anything to resolve module names, and as such it's only able to handle straightforward declarative modules. I like this method because it avoids any potential side effects, but in this case because we are importing things anyway, it might be possible to consider dynamic methods for converting between module names and paths. This opens a can of worms though, as we have to operate in a Turing complete environment, so almost all guarantees fly out the window, but it would be possible to specify numpy.round and get an understanding of what that was in this case. I do recommend sticking to only statically specifiable things though and document that as a restriction. It will save us both headaches.

[ta946]

@Erotemic
figured out the problems. i had __init__.py files in the repo & in the pythonpath_module where they shouldn't belong.
followed your code to create the poc repo and things worked finally.

got the name mapping sorted as well as the ast profiling.

the only thing im stuck on is that line profiler will only start profiling an import if it was used inside a function that is being profiled.

ive put together all the poc code. its all in one place so its alot to go through to understand but shouldn't be too hard

to get line_profiler to profile the script, uncomment line 268 # str(modpaths['script']), # special case to profile all items

main.py

import sys
import os
import ast

import ubelt as ub
from xdoctest.static_analysis import package_modpaths

try:
    unparse = ast.unparse
except AttributeError:
    try:
        import astunparse
        unparse = astunparse.unparse
    except ModuleNotFoundError:
        unparse = None


def create_poc(dry_run=False):
    root = ub.Path.appdir('line_profiler/test/poc/')
    repo = (root / 'repo')
    modpaths = {}
    modpaths['script'] = (root / 'repo/script.py')
    modpaths['foo'] = (root / 'repo/foo')
    modpaths['foo.__init__'] = (root / 'repo/foo/__init__.py')
    modpaths['foo.bar'] = (root / 'repo/foo/bar.py')
    modpaths['foo.baz'] = (root / 'repo/foo/baz')
    modpaths['foo.baz.__init__'] = (root / 'repo/foo/baz/__init__.py')
    modpaths['foo.baz.spam'] = (root / 'repo/foo/baz/spam.py')
    modpaths['foo.baz.eggs'] = (root / 'repo/foo/baz/eggs.py')

    if not dry_run:
        root.delete().ensuredir()
        repo.ensuredir()
        modpaths['script'].touch()
        modpaths['foo'].ensuredir()
        modpaths['foo.__init__'].touch()
        modpaths['foo.bar'].touch()
        modpaths['foo.bar'].write_text('def asdf():\n    2**(1/65536)')
        modpaths['foo.baz'].ensuredir()
        modpaths['foo.baz.__init__'].touch()
        modpaths['foo.baz.spam'].touch()
        modpaths['foo.baz.spam'].write_text('def spamfunc():\n    ...')
        modpaths['foo.baz.eggs'].touch()
        modpaths['foo.baz.eggs'].write_text('def eggfunc():\n    ...')

        """different import variations to handle"""
        script_text = ub.codeblock(
            '''
            import foo # mod
            import foo.bar # py
            from foo import bar # py
            from foo.bar import asdf # fn
            import foo.baz as foodotbaz # mod
            from foo import baz as foobaz # mod
            from foo import bar, baz as baz2
            import foo.baz.eggs # py
            from foo.baz import eggs # py
            from foo.baz.eggs import eggfunc # fn
            from foo.baz.spam import spamfunc as yum # fn
            from numpy import round # fn
            @profile
            def test():
                2**65536
                # foo.bar.asdf()
            def main():
                2**65536
                foo.bar.asdf()
            main()
            test()
            foo.bar.asdf()
            ''')
        ub.writeto(modpaths['script'], script_text)

    return root,repo,modpaths

def is_path(text):
    return '/' in text.replace('\\','/')

def get_modnames_to_profile(script_file, prof_mod):
    script_directory = os.path.realpath(os.path.dirname(script_file))
    new_sys_path = [script_directory]+sys.path

    modnames_to_profile = []
    for mod in prof_mod:
        modpath = ub.util_import.modname_to_modpath(mod, sys_path=new_sys_path)
        if modpath is None:
            if not os.path.exists(mod):
                if is_path(mod):
                    # print('^ mod file does not exist')
                    continue
                modnames_to_profile.append(mod)
                continue
            modpath = mod

        try:
            modname = ub.util_import.modpath_to_modname(modpath)
        except ValueError:
            continue
        if modname not in modnames_to_profile:
            modnames_to_profile.append(modname)

        for submod_path in package_modpaths(modpath):
            submod_name = ub.util_import.modpath_to_modname(submod_path)
            if submod_name not in modnames_to_profile:
                modnames_to_profile.append(submod_name)
    return modnames_to_profile

def ast_get_imports_from_tree(tree):
    modules = []
    modules_alias = []
    modules_index = []
    for idx,node in enumerate(tree.body):
        if isinstance(node, ast.Import):
            for name in node.names:
                modname = name.name
                if modname not in modules:
                    modules.append(modname)
                    modules_alias.append(name.asname)
                    modules_index.append(idx)
        elif isinstance(node, ast.ImportFrom):
            for name in node.names:
                modname = node.module+'.'+name.name
                alias = name.asname or name.name
                if modname not in modules:
                    modules.append(modname)
                    modules_alias.append(alias)
                    modules_index.append(idx)
    return modules, modules_alias, modules_index

def ast_get_imports(script_file):
    with open(script_file,'r') as f:
        script_text = f.read()
    tree = ast.parse(script_text, filename=script_file)
    modules, modules_alias, modules_index = ast_get_imports_from_tree(tree)
    return modules, modules_alias, modules_index, tree

def get_names_to_profile(modnames_to_profile, modules, modules_alias):
    chosen_indexes = []
    mod_imports_added = []
    names_to_profile = []

    """match prof modules when submodule or function(hence the extra .) imported"""
    for i,mod_import in enumerate(modules):
        if mod_import not in modnames_to_profile and mod_import.rsplit('.',1)[0] not in modnames_to_profile:
            continue
        name = modules_alias[i] or mod_import
        if mod_import in mod_imports_added:
            continue
        mod_imports_added.append(mod_import)
        names_to_profile.append(name)
        chosen_indexes.append(i)

    # """match prof submodules when parent imported. doesnt work."""
    # for modname in modnames_to_profile:
    #     # parent_name = modname.rsplit('.',1)[0]
    #     for i,mod_import in enumerate(modules):
    #         if len(modname) <= len(mod_import) or not modname.startswith(mod_import) or modname[len(mod_import)] != '.':
    #         # if mod_import != parent_name or modname[len(parent_name)] != '.':
    #             continue
    #         alias = modules_alias[i]
    #         if alias:
    #             name = alias+modname[len(mod_import):]
    #         else:
    #             name = modname
    #         if mod_import not in mod_imports_added:
    #             mod_imports_added.append(mod_import)
    #             names_to_profile.append(name)
    #             chosen_indexes.append(i)
    #         break
    return names_to_profile, chosen_indexes

def ast_profile_module(modname, attr='add_module'):
    func = ast.Attribute(value=ast.Name(id='profile', ctx=ast.Load()), attr=attr, ctx=ast.Load())
    names = modname.split('.')
    value = ast.Name(id=names[0], ctx=ast.Load())
    for name in names[1:]:
        value = ast.Attribute(attr=name,ctx=ast.Load(),value=value)
    expr = ast.Expr(value=ast.Call(func=func, args=[value], keywords=[]))
    return expr

class ASTProfile(ast.NodeTransformer):
    def __init__(self, profile_imports=False, profiled_imports=None):
        self._profile_imports = bool(profile_imports)
        self._profiled_imports = profiled_imports if profiled_imports is not None else []

    def visit_FunctionDef(self, node):
        if 'profile' not in (d.id for d in node.decorator_list):
            node.decorator_list.append(ast.Name(id='profile', ctx=ast.Load()))
        return self.generic_visit(node)

    def visit_Import(self, node):
        if not self._profile_imports:
            return self.generic_visit(node)
        visited = [self.generic_visit(node)]
        for names in node.names:
            node_name = names.name if names.asname is None else names.asname
            if node_name in self._profiled_imports:
                continue
            self._profiled_imports.append(node_name)
            expr = ast_profile_module(node_name)
            visited.append(expr)
        return visited

    # """
    # needs modified line_profiler.py with add_imported_function_or_module()
    # not needed for this test, will add on release
    # """
    # def visit_ImportFrom(self, node):
    #     if not self._profile_imports:
    #         return self.generic_visit(node)
    #     visited = [self.generic_visit(node)]
    #     for names in node.names:
    #         node_name = names.name if names.asname is None else names.asname
    #         if node_name in self._profiled_imports:
    #             continue
    #         self._profiled_imports.append(node_name)
    #         expr = ast_profile_module(node_name, attr='add_imported_function_or_module')
    #         visited.append(expr)
    #     return visited

def argsort(seq):
    return sorted(range(len(seq)), key=seq.__getitem__)

def profile_ast_tree(tree, names_to_profile, chosen_indexes, modules_index, profile_script=False, profile_script_imports=False):
    argsort_indexes = reversed(argsort(chosen_indexes))
    for i in argsort_indexes:
        index = chosen_indexes[i]
        name = names_to_profile[i]
        idx = modules_index[index]
        expr = ast_profile_module(name)
        tree.body.insert(idx+1,expr)
    if profile_script:
        tree = ASTProfile(profile_imports=profile_script_imports, profiled_imports=names_to_profile).visit(tree)
    ast.fix_missing_locations(tree)
    return tree


def main():
    root,repo,modpaths = create_poc(dry_run=False)

    script_file = str(modpaths['script'])

    """separate from prof_mod, profile all imports in script"""
    profile_script_imports = False


    """modnames to profile"""
    modnames = [
        # 'fool', # doesn't exist
        # 'foo',
        'foo.bar',
        # 'foo.baz',
        # 'foo.baz.eggs',
        # 'foo.baz.spam',
        # 'numpy.round',
    ]
    """modpaths to profile"""
    modpaths = [
        # str(root),
        # str((repo / 'fool')), # doesn't exist
        # str(modpaths['foo']),
        # str(modpaths['foo.__init__']),
        # str(modpaths['foo.bar']),
        # str(modpaths['foo.baz']),
        # str(modpaths['foo.baz.__init__']),
        # str(modpaths['foo.baz.spam']),
        # str(modpaths['foo.baz.eggs']),
        # str(modpaths['script']), # special case to profile all items
    ]

    prof_mod = modnames+modpaths
    # prof_mod = modpaths
    # prof_mod = modnames


    modnames_to_profile = get_modnames_to_profile(script_file, prof_mod)

    print('modnames_to_profile')
    print('='*10)
    print('\n'.join(modnames_to_profile))

    modules, modules_alias, modules_index, tree = ast_get_imports(script_file)
    print('\nscript imports')
    print('='*10)
    for idx,mod,alias in zip(modules_index,modules, modules_alias):
        print(idx,mod,'as '+alias if alias else '')

    names_to_profile, chosen_indexes = get_names_to_profile(modnames_to_profile, modules, modules_alias)

    print('\nnames to profile')
    print('='*10)
    for idx,name in zip(chosen_indexes,names_to_profile):
        print(idx,name)

    script_file_realpath = os.path.realpath(script_file)
    profile_script = False
    for mod in prof_mod:
        if os.path.realpath(mod) == script_file_realpath:
            profile_script = True
            break

    tree_profiled = profile_ast_tree(tree, names_to_profile, chosen_indexes, modules_index, profile_script=profile_script, profile_script_imports=profile_script_imports)

    if unparse is not None:
        script_parsed_text = unparse(tree_profiled)
        with open(os.path.join(root,'script_profiled.py'),'w') as f:
            f.write(script_parsed_text)

    import builtins
    __file__ = script_file
    __name__ = '__main__'
    script_directory = os.path.realpath(os.path.dirname(script_file))
    sys.path.insert(0, script_directory)
    import line_profiler
    prof = line_profiler.LineProfiler()
    builtins.__dict__['profile'] = prof
    ns = locals()
    exec(compile(tree_profiled, script_file, 'exec'), ns, ns)

    print('\nprofiled')
    print('='*10)
    prof.print_stats(output_unit=1e-6, stripzeros=True, stream=sys.stdout)

if __name__ == '__main__':
    main()

[Erotemic]

Sorry for the long wait time on this. I had lost track of it, until you reminded me in #188.

Overall this is a solid POC, and I was able to run it.

the only thing im stuck on is that line profiler will only start profiling an import if it was used inside a function that is being profiled.

Can you expand on this? A function must be run in order to ever be profiled. I tried removing the profile from the test function in repo/script.py but it still reported its results. Can you show an example of the behavior?

Overall this looks this has all the pieces needed to start a PR that integrates this feature. So now we need to organize it as a module with a maintainable API so that we can add a CLI argument to the main script to enable the behavior with minimal code.

Some thoughts and comments:

• I think the scope of ast_get_imports_from_tree might be more clear as a class that inherits from NodeVisitor, which is the standard way to get information about specific types of nodes. Grouping other functionality like get_names_to_profile into that class might make sense.

• The result of the visitor might then be passed to the ASTProfile transformer, where perhaps the functionality in profile_ast_tree would make sense as a class method (the idea being grouping the functions related to visiting / transforming into their respective classes).

• A good new module name to put the AST related code might be line_profiler/auto_decorator.py or line_profiler/ast_decorator.py. (Edit, looking at your other message, autoprofile.py is also a good name).

• The functions from xdoctest / ubelt will have to be vendored into this library to minimize the dependency surface (as much as I would love to have them piggy back on the downloads from this repo). Fortunately I've written a library called liberator to make this fairly easy. I may have to publish my development branch to make the following code work, but after I do that this code crawls xdoctest to extract only the minimum necessary code that needs to be copied over (probably into a new submodule called line_profiler/utils/util_import.py).

import liberator
lib = liberator.Liberator()

from xdoctest.static_analysis import package_modpaths
from xdoctest.utils import util_import

lib.add_dynamic(package_modpaths)
lib.add_dynamic(util_import.modname_to_modpath)
lib.add_dynamic(util_import.modpath_to_modname)
lib.expand(['xdoctest'])

print(lib.current_sourcecode())

[ta946]

no worries, i figured that was the case then forgot myself.

Can you expand on this? A function must be run in order to ever be profiled. I tried removing the profile from the test function in repo/script.py but it still reported its results. Can you show an example of the behavior?

The poc is set up to highlight this already
So we have foo.bar in prof_mod as a module to profile
foo.bar.asdf() gets called in main() and in the global space
but because we are not profiling main or the whole script file (since # str(modpaths['script']), # special case to profile all items is commented out) it will not profile foo.bar.asdf()

This might be the desired behaviour, but to me it seems to contradict the expectation that if i add foo.bar as a module to profile in an "autoprofiler" then It should automatically profile the functions under foo.bar ie: asdf() when it gets used in my script without having to explicitly profile main(), or anywhere else it is run in the script.
Eg: I only want to profile asdf() but not main()

• I think the scope of ast_get_imports_from_tree might be more clear as a class that inherits from NodeVisitor, which is the standard way to get information about specific types of nodes.

the reason I didn't use nodevisitor and instead traverse the tree body manually is because i need to know the indexes of the Import statements at the top level in the tree. I then use those indexes to add the profiling decorators/code after the statements to ensure i am not trying to call profile(module) before the module is imported.
Couldn't figure out how to do that with nodevisitor and its visitors in a non-convoluted way.

• The result of the visitor might then be passed to the ASTProfile transformer

same thing here. i need to add the profiling decorators/code at those specific indexes, not sure how to do that with nodetransformer.
Another way to look at what I want to do, without relying on index, is that i need to add the profiling code immediately after the respective import statements.

ASTProfile that uses nodetransformer is actually a separate workflow than prof_mod that profiles everything in the script (as opposed to select items) which is only used when adding the script path # str(modpaths['script']), # special case to profile all items to prof_mod

• The functions from xdoctest / ubelt will have to be vendored into this library to minimize the dependency surface

I thought about that aswell. I'm averse to having 2 copies of the same code, makes maintaining them a headache. but I was sure you wouldn't want to add them as a dependancy. So I guess its a compromise we would have to deal with.

I'm travelling to europe tomorrow for a month so I won't be able to do much coding, but i will be able to reply

[ta946]

@Erotemic I'm back now so I can put some time aside to finishing this. I'm not sure how notifications work wrt replies so tagging you just incase you didn't see the previous reply

[zig1000]

Thanks for working on this! As an aside I thought I'd mention that I have a hacky local tool that adds a similar feature of profiling CLI-specified functions without needing decorators, but doesn't work unless the called module has a main (not too bad a restriction and dodges AST-parsing, but still). So this feature being native would be a big help.

Here's the code for my quick-and-dirty tool, for posterity, though I think it's strictly worse than the native implementation you're working on:

"""Profile the given unmodified python module (normally line_profiler requires adding @profile decorators).
The module must expose a main() function, which will be executed.
"""

import argparse
import importlib
import inspect
import sys

import line_profiler


if __name__ == '__main__':
    parser = argparse.ArgumentParser(
         usage="%(prog)s [-h] functions module [your module's args]",
         description="Profile the given module's function(s) line-by-line.\n"
                     "Unlike pypi's line-profiler, does not require adding decorators.",
         formatter_class=argparse.RawTextHelpFormatter)
    parser.add_argument('functions',
        help="Function(s) to be profiled from the given module, comma-separated.\n"
             "Specify each relative to the module.\n"
             "E.g.: `my_public_function,MyPublicClass.my_method,my_submodule.MyInternalClass.my_other_method`.")
    parser.add_argument('module',
        help="Py module to profile. module.main() will be executed.\n"
             "If it is a package and doesn't expose a `main`, but its __main__.py has one, that will be used instead.")

    # Assume any unknown arguments are intended for the module being profiled
    args, unused_args = parser.parse_known_args()
    sys.argv = sys.argv[:1] + unused_args

    # Import the module, sanitizing if it was passed as a file name
    module = importlib.import_module(args.module.strip('.py'))

    # Set up the line profiler and add the specified module functions to it
    profiler = line_profiler.LineProfiler()
    for fn in args.functions.split(','):
        # Handle arbitrary length 'a.b.c' specification, e.g. 'some_submodule.SomeClass.some_method' from module
        obj = module
        for attr in fn.split('.'):
            try:
                obj = getattr(obj, attr)
            except AttributeError as e:
                raise AttributeError(f"Couldn't find {fn}, maybe you need <submodule>.{fn} based on its file?") from e
        profiler.add_function(obj)

    # If `main` is missing, do a last-ditch check if the module is a package with __main__.py and check it instead
    if not hasattr(module, 'main'):
        try:
            main_module = importlib.import_module(args.module.strip('.py') + '.__main__')
            if hasattr(main_module, 'main'):
                module = main_module
        except ModuleNotFoundError:
            sys.tracebacklimit = 0  # Hide traceback to prett-ify what will be the most common error
            raise AttributeError(f"Module `{args.module}` has no function `main`. Please move your __name__ == '__main__' block into one.") from None  # Hide the failed __main__.py check via `from None`

    # Run with asyncio if main() is async
    if inspect.iscoroutinefunction(module.main):
        import asyncio
        profiler.run('asyncio.run(module.main())')
    else:
        profiler.run('module.main()')

    profiler.print_stats()

Sample usage: python -m line_profiler_helper function1,my_submodule.MyClass.my_method,function3 mymodule [my module's args]

[Erotemic]

This assumes you have a function named main in your module?

[zig1000]

Yes, which is why a native AST-based implementation without this restriction would be a nice improvement.

[zig1000]

The [[submodule.]Class.]method specification syntax being all that's required is perhaps the nicest part of this hack.

[ta946]

The [[submodule.]Class.]method specification syntax being all that's required is perhaps the nicest part of this hack.

I like the idea because it avoids having to modify the code, you just specify the functions to profile in cli.

My main goal was also to profile without modifying the code, but also to profile the whole script without needing to specify any functions. But you can still choose to specify them to reduce the scope of the profiling.

[zig1000]

Yes, since my main use case has too much code to look at at once, but isn't too slow to run, I typically just profile one function at a time and use the results to decide what to inspect next, though of course for very slow scripts being able to get multiple at once is pretty desirable.

Full-script profiling is also definitely nice (especially for small single-file scripts), but I'd definitely want to see this feature have a function-specified implementation like my tool too, or even just the single-function version, if the CLI argument-juggling were a hangup.

[ta946]

@Erotemic i've solved the issue i was referring to before, where asdf() was not being profiled unless it was inside a function that was profiled. So I'm happy to go ahead and begin cleaning things up to prepare for a PR.

I've created a new branch in my fork if you want to take a look at the code
main...ta946:line_profiler:autoprofile_prof_mod

Theres a few things I wanted to bring up.

1. The code definitely needs abit of refactoring. Whether its rearranging the functions or creating a class to encapsulate the loose functions. I'd need your feedback on this
2. All the new functionality is under line_profiler/autoprofile. I was using pip install . to install my local version but I couldn't get it to copy the folder to site-packages (I'm using miniconda). not sure what needs to be changed in the setup.py
3. I wasn't sure how to deal with the imports since kernprof imports autoprofile so in autoprofile.py and profmod_extractor.py the imports are from line_profiler.autoprofile.---. i believe they should be a relative imports instead?
4. To allow profiling functions, modules and classes, I had to create a new method to extend LineProfiler add_imported_function_or_module. To avoid modifying line_profiler.py and to keep this separate as a submodule, autoprofile.py calls extend_line_profiler_import_profiling which binds the method to the profiler instance. If we want to avoid this, we could just add the method to the LineProfiler class.
5. to make the profiler work for asdf() when its not in a profiled function. I had to add self.enable_by_count() in the function i just mentioned add_imported_function_or_module .
   The reason for adding it was that usually a function gets decorated with profile(func/module) and the wrapper calls enable_by_count when the function is run which activates the profiling for it. but profile.add_module or my modified version add_imported_function_or_module only adds the function name to LineProfiler but never activates the profiling for it as it doesn't call enable_by_count, so I manually call it. I'm not 100% sure if there will be any side effects by doing that.
6. Atm, im using the cli arg -p --prof-mod as a comma-separated list. we can change that to whatever we like
7. i also added another arg -m --prof-imports which only works when you profile the full script by adding the script path to --prof-mod. this option will profile the imported modules automatically (but only the top level functions, not recursively)
8. i moved the poc code to poc.py

[Erotemic]

I think your code is at a point where it is ready for a PR. Start that and we can continue the technical discussion there.

For avoiding nargs="+", your concern makes sense. It is possible to use -- to distinguish positional arguments from a multi-valued argument, but for now I think a comma separated list is fine. We can keep it simple for now and expand it if we need to. Let's focus on getting the core functionality right.

@Theelx if you have time to also take a look at Point 4, that would help me out. Do you see any side effects of calling enable_by_count in this way at import time, and not call time?

[ta946]

should I start the PR from my fork or create a new branch here? Also should the target be main branch?

[Theelx]

My very basic analysis of the code indicates that calling enable_by_count at import time won't matter. The only potential side effect I can think of is that wrappers that programmatically call enable_by_count will cause duplicated line timings, but that should be at a very small scale since enable_by_count is only meant to be called once per function iirc. It might be worth looking into making the function idempotent.

I will say that I'm on my phone right now so I can't do a full analysis, I'll update this when I get on my computer.

[Erotemic]

should I start the PR from my fork or create a new branch here? Also should the target be main branch?

Yes, use your fork. Just start a new PR where you request that your branch is merged into main on this repo (click pull request, and if it doesn't show both repos, click compare across forks, then find your branch for your repo and main for this one). Also ensure you are rebased on the latest main.

[ta946]

#201