Pydeps: Support compiled python filename

[ZiqianXu]

Hi pydeps team,

Does pydeps support analyzing pyc files?
I got the following error when set fname to compiled python file:

pydeps HelloWorld.pyc --show-deps --max-bacon 1 --pylib --no-output --include-missing 
Traceback (most recent call last):
  File "/usr/local/bin/pydeps", line 8, in <module>
    sys.exit(pydeps())
             ^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydeps/pydeps.py", line 175, in pydeps
    return _pydeps(inp, **_args)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydeps/pydeps.py", line 39, in _pydeps
    dep_graph = py2depgraph.py2dep(trgt, **kw)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydeps/py2depgraph.py", line 222, in py2dep
    mf.run_script(dummy.fname)
  File "/usr/local/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/modulefinder.py", line 153, in run_script
    self.load_module('__main__', fp, pathname, stuff)
  File "/usr/local/lib/python3.11/site-packages/pydeps/py2depgraph.py", line 145, in load_module
    module = mf27.ModuleFinder.load_module(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydeps/mf27.py", line 58, in load_module
    co = compile(
         ^^^^^^^^
ValueError: source code string cannot contain null bytes

I tried with with both py2 and py3 compiled file, both did not work.
_PY_COMPILED seems to be supported in mf27.py, did I do something wrong here?

Thanks for any help on this!

[thebjorn]

Hi @ZiqianXu and thank you for your interest in pydeps.

The python stdlib module modulefinder has hardcoded PY_SOURCE in the initial call

    def run_script(self, pathname):
        self.msg(2, "run_script", pathname)
        with io.open_code(pathname) as fp:
            stuff = ("", "rb", _PY_SOURCE)
            self.load_module('__main__', fp, pathname, stuff)

I've overriden this now to allow for .pyc files as well.

The following works for me with v.1.12.17:

$> ls -F
a/
$> cat a/b.py
import c
$> cat a/c.py
<empty>
$> python -m compileall a
$> mkdir b
$> cp a/__pycache__/b.cpython-38.pyc b/b.pyc
$> cp a/__pycache__/c.cpython-38.pyc b/c.pyc
$>  pydeps b\b.pyc --no-show --show-deps
{
    "b.pyc": {
        "bacon": 0,
        "imports": [
            "c"
        ],
        "name": "b.pyc",
        "path": null
    },
    "c": {
        "bacon": 1,
        "imported_by": [
            "b.pyc"
        ],
        "name": "c",
        "path": "/srv/tmp/pydeps194/b/c.pyc"
    }
}

[ZiqianXu]

Hi @thebjorn , thanks for the replay and quick fix! I'm able to use pydeps to analyze pyc file from my end as well!