Find dependencies in a python source/script

You can tokenize Python source code using the tokenize module from the standard library. This will allow you to find all variable names used in the script.

Now suppose we define a "non-dependency" as any variable name that comes immediately before an = sign. Then, depending on how simple your script code really is (see the Caveats below), you may be able to determine the variable names which are not non-dependencies this way:

import tokenize
import io
import token
import collections
import keyword

kwset = set(keyword.kwlist)
class Token(collections.namedtuple('Token', 'num val start end line')):
    @property
    def name(self):
        return token.tok_name[self.num]

source = '''
C = A+B
D = C * 4
'''

lastname = None
names = set()
not_dep = set()
for tok in tokenize.generate_tokens(io.BytesIO(source).readline):
    tok = Token(*tok)
    print(tok.name, tok.val)
    if tok.name == 'NAME':
        names.add(tok.val)
        lastname = tok.val
    if tok.name == 'OP' and tok.val == '=':
        not_dep.add(lastname)

print(names)
# set(['A', 'C', 'B', 'D'])
print(not_dep)
# set(['C', 'D'])

deps = dict.fromkeys(names - not_dep - kwset, 1)
print(deps)
# {'A': 1, 'B': 1}

Caveats:

• If your scripts contain statements other than simple assignments, then names may become populated with undesired variable names. For example,

  import numpy
  

  would add both 'import' and 'numpy' to the set names.

• If your script contains an assignment that makes use of left-hand side tuple unpacking, such as

  E, F = 1, 2
  

  then the naive code above will only recognize that F is not a dependency.