我想以编程方式编辑python源代码。基本上我想读取一个.py文件,生成AST,然后写回修改后的python源代码(即另一个.py文件)。
有一些方法可以使用标准的python模块(如ast或compiler)来解析/编译python源代码。但是,我认为它们都不支持修改源代码的方法(例如删除这个函数声明),然后再写回修改的python源代码。
更新:我想这样做的原因是我想为python写一个突变测试库,主要是通过删除语句/表达式,重新运行测试,看看有什么破坏。
我想以编程方式编辑python源代码。基本上我想读取一个.py文件,生成AST,然后写回修改后的python源代码(即另一个.py文件)。
有一些方法可以使用标准的python模块(如ast或compiler)来解析/编译python源代码。但是,我认为它们都不支持修改源代码的方法(例如删除这个函数声明),然后再写回修改的python源代码。
更新:我想这样做的原因是我想为python写一个突变测试库,主要是通过删除语句/表达式,重新运行测试,看看有什么破坏。
当前回答
您可能不需要重新生成源代码。当然,这对我来说有点危险,因为您实际上并没有解释为什么您认为需要生成一个充满代码的.py文件;但是:
If you want to generate a .py file that people will actually use, maybe so that they can fill out a form and get a useful .py file to insert into their project, then you don't want to change it into an AST and back because you'll lose all formatting (think of the blank lines that make Python so readable by grouping related sets of lines together) (ast nodes have lineno and col_offset attributes) comments. Instead, you'll probably want to use a templating engine (the Django template language, for example, is designed to make templating even text files easy) to customize the .py file, or else use Rick Copeland's MetaPython extension. If you are trying to make a change during compilation of a module, note that you don't have to go all the way back to text; you can just compile the AST directly instead of turning it back into a .py file. But in almost any and every case, you are probably trying to do something dynamic that a language like Python actually makes very easy, without writing new .py files! If you expand your question to let us know what you actually want to accomplish, new .py files will probably not be involved in the answer at all; I have seen hundreds of Python projects doing hundreds of real-world things, and not a single one of them needed to ever writer a .py file. So, I must admit, I'm a bit of a skeptic that you've found the first good use-case. :-)
更新:既然你已经解释了你要做的事情,我还是想做AST手术。您可能希望通过删除整个语句而不是删除文件中的行(这可能导致半条语句,并简单地使用SyntaxError终止)来进行更改—还有什么地方比在AST中更好地做到这一点呢?
其他回答
Pythoscope对它自动生成的测试用例执行此操作,就像python 2.6的2to3工具一样(它转换python 2。X源代码转换为python 3。x源)。
这两个工具都使用lib2to3库,它是python解析器/编译器机制的实现,可以在从source -> AST -> source循环绊倒源代码时保留源代码中的注释。
如果您想进行更多的重构(如转换),rope项目可以满足您的需求。
ast模块是另一个选择,还有一个关于如何将语法树“解解析”回代码的旧示例(使用解析器模块)。但是ast模块在对随后转换为代码对象的代码进行ast转换时更有用。
红男爵计划也可能是个不错的选择(泽维尔·康贝尔)
内置ast模块似乎没有转换回源代码的方法。但是,这里的codegen模块为ast提供了一个漂亮的打印机,使您能够这样做。 如。
import ast
import codegen
expr="""
def foo():
print("hello world")
"""
p=ast.parse(expr)
p.body[0].body = [ ast.parse("return 42").body[0] ] # Replace function body with "return 42"
print(codegen.to_source(p))
这将打印:
def foo():
return 42
请注意,您可能会丢失确切的格式和注释,因为这些没有保留。
但是,您可能不需要这样做。如果您所需要的只是执行替换的AST,那么只需在AST上调用compile()并执行结果代码对象即可。
我们也有类似的需求,这里的其他答案并没有解决这个问题。因此,我们为此创建了一个库ASTTokens,它采用AST或astroid模块生成的AST树,并用原始源代码中的文本范围标记它。
它不直接修改代码,但在上面添加代码并不难,因为它会告诉您需要修改的文本范围。
例如,这将在WRAP(…)中包装一个函数调用,保留注释和其他内容:
example = """
def foo(): # Test
'''My func'''
log("hello world") # Print
"""
import ast, asttokens
atok = asttokens.ASTTokens(example, parse=True)
call = next(n for n in ast.walk(atok.tree) if isinstance(n, ast.Call))
start, end = atok.get_text_range(call)
print(atok.text[:start] + ('WRAP(%s)' % atok.text[start:end]) + atok.text[end:])
生产:
def foo(): # Test
'''My func'''
WRAP(log("hello world")) # Print
希望这能有所帮助!
您可能不需要重新生成源代码。当然,这对我来说有点危险,因为您实际上并没有解释为什么您认为需要生成一个充满代码的.py文件;但是:
If you want to generate a .py file that people will actually use, maybe so that they can fill out a form and get a useful .py file to insert into their project, then you don't want to change it into an AST and back because you'll lose all formatting (think of the blank lines that make Python so readable by grouping related sets of lines together) (ast nodes have lineno and col_offset attributes) comments. Instead, you'll probably want to use a templating engine (the Django template language, for example, is designed to make templating even text files easy) to customize the .py file, or else use Rick Copeland's MetaPython extension. If you are trying to make a change during compilation of a module, note that you don't have to go all the way back to text; you can just compile the AST directly instead of turning it back into a .py file. But in almost any and every case, you are probably trying to do something dynamic that a language like Python actually makes very easy, without writing new .py files! If you expand your question to let us know what you actually want to accomplish, new .py files will probably not be involved in the answer at all; I have seen hundreds of Python projects doing hundreds of real-world things, and not a single one of them needed to ever writer a .py file. So, I must admit, I'm a bit of a skeptic that you've found the first good use-case. :-)
更新:既然你已经解释了你要做的事情,我还是想做AST手术。您可能希望通过删除整个语句而不是删除文件中的行(这可能导致半条语句,并简单地使用SyntaxError终止)来进行更改—还有什么地方比在AST中更好地做到这一点呢?
另一种回答建议使用密码原,它似乎已被阿斯特取代。PyPI上的astor版本(撰写本文时的版本为0.5)似乎也有点过时,因此您可以按如下方式安装astor的开发版本。
pip install git+https://github.com/berkerpeksag/astor.git#egg=astor
然后你可以使用阿斯特。to_source将Python AST转换为人类可读的Python源代码:
>>> import ast
>>> import astor
>>> print(astor.to_source(ast.parse('def foo(x): return 2 * x')))
def foo(x):
return 2 * x
我已经在Python 3.5上进行了测试。