如何在Python中将标准输出重定向到任意文件?

When a long-running Python script (e.g, web application) is started from within the ssh session and backgounded, and the ssh session is closed, the application will raise IOError and fail the moment it tries to write to stdout. I needed to find a way to make the application and modules output to a file rather than stdout to prevent failure due to IOError. Currently, I employ nohup to redirect output to a file, and that gets the job done, but I was wondering if there was a way to do it without using nohup, out of curiosity.

我已经试过了。Stdout = open('somefile', 'w'),但这似乎并没有阻止一些外部模块仍然输出到终端(或者sys. exe)。Stdout =…Line根本没有开火)。我知道它应该在我测试过的更简单的脚本上工作,但我还没有时间在web应用程序上进行测试。


import sys
sys.stdout = open('stdout.txt', 'w')

如果您想在Python脚本中进行重定向,请设置sys. exe。对文件对象的Stdout是有用的:

# for python3
import sys
with open('file', 'w') as sys.stdout:
    print('test')

一个更常见的方法是在执行时使用shell重定向(在Windows和Linux上相同):

$ python3 foo.py > file

你可以尝试更好的方法

import sys

class Logger(object):
    def __init__(self, filename="Default.log"):
        self.terminal = sys.stdout
        self.log = open(filename, "a")

    def write(self, message):
        self.terminal.write(message)
        self.log.write(message)

sys.stdout = Logger("yourlogfilename.txt")
print "Hello world !" # this is should be saved in yourlogfilename.txt

其他答案不包括您希望分叉进程共享您的新标准输出的情况。

做那件事:

from os import open, close, dup, O_WRONLY

old = dup(1)
close(1)
open("file", O_WRONLY) # should open on 1

..... do stuff and then restore

close(1)
dup(old) # should dup to 1
close(old) # get rid of left overs

引用自PEP 343 -“with”语句(添加了import语句):

临时重定向标准输出:

import sys
from contextlib import contextmanager
@contextmanager
def stdout_redirected(new_stdout):
    save_stdout = sys.stdout
    sys.stdout = new_stdout
    try:
        yield None
    finally:
        sys.stdout = save_stdout

用途如下:

with open(filename, "w") as f:
    with stdout_redirected(f):
        print "Hello world"

当然,这不是线程安全的,但手动执行相同的操作也不是。在单线程程序(例如脚本)中,这是一种流行的做事方式。

用其他语言(例如C)编写的程序必须使用特殊的魔法(称为双分叉)来明确地与终端分离(并防止僵尸进程)。所以,我认为最好的解决办法是模仿他们。

重新执行程序的一个好处是,你可以在命令行上选择重定向,例如/usr/bin/python mycoolscript.py 2>&1 1>/dev/null

查看这篇文章了解更多信息:在创建守护进程时执行双fork的原因是什么?

Python 3.4+中有contextlib.redirect_stdout()函数:

from contextlib import redirect_stdout

with open('help.txt', 'w') as f:
    with redirect_stdout(f):
        print('it now prints to `help.text`')

它类似于:

import sys
from contextlib import contextmanager

@contextmanager
def redirect_stdout(new_target):
    old_target, sys.stdout = sys.stdout, new_target # replace sys.stdout
    try:
        yield new_target # run some code with the replaced stdout
    finally:
        sys.stdout = old_target # restore to the previous value

可以在早期的Python版本中使用。后一个版本是不可重用的。如果需要,它可以成为一个。

它不会在文件描述符级别重定向标准输出,例如:

import os
from contextlib import redirect_stdout

stdout_fd = sys.stdout.fileno()
with open('output.txt', 'w') as f, redirect_stdout(f):
    print('redirected to a file')
    os.write(stdout_fd, b'not redirected')
    os.system('echo this also is not redirected')

B 'not重定向'和'echo this also is not重定向'不会重定向到output.txt文件。

要在文件描述符级别重定向,可以使用os.dup2():

import os
import sys
from contextlib import contextmanager

def fileno(file_or_fd):
    fd = getattr(file_or_fd, 'fileno', lambda: file_or_fd)()
    if not isinstance(fd, int):
        raise ValueError("Expected a file (`.fileno()`) or a file descriptor")
    return fd

@contextmanager
def stdout_redirected(to=os.devnull, stdout=None):
    if stdout is None:
       stdout = sys.stdout

    stdout_fd = fileno(stdout)
    # copy stdout_fd before it is overwritten
    #NOTE: `copied` is inheritable on Windows when duplicating a standard stream
    with os.fdopen(os.dup(stdout_fd), 'wb') as copied: 
        stdout.flush()  # flush library buffers that dup2 knows nothing about
        try:
            os.dup2(fileno(to), stdout_fd)  # $ exec >&to
        except ValueError:  # filename
            with open(to, 'wb') as to_file:
                os.dup2(to_file.fileno(), stdout_fd)  # $ exec > to
        try:
            yield stdout # allow code to be run with the redirected stdout
        finally:
            # restore stdout to its previous value
            #NOTE: dup2 makes stdout_fd inheritable unconditionally
            stdout.flush()
            os.dup2(copied.fileno(), stdout_fd)  # $ exec >&copied

如果使用stdout_redirected()而不是redirect_stdout(),同样的例子现在也可以工作:

import os
import sys

stdout_fd = sys.stdout.fileno()
with open('output.txt', 'w') as f, stdout_redirected(f):
    print('redirected to a file')
    os.write(stdout_fd, b'it is redirected now\n')
    os.system('echo this is also redirected')
print('this is goes back to stdout')

只要stdout_redirect()上下文管理器处于活动状态,以前在stdout上打印的输出现在就会转到output.txt。

注意:stdout.flush()不刷新 Python 3上的C stdio缓冲区,其中I/O直接在read()/write()系统调用上实现。要刷新所有打开的C stdio输出流,如果某些C扩展使用基于stdio的I/O,可以显式调用libc.fflush(None):

try:
    import ctypes
    from ctypes.util import find_library
except ImportError:
    libc = None
else:
    try:
        libc = ctypes.cdll.msvcrt # Windows
    except OSError:
        libc = ctypes.cdll.LoadLibrary(find_library('c'))

def flush(stream):
    try:
        libc.fflush(None)
        stream.flush()
    except (AttributeError, ValueError, IOError):
        pass # unsupported

你可以使用stdout参数重定向其他流,而不仅仅是sys。标准输出,例如,合并sys。Stderr和sys.stdout:

def merged_stderr_stdout():  # $ exec 2>&1
    return stdout_redirected(to=sys.stdout, stdout=sys.stderr)

例子:

from __future__ import print_function
import sys

with merged_stderr_stdout():
     print('this is printed on stdout')
     print('this is also printed on stdout', file=sys.stderr)

注:stdout_redirects()混合缓冲I/O (sys。通常是stdout)和非缓冲I/O(直接对文件描述符进行操作)。注意,可能存在缓冲问题。

要回答这个问题,您的编辑:您可以使用Python -daemon来守护脚本,并使用日志记录模块(如@erikb85所建议的),而不是打印语句,并仅为您现在使用nohup运行的长时间运行的Python脚本重定向stdout。

根据这个答案:https://stackoverflow.com/a/5916874/1060344,这是我在我的一个项目中使用的另一种方法。无论你用什么替换sys。Stderr或sys。stderr/ Stdout被用在一些不受你控制的库中,你必须确保替换符合文件接口,特别是当你这样做的时候。该库可能正在使用文件对象的其他方法。

以这种方式检查,我仍然让所有内容都执行stderr/stdout(或任何文件),并使用Python的日志记录功能将消息发送到日志文件(但你真的可以用它做任何事情):

class FileToLogInterface(file):
    '''
    Interface to make sure that everytime anything is written to stderr, it is
    also forwarded to a file.
    '''

    def __init__(self, *args, **kwargs):
        if 'cfg' not in kwargs:
            raise TypeError('argument cfg is required.')
        else:
            if not isinstance(kwargs['cfg'], config.Config):
                raise TypeError(
                    'argument cfg should be a valid '
                    'PostSegmentation configuration object i.e. '
                    'postsegmentation.config.Config')
        self._cfg = kwargs['cfg']
        kwargs.pop('cfg')

        self._logger = logging.getlogger('access_log')

        super(FileToLogInterface, self).__init__(*args, **kwargs)

    def write(self, msg):
        super(FileToLogInterface, self).write(msg)
        self._logger.info(msg)

您需要一个终端多路复用器,比如tmux或GNU屏幕

我很惊讶Ryan Amos对原始问题的一个小评论是唯一一个比其他所有问题都更可取的解决方案,不管蟒蛇的诡计有多聪明,他们得到了多少点赞。根据Ryan的评论,tmux是GNU屏幕的一个很好的替代品。

But the principle is the same: if you ever find yourself wanting to leave a terminal job running while you log-out, head to the cafe for a sandwich, pop to the bathroom, go home (etc) and then later, reconnect to your terminal session from anywhere or any computer as though you'd never been away, terminal multiplexers are the answer. Think of them as VNC or remote desktop for terminal sessions. Anything else is a workaround. As a bonus, when the boss and/or partner comes in and you inadvertently ctrl-w / cmd-w your terminal window instead of your browser window with its dodgy content, you won't have lost the last 18 hours-worth of processing!

以下是Yuda Prawira的回答:

实现flush()和所有文件属性 将其编写为上下文管理器 也捕获stderr

.

import contextlib, sys

@contextlib.contextmanager
def log_print(file):
    # capture all outputs to a log file while still printing it
    class Logger:
        def __init__(self, file):
            self.terminal = sys.stdout
            self.log = file

        def write(self, message):
            self.terminal.write(message)
            self.log.write(message)

        def __getattr__(self, attr):
            return getattr(self.terminal, attr)

    logger = Logger(file)

    _stdout = sys.stdout
    _stderr = sys.stderr
    sys.stdout = logger
    sys.stderr = logger
    try:
        yield logger.log
    finally:
        sys.stdout = _stdout
        sys.stderr = _stderr


with log_print(open('mylogfile.log', 'w')):
    print('hello world')
    print('hello world on stderr', file=sys.stderr)

# you can capture the output to a string with:
# with log_print(io.StringIO()) as log:
#   ....
#   print('[captured output]', log.getvalue())

我知道这个问题已经得到了回答(使用python abc.py > output.log 2>&1),但我仍然不得不说:

在编写程序时,不要写入标准输出。总是使用日志来输出您想要的任何东西。这在以后你想重定向,过滤,旋转输出文件的时候会给你很大的自由。

正如@jfs提到的,大多数解决方案不能正确处理某些类型的stdout输出,例如来自C扩展的输出。在PyPI上有一个名为wurlitzer的模块负责处理所有这些。您只需要它的sys_pipes上下文管理器。简单易用:

from contextlib import redirect_stdout
import os
from wurlitzer import sys_pipes
        
log = open("test.log", "a")
with redirect_stdout(log), sys_pipes():
    print("print statement")
    os.system("echo echo call")

基于这篇文章之前的回答,我为自己编写了这个类,作为一种更紧凑和灵活的方式来重定向代码段的输出——这里只是一个列表——并确保之后的输出是规范化的。

class out_to_lt():
    def __init__(self, lt):
        if type(lt) == list:
            self.lt = lt
        else:
            raise Exception("Need to pass a list")            
    def __enter__(self):
        import sys
        self._sys = sys
        self._stdout = sys.stdout
        sys.stdout = self
        return self
    def write(self,txt):
        self.lt.append(txt)    
    def __exit__(self, type, value, traceback):
        self._sys.stdout = self._stdout

用作:

lt = []
with out_to_lt(lt) as o:
    print("Test 123\n\n")
    print(help(str))

更新。刚刚发现了一个场景,我必须添加两个额外的方法,但很容易适应:

class out_to_lt():
    ...
    def isatty(self):
        return True #True: You're running in a real terminal, False:You're being piped, redirected, cron
    def flush(self):
        pass

还有其他使用上下文的版本,但没有这么简单。实际上,我只是在谷歌上仔细检查了一下,并惊讶地没有看到它,所以对于其他人来说,寻找一个快速的解决方案是安全的,只针对上下文块中的代码,这里是:

import sys
with open('test_file', 'w') as sys.stdout:
    print('Testing 1 2 3')

测试如下:

$ cat redirect_stdout.py 导入系统 将open('test_file', 'w')作为sys.stdout: 打印('测试1 2 3') $ python redirect_stdout.py $ cat test_file 测试1 2 3