我有一个Python脚本，它把一个整数列表作为输入，我需要一次处理四个整数。不幸的是，我无法控制输入，否则我将它作为一个四元素元组列表传入。目前，我以这种方式迭代它:
for i in range(0, len(ints), 4):
# dummy op for example code
foo += ints[i] * ints[i + 1] + ints[i + 2] * ints[i + 3]
不过，它看起来很像“C-think”，这让我怀疑有一种更python的方式来处理这种情况。该列表在迭代后被丢弃，因此不需要保留。也许这样会更好?
while ints:
foo += ints[0] * ints[1] + ints[2] * ints[3]
ints[0:4] = []
不过，感觉还是不太对。: - /
相关问题:在Python中如何将列表分割成大小均匀的块?
当前回答
我从来不想填充我的块，所以这个要求是必要的。我发现在任何可迭代对象上工作的能力也是必需的。鉴于此，我决定扩展公认的答案，https://stackoverflow.com/a/434411/1074659。
如果由于需要比较和筛选填充值而不需要填充，则这种方法的性能会受到轻微的影响。然而，对于大块大小，这个实用程序是非常高性能的。
#!/usr/bin/env python3
from itertools import zip_longest
_UNDEFINED = object()
def chunker(iterable, chunksize, fillvalue=_UNDEFINED):
"""
Collect data into chunks and optionally pad it.
Performance worsens as `chunksize` approaches 1.
Inspired by:
https://docs.python.org/3/library/itertools.html#itertools-recipes
"""
args = [iter(iterable)] * chunksize
chunks = zip_longest(*args, fillvalue=fillvalue)
yield from (
filter(lambda val: val is not _UNDEFINED, chunk)
if chunk[-1] is _UNDEFINED
else chunk
for chunk in chunks
) if fillvalue is _UNDEFINED else chunks
其他回答
如果你不介意使用外部包，你可以使用iteration_utilities。Grouper from iteration_utilities它支持所有可迭代对象(不仅仅是序列):
from iteration_utilities import grouper
seq = list(range(20))
for group in grouper(seq, 4):
print(group)
打印:
(0, 1, 2, 3)
(4, 5, 6, 7)
(8, 9, 10, 11)
(12, 13, 14, 15)
(16, 17, 18, 19)
如果长度不是组大小的倍数，它还支持填充(不完整的最后一组)或截断(丢弃不完整的最后一组)最后一个:
from iteration_utilities import grouper
seq = list(range(17))
for group in grouper(seq, 4):
print(group)
# (0, 1, 2, 3)
# (4, 5, 6, 7)
# (8, 9, 10, 11)
# (12, 13, 14, 15)
# (16,)
for group in grouper(seq, 4, fillvalue=None):
print(group)
# (0, 1, 2, 3)
# (4, 5, 6, 7)
# (8, 9, 10, 11)
# (12, 13, 14, 15)
# (16, None, None, None)
for group in grouper(seq, 4, truncate=True):
print(group)
# (0, 1, 2, 3)
# (4, 5, 6, 7)
# (8, 9, 10, 11)
# (12, 13, 14, 15)
基准
我还决定比较上面提到的几种方法的运行时间。这是一个对数-对数图，根据不同大小的列表将“10”个元素分组。对于定性结果:较低意味着更快:
至少在这个基准测试中iteration_utilities。石斑鱼表现最好。接着是Craz。
基准是用simple_benchmark1创建的。运行这个基准测试的代码是:
import iteration_utilities
import itertools
from itertools import zip_longest
def consume_all(it):
return iteration_utilities.consume(it, None)
import simple_benchmark
b = simple_benchmark.BenchmarkBuilder()
@b.add_function()
def grouper(l, n):
return consume_all(iteration_utilities.grouper(l, n))
def Craz_inner(iterable, n, fillvalue=None):
args = [iter(iterable)] * n
return zip_longest(*args, fillvalue=fillvalue)
@b.add_function()
def Craz(iterable, n, fillvalue=None):
return consume_all(Craz_inner(iterable, n, fillvalue))
def nosklo_inner(seq, size):
return (seq[pos:pos + size] for pos in range(0, len(seq), size))
@b.add_function()
def nosklo(seq, size):
return consume_all(nosklo_inner(seq, size))
def SLott_inner(ints, chunk_size):
for i in range(0, len(ints), chunk_size):
yield ints[i:i+chunk_size]
@b.add_function()
def SLott(ints, chunk_size):
return consume_all(SLott_inner(ints, chunk_size))
def MarkusJarderot1_inner(iterable,size):
it = iter(iterable)
chunk = tuple(itertools.islice(it,size))
while chunk:
yield chunk
chunk = tuple(itertools.islice(it,size))
@b.add_function()
def MarkusJarderot1(iterable,size):
return consume_all(MarkusJarderot1_inner(iterable,size))
def MarkusJarderot2_inner(iterable,size,filler=None):
it = itertools.chain(iterable,itertools.repeat(filler,size-1))
chunk = tuple(itertools.islice(it,size))
while len(chunk) == size:
yield chunk
chunk = tuple(itertools.islice(it,size))
@b.add_function()
def MarkusJarderot2(iterable,size):
return consume_all(MarkusJarderot2_inner(iterable,size))
@b.add_arguments()
def argument_provider():
for exp in range(2, 20):
size = 2**exp
yield size, simple_benchmark.MultiArgument([[0] * size, 10])
r = b.run()
1免责声明:我是iteration_utilities和simple_benchmark库的作者。
修改自Python的itertools文档中的Recipes部分:
from itertools import zip_longest
def grouper(iterable, n, fillvalue=None):
args = [iter(iterable)] * n
return zip_longest(*args, fillvalue=fillvalue)
例子
grouper('ABCDEFG', 3, 'x') # --> 'ABC' 'DEF' 'Gxx'
注意:在Python 2上使用izip_longest而不是zip_longest。
下面是我的go works on lists,iter和range…懒洋洋地:
def chunker(it,size):
rv = []
for i,el in enumerate(it,1) :
rv.append(el)
if i % size == 0 :
yield rv
rv = []
if rv : yield rv
几乎变成了一句俏皮话;(
In [95]: list(chunker(range(9),2) )
Out[95]: [[0, 1], [2, 3], [4, 5], [6, 7], [8]]
In [96]: list(chunker([1,2,3,4,5],2) )
Out[96]: [[1, 2], [3, 4], [5]]
In [97]: list(chunker(iter(range(9)),2) )
Out[97]: [[0, 1], [2, 3], [4, 5], [6, 7], [8]]
In [98]: list(chunker(range(9),25) )
Out[98]: [[0, 1, 2, 3, 4, 5, 6, 7, 8]]
In [99]: list(chunker(range(9),1) )
Out[99]: [[0], [1], [2], [3], [4], [5], [6], [7], [8]]
In [101]: %timeit list(chunker(range(101),2) )
11.3 µs ± 68.2 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
使用小的函数和东西真的不吸引我;我更喜欢使用切片:
data = [...]
chunk_size = 10000 # or whatever
chunks = [data[i:i+chunk_size] for i in xrange(0,len(data),chunk_size)]
for chunk in chunks:
...
关于J.F. Sebastian给出的解决方案:
def chunker(iterable, chunksize):
return zip(*[iter(iterable)]*chunksize)
它很聪明，但有一个缺点——总是返回元组。如何获得字符串代替? 当然，你可以写“.join(chunker(…))”，但无论如何都要构造临时元组。
你可以通过编写自己的zip来摆脱临时元组，就像这样:
class IteratorExhausted(Exception):
pass
def translate_StopIteration(iterable, to=IteratorExhausted):
for i in iterable:
yield i
raise to # StopIteration would get ignored because this is generator,
# but custom exception can leave the generator.
def custom_zip(*iterables, reductor=tuple):
iterators = tuple(map(translate_StopIteration, iterables))
while True:
try:
yield reductor(next(i) for i in iterators)
except IteratorExhausted: # when any of iterators get exhausted.
break
Then
def chunker(data, size, reductor=tuple):
return custom_zip(*[iter(data)]*size, reductor=reductor)
使用示例:
>>> for i in chunker('12345', 2):
... print(repr(i))
...
('1', '2')
('3', '4')
>>> for i in chunker('12345', 2, ''.join):
... print(repr(i))
...
'12'
'34'
