UU Blog

Python生成器 生产消费者模型 迭代器

环境: Python 3.5

列表解析和生成器

假设要生成一个1到10000的列表a,供其它功能去遍历它的话。

用Python列表解析的话,如下:

1
a = [ x for x in range(1,10000)]

但是有这么一个情况。我每次也不是非要遍历完这个列表,而且某些情况我可能根本不会用到这个列表。

但是一开始就定义好这个列表。它是会即时生成这个列表。如果这个表很大,毫无疑问比较浪费时间,和占用内存。

应对上述的问题,可以用生成器来解决。

1
a = ( x for x in range(1,10000))

语法上和列表解析的差别主要就是一个是用[],一个是用()

同样,可以用for去遍历它

1
2
for n in a:
print(n)

另外可以通过next属性去访问它的值。

1
2
3
4
5
6
7
8
9
10
# python 2.x  属性是 next()
# python 3.x 属性是 __next()
# 所以建议用next()获取 比较通用点。
whlie True:
try:
n = next(a)
print('n:', n)
except StopIteration as e:
print('Generator return value:', e.value)
break

Generator function

现在知道了( x for x in range(1,10000))这种生成器形式。但是我想要生成更复杂点的数据怎么办?

比如斐波那契数列。这个时候就用到yield了,通过yield去生成生成器的数据。简单点理解,它就是普通函数的return,只不过yield是专门将数据返回给生成器。

1
2
3
4
5
6
# 生成10位fabnacci数列
def fab():
a,b=1,1
for i in range(1,10):
yield a
a,b=b,a+b

console查看类型如下:

1
2
3
4
In [2]: fabnacci = fab()

In [3]: type(fabnacci)
Out[3]: generator

打印数据:

1
2
3
4
5
6
7
8
9
10
11
In [6]: for i in fabnacci:
print(i)
1
1
2
3
5
8
13
21
34

生产者消费者模型 协程

关于对生成器的应用,一个很好的例子是实现生产者消费者模型

执行由串行变成并行。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

def consumer(name):
print("I'm crazy {name},I'm ready to kill some mother fucker.".format(name=name))
while True:
fucker = yield
print('{name}:{fucker} guys has been killed.'.format(name=name,fucker=fucker))

def producer():
tom = consumer('Tom')
bob = consumer('Bob')
next(tom)
next(bob)
for i in range(1,10):
tom.send(i)
bob.send(i)

producer()

Out:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
I'm crazy Tom,I'm ready to kill some mother fucker.
I'm crazy Bob,I'm ready to kill some mother fucker.
Tom:1 guys has been killed.
Bob:1 guys has been killed.
Tom:2 guys has been killed.
Bob:2 guys has been killed.
Tom:3 guys has been killed.
Bob:3 guys has been killed.
Tom:4 guys has been killed.
Bob:4 guys has been killed.
Tom:5 guys has been killed.
Bob:5 guys has been killed.
Tom:6 guys has been killed.
Bob:6 guys has been killed.
Tom:7 guys has been killed.
Bob:7 guys has been killed.
Tom:8 guys has been killed.
Bob:8 guys has been killed.
Tom:9 guys has been killed.
Bob:9 guys has been killed.

可迭代对象和迭代器

  • 可以迭代的主要分两种,一个是可迭代对象(Iterable)的和迭代器(Iterator)。

  • 迭代器一定是可迭代对象,可迭代对象不一定是迭代器。

  • 可迭代对象都可以用for x in xx:语句来遍历对象。

  • 而迭代器还可以用next()函数逐个访问下一个值。遍历结束会遇到StopIteration异常。

  • 可以看做迭代器是可迭代对象一个更高级的实现。

如何判断

  1. 可以用isinstance函数来查看。

记得先导入Iterable,Iterator。

1
2
from collections import Iterable
from collections import Iterator
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
In [19]: fabnacci = fab()

In [20]: a = [ x for x in range(1,10000)]

In [21]: isinstance(fabnacci,Iterable)
Out[21]: True

In [22]: isinstance(fabnacci,Iterator)
Out[22]: True

In [23]: isinstance(a,Iterable)
Out[23]: True

In [24]: isinstance(a,Iterator)
Out[24]: False
  1. 可以用dir函数查看是否有next的方法

如果存在就是迭代器。

如何转换

iter()函数。可以将iterable的数据变为iterator

给作者打一针鸡血