• 售前

  • 售后

热门帖子
入门百科

python 异步编程——asyncio

[复制链接]
张王赵高痔 显示全部楼层 发表于 2022-1-15 13:19:59 |阅读模式 打印 上一主题 下一主题
python 异步编程——asyncio



摘要

本文主要介绍了python通过asyncio模块如何实现异步编程。
1. 协程

1.1 基本概念


  • 协程不是计算机提供的,程序员人为创造;
  • 协程(coroutine,微线程),即一个线程实现代码块相互切换执行;
1.2 实现方法


  • greenlet模块(早期)
  • yield关键字
  • asyncio模块(py3.4)
  • async、await关键字(py3.5)
1.2.1 greenlet

  1. pip install greenlet
复制代码
  1. from greenlet import greenlet
  2. def func_a():
  3.     print("first in func_a!")
  4.     gr_b.switch()
  5.     print("second in func_a!")
  6.     gr_b.switch()
  7.     print("third in func_a!")
  8.     gr_b.switch()
  9.    
  10. def func_b():
  11.     print("first in func_b!")
  12.     gr_a.switch()
  13.     print("second in func_b!")
  14.     gr_a.switch()
  15.     print("third in func_b!")
  16. gr_a = greenlet(func_a)
  17. gr_b = greenlet(func_b)
  18. gr_a.switch()
复制代码
  1. <code>first in func_a!
  2. first in func_b!
  3. second in func_a!
  4. second in func_b!
  5. third in func_a!
  6. third in func_b!
复制代码
1.2.2 yield

  1. def func_a():
  2.     yield "first in func_a!"
  3.     yield from func_b()
  4.     yield "second in func_a!"
  5.     yield "third in func_a!"
  6. def func_b():
  7.     yield "first in func_b!"
  8.     yield "second in func_b!"
  9.     yield "third in func_b!"
  10. iters = func_a()
  11. for i in iters:
  12.     print(i)
复制代码
  1. <code>first in func_a!
  2. first in func_b!
  3. second in func_b!
  4. third in func_b!
  5. second in func_a!
  6. third in func_a!
复制代码
1.2.3 asyncio模块

  1. import asyncio
  2. @asyncio.coroutine
  3. def func_a():
  4.     print("first in func_a!")
  5.     yield from asyncio.sleep(2)
  6.     print("second in func_a!")
  7.     yield from asyncio.sleep(2)
  8.     print("third in func_a!")
  9. @asyncio.coroutine
  10. def func_b():
  11.     print("first in func_b!")
  12.     yield from asyncio.sleep(2)
  13.     print("second in func_b!")
  14.     yield from asyncio.sleep(2)
  15.     print("third in func_b!")
  16. tasks = [asyncio.ensure_future(func_a()),
  17.          asyncio.ensure_future(func_b())]
  18. loop = asyncio.get_event_loop()
  19. loop.run_until_complete(asyncio.wait(tasks))
复制代码
  1. <code>first in func_a!
  2. first in func_b!
  3. second in func_a!
  4. second in func_b!
  5. third in func_a!
  6. third in func_b!
复制代码
1.2.4 async、await关键字(推荐)

  1. import asyncio
  2. async def func_a():
  3.     print("first in func_a!")
  4.     await asyncio.sleep(2)
  5.     print("second in func_a!")
  6.     await asyncio.sleep(2)
  7.     print("third in func_a!")
  8. async def func_b():
  9.     print("first in func_b!")
  10.     await asyncio.sleep(2)
  11.     print("second in func_b!")
  12.     await asyncio.sleep(2)
  13.     print("third in func_b!")
  14. tasks = [asyncio.ensure_future(func_a()),
  15.          asyncio.ensure_future(func_b())]
  16. loop = asyncio.get_event_loop()
  17. loop.run_until_complete(asyncio.wait(tasks))
复制代码
  1. <code>first in func_a!
  2. first in func_b!
  3. second in func_a!
  4. second in func_b!
  5. third in func_a!
  6. third in func_b!
复制代码
1.2.5 协程的意义

在一个线程中,如果遇到IO等待时间,利用该空闲时间去处理其余任务。
2. 异步编程(asyncio模块)

2.1 事件循环

2.1.1 定义

死循环:检测并执行某些代码。
  1. import asyncio
  2. # 获取一个事件循环
  3. loop = asyncio.get_event_loop()
  4. #将任务放到任务列表
  5. loop.run_until_complete(tasks)
复制代码
2.2 协程函数(async关键字) & 携程对象

  1. 协程函数
  2. async def 函数名
  3. 携程对象
  4.         执行携程函数得到的对象 func()
  5. async func():
  6.         pass
  7. result = func() # 得到协程对象时,函数内部代码不会执行。
复制代码
执行协程对象
  1. import asyncio
  2. async def func():
  3.         pass
  4. asyncio.run(func())
复制代码
2.3 await关键字

  1. await 后接可等待的对象
  2.          (协程对象、Future对象、Task对象)
  3.           即IO等待
  4. # 举例1
  5. import asyncio
  6. async def func():
  7.         print("开始执行!")
  8.         response = await asyncio.sleep(2)
  9.         print("已结束!",response)
  10. asyncio.run(func())
  11. # 举例2
  12. import asyncio
  13. async def fun_a():
  14.     print("func_a:1!")
  15.     await asyncio.sleep(2)
  16.     print("func_a:2!")
  17.     return "func_a:over!"
  18. async def fun_b():
  19.     print("func_b:1")
  20.     response = await fun_a()
  21.     print(response)
  22. asyncio.run(fun_b())
  23. 结果为:
  24. func_b:1
  25. func_a:1!
  26. func_a:2!
  27. func_a:over!
复制代码
2.4 Task对象

在事件循环中添加多个任务。
  1. <code>Task用于并发调度协程,
  2. asyncio.create_task(协程对象)
复制代码
  1. import asyncio
  2. async def fun_a():
  3.     print("func_a:1!")
  4.     await asyncio.sleep(2)
  5.     print("func_a:2!")
  6.     return "func_a:over!"
  7. async def fun_b():
  8.     print("func_b:1")
  9.     tasks = [asyncio.create_task(fun_a()),
  10.              asyncio.create_task(fun_a()),
  11.              ]
  12.     done,pending = await asyncio.wait(tasks)
  13.     print(done)
  14. asyncio.run(fun_b())
复制代码
  1. <code>func_b:1
  2. func_a:1!
  3. func_a:1!
  4. func_a:2!
  5. func_a:2!
复制代码
2.5 future 对象

Task继承Future对象,Task的await结果由Future处理而来。(很少使用)
2.6 concurrent.futures.Future对象

使用进程池或线程池实现异步操作。
  1. from concurrent.futures import Future
  2. from concurrent.futures.thread import ThreadPoolExecutor
  3. from concurrent.futures.process import ProcessPoolExecutor
  4. def func(val):
  5.     print(val)
  6.     return val
  7. thread_pool = ThreadPoolExecutor(max_workers=5)
  8. for i in range(10):
  9.     fut = thread_pool.submit(func,i)
  10.     print(fut)
复制代码
协程与线程(进程)的交叉使用
  1. import asyncio
  2. from concurrent.futures import Future
  3. from concurrent.futures.thread import ThreadPoolExecutor
  4. from concurrent.futures.process import ProcessPoolExecutor
  5. async def func_a():
  6.     print("func_a:1!")
  7.     await asyncio.sleep(2)
  8.     print("func_a:2!")
  9.     return 200
  10. def func_b(val):
  11.     print(val)
  12.     print("I am in func_b!")
  13.     return 100
  14. async def main():
  15.     print("main func:1!")
  16.     result = await func_a()
  17.     print(result)
  18.     loop = asyncio.get_running_loop()
  19.     # 1、使用线程调用func_b
  20.     future = loop.run_in_executor(None,func_b,2011)
  21.     # # 2、使用线程调用func_b
  22.     # with ThreadPoolExecutor(max_workers=5) as executor:
  23.     #     future = loop.run_in_executor(executor, func_b, 2011)
  24.     #
  25.     # # 3、使用进程调用func_b
  26.     # with ProcessPoolExecutor(max_workers=5) as executor:
  27.     #     future = loop.run_in_executor(executor, func_b, 2011)
  28.     response = await future
  29.     print(response)
  30. asyncio.run(main())
复制代码
  1. <code>main func:1!
  2. func_a:1!
  3. func_a:2!
  4. 200
  5. 2011
  6. I am in func_b!
  7. 100
复制代码
案例(asyncio+不支持异步的模块)
  1. // An highlighted block
  2. import asyncio
  3. from concurrent.futures.thread import ThreadPoolExecutor
  4. import requests
  5. import random
  6. async def spider(url):
  7.     loop = asyncio.get_running_loop()
  8.     print("start to download {}".format(url))
  9.     with ThreadPoolExecutor(max_workers=5) as executor:
  10.         result = loop.run_in_executor(executor,requests.get,url)
  11.     response = await result
  12.     print("end to download {}".format(url))
  13.     with open("{}.png".format(random.choice([i for i in range(1000)])),"wb") as f:
  14.         f.write(response.content)
  15. urls = ["https://image.baidu.com/search/detail?ct=503316480&z=0&ipn=false&word=%E4%BA%94%E6%9C%88%E5%A4%A9%E5%9B%BE%E7%89%87&step_word=&hs=0&pn=0&spn=0&di=101860&pi=0&rn=1&tn=baiduimagedetail&is=0%2C0&istype=0&ie=utf-8&oe=utf-8&in=&cl=2&lm=-1&st=undefined&cs=548937810%2C3900574322&os=2218065150%2C2242708278&simid=4253052539%2C539865446&adpicid=0&lpn=0&ln=1334&fr=&fmq=1642142581695_R&fm=&ic=undefined&s=undefined&hd=undefined&latest=undefined&copyright=undefined&se=&sme=&tab=0&width=undefined&height=undefined&face=undefined&ist=&jit=&cg=star&bdtype=0&oriquery=&objurl=https%3A%2F%2Fgimg2.baidu.com%2Fimage_search%2Fsrc%3Dhttp%3A%2F%2Fpic1.zhimg.com%2F50%2Fv2-3b13501c1397e36b11a0efb9fc2ca195_hd.jpg%26refer%3Dhttp%3A%2F%2Fpic1.zhimg.com%26app%3D2002%26size%3Df9999%2C10000%26q%3Da80%26n%3D0%26g%3D0n%26fmt%3Djpeg%3Fsec%3D1644734586%26t%3Dedf6d65698f1c31375d2689bfcd59533&fromurl=ippr_z2C%24qAzdH3FAzdH3Fooo_z%26e3Bziti7_z%26e3Bv54AzdH3Fq7jfpt5gAzdH3Fdbd00mm9aAzdH3Fwgfoj6AzdH3F00d9nmdmc&gsm=1&rpstart=0&rpnum=0&islist=&querylist=&nojc=undefined&dyTabStr=MCwzLDEsNiw0LDIsNSw3LDgsOQ%3D%3D",
  16.         "https://image.baidu.com/search/detail?ct=503316480&z=0&ipn=false&word=%E4%BA%94%E6%9C%88%E5%A4%A9%E5%9B%BE%E7%89%87&step_word=&hs=0&pn=1&spn=0&di=70180&pi=0&rn=1&tn=baiduimagedetail&is=0%2C0&istype=0&ie=utf-8&oe=utf-8&in=&cl=2&lm=-1&st=undefined&cs=301737693%2C3447630243&os=1460876122%2C3473358269&simid=4250352230%2C715662432&adpicid=0&lpn=0&ln=1334&fr=&fmq=1642142581695_R&fm=&ic=undefined&s=undefined&hd=undefined&latest=undefined&copyright=undefined&se=&sme=&tab=0&width=undefined&height=undefined&face=undefined&ist=&jit=&cg=star&bdtype=0&oriquery=&objurl=https%3A%2F%2Fgimg2.baidu.com%2Fimage_search%2Fsrc%3Dhttp%3A%2F%2Fb-ssl.duitang.com%2Fuploads%2Fitem%2F201704%2F03%2F20170403095157_yfQ5i.thumb.700_0.jpeg%26refer%3Dhttp%3A%2F%2Fb-ssl.duitang.com%26app%3D2002%26size%3Df9999%2C10000%26q%3Da80%26n%3D0%26g%3D0n%26fmt%3Djpeg%3Fsec%3D1644734586%26t%3Db079866dcc8b21bbdae0450fea819256&fromurl=ippr_z2C%24qAzdH3FAzdH3Fooo-kjpw8_z%26e3B17tpwg2_z%26e3Bv54AzdH3Fks52AzdH3F%3Ft1%3D0n98n9bn9&gsm=1&rpstart=0&rpnum=0&islist=&querylist=&nojc=undefined&dyTabStr=MCwzLDEsNiw0LDIsNSw3LDgsOQ%3D%3D",
  17.         "https://image.baidu.com/search/detail?ct=503316480&z=0&ipn=false&word=%E4%BA%94%E6%9C%88%E5%A4%A9%E5%9B%BE%E7%89%87&step_word=&hs=0&pn=4&spn=0&di=164890&pi=0&rn=1&tn=baiduimagedetail&is=0%2C0&istype=0&ie=utf-8&oe=utf-8&in=&cl=2&lm=-1&st=undefined&cs=554303154%2C2476822283&os=2894650807%2C2922528529&simid=3343407959%2C137235789&adpicid=0&lpn=0&ln=1334&fr=&fmq=1642142581695_R&fm=&ic=undefined&s=undefined&hd=undefined&latest=undefined&copyright=undefined&se=&sme=&tab=0&width=undefined&height=undefined&face=undefined&ist=&jit=&cg=star&bdtype=0&oriquery=&objurl=https%3A%2F%2Fgimg2.baidu.com%2Fimage_search%2Fsrc%3Dhttp%3A%2F%2Finews.gtimg.com%2Fnewsapp_match%2F0%2F11861039407%2F0.jpg%26refer%3Dhttp%3A%2F%2Finews.gtimg.com%26app%3D2002%26size%3Df9999%2C10000%26q%3Da80%26n%3D0%26g%3D0n%26fmt%3Djpeg%3Fsec%3D1644734586%26t%3D7dcf718e3ef7b029dbfbb0312ac5c301&fromurl=ippr_z2C%24qAzdH3FAzdH3Fh7wtkw5_z%26e3Bqq_z%26e3Bv54AzdH3FfAzdH3FdadaamadAaHbUDaa%3F6juj6%3Dfrt1j6&gsm=1&rpstart=0&rpnum=0&islist=&querylist=&nojc=undefined&dyTabStr=MCwzLDEsNiw0LDIsNSw3LDgsOQ%3D%3D",
  18.         "https://image.baidu.com/search/detail?ct=503316480&z=0&ipn=false&word=%E4%BA%94%E6%9C%88%E5%A4%A9%E5%9B%BE%E7%89%87&step_word=&hs=0&pn=18&spn=0&di=125620&pi=0&rn=1&tn=baiduimagedetail&is=0%2C0&istype=0&ie=utf-8&oe=utf-8&in=&cl=2&lm=-1&st=undefined&cs=3919725283%2C4047357414&os=1077457953%2C3020600078&simid=4210168092%2C663966287&adpicid=0&lpn=0&ln=1334&fr=&fmq=1642142581695_R&fm=&ic=undefined&s=undefined&hd=undefined&latest=undefined&copyright=undefined&se=&sme=&tab=0&width=undefined&height=undefined&face=undefined&ist=&jit=&cg=star&bdtype=0&oriquery=&objurl=https%3A%2F%2Fgimg2.baidu.com%2Fimage_search%2Fsrc%3Dhttp%3A%2F%2Fb-ssl.duitang.com%2Fuploads%2Fitem%2F201707%2F21%2F20170721002211_RkCmF.thumb.700_0.jpeg%26refer%3Dhttp%3A%2F%2Fb-ssl.duitang.com%26app%3D2002%26size%3Df9999%2C10000%26q%3Da80%26n%3D0%26g%3D0n%26fmt%3Djpeg%3Fsec%3D1644734693%26t%3D56b4d33f55e7483f69343b4e9ebaab21&fromurl=ippr_z2C%24qAzdH3FAzdH3Fooo_z%26e3B17tpwg2_z%26e3Bv54AzdH3Fks52AzdH3F%3Ft1%3Dbalcllal0&gsm=13&rpstart=0&rpnum=0&islist=&querylist=&nojc=undefined&dyTabStr=MCwzLDEsNiw0LDIsNSw3LDgsOQ%3D%3D",
  19.         "https://image.baidu.com/search/detail?ct=503316480&z=0&ipn=false&word=%E4%BA%94%E6%9C%88%E5%A4%A9%E5%9B%BE%E7%89%87&step_word=&hs=0&pn=38&spn=0&di=69740&pi=0&rn=1&tn=baiduimagedetail&is=0%2C0&istype=0&ie=utf-8&oe=utf-8&in=&cl=2&lm=-1&st=undefined&cs=3698734864%2C2135288157&os=1009166602%2C1961084779&simid=3421568423%2C430913388&adpicid=0&lpn=0&ln=1334&fr=&fmq=1642142581695_R&fm=&ic=undefined&s=undefined&hd=undefined&latest=undefined&copyright=undefined&se=&sme=&tab=0&width=undefined&height=undefined&face=undefined&ist=&jit=&cg=star&bdtype=0&oriquery=&objurl=https%3A%2F%2Fgimg2.baidu.com%2Fimage_search%2Fsrc%3Dhttp%3A%2F%2F5b0988e595225.cdn.sohucs.com%2Fimages%2F20181103%2Fac15613329de46099b3bc0d5398ff9ea.jpeg%26refer%3Dhttp%3A%2F%2F5b0988e595225.cdn.sohucs.com%26app%3D2002%26size%3Df9999%2C10000%26q%3Da80%26n%3D0%26g%3D0n%26fmt%3Djpeg%3Fsec%3D1644734709%26t%3D60e503e73c3c9a23d100ebd435e17c48&fromurl=ippr_z2C%24qAzdH3FAzdH3Fooo_z%26e3Bf5i7_z%26e3Bv54AzdH3FwAzdH3Fd0dlcmamd_m09baa&gsm=26&rpstart=0&rpnum=0&islist=&querylist=&nojc=undefined&dyTabStr=MCwzLDEsNiw0LDIsNSw3LDgsOQ%3D%3D",]
  20. task = [spider(url) for url in urls]
  21. asyncio.run(asyncio.wait(task))
复制代码
2.7 异步迭代器

不常用
2.8 异步上下文管理器

  1. import asyncio
  2. async def main():
  3.         async with AsyncManager() as f:
  4.                 f.do_sth()
  5. asyncio.run(main())
复制代码
3. uvloop

  1. 使用uvloop替代asyncio的默认事件循环,效率高出两倍
  2. pip install uvloop
  3. import asyncio
  4. import uvloop
  5. asyncion.set_event_loop_policy(uvloop.EventLoopPolicy())
复制代码
4. 案例

4.1 连接MySQL

  1. <code>pip insatll aiomysql
复制代码
4.2 FastAPI框架

  1. <code>pip insatll fastapi
  2. pip insatll uvicorn
复制代码
by CyrusMay 2022 01 14
世界再大
不过你和我
用最小回忆
堆成宇宙
—————五月天(因为你 所以我)—————


来源:https://blog.caogenba.net/Cyrus_May/article/details/122491335
免责声明:如果侵犯了您的权益,请联系站长,我们会及时删除侵权内容,谢谢合作!

帖子地址: 

回复

使用道具 举报

分享
推广
火星云矿 | 预约S19Pro,享500抵1000!
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

草根技术分享(草根吧)是全球知名中文IT技术交流平台,创建于2021年,包含原创博客、精品问答、职业培训、技术社区、资源下载等产品服务,提供原创、优质、完整内容的专业IT技术开发社区。
  • 官方手机版

  • 微信公众号

  • 商务合作