标签： python奇淫技巧

最优线程数

• Ncpu=CPU的数量
• Ucpu=目标CPU使用率
• W/C=等待时间与计算时间的比率

为保持处理器达到期望的使用率，最优的线程池的大小等于

• cpu密集型任务，即$W<<C$，则$W/C≈0$，则$Nthreads=Ncpu*Ucpu$

如果希望CPU利用率为100%，则$Nthreads=Ncpu$

• IO密集型任务，即系统大部分时间在跟I/O交互，而这个时间线程不会占用CPU来处理，即在这个时间范围内，可以由其他线程来使用CPU，因而可以多配置一些线程。
• 混合型任务，二者都占有一定的时间

线城池

对于任务数量不断增加的程序，每有一个任务就生成一个线程，最终会导致线程数量的失控。对于任务数量不端增加的程序，固定线程数量的线程池是必要的。

方法一：使用threadpool模块

threadpool是一个比较老的模块了，支持py2 和 py3 。

import threadpoolimport timedef sayhello (a):    print("hello: "+a)    time.sleep(2)def main():    global result    seed=["a","b","c"]    start=time.time()    task_pool=threadpool.ThreadPool(5)    requests=threadpool.makeRequests(sayhello,seed)    for req in requests:        task_pool.putRequest(req)    task_pool.wait()    end=time.time()    time_m = end-start    print("time: "+str(time_m))    start1=time.time()    for each in seed:        sayhello(each)    end1=time.time()    print("time1: "+str(end1-start1))if __name__ == '__main__':    main(

方法二：使用concurrent.futures模块

from concurrent.futures import ThreadPoolExecutorimport timeimport timefrom concurrent.futures import ThreadPoolExecutor, wait, as_completedll = []def sayhello(a):    print("hello: "+a)    ll.append(a)    time.sleep(0.8)def main():    seed=["a","b","c","e","f","g","h"]    start1=time.time()    for each in seed:        sayhello(each)    end1=time.time()    print("time1: "+str(end1-start1))    start2=time.time()    with ThreadPoolExecutor(2) as executor:        for each in seed:            executor.submit(sayhello,each)    end2=time.time()    print("time2: "+str(end2-start2))def main2():    seed = ["a", "b", "c", "e", "f", "g", "h"]    executor = ThreadPoolExecutor(max_workers=10)    f_list = []    for each in seed:        future = executor.submit(sayhello, each)        f_list.append(future)    wait(f_list)    print(ll)    print('主线程结束')def main3():    seed = ["a", "b", "c", "e", "f", "g", "h"]    with ThreadPoolExecutor(max_workers=2) as executor:        f_list = []        for each in seed:            future = executor.submit(sayhello, each)            f_list.append(future)        wait(f_list,return_when='ALL_COMPLETED')        print(ll)        print('主线程结束')if __name__ == '__main__':    main3()

方法三：使用vthread模块

参考：

demo1

import vthread @vthread.pool(6)def some(a,b,c):    import time;time.sleep(1)    print(a+b+c) for i in range(10):    some(i,i,i)

demo2:分组线程池

import vthreadpool_1 = vthread.pool(5,gqueue=1) # open a threadpool with 5 threads named 1pool_2 = vthread.pool(2,gqueue=2) # open a threadpool with 2 threads named 2@pool_1def foolfunc1(num):    time.sleep(1)    print(f"foolstring1, test3 foolnumb1:{num}")@pool_2def foolfunc2(num):    time.sleep(1)    print(f"foolstring2, test3 foolnumb2:{num}")@pool_2def foolfunc3(num):    time.sleep(1)    print(f"foolstring3, test3 foolnumb3:{num}")for i in range(10): foolfunc1(i)for i in range(4): foolfunc2(i)for i in range(2): foolfunc3(i)