python开发之基于thread线程搜索本地文件的方法
作者:Hongten 发布时间:2023-05-11 18:49:12
标签:python,thread,线程
本文实例讲述了python开发之基于thread线程搜索本地文件的方法。分享给大家供大家参考,具体如下:
先来看看运行效果图:
利用多个线程处理搜索的问题,我们可以发现他很快....
下面是代码部分:
# A parallelized "find(1)" using the thread module.
# This demonstrates the use of a work queue and worker threads.
# It really does do more stats/sec when using multiple threads,
# although the improvement is only about 20-30 percent.
# (That was 8 years ago. In 2002, on Linux, I can't measure
# a speedup. :-( )
# I'm too lazy to write a command line parser for the full find(1)
# command line syntax, so the predicate it searches for is wired-in,
# see function selector() below. (It currently searches for files with
# world write permission.)
# Usage: parfind.py [-w nworkers] [directory] ...
# Default nworkers is 4
import sys
import getopt
import time
import os
from stat import *
import _thread as thread
# Work queue class. Usage:
# wq = WorkQ()
# wq.addwork(func, (arg1, arg2, ...)) # one or more calls
# wq.run(nworkers)
# The work is done when wq.run() completes.
# The function calls executed by the workers may add more work.
# Don't use keyboard interrupts!
class WorkQ:
# Invariants:
# - busy and work are only modified when mutex is locked
# - len(work) is the number of jobs ready to be taken
# - busy is the number of jobs being done
# - todo is locked iff there is no work and somebody is busy
def __init__(self):
self.mutex = thread.allocate()
self.todo = thread.allocate()
self.todo.acquire()
self.work = []
self.busy = 0
def addwork(self, func, args):
job = (func, args)
self.mutex.acquire()
self.work.append(job)
self.mutex.release()
if len(self.work) == 1:
self.todo.release()
def _getwork(self):
self.todo.acquire()
self.mutex.acquire()
if self.busy == 0 and len(self.work) == 0:
self.mutex.release()
self.todo.release()
return None
job = self.work[0]
del self.work[0]
self.busy = self.busy + 1
self.mutex.release()
if len(self.work) > 0:
self.todo.release()
return job
def _donework(self):
self.mutex.acquire()
self.busy = self.busy - 1
if self.busy == 0 and len(self.work) == 0:
self.todo.release()
self.mutex.release()
def _worker(self):
time.sleep(0.00001) # Let other threads run
while 1:
job = self._getwork()
if not job:
break
func, args = job
func(*args)
self._donework()
def run(self, nworkers):
if not self.work:
return # Nothing to do
for i in range(nworkers-1):
thread.start_new(self._worker, ())
self._worker()
self.todo.acquire()
# Main program
def main():
nworkers = 4
#print(getopt.getopt(sys.argv[1:], '-w:'))
opts, args = getopt.getopt(sys.argv[1:], '-w:')
for opt, arg in opts:
if opt == '-w':
nworkers = int(arg)
if not args:
#print(os.curdir)
args = [os.curdir]
wq = WorkQ()
for dir in args:
wq.addwork(find, (dir, selector, wq))
t1 = time.time()
wq.run(nworkers)
t2 = time.time()
sys.stderr.write('Total time %r sec.\n' % (t2-t1))
# The predicate -- defines what files we look for.
# Feel free to change this to suit your purpose
def selector(dir, name, fullname, stat):
# Look for world writable files that are not symlinks
return (stat[ST_MODE] & 0o002) != 0 and not S_ISLNK(stat[ST_MODE])
# The find procedure -- calls wq.addwork() for subdirectories
def find(dir, pred, wq):
try:
names = os.listdir(dir)
except os.error as msg:
print(repr(dir), ':', msg)
return
for name in names:
if name not in (os.curdir, os.pardir):
fullname = os.path.join(dir, name)
try:
stat = os.lstat(fullname)
except os.error as msg:
print(repr(fullname), ':', msg)
continue
if pred(dir, name, fullname, stat):
print(fullname)
if S_ISDIR(stat[ST_MODE]):
if not os.path.ismount(fullname):
wq.addwork(find, (fullname, pred, wq))
# Call the main program
main()
希望本文所述对大家Python程序设计有所帮助。
0
投稿
猜你喜欢
- 在jQuery的官方文档中,提示用户这是一个低级的方法,应该用.data()方法来代替。$.data( element, key, valu
- 新云4.0模版标签是全新改的了,加了前缀。如果你怀旧,请查看新云CMS3.1常用模板标签。下面的标签说明,后台就有,为了方便查看转到这里。{
- 5.0版本和之前版本的差异较大,本篇对熟悉3.2版本的用户给出了一些5.0的主要区别。URL和路由5.0的URL访问不再支持普通URL模式,
- 制作爬虫的步骤制作一个爬虫一般分以下几个步骤:分析需求分析网页源代码,配合开发者工具编写正则表达式或者XPath表达式正式编写 python
- 本文实例讲述了Python只用40行代码编写的计算器。分享给大家供大家参考,具体如下:效果图:代码:from tkinter import
- 一般事件事件浏览器支持描述onClickIE3|N2|O3鼠标点击事件,多用在某个对象控制的范围内的鼠标点击onDblClickIE4|N4
- isset和is_null啥区别,看手册上讲的话, isset和is_null的功能几乎完全”相反的一样”..是不是isset就是一个is_
- 字符串索引示意图字符串切片也就是截取字符串,取子串Python中字符串切片方法字符串[开始索引:结束索引:步长]切取字符串为开始索引到结束索
- 转自微信公众号:机器学习社区,经作者授权转载时间序列分析是数据科学中一个非常重要的领域,它主要包含统计分析、检测变化点、异常检测和预测未来趋
- <% '#######以下是一个类文件,下面的注解是调用类的方法#########################
- 首先我很不喜欢验证码这东西。但现在越来越多的网站使用验证码来抵御spam的冲击。太揪心了。背景介绍验证码的技术叫CAPTCHA验证,CAPT
- 前言这次,我们要用Pygame写一个Pong游戏先看看效果:需要的模块:Pygame在python文件同目录下新建resources文件夹,
- 前言使用 Pandas 的between 、cut、qcut 和 value_count离散化数值变量。分箱是一种常见的数据预处理技术有时也
- public static char doVerify(String id) { char pszSrc[]=id.toCharArray(
- 此前piscdong已经做过一次评测了,这次的Beta正式推出,我也来参与一下.当回小白鼠吧.新的界面设计非常让人兴奋750){this.r
- 你知道(X)HTML中最多余的标签中是什么吗?在我看来就是这个<a>标签,不错,就是每个网站使用最多的超级链接标签<a&g
- 依次前移,实现聊友们的发言的更迭:function form1_onsubmit()if document.form1.
- 语言是信息传播的主要障碍。多语言网站,顾名思义就是能够以多种语言(而不是单种语言)为用户提供信息服务,让使用不同语言的用户都能够从同个网站获
- 最近正在用功的学习jQuery,在琢磨了不少别人写的功能之后,也开始尝试着自己开发一些功能。今天我做了一个简单的密码强度测试工具。这可功能的
- 要求必备知识熟悉基本编程环境搭建。运行环境windows 7(64位);php-5.3; memcached-1.2.6下载地址环境下载什么