详解pytest分布式执行插件 pytest-xdist 的高级用法
作者:SoonPan 发布时间:2022-11-04 21:41:51
标签:pytest,分布式,插件,pytest-xdist
想要使用多个CPU核心来进行测试,可以使用 -n 参数( 或者 --numprocesses)
(使用8个核心来跑测试用例)
pytest -n 8
使用 -n auto
参数可以利用电脑的所有核心来跑测试用例
测试时使用的算法可以根据--dist
命令参数定制:
--dist load
(默认选项):给每个CPU核心随机分配用例,不保证执行顺序。--dist loadscope
:对于测试函数,测试按模块分组,对于测试方法,测试按类分组。每组作为一个整体分配给可用的worker。这保证了组中的所有测试都在同一进程中运行。如果的模块级或类级fixtures,这将非常有用。按类分组优先于按模块分组。--dist loadfile
: 测试用例按其所在文件分组。每组作为一个整体分配给可用的worker。这保证了文件中的所有测试都在同一个辅助进程中运行。--dist loadgroup
: 测试按xdist_group标记分组。每组作为一个整体分配给可用的执行器。这保证了具有相同xdist_ group名称的所有测试都在同一个worker中运行。
@pytest.mark.xdist_group(name="group1")
def test1():
pass
class TestA:
@pytest.mark.xdist_group("group1")
def test2():
pass-
这将确保test1和TestA::test2将在同一个worker中运行。没有xdist_ group标记的测试在--dist=load模式下正常运行。
--dist no
:正常的pytest执行模式,一次运行一个测试(完全没有分发)。
例子:
项目目录结构
xdist_test.py
import logging
import pytest
class TestXdist(object):
@pytest.mark.xdist_group("group1")
def test_one(self):
logging.info("1")
assert True
@pytest.mark.xdist_group("group1")
def test_two(self):
logging.info("2")
assert True
@pytest.mark.xdist_group("group2")
def test_three(self):
logging.info("3")
assert True
@pytest.mark.xdist_group("group2")
def test_four(self):
logging.info("4")
assert True
xdist_dummy_test.py
import logging
import pytest
class TestXdist(object):
@pytest.mark.run(order=1)
@pytest.mark.xdist_group("group1")
def test_dummy_one(self):
logging.info("d1")
assert True
@pytest.mark.xdist_group("group1")
@pytest.mark.run(order=2)
def test_dummy_two(self):
logging.info("d2")
assert True
@pytest.mark.run(order=3)
@pytest.mark.xdist_group("group2")
def test_dummy_three(self):
logging.info("d3")
assert True
@pytest.mark.xdist_group("group2")
@pytest.mark.run(order=4)
def test_dummy_four(self):
logging.info("d4")
assert True
(venv) ? pytest pytest -n auto --dist loadscope
=========================================================================== test session starts ===========================================================================
platform darwin -- Python 3.8.9, pytest-7.1.2, pluggy-1.0.0
rootdir: /Users/spock/PycharmProjects/pytest, configfile: pytest.ini
plugins: xdist-2.5.0, forked-1.4.0, ordering-0.6
[gw0] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw1] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw2] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw3] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw4] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw5] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw6] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw7] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8] / gw6 [8] / gw7 [8]
scheduling tests via LoadScopeScheduling
test2/xdist_dummy_test.py::TestXdist::test_dummy_one
test/xdist_test.py::TestXdist::test_one
[gw0] [ 12%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_one
[gw1] [ 25%] PASSED test/xdist_test.py::TestXdist::test_one
test2/xdist_dummy_test.py::TestXdist::test_dummy_two
test/xdist_test.py::TestXdist::test_two
[gw1] [ 37%] PASSED test/xdist_test.py::TestXdist::test_two
test/xdist_test.py::TestXdist::test_three
[gw0] [ 50%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_two
test2/xdist_dummy_test.py::TestXdist::test_dummy_three
[gw1] [ 62%] PASSED test/xdist_test.py::TestXdist::test_three
test/xdist_test.py::TestXdist::test_four
[gw0] [ 75%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_three
test2/xdist_dummy_test.py::TestXdist::test_dummy_four
[gw1] [ 87%] PASSED test/xdist_test.py::TestXdist::test_four
[gw0] [100%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_four
============================================================================ 8 passed in 0.40s ============================================================================
(venv) ? pytest pytest -n auto --dist loadfile
=========================================================================== test session starts ===========================================================================
platform darwin -- Python 3.8.9, pytest-7.1.2, pluggy-1.0.0
rootdir: /Users/spock/PycharmProjects/pytest, configfile: pytest.ini
plugins: xdist-2.5.0, forked-1.4.0, ordering-0.6
[gw0] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw1] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw2] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw3] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw4] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw5] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw6] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw7] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8] / gw6 [8] / gw7 [8]
scheduling tests via LoadFileScheduling
test/xdist_test.py::TestXdist::test_one
test2/xdist_dummy_test.py::TestXdist::test_dummy_one
[gw1] [ 12%] PASSED test/xdist_test.py::TestXdist::test_one
[gw0] [ 25%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_one
test/xdist_test.py::TestXdist::test_two
test2/xdist_dummy_test.py::TestXdist::test_dummy_two
[gw1] [ 37%] PASSED test/xdist_test.py::TestXdist::test_two
[gw0] [ 50%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_two
test2/xdist_dummy_test.py::TestXdist::test_dummy_three
test/xdist_test.py::TestXdist::test_three
[gw1] [ 62%] PASSED test/xdist_test.py::TestXdist::test_three
[gw0] [ 75%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_three
test2/xdist_dummy_test.py::TestXdist::test_dummy_four
test/xdist_test.py::TestXdist::test_four
[gw1] [ 87%] PASSED test/xdist_test.py::TestXdist::test_four
[gw0] [100%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_four
============================================================================ 8 passed in 0.38s ============================================================================
(venv) ? pytest pytest -n auto --dist loadgroup
=========================================================================== test session starts ===========================================================================
platform darwin -- Python 3.8.9, pytest-7.1.2, pluggy-1.0.0
rootdir: /Users/spock/PycharmProjects/pytest, configfile: pytest.ini
plugins: xdist-2.5.0, forked-1.4.0, ordering-0.6
[gw0] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw1] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw2] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw3] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw4] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw5] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw6] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
[gw7] Python 3.8.9 (default, May 17 2022, 12:55:41) -- [Clang 13.1.6 (clang-1316.0.21.2.5)]
gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8] / gw6 [8] / gw7 [8]
scheduling tests via LoadGroupScheduling
test2/xdist_dummy_test.py::TestXdist::test_dummy_three@group2
test2/xdist_dummy_test.py::TestXdist::test_dummy_one@group1
[gw1] [ 12%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_three@group2
test2/xdist_dummy_test.py::TestXdist::test_dummy_four@group2
[gw0] [ 25%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_one@group1
[gw1] [ 37%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_four@group2
test2/xdist_dummy_test.py::TestXdist::test_dummy_two@group1
[gw0] [ 50%] PASSED test2/xdist_dummy_test.py::TestXdist::test_dummy_two@group1
test/xdist_test.py::TestXdist::test_three@group2
[gw1] [ 62%] PASSED test/xdist_test.py::TestXdist::test_three@group2
test/xdist_test.py::TestXdist::test_four@group2
[gw1] [ 75%] PASSED test/xdist_test.py::TestXdist::test_four@group2
test/xdist_test.py::TestXdist::test_one@group1
[gw0] [ 87%] PASSED test/xdist_test.py::TestXdist::test_one@group1
test/xdist_test.py::TestXdist::test_two@group1
[gw0] [100%] PASSED test/xdist_test.py::TestXdist::test_two@group1
============================================================================ 8 passed in 0.40s ============================================================================
来源:https://www.cnblogs.com/SoonPan/archive/2022/08/12/16578391.html
0
投稿
猜你喜欢
- 模块内部封装了很多实用的功能,有时在模块外部调用就需要将其导入。常见的方式有如下几种:1 . import>>> impo
- PyQt5动态(可拖动控件大小)布局控件QSplitter简介PyQt还提供了特殊的布局管理器QSplitter。它可以动态地拖动子控件之间
- 对于熟悉 C/C++ 或 Java 语言的工程师来说,JavaScript 显得灵活,简单易懂,对代码的格式的要求也相对松散。很容易学习,并
- 本文说明向外扩展数据库系统的两个选项,从而实现更高的可扩展性:水平数据划分和垂直数据划分当我提到向外扩展数据库系统时,我实际上只是讨论对数据
- 一:从版本3.23升级到4.104.10新特性:支持事务处理和存储过程升级过程的注意事项:1:升级采用直接复制安装文件的方法,简单实用,但要
- 前言很多人都认为人脸识别是一项非常难以实现的工作,看到名字就害怕,然后心怀忐忑到网上一搜,看到网上N页的教程立马就放弃了。这些人里包括曾经的
- 本文为大家分享了Python2.7与Python3.6环境切换的具体方法,供大家参考,具体内容如下系统支持为:Ubuntu18.04系统默认
- RSA算法RSA算法是一种公钥加密技术,被认为是最安全的加密方式.它是由Rivest,Shamir和Adleman于1978年发明的,因此命
- 因为要将js的一个签名算法移植到python上,遇到一些麻烦。int无限宽度,不会溢出算法中需要用到了32位int的溢出来参与运算,但是py
- 前言不管是做开发还是做过网站的朋友们,应该对于User Agent一点都不陌生,User Agent 中文名为用户代理,简称 UA,它是一个
- Linux Journal 发表了一篇优化 Oracle 数据库的文章,感觉十分的有用。简要介绍其摘要和大家共同分享 Linux 在企业级数
- 映射类型操作符(1)标准类型操作符 字典可以和所有的标准
- 字符串类型代码的执行字符串类型代码的执行函数有三个,都是Python的内置函数。eval()执行字符串类型的代码,并返回最终结果。exec(
- 本文实例讲述了Python可变参数用法。分享给大家供大家参考,具体如下:#!/usr/bin/env python# -*- coding:
- 本文实例讲述了Python enumerate函数功能与用法。分享给大家供大家参考,具体如下:eunmerate在英文中是列举、枚举的意思,
- with 用法理解Overviewwith 与with之后的object一起,起到了抛出异常和单独生成一个空间让代码在空间里运行的效果。实验
- 本文实例讲述了PHP编程实现多维数组按照某个键值排序的方法。分享给大家供大家参考,具体如下:实现对多维数组按照某个键值排序的两种解决方法(a
- CSS2.1 中规定了关于 CSS 规则 Specificity(特异性)的计算方式,用一个四位的数字串(注:CSS2 中是用三位)来表示,
- 简单的合并,本例是横向合并,纵向合并可以自行调整。import xlrd import xlwtimport shutil from xlu
- 一、常用文件函数库1、basename(); -- 返回路径中的文件名部分。string basename ( string $path [