Pytorch 和 Tensorflow v1 兼容的环境搭建方法
作者:daimashiren 发布时间:2022-04-05 12:19:06
Github 上很多大牛的代码都是Tensorflow v1 写的,比较新的文章则喜欢用Pytorch,这导致我们复现实验或者对比实验的时候需要花费大量的时间在搭建不同的环境上。这篇文章是我经过反复实践总结出来的环境配置教程,亲测有效!
首先最基本的Python 环境配置如下:
conda create -n py37 python=3.7
python版本不要设置得太高也不要太低,3.6~3.7最佳,适用绝大部分代码库。(Tensorflow v1 最高支持的python 版本也只有3.7)
然后是Pytorch 环境 (因为最简单省力,哈哈哈)
# ROCM 5.1.1 (Linux only)
pip install torch==1.12.1+rocm5.1.1 torchvision==0.13.1+rocm5.1.1 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/rocm5.1.1
# CUDA 11.6
pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu116
# CUDA 11.3
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113
# CUDA 10.2
pip install torch==1.12.1+cu102 torchvision==0.13.1+cu102 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu102
# CPU only
pip install torch==1.12.1+cpu torchvision==0.13.1+cpu torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cpu
推荐使用pip 安装,用conda 安装的有时候会出现torch 识别不到GPU 的问题....
官网教程链接
Previous PyTorch Versions | PyTorch
然后是显卡相关的配置, cudatoolkit 和 cudnn. 前面这个是pytorch 环境必须具备的包,后面这个则是tensorflow 要使用GPU所必需的。前面安装完pytorch 其实已经装好了一个cudatoolkit,我的电脑是Cuda 10.2 ,所以现在环境中已经有了一个cudatookit=10.2的包了,但是Tensorflow v1 最高只支持到 Cuda 10,所以得降低cudatoolkit的版本到10.0 (Cuda 环境是向下兼容的,我的Cuda 环境是10.2 但是cudatoolkit=10.0 也一样能用,这是Tensorflow v1 最高支持的版本,只能妥协......)
conda install cudatoolkit=10.0
然后装cudnn
conda install cudnn=7.6.5=cuda10.0_0
亦可使用如下命令搜索你所要的cudnn版本
conda search cudnn
如果conda 下载太慢请切换国内源
https://www.jb51.net/article/199913.htm
最后把Tensorflow v1装上
pip install tensorflow-gpu==1.15.0 -i https://pypi.tuna.tsinghua.edu.cn/simple
推荐的Tensorflow v1 的版本是1.15.0 和1.14.0,其他版本尚未测试。
最后分别测试Pytorch 和Tensorflow 能否使用GPU如下:
import torch
print(torch.cuda.is_available()
Pytorch 检测GPU的方法相信大家都知道,不再赘述。Tensorflow v1 检测GPU的方法如下:
from tensorflow.python.client import device_lib
print(device_lib.list_local_devices())
如果输出结果为:
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
则降低protobuf 的版本
pip install protobuf==3.19.6 -i https://pypi.tuna.tsinghua.edu.cn/simple
正确的输出为:
2022-10-30 21:46:59.982971: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2022-10-30 21:47:00.006072: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3699850000 Hz
2022-10-30 21:47:00.006792: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55d1633f2750 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2022-10-30 21:47:00.006808: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2022-10-30 21:47:00.008473: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2022-10-30 21:47:00.105474: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-10-30 21:47:00.105762: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55d1635c3f60 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2022-10-30 21:47:00.105784: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce GTX 1080 Ti, Compute Capability 6.1
2022-10-30 21:47:00.105990: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-10-30 21:47:00.106166: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:01:00.0
2022-10-30 21:47:00.106369: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-10-30 21:47:00.107666: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2022-10-30 21:47:00.108687: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2022-10-30 21:47:00.108929: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2022-10-30 21:47:00.111721: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2022-10-30 21:47:00.112861: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2022-10-30 21:47:00.116688: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2022-10-30 21:47:00.116826: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-10-30 21:47:00.117018: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-10-30 21:47:00.117127: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2022-10-30 21:47:00.117170: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-10-30 21:47:00.117421: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-10-30 21:47:00.117435: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
2022-10-30 21:47:00.117446: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
2022-10-30 21:47:00.117529: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-10-30 21:47:00.117678: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-10-30 21:47:00.117813: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/device:GPU:0 with 10361 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
[name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 10409023728072267246
, name: "/device:XLA_CPU:0"
device_type: "XLA_CPU"
memory_limit: 17179869184
locality {
}
incarnation: 7385902535139826165
physical_device_desc: "device: XLA_CPU device"
, name: "/device:XLA_GPU:0"
device_type: "XLA_GPU"
memory_limit: 17179869184
locality {
}
incarnation: 7109357658802926795
physical_device_desc: "device: XLA_GPU device"
, name: "/device:GPU:0"
device_type: "GPU"
memory_limit: 10864479437
locality {
bus_id: 1
links {
}
}
incarnation: 6537278509263123219
physical_device_desc: "device: 0, name: NVIDIA GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1"
]
最关键的地方是你得看到你的GPU 型号,我的是 GTX 1080Ti,检测成功!
以上环境适用绝大多数深度学习模型,希望能帮到你!喜欢请点赞!
完结!撒花!
参考文献
Could not load dynamic library 'libcudart.so.10.0' - 知乎
https://medium.com/analytics-vidhya/solution-to-tensorflow-2-not-using-gpu-119fb3e04daa
How to tell if tensorflow is using gpu acceleration from inside python shell? - Stack Overflow
来源:https://blog.csdn.net/daimashiren/article/details/127605221


猜你喜欢
- 一、 什么是遗传算法?遗传算法是仿真生物遗传学和自然选择机理,通过人工方式所构造的一类搜索算法,从某种程度上说遗传算法是对生物进化过程进行的
- 检测这些圆,先找轮廓后通过轮廓点拟合椭圆import cv2import numpy as npimport matplotlib.pypl
- 这个系列记录我在一年vue开发中总结的一些经验和技巧。利用Object.freeze()提升性能Object.freeze()是ES5新增的
- #/usr/bin/env/python#coding=utf-8import sys,re,time,osmaxdata = 50000
- 前言在做自己的项目的时候有用到判断设备是否有切屏,一般用的多的地方就是考试系统,切屏我们都知道,一般可以很容易的进行监控,只不过当开启了小窗
- 算法是计算机科学中一个重要的研究方向,是解决复杂问题的关键。在计算机世界中,算法无处不在。数据库是存储数据和执行大批量计算的场所,在数据库中
- 本文实例讲述了PHP会话控制技巧。分享给大家供大家参考,具体如下:Demo1.php<form method="get&qu
- 我们在学习keras经常会看到下面这样的代码段:查阅官方文档可以知道:我们知道彩色图像一般会有Width, Height, Channels
- 在开发一些需要网络通信的应用中,经常会用到各种网络协议进行通信,博主在开发实验室的机器人的时候就遇到了需要把机器人上采集到的图片传回服务器进
- 安装时建议你为MySQL管理创建一个用户和组。由该组用户运行mysql服务器并执行管理任务。(也可以以root身份运行服务器,但是不推荐)第
- Python运算符重载 Python语言提供了运算符重载功能,增强了语言的灵活性
- 对于爬虫中部分网站设置了请求次数过多后会封杀ip,现在模拟浏览器进行爬虫,也就是说让服务器认识到访问他的是真正的浏览器而不是机器操作简单的直
- 增加操作:变量名[key] = value # 通过key添加value值,如果key存在则覆盖 &nbs
- 需求:从接口动态获取子菜单数据 动态加载 要求只有展开才加载子菜单数据 支持刷新,页面显示正常思路:一开始比较乱,思路很多。想了很多首先路由
- 前言提示:这里可以添加本文要记录的大概内容:将一个EXCEL等份拆成多个EXCEL将多个小EXCEL合并成一个大EXCEL并标记来源提示:以
- 1.直方图直方图: (1) 图像中不同像素等级出现的次数 (2) 图像中具有不同等级的像素关于总像素数目的比值。我们使用cv2.calcHi
- SQL Update常见写法Oralce和DB2都支持的语法:update test1?set (test1.name,test1.age)
- 背景重装系统,发现之前装在E盘的python可以直接使用,就只是将python的安装目录加入到环境变量中,也一直没有管它,今天跟天软交互的时
- PHP 5.0.0 和PHP 4.0.38 于2004年7月13日同时发布,这是一个值得我们PHP爱好者的一大喜讯。期盼已久的PHP5终于出
- 在MySQL的管理过程中,会遇到PC Server脱机或者重启,我需要在主机启动后再将MySQL服务启动。如果上百台或者更多的MySQL主机