tensorflow模型转ncnn的操作方式
作者:MatthewHsw 发布时间:2022-10-29 15:55:19
第一步把tensorflow保存的.ckpt模型转为pb模型, 并记下模型的输入输出名字.
第二步去ncnn的github上把仓库clone下来, 按照上面的要求装好依赖并make.
第三步是修改ncnn的CMakeList, 具体修改的位置有:
ncnn/CMakeList.txt 文件, 在文件开头处加入add_definitions(-std=c++11), 末尾处加上add_subdirectory(examples), 如果ncnn没有examples文件夹,就新建一个, 并加上CMakeList.txt文件.
ncnn/tools/CMakeList.txt 文件, 加入add_subdirectory(tensorflow)
原版的tools/tensorflow/tensorflow2ncnn.cpp里, 不支持tensorflow的elu, FusedBathNormalization, Conv2dBackpropback操作, 其实elu是支持的,只需要仿照relu的格式, 在.cpp文件里加上就行. FusedBatchNormalization就是ncnn/layer/里实现的batchnorm.cpp, 只是`tensorflow2ncnn里没有写上, 可以增加下面的内容:
else if (node.op() == "FusedBatchNorm")
{
fprintf(pp, "%-16s", "BatchNorm");
}
...
else if (node.op() == "FusedBatchNorm")
{
std::cout << "node name is FusedBatchNorm" << std::endl;
tensorflow::TensorProto tensor;
find_tensor_proto(weights, node, tensor);
const tensorflow::TensorShapeProto& shape = tensor.tensor_shape();
const tensorflow::TensorProto& gamma = weights[node.input(1)];
const tensorflow::TensorProto& Beta = weights[node.input(2)];
const tensorflow::TensorProto& mean = weights[node.input(3)];
const tensorflow::TensorProto& var = weights[node.input(4)];
int channels = gamma.tensor_shape().dim(0).size(); // data size
int dtype = gamma.dtype();
switch (dtype){
case 1:
{
const float * gamma_tensor = reinterpret_cast<const float *>(gamma.tensor_content().c_str());
const float * mean_data = reinterpret_cast<const float *>(mean.tensor_content().c_str());
const float * var_data = reinterpret_cast<const float *>(var.tensor_content().c_str());
const float * b_data = reinterpret_cast<const float *>(Beta.tensor_content().c_str());
for (int i=0; i< channels; ++i)
{
fwrite(gamma_tensor+i, sizeof(float), 1, bp);
}
for (int i=0; i< channels; ++i)
{
fwrite(mean_data+i, sizeof(float), 1, bp);
}
for (int i=0; i< channels; ++i)
{
fwrite(var_data+i, sizeof(float), 1, bp);
}
for (int i=0; i< channels; ++i)
{
fwrite(b_data+i, sizeof(float), 1, bp);
}
}
default:
std::cerr << "Type is not supported." << std::endl;
}
fprintf(pp, " 0=%d", channels);
tensorflow::AttrValue value_epsilon;
if (find_attr_value(node, "epsilon", value_epsilon)){
float epsilon = value_epsilon.f();
fprintf(pp, " 1=%f", epsilon);
}
}
同理, Conv2dBackpropback其实就是ncnn里的反卷积操作, 只不过ncnn实现反卷积的操作和tensorflow内部实现反卷积的操作过程不一样, 但结果是一致的, 需要仿照普通卷积的写法加上去.
ncnn同样支持空洞卷积, 但无法识别tensorflow的空洞卷积, 具体原理可以看tensorflow空洞卷积的原理, tensorflow是改变featuremap做空洞卷积, 而ncnn是改变kernel做空洞卷积, 结果都一样. 需要对.proto文件修改即可完成空洞卷积.
总之ncnn对tensorflow的支持很不友好, 有的层还需要自己手动去实现, 还是很麻烦.
补充知识:pytorch模型转mxnet
介绍
gluon把mxnet再进行封装,封装的风格非常接近pytorch
使用gluon的好处是非常容易把pytorch模型向mxnet转化
唯一的问题是gluon封装还不成熟,封装好的layer不多,很多常用的layer 如concat,upsampling等layer都没有
这里关注如何把pytorch 模型快速转换成 mxnet基于symbol 和 exector设计的网络
pytorch转mxnet module
关键点:
mxnet 设计网络时symbol 名称要和pytorch初始化中各网络层名称对应
torch.load()读入pytorch模型checkpoint 字典,取当中的'state_dict'元素,也是一个字典
pytorch state_dict 字典中key是网络层参数的名称,val是参数ndarray
pytorch 的参数名称的组织形式和mxnet一样,但是连接符号不同,pytorch是'.',而mxnet是'_'比如:
pytorch '0.conv1.0.weight'
mxnet '0_conv1_0_weight'
pytorch 的参数array 和mxnet 的参数array 完全一样,只要名称对上,直接赋值即可初始化mxnet模型
需要做的有以下几点:
设计和pytorch网络对应的mxnet网络
加载pytorch checkpoint
调整pytorch checkpoint state_dict 的key名称和mxnet命名格式一致
FlowNet2S PytorchToMxnet
pytorch flownet2S 的checkpoint 可以在github上搜到
import mxnet as mx
from symbol_util import *
import pickle
def get_loss(data, label, loss_scale, name, get_input=False, is_sparse = False, type='stereo'):
if type == 'stereo':
data = mx.sym.Activation(data=data, act_type='relu',name=name+'relu')
# loss
if is_sparse:
loss =mx.symbol.Custom(data=data, label=label, name=name, loss_scale= loss_scale, is_l1=True,
op_type='SparseRegressionLoss')
else:
loss = mx.sym.MAERegressionOutput(data=data, label=label, name=name, grad_scale=loss_scale)
return (loss,data) if get_input else loss
def flownet_s(loss_scale, is_sparse=False, name=''):
img1 = mx.symbol.Variable('img1')
img2 = mx.symbol.Variable('img2')
data = mx.symbol.concat(img1,img2,dim=1)
labels = {'loss{}'.format(i): mx.sym.Variable('loss{}_label'.format(i)) for i in range(0, 7)}
# print('labels: ',labels)
prediction = {}# a dict for loss collection
loss = []#a list
#normalize
data = (data-125)/255
# extract featrue
conv1 = mx.sym.Convolution(data, pad=(3, 3), kernel=(7, 7), stride=(2, 2), num_filter=64, name=name + 'conv1_0')
conv1 = mx.sym.LeakyReLU(data=conv1, act_type='leaky', slope=0.1)
conv2 = mx.sym.Convolution(conv1, pad=(2, 2), kernel=(5, 5), stride=(2, 2), num_filter=128, name=name + 'conv2_0')
conv2 = mx.sym.LeakyReLU(data=conv2, act_type='leaky', slope=0.1)
conv3a = mx.sym.Convolution(conv2, pad=(2, 2), kernel=(5, 5), stride=(2, 2), num_filter=256, name=name + 'conv3_0')
conv3a = mx.sym.LeakyReLU(data=conv3a, act_type='leaky', slope=0.1)
conv3b = mx.sym.Convolution(conv3a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=256, name=name + 'conv3_1_0')
conv3b = mx.sym.LeakyReLU(data=conv3b, act_type='leaky', slope=0.1)
conv4a = mx.sym.Convolution(conv3b, pad=(1, 1), kernel=(3, 3), stride=(2, 2), num_filter=512, name=name + 'conv4_0')
conv4a = mx.sym.LeakyReLU(data=conv4a, act_type='leaky', slope=0.1)
conv4b = mx.sym.Convolution(conv4a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=512, name=name + 'conv4_1_0')
conv4b = mx.sym.LeakyReLU(data=conv4b, act_type='leaky', slope=0.1)
conv5a = mx.sym.Convolution(conv4b, pad=(1, 1), kernel=(3, 3), stride=(2, 2), num_filter=512, name=name + 'conv5_0')
conv5a = mx.sym.LeakyReLU(data=conv5a, act_type='leaky', slope=0.1)
conv5b = mx.sym.Convolution(conv5a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=512, name=name + 'conv5_1_0')
conv5b = mx.sym.LeakyReLU(data=conv5b, act_type='leaky', slope=0.1)
conv6a = mx.sym.Convolution(conv5b, pad=(1, 1), kernel=(3, 3), stride=(2, 2), num_filter=1024, name=name + 'conv6_0')
conv6a = mx.sym.LeakyReLU(data=conv6a, act_type='leaky', slope=0.1)
conv6b = mx.sym.Convolution(conv6a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=1024,
name=name + 'conv6_1_0')
conv6b = mx.sym.LeakyReLU(data=conv6b, act_type='leaky', slope=0.1, )
#predict flow
pr6 = mx.sym.Convolution(conv6b, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
name=name + 'predict_flow6')
prediction['loss6'] = pr6
upsample_pr6to5 = mx.sym.Deconvolution(pr6, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
name=name + 'upsampled_flow6_to_5', no_bias=True)
upconv5 = mx.sym.Deconvolution(conv6b, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=512,
name=name + 'deconv5_0', no_bias=False)
upconv5 = mx.sym.LeakyReLU(data=upconv5, act_type='leaky', slope=0.1)
iconv5 = mx.sym.Concat(conv5b, upconv5, upsample_pr6to5, dim=1)
pr5 = mx.sym.Convolution(iconv5, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
name=name + 'predict_flow5')
prediction['loss5'] = pr5
upconv4 = mx.sym.Deconvolution(iconv5, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=256,
name=name + 'deconv4_0', no_bias=False)
upconv4 = mx.sym.LeakyReLU(data=upconv4, act_type='leaky', slope=0.1)
upsample_pr5to4 = mx.sym.Deconvolution(pr5, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
name=name + 'upsampled_flow5_to_4', no_bias=True)
iconv4 = mx.sym.Concat(conv4b, upconv4, upsample_pr5to4)
pr4 = mx.sym.Convolution(iconv4, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
name=name + 'predict_flow4')
prediction['loss4'] = pr4
upconv3 = mx.sym.Deconvolution(iconv4, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=128,
name=name + 'deconv3_0', no_bias=False)
upconv3 = mx.sym.LeakyReLU(data=upconv3, act_type='leaky', slope=0.1)
upsample_pr4to3 = mx.sym.Deconvolution(pr4, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
name= name + 'upsampled_flow4_to_3', no_bias=True)
iconv3 = mx.sym.Concat(conv3b, upconv3, upsample_pr4to3)
pr3 = mx.sym.Convolution(iconv3, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
name=name + 'predict_flow3')
prediction['loss3'] = pr3
upconv2 = mx.sym.Deconvolution(iconv3, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=64,
name=name + 'deconv2_0', no_bias=False)
upconv2 = mx.sym.LeakyReLU(data=upconv2, act_type='leaky', slope=0.1)
upsample_pr3to2 = mx.sym.Deconvolution(pr3, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
name=name + 'upsampled_flow3_to_2', no_bias=True)
iconv2 = mx.sym.Concat(conv2, upconv2, upsample_pr3to2)
pr2 = mx.sym.Convolution(iconv2, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
name=name + 'predict_flow2')
prediction['loss2'] = pr2
flow = mx.sym.UpSampling(arg0=pr2,scale=4,num_filter=2,num_args = 1,sample_type='nearest', name='upsample_flow2_to_1')
# ignore the loss functions with loss scale of zero
keys = loss_scale.keys()
# keys.sort()
#obtain the symbol of the losses
for key in keys:
# loss.append(get_loss(prediction[key] * 20, labels[key], loss_scale[key], name=key + name,get_input=False, is_sparse=is_sparse, type='flow'))
loss.append(mx.sym.MAERegressionOutput(data=prediction[key] * 20, label=labels[key], name=key + name, grad_scale=loss_scale[key]))
# print('loss: ',loss)
#group 暂时不知道为嘛要group
loss_group =mx.sym.Group(loss)
# print('net: ',loss_group)
return loss_group,flow
import gluonbook as gb
import torch
from utils.frame_utils import *
import numpy as np
if __name__ == '__main__':
checkpoint = torch.load("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/flownet2_pytorch/FlowNet2-S_checkpoint.pth.tar")
# # checkpoint是一个字典
print(isinstance(checkpoint['state_dict'], dict))
# # 打印checkpoint字典中的key名
print('keys of checkpoint:')
for i in checkpoint:
print(i)
print('')
# # pytorch 模型参数保存在一个key名为'state_dict'的元素中
state_dict = checkpoint['state_dict']
# # state_dict也是一个字典
print('keys of state_dict:')
for i in state_dict:
print(i)
# print(state_dict[i].size())
print('')
# print(state_dict)
#字典的value是torch.tensor
print(torch.is_tensor(state_dict['conv1.0.weight']))
#查看某个value的size
print(state_dict['conv1.0.weight'].size())
#flownet-mxnet init
loss_scale={'loss2': 1.00,
'loss3': 1.00,
'loss4': 1.00,
'loss5': 1.00,
'loss6': 1.00}
loss,flow = flownet_s(loss_scale=loss_scale,is_sparse=False)
print('loss information: ')
print('loss:',loss)
print('type:',type(loss))
print('list_arguments:',loss.list_arguments())
print('list_outputs:',loss.list_outputs())
print('list_inputs:',loss.list_inputs())
print('')
print('flow information: ')
print('flow:',flow)
print('type:',type(flow))
print('list_arguments:',flow.list_arguments())
print('list_outputs:',flow.list_outputs())
print('list_inputs:',flow.list_inputs())
print('')
name_mxnet = symbol.list_arguments()
print(type(name_mxnet))
for key in name_mxnet:
print(key)
name_mxnet.sort()
for key in name_mxnet:
print(key)
print(name_mxnet)
shapes = (1, 3, 384, 512)
ctx = gb.try_gpu()
# exe = symbol.simple_bind(ctx=ctx, img1=shapes,img2=shapes)
exe = flow.simple_bind(ctx=ctx, img1=shapes, img2=shapes)
print('exe type: ',type(exe))
print('exe: ',exe)
#module
# mod = mx.mod.Module(flow)
# print('mod type: ', type(exe))
# print('mod: ', exe)
pim1 = read_gen("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/data/0000007-img0.ppm")
pim2 = read_gen("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/data/0000007-img1.ppm")
print(pim1.shape)
'''使用pytorch 的state_dict 初始化 mxnet 模型参数'''
for key in state_dict:
# print(type(key))
k_split = key.split('.')
key_mx = '_'.join(k_split)
# print(key,key_mx)
try:
exe.arg_dict[key_mx][:]=state_dict[key].data
except:
print(key,exe.arg_dict[key_mx].shape,state_dict[key].data.shape)
exe.arg_dict['img1'][:] = pim1[np.newaxis, :, :, :].transpose(0, 3, 1, 2).data
exe.arg_dict['img2'][:] = pim2[np.newaxis, :, :, :].transpose(0, 3, 1, 2).data
result = exe.forward()
print('result: ',type(result))
# for tmp in result:
# print(type(tmp))
# print(tmp.shape)
# color = flow2color(exe.outputs[0].asnumpy()[0].transpose(1, 2, 0))
outputs = exe.outputs
print('output type: ',type(outputs))
# for tmp in outputs:
# print(type(tmp))
# print(tmp.shape)
#来自pytroch flownet2
from visualize import flow2color
# color = flow2color(exe.outputs[0].asnumpy()[0].transpose(1,2,0))
flow_color = flow2color(exe.outputs[0].asnumpy()[0].transpose(1, 2, 0))
print('color type:',type(flow_color))
import matplotlib.pyplot as plt
#来自pytorch
from torchvision.transforms import ToPILImage
TF = ToPILImage()
images = TF(flow_color)
images.show()
# plt.imshow(color)
来源:https://blog.csdn.net/Murdock_C/article/details/88043784
猜你喜欢
- 我正在开发某种RESTful API.发生一些错误时,我会抛出一个App :: abort($code,$message)错误.问题是:我希
- 摘要:神经网络的训练的主要流程包括图像输入神经网络, 得到模型的输出结果,计算模型的输出与真实值的损失, 计算损失值的梯度,最后用梯度下降算
- 一、UDP编程1.客户端Client:发起访问的一方。2.服务器端3.server段编程(1)建立socket,socket是负责具体通信的
- 找到对应页面,将Reopen last project on startup前面的勾去掉来源:https://blog.csdn.net/l
- python列表元素去重后如何保持原来的顺序不变原列表:list1 = [1,2,1,4,9,3,5,2,6,7,3,1,6,8,4,0]去
- 今天刚看到ChinaUI.com新改版的网站,觉得优艾网是越来越漂亮了,这次的变化的确是挺大的。一早大概的看了一下就关了,后来晚上又打开看时
- 文章目录 微信登录问题Python chrome driver操作导入库并声明浏览器:完整流程:用js来预约生成js代码 主函数——程序出错
- 有时候,我们需要将TensorFlow的模型导出为单个文件(同时包含模型架构定义与权重),方便在其他地方使用(如在c++中部署网络)。利用t
- 本文介绍了用ASP的AdoDb.Stream读取/写入UTF-8编码格式的文件的方法:函数名称:ReadTextFile 作用:利用AdoD
- 1、ModuleNotFoundError: No module named ‘scipy.spatial.transf
- 一、web.py简介web.py是一款轻量级的Python web开发框架,简单、高效、学习成本低,特别适合作为python web开发的入
- 本文实例讲述了Python3中函数参数传递方式。分享给大家供大家参考,具体如下:之前在看北理工嵩天等老师的python3的课程,在第五周中老
- (1) 单人脸情况import cv2import dlibpath = "1.jpg"img = cv2.imread
- 使用邻接矩阵实现图及Dijkstra算法# 邻接矩阵实现无向图 Dijkstra算法inf = float("inf")
- 由于CPython实现中的GIL的限制,python中的多线程其实并不是真正的多线程,如果想要充分地使用多核CPU的资源,在python中大
- PHP echo() 函数实例输出一些文本:<?php echo "Hello world!"; ?>定义和
- Pythonpython 真的太好用了,但是它真的好慢啊(哭死) ; C++ 很快,但是真的好难写啊,此生能不碰它就不碰它。老天啊,有没有什
- 套接字socket套接字(socket)是计算机之间进行通信的一种技术,它允许不同主机上的进程之间进行数据交换。在Python中,我们可以使
- 下午的时候,配好了OpenCV的Python环境,OpenCV的Python环境搭建。于是迫不及待的想体验一下opencv的人脸识别,如下文
- 1.在线定制下载echartshttps://echarts.apache.org/zh/builder.html2.创建一个django项