pytorch+lstm实现的pos示例
作者:say_c_box 发布时间:2023-08-11 22:02:10
标签:pytorch,lstm,pos
学了几天终于大概明白pytorch怎么用了
这个是直接搬运的官方文档的代码
之后会自己试着实现其他nlp的任务
# Author: Robert Guthrie
import torch
import torch.autograd as autograd
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
torch.manual_seed(1)
lstm = nn.LSTM(3, 3) # Input dim is 3, output dim is 3
inputs = [autograd.Variable(torch.randn((1, 3)))
for _ in range(5)] # make a sequence of length 5
# initialize the hidden state.
hidden = (autograd.Variable(torch.randn(1, 1, 3)),
autograd.Variable(torch.randn((1, 1, 3))))
for i in inputs:
# Step through the sequence one element at a time.
# after each step, hidden contains the hidden state.
out, hidden = lstm(i.view(1, 1, -1), hidden)
# alternatively, we can do the entire sequence all at once.
# the first value returned by LSTM is all of the hidden states throughout
# the sequence. the second is just the most recent hidden state
# (compare the last slice of "out" with "hidden" below, they are the same)
# The reason for this is that:
# "out" will give you access to all hidden states in the sequence
# "hidden" will allow you to continue the sequence and backpropagate,
# by passing it as an argument to the lstm at a later time
# Add the extra 2nd dimension
inputs = torch.cat(inputs).view(len(inputs), 1, -1)
hidden = (autograd.Variable(torch.randn(1, 1, 3)), autograd.Variable(
torch.randn((1, 1, 3)))) # clean out hidden state
out, hidden = lstm(inputs, hidden)
#print(out)
#print(hidden)
#准备数据
def prepare_sequence(seq, to_ix):
idxs = [to_ix[w] for w in seq]
tensor = torch.LongTensor(idxs)
return autograd.Variable(tensor)
training_data = [
("The dog ate the apple".split(), ["DET", "NN", "V", "DET", "NN"]),
("Everybody read that book".split(), ["NN", "V", "DET", "NN"])
]
word_to_ix = {}
for sent, tags in training_data:
for word in sent:
if word not in word_to_ix:
word_to_ix[word] = len(word_to_ix)
print(word_to_ix)
tag_to_ix = {"DET": 0, "NN": 1, "V": 2}
# These will usually be more like 32 or 64 dimensional.
# We will keep them small, so we can see how the weights change as we train.
EMBEDDING_DIM = 6
HIDDEN_DIM = 6
#继承自nn.module
class LSTMTagger(nn.Module):
def __init__(self, embedding_dim, hidden_dim, vocab_size, tagset_size):
super(LSTMTagger, self).__init__()
self.hidden_dim = hidden_dim
#一个单词数量到embedding维数的矩阵
self.word_embeddings = nn.Embedding(vocab_size, embedding_dim)
#传入两个维度参数
# The LSTM takes word embeddings as inputs, and outputs hidden states
# with dimensionality hidden_dim.
self.lstm = nn.LSTM(embedding_dim, hidden_dim)
#线性layer从隐藏状态空间映射到tag便签
# The linear layer that maps from hidden state space to tag space
self.hidden2tag = nn.Linear(hidden_dim, tagset_size)
self.hidden = self.init_hidden()
def init_hidden(self):
# Before we've done anything, we dont have any hidden state.
# Refer to the Pytorch documentation to see exactly
# why they have this dimensionality.
# The axes semantics are (num_layers, minibatch_size, hidden_dim)
return (autograd.Variable(torch.zeros(1, 1, self.hidden_dim)),
autograd.Variable(torch.zeros(1, 1, self.hidden_dim)))
def forward(self, sentence):
embeds = self.word_embeddings(sentence)
lstm_out, self.hidden = self.lstm(embeds.view(len(sentence), 1, -1), self.hidden)
tag_space = self.hidden2tag(lstm_out.view(len(sentence), -1))
tag_scores = F.log_softmax(tag_space)
return tag_scores
#embedding维度,hidden维度,词语数量,标签数量
model = LSTMTagger(EMBEDDING_DIM, HIDDEN_DIM, len(word_to_ix), len(tag_to_ix))
#optim中存了各种优化算法
loss_function = nn.NLLLoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)
# See what the scores are before training
# Note that element i,j of the output is the score for tag j for word i.
inputs = prepare_sequence(training_data[0][0], word_to_ix)
tag_scores = model(inputs)
print(tag_scores)
for epoch in range(300): # again, normally you would NOT do 300 epochs, it is toy data
for sentence, tags in training_data:
# Step 1. Remember that Pytorch accumulates gradients.
# We need to clear them out before each instance
model.zero_grad()
# Also, we need to clear out the hidden state of the LSTM,
# detaching it from its history on the last instance.
model.hidden = model.init_hidden()
# Step 2. Get our inputs ready for the network, that is, turn them into
# Variables of word indices.
sentence_in = prepare_sequence(sentence, word_to_ix)
targets = prepare_sequence(tags, tag_to_ix)
# Step 3. Run our forward pass.
tag_scores = model(sentence_in)
# Step 4. Compute the loss, gradients, and update the parameters by
# calling optimizer.step()
loss = loss_function(tag_scores, targets)
loss.backward()
optimizer.step()
# See what the scores are after training
inputs = prepare_sequence(training_data[0][0], word_to_ix)
tag_scores = model(inputs)
# The sentence is "the dog ate the apple". i,j corresponds to score for tag j
# for word i. The predicted tag is the maximum scoring tag.
# Here, we can see the predicted sequence below is 0 1 2 0 1
# since 0 is index of the maximum value of row 1,
# 1 is the index of maximum value of row 2, etc.
# Which is DET NOUN VERB DET NOUN, the correct sequence!
print(tag_scores)
来源:https://blog.csdn.net/say_c_box/article/details/78802770
0
投稿
猜你喜欢
- python语言的一大优势:为科学计算提供了大量的支持功能,math模块提供了很多数学计算函数。math模块定义了一些数学模块,这个模块属于
- 基本开发环境· Python 3.6· Pycharm相关模块使用import requestsimport timefrom tkinte
- 如下所示:class Login(QMainWindow): """登录窗口""
- 本文实例讲述了Python实现返回数组中第i小元素的方法。分享给大家供大家参考,具体如下:#! /usr/bin/env python#co
- 看了一个月的文档和资料以后,终于让我参与到项目中来了,哈哈,痛快!虽然只是让我解决一个小问题,不过有活干就是好。在写代码的过程中遇到了一个小
- ASP里两种常用的生成文件的方式是:利用ADODB.Stream生成文件和利用Scripting.FileSystemObject(fso)
- Mysqli是php5之后才有的功能,没有开启扩展的朋友可以打开您的php.ini的配置文件。 查找下面的语句:;extension=php
- 本段代码是先将需要转换经纬度的地址爬取在 ‘地址.csv' 文件里,文件截图示例:代码展示# coding=utf-8# SPL#
- python 的语法定义和C++、matlab、java 还是很有区别的。1. 括号与函数调用def devided_3(x):  
- 为什么要用jsonpath就跟为什么要用xpath一样,jsonpath的设计灵感来源于xpath。一个强大的json数据提取工具。让用户不
- 在国内利用Python从Internet上爬取数据时,有些网站或API接口被限速或屏蔽,这时使用代理可以加速爬取过程,减少请求失败,Pyth
- python的PIL库简直好用的不得了,PIL下面的Image库更是封装了很多对图片处理的函数,关于Image库的介绍和使用,看这里:htt
- 一、介绍Django特点:具有完整的封装,开发者可以高效率的开发项目,Django将大部分的功能进行了封装,开发者只需要调用即可,如此,大大
- 在编写自动化测试用例的时候,每次登录都需要输入验证码,后来想把让python自己识别图片里的验证码,不需要自己手动登陆,所以查了一下识别功能
- 1 安装pyqt51.1 安装pyqt5pip install PyQt51.2 安装pyqt5常用工具包pip install PyQt5
- Python支持四种不同的数值类型,包括int(整数)long(长整数)float(浮点实际值)complex (复数),本文章向码农介绍p
- Python爬虫:一些常用的爬虫技巧总结爬虫在开发过程中也有很多复用的过程,这里总结一下,以后也能省些事情。1、基本抓取网页get方法imp
- 实例如下所示:from tensorflow.python import pywrap_tensorflowcheckpoint_path
- 个人想到的解决方法有两种,一种是 .replace(' old ',' new ')
- 话说土匪老湿在他的大作 《交互设计之回归篇》 里曝光了上次有意思小组竞赛我们小组分享的话题 “瞬间的快感”,但这一极具噱