viewsetting

矩阵求导的Trace Trick

公式推导

定义对于标量函数对于矩阵的导数，可以记做：首先对于标量函数对于标量的微分，可以写作：然后推广到标量函数对于向量的微分，可以理解为每一个向量的分量对于标量微分的贡献之和。观察上面的式子，向量微分的左边是函数对于向量的偏微分的转置，从而转化为了向量的内积形式。因此，推广到矩阵 : 上式中，表示的是矩阵的trace，迹，含义是矩阵主对角线元素之和，之所以映入矩阵的trace是因为...

Posted by viewsetting on May 17, 2020

CS224n Lecture 17 Coreference Resolution

Lecture Note

指代消解的步骤指代消解任务可以分为两步式的任务：检测Mention的边界对于Mention进行聚类对于第一个任务，举个例子： “I voted for Nader because he was most aligned with my values,” she said 在这一句中，如果只是做mention检测的任务的话我们得到的结果应该是： “...

Posted by viewsetting on December 2, 2019

HotpotQA数据集探索

Dataset笔记

HotpotQA数据集备注文件 TRAIN: hotpot_train_v1.1.json http://curtis.ml.cmu.edu/datasets/hotpot/hotpot_train_v1.1.json DEV: hotpot_dev_distractor_v1.json http://curtis.ml.cmu.edu/datasets/hotpot/hotpot_...

Posted by viewsetting on October 20, 2019

pytorch-transformers本地加载

备忘录

本文将以Pytorch-transformers库中的Bert为例。从本地文件中加载模型下载至本地 BERT的预训练模型下载地址如下: https://github.com/huggingface/transformers/blob/master/transformers/modeling_bert.py line 37 BERT_PRETRAINED_MODEL_ARCHIV...

Posted by viewsetting on October 17, 2019

闲聊Attention

机器翻译的一组实验demo

实验目的比较三种Attention机制在机器翻译任务上的表现。分别为：Bahdanau Attention(D. Bahdanau et al. ,2015)，Convolutional Attention(Jonas Gehring et al. , 2017)，以及基于Self Attention机制的Transformer(Vaswani et al. ,2017)。 ...

Posted by viewsetting on October 15, 2019

NLTK Tokenizer

notes

nltk.tokenize To be updated… sent_tokenize nltk.tokenize.sent_tokenize(context) code: Tokenize a paragraph in sentence level, then shuffle it. def shuffle_context(context): sent_tokenize_...

Posted by viewsetting on September 28, 2019

自然语言处理课堂作业

NLP Course Assignment

Regular Syntax to NFA Data 示例数据位于：\data.txt 中,如下所示格式： S->aA|$ A->aB|bA B->aS|bA|$ S->bB 实现位于/model.py中，基于Python 3。节点类：每个节点的属性就是名字，作为键值在Graph对象中可以直接查询以访问。节点中还保存了后继节点的信息。 class ...

Posted by viewsetting on September 25, 2019

PyTorch模型保存与加载

pytorch notes

整个模型的保存与加载 torch.save(model_object, 'model.pth') model = torch.load('model.pth') 只保存以及加载模型参数 torch.save(model_object.state_dict(), 'params.pth') model_object.load_state_dict(torch.load('p...

Posted by viewsetting on September 24, 2019

Shift from CPU to GPU

PyTorch Notes

Check CUDA status if torch.cuda.is_available(): print("CUDA is READY") Model model = Seq2Seq() model.cuda(0) Variable hidden = Variable(torch.zeros(1, 1, n_hidden)) # Previous hidden =...

Posted by viewsetting on May 25, 2019

Numpy Guidebook

A Memo

Basic Import Numpy import numpy as np Initialize An Array a = np.array([4,5,6]) print(a.dtype) print(a.shape) print: int32 (3,) 2-D Array b = np.array([[4,5,6],[1,2,3]]) print(b.shape) p...

Posted by viewsetting on May 24, 2019

Blog of viewsetting