Jiakui/awesome-bert: bert nlp papers, applications and github resources, includ ...

原作者: [db:作者] 来自: 网络收藏邀请

开源软件名称：

Jiakui/awesome-bert

开源软件地址：

https://github.com/Jiakui/awesome-bert

开源编程语言：

开源软件介绍：

This repository is to collect BERT related resources.

AD: a repository for graph convolutional networks at https://github.com/Jiakui/awesome-gcn (resources for graph convolutional networks （图卷积神经网络相关资源）).

Papers:

arXiv:1810.04805, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , Authors: Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova

Click to see more

arXiv:1812.06705, Conditional BERT Contextual Augmentation, Authors: Xing Wu, Shangwen Lv, Liangjun Zang, Jizhong Han, Songlin Hu
arXiv:1812.03593, SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering, Authors: Chenguang Zhu, Michael Zeng, Xuedong Huang
arXiv:1901.02860, Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context, Authors: Zihang Dai, Zhilin Yang, Yiming Yang, William W. Cohen, Jaime Carbonell, Quoc V. Le and Ruslan Salakhutdinov.
arXiv:1901.04085, Passage Re-ranking with BERT, Authors: Rodrigo Nogueira, Kyunghyun Cho
arXiv:1902.02671, BERT and PALs: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning, Authors: Asa Cooper Stickland, Iain Murray
arXiv:1904.02232, BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis, Authors: Hu Xu, Bing Liu, Lei Shu, Philip S. Yu, [code]

Github Repositories:

official implement:

google-research/bert, officical TensorFlow code and pre-trained models for BERT ,

implement of BERT besides tensorflow:

codertimo/BERT-pytorch, Google AI 2018 BERT pytorch implementation,
huggingface/pytorch-pretrained-BERT, A PyTorch implementation of Google AI's BERT model with script to load Google's pre-trained models,
dmlc/gluon-nlp, Gluon + MXNet implementation that reproduces BERT pretraining and finetuning on GLUE benchmark, SQuAD, etc,
dbiir/UER-py, UER-py is a toolkit for pre-training on general-domain corpus and fine-tuning on downstream task. UER-py maintains model modularity and supports research extensibility. It facilitates the use of different pre-training models (e.g. BERT), and provides interfaces for users to further extend upon.
BrikerMan/Kashgari, Simple, Keras-powered multilingual NLP framework, allows you to build your models in 5 minutes for named entity recognition (NER), part-of-speech tagging (PoS) and text classification tasks. Includes BERT, GPT-2 and word2vec embedding.
kaushaltrivedi/fast-bert, Super easy library for BERT based NLP models,

Click to see more

Separius/BERT-keras, Keras implementation of BERT with pre-trained weights,
soskek/bert-chainer, Chainer implementation of "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding",
innodatalabs/tbert, PyTorch port of BERT ML model
guotong1988/BERT-tensorflow, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
dreamgonfly/BERT-pytorch, PyTorch implementation of BERT in "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"
CyberZHG/keras-bert, Implementation of BERT that could load official pre-trained models for feature extraction and prediction
soskek/bert-chainer, Chainer implementation of "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"
MaZhiyuanBUAA/bert-tf1.4.0, bert-tf1.4.0
dhlee347/pytorchic-bert, Pytorch Implementation of Google BERT,
kpot/keras-transformer, Keras library for building (Universal) Transformers, facilitating BERT and GPT models,
miroozyx/BERT_with_keras, A Keras version of Google's BERT model,
conda-forge/pytorch-pretrained-bert-feedstock, A conda-smithy repository for pytorch-pretrained-bert. ,
Rshcaroline/BERT_Pytorch_fastNLP, A PyTorch & fastNLP implementation of Google AI's BERT model.
nghuyong/ERNIE-Pytorch, ERNIE Pytorch Version,

Pretrained BERT weights:

brightmart/roberta_zh, RoBERTa for Chinese, 中文预训练RoBERTa模型,
ymcui/Chinese-BERT-wwm, Pre-Training with Whole Word Masking for Chinese BERT（中文BERT-wwm预训练模型） https://arxiv.org/abs/1906.08101,
thunlp/OpenCLaP,Open Chinese Language Pre-trained Model Zoo, OpenCLaP：多领域开源中文预训练语言模型仓库,
ymcui/Chinese-PreTrained-XLNet, Pre-Trained Chinese XLNet（中文XLNet预训练模型）,
brightmart/xlnet_zh, 中文预训练XLNet模型: Pre-Trained Chinese XLNet_Large,

improvement over BERT:

thunlp/ERNIE, Source code and dataset for ACL 2019 paper "ERNIE: Enhanced Language Representation with Informative Entities", imporove bert with heterogeneous information fusion.
PaddlePaddle/LARK, LAnguage Representations Kit, PaddlePaddle implementation of BERT. It also contains an improved version of BERT, ERNIE, for chinese NLP tasks. BERT 的中文改进版 ERNIE,
zihangdai/xlnet, XLNet: Generalized Autoregressive Pretraining for Language Understanding,
kimiyoung/transformer-xl, Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context, This repository contains the code in both PyTorch and TensorFlow for our paper.
GaoPeng97/transformer-xl-chinese, transformer xl在中文文本生成上的尝试。（transformer xl for text generation of chinese）,
PaddlePaddle/ERNIE, An Implementation of ERNIE For Language Understanding (including Pre-training models and Fine-tuning tools) BERT 的中文改进版 ERNIE,
pytorch/fairseq, Facebook AI Research Sequence-to-Sequence Toolkit written in Python. RoBERTa: A Robustly Optimized BERT Pretraining Approach,
facebookresearch/SpanBERT, Code for using and evaluating SpanBERT. , This repository contains code and models for the paper: SpanBERT: Improving Pre-training by Representing and Predicting Spans.,
brightmart/albert_zh, 海量中文预训练ALBERT模型, A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS https://arxiv.org/pdf/1909.11942.pdf,
lonePatient/albert_pytorch, A Lite Bert For Self-Supervised Learning Language Representations,
kpe/bert-for-tf2, A Keras TensorFlow 2.0 implementation of BERT, ALBERT and adapter-BERT. https://github.com/kpe/bert-for-tf2,

other resources for BERT:

brightmart/bert_language_understanding, Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN,
Y1ran/NLP-BERT--ChineseVersion, 谷歌自然语言处理模型BERT：论文解析与python代码,

Click to see more

yangbisheng2009/cn-bert, BERT在中文NLP的应用, 语法检查
JayYip/bert-multiple-gpu, A multiple GPU support version of BERT,
HighCWu/keras-bert-tpu, Implementation of BERT that could load official pre-trained models for feature extraction and prediction on TPU,
Willyoung2017/Bert_Attempt, PyTorch Pretrained Bert,
Pydataman/bert_examples, some examples of bert, run_classifier.py 是基于谷歌bert实现了Quora Insincere Questions Classification二分类比赛。run_ner.py是基于瑞金医院AI大赛第一赛季数据和bert写的一个命名实体识别。
guotong1988/BERT-chinese, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 中文汉语
zhongyunuestc/bert_multitask, 多任务task
Microsoft/AzureML-BERT, End-to-end walk through for fine-tuning BERT using Azure Machine Learning ,
bigboNed3/bert_serving, export bert model for serving,
yoheikikuta/bert-japanese, BERT with SentencePiece for Japanese text.
whqwill/seq2seq-keyphrase-bert, add BERT to encoder part for https://github.com/memray/seq2seq-keyphrase-pytorch,
algteam/bert-examples, bert-demo,
cedrickchee/awesome-bert-nlp, A curated list of NLP resources focused on BERT, attention mechanism, Transformer networks, and transfer learning.
cnfive/cnbert, 中文注释一下bert代码功能,
brightmart/bert_customized, bert with customized features,
JayYip/bert-multitask-learning, BERT for Multitask Learning,
yuanxiaosc/BERT_Paper_Chinese_Translation, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 论文的中文翻译。Chinese Translation! https://yuanxiaosc.github.io/2018/12/…,
yaserkl/BERTvsULMFIT, Comparing Text Classification results using BERT embedding and ULMFIT embedding,
kpot/keras-transformer, Keras library for building (Universal) Transformers, facilitating BERT and GPT models,
1234560o/Bert-model-code-interpretation, 解读tensorflow版本Bert中modeling.py数据流
cdathuraliya/bert-inference, A helper class for Google BERT (Devlin et al., 2018) to support online prediction and model pipelining.
gameofdimension/java-bert-predict, turn bert pretrain checkpoint into saved model for a feature extracting demo in java
1234560o/Bert-model-code-interpretation, 解读tensorflow版本Bert中modeling.py数据流