# Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey

Article · Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, Dan Roth

- Archive id: arxiv:2111.01243
- Published: 2023-06-27
- Venue: ACM Computing Surveys
- DOI: https://doi.org/10.1145/3605943
- arXiv: https://arxiv.org/abs/2111.01243
- Citations: 1,386
- Source URL: https://doi.org/10.1145/3605943
- PDF: https://dl.acm.org/doi/pdf/10.1145/3605943
- Provenance: openalex+arxiv

## Abstract

Large, pre-trained transformer-based language models such as BERT have drastically changed the Natural Language Processing (NLP) field. We present a survey of recent work that uses these large language models to solve NLP tasks via pre-training then fine-tuning, prompting, or text generation approaches. We also present approaches that use pre-trained language models to generate data for training augmentation or other purposes. We conclude with discussions on limitations and suggested directions for future research.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [LLM Architectures & Pretraining](https://space-kitty.com/topics/llm-architectures)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.