{"id":"arxiv:2111.01243","kind":"article","title":"Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey","authors":["Bonan Min","Hayley Ross","Elior Sulem","Amir Pouran Ben Veyseh","Thien Huu Nguyen","Oscar Sainz","Eneko Agirre","Ilana Heintz","Dan Roth"],"author_count":9,"year":2023,"published":"2023-06-27","venue":"ACM Computing Surveys","cited_by":1386,"doi":"10.1145/3605943","arxiv_id":"2111.01243","nct_id":null,"source_url":"https://doi.org/10.1145/3605943","archive_url":"https://space-kitty.com/papers/arxiv:2111.01243","updated":"2021-11-01","pmid":null,"pmcid":null,"openalex_id":"W4382246105","pdf_url":"https://dl.acm.org/doi/pdf/10.1145/3605943","abstract":"Large, pre-trained transformer-based language models such as BERT have drastically changed the Natural Language Processing (NLP) field. We present a survey of recent work that uses these large language models to solve NLP tasks via pre-training then fine-tuning, prompting, or text generation approaches. We also present approaches that use pre-trained language models to generate data for training augmentation or other purposes. We conclude with discussions on limitations and suggested directions for future research.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Topic Modeling"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"llm-architectures","name":"LLM Architectures & Pretraining","domain_slug":"ai"}],"cited_in_dossiers":[]}