{"topic":"llm-architectures","sort":"top","limit":25,"offset":0,"count":25,"papers":[{"id":"arxiv:2005.14165","kind":"preprint","title":"Language Models are Few-Shot Learners","authors":["Brown, Tom B.","Benjamin Mann","Nick Ryder","Melanie Subbiah","Jared Kaplan A","Prafulla Dhariwal","Arvind Neelakantan","Pranav Shyam","Girish Sastry","Amanda Askell","Sandhini Agarwal","Ariel Herbert-Voss","Gretchen Marie Krueger","Tom Henighan","Rewon Child","Ramesh, Aditya","Ziegler, Daniel M.","Jeffrey Wu","Clemens Winter","Christopher Hesse","Mark I-Cheng Chen","Sigler, Eric","Mateusz Litwin","Scott Gray","Benjamin Chess"],"author_count":31,"year":2020,"published":"2020-05-28","venue":null,"cited_by":23014,"doi":null,"arxiv_id":"2005.14165","nct_id":null,"source_url":"http://arxiv.org/abs/2005.14165","archive_url":"https://space-kitty.com/papers/arxiv:2005.14165","updated":"2020-07-22","pmid":null,"pmcid":null,"openalex_id":"W3030163527","pdf_url":"https://arxiv.org/pdf/2005.14165","abstract":"Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting. For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model. GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic. At the same time, we also identify some datasets where GPT-3's few-shot learning still struggles, as well as some datasets where GPT-3 faces methodological issues related to training on large web corpora. Finally, we find that GPT-3 can generate samples of news articles which human evaluators have difficulty distinguishing from articles written by humans. We discuss broader societal impacts of this finding and of GPT-3 in general.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":false,"categories":["Topic Modeling"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2303.18223","kind":"article","title":"A Survey of Large Language Models","authors":["Wayne Xin Zhao","Kun Zhou","Junyi Li","Tianyi Tang","Zican Dong","Yupeng Hou","Beichen Zhang","Yingqian Min","Junjie Zhang","Peiyu Liu","Xiaolei Wang","Yifan Du","Yushuo Chen","Zhipeng Chen","Jinhao Jiang","Ruiyang Ren","Yifan Li","Xinyu Tang","Yiwen Hu","Jian‐Yun Nie","Ji-Rong Wen"],"author_count":23,"year":2026,"published":"2026-05-09","venue":"Frontiers of Computer Science","cited_by":2085,"doi":"10.1007/s11704-026-60308-3","arxiv_id":"2303.18223","nct_id":null,"source_url":"https://doi.org/10.1007/s11704-026-60308-3","archive_url":"https://space-kitty.com/papers/arxiv:2303.18223","updated":"2026-03-18","pmid":null,"pmcid":null,"openalex_id":"W4362515116","pdf_url":"https://link.springer.com/content/pdf/10.1007/s11704-026-60308-3.pdf","abstract":"Language is essentially a complex, intricate system of human expressions governed by grammatical rules. It poses a significant challenge to develop capable AI algorithms for comprehending and grasping a language. As a major approach, language modeling has been widely studied for language understanding and generation in the past two decades, evolving from statistical language models to neural language models. Recently, pre-trained language models (PLMs) have been proposed by pre-training Transformer models over large-scale corpora, showing strong capabilities in solving various NLP tasks. Since researchers have found that model scaling can lead to performance improvement, they further study the scaling effect by increasing the model size to an even larger size. Interestingly, when the parameter scale exceeds a certain level, these enlarged language models not only achieve a significant performance improvement but also show some special abilities that are not present in small-scale language models. To discriminate the difference in parameter scale, the research community has coined the term large language models (LLM) for the PLMs of significant size. Recently, the research on LLMs has been largely advanced by both academia and industry, and a remarkable progress is the launch of ChatGPT, which has attracted widespread attention from society. The technical evolution of LLMs has been making an important impact on the entire AI community, which would revolutionize the way how we develop and use AI algorithms. In this survey, we review the recent advances of LLMs by introducing the background, key findings, and mainstream techniques. In particular, we focus on four major aspects of LLMs, namely pre-training, adaptation tuning, utilization, and capacity evaluation. Besides, we also summarize the available resources for developing LLMs and discuss the remaining issues for future directions.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":"cc-by","is_open_access":true,"peer_reviewed":null,"categories":["Topic Modeling"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:1901.08746","kind":"article","title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","authors":["Jinhyuk Lee","Wonjin Yoon","Sungdong Kim","Donghyeon Kim","Sunkyu Kim","Chan Ho So","Jaewoo Kang"],"author_count":7,"year":2019,"published":"2019-09-05","venue":"Bioinformatics","cited_by":7208,"doi":"10.1093/bioinformatics/btz682","arxiv_id":"1901.08746","nct_id":null,"source_url":"https://doi.org/10.1093/bioinformatics/btz682","archive_url":"https://space-kitty.com/papers/arxiv:1901.08746","updated":"2019-10-18","pmid":"31501885","pmcid":null,"openalex_id":"W2911489562","pdf_url":"https://academic.oup.com/bioinformatics/article-pdf/36/4/1234/48983216/bioinformatics_36_4_1234.pdf","abstract":"Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processing (NLP), extracting valuable information from biomedical literature has gained popularity among researchers, and deep learning has boosted the development of effective biomedical text mining models. However, directly applying the advancements in NLP to biomedical text mining often yields unsatisfactory results due to a word distribution shift from general domain corpora to biomedical corpora. In this article, we investigate how the recently introduced pre-trained language model BERT can be adapted for biomedical corpora. We introduce BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining), which is a domain-specific language representation model pre-trained on large-scale biomedical corpora. With almost the same architecture across tasks, BioBERT largely outperforms BERT and previous state-of-the-art models in a variety of biomedical text mining tasks when pre-trained on biomedical corpora. While BERT obtains performance comparable to that of previous state-of-the-art models, BioBERT significantly outperforms them on the following three representative biomedical text mining tasks: biomedical named entity recognition (0.62% F1 score improvement), biomedical relation extraction (2.80% F1 score improvement) and biomedical question answering (12.24% MRR improvement). Our analysis results show that pre-training BERT on biomedical corpora helps it to understand complex biomedical texts. We make the pre-trained weights of BioBERT freely available at https://github.com/naver/biobert-pretrained, and the source code for fine-tuning BioBERT available at https://github.com/dmis-lab/biobert.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":"cc-by","is_open_access":true,"peer_reviewed":null,"categories":["Biomedical Text Mining and Ontologies"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2101.01169","kind":"article","title":"Transformers in Vision: A Survey","authors":["Salman Hameed Khan","Muzammal Naseer","Munawar Hayat","Syed Waqas Zamir","Fahad Shahbaz Khan","Mubarak A. Shah"],"author_count":6,"year":2022,"published":"2022-01-06","venue":"ACM Computing Surveys","cited_by":3593,"doi":"10.1145/3505244","arxiv_id":"2101.01169","nct_id":null,"source_url":"https://doi.org/10.1145/3505244","archive_url":"https://space-kitty.com/papers/arxiv:2101.01169","updated":"2022-01-19","pmid":null,"pmcid":null,"openalex_id":"W3119997354","pdf_url":"https://arxiv.org/pdf/2101.01169","abstract":"Astounding results from Transformer models on natural language tasks have intrigued the vision community to study their application to computer vision problems. Among their salient benefits, Transformers enable modeling long dependencies between input sequence elements and support parallel processing of sequence as compared to recurrent networks e.g., Long short-term memory (LSTM). Different from convolutional networks, Transformers require minimal inductive biases for their design and are naturally suited as set-functions. Furthermore, the straightforward design of Transformers allows processing multiple modalities (e.g., images, videos, text and speech) using similar processing blocks and demonstrates excellent scalability to very large capacity networks and huge datasets. These strengths have led to exciting progress on a number of vision tasks using Transformer networks. This survey aims to provide a comprehensive overview of the Transformer models in the computer vision discipline. We start with an introduction to fundamental concepts behind the success of Transformers i.e., self-attention, large-scale pre-training, and bidirectional encoding. We then cover extensive applications of transformers in vision including popular recognition tasks (e.g., image classification, object detection, action recognition, and segmentation), generative modeling, multi-modal tasks (e.g., visual-question answering, visual reasoning, and visual grounding), video processing (e.g., activity recognition, video forecasting), low-level vision (e.g., image super-resolution, image enhancement, and colorization) and 3D analysis (e.g., point cloud classification and segmentation). We compare the respective advantages and limitations of popular techniques both in terms of architectural design and their experimental value. Finally, we provide an analysis on open research directions and possible future works.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":null,"categories":["Advanced Neural Network Applications"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2106.09685","kind":"preprint","title":"LoRA: Low-Rank Adaptation of Large Language Models","authors":["J. Edward Hu","Yelong Shen","Phillip Wallis","Zeyuan Allen-Zhu","Yuanzhi Li","Shean Wang","Wang, Lu","Chen, Weizhu"],"author_count":8,"year":2021,"published":"2021-06-17","venue":null,"cited_by":2501,"doi":null,"arxiv_id":"2106.09685","nct_id":null,"source_url":"http://arxiv.org/abs/2106.09685","archive_url":"https://space-kitty.com/papers/arxiv:2106.09685","updated":"2021-10-16","pmid":null,"pmcid":null,"openalex_id":"W3168867926","pdf_url":"https://arxiv.org/pdf/2106.09685","abstract":"An important paradigm of natural language processing consists of large-scale pre-training on general domain data and adaptation to particular tasks or domains. As we pre-train larger models, full fine-tuning, which retrains all model parameters, becomes less feasible. Using GPT-3 175B as an example -- deploying independent instances of fine-tuned models, each with 175B parameters, is prohibitively expensive. We propose Low-Rank Adaptation, or LoRA, which freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks. Compared to GPT-3 175B fine-tuned with Adam, LoRA can reduce the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times. LoRA performs on-par or better than fine-tuning in model quality on RoBERTa, DeBERTa, GPT-2, and GPT-3, despite having fewer trainable parameters, a higher training throughput, and, unlike adapters, no additional inference latency. We also provide an empirical investigation into rank-deficiency in language model adaptation, which sheds light on the efficacy of LoRA. We release a package that facilitates the integration of LoRA with PyTorch models and provide our implementations and model checkpoints for RoBERTa, DeBERTa, and GPT-2 at https://github.com/microsoft/LoRA.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":false,"categories":["Topic Modeling"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2307.06435","kind":"article","title":"A Comprehensive Overview of Large Language Models","authors":["Humza Naveed","Asad Ullah Khan","Shi Qiu","Muhammad Saqib","Saeed Anwar","Muhammad Usman","Naveed Akhtar","Nick M. Barnes","Ajmal Saeed Mian"],"author_count":9,"year":2025,"published":"2025-06-18","venue":"ACM Transactions on Intelligent Systems and Technology","cited_by":1255,"doi":"10.1145/3744746","arxiv_id":"2307.06435","nct_id":null,"source_url":"https://doi.org/10.1145/3744746","archive_url":"https://space-kitty.com/papers/arxiv:2307.06435","updated":"2024-10-17","pmid":null,"pmcid":null,"openalex_id":"W4411403346","pdf_url":"https://arxiv.org/pdf/2307.06435","abstract":"Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks and beyond. This success of LLMs has led to a large influx of research contributions in this direction. These works encompass diverse topics such as architectural innovations, better training strategies, context length improvements, fine-tuning, multi-modal LLMs, robotics, datasets, benchmarking, efficiency, and more. With the rapid development of techniques and regular breakthroughs in LLM research, it has become considerably challenging to perceive the bigger picture of the advances in this direction. Considering the rapidly emerging plethora of literature on LLMs, it is imperative that the research community is able to benefit from a concise yet comprehensive overview of the recent developments in this field. This article provides an overview of the existing literature on a broad range of LLM-related concepts. Our self-contained comprehensive overview of LLMs discusses relevant background concepts along with covering the advanced topics at the frontier of research in LLMs. This review article is intended to not only provide a systematic survey but also a quick comprehensive reference for the researchers and practitioners to draw insights from extensive informative summaries of the existing works to advance the LLM research.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":"cc-by","is_open_access":true,"peer_reviewed":null,"categories":["Topic Modeling"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:1908.02265","kind":"preprint","title":"ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks","authors":["Jiasen Lu","Dhruv Batra","Devi Parikh","Stefan Lee"],"author_count":4,"year":2019,"published":"2019-08-06","venue":null,"cited_by":2111,"doi":null,"arxiv_id":"1908.02265","nct_id":null,"source_url":"http://arxiv.org/abs/1908.02265","archive_url":"https://space-kitty.com/papers/arxiv:1908.02265","updated":"2019-08-06","pmid":null,"pmcid":null,"openalex_id":"W2966715458","pdf_url":"https://arxiv.org/pdf/1908.02265","abstract":"We present ViLBERT (short for Vision-and-Language BERT), a model for learning task-agnostic joint representations of image content and natural language. We extend the popular BERT architecture to a multi-modal two-stream model, pro-cessing both visual and textual inputs in separate streams that interact through co-attentional transformer layers. We pretrain our model through two proxy tasks on the large, automatically collected Conceptual Captions dataset and then transfer it to multiple established vision-and-language tasks -- visual question answering, visual commonsense reasoning, referring expressions, and caption-based image retrieval -- by making only minor additions to the base architecture. We observe significant improvements across tasks compared to existing task-specific models -- achieving state-of-the-art on all four tasks. Our work represents a shift away from learning groundings between vision and language only as part of task training and towards treating visual grounding as a pretrainable and transferable capability.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":false,"categories":["Multimodal Machine Learning Applications"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:1807.10854","kind":"review","title":"A Survey of the Usages of Deep Learning for Natural Language Processing","authors":["Daniel W. Otter","Julian Richard Medina","Jugal Kumar Kalita"],"author_count":3,"year":2020,"published":"2020-04-21","venue":"IEEE Transactions on Neural Networks and Learning Systems","cited_by":1836,"doi":"10.1109/tnnls.2020.2979670","arxiv_id":"1807.10854","nct_id":null,"source_url":"https://doi.org/10.1109/tnnls.2020.2979670","archive_url":"https://space-kitty.com/papers/arxiv:1807.10854","updated":"2019-12-21","pmid":"32324570","pmcid":null,"openalex_id":"W2884159461","pdf_url":"https://arxiv.org/pdf/1807.10854","abstract":"Over the last several years, the field of natural language processing has been propelled forward by an explosion in the use of deep learning models. This survey provides a brief introduction to the field and a quick overview of deep learning architectures and methods. It then sifts through the plethora of recent studies and summarizes a large assortment of relevant contributions. Analyzed research areas include several core linguistic processing issues in addition to a number of applications of computational linguistics. A discussion of the current state of the art is then provided along with recommendations for future research in the field.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":null,"categories":["Natural Language Processing Techniques"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:1506.00019","kind":"preprint","title":"A Critical Review of Recurrent Neural Networks for Sequence Learning","authors":["Zachary C. Lipton","John A. Berkowitz","Charles Elkan"],"author_count":3,"year":2015,"published":"2015-05-29","venue":null,"cited_by":2074,"doi":null,"arxiv_id":"1506.00019","nct_id":null,"source_url":"http://arxiv.org/abs/1506.00019","archive_url":"https://space-kitty.com/papers/arxiv:1506.00019","updated":"2015-10-17","pmid":null,"pmcid":null,"openalex_id":"W1598796236","pdf_url":"https://arxiv.org/pdf/1506.00019","abstract":"Countless learning tasks require dealing with sequential data. Image captioning, speech synthesis, and music generation all require that a model produce outputs that are sequences. In other domains, such as time series prediction, video analysis, and musical information retrieval, a model must learn from inputs that are sequences. Interactive tasks, such as translating natural language, engaging in dialogue, and controlling a robot, often demand both capabilities. Recurrent neural networks (RNNs) are connectionist models that capture the dynamics of sequences via cycles in the network of nodes. Unlike standard feedforward neural networks, recurrent networks retain a state that can represent information from an arbitrarily long context window. Although recurrent neural networks have traditionally been difficult to train, and often contain millions of parameters, recent advances in network architectures, optimization techniques, and parallel computation have enabled successful large-scale learning with them. In recent years, systems based on long short-term memory (LSTM) and bidirectional (BRNN) architectures have demonstrated ground-breaking performance on tasks as varied as image captioning, language translation, and handwriting recognition. In this survey, we review and synthesize the research that over the past three decades first yielded and then made practical these powerful learning models. When appropriate, we reconcile conflicting notation and nomenclature. Our goal is to provide a self-contained explication of the state of the art together with a historical perspective and references to primary research.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":false,"categories":["Neural Networks and Applications"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2001.08361","kind":"preprint","title":"Scaling Laws for Neural Language Models","authors":["Jared Kaplan A","Sam McCandlish","Tom Henighan","Brown, Tom B.","Benjamin Chess","Rewon Child","Scott Gray","Alec Radford","Jeffrey Wu","Dario Amodei"],"author_count":10,"year":2020,"published":"2020-01-23","venue":null,"cited_by":1751,"doi":null,"arxiv_id":"2001.08361","nct_id":null,"source_url":"http://arxiv.org/abs/2001.08361","archive_url":"https://space-kitty.com/papers/arxiv:2001.08361","updated":"2020-01-23","pmid":null,"pmcid":null,"openalex_id":"W3001279689","pdf_url":"https://arxiv.org/pdf/2001.08361","abstract":"We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends spanning more than seven orders of magnitude. Other architectural details such as network width or depth have minimal effects within a wide range. Simple equations govern the dependence of overfitting on model/dataset size and the dependence of training speed on model size. These relationships allow us to determine the optimal allocation of a fixed compute budget. Larger models are significantly more sample-efficient, such that optimally compute-efficient training involves training very large models on a relatively modest amount of data and stopping significantly before convergence.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":false,"categories":["Artificial Intelligence Applications"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2204.14198","kind":"preprint","title":"Flamingo: a Visual Language Model for Few-Shot Learning","authors":["Jean-Baptiste Alayrac","Jeff Donahue","Pauline Luc","Antoine Miech","Barr, Iain","Yana Hasson","Karel Lenc","Arthur Mensch","Katie Millican","Malcolm Reynolds","Roman Ring","Eliza Rutherford","Serkan Cabi","Tengda Han","Gong, Zhitao","Sina Samangooei","Marianne Monteiro","Jacob Menick","Sebastian Borgeaud","Brock, Andrew","Aida Nematzadeh","Sahand Sharifzadeh","Mikołaj Bińkowski","Ricardo Barreira","Oriol Vinyals"],"author_count":27,"year":2022,"published":"2022-04-29","venue":null,"cited_by":1555,"doi":null,"arxiv_id":"2204.14198","nct_id":null,"source_url":"http://arxiv.org/abs/2204.14198","archive_url":"https://space-kitty.com/papers/arxiv:2204.14198","updated":"2022-11-15","pmid":null,"pmcid":null,"openalex_id":"W4225323055","pdf_url":"https://arxiv.org/pdf/2204.14198","abstract":"Building models that can be rapidly adapted to novel tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research. We introduce Flamingo, a family of Visual Language Models (VLM) with this ability. We propose key architectural innovations to: (i) bridge powerful pretrained vision-only and language-only models, (ii) handle sequences of arbitrarily interleaved visual and textual data, and (iii) seamlessly ingest images or videos as inputs. Thanks to their flexibility, Flamingo models can be trained on large-scale multimodal web corpora containing arbitrarily interleaved text and images, which is key to endow them with in-context few-shot learning capabilities. We perform a thorough evaluation of our models, exploring and measuring their ability to rapidly adapt to a variety of image and video tasks. These include open-ended tasks such as visual question-answering, where the model is prompted with a question which it has to answer; captioning tasks, which evaluate the ability to describe a scene or an event; and close-ended tasks such as multiple-choice visual question-answering. For tasks lying anywhere on this spectrum, a single Flamingo model can achieve a new state of the art with few-shot learning, simply by prompting the model with task-specific examples. On numerous benchmarks, Flamingo outperforms models fine-tuned on thousands of times more task-specific data.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":false,"categories":["Multimodal Machine Learning Applications"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2111.01243","kind":"article","title":"Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey","authors":["Bonan Min","Hayley Ross","Elior Sulem","Amir Pouran Ben Veyseh","Thien Huu Nguyen","Oscar Sainz","Eneko Agirre","Ilana Heintz","Dan Roth"],"author_count":9,"year":2023,"published":"2023-06-27","venue":"ACM Computing Surveys","cited_by":1386,"doi":"10.1145/3605943","arxiv_id":"2111.01243","nct_id":null,"source_url":"https://doi.org/10.1145/3605943","archive_url":"https://space-kitty.com/papers/arxiv:2111.01243","updated":"2021-11-01","pmid":null,"pmcid":null,"openalex_id":"W4382246105","pdf_url":"https://dl.acm.org/doi/pdf/10.1145/3605943","abstract":"Large, pre-trained transformer-based language models such as BERT have drastically changed the Natural Language Processing (NLP) field. We present a survey of recent work that uses these large language models to solve NLP tasks via pre-training then fine-tuning, prompting, or text generation approaches. We also present approaches that use pre-trained language models to generate data for training augmentation or other purposes. We conclude with discussions on limitations and suggested directions for future research.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":null,"categories":["Topic Modeling"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"doi:10.3390/electronics8030292","kind":"article","title":"A State-of-the-Art Survey on Deep Learning Theory and Architectures","authors":["Md Zahangir Alom","Tarek M. Taha","Chris Yakopcic","Stefan K.E. Westberg","Paheding Sidike","Mst Shamima Nasrin","Mahmudul Hasan","Brian C. Van Essen","Abdul Ahad S. Awwal","Vijayan K. Asari"],"author_count":10,"year":2019,"published":"2019-03-05","venue":"Electronics","cited_by":1622,"doi":"10.3390/electronics8030292","arxiv_id":null,"nct_id":null,"source_url":"http://dx.doi.org/10.3390/electronics8030292","archive_url":"https://space-kitty.com/papers/doi:10.3390/electronics8030292","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W2919358988","pdf_url":null,"abstract":"In recent years, deep learning has garnered tremendous success in a variety of application domains. This new field of machine learning has been growing rapidly and has been applied to most traditional application domains, as well as some new areas that present more opportunities. Different methods have been proposed based on different categories of learning, including supervised, semi-supervised, and un-supervised learning. Experimental results show state-of-the-art performance using deep learning when compared to traditional machine learning approaches in the fields of image processing, computer vision, speech recognition, machine translation, art, medical imaging, medical information processing, robotics and control, bioinformatics, natural language processing, cybersecurity, and many others. This survey presents a brief survey on the advances that have occurred in the area of Deep Learning (DL), starting with the Deep Neural Network (DNN). The survey goes on to cover Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL). Additionally, we have discussed recent developments, such as advanced variant DL techniques based on these DL approaches. This work considers most of the papers published after 2012 from when the history of deep learning began. Furthermore, DL approaches that have been explored and evaluated in different application domains are also included in this survey. We also included recently developed frameworks, SDKs, and benchmark datasets that are used for implementing and evaluating deep learning approaches. There are some surveys that have been published on DL using neural networks and a survey on Reinforcement Learning (RL). However, those papers have not discussed individual advanced techniques for training large-scale deep learning models and the recently developed method of generative models.","abstract_license":"CC-BY (publisher license, via OpenAlex)","license":"cc-by","is_open_access":true,"peer_reviewed":null,"categories":["Neural Networks and Applications"],"details":null,"provenance":"openalex","attribution":"Abstract of \"A State-of-the-Art Survey on Deep Learning Theory and Architectures\" by Md Zahangir Alom, Tarek M. Taha, Chris Yakopcic et al., https://doi.org/10.3390/electronics8030292, used under its Creative Commons Attribution license (CC-BY); markup removed, otherwise unmodified."},{"id":"doi:10.1109/access.2024.3365742","kind":"article","title":"A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges","authors":["Mohaimenul Azam Khan Raiaan","Md. Saddam Hossain Mukta","Kaniz Fatema","Nur Mohammad Fahad","Sadman Sakib","Most. Marufatul Jannat Mim","Jubaer Ahmad","Mohammed Eunus Ali","Sami Azam"],"author_count":9,"year":2024,"published":"2024-01-01","venue":"IEEE Access","cited_by":933,"doi":"10.1109/access.2024.3365742","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.1109/access.2024.3365742","archive_url":"https://space-kitty.com/papers/doi:10.1109/access.2024.3365742","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W4391855109","pdf_url":"https://ieeexplore.ieee.org/ielx7/6287639/6514899/10433480.pdf","abstract":null,"abstract_license":null,"license":"other-oa","is_open_access":true,"peer_reviewed":null,"categories":["Topic Modeling"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."},{"id":"arxiv:1612.08083","kind":"preprint","title":"Language Modeling with Gated Convolutional Networks","authors":["Yann Dauphin","Angela Fan","Michael Auli","David Grangier"],"author_count":4,"year":2016,"published":"2016-12-23","venue":null,"cited_by":1577,"doi":null,"arxiv_id":"1612.08083","nct_id":null,"source_url":"http://arxiv.org/abs/1612.08083","archive_url":"https://space-kitty.com/papers/arxiv:1612.08083","updated":"2017-09-08","pmid":null,"pmcid":null,"openalex_id":"W2567070169","pdf_url":"https://arxiv.org/pdf/1612.08083","abstract":"The pre-dominant approach to language modeling to date is based on recurrent neural networks. Their success on this task is often linked to their ability to capture unbounded context. In this paper we develop a finite context approach through stacked convolutions, which can be more efficient since they allow parallelization over sequential tokens. We propose a novel simplified gating mechanism that outperforms Oord et al (2016) and investigate the impact of key architectural decisions. The proposed approach achieves state-of-the-art on the WikiText-103 benchmark, even though it features long-term dependencies, as well as competitive results on the Google Billion Words benchmark. Our model reduces the latency to score a sentence by an order of magnitude compared to a recurrent baseline. To our knowledge, this is the first time a non-recurrent approach is competitive with strong recurrent models on these large scale language tasks.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":false,"categories":["Topic Modeling"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2009.06732","kind":"article","title":"Efficient Transformers: A Survey","authors":["Yi Tay","Mostafa Dehghani","Dara Bahri","Donald Metzler"],"author_count":4,"year":2022,"published":"2022-04-22","venue":"ACM Computing Surveys","cited_by":1267,"doi":"10.1145/3530811","arxiv_id":"2009.06732","nct_id":null,"source_url":"https://doi.org/10.1145/3530811","archive_url":"https://space-kitty.com/papers/arxiv:2009.06732","updated":"2022-03-14","pmid":null,"pmcid":null,"openalex_id":"W3085139254","pdf_url":"https://arxiv.org/pdf/2009.06732","abstract":"Transformer model architectures have garnered immense interest lately due to their effectiveness across a range of domains like language, vision and reinforcement learning. In the field of natural language processing for example, Transformers have become an indispensable staple in the modern deep learning stack. Recently, a dizzying number of \"X-former\" models have been proposed - Reformer, Linformer, Performer, Longformer, to name a few - which improve upon the original Transformer architecture, many of which make improvements around computational and memory efficiency. With the aim of helping the avid researcher navigate this flurry, this paper characterizes a large and thoughtful selection of recent efficiency-flavored \"X-former\" models, providing an organized and comprehensive overview of existing work and models across multiple domains.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":"cc-by-nd","is_open_access":true,"peer_reviewed":null,"categories":["Topic Modeling"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2306.13549","kind":"review","title":"A survey on multimodal large language models","authors":["Shukang Yin","Chaoyou Fu","Sirui Zhao","Ke Li","Xing Wei Sun","Tong Xu","Enhong Chen"],"author_count":7,"year":2024,"published":"2024-11-11","venue":"National Science Review","cited_by":893,"doi":"10.1093/nsr/nwae403","arxiv_id":"2306.13549","nct_id":null,"source_url":"https://doi.org/10.1093/nsr/nwae403","archive_url":"https://space-kitty.com/papers/arxiv:2306.13549","updated":"2024-11-29","pmid":"39679213","pmcid":null,"openalex_id":"W4404356490","pdf_url":"https://arxiv.org/pdf/2306.13549","abstract":"Recently, Multimodal Large Language Model (MLLM) represented by GPT-4V has been a new rising research hotspot, which uses powerful Large Language Models (LLMs) as a brain to perform multimodal tasks. The surprising emergent capabilities of MLLM, such as writing stories based on images and OCR-free math reasoning, are rare in traditional multimodal methods, suggesting a potential path to artificial general intelligence. To this end, both academia and industry have endeavored to develop MLLMs that can compete with or even better than GPT-4V, pushing the limit of research at a surprising speed. In this paper, we aim to trace and summarize the recent progress of MLLMs. First of all, we present the basic formulation of MLLM and delineate its related concepts, including architecture, training strategy and data, as well as evaluation. Then, we introduce research topics about how MLLMs can be extended to support more granularity, modalities, languages, and scenarios. We continue with multimodal hallucination and extended techniques, including Multimodal ICL (M-ICL), Multimodal CoT (M-CoT), and LLM-Aided Visual Reasoning (LAVR). To conclude the paper, we discuss existing challenges and point out promising research directions. In light of the fact that the era of MLLM has only just begun, we will keep updating this survey and hope it can inspire more research. An associated GitHub link collecting the latest papers is available at https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":"cc-by","is_open_access":true,"peer_reviewed":null,"categories":["Multimodal Machine Learning Applications"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"doi:10.1109/tpami.2016.2599174","kind":"article","title":"Long-Term Recurrent Convolutional Networks for Visual Recognition and Description","authors":["Jeff Donahue","Lisa Anne Hendricks","Marcus Rohrbach","Subhashini Venugopalan","Sergio Guadarrama","Kate Saenko","Trevor Jackson Darrell"],"author_count":7,"year":2016,"published":"2016-09-01","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","cited_by":1549,"doi":"10.1109/tpami.2016.2599174","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.1109/tpami.2016.2599174","archive_url":"https://space-kitty.com/papers/doi:10.1109/tpami.2016.2599174","updated":null,"pmid":"27608449","pmcid":null,"openalex_id":"W2508429489","pdf_url":null,"abstract":null,"abstract_license":null,"license":null,"is_open_access":false,"peer_reviewed":null,"categories":["Multimodal Machine Learning Applications"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."},{"id":"arxiv:1609.07843","kind":"preprint","title":"Pointer Sentinel Mixture Models","authors":["Stephen Merity","Caiming Xiong","James Bradbury","Richard Socher"],"author_count":4,"year":2016,"published":"2016-09-26","venue":null,"cited_by":1120,"doi":null,"arxiv_id":"1609.07843","nct_id":null,"source_url":"http://arxiv.org/abs/1609.07843","archive_url":"https://space-kitty.com/papers/arxiv:1609.07843","updated":"2016-09-26","pmid":null,"pmcid":null,"openalex_id":"W2525332836","pdf_url":"https://arxiv.org/pdf/1609.07843","abstract":"Recent neural network sequence models with softmax classifiers have achieved their best language modeling performance only with very large hidden states and large vocabularies. Even then they struggle to predict rare or unseen words even if the context makes the prediction unambiguous. We introduce the pointer sentinel mixture architecture for neural sequence models which has the ability to either reproduce a word from the recent context or produce a word from a standard softmax classifier. Our pointer sentinel-LSTM model achieves state of the art language modeling performance on the Penn Treebank (70.9 perplexity) while using far fewer parameters than a standard softmax LSTM. In order to evaluate how well language models can exploit longer contexts and deal with more realistic vocabularies and larger corpora we also introduce the freely available WikiText corpus.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":false,"categories":["Topic Modeling"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"doi:10.1109/tip.2023.3293771","kind":"article","title":"nnFormer: Volumetric Medical Image Segmentation via a 3D Transformer","authors":["Hong-Yu Zhou","Jiansen Guo","Yinghao Zhang","Xiaoguang Han","Lequan Yu","Liansheng Wang","Yizhou Yu"],"author_count":7,"year":2023,"published":"2023-01-01","venue":"IEEE Transactions on Image Processing","cited_by":748,"doi":"10.1109/tip.2023.3293771","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.1109/tip.2023.3293771","archive_url":"https://space-kitty.com/papers/doi:10.1109/tip.2023.3293771","updated":null,"pmid":"37440404","pmcid":null,"openalex_id":"W4384159609","pdf_url":null,"abstract":null,"abstract_license":null,"license":null,"is_open_access":false,"peer_reviewed":null,"categories":["Medical Image Segmentation Techniques"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."},{"id":"arxiv:1701.06538","kind":"preprint","title":"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer","authors":["Noam Shazeer","Azalia Mirhoseini","Krzysztof Maziarz","Andy Davis","Quoc Viet Le","Geoffrey E. Hinton","Jeff Dean"],"author_count":7,"year":2017,"published":"2017-01-23","venue":null,"cited_by":933,"doi":null,"arxiv_id":"1701.06538","nct_id":null,"source_url":"http://arxiv.org/abs/1701.06538","archive_url":"https://space-kitty.com/papers/arxiv:1701.06538","updated":"2017-01-23","pmid":null,"pmcid":null,"openalex_id":"W2581624817","pdf_url":"https://arxiv.org/pdf/1701.06538","abstract":"The capacity of a neural network to absorb information is limited by its number of parameters. Conditional computation, where parts of the network are active on a per-example basis, has been proposed in theory as a way of dramatically increasing model capacity without a proportional increase in computation. In practice, however, there are significant algorithmic and performance challenges. In this work, we address these challenges and finally realize the promise of conditional computation, achieving greater than 1000x improvements in model capacity with only minor losses in computational efficiency on modern GPU clusters. We introduce a Sparsely-Gated Mixture-of-Experts layer (MoE), consisting of up to thousands of feed-forward sub-networks. A trainable gating network determines a sparse combination of these experts to use for each example. We apply the MoE to the tasks of language modeling and machine translation, where model capacity is critical for absorbing the vast quantities of knowledge available in the training corpora. We present model architectures in which a MoE with up to 137 billion parameters is applied convolutionally between stacked LSTM layers. On large language modeling and machine translation benchmarks, these models achieve significantly better results than state-of-the-art at lower computational cost.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":false,"categories":["Topic Modeling"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:1402.1128","kind":"preprint","title":"Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition","authors":["Haşim Sak","Andrew Senior","Françoise Beaufays"],"author_count":3,"year":2014,"published":"2014-02-05","venue":null,"cited_by":859,"doi":null,"arxiv_id":"1402.1128","nct_id":null,"source_url":"http://arxiv.org/abs/1402.1128","archive_url":"https://space-kitty.com/papers/arxiv:1402.1128","updated":"2014-02-05","pmid":null,"pmcid":null,"openalex_id":"W1499864241","pdf_url":"https://arxiv.org/pdf/1402.1128","abstract":"Long Short-Term Memory (LSTM) is a recurrent neural network (RNN) architecture that has been designed to address the vanishing and exploding gradient problems of conventional RNNs. Unlike feedforward neural networks, RNNs have cyclic connections making them powerful for modeling sequences. They have been successfully used for sequence labeling and sequence prediction tasks, such as handwriting recognition, language modeling, phonetic labeling of acoustic frames. However, in contrast to the deep neural networks, the use of RNNs in speech recognition has been limited to phone recognition in small scale tasks. In this paper, we present novel LSTM based RNN architectures which make more effective use of model parameters to train acoustic models for large vocabulary speech recognition. We train and compare LSTM, RNN and DNN models at various numbers of parameters and configurations. We show that LSTM models converge quickly and give state of the art speech recognition performance for relatively small sized models.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"peer_reviewed":false,"categories":["Speech Recognition and Synthesis"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2401.04081","kind":"preprint","title":"MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts","authors":["Maciej Pióro","Kamil Ciebiera","Krystian Król","Jan Ludziejewski","Michał Krutul","Jakub Krajewski","Szymon Antoniak","Piotr Miłoś","Marek Cygan","Sebastian Jaszczur"],"author_count":10,"year":2024,"published":"2024-01-08","venue":null,"cited_by":24,"doi":null,"arxiv_id":"2401.04081","nct_id":null,"source_url":"https://arxiv.org/abs/2401.04081","archive_url":"https://space-kitty.com/papers/arxiv:2401.04081","updated":"2024-02-26","pmid":null,"pmcid":null,"openalex_id":"W4390723179","pdf_url":"https://arxiv.org/pdf/2401.04081","abstract":"State Space Models (SSMs) have become serious contenders in the field of sequential modeling, challenging the dominance of Transformers. At the same time, Mixture of Experts (MoE) has significantly improved Transformer-based Large Language Models, including recent state-of-the-art open models. We propose that to unlock the potential of SSMs for scaling, they should be combined with MoE. We showcase this on Mamba, a recent SSM-based model that achieves remarkable performance. Our model, MoE-Mamba, outperforms both Mamba and baseline Transformer-MoE. In particular, MoE-Mamba reaches the same performance as Mamba in $2.35\\times$ fewer training steps while preserving the inference performance gains of Mamba against Transformer.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"peer_reviewed":false,"categories":["cs.LG","cs.AI","cs.CL"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"doi:10.1109/jstsp.2017.2763455","kind":"article","title":"Hybrid CTC/Attention Architecture for End-to-End Speech Recognition","authors":["Shinji Watanabe","Takaaki Hori","Suyoun Kim","John R. Hershey","Tomoki Hayashi"],"author_count":5,"year":2017,"published":"2017-10-25","venue":"IEEE Journal of Selected Topics in Signal Processing","cited_by":801,"doi":"10.1109/jstsp.2017.2763455","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.1109/jstsp.2017.2763455","archive_url":"https://space-kitty.com/papers/doi:10.1109/jstsp.2017.2763455","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W2766219058","pdf_url":null,"abstract":null,"abstract_license":null,"license":null,"is_open_access":false,"peer_reviewed":null,"categories":["Speech Recognition and Synthesis"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."},{"id":"openalex:W2970608575","kind":"article","title":"ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks","authors":["Jiasen Lu","Dhruv Batra","Devi Parikh","Stefan Lee"],"author_count":4,"year":2019,"published":"2019-08-06","venue":"neural information processing systems","cited_by":772,"doi":null,"arxiv_id":null,"nct_id":null,"source_url":"https://papers.nips.cc/paper/8297-vilbert-pretraining-task-agnostic-visiolinguistic-representations-for-vision-and-language-tasks.pdf","archive_url":"https://space-kitty.com/papers/openalex:W2970608575","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W2970608575","pdf_url":null,"abstract":null,"abstract_license":null,"license":null,"is_open_access":false,"peer_reviewed":null,"categories":["Multimodal Machine Learning Applications"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."}],"next":"https://space-kitty.com/api/v1/topics/llm-architectures/papers?sort=top&limit=25&offset=25"}