← Back to Research Radar
Academic Publication Academic Publication

Pre-Trained Language Models for Text Generation: A Survey

161
Citations
October 31, 2024
Published Date

Research Abstract & Technology Focus

Text Generation aims to produce plausible and readable text in human language from input data. The resurgence of deep learning has greatly advanced this field, in particular, with the help of neural generation models based on pre-trained language models (PLMs). Text generation based on PLMs is viewed as a promising approach in both academia and industry. In this article, we provide a survey on the utilization of PLMs in text generation. We begin with introducing two key aspects of applying PLMs to text generation: (1) how to design an effective PLM to serve as the generation model; and (2) how to effectively optimize PLMs given the reference text and to ensure that the generated texts satisfy special text properties. Then, we show the major challenges that have arisen in these aspects, as well as possible solutions for them. We also include a summary of various useful resources and typical text generation applications based on PLMs. Finally, we highlight the future research directions which will further improve these PLMs for text generation. This comprehensive survey is intended to help researchers interested in text generation problems to learn the core concepts, the main techniques and the latest developments in this area based on PLMs.
Read Full Literature

AI Semantic Synergy Context

Connecting this academic literature to real-world market discussions and products.

crossref.org › academic paper
28%

Pre-Trained Language Models for Text Generation: A Survey

Text Generation aims to produce plausible and readable text in human language from input data. The resurgence of deep learning has greatly advanced this field, in particular, with the help of neura...

crossref.org › academic paper
0%

Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond

This article presents a comprehensive and practical guide for practitioners and end-users working with Large Language Models (LLMs) in their downstream Natural Language Processing (NLP) tasks. We p...

crossref.org › academic paper
0%

AI models collapse when trained on recursively generated data

Abstract Stable diffusion revolutionized image creation from descriptive text. GPT-2 (ref. 1), GPT-3(.5) (ref. 2) and GPT-4 (ref. 3) demonstrated high performance across a variety of lang...

crossref.org › academic paper
0%

When large language models meet personalization: perspectives of challenges and opportunities

AbstractThe advent of large language models marks a revolutionary breakthrough in artificial intelligence. With the unprecedented scale of training and model parameters, the capability of large lan...

crossref.org › academic paper
0%

AI–Human Hybrids for Marketing Research: Leveraging Large Language Models (LLMs) as Collaborators

The authors’ central premise is that a human–LLM (large language model) hybrid approach leads to efficiency and effectiveness gains in the marketing research process. In qualitative research, they ...

Frequently Asked Questions (FAQ)

Curated market intelligence mapped to this research.

What is the core focus of the research titled 'Pre-Trained Language Models for Text Generation: A Survey'?

This literature focuses on: Text Generation aims to produce plausible and readable text in human language from input data. The resurgence of deep learning has greatly advanced this field, in particular, with the help of neural generation models based on pre-trained language ...

Are there open-source GitHub repositories related to Pre-Trained Language Models for Text Generation: A Survey?

Yes, open-source projects like alvinunreal/awesome-opensource-ai (Curated list of the best truly open-source AI projects, models, tools, and infrastructure.) are actively building upon these concepts.

Which startups are commercializing the technology behind Pre-Trained Language Models for Text Generation: A Survey?

Products like Ollang DX are bringing this to market. Their focus is: The AI Language Execution Layer for Enterprise.

What other academic literature is closely related to 'Pre-Trained Language Models for Text Generation: A Survey'?

Yes, highly correlated activity was mapped. An entry titled 'Pre-Trained Language Models for Text Generation: A Survey' discusses this: Text Generation aims to produce plausible and readable text in human language from input data. The resurgence of deep learning has greatly advanced...

Cite this Market Intelligence Report

Reference our AI-mapped synergy between this research and the commercial market to instantly build authority.

Commercial Realization

Startups and Open Source tools heavily associated with the concepts explored in this paper.

Associated Media Narrative