← Back to Research Radar
Academic Publication Academic Publication

A survey on multimodal large language models

519
Citations
November 14, 2024
Published Date

Research Abstract & Technology Focus

ABSTRACT
Recently, the multimodal large language model (MLLM) represented by GPT-4V has been a new rising research hotspot, which uses powerful large language models (LLMs) as a brain to perform multimodal tasks. The surprising emergent capabilities of the MLLM, such as writing stories based on images and optical character recognition–free math reasoning, are rare in traditional multimodal methods, suggesting a potential path to artificial general intelligence. To this end, both academia and industry have endeavored to develop MLLMs that can compete with or even outperform GPT-4V, pushing the limit of research at a surprising speed. In this paper, we aim to trace and summarize the recent progress of MLLMs. First, we present the basic formulation of the MLLM and delineate its related concepts, including architecture, training strategy and data, as well as evaluation. Then, we introduce research topics about how MLLMs can be extended to support more granularity, modalities, languages and scenarios. We continue with multimodal hallucination and extended techniques, including multimodal in-context learning, multimodal chain of thought and LLM-aided visual reasoning. To conclude the paper, we discuss existing challenges and point out promising research directions.
survey multimodal large language models
Read Full Literature

AI Semantic Synergy Context

Connecting this academic literature to real-world market discussions and products.

crossref.org › academic paper
13%

A survey on multimodal large language models

ABSTRACT Recently, the multimodal large language model (MLLM) represented by GPT-4V has been a new rising research hotspot, which uses powerful large language models (LLMs) as a brai...
crossref.org › academic paper
0%

A Comprehensive Overview of Large Language Models

Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks and beyond. This success of LLMs has led to a large influx of research contribut...
crossref.org › academic paper
0%

A Survey on Evaluation of Large Language Models

Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to play a vital role...
crossref.org › academic paper
0%

Large Language Models in Healthcare and Medical Domain: A Review

The deployment of large language models (LLMs) within the healthcare sector has sparked both enthusiasm and apprehension. These models exhibit the remarkable ability to provide proficient responses...
crossref.org › academic paper
0%

Frontiers: Can Large Language Models Capture Human Preferences?

This paper examines the potential of large language models to mimic human survey respondents and to derive their preferences.

Frequently Asked Questions (FAQ)

Curated market intelligence mapped to this research.

What is the core focus of the research titled 'A survey on multimodal large language models'?

This literature focuses on: ABSTRACT Recently, the multimodal large language model (MLLM) represented by GPT-4V has been a new rising research hotspot, which uses powerful large language models (LLMs) as a brain to perform multimodal tasks. The surprising emer...

Are there open-source GitHub repositories related to A survey on multimodal large language models?

Yes, open-source projects like FreedomIntelligence/OpenClaw-Medical-Skills (The largest open-source medical AI skills library for OpenClaw🦞.) are actively building upon these concepts.

Which startups are commercializing the technology behind A survey on multimodal large language models?

Products like Qwen3.6-Plus are bringing this to market. Their focus is: Multimodal AI optimized for real-world coding agents.

What other academic literature is closely related to 'A survey on multimodal large language models'?

Yes, highly correlated activity was mapped. An entry titled 'A survey on multimodal large language models' discusses this: ABSTRACT Recently, the multimodal large language model (MLLM) represented by GPT-4V has been a new rising research hotspot, which us...

Cite this Market Intelligence Report

Reference our AI-mapped synergy between this research and the commercial market to instantly build authority.

Commercial Realization

Startups and Open Source tools heavily associated with the concepts explored in this paper.

Associated Media Narrative