SaaS Metrics
Large Language Models (LLMs) have shown impressive abilities in code generation. Chain-of-Thought (CoT) prompting is the state-of-the-art approach to utilizing LLMs. CoT prompting asks LLMs first to generate CoTs (i.e., intermediate natural language reasoning steps) and then output the code. However, the accuracy of CoT prompting still cannot satisfy practical applications. For example, gpt-3.5-turbo with CoT prompting only achieves 53.29% Pass@1 in HumanEval. In this article, we propose Structured CoTs (SCoTs) and present a novel prompting technique for code generation named SCoT prompting. Our motivation is that human developers follow structured programming. Developers use three programming structures (i.e., sequential, branch, and loop) to design and implement structured programs. Thus, we ask LLMs to use three programming structures to generate SCoTs (structured reasoning steps) before outputting the final code. Compared to CoT prompting, SCoT prompting explicitly introduces programming structures and unlocks the structured programming thinking of LLMs. We apply SCoT prompting to two LLMs (i.e., gpt-4-turbo, gpt-3.5-turbo, and DeepSeek Coder-Instruct-
\(\{\)
1.3B, 6.7B, 33B
\(\}\)
) and evaluate it on three benchmarks (i.e., HumanEval, MBPP, and MBCPP). SCoT prompting outperforms CoT prompting by up to 13.79% in Pass@1. SCoT prompting is robust to examples and achieves substantial improvements. The human evaluation also shows human developers prefer programs from SCoT prompting.
Academic Publication Structured Chain-of-Thought Prompting for Code Generation
Research Abstract & Technology Focus
AI Semantic Synergy Context
Connecting this academic literature to real-world market discussions and products.
Graph of Thoughts: Solving Elaborate Problems with Large Language Models
We introduce Graph of Thoughts (GoT): a framework that advances prompting capabilities in large language models (LLMs) beyond those offered by paradigms such as Chain-of-Thought or Tree of Thought...
DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
Abstract General reasoning represents a long-standing and formidable challenge in artificial intelligence (AI). Recent breakthroughs, exemplified by large language models (LLMs)1,2 and ch...
顺着提示词开源的思路写了一个提示词工具，请过目
This issue reveals a developer's initiative to streamline prompt engineering through a visual, flowchart-based tool. The 'vibe coding' approach to creating a 'flowchart-based prompt tool' for 'open...
Self-Collaboration Code Generation via ChatGPT
Although large language models (LLMs) have demonstrated remarkable code-generation ability, they still struggle with complex tasks. In real-world software development, humans usually tackle complex...
Show HN: I made a "programming language" looking for feedback
I've been thinking about something along these lines, but coupled with deterministic inference. At each "macro" invocation you'd also include hash-of-model, and hash-of-generated-text. (Note, deter...
Frequently Asked Questions (FAQ)
Curated market intelligence mapped to this research.
What is the core focus of the research titled 'Structured Chain-of-Thought Prompting for Code Generation'?
This literature focuses on: Large Language Models (LLMs) have shown impressive abilities in code generation. Chain-of-Thought (CoT) prompting is the state-of-the-art approach to utilizing LLMs. CoT prompting asks LLMs first to generate CoTs (i.e., intermediate natural langua...
Are there open-source GitHub repositories related to Structured Chain-of-Thought Prompting for Code Generation?
Yes, open-source projects like gsd-build/gsd-2 (A powerful meta-prompting, context engineering and spec-driven development system that enables agents to work for long periods of time autonomously...) are actively building upon these concepts.
Which startups are commercializing the technology behind Structured Chain-of-Thought Prompting for Code Generation?
Products like gigabrainz — Learn Anything, 10x Faster are bringing this to market. Their focus is: Upload whatever you need to learn to get a structured course.
What other academic literature is closely related to 'Structured Chain-of-Thought Prompting for Code Generation'?
Yes, highly correlated activity was mapped. An entry titled 'Graph of Thoughts: Solving Elaborate Problems with Large Language Models' discusses this: We introduce Graph of Thoughts (GoT): a framework that advances prompting capabilities in large language models (LLMs) beyond those offered by para...
How is the concept of 'Structured Chain-of-Thought Prompting for Code Generation' being discussed by engineers on Hacker News?
Yes, highly correlated activity was mapped. An entry titled 'Show HN: I made a "programming language" looking for feedback' discusses this: I've been thinking about something along these lines, but coupled with deterministic inference. At each "macro" invocation you'd also include hash-...
Cite this Market Intelligence Report
Reference our AI-mapped synergy between this research and the commercial market to instantly build authority.
Commercial Realization
Startups and Open Source tools heavily associated with the concepts explored in this paper.
-
GitHubgsd-build/gsd-2
-
GitHubgoogle-labs-code/design.md
-
Product Huntgigabrainz — Learn Anything, 10x Faster
-
Product HuntBeezi AI