Study Notes: Advanced Prompting Techniques for Generative AI (微软AI学习Advanced Prompting)

1. Why Advanced Prompting Matters

LLMs do not know everything. Like humans (especially children), when an AI lacks a correct answer, it will still try to give its “best guess” — this is known as hallucination.
The goal of these techniques is to ensure the output is correct, consistent, and useful.
These methods act as ways to ask the model, “Are you really sure about this response? You didn’t make it up?”

2. Overview of Prompting Principles Discussed

Zero-shot prompting – the most basic form; you send a prompt and hope for the best.
Chain-of-thought – guiding the model step-by-step through reasoning.
Generated knowledge / RAG – injecting extra context or facts to improve responses.
Least-to-most – breaking a large problem into a series of smaller, ordered sub‑problems.
Self-refine – critiquing the LLM’s own output and iteratively improving it.
Myotic prompting (likely ‘maieutic prompting’) – validating each part of an answer by asking the model to explain itself and check for consistency.

3. Zero-Shot Prompting

The simplest form: construct a prompt, send it to the AI assistant, and accept whatever response comes back.
Works well for general knowledge but provides no guidance or verification.

4. Chain-of-Thought (CoT)

Analogy: Imagine a teacher showing a student how to solve a problem step-by-step. In CoT, you are the teacher and the LLM is the student. You show it examples with full reasoning chains so it learns to mimic that behaviour.

How it works:

Provide a similar example problem with the full calculation/working shown.
Then present the actual question you need answered.
The LLM uses the pattern you demonstrated to produce a correct, reasoned response.

Example (Apple Problem):

Original (zero-shot) question:
“Alice has five apples. She throws away three apples. She gives two apples to Bob, and Bob gives one back. How many apples does Alice have?”
Result: The model incorrectly answers 5.
Chain-of-thought approach:
- First, give a worked example:
  “Lisa has seven apples. She throws away one apple. She gives four apples to Bart, and Bart gives one back.
  Calculation: 7 – 1 = 6 → 6 – 4 = 2 → 2 + 1 = 3. So Lisa has 3 apples.”
- Then add the original Alice question.
- New result: The model now correctly answers 1 (5 – 3 – 2 + 1 = 1).

Why it works: You show the LLM the “chain” — the entire reasoning from start to finish — so it replicates that structured thinking.

5. Generated Knowledge (and RAG – Retrieve Augmented Generation)

Core idea: Give the model extra factual context that it wouldn’t have on its own, leading to a much better, more relevant answer.

Mechanism:

Start with a template containing placeholders (e.g., {company_name}, {products_list}).
Fetch the missing data from your own company systems or databases (e.g., via an API call).
Insert that data into the template and send the complete prompt to the LLM.
The LLM now has the exact knowledge it needs to generate a precise answer.

Example (Insurance):

Generic question without context:
“Please suggest an insurance. My budget is $1,000 and it should cover car and home.”
Result: A very generic reply – “There are multiple insurances…” – not helpful for a specific company.
With generated knowledge (RAG):
First, you fetch the actual product list from “Acme Insurance” and inject it into the prompt:
e.g., “Acme Insurance offers: 1. Basic Auto ($500), 2. Full Home ($600), 3. Combined Car+Home ($900) …”
Then ask the same budget question.
Result: The LLM now recommends specific products that fit within the $1,000 budget, using the real data.
Note: Building the data-fetching pipeline is the “devil in the detail”; Azure provides many code samples for Python, JavaScript, Java, .NET, etc.

6. Least-to-Most

Concept: Break a complex problem down into a sequence of smaller, easier sub‑problems, then solve each one in order. Very similar to chain-of-thought, but explicitly about decomposing a large task.

How to apply:

First, ask the LLM for the high-level steps needed to solve a problem.
Then, for each step, probe deeper to get detailed, actionable answers (often including code).

Example (Data Science):

Initial prompt: “How would I do data science in five steps?”
LLM responds: “1. Collect data, 2. Clean data, 3. Analyze data, 4. Plot data, 5. Present data.”
Now you use that breakdown to guide more specific prompts:
Follow-up prompt: “How would I do collect data in data science?”
(The model now gives a more focused explanation about data collection.)
To get even better output, add extra context and instructions:
Enhanced prompt: “Show me Python code for collect data in data science.”
Result: Practical Python code to read a CSV, inspect headers, select features, etc.

Takeaway: The more specific the sub‑step and the richer the instruction (e.g., “show me Python code”), the better the response.

7. Self-Refine

Idea: Treat the LLM’s output as a draft and critique it iteratively to improve it. You act as an editor asking the model to reconsider and enhance its own work.

Process:

Give the model an initial task (e.g., “Create a Python web API with routes for products and customers”).
After receiving the first answer, ask for improvements: “Suggest three improvements to the above code.”
The model will reflect on its earlier output and supply a refined version.
You can repeat this loop as many times as necessary until you are satisfied.

Key insight: This makes the LLM its own critic, helping to catch errors and enhance quality without you having to manually rewrite everything.

8. Myotic (Maieutic) Prompting

Concept: Validate the correctness of a multi‑part answer by cross‑examining each piece. The name likely references the Socratic method of eliciting truth through questioning.