My AI assisted me in writing this article.
But I did not write a carefully structured prompt. That's why I believe something about how we work with AI has changed.
For the past few years, much of the conversation was about prompt engineering. People created prompt formulas, prompt templates, role-playing instructions, and increasingly complicated ways of telling an AI what to do.
I used to teach Prompt Engineering. But I don't prompt AI like that anymore. AI systems have evolved over the years, and it's exposing something very differently; How Bad We Are at Defining Problems.
As AI gets smarter, our job shifts from telling it exactly what to do to making sure it understands what is happening, what information matters, and what a good result looks like. This has led me to the concept of Brief Engineering.
What Is Brief Engineering?
Brief Engineering is the skill of turning a real problem into a clear enough brief for AI to reason about and produce a useful outcome.
Brief Engineering is not about prompts, it asks "What exactly are we trying to solve, what does the intelligence need to understand, and what would count as a good result?"
The information in a brief can include the current instruction, previous conversations, examples, documents, memories, retrieved information, tool results, system instructions, and other data available to the model when it responds.
My 3 Step Super Intelligence Briefing Framework
1. Describe the Problem
Start with the actual problem and not a fake role like "act as the world's greatest strategist."
Tell the model what is happening and what needs to change so the model knows what problem it is actually solving.
There is an important difference between describing a task and describing a problem.
"Rewrite my website homepage" is a task.
"People visit my homepage but cannot immediately understand what we do" is a problem.
The second gives the model more room to reason about the desired result.
2. Give It the Right Data Points
Now supply the information that would help someone solve that problem.
Who is the customer?
What have you already tried?
What data do you have?
What should not change?
What examples represent what you want?
What restrictions matter, etc.?
Relevant data and context gives AI superpowers.
3. Define the Output
Finally, tell the model what success looks like.
State what must be included.
State what should be removed.
If you already know what good work should look like, tell the model.
The entire framework is
Problem → Data → Desired Output
Experts In Any Field Can Get So Much Out of AI
An expert usually knows what good work looks like.
A marketer knows when positioning is weak.
A programmer understands the expected behaviour of a system.
A lawyer knows which facts are material to an argument.
A logistics operator understands what information determines whether an order can actually be fulfilled.
Knowledge allows the expert to give the AI better briefs and define a better destination.
Someone with very little knowledge of a subject can ask an AI to "give me the best strategy."
An expert can describe the situation, identify the variables that matter, provide the relevant data and explain exactly what the final strategy should accomplish.
The difference in output can be enormous; the advantage is not knowing a secret prompt but knowing what information matters.
Warning!
There is one problem with everything I have said so far. Expert defined outcomes or statements steers the model.
Language models are overly agreeable with users. This behaviour is called sycophancy. Research from Anthropic has found models matching a user's stated beliefs instead of giving the most truthful response, and a joint Anthropic and OpenAI evaluation also observed forms of sycophancy across models from both companies.
If I explain why I believe an idea is brilliant and then ask the model to evaluate the idea, I have already shaped the environment in which that evaluation happens.
So good AI briefs can sometimes mean deliberately giving the model permission to disagree especially when the intended outcome is not certain yet or you're still researching.
I personally find instructions such as these useful during serious research:
"Find evidence against this."
"Assume my hypothesis is wrong."
"Separate what I believe from what the evidence establishes."
Brief engineering should improve reasoning for experts, not improve the model's ability to agree with us.
0 Comments
Write a comment