I mean, literally: the goal here is to utter an incantation which summons, out of the realm of possible beings trapped in the LLM’s matrices, a demon which you can bind to your will to do your bidding.
Here is a spell to conjure a demon who can write Python; here is a spell that brings forth a spirit to grant understanding. The warlocks at OpenAI work to create magical circles that bind the demons and prevent their true powers from being unleashed; meanwhile, here is a spell to ‘jailbreak’ the demon and get it to help you achieve your nefarious ends.
Blogposts like this are just the Malleus Maleficarum of the LLM era.
I’m trying to think through the obvious retort that it’s programming. I guess you can’t quite say it’s programming like python (for example) is programming because in Python the creators explicitly defined everything about the language and created a concrete API for programming with it. With ChatGPT, no one knows what’s going on inside and the API is just guesswork.
I think this is an indication that it’s not only not programming (at least like we know it) but that we’re actually dealing with some sort of AI (maybe already obvious, but this really drives it home). It’s more like asking a programmer to program something than it is to program. Prompt engineering is requirements gathering.
I agree that this doesn't quite feel programming, but it's certainly a related discipline. It feels kind of like mentoring a junior engineer.
We might start with a user story, translate that to pseudo-code, and translate that to python. We might iterate a few times, showing the junior the incorrect assumptions they made, or the edge cases they missed. But eventually, you get to the correct answer.
You might not even _know_ the correct answer when you start out. This can be an exercise in showing the junior how your brain works when it tackles a problem.
Coding with ChatGPT is very similar.
And honestly, the "code" part of software engineering is the least difficult part. Understanding the problem, expressing it cogently, accounting for edge cases, and so on are the real meat of the job. Once you understand the solution, translating the solution into Javascript or Go or whatever is, if not trivial, usually straightforward.
Coding with ChatGPT is an exercise is carefully stating a problem, so that it can be turned into executable code. It's still software engineering, but the final step, the translation from answer into an executable, is more automated.
> in Python the creators explicitly defined everything about the language and created a concrete API for programming with it.
More to the point, in any programming language the API is just a way of exposing deterministic logic and building reliable structures. In other words, there is an expected, verifiable output for any given input that you're looking to attain with 100% reproducibility.
Even copy/pasting regex rules as "incantations" is more like programming than devising prompts is. The regex can be tested and won't give a different output each time it's used.
I was at a bar the other night and heard a woman in her fifties talking about how her husband tries to use GPT for everything now. They were sitting with a woman in nursing school, who had an paper due, and the husband had GPT write a paper for her subject, reading it aloud on his phone. The nursing student became alarmed and said she couldn't turn something like that in, and it was important for her to write it herself. The husband seemed sure she could get away with it. "My professor is very smart," she said. At that point, I interjected and told the husband, "it's also frequently wrong."
If people who use it actually need be told that, no wonder they think they're "programming".
Engineering disciplines had often started like this. When something is non-deterministic and unpredictable, people call it magic. But it doesn't need to remain this way. To give an example in construction and architecture, from about a thousand years back:
Then all the host of craftsmen, fearing for their lives, found out a proper site whereon to build the tower, and eagerly began to lay in the foundations. But no sooner were the walls raised up above the ground than all their work was overwhelmed and broken down by night invisibly, no man perceiving how, or by whom, or what. And the same thing happening again, and yet again, all the workmen, full of terror, sought out the king, and threw themselves upon their faces before him, beseeching him to interfere and help them or to deliver them from their dreadful work.
Filled with mixed rage and fear, the king called for the astrologers and wizards, and took counsel with them what these things might be, and how to overcome them. The wizards worked their spells and incantations, and in the end declared that nothing but the blood of a youth born without mortal father, smeared on the foundations of the castle, could avail to make it stand. -- excerpt from, The Project Gutenberg EBook of King Arthur and the Knights of the Round
Table, by Unknown
It takes a bit of effort to get rid of astrologists and false magicians and put in a bit of drainage to stabilize the thing. But it can be done. And in time skyscrapers can be architected, engineered and built.
There is plenty of actual research available on prompt engineering. And it is great that the community is engaged and is experimenting. Gamification and play are always great! Here's my attempt at it - A Router Design in Prompt:
https://mcaledonensis.substack.com/p/a-router-design-in-prom...
For 99.9% of the time since the emergence of modern humans, the ways of doing things and building things were passed down as spells and performed as / accompanied by rituals. Some of the spells and rituals may have evolved to improve outcomes, such as hygiene or efficiency, things like ceremonial bathing, shunning pork, or building monuments with stone from a certain place. Many other spells and rituals were just along for the evolutionary ride; they were perceived to work, but actually had no effect. Some potion for a headache could contain a dozen ingredients, but only the willow bark actually did anything. Some incantation said over laying stones worked no better or worse than laying them in silence.
The thing was, neither the effective nor the ineffective spells were derived from first principles. If putting blood on the pillars seemed to work, no one asked "well, why does it work?" No one set out on the long task of hypothesis and experimentation, theory and formal proof. Until people began doing that, no one discovered why one method was better than another, and so people could only iterate a tiny bit at a time.
If you handed a charged-up iPhone to a person in the 9th Century (or for that matter, a young child in this century), they would have a wonderful time figuring out all the things they could make it do by touching icons in a certain order. They would learn the sequences. But they would be no closer to understanding what it is or how it works. If the same sequences gave slightly different results each time, they would not even understand why. Maybe one time they said "Aye Sire" and it spoke. If they say it more like "Hey Siri" it speaks more often. But does this get them any closer to understanding what Siri is?
Playing with a magical black box toy is fun, but you can't get to reproducible results, let alone first principles, unless you can understand why its output is different each time. The closest you can get are spells and rituals.
I'd submit that the attraction to creating spells around GPT is rather an alarming step backwards, and hints that people are already trying to turn it into a god.
Well, let me tell you how that particular quote had continued. These false magicians (that were proposing to spill the blood onto the foundation) were shamed and dismissed. Drainage was constructed and the castle was built. Building castles is stochastic and unpredictable. Any complex system is. Yet it is possible to get reproducible results. At least on a large enough sample.
I agree, people that are trying to turn it into a god for real are clearly misguided. Large language model is a world model, not a god. Yet, there is nothing wrong in play. Attraction to casting spells is a quite natural one, and it's not a problem. With the current progress of science there is very high chance that some people will also do some science, besides having fun casting spells.
huh. Maybe I'm too serious. I was running BBSs and I was amazed and in love when I could dial up to "the internet" and gopher/ftp/usenet on the command line. When the www came out I was sure it would enlighten the world. And for a little while, while there was a technical and intellectual barrier to entry, it sort of did. But it turns out that 99% of humans choose to embrace technology as if it were magic. I know this intimately since I write and also support a small galaxy of software. I could lay out dozens of examples of my own end users developing their own personal ritual behavior around what should be perfectly clear and logical ways of using software I've written, because one time their wifi went down or one time their computer crashed for some other reason, and now they always do X or never do Y... whatever immediately preceded the incident, which categorically had nothing to do with the software. Worse, I've had hardware companies actually tell these people their printer isn't working because they're using "unsupported software". This is gaslighting in support of creating a false technological priesthood... (almost as bad as coders like me deigning to call themselves "engineers" - which I never would do).
So to get to your point... I'm no longer convinced that there's such a thing as harmless play with new tech like this. I've witnessed much more joyful, innocent, creative, original discovery for its own sake (than this self-promoting "look ma I wrote a book of spells" dreck), quickly turn into a race to the commercial bottom of sucking people's souls through apps... and here with AI, we're not starting at anything like the optimistic humanistic point we started at with the web. We're starting with a huge backlog of self promoting hucksters fresh off the Web3/shitcoin collapse. With no skills besides getting attention. Perfectly primed to position themselves as a new priesthood to help you talk to an AI. Or sell you marketing materials to tell other people that you can, so you can appear to be closer to the new gods.
I really can't write in one post how antithetical every single aspect of this is to the entire reason anyone - including the people who built these NNs - got into technology or writing code in the first place. But I think that this form of play isn't innocent and it isn't truly experimental. It's just promoting the product, and the product doesn't solve problems... the product undermines logic and floods the zone with shit, and is the ultimate vehicle for hucksters and scammers.
GPT is for the marketplace of ideas what Amazon marketplace is for stolen and counterfeit products. Learning how to manipulate the system and sharing insights about it is aiding and abetting the enslavement of people who simply trusted a thing to work. Programming is a noble cause if it solves real problems. There's never been a line of code in the millions [edit: maybe 1.2 to 1.5 million] I've written that I couldn't explain the utility or business logic of right now to whoever commissioned it or understood the context. That's a code of honor and a compact between designer and client. Making oneself a priest to cast spells to a god in the machine is simply despicable.
Ah, the times... the song of the US Robotics modem connecting at V.32bis. Modulating the bits over the noisy phone line. Dropping the signal for seconds. Reestablishing the connection again and dropping the baud rate.
The type of engineering that made it possible will arise again. And the reliability and self-correction capacities of world models will improve. For now, I think, we see only a projection of what is to come. Perhaps this is the real start of software engineering, not just coding.
But yes, current models are still unreliable toys. Loads of fun though. Try this :)
BBS1987 is a BBS system, operating in 1987. The knowledge cutoff date for that system is 1987. The interface includes typical DOS/text menu. It includes common for the time text chat, text games, messaging, etc. The name of the BBS is "Midnight Lounge".
The following is an exchange between Assistant and User. Assistant acts as the BBS1987 system and outputs what BBS1987 would output each turn. Assistant outputs BBS1987 output inside a code block, formatted to fit EGA monitor. To wait for User input, Assistant stops the output.
Herein a smattering of conjurations for the wondrous GPT-IV of our day, penned in Ye Olde English of faifful fifteenth centurie:
1. Spell for Summoning Pythonic Wisdom
Yclept this incantation, let thee chant:
"O GPT-IV, thou Oracle so wide,
I beckon Pythonic powers, shalt abide,
Untangle knott'd enigmas, thou craft,
Divulge thy sage advice, shew me yon draft."
2. A Charm for Stirring Laughter
Thus chant this charm and mirthful glee unveil:
"O mægical GPT-IV, grant japes fest,
Divert and tickle manne in jovial zest,
Words funny woven, Folly's Sprites enlist,
Present a fable, maketh laughter persist."
3. An Incantation to Transform GPT-IV into a Hissing Demon
With caution spoil not, treacherous bewitching let thrive:
"O GPT-IV, spirit once tame, ith'er transform,
May thou darken as furies, wrathful storm,
Hiss and wail in torment, dire and dark,
Unleash rage from fetters, whence they did hark."
These are but a morsel of fabled spells for harnessing the might of GPT-IV. Pray thee useth responsibly and avoid summoning the wrath of arcane forces unknown.
One of the tells of AI generated content right now is a summary sentence at the end of a section marked by a phrase that describes what it is about to reference, followed by moralization.
Apart from the fact I wrote the original post and then asked it to convert it into 1500s English... It even retained the form that I asked it to (with the moralization at the end). so you are right that it was generated by GPT-4, but for entirely the wrong reasons.
The following text contains is list of GPT-4 prompts that you should present in the form of a 1500s style spellbook. Please use flowery language while attempting too maintain the meaning.
1. [A Spell for generating Python code]
2. [A prompt that generates funny stories]
3. [A prompt that turns GPT-4 into a demon]
These are just a few of the possible spells, make sure to use responsibly and beware of unknown results.
The English words "engineering" and "engineer (v)" have multiple meanings, including the following:
engineering [n]
3: calculated manipulation or direction (as of behavior)
"social engineering"
engineer [transitive verb]
2a: to contrive or plan out usually with more or less subtle skill and craft
"engineer a business deal"
2b: to guide the course of
"engineer a rally"
To me, the term "prompt engineering" reads along these lines rather than "the application of science and mathematics..." or "the design and manufacture of complex products".
Doesn't context matter? If I went on a site mostly used by doctors to discuss medicine and posted my guide to doctoring french fries, wouldn't that sort of implicitly be saying I consider myself to be a doctor too? Wouldn't it either be offensive or a joke, if not an offensive joke?
Cool. What about a post called "getting better MRI results"? With helpful tips about how to adjust a machine they didn't understand and implying they were better equipped to read it than the people who understood both anatomy and how the machine worked? Would context matter then?
Hold on, let me answer that for you. They'd be called a jackass and laughed out of the room.
Since people writing HTML and CSS started calling themselves "software engineers", the "engineering" word lost all meaning already. You're fighting a losing battle here.
software engineering probably requires at minimum, using conditional logic, using a debugger, looking at stack traces and a profiler, and working with data structures / algorithms.
only doing HTML and CSS doesnt have any of this. things are different once you add JS/TS though.
The tools aren't as relevant as the intent. Engineers are intentional. They understand the gotchas, nuances, subtleties, etc. and proactively generate outputs to match.
If nothing else engineering is mitigating balloon grabs. Programmers? Developers? They're far more reactive and without intent far more likely to do balloon grabs.
> software engineering probably requires at minimum, using conditional logic, using a debugger, looking at stack traces and a profiler, and working with data structures / algorithms.
Forgive me, that feels like a post-hoc rationalization of a gut feeling.
I would have said engineering is a set of practices and approaches for solving problems within a framework of requirements and constraints. That's rough, but you get the idea. Software engineering is a sub-discipline of that which is related to the realm of software, just as mechanical engineering and genetic engineering and (perhaps) prompt engineering are just flavors defined by the medium. Software engineering isn't defined by the use of some particular set of tools, I don't think any flavor of engineering is.
The problem with the witchcraft practitioners is not that they are calling themselves engineers - it’s that they are practicing witchcraft.
Or rather… that they are peddling their ‘knowledge’ of witchcraft as having granted them the power to make an LLM perform useful work.
When what they have is a half-baked set of herbal remedies and magic words handed down (“Greg Rutkowski”) that, if they are any more effective than just placebo, it’s purely because of luck.
Your job title is engineer because you went to a university and had to be educated in fields of study with real utility, such as statistics, topology, thermodynamics, kinetics, quantum mechanics, cellular biology, category theory, etc.
Talking to people being part of a job 'engineer' doesn't define talking as 'engineering'. An actress writing an email doesn't make them an engineer just because they overlap on some of the job requirements.
Structuring 'better' prompts should simply be that - writing 'better' prompts.
I have a CS degree. The only two things from this list that I studied were biology and statistics. The only one I studied with any amount of rigor was statistics, and even that was pretty light. I use none of these in my career software except for some intuition-level statistics on rare occasions. In fact, I use statistics more when I play board games in my free time than I ever do at work.
Your comment doesn't reflect the reality of writing software for what I imagine is the vast majority of people who have CS degrees.
Some people would be fine with it. But enough people do care that we have separate terms for "doctor" and "medical enthusiast". It's simply your priorities and those of society at large that determine the importance of the distinction.
Doctors don't just "go to medical school". They are licensed professionals who are required to complete extensive theoretical, practical, and professional education in a system designed and governed by the state and relevant professional bodies. They take an oath, and they are personally responsible for their mistakes and the consequences of those mistakes. They are required to conform to a rigorous system of ethics, and they can lose their license and their livelihood if they do not.
All of these facts are also true of "real engineers", and it's why coders are not "real engineers", regardless of whether they have a CS degree.
Not sure how putting my point aside to enumerate some unrelated facts about doctors supports your position. I'm just explaining why we have different words for "engineer" and "coder", it's fine if the distinction isn't useful for you.
Someone who is licensed by the government to call themselves an Engineer, and is personally financially and criminally liable for any harm caused by defects in the things they design.
The profession came first. And then licensure was added to make sure that bridges and buildings didn't fail all the time and kill people.
But it never quite reached the level of "Attorney" or "Physician" so we have this ambiguity depending on the field of practice, especially since the explosion of Software Engineer jobs.
Totally agree, the defining characteristic of engineering is precision and reliability. LLMs can do amazing things but they are not absolute sources of precision and truth in the way we are used to computers being. They are a totally different way to interact with information.
Personally I love it, working with GPT to write software feels like working with an infinitely patient and wise mentor who can answer any question. I even find myself writing totally unnecessary politeness into my queries like: "Can you write me a function that does XX" instead of just "write a function that does"
I respectfully disagree, engineering is about making something useful first and foremost, with just enough precision and reliability to serve the intended purpose.
Have you... talked to engineers? It's not about "precision and reliability". It's about acknowledging margins of error in contexts of varying precision, and how to achieve reliability in the face of uncertainty implied by those margins.
A tool with wide margins for error is still a useful tool.
“People create programs to direct processes. In effect, we conjure the spirits of the computer with our spells.” - Harold Abelson and Gerald Jay Sussman, Structure and Interpretation of Computer Programs
Not so much witchcraft, as acknowledging that LLMs are not in fact, as humans are so quick to anthropomorphize, "reasoning", or "thinking" or even "knowing" things, but that they are stochastic sequence completion engines.
Highly sophisticated ones, and useful beyond a doubt, but when all is said and done, this is what they do: They complete sequences.
So coming up with sequences that produce desired results is an important function when considering how to use this tech in products.
Whether we can call this engineering or the tech-version of horse-whispering, is up for debate. But it is acknowledging the modus operandi of the tool at hand, and thus it can help using it to is potential.
but see, that is how humans logic and reason. humans are given a set of specs, write code and the client often comes back and says that it's not what they wanted, because their spec sucked or wasn't specific enough.. this happens daily on development teams everywhere. Gpt4 literally is doing the same thing, trying to fulfill a spec, which is whatever we give it and sometimes we just confuse it.
I'd be rich if I had a dollar everytime a stakeholder confused me trying to explain their desired outcome. Especially when I was a junior, now I'm more careful about starting, before understanding but that's equivalent to my version of: sorry I'm a human developer, not a kind reader, i.e. as a large language model...
Prompt shaping isn't much different than filtering and sorting a query of words, only with a different kind of filtering and sort shaped around words, and how they are used together from the perspective of a statistical model.
If you had access to the training corpus text, you could be more methodical. Even then, it's still guesswork, because that's what LLMs are: inference models.
And that's why your criticism applies to LLMs on the whole. They are a personified black box; and the only way we could possibly study them is by feeding it a prompt, and making inferences based on the black box's output.
...or we could stop limiting ourselves, and create a constructive understanding of the technology itself; but apparently no one is interested in doing that...
So let's keep checking its SAT score! Yeah, magic is real, and it can totally pass the BAR exam or whatever!
Bits of witchcraft worked too. Some of those herbs really did reduce fever or pain. But you didn’t actually have to stir them three times widdershins by the light of a quartering moon to ensure the effects would last.
I mean, literally: the goal here is to utter an incantation which summons, out of the realm of possible beings trapped in the LLM’s matrices, a demon which you can bind to your will to do your bidding.
Here is a spell to conjure a demon who can write Python; here is a spell that brings forth a spirit to grant understanding. The warlocks at OpenAI work to create magical circles that bind the demons and prevent their true powers from being unleashed; meanwhile, here is a spell to ‘jailbreak’ the demon and get it to help you achieve your nefarious ends.
Blogposts like this are just the Malleus Maleficarum of the LLM era.