I have tried to get useful code out of ChatGPT with GPT-4 and so far been unable to find a case where it was easier for me to verify that chatGPT was right than to just do it myself. This was for toy examples. For more complicated things it's downright useless.

Everyone seems to be raving about its ability to write code. What am I doing wrong?

Yeah frankly it’s hard to understand how you can’t get value from ChatGPT as a programming companion.

Either you already know how to do everything or you’re not asking it a wide range of questions about the work you’re doing.

It’s not there to write your code for you, although it can write snippets and short functions. It’s thereto answer questions like “how do I….”, “how does this work”, “explain this code”, “what’s the correct typescript type for….”, “what’s the maximum decimal of uint16”, “difference between and y”, “give me an example of”, “what are the arguments to”, “what causes this error”, “explain this code”, “given this data structure, how do I get the third row, first item”, “how do I Centre a div”, “create a header file for these functions”, “what’s the command to compile this code”, “what’s the regex to get lines containing foo”, “ffmpeg command to convert X to y”, “given this sql table, write a query to …”, “make 1000 rows of sample json data with fields name address phone”, “write a sed command to search and replace X with y”, “create a cmakelists to compile this”, etc etc and critically important “that didn’t work”, “another approach”, “any other ideas?”.

Then you debug the code it gives you.

To be fair, it does strike me that not everyone will derive the same value from chatGPT.

I have always wanted to rap and it has wrote rap lyrics that are way beyond what I could possibly do myself. If I was Andre 3000 or Tupac though I wouldn't be impressed at all.

That is why asking experts in their domain what they think about chatGPT seems to miss the point to me.

Exactly. I've been very frustrated in my attempts to make use of LLMs for programming until I tried using it in the context of languages and technology I wasn't familiar with. There it improves productivity immensely.

The primary use-case of LLMs is to reduce the barrier to entry for tasks that used to require a lot of learning. In the past, some people were employed mostly just for knowing how use some library or framework. That will no longer be the case. The value of knowledge that can be found in a book (not the same as "learned from a book") has been going down with digitization (internet, Wikipedia, search engines) and with LLMs its value will be reduced to almost nothing.

The problem is, how do you trust what GPT gives you any more than the top 5 google results you'll get? What makes its answers trustworthy?

You don't trust it more than the top 5 google results. Surely, you wouldn't argue that search engines are useless because not everything that's written on the internet is trustworthy.

For programming, you can just test if it works. That's how most people[1] check if their own code is trustworthy, absent considerations of security against malicious attackers.

In general, knowledge and authority or trustworthiness are different things. I said this makes knowledge worthless. It does not make authority or proof of authenticity worthless; probably, the opposite.

[1]: those who aren't able to always write 100% perfect code on the first try

I think the issue is the authority is worthless. At least with Google you can see the source of those 5 responses and check their quality (i.e. SEO content), ChatGPT is just 'trust me bro.'

With some LLMs, there are efforts to make them cite sources (of course this is hard). On the other hand, existing LLMs are becoming more truthful over time (GPT4 is more truthful than GPT3, etc.).

The possible applications of the technology have known required levels of trustworthiness. As the tools surpass any given threshold, new use-cases become attractive. Already at current levels of performance there are many applications where it's easier to test or continue researching a hypothesis than to find plausible and relevant ones.

No source of information is perfectly trustworthy and we have to deal with that. This is sometimes called "critical thinking" and might become a more important skill. There is also a good chance that AI systems can be trained to do this themselves and "fact-check" the responses of others while providing sources and reasoning, which would be one tool to increase the level of trustworthiness.

Or we could use our own brains, I imagine critical thinking is an important skill we shouldn't really outsource. But hey, we march to the beat of technology.

If you come to 'trust' AI for critical thinking, how do you then know when or if it has been manipulated (surely it is naive to think the governments and power centres won't do everything they can to make AI say the 'right' things)?

The value is not decreasing. The price is. The information is still as useful as it was, but supply is increasing.

I've come to much the same conclusion with getting it to produce prose. Having been paid to write for the last ~25 years I would be embarrassed to copy/paste directly from chatGPT - it manages to be both bland and verbose consistently.

But seeing how much trouble non-writers have in producing anything longer than a text message, I can see that it would have real value for them.

It's worth remembering how hard it is to make a start when you have no idea what you're doing, and the worst part of the learning process is when you know something is wrong but you don't know how to fix it.

ChatGPT seems to excel in a few narrow fields, but I think it can take a lot of people from mediocre/non-functional to somewhat acceptable and that's a pretty big leap.

I’ll speak up.

I am not a good coder, nowhere near the level of even your average professional developer.

I’m OK, and I have been writing code of some kind or another since I was a kid in the late 80’s and have managed many technical projects and I would say I have a pretty deep understanding of how it all works.

The thing I’m best at is devops type stuff and rails but even with that I’m wildly out of practice since I’m a founder/CEO type of a non software company and I can go a week or two without even touching code.

These tools have completely changed everything for me in a way that is really disconcerting.

Previously every time I wanted to change an app I had worked on, add a feature to something, or do something like call an API to grab something interesting from another business resource and pass it along, I was in for a couple hours just getting my head into what I was doing. I’d sort of block out a weekend and by sometime Sunday afternoon I would be getting there.

Now I can just start talking to GPT like it’s an experienced developer sitting next to me. Hey so if I wanted to call this service how would I structure the query. Hey look at this curl and tell me some reasons it might be getting this error.

It definitely can’t do things for me. Without a pretty deep background in software as a concept and practical experience I wouldn’t even know where to start.

But I do have those things. And also I’m the guy who just knows the business logic and what I want.

Now I can get what I want like 30-40x faster. To be honest it feels very strange, after being sort of frustrated by translating my requirements into specs for 20 years to work with developers and wasting a lot of time it feels like someone popped a babel fish in my ear.

So there’s one anecdote.

It's useful for small scripts. For example, I needed to convert a bunch of SVGs to PNGs. GTP-4 made me a node.js script in a few seconds, which saved me from the headache of googling and figuring out how to do it myself.

It's also useful for suggesting basic boilerplate code for something you're not very familiar with. If you're not familiar with something, you might get stuck in the phase of how to even get started.

In addition, it's useful for short but mathy code. I needed to place a bunch of items in a circle on an arbitrary plane in 3D space, and GPT-4 gave me a perfect answer for it.

I program in a lot of different programming languages on the regular and there is some cognitive overhead in switching back and forth between them. GPT4 is pretty helpful when I don't want to remind myself for the 10th time how one does X in language Y when I've been programming in language Z for the last 3 months.

Would you consider a CNN that predicts what chord is being played on a guitar based on still video frames of a hand on a fretboard to be complicated?

Because GPT-4 helped immensely with what was my very first NN ML project.

I didn’t approach the problem with no knowledge as I’ve had a few ML projects over the decades (graphs clustering, PageRank, eigenfaces) and I’ve skimmed a few books on PyTorch and neural nets, worked through Karpasky’s zero to hero, etc.

So I knew how to ask very specific questions and I knew the general overview but I had literally never written a single line of PyTorch code.

I’m now working on the iOS app to host the model and send MIDI and OSC data to a DAW. I haven’t made an iOS app in about a decade and that was in the Obj-C days so I’ve never written a line of Swift.

I have a demo app written in Swift that accesses the camera, converts frames to the model input, and makes predictions, and now I’m ready to use this to assist with labeling many more images!

Depends what you are doing and how much code. Especially if you work with new frameworks or apis ChatGPT is extremly helpful. I think especially on small code snippets it is strong. Guess you would have to give an example, what you tried and what you got...

I can see the argument for using it with frameworks and libraries as it can discover the library functions that do what your prompt describes for you.

It seems to have a lot of trouble with building code that works in a certain way. For example if you ask it to build a simple graph container with access functions that give you iterators over neighbourhoods of nodes or whatever it makes a mess. Sometimes it technically works, but the underlying graph datastructure is just ripped out of randomseocodingsite.com using nodes owning pointers to their connected nodes.

If you ask it to use a specific underlying structure like a vector of nodes and indexes to represent edges it doesn't quite know what to do.

So yes, I guess the initial implementation "works", but not well.

Unless it is too new.

For wgsl and webgpu, ChatGPT4 is quite useless.

To be fair, the spec is still changing, but this means, that ChatGPT4 will mix in various incompatible bits. I hope they handtune it.

I started learning Rust last week, and figured I could let copilot help me out a bit. Spent a while trying to get a very simple example to compile, with no luck. So even for a 5 year old language, it derps it up.

I use it as a replacement for google because it become more and more shit. I don't just copy/paste the result though and try to check and understand it before

It really doesn't work well unless you have a very specific focus, usually a specific function.

I'm working on something to make this more manageable, hopefully a lot more. You can get an email if you sign-in at https://inventai.xyz.

You're just not the variously skilled/unskilled people or marketing reps jumping on the hype train...

I have a feeling most of the excitement is coming from the "citizen developer" space.

"citizen developer" no longer need to hire a .... patrician(?) developer. Not sure what your classes are.

And from the people trying to sell AI tech.

I asked it to create a script to download info from the Hackernews api. It found the right url to get the information from, with me having to look it up.

Right, but you're a developer.

If you literally cannot code at all, GPT-4 is a solution.

Similarly, I cannot draw at all. But I can endlessly prompt Midjourney until I get something acceptable.

People who can't code can try GPT over and over again, and hopefully eventually get something that works (even if more by coincidence, who cares as long as it works!)

People have said this about "no-code" tools for decades. It wasn't true for any of them, and I don't see how it is true now. These tools, from labview to copilot, have a 90% problem: they can get you about 90% of the way to a complete project, but without the last 10%, they are functionally useless unless you are an expert. That is very different than art, for example, which does not have the same kind of problem unless you are trying to do something that demands accuracy (like scientific illustration).

I have no idea how you would measure the accuracy of that 90%. The huge majority of no code solutions never get published - they just quietly tick away doing their thing in some corner of the business. The only ones you're really likely to hear about are the ones that aren't working or need modifying by an expert.

Maybe there's a parallel there, that 90% of no code projects do provide complete solutions and 10% are problematic

I am pretty sure, people do 100% projects with labview without "code".

It surely is not suitable for everything, but it would not be broadly in use, if it would be a toy.

This time it's different. And I mean that sincerely, not sarcastically.

I use it to code in languages I don't know.

That's cool, but you're still a software engineer, and I bet you're doing plenty of work to debug those programs. Non-programmers don't know how to do that.

Your own example is not proof that this time it's different - this is just a no-code tool that you happen to gain productivity by using.

This sounds like a fun direct path to people living as they do in Wall-E. I for one think that's a terrible future.

I finally decided to install Copilot recently now that they have the business subscription with a more sensible privacy policy, and it's so much better than expected. Even on internal code it hasn't been trained against it often just works anyway based on the context and other nearby code.

If something requires even a slight bit of boilerplate that has a pattern to it, it writes it much faster than I could. I find it almost uncanny sometimes how well it knows what I wanted to do next even without writing any comment telling it what to do. Not to mention the hundreds of random small suggestions like writing similar method definitions, completing all of the parameters I was going to pass to something, writing out the rest of some math formula after I typed the name of the variable for the result I want to compute, etc.

People weren't kidding about the productivity boost. On the other hand, I found trying to get ChatGPT to write large chunks of code almost completely useless - it would always be incorrect in some subtle way or failing to use the style I gave it as an example, unless the task is to write some simple python scripts.

I kind of agree with you and I followed almost the same path.

Excerpt one thing I noticed. That the times it is subtly wrong I waste a ton of time finding the bug. It’s like looking for bugs in someone else’s code. I lack the context and muscle memory of the code written by the assistant.

I’m still not sure if it’s net positive. I wish there was an objective way to measure the effect on a personal, project and org level.

This was my experience as well. The time it saved me didn't add up to the time I spent in the debugger fixing subtle bugs in code that looked fine but didn't quite work. I think it would be very easy to ship hard to find bugs in production code with copilot.

I think there’s a “eyes glaze over” effect with generating lots of boilerplate, where things look generally correct, but have issues you need to be paying attention to notice and correct.

One example recently: I watched a developer scaffolding a database schema with an ORM. Plenty of boilerplate defining the fields and stuff, nice to have Copilot running, however the suggestions were baking in bad design that would work now, but almost certainly cause pain later: nullable foreign keys, text fields with inappropriate max lengths, redundant FK relationships where a relationship could be ambiguous depending on whether you joined directly on the FK or through another related table, etc.

Obviously all of that is on the developer to notice and be responsible for, but it’s really easy to just tab through the completions, see that it works (right now) and move on.

I already jokingly call myself a cyborg. I think I’m in the perfect position to take advantage of AI in my daily workflow. Why?

I research all the time because I find what I do fun. I read books, listen to conference talks, practice synthesizing, and project to explore ideas. (I will say there is a lot of conflicting advise out there so it’s important to try to mentalize the perspective to understand why the particular advise is offered. Anyways, I digress…)

I also have ~10 years programming as an IC.

Because of what I stated above I know the high level “strategy” of what I’ll want to do and most (if not some) of the implementation details. AI helps me with the “tactics” of implementation. I can ask AI or allow copilot to make implementation suggestions and I know enough to tailor the suggestions to my project (and to my coding preferences). Typically I still cross reference using documentation but using AI has helped reduce the amount of cognitive load required to do something interesting.

We live in exciting times as programmers. It’s only going to get better as our tooling becomes more sophisticated.

My experience with coding-oriented LLMs has been that they fail pretty spectacularly on non-popular languages and are incapable of fixing errors in their own code.

I've wrestled for many hours trying to get them to write me a short, simple function in a non-popular language and they failed again and again.

I also suspect these LLMs are going to have huge problems when APIs and language definitions change, because the data they're trained on usually doesn't have API/language version metadata attached to it, so they have no way of knowing which version is applicable to whatever you're trying to do.

They may spit out code in an obsolete API version, for example, or even give you code containing mixed API versions. This will be a nightmare to debug.

Where they do shine for me is for repetitive tasks. Say I am starting to write a list and start it out with "item 1", "item 2".. it will then let me tab complete some more items like value completion in e.g. excel. It has definitely saved me time.

I spent 30mins trying to get chatGPT to add a feature in some code it provided, which was not working.

It kept apologizing and giving slightly tweaked code, which I could see was not actually addressing the issue.

Then when I asked it if the feature was actually even possible, it said No.

What kind of feature?

synchronised scrolling between two windows in a chrome extension

I've just restarted writing Ruby on Rails for an interview take-home after a couple of years away from it, and literally every single step, I had to look up the exact syntax/methods, or more commonly things just didn't work out of the box, or as they should have or as expected/remembered and I had to look things up. Often the online answers are wrong (yeah go ahead and use 'first()' and 'reverse' in your model scopes to get reverse ordering, forcing rails to load the entire set from db).

Oh yes, you have to run rspec init after you bundle install it. Don't forget factorybot needs this little config file. Oh didn't I mention, form_with breaks error messages? And on and on. I knew exactly what I wanted (I used to know it pretty in-depth), but just experienced constant shin-scrapes and paper-cuts getting back into it.

This is where I imagine GPT shines - though I don't know because I can't get an account.

I have been coding collaboratively with GPT-4 extensively for the last few weeks using aider [1]. Prior to that I had been using an ad hoc workflow to code with GPT-3.5. Here's are some thoughts on my experience collaborating with AI on code:

  - Use GPT-4. It's vastly better than 3.5.
  - Use an integrated workflow like copilot or aider. Cutting and pasting code between your editor and a chat window/tool is tedious, and doesn't give GPT any context about the rest of your code. It forces you to do extra up front work to prepare your code to call a bite-sized function that can be easily cut/pasted. Asking GPT for revisions is slow with more cut/paste at every step.
  - Coding with AI lowers the activation energy to start a new change, since you expect it to be easy.
  - GPT is a champ at boilerplate, and can quickly prepare the ground for the interesting work.
  - Typing a few terse, informal sentences describing the task to GPT can help organize my own thinking.
  - I spend so much less time fiddling with annoying details of syntax, method names, parameter names/order, regex syntax, etc.
  - It's important to break down a big change into a series of smaller, self contained steps.
  - Watching GPT type out code is good "think time" for me.
  - GPT is great at "busy work" like updating docs, adding small quality of life features, writing tests, etc. I accomplish 10x as much of this work now because it's so easy to just ask for it.
  - GPT has astounding knowledge about available libraries and packages that can simplify your task.
  - Expect to iterate rapidly with GPT on more complex code changes, offering it feedback/direction.

I'm not sure I can go back to coding the old fashioned way. I realized this when GPT-4 went down for an hour or so last week.

[1] https://github.com/paul-gauthier/aider

Corporate security will be all over this when they realize that all it takes to get really bad malware or vulnerabilities inserted into your codebase is to have some “AI” that got trained nasty stuff insert something really bad into a critical application. I see the value in these companions in early training, but it could get really dangerous after that and it’s only a matter of time until some really nasty breach was caused by terrible code some AI came up with.

This will be like Solar Winds 10x by letting the attackers right into the most sacred parts of your organization… writing the code that will make or break your company.

But can’t we just filter the training set for bad things? No. The whole point of zero days is that bad folks know what they need to get inserted into your codebase before you (or companies training an AI) know it’s bad.

"insert something really bad into a critical application"

Dear god. Please let me know that you need to have code reviews, unit tests, and a QA staff for "critical applications". An AI doesn't replace any of that.

I need to hear you say it.

Has anyone compared Google's new Codey, Code Whisperer, and GPT-4 for code generation?

For my service to work at all, I feel like I really need GPT-4 or something better if that ever comes along.

But aside from GPT-4 trying to use APIs from two years ago, it can definitely make simple software without necessarily needing an actual programmer. Especially when you put it in a closed loop.

As the models get better with larger context windows and more up-to-date training, software engineers will probably defer more and more to the AI. And non-programmers will be able to build software with it also.

The time where this is just minor code completion or something is going to be pretty limited. In 3-5 years, I would be very surprised it was considered good software engineering to have humans writing code at all.

I find that GPT-4 always wins in task specific, clear prompts.

E.g. I used it on a project recently to translate code from one version to another, and keep in mind that the syntax was very different

A comparable example would be translating old class based React code to functional TSX with hooks.

And it does extremely well there. Other coding tools can’t even do that as there’s no way to load a whole prompt and the entire original file as a prompt.

Every version I’ve seen of GPT has shown me that if GPT can write most of your code, then you’re not really doing difficult work anyway. Real world problems are far too complicated for GPT to make serious impact beyond being some idea generator or stack overflow alternative.

I guess I'm old enough to remember when developers regularly used regular text editors and had to compile and link in two different steps.

I also remember how much just having an integrated development environment helped with the process.

And then intellisense! Wow -- I thought ANYbody could be a programmer then, and got worried for my job. :-)

I think this is just the next iteration. Yes -- I would expect AI large language models and general purpose AI to change the landscape of how developers do their work. It will be uncomfortable for a while -- but go to your user groups, listen to your podcasts, share the adventure with other people. We get to make it fun.

I think this will become one of those the first 80% is easy, last 20% is hard and expensive and it all falls away.

Microsoft is really trying to hear up how awesome copilot is going to be, but the real world doesn't operate in an MS Lab.

I manage a dev team and have some team members trying different coding assistants. AWS Code Whisperer is getting good marks — while it’s impressive when CoPilot or ChatGPT can spit out an entire example webapp, that’s not actually useful. Code Whisperer provides small suggestions but this seems to be more of an aid, and it’s easier to ignore/adjust when incorrect.

Our AWS rep has also said that they’ll keep our code private and won’t add them to the training set, you know, actually paying attention to legal issues like grown ups rather than creating a yawning chasm of risk.

Sandeep Pokkunuri: "Don’t get me wrong, ultimately the developer is in control. They are the ones who decide whether this code is good. They’re the ones that will run and verify that it is working as expected. They’re the ones that will ship."

Sandeep is being very optimistic. AI coding companions open up coding capabilities to even less experienced coders who mostly care of the code works. Whether the code is good will be questionable.

Until they try to do something slightly novel and find they need an actual developer to help them along.

Not to mention security or other concerns.

In a sense, ChatGPT for coding is the ultimate "build something for other start-ups" business idea isn't it? It allows barely tech literate, read people knowing more about actual coding than I do but nowhere near enough to qualify as a developer, to found tech start-ups without a tech co-founder. Considering how often this question gets asked, well, ChatGPT is just genious. Not that I am convinced it'll actually serve a purpose other than allowing peiple to be lazy enough to not study and learn stuff themselves.

I use it to ask questions on how to do things that I am not clear about in frameworks I am not familiar with. For instance the best way to implement paging using a spring webclient. You have to chat with it to get it to provide the most optimal answer— the first solution was recursive.

I think it would slow me down in areas I already know about. The IntelliJ tools are already pretty good.

What are those brackets and why can't I copy them? They behave like it's some kind of RTL character when trying to select on mobile.