Well I have personally tested it on the green field projects I mostly work on and it does the grunt work of IAC (Terraform) and even did a decently complicated API with some detailed instructions like I would give another developer.
I’ve done literally dozens of short term quick turn around POCs from doing the full stack from an empty AWS account to “DevOps” to the software development -> training customers how to fish and showing them the concepts -> move on to next projects between working at AWS ProServe and now a third party consulting company. I’m familiar with the level of effort for these types of projects. I know how many fewer man hours it takes me now.
I have avoided front end work for well over a decade. I had to modify the front end part of the project we released to the customer that another developer did to remove all of the company specific stuff to make it generic so I could put it in our internal repo. I didn’t touch one line of front end code to make the decently extensive modifications, honestly I didn’t even look at the front end changes. I just made sure it worked as expected.
If you are “consulting” on an hourly rate, you’re doing it wrong. The company and I get paid for delivering projects not the number of hours we work. A smaller project may just say they have me for 6 weeks with known deliverable. I’m rarely working 40 hours a week.
When I did do one short term project independently, I gave them the amount I was going to charge for the project based on the requirements.
All consulting companies - including the division at AWS - always eventually expand to the staff augmentation model where you assign warm bodies and the client assigns the work. I have always refused to touch that kind of work with a ten foot pole.
All of my consulting work has been working full time and salaries for either the consulting division of AWS where I got the same structured 4 year base + RSUs as every other employee or now making the same amount (with a lot less stress and better benefits) in cash.
I’m working much less now than I ever have in my life partially because I’m getting paid for my expertise and not for how much code I can pump out.
I am working fewer hours. I at most work 4 hours a day unless it’s a meeting heavy day. I haven’t typed a line of code in the last 8 months yet I’ve produced just as much work as I did before LLMs.
Right now its just a tool you can use or not and if you are smart enough, you figure out very quickly when to use a tool for efficency and when not.
I do not vibe code my core architecture because i control it and know it very well. I vibe code some webui i don't care about or a hobby idea in 1-4h on a weekend because otherwise it would take me 2 full weekends.
I fix emails, i get feedback etc.
When I do experiemnts with vibe coding, i'm very aware what i'm doing.
Nonetheless, its 2025. Alone 2026 we will add so much more compute and the progress we see is just crazy fast. In a few month there will be the next version of claude, gpt, gemini and co.
And this progress will not stop tomorrow. We don't know yet how fast it will progress and when it will be suddenly a lot better then we are.
Additionally you do need to learn how to use these tools. I learned through vibe coding that i have to specify specific things i just assume the smart LLM will do right without me telling for example.
Now i'm thinking about doing an experiemnt were i record everything about a small project i want to do, to then subscribe it into text and then feeding it into an llm to strucuture it and then build me that thing. I could walk around outside with a headset to do so and it would be a fun experiemnt how it would feel like.
I can imagine myself having some non intrusive AR Google and the ai sometimes shows me results and i basically just give feedback .
Except that's unproven. It might make you more productive, but whether you get any of that new value is untested.