And if not using git, just fiddle around with a scripting language to automate the update, like check github's release API to see if there was some change between your local and the remote version somehow.
In principle, the answer should be (if we ignored the language community and the model of the stdlib, which we shouldn't do), that structs should be used for most things, and classes only when they are needed. There's nothing a class can do that a struct can't, and structs are not automatically allocated in the heap, so they take some pressure off the GC. Go showed that you can have a fully managed GC language where all types are value types, and get pretty good ergonomics. Java is adding support for value types specifically for performance reasons, and so may have a similar attitude in the far future.
Note that Go does have one feature that makes value types more ergonomic - the ability to explicitly take a reference to one and use it as any other variable, using the syntax of C pointers (but without pointer arithmetic). In C#, if you need a reference to a structs type, your options are more limited. There is `ref` for function parameters, but if you want to store it in a field, you need to use some class type as a box, or perhaps an array of 1 element, since there are no "ref fields" in this sense (there are ref fields, but they are a completely different thing, related to the "ref structs").
Now, in working with real C# code and real C# programmers, all this is false. The community has always preferred using structs almost exclusively for small immutable values, like Color. The standard library and most C# code is not designed with wide use of structs in mind, so it's possible various methods will copy structs unnecessarily around. People aren't used to the semantics of structs, and there is no syntactic difference between structs and classes, so mutable structs will often cause confusion where people won't realize they're modifying a temporary copy instead of the modifying the original.
> using the syntax of C pointers (but without pointer arithmetic)
Go structs, when they have their address taken, act the same way object references do in C# or Java (that is, *MyStruct). Taking an address of a struct in Go and assigning it to a location likely turns it into a heap allocation. Even when not, it is important to understand what a stack in Go is and how it is implemented:
Go makes a different set of tradeoffs by using a virtual/managed stack. When more memory on stack is required - the request goes through an allocator and the new memory range is attached to a previous one in a linked-list fashion (if this has changed in recent versions - please correct me). This is rather effective but notably can suffer from locality issues where, for example, two adjacently accessed structs in a hot loop are placed in different memory locations. Modern CPUs can deal with such non-locality of data well, but it is a concern still.
It also, in a way, makes it somewhat similar to the role Gen0 / Nursery GC heaps play in .NET and OpenJDK GC implementations - heap allocations just bump a pointer within thread-local allocation context, and if a specific heap has ran out of free memory - more memory is requested from GC (or a collection is triggered, or both). By nature of both having generational heaps and moving garbage collector, the data in the heap has high locality and consecutively allocated objects are placed in a linear fashion. One of the main drawbacks of such approach is much higher implementation complexity.
Additionally, the Go choice comes at a tradeoff of making FFI (comparatively) expensive - you pay about 0.5-2ns for a call across interop into C in .NET (when statically linked, it's just a direct call + branch), where-as the price of FFI in GoGC is around 50ns (and also interacts worse with the executor if you block goroutines).
Overall, the design behind memory management in Go is interesting and makes a lot of sense for the scenarios Go was intended for (lightweight networked microservices, CLI tooling, background daemons, etc.).
However, it trades off smaller memory footprint for a significantly lower allocation throughput. Because GC in Go is, at least partially, write-barrier driven, it makes WBs much more expensive - something you pay for when you assign a Go struct pointer to a heap or not provably local location (I don't know if Go performs WB elision/cheap WB selection the way OpenJDK and .NET do).
To explore this further, I put together a small demonstration[0] based on BenchmarksGame's BinaryTrees suite which stresses this exact scenario. Both Go and C# there can be additionally inspected with e.g. dtrace on macOS or most other native profilers like Samply[1].
> Now, in working with real C# code and real C# programmers, all this is false...
> but if you want to store it in a field, you need to use some class type as a box,
This does not correspond to language specification or the kind of code that is being written outside of enterprise.
First of all, `ref T` syntax in C# is a much more powerful concept than a simple reference to a local variable. `ref T` in .NET is a 'byref' aka managed pointer. Byrefs can point to arbitrary memory and are GC-aware. This means that they can point to stack, unmanged memory (you can even mmap it directly) GC on NonGC heap object interiors, etc. If they point to GC heap object interiors, they are appropriately updated by GC when it moves the objects, and are ignored when they are not. They cannot be stored on heap, which does cause some tension, but if you have a pointer-rich deep data structure, then classes are a better choice almost every time.
Byrefs can be held by ref structs and the most common example used everywhere today is Span<T> - internally it is (ref T _reference, int _length) and is used for interacting with and slicing of arbitrary contiguous memory. You can find additional details on low-level memory management techniques here: https://news.ycombinator.com/item?id=40963672
Byrefs, alongside regular C pointers in C#, support pointer arithmetics as well. Of course it is just as unsafe, but for targeted hot paths this is indispensable. You can use it for fancy things like vectorized byte pair count within a sequence without having to pin the memory: https://github.com/U8String/U8String/blob/split-refactor/Sou...
Last but not least, the average line-of-business code indeed rarely uses structs - in the past it was mostly classes, today it is a mix of classes, records, and sometimes record structs for single-field wrappers (still rarely). However, C# is a multi-paradigm language with strong low-level capabilities and there is a sea of projects beyond enterprise that make use of all the new and old low-level features. It's something that both C# and .NET were designed in mind from the very beginning. In this regard, they brings you much closer to the metal than Go unless significant changes are introduced to it.
If you're interested, feel free to explore Ryujinx[2] and Garnet[3] which are more recent examples of projects demonstrating suitability of C# in domains historically reserved for C, C++ and Rust.
I believe the Go design of having all types be structs and supporting explicit pointers to them is separate from their managed stack design, and both are separate still from the GC design.
You can have value types as the base type, and support (managed) pointers to the value types freely in a runtime like .NET or the JVM. I tend to think the data type design is better in Go, but the coroutine implementation and simplistic mark-and-sweep GC are inferior.
I'll also note that I should have checked how things had changed in C#. I hadn't used it sine C# 4 or something, when the ref keyword was limited to method parameters. Thanks for explaining the wealth of improvements since. This makes it indeed a core type of pointer, instead of being limited to a parameter passing scheme.
And yes, you're right that there exist many c# communities with different standards of use, and some are much more low level than Go can even... Go. I was talking mostly about the kind of things you'd find as recommendations from official MS docs or John Skeet answers on SO when I was saying "C# community", but you're absolutely right that this is a limited view and doesn't accurately encompass even some major forces in the ecosystem.
If they are small, then there can be a significant performance advantage to using a struct.
Imagine you have a Dictionary<TKey,TValue>. (That is, a dictionary (hash lookup) from type TKey to type TValue ).
Imagine your choice of key is a composite of 4 bytes, and you want to preserve the meaning of each, so let's say you define a class (shorthand to avoid all the get/set cruft):
class MyKey {
byte A;
byte B;
byte C;
byte D;
}
When you profile your memory usage, to your horror you discover you actually have a keysize of 64 bits for the reference to the location on the heap.
If however you use a struct, then your key is actually the 4 bytes that defines your key.
In modern .Net, you'd probably be better off defaulting to use a record (correction: record struct) type for this purpose unless you have very specific reasons for wanting fine-grained control over memory layout with the StructLayout Attribute.
Please note that the article is quite old and does not encompass the wide variety of scenarios C# is effective at.
Structs are used for but not limited to: all kinds of "transient" data containers, pass by value semantics, precise control over layout of data in memory, low-level programming and interoperating with C/C++, zero-cost abstractions via struct generics (ala Rust or templates in C++), lightweight wrappers over existing types, etc.
Most C# codebases use them without even noticing in `foreach` loops - enumerators are quite often structs, and so are Span<T> and Memory<T> (which are .NET slice types for arrays, strings and pretty much every other type of contiguous memory, including unmanaged). Tuple syntax uses structs too - `(int x, int y)` is `ValueTuple<int, int>` behind the scenes.
.NET has gotten quite good at optimizing structs, so the more general response is "you use them for the same things you use structs in C, C++". Or the same reason you would pick plain T struct in Rust over Box/Arc<T>.
Even if you exclude performance reasons, some things just make more sense as values rather than instances. A good example is Color. A Color might be an object that holds 3 integers (red, green, blue) but you want to treat it like you would a scalar value. For example, two color instances should be equal if they contain all the same values. Class instances all have their own identity -- two classes instances are not equal even if they contain the same values.
I enjoyed Rogue One, they took a unexplored slice of the story after the legends thing and created this self contained, well written journey of a group that is only mentioned in the 4th episode.
Takin may have felt a bit off (uncanny valley), but I think it was a good choice to have him included in the story nonetheless. I like the cold, unhinged personality of this character; I've grown used to Peter Cushing's acting and facial features.
I feel like, there should be more exploration of Star Wars in the aspect of "mundane" life, like it's done in Andor. There's a big universe already established. Andor really helped me understand 2 things of SW universe: the oppression which built up the motivation for Cassian to join the Rebels, and, effectiveness of the Empire, specifically the ISB. God, the exchange between Daedra Meero and Blevin, with an added mediation of the cunning Major Partagaz was excellent. Reminded me of the discussion in Jurassic Park about ethics.
>God, the exchange between Daedra Meero and Blevin, with an added mediation of the cunning Major Partagaz was excellent.
The writing in that scene is on par with the best in HBO. The political maneuvering and intrigue are reminiscent of House of Cards, and Partagaz stands out as a truly formidable leader.
So did I: I love it that it ends just where episode IV start and I like it too, spoiler alert, that the likeable protagonist do not make it (which we knew from episode IV but still).
I liked everything except for how they just give up at the end. That annoyed me. Them not making it is fine, but them all giving up when it looked like there were many viable methods of escape annoyed me to no end. I still think it's the best Star Wars movie overall.
Yes, Andor is far and away the best SW content produced in recent years. What I really want to see is what happens with Daedra Meero. She’s such an interesting character. She says at one point that the Empire’s cracking down and sharp authoritarianism is only playing into the rebels’ hands. But of course that is what the Empire is, given that it’s run by a Sith. So we have a “good” Empire employee who wants to be effective at her job. How does she reconcile all those things? What does she do after the Death Star destroys Alderaan? Does she really buy into the Empire and its ways 100%? I’d love to know more about her backstory and frontstory, so to speak, from Andor.
Rogue One and Andor seem to me to be perfectly adjusted SW stories for the fans of the originals who are now 40+ years older. I hope Andor gets to stretch its legs fully story-wise, there is immense potential in the buildup so far!
Not sure why this was downvoted, but I also assumed they were referring to this scene, the Hammond + Grant + Sadler + Malcolm + Gennaro lunch debate in the original.
I think I saw all of them in a CSS Tricks post, but I too took the "simple" way. At first, I marvelled at the complexity of CSS units, but it didn't take long for me to tire of them.
Some tricks are still useful though, like using `calc` with `v[w|h]`, `[r]em` and `%`, that yield favourable and interesting results; fortunately you only have to remember and understand a subset of units to take advantage of this.
I myself use 2 terminals: Alacritty and WezTerm. The former for local work, and the latter for my VMs, and occasional remote server work.
I couldn't explain exactly why I use separated applications, since WezTerm works well for local (which even has a menu entry for that) and remote alike, but I'm comfortable with the workflow that I have today, and for me it's enough and what matters.
Do you not use it any more? Have you switched to Eglot or something?
I use lsp-mode, was going to switch to eglot, but it doesn't have the features that lsp-mode has, like lsp-ui (as far as I can remember), and the last time a compared (last year), the performance impact was somewhat noticeable.
Eglot philosophy is to hook into existing emacs features instead of reinventing the wheel, so you may find some of the lsp-mode/lsp-ui features in other packages. imenu-list for sideline navigation, breadcrumb, flymake has a way to display inline diagnostics (can never remember how as I don't use it and google does not help).
I work on a huge codebase with eglot+clangd and the few times where it gets slow is where there are tons of flymake warnings. Like when you enable all the clang-tidy checks.
eglot is part of emacs 29.
I have been testing it out for my .NET development (with Omnisharp)
I officially started with .NET back in 2008 and, at the time, you really had to use Visual Studio. I knew emacs was not an option at that time. Since .NET Core I knew I was getting closer and closer for emacs becomming an option.
Then comes Visual Studio Code.
I have experiemented with C# development in emacs for a few years, now. I cannot remember now but it was likely with lsp-mode. When I found out eglot is in v29, it was time to revisit it. It has been positive so far.
I actually forced myself doing .net development in emacs this weekend. I am 90% there! Need to see if I can debug in emacs, now. I hope so!
Once I am happy, I will continue to add more interactive functions, saving me typing 'dotnet cli..' commands. I also need to revisit yasnippet to make coding more effecient in C#.
I had a customer request a project that required C# due to a few fancy dependencies they had purchased back in 2008 - which was my first proper contact with .NET. I got frustrated with the Visual Studio UI after a few hours, and switched to editing in Emacs and compiling via CLI tools, only using Visual Studio for editing the project data. It wasn't too complicated to kick cc-mode into behaving somewhat sensibly with C# code.
It was not just about C# editing. There were other things being used in Visual Studio at that time. Also being inexperienced in VB.NET (my first job) surrounded by 10 other seasoned devs, they discouraged anything outside of Visual Studio. I understand why when looking back.
This was also the Visual SourceSafe days. How I do not miss that!
C# wiggled its way in eventually.
Back in 2008, I was pretty much using Emacs for most programming languages. The only exception was C# and Javascript.
As the years pass to now, I rarely use a programming language outside of emacs. I have enough years in my career to not tell me what I can and cannot do to get things done.
I had my emacs setup to edit LotusScript (one of the scripting options for Lotus Notes), which is close enough to VB that it could handle the occasional file - this was a new codebase, though, so it made sense to just start with C#.
That was also the project where I tested if git reached the point where you can have "normal" people work with it as well (which worked pretty well). I never accepted any project where they'd have me work with idiotic proprietary source control, unless the project was "migrate away from that".
I like how people think I am purposely being different or difficult because I do C# development in emacs. Many have never heard of it. If they have its from forums or youtube videos.
Think of their mindset -- Its not like I have moved away from Visual Studio to Visual Studio Code. I am using Emacs. They must think I am trying to be difficult or special.
I have also swapped my L-CTRL key with CAPS specially for improving my emacs experience.
Regardless, when we do pair programming (or something) on my screen and they watch me use Emacs on a C# project, they soon understand. :-)
I've found lsp-mode to troublesome to configure for what it provided, and occasionally getting in the way, so with emacs 29 I just moved to the integrated eglot. Not that I'm using it much, though - I've found it marginally useful for getting used to new languages, but generally prefer to just have full documentation open - once I'm familiar with the code base completion tends to just get in the way, so I often have any form of completion switched off anyway.
I'm a newcomer, started using Emacs in 2022, well, not vanilla, but Doom Emacs. I was immediately hooked! The defaults are sane for my workflow, and to make it short: it boosted my productivity. But, I've been bitten many times by Emacs' UI nature, unfortunately, mainly in big repos. It's not bad enough to drive me away though.
I has gotten better over this past year, I've noticed many improvements with Nativa Compilation and Tree-Sitter, not sure if it can get even better; I sure hope so.
I bought an iPhone last year. I started regretting it after 3 months of usage. Trivial things are a burden to set up, like:
- custom ringtone;
- emulators;
- lock screen rotation in landscape;
- listen to music from local storage without an Apple subscription;
I could go on and on. Most of them have solutions, but again, a burden to configure compared to an Android device. My next smartphone will be Android, once again. I just wish small smartphones were more common and had battery life comparable to Apple's.
I bought a second hand MacBook Pro 2015 to use Linux on. I'll let Asahi cook a bit more before moving to an M series.
I stood in line on launch day in June 2007 for the first iPhone, and used one exclusively until 2022 when I started running it side-by-side with a Pixel running GrapheneOS.
I personally tried it with Apple Music and it did not work if you had no connectivity — and, in fact, because about half of the app itself appears to be loaded on demand, parts of the UI were non-functional as well.
Also, it was not possible for me to download the equivalent of my Discover Weekly playlist for use offline; it HAD to be streamed for each. and. every. play.
I gave up and went back to Spotify. A few months later, I wiped the iPhone and sold it, leaving me with GrapheneOS exclusively.
Just last night I wiped the storage of my last Mac and it will be given away.
Given the direction Apple is going, I made the right call, which is a damn shame.
I am in the same boat and I suspect a lot of the early hour Apple users are in a similar process of thinking about it.
Some of the main draws of Apple were the freedom of the platform as a whole and the total cost of ownership that was much lower than the price tag would suggest at first.
Nowadays not only do you pay for the privilège of locked down hardware but the software is very often "meh" at best, when it's not completely trash compared to the alternatives. And their push to everything subscription with the inability to both upgrade/repair the hardware easily makes the price tag unpalatable. They aren't better built than many expensive options nowadays...
Most of the current Apple software works well only if you want to pay for content, for your own stuff the softwares are rather lacking which makes the base offering uninteresting and since 3rd party software has become extremely expensive in the ecosystem it truly is a luxury platform.
In the end, there is nothing that is better done on Apple's ecosystem nowadays, most of their historical advantage have evaporated and arguing about the battery life of Apple Silicon seems like splitting hair considering the overall cost...
A few months ago, I asked Siri to play a specific song from my iTunes library. I've done this successfully many times before, but this time it misunderstood which song I requested. The song it thought I asked for wasn't present in my library, but it was present in Apple Music. Without asking for confirmation, it enrolled me in a trial subscription and begin streaming.
Hello, I would like to take this opportunity and ask for help here, about using A.I. with my own codebase.
Context:
I missed [almost] the entire A.I. wave, but I knew that one day I would have to learn something about and/or use it. That day has come. I'm allocated in one team, that is migrating to another engine, let's say "engine A → engine B". We are looking from the perspective of A, to map the entries for B (inbound), and after the request to B is returned, we map back to A's model (outbound). This is a chore, and much of the work is repetitive, but it comes with its edge cases that we need to look out for and unfortunately there isn't a solid foundation of patterns apart from the Domain-driven design (DDD) thing. It seemed like a good use case for an A.I.
Attempts: I began by asking to ChatGPT and Bard, with questions similar to: "how to train LLM on own codebase" and "how to get started with prompt engineering using own codebase".
I concluded that, fine-tuning is expensive, for large models, unrealistic for my RTX 3060 with 6Gb VRAM, no surprise there; so, I searched here, in Hacker News, for keywords like "llama", "fine-tuning", "local machine", etc, and I found out about ollama and DeepSeek.
I tried both ollama and DeepSeek, the former was slow but not as slow as the latter, which was dead slow, using a 13B model. I tried the 6/7B model (I think it was codellama) and I got reasonable results and speed. After feeding it some data, I was on my way to try and train on the codebase when a friend of mine came and suggested that I use Retrieval-Augmented Generation (RAG), I have yet to try it, with a setup Langchain + Ollama.
Any thoughts, suggestions or experiences to share?
> much of the work is repetitive, but it comes with its edge cases
for the repetitive stuff, just use copilot embedded in whatever editor you use.
the edge cases are tricky, to actually avoid these the model would need an understanding of both the use case (which is easy to describe to the model) and the code base itself (which is difficult, since description/docstring is not enough to capture the complex behaviors that can arise from interactions between parts of your codebase).
idk how you would train/finetune a model to somehow have this understanding of your code base, I doubt just doing next token prediction would help, you'd likely have to create chat data discussing the intricacies of your code base and do DPO/RLFH to bake it into your model.
look into techniques like qlora that'll reduce the needed memory during tuning. look into platforms like vast ai to rent GPUs for cheap.
RAG/Agents could be useful but probably not. could store info about functions in your codebase such as the signature, the function it calls, its docstring, and known edge cases associated with it. if you don't have docstrings using a LLM to generate them is feasible.
You could try using aider [0], which is my open source ai coding tool. You need an OpenAI API key, and aider supports any of the GPT 3.5 or 4 models. Aider has features that make it work well with existing code bases, like git integration and a "repository map". The repo map is used to send GPT-4 a distilled map of your code base, focused specifically on the coding task at hand [1]. This provides useful context so that GPT can understand the relevant parts of the larger code base when making a specific change.
I gave aider an afternoon this week (which is not a lot of time to learn anything, of course). It did too many wild things to the project and repository I was using it with (Rails 7 base) to comfortably explore this further – for now.
Paul, if you are up for that, it would be tremendously helpful to have a video(series) that shows what aider can realistically do given a boring, medium sized CRUD code base. The logs in the examples are too narrow and also build not intuition about what to do when things go wrong.
Thanks for trying aider, and sorry to hear you had trouble getting the hang of it. It might be worth looking through some of the tips on the aider github page:
In particular, this is one of the most important tips: Large changes are best performed as a sequence of thoughtful bite sized steps, where you plan out the approach and overall design. Walk GPT through changes like you might with a junior dev. Ask for a refactor to prepare, then ask for the actual change. Spend the time to ask for code quality/structure improvements.
Not sure if this was a factor in your attempts? But it's best not to ask for a big sweeping change all at once. It's hard to unambiguously and completely specify what you want, and it's also harder for GPT to succeed at bigger changes in one bite.
My goal is to (1) try running an A.I. locally and see if it works, out curiosity, and (2) delve into A.I. concepts. I do not intend to use it as the definitive tool to code for me, and maybe I shouldn't.
Realistically, since we are in a Azure ecosystem, I would use Codex to try out a solution.
I think you are going the wrong way - I would start with the best possible AI (which will probably be GPT4) and see if it can do it, and then walk backwards to local AI deployments (which are currently significantly weaker).
> My goal is to (1) try running an A.I. locally and see if it works, out curiosity, and (2) delve into A.I. concepts.
No this isn't what I mean.
Unrelated to where the model runs, it is unclear how you are specifying what you want the model to do. Usually you input the code and some natural language instructions into the context of the model and it will follow the instructions to generate new code.
But you say you are fine tuning, and it's unclear why this is.
There could be good reasons to do this: Maybe you have an in-house language it doesn't know, or maybe you are fine tuning on old-data -> old-transform-code -> new data + errors or something.
But usually you don't need to do fine tuning for this task.
Is writing the code the hard part, or is ensuring what you've written is correct the hard part? I'd guess the latter and AI will not ensure the code is correct.
Can you record input and output at some layers of your system and then use that data to test the ported code? Make sure the inputs produce the same outputs.
Yes, and I also imagine some kind of AI thing would be useful for reading logs and writing other tests that document what the system does in a nice bdd style.
But you still have to read the tests and decide if that's what you want the code to do, and make sure the descriptions aren't gobbledygook.
> We are looking from the perspective of A, to map the entries for B (inbound), and after the request to B is returned, we map back to A's model (outbound). This is a chore, and much of the work is repetitive, but it comes with its edge cases that we need to look out for and unfortunately there isn't a solid foundation of patterns apart from the Domain-driven design (DDD) thing.
This sounds like a job for protobufs or some kind of serialization solution. And you already know there are dragons here, so letting a LLM try and solve this is just going to mean more rework/validation for you.
If you don't understand the problem space, hire a consultant. LLMs are not consultants (yet). Either way, I'd quit wasting time on trying to feed your codebase into a LLM and just do the work.
> much of the work is repetitive, but it comes with its edge cases that we need to look out for
Then don't use AI for it.
Bluntly.
This is a poor use-case; it doesn't matter what model you use, you'll get a disappointing result.
These are the domains where using AI coding currently shines:
1) You're approaching a new well established domain (eg. building an android app in kotlin), and you already know how to build things / apps, but not specifically that exact domain.
Example: How do I do X but for an android app in kotlin?
2) You're building out a generic scaffold for a project and need some tedious (but generic) work done.
3) You have a standard, but specific question regarding your code, and although related Q/A answers exist, nothing seems to specifically target the issue you're having.
Example: My nginx configuration is giving me [SPECIFIC ERROR] for [CONFIG FILE]. What's wrong and how can I fix it?
The domains where it does not work are:
1) You have some generic code with domain/company/whatever specific edge cases.
The edge cases, broadly speaking, no matter how well documented, will not be handled well by the model.
Edge cases are exactly that; edge cases; the common medium of 'how to x' does not cover edge cases; the edge cases will not be covered and the results will require you to review and complete them manually.
2) You have some specific piece of code you want to refactor 'to solve xxx', but the code is not covered well by tests.
LLMs struggle to refactor existing code, and the difficulty is proportional to the code length. There are technical reasons for this (mainly randomizing token weights), but tldr; it's basically a crap shot.
Might work. Might not. If you have no tests who knows? You have to manually verify both the new functionality and the old functionality, but maybe it helps a bit, at scale, for trivial problems.
3) You're doing something obscure or using a new library / new version of the library.
The LLM will have no context for this, and will generate rubbish / old deprecated content.
Obscure requirements have an unfortunate tendency to mimic the few training examples that exist, and may generate verbatim copies, depending on the model you use.
...
So. Concrete advice:
1) sigh~
> a friend of mine came and suggested that I use Retrieval-Augmented Generation (RAG), I have yet to try it, with a setup Langchain + Ollama.
Ignore this advice. RAG and langchain are not the solutions you are looking for.
2) Use a normal coding assistant like copilot.
This is the most effective way to use AI right now.
There are some frameworks that let you use open source models if you don't want to use openAI.
3) Do not attempt to bulk generate code.
AI coding isn't at that level. Right now, the tooling is primitive, and large scale coherent code generation is... not impossible, but it is difficult (see below).
You will be more effective using an existing proven path that uses 'copilot' style helpers.
However...
...if you do want to pursue code generation, here's a broad blueprint to follow:
- decompose your task into steps
- decompose you steps in functions
- generate or write tests and function definitions
- generate an api specification (eg. .d.ts file) for your function definitions
- for each function definition, generate the code for the function passing the api specification in as the context. eg. "Given functions x, y, z with the specs... ; generate an implementation of q that does ...".
- repeated generate multiple outputs for the above until you get one that passes the tests you wrote.
This approach broadly scales to reasonably complex problems, so long as you partition your problem into module sized chunks.
I personally like to put something like "you're building a library/package to do xxx" or "as a one file header" as a top level in the prompt, as it seems to link into the 'this should be isolated and a package' style of output.
However, I will caveat this with two points:
1) You generate a lot of code this way, and that's expensive if you use a charge-per-completion API.
2) The results are not always coherent and functions tend to (depending on the model, eg. 7B mistral) inline implementations for 'trivial' functions instead of using functions (eg. if you define Vector::add, the model will 50/50 just go a = new Vector(a.x + b.x, a.y + b.y)).
I've found that the current models other than GPT4 are prone to incoherence as the problem size scales.
7B models, specifically, perform significantly worse than larger models.
I have limited success with feeding a LLM (dolphin finetune of mixtral) a content of a merge request coming from my team. It was few thousand lines of added integration test code and I just couldn't be bothered/had little time to really delve.
I slapped the diff and used about 10 prompt strategies to get anything meaningful. So my first initial impressions were: clearly it was finetuned on too short responses. It kept putting in "etc.", "and other input parameters", "and other relevant information". At one point I was ready to give up; it clearly hallucinated.
Or that's what I thought: turned out there was some new edge case of a existing functionality added that was added, without ever me noticing (despite being on the same meetings).
I think it actually saved me a lot of hours or pestering other team members.
We use Preact nowadays. It feels so much better and intuitive having first class support for ESM and use the browser's technologies for events, for instance. The TSX to JS transpilation is a bonus.
In general, for us: the Vite + Preact team feels just right.
