LEAN manufacturing has a high-level belief that machines should be made to work around humans rather than making humans work around the machine. Humans shouldn't have to make complex movements when working with the machine, rather the machines should be designed in a way that accounts for how humans work. Doing so improved outcomes
I think if we swap “human” with “worker” that is spot on with this line of thinking. In the case of LEAN manufacturing, the system is optimized for the thing doing the work (the worker). If our goal is for the “worker” to be an agentic process and not a human, then we should be optimizing the system for that worker’s needs and not forcing the process to do complex movements around a code base optimized for humans.
The problem is that you are unlikely to be able to remove the human, so the worker will eventually become a human, because the liability will eventually land on a human and the code will eventually need to be read and modified by a human. The agents will never be mistake free, much like us
Humans will need to have visibility into application code, sure, but not likely optimized visibility - compiling and transpiling are good precedents for this. There was(is?) likely an argument to be made that humans need to be able to directly review bytecode or minified JS, but these certainly aren't optimized for humans. I do think there would be some version of an IDE that makes human traversal of a bot-optimized codebase easier, but the more we expect the LLM to do, the more we will need to favor optimizing for the LLM over optimizing for human developers.
I tend to believe we are nowhere near the point it makes sense to deploy LLMs at a scale where it makes sense to optimize code for them. They make far too many mistakes. You can always reorganize the code when you send it to them, which is what is happening anyway since it all goes into a flat context as part of the stream of input tokens.
Additionally
- a number of issues around how specific languages work will arise because they expect code to be organized a specific way
- the snippet assembly phase with LLMs will introduce non-determinism to the software building process, which in itself should give pause to reconsider the approach. Reproducibility and provenance are important to the software supply chain
I know we had a 486 at some point when I was a kid, but this code would have predated that machine by a few years so I'm not entirely sure. There were also machines at the school (where he worked) but not 100% sure what they had when.
I got fortunate with the disks - I just bought a USB 3.5mm reader and hooked it up to my Ubuntu machine, and there the files were, no corruption that I can tell. I'm glad other people are also enjoying the find!
These scripts were written by my father in the 1980s, he was a middle school science teacher and wrote these to help him manage classrooms. I think it’s a cool example of the way personal computing changed life in that era; he was never a software engineer, just a guy that needed to automate some tedious work. I encoded the BAS files in utf8 so they are easier to read.
A bit tangential, but I had a chemistry teacher in high school (~2007) who I liked a lot, but was not super tech-savy. I was staying after class to get some help with a homework assignment, but we ended up chatting a bit because he was funny and very friendly.
He mentioned how long grades took him to calculate because he had a lot of custom weighting for stuff, and he was doing everything with a Texas Instruments calculator, and he was dreading it because the semester was almost over. I said something like "Couldn't you do this with Excel or something?"
He didn't really know what Excel was, but my dad had like a week before taught me how to use it for basic number crunching stuff, so I loaded it up on his computer, and showed him some very basic arithmetic stuff, and how to use cells as variables and roughly how copying and pasting worked. He was immediately amazed and I left feeling super smart.
The next week, I chatted with him again, and he had completely overtaken my Excel ability. He learned a bunch of little tricks I was unfamiliar with, had redone all of the gradebooks with it, added a lot of conditionals, color coding, and I think might have even been using the VBA. He also added graphs to look at how much he had to curve things and to see if tests were fair. It was life-changing for him; a task that used to take him hours of manual work was reduced to about fifteen minutes of playing copypaste.
Excel is particularly interesting to me, because it's very immediately useful, but you can also go super deep with learning new features, and it keeps getting more useful!
Spreadsheets are one of the best technological advancements in the past 100 years - and I'm not exaggerating. There is just so much you can do with them; they are probably the most accessible way for a human to access the power of CPUs and computation. For most problems, I'd rather use a spreadsheet than a general-purpose programming language, because the spreadsheet itself offers a quick away to leverage data structures and functions.
Yep, completely agree. I remember I saw a talk six or seven years ago where they said something to effect of "spreadsheets are the best programming language because they allow people to program without even realizing they're programming", and I agree with that sentiment.
They give you access to a lot of programming logic, they're visual, they're reactive, they're easy to learn, and maybe most importantly, the results are instant. Instead of having to try and understand data types and scope and having to run a program from scratch for every change, you can just change a value and see the result right away.
I use spreadsheets for nearly anything involving "numbers" (outside of stuff with lots and lots of data), because it's a lot less of a headache than setting up a "real" language. They're wonderful.
One of the better ways I've seen dbt described is very similar to an MVC framework for data (I didn't see that mentioned yet, if it has been sorry I missed it).
Where a rough mapping would be:
model >> source
controller >> ephemeral model
view >> materialized (view, table, incremental) model
Like many MVCs it provides abstractions and helpers for common tasks such as performing incremental updates, snapshotting time series, common manipulations like creating date spines. Like many MVCs it supports a robust ecosystem of plugins that make it easy to re-use stable transforms for common datasets. Where Django passes you the request object instead of hand-parsing http responses, dbt allows you to loop through a set of derived schemas and dry out your SQL code.
You would generally use dbt as a transform framework for the same reason you'd use Ruby on Rails or Django etc as a web framework - because it provides you with a ton of otherwise repetitive non-differentiating code pre-baked and ready to go. You could keep a folder of sql files you arbitrarily run, and you could roll your own web framework from scratch. Personally I wouldn't do those things.
Health Union | Data Engineer & Analytics Engineer | Philadelphia PA | PA or possible REMOTE |
We are building some super cool stuff to help people living with chronic medical conditions find community. You get to drive Open Source Software and proprietary projects we maintain, build cutting edge stuff in microservices and work with a badass data team. Check out this video walkthrough of our tech stack and codebase to see what you'll be doing if you join us:
We have an amazing office in Philly, and remote is quickly becoming an option as the cultural tides shift. Hit me up for details, or if you want to talk shop about the company or what we're building before applying.
