I have a kind of similar story when I started studying in the late 2000s years. There was a company that had a specialized and really expensive measurement device. But the vendor went out of service. They changed part of their system but kept the measurement device only to find out it could not talk to the new system because the file system of the data was proprietary to the measurement device.
I got a student job at that company and one of my first tasks (and that of several students before me) was to open measurement results on the device and type them into excel spreadsheets. I did this for an hour or so until I became totally bored so I started to tinker around. The measurement device had it's own PC that booted Windows (I think it was 95 or 98) and autostarted their software in full screen/some sort of . This was easy to bypass via the task manager and running explorer.exe. I found out that the proprietary file format was simplay an MS Access file with a different extension. I tried to open it, but the file was password protected.
At this time I had little to none experience with programming or anything else that was "low level" computer stuff, but I occasionally stumbled about writeups about hacks and exploits and skimmed over them. So I was pretty sure that there had to be a hardcoded password somewhere. I started to open every file I could in a text editor with no luck. Then I got a hex editor and opened the binaries and finally, in a dll there was a password.
The next few days at this job I spent teaching myself enough Python to read the Access files and write the contents into an Excel file.
This worked and I used the free time to study/eat/sleep while getting paid for it but then one of my supervisors found out that I wasn't doing anything but still got results and wondered how I did it. He immediately put me onto another problem they had, thus starting my career as a software engineer.
Its weird how luck plays a part in all this - I remember as a Student temping for the giant Audit firm, Arthur Andersen (become accenture eventually). I was doing something like typing from one system to another. I think I found VBA and demonstrated how I could do a weeks work in a lunch hour.
My boss took one look, freaked out and I was back at the Temping agency.
We are still very far from a Software Literate society.
My dad did this for someone's whole team as a favour because he overheard what they were doing over lunch or something like that. Later, he met the manager of the team he'd done it for and asked if they were going to transfer any of the employees that got freed up for the work and she said "no, there's still lots of work to do and the speedup, while helpful, wasn't major."
Went to her retirement party a year later and got the truth: Her pension was tied to her salary as a manager as an average of her last three years, and her salary as a manager was directly tied to her number of reports. If she'd have given up half the team as she could have, she would have lost hundreds of thousands of dollars over the rest of her life.
There was a lot of that type of stuff going on when computers first came in and hackers here and there started optimizing things. Individual interest and politics doesn't disappear.
It wasn't the humble beginnings of my programming career (far from it), but I still ran into a similar situation in an internship at a recycling company, just a couple of years ago. "Hey we got all these daily excel sheets that someone needs to sit down and aggregate into these monthly balance sheets." A couple hours of VBA later and I had automated it.
Then a few days later, I happened to talk to someone from the accounting side about this experience. She mentioned that they were actually also tracking these same numbers and apparently had an automated system already. The production floor just knew nothing of it and had been doing the same task by hand forever. I guess the realization how inefficient organizations can be was probably the greater learning experience for me there.
Same story here. Was working for a manufacturer doing help desk/support. One of two people in the dept. First task was to help someone in a different dept sort through PDF files, and rename them to the company standard format. I wrote a python script to do this instead of renaming one by one.
Got reamed because "there's no way that is accurate and it might mess something up".
They where renaming the file based on the date of review, and the creators name.... Both of which were in the damn metadata.
If your position was independent enough, you could maybe get by with a "show me a mistake, and I'll acknowledge and fix it and eat crow. If you are just guessing, go look for an error."
Or if there was storage space enough, make a mirror directory with copies renamed, so there was obviously no loss of original files, then invite them to check as many as they wanted.
I do understand that many people don't trust automation like this, I've been there too.
I have a rule that helps me be a decent human being. Automate to make my jobs easy, never automate someone else's job. Too many hot shot software developers think that putting someone out of a job is a good thing
Your story is probably the more common one by far. Most employers and managers (especially for starter/entry-level jobs) are incredibly insecure, and any sign of intellect or creativity scares them. I've learned the hard way to never ruffle feathers by trying to think or solve problems at full power on a job. Just do what's expected and move on.
It's also why the nimble newcomer can often disrupt and slay the giant incumbent. It's a lot easier to create high-performance environments when you have teams small enough for all members to know each other personally than when you have hordes of people and have to use lowest-common-denominator bureaucracy to manage them in bulk.
I'm doubt much of this behaviour is driven insecurity / fear of intellect and creativity - it's mainly an incentive problem IMO. If I'm incentivised to keep an inefficient manual system going, I'll fight for that inefficient manual system until the bitter end. Sending a smart but non-business-savvy temp back to the temp agency is small potatoes.
I don't think it's insecurity all the time, although I have run into people who refuse to trust an automated process because they can't see the work getting done...
I think a lot of resistance is due to, like others have said in this thread, unexpected incentives.
If someone's career progress, financial incentives, or work politics require the status quo, no changes will be welcome, and might even be perceived as an attack. (Yep, been there.)
If you miraculously can get the incentives changed, then you can make progress...
This is super similar to my start. I was hired at 17 one summer to do data entry for a surveying company, by putting timesheets created in Excel into a central system for billing clients. That got boring after a day, so I figured out how to use VBA (Excel on Macintosh System 8!) and wrote a macro that I linked to a button and put on the spreadsheet template, hidden off in the corner somewhere. When I got the next set of sheets back I hit the buttons and my job was done.
I showed the bosses, and was immediately put to work on some much more interesting stuff linking Lotus Notes with SQL Server for reporting and dashboarding, and then I was off to the races.
(The previous year I'd spent the summer making concrete garden ornaments with a group of ex-cons in a shed in the back of a farm - an experience which certainly made me appreciate the comforts of doing spreadsheets in an air-conditioned office, though my muscles were never quite as good.)
I got a student job at that company and one of my first tasks (and that of several students before me) was to open measurement results on the device and type them into excel spreadsheets. I did this for an hour or so until I became totally bored so I started to tinker around. The measurement device had it's own PC that booted Windows (I think it was 95 or 98) and autostarted their software in full screen/some sort of . This was easy to bypass via the task manager and running explorer.exe. I found out that the proprietary file format was simplay an MS Access file with a different extension. I tried to open it, but the file was password protected. At this time I had little to none experience with programming or anything else that was "low level" computer stuff, but I occasionally stumbled about writeups about hacks and exploits and skimmed over them. So I was pretty sure that there had to be a hardcoded password somewhere. I started to open every file I could in a text editor with no luck. Then I got a hex editor and opened the binaries and finally, in a dll there was a password. The next few days at this job I spent teaching myself enough Python to read the Access files and write the contents into an Excel file.
This worked and I used the free time to study/eat/sleep while getting paid for it but then one of my supervisors found out that I wasn't doing anything but still got results and wondered how I did it. He immediately put me onto another problem they had, thus starting my career as a software engineer.