Having personally worked with the DICOM standard, I'll say it's not the worst, but definitely not the best spec to work with.
The main issue I've found is the manufacturer's implementation. They range from okay to barely meeting the minimum standards.
One thing that always annoyed me was the time stamps. They're mandated by the standard, so there's always a timestamp, but it's often useless. For example, it seems some scanners don't get their time set, over the network or otherwise, _ever_, so you might have a bunch of scans performed a few years ago on scanner A with an accurate timestamp of say 2017, but in a new scan from last week that was taken by scanner B, which hasn't had its clock configured properly, it reports a date closer to the start of the unix epoch rather than 2020.
Of course some of this is out of the manufacturers control, but temporal data is critical to analyzing changes to patient over time. I think this is one area where the DICOM standard couldn't be a bit more strict with a lot of positive impact.
One thing that always annoyed me was the time stamps
So much this. Now your example can still be countered somewhat by taking notes and writing down which scan was taken when. This is usually common practice in research.
But it can get worse when trying to integrate integrating MRI scanner software with external systems which do get a correct timestamp (or at least, different one) and which are used for research experiments like presenting stimuli and recording behaviour all together with getting MRI data.
Scanner software usually doesn't provide any means of starting it except for clicking in the UI so that means the people doing such experiment have to start 2 or more pieces of software manually (and hope they don't forget one). Now if one session lasts 10 minutes and it's not uncommon to do test sessions (with no scan) or test scans (with no corresponding experiment) or make a mistake (start scan with wrong parameters, abort, start again) and a complete test takes a couple of hours, imagine the fun of linking all of that session data together correctly based on just timestamps. I don't even know how much time has been lost or many mistakes were made (and possibly published) because of that. But it won't be 0, it's just too easy to mess up.
Of course some of this is out of the manufacturers contro
Only partly. Some of these systems are always connected to the internet anway so then at least try to make sure NTP works.
Do you know what software is usually used to get fMRI timestamps? My last position was in an fMRI lab, and we had this in-house piece of Windows 98 software that did real-time recording down to <10ms, but it was a huge codebase to maintain and pretty much impossible for me to migrate it to Windows 10. The output file was pretty hard to decode too. I've always wondered what else is out there.
Not 100% sure what you mean with 'fMRI timestamps' but since you say 'recording' I assume it's the TTL/optical trigger produced at the start of each volume? And since you mention windows 98 and < 10mSec the in-house software is maybe using the parallal port? From former experience with such software: that's usually a mess, especially on later versions of Windows where direct access to such port was being ruled out.
Anyway the principle is you use an A/D converter to digitize the signal then save it (either the complete signal or just timestamps of rising edge, the latter can also be done directly on some A/D cards). In the system I'm currently working on I'm using National Instruments PCI-X cards their C SDK to get the data but that's just because I need I/O channels for other things as well and because it has to include online processing of data. Otherwise you could just save the data at a nice and slow rate using Labview or Matlab or ... (pretty sure there are open source solutions as well). And there are many alternatives to National instruments as well of course. All woth resolution down to a couple of nanoseconds if needed. But note that it's still not trivial to get an accurate 'time of day' timestamp from it (should that be needed): at the recording site there's no way of knowing exactly when the data acquisition starts. And there can be clock drift.
Yes, that's it. Sorry it's been a while, and yes it was using the parallel port. I wish I had thought to try modern PCIe cards and different software like Matlab, that would probably have been much easier.
This is why I hate DICOM. DICOM itself is fine, it's just that every device manufacturer mucks it up. I remember once working with a device that claimed to meet a certain supplement. Got some sample data from a real world device, implemented the standard in our system, and started playing with it, but every image I got back was just a garbled mess. After tons of attempts, I eventually got in contact with a developer at the manufacturer who told me they encrypt the image data so that it can only be used by their supporting software...
I think I know which manufacturer you are talking about, and this is super shady. They also go around in conferences talking about how "the data belongs to the patient" and how they "support standards".
Maybe now that the EU has started mandating data interoperability this could improve, but I have a feeling they will just find a way to tick that box without actually offering any diagnostically relevant data.
All manufacturers do this indeed. The “good” data is in the private fields that can only be read by the manufacturer’s proprietary software. The fields that are open for everyone usually have a lower quality version of the actual data. The latter is usually good enough for a small preview, but not good enough for proper diagnostic usage.
This was an ultrasound machine. Philips would first have you import the data in their own QLab software and from there export a copy (which misses some of the data from the private fields). With GE reading the data was completely impossible.
I don't think it is, at least in the US. What most people forget about HIPAA is that the P stands for "Portability". The entire point behind it was that someone should be able to take their records from one hospital, doctor's office, or imaging clinic and then bring it to another one without having to go through another scan. DICOM was the industry's response to those portability requirements that our government put into place.
I suppose it could be false advertising to say you support a standard but only in a way that's meaningful to your software? As a 3rd party developer, it's hard to demand another company build something into their API, based on standards or otherwise. The best you can do is throw around your market share and lean on your mutual unhappy customers to make it happen.
This is one of those times I feel like the cell networks (or other network) should broadcast an accurate time without the need for an account/SIM and a cheap chip to receive it. I would say there's a lot of medical facilities that don't want to connect their equipment to the internet for time information and for good reason.
That'd depend on the hospital building out an RS485 infrastructure to support the device, though, which is a nontrivial ask.
Piggybacking off of a cell network is honestly probably the right solution. It doesn't have quite as high resolution as GPS, but that's hardly necessary for the sort of timestamps we're discussing -- even a resolution of a minute or two would be a huge improvement on "hope the tech who installed the equipment set the time correctly".
Remember that anything that is harder than setting the date and time is unlikely to be done in practice, since we're looking for something easier than that. Routing cabling out of a room seems unlikely.
Dicom gets a lot of flak, but as you say, the thing that makes it bad is shitty implementations and shitty vendors. After a while, you realize there aren't any nonshitty vendors in the medical field.
And then you count your blessings that at least dicom gives you the beginning of interop.
My software gig largely involved converting DICOM files into the format used by the company. I very very very much can relate with all this. Basically almost every client needed a custom script to correct the issues in their use of DICOM
Pretty much all of the medical instrument software I've ever seen is awful. I'd always imagined that that was because it was written by EEs. No less scary for that, though.
Be glad for DICOM. In digital pathology we don't even have that. Every manufacturer has their own format. We don't even have proper OS libraries, there is a LGPL library that hasn't been updated for 2 years and GPL one that's up to date but slow as molasses.
EDIT: yes, I meant OpenSlide and BioFormats. BioFormats is in unoptimized Java. Oh yeah, and DICOM for pathology exists and some software tools use it exclusively but none of the manufacturers do. As the only standard some organization press for standardizing on it - forcing all vendors to provide convertors. But those are often slow, buggy and produce huge files.
Also, I am sure you are aware the DICOM folks have been trying to get digital pathology pulled into the standard. I am not sure how much success they have had, but David Clunie (the current editor of the standard and a very friendly and open-minded person) had a funny set of power point slides about this topic. I recommend taking a look just for the humor:
FWIW, Innolitics has a better data dictionary (information object definitions) for DICOM than the official standards website. https://dicom.innolitics.com/ciods
DICOM has its issues, but it’s much better than HL7.
@korijn I'm glad you find our DICOM Standard Browser to be useful!
You (and others reading this post) may be interested to know that the JSON data the powers the browser is extracted using a set of open-sourced Python scripts:
Sometimes when it doesn't really matter what is chosen, one powerful entity can just pick and it doesn't matter. Even if maybe option B is very slightly better, clarity that we're all doing A is arguably better than some do A and some B and all suffer.
Unicode / ISO 10646-1 is an example like that. Some people don't like Han Unification, some people think UTF-16 reserved codepoints is an abomination, but mostly everybody is happy to live in a world where Unicode and ISO-10646 are effectively the same thing seen from different angles, not competing standards (Yes that could have happened)
Other times though there is no single entity powerful enough and willing to insist on one way forward, and no natural agreement on whether to do A or B. Instead parties that want to do A, and parties that want to do B just agree either is fine. The result is "standards" like USB Storage, TIFF or DICOM that leave far too much as unresolved choices.
Example: Some scanner manufacturers agreeing TIFF had designed scanners that start from the top-left of the eventual document, others started top-right or even bottom-right. If TIFF said "Images are top-to-bottom" the bottom-right scanners become expensive because they need a huge RAM buffer. Vice versa if TIFF says "Images are bottom-to-top". So... the actual TIFF standard says either can be true, you just flip a tag in the header to declare which way up the image is. Decoders have to carry all the burden of dealing with this, or be incompatible with some images to the confusion of users. ("Why is this picture upside down?")
These standards are marginally better than nobody agreeing anything, but only marginally. They're like that mediocre fusion restaurant you end up at when half the group wants Chinese and half wants Indian. Nobody likes this (hypothetical) fusion restaurant, so now nobody is happy, is that really better? I guess it's an improvement on spending the evening arguing pointlessly, because at least you can eat now.
I work with DICOM and HL7 and these things bring me to daydream about becoming a farmer and never looking back... Public healthcare IT is not where you want to be.
I think some of the frustration towards DICOM is misattributed. Yes, there are many instances of the standard being implemented poorly, and yes it is not always the most convenient having to index into your metadata using hex digits. But at least there IS a well documented, universally adopted standard that captures the most important information relevant to medical images right there next to the pixels. If this weren't the case, I think it would be much harder to have reliable interoperability between the highly complex + diverse types of systems that need to talk to each other in order to conduct an imaging workflow. There is a reason it exists and is so widely adopted -- and if you dig in to the details, a lot of thought has been put into its design. I highly recommend the book https://www.amazon.com/Digital-Imaging-Communications-Medici..., which believe it or not is a very entertaining read on the subject.
We're continuing to see hospitals and other healthcare-related organizations expose services that fall through the cracks due to the relative obscurity of these types of protocols in IT.
Hospitals tend to have piss-poor security. Their IT budget is usually low, they have one or two sysadmins who are eternally busy fixing things that users have messed up. Pentests are slowly becoming more accepted as a requirement, mostly because the GDPR has come into effect. And those inevitably turn up large lists of criticals.
Typically, gear is left with all the settings at 'default', things are opened up to the net for temporary access which is then never removed again, it's possible for the whole hospital to have only one LAN segment and so on. Ransomware, directed attacks to gain admin privileges and substantial violations of patient confidentiality have all been recorded with great regularity.
I'm assuming that if I have to go into the hospital again for whatever reason that that data will be on the street before I leave the building.
As standards come DICOM is actually pretty good. Sure, manufacturers will do their damnest to mess it up but interop tends to be surprisingly good because - just like with HTML - people tend to write for the lowest common denominator on reception. A typical PACS system will talk to a large number of machines all producing different output for different specialists to look at.
I've looked at about 10 companies using DICOM over the last couple of years and interop was typically the least of their worries. Think about the level of complexity here: anything from single frame BW at high res (say an X-ray) to time series of volumetric scans (for instance CAT).
The smarter companies will use one of the better FOSS libraries to implement the standard, the not-so-smart ones will try to roll their own, burn a ton of resources and will end up playing catch-up.
I used to work with DICOM about 7 years ago. It was a complete shitshow. Most of the important data was in "private fields". Imagine all websites using CSS, but they all use different vendor-specific attributes like "-moz-border-radius: 5". Might as well use a proprietary format at that point.
In the pre-DICOM days, I made money by making video capture cards. Back then, the only standard to get images from CT and MRI machines was the composite video on the 75 ohm coax going to the operator's console screen. Laser cameras (medical image film printers) worked the same way.
Actually that's not entirely true: there was also a 3M / Kodak defacto standard using a parallel RS-485 bus.
While this is undoubtedly one take on the subject, allow me to give you another perspective:
One of my friends had an MRI taken for suspected nasopharyngeal cancer. The physician that read the scan did not find cancer. Thankfully, his ENT physician did not believe the diagnosis and took another look at his MRI, which happened to be in DICOM format. He was able to load the images on a DICOM compliant image viewer and saw the tumor right away. My friend underwent surgery and was confirmed to have cancer by microscopy.
I think DICOM saved his life. It allowed his ENT physician to take another look, which would have probably been significantly harder with a proprietary file format.
If any of this were true you'd have tech giants, with their armies of "top" developers and massive legal teams disrupting the status quo but instead they announce a foray into healthcare, with much fanfare, only to scrap the project or have it fade into obscurity a few years later.
Turns out healthcare is hard, both the practice itself and the industrial/government bureaucracy. Probably why the better EHR shops pay just as well as top tech companies.
Having worked on a MVP for a healthcare company in Europe I can tell you that the users are not looking for better solutions. The doctors I worked with wanted a better pencil not some fancy solution. In our case it was improved traceability of medical implants and one doctor, when asked in private, told me that the current status quo suites them fine as it allows for more creative answers when the sh*t hits the fan. After that meeting I decided that the healthcare business was not the most healthy to work with.
Agreed, but this is slowly changing. Of the doctors/nurses I've worked with (hundreds at this point), I'd say only half of them know how to type. The younger docs though, they can be real power users. Europe is notable in how low tech they can be, which is why they've been the biggest new market in recent years for a lot of health IT companies. The idea of a hospital system doing all of their charting on paper is unheard of in the US, but there's still plenty of places doing that in Europe.
Healthcare in Europe is weird you end up with a lot of parallel systems, all healthcare data in Europe have to be digitized and there is a standard for digital EU healthcare records including mandated interoperability between all providers in various member states.
Even within the same healthcare system you can have drastic differences and a layer that sorts (or tries too) the entire mess out.
Oh, man. Who lied to you about healthcare compensation?
Just go check glassdoor.
Major IT companies bail on healthcare because margins suck and it's enormously regulated.
I give you, for instance, DICOM, Hl7, Meaningful Use, Infobutton, or any of the other half-useless dead-letter, best of the 80s-era mainframe technologies that are required to meet healthcare regulatory burden.
The best thing that's come out lately if FHIR, which is 9 years old and still doesn't qualify an EHR for meaningful use. Instead, it's all pipe-delimited TCP/MLLP and WTF.
I work in health IT and I've turned down offers from two big-n companies because they couldn't match my current compensation. I'm sure there's some no-name health system in the middle of nowhere paying $40k/year for dev work the same I'm sure there's some android fart app paying some developer peanuts as well. Be fair and compare the exceptions to the exceptions.
The main issue I've found is the manufacturer's implementation. They range from okay to barely meeting the minimum standards.
One thing that always annoyed me was the time stamps. They're mandated by the standard, so there's always a timestamp, but it's often useless. For example, it seems some scanners don't get their time set, over the network or otherwise, _ever_, so you might have a bunch of scans performed a few years ago on scanner A with an accurate timestamp of say 2017, but in a new scan from last week that was taken by scanner B, which hasn't had its clock configured properly, it reports a date closer to the start of the unix epoch rather than 2020.
Of course some of this is out of the manufacturers control, but temporal data is critical to analyzing changes to patient over time. I think this is one area where the DICOM standard couldn't be a bit more strict with a lot of positive impact.