> but even still, that adds only another layer of testing to ensure that your techniques for converting the raw data to structured data are sound enough to be reproduced.
That's actually not correct.
What if your validation method is broken? What if that unit test you wrote has a bug in the test itself?
In science you have to keep the raw data available.
This is not only for reproduction of your particular structured metric (i.e. tail flick angle), but for other groups to design novel metrics that may add to or trivialize the published result.
>In science you have to keep the raw data available.
There is nothing I'm aware of saying that you can't losslessly convert tiff to png or even tiff to tiff.gz. Nobody needs to collect pcap files for their sensors (the raw data). They just collect the data and stick it in an appropriate file.
Most bio-peeps and medical personnel have an understanding that there is a difference between .png, .jpg, and .tif. But the diff of .tif and .tiff? No idea, let alone the +/- of a .png to a .jpg. It's all just a picture to them.
So, trying to tell them that you can losslessly convert them, though maybe true, is not a good idea. .tiff can handle a stack of images and you only have to call it once in a MatLab script, .jpg cannot and will result in a shitfit as they try to load in an entire folder of images. Then you get into bit-depth and God help you if the conversation ever has you trying to pronounce 'int', 'float', 'str', or 'double'. The class of a variable? Dude, this person could care less and will get out a protractor and measure tail flicks off a print-out before any of that will ever sink in.
I think you underestimate people. If you're an admin doing user support for such a system and someone says they need more storage because of blah blah blah then you offer ways to alter the workflow so they don't need 500G of tiff files. It might even speed up their workflow since it's not streaming 500G off disk.
No need to get into bit depth and int float str and double (though all of the biomedial researchers I've worked with knew these very well). Just: 'you can convert to png and back again without losing data. Just like zipping a file and unzipping it again. Let me show you how...'
That's actually not correct. What if your validation method is broken? What if that unit test you wrote has a bug in the test itself?
In science you have to keep the raw data available.
This is not only for reproduction of your particular structured metric (i.e. tail flick angle), but for other groups to design novel metrics that may add to or trivialize the published result.