Hacker Newsnew | past | comments | ask | show | jobs | submit | theophrastus's commentslogin

Had a QuantSci Prof who was fond of asking "Who can name a data collection scenario where the x data has no error?" and then taught Deming regression as a generally preferred analysis [1]

[1] https://en.wikipedia.org/wiki/Deming_regression


Most of the time, if you have a sensor that you sample at, say 1 KHz and you’re using a reliable MCU and clock, the noise terms in the sensor will vastly dominate the jitter of sampling.

So for a lot of sensor data, the error in the Y coordinate is orders of magnitude higher than the error in the X coordinate and you can essentially neglect X errors.


That is actually the case in most fields outside of maybe clinical chemistry and such, where Deming became famous for explaining it (despite not even inventing the method). Ordinary least squares originated in astronomy, where people tried to predict movement of celestial objects. Timing a planet's position was never an issue (in fact time is defined by celestian position), but getting the actual position of a planet was.

Total least squares regression also is highly non-trivial because you usually don't measure the same dimension on both axes. So you can't just add up errors, because the fit will be dependent on the scale you chose. Deming skirts around this problem by using the ratio of variances of errors (division also works for different units), but that is rarely known well. Deming works best when the measurement method for both dependent and independent variable is the same (for example when you regress serum levels against one another), meaning the ratio is simply one. Which of course implies that they have the same unit. So you don't run into the scale-invariance issues, which you would in most natural science fields.


From that wikipedia article, delta is the ratio of y variance to x variance. If x variance is tiny compared to y variance (often the case in practice) then will we not get an ill-conditioned model due to the large delta?


If you take the limit of delta -> infinity then you will get beta_1 = s_xy / s_xx which is the OLS estimator.

In the wiki page, factor out delta^2 from the sqrt and take delta to infinity and you will get a finite value. Apologies for not detailing the proof here, it's not so easy to type math...


In my field, the X data error (measurement jitter) is generally <10ns, which might as well be no error.


For most time series, noise in time measurement is negligible. However, this does not prevent complex coupling phenomena from occurring for other parameters, such as GPS coordinates.


The issue in that case is that OLS is BLUE, the best linear unbiased estimator (best in the sense of minimum variance). This property is what makes OLS exceptional.


Merry Christmas!! you gentle nerds you! Peace to us all.


What was the source of the oxygen to maintain ethyl alcohol combustion in a sealed WWII torpedo?



Curiously the default has audio output off. That is, was the little speaker icon unmuted?


Here's from 2014 with some capacity calculations [0]. "ssh private key (900 bytes): 15 feet of tape."

[0] https://heepy.net/index.php/Data_storage_capacity_of_teletyp...


If we prefer paper but relax the teletype constraint, Oleh Yuschuk's PaperBack [0] allows encoding 500 KB on a sheet of printer paper.

[0] https://ollydbg.de/Paperbak (posted various times to HN)


Circa 1980, as a hobbyist beekeeper with six hives nearby Seattle, they came down with foul-brood. (It never was certain if it was American Foul-brood or European) Duly reported and the county agent came in, sealed them, and carted them away. For an additional fee (which I paid) they would fumigate them and return just the hive bodies, but none of the frames, (some of which would contain the infected brood). Those were burned. I believe the fumigant at the time was phosphine[1]

[1] https://en.wikipedia.org/wiki/Phosphine


Shouldn't this make some initial direct reference to the author: Silvanus P. Thompson[1]?

[1] https://en.wikipedia.org/wiki/Calculus_Made_Easy


https://calculusmadeeasy.org/

The link isn’t to the front page with his name.


Ohh, this really is from 1910. And here I just thought the author was being obnoxiously cutesy with their language.


Additional small molecule pharmaceutical candidates via molecular descriptors


Well done: 3d is an option! Always wondered what emergent properties result from simple rules worlds when the dimensionality goes from 2d to 3d.


Lately in advertisements there have been a lot of circular QR codes[1]. Which often apparently present data outside of the alignment patterns. Is that merely an artistic device, or is there some standard permitting information extending beyond the square region?

[1] https://www.google.com/search?tbm=isch&q=circular%20qr%20cod...


I think most of them are merely artistic, probably all data outside of the alignment patterns is just random stuff. It looks good though. [0]

Some of them however are implemented to be read by a specific non-standard reader, something such as the snap chat codes.

[0]: https://stackoverflow.com/questions/66837985/damaging-qr-cod...


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: