While I fully agree with you that the practice of curling a script and piping it into `sh` is—to say the least—risky, notice that this risk has been widely accepted long time ago. Each time you download an executable file—be it an exe for Windows, apk for Android, a Linux binary, an OS X executable—you're doing the same thing. I'll go one step further: each time you download a free/open source tarball you do not read the code before typing `make`. You make your machine run some code of unknown functionality and only plausible origin.
Arguably, HTTPS is one step forward, however vulnerabilities like the one discussed here make us defenceless. To make matters worse the line of defence based on reading the script works only in the case of relatively short, unobfuscated and unminified scripts written in plain text. It also requires the person who's downloading to have skills which despite being common for this community's audience are not widely spread across the population.
Sure, many projects sign their releases or announce cryptographic hashes of published files. But let's be honest: how many of us actually run `gpg` od `sha256sum -c` to verify them?
Spreading paranoia is not my goal here, however I hope that this comment will end up being thought-provoking.
I think the point he was making is moreso that these rails-centric sites are going to get nailed &, as a result, one should be more wary during the next few weeks using this sort of installation method for stuff.
O should be generally quite wary of it in the first place given the ease one could swap out a single file & wreck havoc.
At the very beginning of the presentation he underlines the importance of the static compiler RubyMotion is based on, yet at 7:30 when he compiles a file with a name error in it the compiler doesn't even file a warning. Such non-existent methods and other similar errors should be caught at compile time if we're talking about solid AOT compilation.
I've been running various flavours of GNU/Linux on my laptop for a couple of years now, with Arch being the most recent one. I find it perfectly adequate for an IDE-less, Vim-centric coding which takes a significant majority of my time spent in front of a keyboard.
I wonder what are experiences of members of the HN community who run Linux on their MacBooks, especially in case of a development-oriented environment. Could you share your opinions about such a setup? Is it worth the price of the machine?
Firsty, comment sections are usually full of noise and it's next to impossible to distill any valuable information out of it. Secondly, don't expect an average person to bother checking for potential malware.
I couldn't sacrifice performance (lots of number crunching was involved with costs tight), so, I couldn't pick Python.
PyPy, Cython or Shed Skin might have been answers to those number crunching problems. I'd love to learn whether they would be sufficient for OP's performance requirements. Some benchmarks look promising, for instance this one published in late 2010: http://geetduggal.wordpress.com/2010/11/25/speed-up-your-pyt...
During the previous semester I spent some time building a recommender using this data as a project for a data mining class. It turned out to be far more challenging than I had initially anticipated.
I've used methods known as collaborative filtration, whose goal was to estimate how a given user would rate a given item basing on knowledge of preferences of other users of similar interests. The initial scope included a naïve Bayesian classifier and a technique called Slope One [1]. The latter one is particularly interesting as according to claims of its authors allows to make a very good estimation in a very short time using solely a very simple linear model. The preprocessing is both time- and space-wise expensive though as it requires you to build a matrix of deviations between rated items.
After reducing the data set to a single subreddit and filtering it from users who weren't avid voters I ran the algorithms and after some tuning I was very content to see promising ROC curves and decent AUC values. Models built around NBC and S1 achieved comparable results when it came to such metrics as precision, recall and F-measure.
When I went to discuss the results with the professor teaching the class I've heard "That's indeed promising, but how about comparing those results with a really naïve model which would just take an average of existing votes by a given user?". Guess what: the model built solely using a single call to the avg function was nearly as good as the NBC and S1 models.
Now I understand why the guys from Reddit are looking for external help with the recommender. It's a way less obvious task than it might seem to be.
Out of curiosity, did you compare to any other baselines? I suspect you did a lot better than you think you did, because that particular baseline is actually very misleading for ranking/recommendation tasks (this is a common source of confusion for newcomers). Here's why, in two parts:
1) Say you estimate (as you propose) that a user will always give their average rating. This might get you good-ish error and ROC as a prediction task, but will give zero recommendation value because the prediction for a given user will be constant for all possible recommendations.
2) Say you estimate that a user will give the average score that the item has received across all users. Again, possibly good-ish in terms of prediction ROC and RMS error, but this offers no personalization (all users get the same predictions, i.e. you're basically just showing the default Reddit ranking).
Both of these baselines are vastly inferior to even really stupid models like "how many times have I upvoted stories from this submitter" in terms of recommendation value, but the latter is (if I recall from my own experiments) much worse when evaluated on the basis of overall ROC.
I would strongly suspect that a correctly implemented NB or S1 would vastly outperform either of the two baselines in terms of actual recommendation utility (even though when you look at the baseline's ability to predict actual numbers, they might be comparably good in an RMS sense).
The moral of the story: one must be very careful when trying to quantify the performance of learning systems; actual utility is often difficult to evaluate merely by looking at standard statistical measures of accuracy.
No, I didn't make any comparisons to other baselines. Thanks a lot for sharing your thoughts; I'll have to reconsider the results I got in the light of your comment.
I implemented Slope One for the netflix prize and found its results pretty unimpressive. So I decided to extend it to build a SVD predictor of Slope One values figuring it might do better than SVD by itself. It didn't.
Turns out increasing the dimensionality of the input 17 thousand times just reduces the amount training data for each attribute. Duh :)
