Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Parent poster used the 5% value, so I did as well. But yes, to answer your question. Shown here on a Debian 11 system that's fairly old:

  $ hyperfine "export LC_ALL=C; cut -d' ' -f1 fake_log.txt | sort -S5% | uniq -c | sort -rn -S5% | head"
  Benchmark 1: export LC_ALL=C; cut -d' ' -f1 fake_log.txt | sort -S5% | uniq -c | sort -rn -S5% | head
  Time (mean ± σ):      1.504 s ±  0.318 s    [User: 2.833 s, System: 0.474 s]
  Range (min … max):    0.942 s …  1.937 s    10 runs
 
  $ hyperfine "export LC_ALL=C; cut -d' ' -f1 fake_log.txt | sort | uniq -c | sort -rn | head"
  Benchmark 1: export LC_ALL=C; cut -d' ' -f1 fake_log.txt | sort | uniq -c | sort -rn | head
  Time (mean ± σ):      3.847 s ±  0.093 s    [User: 4.165 s, System: 0.613 s]
  Range (min … max):    3.591 s …  3.919 s    10 runs
Setting the buffer value to ~half the file size (100 MB) resulted in a mean time of 2.291 seconds. Setting it to the size of the file resulted in a mean time of 1.549 seconds, which is close enough to the 5% to call it equal - not like this server isn't busy with other stuff, so it's hardly a good place for perfect benchmarking.

Syscalls aren't free, nor are disk reads, so if you have the RAM to support slurping the entire file at once, it's sometimes faster.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: