This is worth a periodic re-share:

"Command-line Tools can be 235x Faster than your Hadoop Cluster"

adamdrake.com/command-line-too

Because sometimes the old ways are good ways. ๐Ÿง“

ยท Web ยท 3 ยท 45 ยท 42

@HerraBRE Here's another story along those lines: andre.arko.net/2018/10/25/pars

Goes from a Ruby script, to a $1000/month hi-tech AWS/Python/Hadoop solution, to a boring Rust program with CPU usage low enough to run for free on Lambda.

@alva @HerraBRE I upgraded Logstash like three versions and am still concerned that I'm not capturing all the logs because the CPU is like 1/5th of what it was.

@HerraBRE This comparison is like saying that a paper and pen are better than programming because you can write โ€œhello worldโ€ with paper and pen faster than writing and executing a program to do it. Hadoop is meant to get more performant as the data increases and is almost explicitly not meant to be used with such tiny amounts of data as this.

Sign in to participate in the conversation
Mastodon

Generalistic and moderated instance. All opinions are welcome, but hate speeches are prohibited. Users who don't respect rules will be silenced or suspended, depending on the violation severity.