bash - tail -f into grep into cut not working properly

Question

i'm trying to build a shell script to monitor some log files. I'm using a command like this:

tail -f /var/somelog | grep --line-buffered " some test and p l a c e h o l d e r" | cut -f 3,4,14 -d " "

the log file is like:

some test and p l a c e h o l d e r 3
some test and p l a c e h o l d e r 4
some test and p l a c e h o l d e r 5
some test and p l a c e h o l d e r 6

and so on.. My issue is that the output of the command does not display the last line

some test and p l a c e h o l d e r 6

until line

some test and p l a c e h o l d e r 7

is added to the log.

I hope I made clear my issue. Can anyone help me to solve this? Thank you :)

Answer 1

the problem is almost certainly related to how grep and cut buffer their output. here's a hack that should get you around the problem, though i'm sure there are prettier ways to do it:

tail -f /var/somelog | while read line; do echo "$line" | grep "some test and p l a c e h o l d e r" | cut -f 3,4,14 -d " "; done

(don't forget the ; done at the end of the command)

alternatively, because gawk doesn't buffer it's output, you could use it in place of cut to avoid the cumbersome while loop:

tail -f log | grep --line-buffered "some test and p l a c e h o l d e r" | gawk '{print $3,$4,$14}'

check out http://www.pixelbeat.org/programming/stdio_buffering/ for more info on buffering problems.