Member Avatar for sparkle85

Hello,

I need some help with the following script:

if ! [ -f ${APATH}/myfile.txt ];then
  echo $(date +%Y%m%d"_"%H%M%S)": Nu am gasit fisierul ${APATH}/myfile.txt"
  ps -fxu pin | grep "/usr/local/coreutils/bin/tail -f ${LOG_PATH}/x.log$" | awk '{system("kill "$2)}'
  cat ${LOG_PATH}/x.log | sed -n -e '/LONG/{x;1!p;g;;}' -e h > ${APATH}/myfile.txt
  
  /usr/local/coreutils/bin/tail -f ${LOG_PATH}/x.log | sed -n -e '/LONG/{x;1!p;g;;}' -e h >> ${APATH}/myfile.txt &
fi
date1=$(tail -1 ${PATH}/myfile.txt | cut -d" " -f2-5)
if [ -f exec_long.date ];then
  date2=$(cat exec_long.date)
else
  date2=$date1
  echo $date1 > exec_long.date
fi
if ! [ "$date1" = "$date2" ];then
        ${CAL_PATH}/${APP_NAME}
        sleep 1
        echo $date1 > exec_long.date
fi

I have a continuously growing log file (x.log) in which i have to look for certain lines that contain "Long". The line above each line containing the word "Long" it contains a time stamp. I want to extract each line containing the time stamp into myfile.txt and check the difference between time stamps. Whenever there is a difference i need to run another script (${CAL_PATH}/${APP_NAME}), then sleep 1, then continue searching.

Lines with "Long" do not appear continuously, but in blasts. The script runs fine until the first pause encountered. Starting with the first pause, tail -f doesn't write in myfile.txt anymore.

Can someone help me understand why "tail -f" it stops writing into myfile.txt? Or does someone know an alternative to "tail -f" of achieving the initial scope of the script?

I'll appreciate any help.
Thanks!

  • 3 Contributors
  • 2 Replies
  • 212 Views
  • 3 Weeks Discussion Span
  • Latest Post Latest Post by JeoSaurus

Dani AI

Generated

Two things are most likely happening here: the file is being rotated so your long-running tail -f keeps following the old inode (as suggested), or the pipeline is being buffered so new lines are not flushed through the filters after a burst. 's tail tip can be corrected to the portable form tail -n 1000 -f if you want the last N lines before following.

Quick checks

  • Confirm rotation: compare the file inode before/after a burst (ls -li or stat -c %i /path/x.log). If the inode changes, rotation moved the file.
  • Check the running tail process (pgrep -a tail / ps) and which file descriptor it has open (ls -l /proc/<pid>/fd).

Simple, robust fixes

  • Use tail that follows the name (auto-switch on rotation): tail -F (same as --follow=name --retry). See the tail manual for details.
  • Avoid pipeline buffering by forcing line buffering: use stdbuf -oL (or use gawk/perl with explicit flush). A compact, practical one-liner that appends the timestamp line preceding each "LONG" to your file is below — it handles rotation and minimizes buffering (adjust paths and awk to use gawk if you need fflush):
stdbuf -oL tail -F /path/x.log | awk '/LONG/ { print prev >> "/path/myfile.txt"; fflush("/path/myfile.txt") } { prev=$0 }' &

Alternative approaches and cautions

  • If you control rotation, copytruncate in logrotate keeps the same inode (no tail restart) but can lose a small window of lines during rotation. See the logrotate docs.
  • For production, prefer a supervised service (systemd / supervisor) or use inotifywait to trigger processing on file modification rather than a persistent tail, to avoid orphaned background processes.

Recommended Answers

All 2 Replies

Member Avatar for vaibhav1983

you can do a

tail-1000f <your-log-file>

This command will show the latest 1000 lines from your log file...also if you log file is updated, it will only show the last 1000 lines...

Member Avatar for JeoSaurus

>> Can someone help me understand why "tail -f" it stops writing into myfile.txt

Does the log file get rotated at some point? If so, tail is still referencing the OLD version, even though it's been renamed.

I've done something similar using tail -f, but also running a monitoring script that checks the log file to see if it's been rotated (I check logfile.txt every so often to see if the inode has changed)

If the file gets rotated, the monitor script kills the tail script and re-starts it.

As far as alternatives, I"m sure there is one, but I can't think of one at the moment using bash...

I hope this helps!

-G

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.