Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
598 views
in Technique[技术] by (71.8m points)

bash - Why does reading and writing to the same file in a pipeline produce unreliable results?

I have a bunch a files that contain many blank lines, and want to remove any repeated blank lines to make reading the files easier. I wrote the following script:

#!/bin/bash
for file in * ; do cat "$file" | sed 's/^ +//' | cat -s > "$file" ; done

However, this had very unreliable results, with most files becoming completely empty and only a few files having the intended results. What's more, the files that did work seemed to change randomly every time I retried, as different files would get correctly edited in every run. What's going on?

Note: This is more of a theoretical question, because I realize I could use a workaround like:

#!/bin/bash
for file in * ; do 
    cat "$file" | sed 's/^ +//' | cat -s > "$file"-tmp
    rm "$file"
    mv "$file"-tmp "$file"
done

But that seems unnecessarily convoluted. So why is the "direct" method so unreliable?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

The unpredictability happens because there's a race condition between two stages in the pipeline, cat "$file" and cat -s > "$file".

The first tries to open the file and read from it, while the other tries to empty the file.

  • If it's emptied before it's read, you get an empty file.
  • If it's read before it's emptied, you get some data (but the file is emptied shortly after and the result is truncated unless it's very short).

If you have GNU sed, you can simply do sed -i 'expression' *


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...