shell - Sort text file after the 16th column with p-values in bash

Question

I have a tab delimited textfile with 18 column and more than 300000 rows. I have also a header line and I would sort the whole text file by the 16th column, which contains p-values. So I would like to sort it, having the lowest p-values above and also leaving the headline as it is. I already have a code, it doesn't give me any error message, but it only shows the header line in the output file, and nothing else.

Here is my file:

filename    CHROM   ID  x11_CT  x12_CT  CT1 CT2 SampleSize  x21_CT  x21 x22_CT  x22 x11 x12 chIGSFA P_value GZD ZGSR
V1003   1   rs3131972   212 1   1068    14  541 856 0.791127541589649   13  0.0120147874306839  0.195933456561922   0.000924214417744917    0.70567673346914    0.400882778478405   0.00649170940375354 0.0361163844076152
V1003   1   rs3131962   170 1   1066    14  540 896 0.82962962962963    13  0.012037037037037   0.157407407407407   0.000925925925925926    0.40191966550969    0.526099523335894   0.00450617283950613 0.027281782875571
V1003   1   rs12562034  128 0   1068    14  541 940 0.868761552680222   14  0.0129390018484288  0.118299445471349   0   0.951515008754774   0.329333964471109   0.00612270697448755 0.041938142300103 
V1003   1   rs12131377  78  0   1060    14  537 982 0.914338919925512   14  0.0130353817504655  0.0726256983240224  0   0.555433052966582   0.456106209942983   0.0037868148101911  0.0321609387794883

Output should look like this:

filename    CHROM   ID  x11_CT  x12_CT  CT1 CT2 SampleSize  x21_CT  x21 x22_CT  x22 x11 x12 chIGSFA P_value GZD ZGSR
V1003   1   rs12562034  128 0   1068    14  541 940 0.868761552680222   14  0.0129390018484288  0.118299445471349   0   0.951515008754774   0.329333964471109   0.00612270697448755 0.041938142300103 
V1003   1   rs3131972   212 1   1068    14  541 856 0.791127541589649   13  0.0120147874306839  0.195933456561922   0.000924214417744917    0.70567673346914    0.400882778478405   0.00649170940375354 0.0361163844076152
V1003   1   rs12131377  78  0   1060    14  537 982 0.914338919925512   14  0.0130353817504655  0.0726256983240224  0   0.555433052966582   0.456106209942983   0.0037868148101911  0.0321609387794883
V1003   1   rs3131962   170 1   1066    14  540 896 0.82962962962963    13  0.012037037037037   0.157407407407407   0.000925925925925926    0.40191966550969    0.526099523335894   0.00450617283950613 0.027281782875571

Here is my code:

awk 'NR==1; NR > 1 {print $0 | "sort -g -rk 16,16"}' file.txt > file_out.txt

question from:https://stackoverflow.com/questions/65951805/sort-text-file-after-the-16th-column-with-p-values-in-bash

Answer 1

I'm guessing your sort doesn't have a -g option and so it's failing and not producing any output. Try this instead just using POSIX options:

$ awk 'NR==1; NR > 1 {print | "sort -nrk 16,16"}' file
filename        CHROM   ID      x11_CT  x12_CT  CT1     CT2     SampleSize      x21_CT  x21     x22_CT  x22     x11     x12     chIGSFA P_value     GZD     ZGSR
V1003   1       rs3131962       170     1       1066    14      540     896     0.82962962962963        13      0.012037037037037       0.157407407407407   0.000925925925925926    0.40191966550969        0.526099523335894       0.00450617283950613     0.027281782875571
V1003   1       rs12131377      78      0       1060    14      537     982     0.914338919925512       14      0.0130353817504655      0.0726256983240224  0       0.555433052966582       0.456106209942983       0.0037868148101911      0.0321609387794883
V1003   1       rs3131972       212     1       1068    14      541     856     0.791127541589649       13      0.0120147874306839      0.195933456561922   0.000924214417744917    0.70567673346914        0.400882778478405       0.00649170940375354     0.0361163844076152
V1003   1       rs12562034      128     0       1068    14      541     940     0.868761552680222       14      0.0129390018484288      0.118299445471349   0       0.951515008754774       0.329333964471109       0.00612270697448755     0.041938142300103