Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
180 views
in Technique[技术] by (71.8m points)

c++ - Read deadlock when using pipes to execute multiple shell commands on the same process

I am making a C++ program that needs to run multiple commands in the same bash shell instance. I need this because some of the commands are setting a bash variable that needs to be read by a subsequent command.

I am using pipes to make file descriptors that are then read and written to using read and write, the other ends of these pipes are connected to a child that is made using fork.

A problem arises when a command doesn't return an output, such as setting a bash variable. In the following code, the read will hang forever on command number 2. I've been searching around for a couple days and there doesn't seem to be a way to detect when the command has finished running without closing the pipe somewhere. I believe that if I close the pipe, I won't be able to reopen it, and that would mean I'd need to make a new bash shell that doesn't have the variables loaded.

Additionally, I can't be sure which commands won't return an output as this code will get the commands it needs to run from a web server and would like to avoid concatenating commands with '&&' for granular error reporting.

#include <unistd.h>
#include <fcntl.h>
#include <cstdlib>
#include <string>
#include <iostream>

using namespace std;

int main(int argc, char *argv[])
{
    int inPipeFD[2];
    int outPipeFD[2];

    // Create a read and write pipe for communication with the child process
    pipe(inPipeFD);
    pipe(outPipeFD);

    // Set the read pipe to be blocking
    fcntl(inPipeFD[0], F_SETFL, fcntl(inPipeFD[0], F_GETFL) & ~O_NONBLOCK);
    fcntl(inPipeFD[1], F_SETFL, fcntl(inPipeFD[1], F_GETFL) & ~O_NONBLOCK);

    // Create a child to run the job commands in
    int pid = fork();

    if(pid == 0) // Child
    {
        // Close STDIN and replace it with outPipeFD read end
        dup2(outPipeFD[0], STDIN_FILENO);

        // Close STDOUT and replace it with inPipe read end
        dup2(inPipeFD[1], STDOUT_FILENO);

        system("/bin/bash");
    }
    else // Parent
    {
        // Close the read end of the write pipe
        close(outPipeFD[0]);

        // Close the write end of the read pipe
        close(inPipeFD[1]);
    }

    // Command 1
    char buf[256];
    string command = "echo test
";
    write(outPipeFD[1], command.c_str(), command.length());
    read(inPipeFD[0], buf, sizeof(buf));
    cout << buf << endl;

    // Command 2
    char buf2[256];
    command = "var=worked
";
    write(outPipeFD[1], command.c_str(), command.length());
    read(inPipeFD[0], buf2, sizeof(buf2));
    cout << buf2 << endl;

    // Command 3
    char buf3[256];
    command = "echo $var
";
    write(outPipeFD[1], command.c_str(), command.length());
    read(inPipeFD[0], buf3, sizeof(buf3));
    cout << buf3 << endl;
}

Is there a way to detect that a child's command has finished without having to close the pipe?

question from:https://stackoverflow.com/questions/65847624/read-deadlock-when-using-pipes-to-execute-multiple-shell-commands-on-the-same-pr

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

One solution could be to set bash in interactive mode by starting it with system("/bin/bash -i"); and to set the prompt to the exit code of the last command.

First, a convenience function to make writing and reading simpler:

std::string command(int write_fd, int read_fd, std::string cmd) {
    write(write_fd, cmd.c_str(), cmd.size());
    cmd.resize(1024); // turn cmd into a buffer
    auto len = read(read_fd, cmd.data(), cmd.size());
    if(len == -1) len = 0;
    cmd.resize(static_cast<std::size_t>(len));
    return cmd;
}

Then in your parent process:

    sleep(1); // ugly way to make reasonably sure the child has started bash
    int& out = outPipeFD[1]; // for convenience
    int& in = inPipeFD[0];   // for convenience

    // first, set the prompt
    std::cout << command(out, in, "export PS1='$?\n'
") << '
';

    // then all these will print something
    std::cout << command(out, in, "echo test
") << '
';
    std::cout << command(out, in, "var=worked
") << '
';
    std::cout << command(out, in, "echo $var
") << '
';

This way you will always have something to read - and you can also use it to verify that the command executed correctly.


If your bash requires a real terminal in -i(nteractive) mode, we have to do it without it. The idea:

  • Add echo $? + a delimiter to every command sent
  • Set the pipes in non-blocking mode to be able to catch misc. bad situations, like if the command exit is sent.
  • Read until the delimiter is found or an error occurs.

To make the delimiter something hard to guess (to not be able to easily force reading to get out of sync) I would generate a new delimiter for every command.

Here is an example of what it could look like with those ideas in place with inline comments:

#include <fcntl.h>
#include <unistd.h>
#include <sys/select.h>

#include <algorithm>
#include <cstdlib>
#include <iostream>
#include <random>
#include <sstream>
#include <utility>
#include <vector>

// a function to generate a random string to use as a delimiter
std::string generate_delimiter() {
    thread_local std::mt19937 prng(std::random_device{}());
    thread_local std::uniform_int_distribution dist('a', 'z');
    thread_local auto gen = [&]() { return dist(prng); };

    std::string delimiter(128, 0);
    std::generate(delimiter.begin(), delimiter.end(), gen);

    return delimiter;
}

// custom exit codes for the command function
enum exit_status_t {
    ES_WRITE_FAILED = 256,
    ES_READ_FAILED,
    ES_EXIT_STATUS_NOT_FOUND
};

// a function executing a command and returning the output and exit code
std::pair<std::vector<std::string>, exit_status_t> command(int write_fd,
                                                           int read_fd,
                                                           std::string cmd) {
    constexpr size_t BufSize = 1024;

    // a string that is unlikely to show up in the output:
    const std::string delim = generate_delimiter() + "
";

    cmd += "
echo -e $?"\n"" + delim; // add echoing of status code
    auto len = write(write_fd, cmd.c_str(), cmd.size()); // send the commands
    if(len <= 0) return {{}, ES_WRITE_FAILED};           // couldn't write, return

    cmd.resize(0); // use cmd to collect all read data
    std::string buffer(BufSize, 0);

    // a loop to extract all data until the delimiter is found
    fd_set read_set{};
    FD_SET(read_fd, &read_set);
    while(true) {
        // wait until something happens on the pipe
        select(read_fd + 1, &read_set, nullptr, nullptr, nullptr);

        if((len = read(read_fd, buffer.data(), buffer.size())) <= 0) {
            // Failed reading - pipe probably closed on the other side.
            // Add a custom exit code and the delimiter and break out.
            cmd += "
" + std::to_string(ES_READ_FAILED) + "
" + delim;
            break;
        }

        // append what was read to cmd
        cmd.append(buffer.begin(), buffer.begin() + len);

        // break out of the loop if we got the delimiter
        if(cmd.size() >= delim.size() &&
           cmd.substr(cmd.size() - delim.size()) == delim)
        {
            break;
        }
    }

    cmd.resize(cmd.size() - delim.size()); // remove the delimiter

    // put what was read in an istringstream for parsing
    std::istringstream is(cmd);

    // extract line by line
    std::vector<std::string> output;
    while(std::getline(is, cmd)) {
        output.push_back(cmd);
    }

    // extract the exit code at the last line
    exit_status_t retval = ES_EXIT_STATUS_NOT_FOUND;
    if(not output.empty()) { // should never be empty but ...
        retval = static_cast<exit_status_t>(std::stoi(output.back(), nullptr));
        output.resize(output.size() - 1);
    }

    return {output, retval}; // return the pair
}

Test driver:

int main() {
    int inPipeFD[2];
    int outPipeFD[2];

    // Create a read and write pipe for communication with the child process
    pipe(inPipeFD);
    pipe(outPipeFD);

    // Set the read pipe to be non-blocking
    fcntl(inPipeFD[0], F_SETFL, fcntl(inPipeFD[0], F_GETFL) | O_NONBLOCK);
    fcntl(inPipeFD[1], F_SETFL, fcntl(inPipeFD[1], F_GETFL) | O_NONBLOCK);

    // Create a child to run the job commands in
    int pid = fork();

    if(pid == 0) // Child
    {
        // Close STDIN and replace it with outPipeFD read end
        dup2(outPipeFD[0], STDIN_FILENO);
        close(outPipeFD[0]); // not needed anymore

        // Close STDOUT and replace it with inPipe read end
        dup2(inPipeFD[1], STDOUT_FILENO);
        close(inPipeFD[1]); // not needed anymore

        // execl() is cleaner than system() since it replaces the process
        // completely. Use /bin/sh instead if you'd like.
        execl("/bin/bash", "bash", nullptr);
        return 1; // to not run the parent code in case execl fails
    }
    // Parent

    // Close the read end of the write pipe
    close(outPipeFD[0]);

    // Close the write end of the read pipe
    close(inPipeFD[1]);

    sleep(1);
    int& out = outPipeFD[1]; // for convenience
    int& in = inPipeFD[0];   // for convenience

    // a list of commands, including an erroneous command(foobar) + exit
    for(std::string cmd : {"echo test", "var=worked", "echo $var", "foobar", "exit"}) 
    {
        std::cout << "EXECUTING COMMAND: " << cmd << '
';
        auto [output, exit_status] = command(out, in, cmd);
        // print what was returned
        for(auto str : output) std::cout << str << '
';
        std::cout << "(exit status=" << exit_status << ")
";
    }
}

Possible output:

EXECUTING COMMAND: echo test
test
(exit status=0)
EXECUTING COMMAND: var=worked
(exit status=0)
EXECUTING COMMAND: echo $var
worked
(exit status=0)
EXECUTING COMMAND: foobar
bash: line 7: foobar: command not found
(exit status=127)
EXECUTING COMMAND: exit

(exit status=257)

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...