c++ - CreateProcess with new console window, but override some std i/o handles

Question

If you use CreateProcess with the flag CREATE_NEW_CONSOLE, the new process has its standard input, output, and error handles directed to the new console window. If you want to override the I/O streams, you can do so by setting the handles in STARTUPINFO fields hStdOutput, hStdInput, and hStdError and setting the flag STARTF_USESTDHANDLES.

But what if you want to override only one of the handles? For example, I might want to redirect stderr to a file while leaving the stdout and stdin connected to the new console window.

The STARTF_USESTDHANDLES flag tells CreateProcess to replace all of the handles instead of connecting them to the ones for the new console window. So it seems we must provide all three handles. Obviously I can set hStdError to the handle of the log file, but what values should be used for hStdInput and hStdOutput?

I tried using NULL, which seems to work on Windows 8.1, but it doesn't work on Windows 7.

I also thought about creating a console window first, and then calling CreateProcess with handles to the new console window's buffers (and omitting the CREATE_NEW_CONSOLE flag). Unfortunately, the parent process is also a console application, and it seems a console application cannot create a second console window.

Answer 1

According to this MSDN Support article:

How to spawn console processes with redirected standard handles

If the parent process only wishes to redirect one or two standard handles, specifying GetStdHandle() for the specific handles causes the child to create the standard handle as it normally would without redirection. For example, if the parent process only needs to redirect the standard output and error of the child process, then the hStdInput member of the STARTUPINFO structure is filled as follows:

hStdInput = GetStdHandle(STD_INPUT_HANDLE);

According to the GetStdHandle() documentation:

STD_INPUT_HANDLE
(DWORD)-10
The standard input device. Initially, this is the console input buffer, CONIN$.

STD_OUTPUT_HANDLE
(DWORD)-11
The standard output device. Initially, this is the active console screen buffer, CONOUT$.

STD_ERROR_HANDLE
(DWORD)-12
The standard error device. Initially, this is the active console screen buffer, CONOUT$.

...

The standard handles of a process may be redirected by a call to SetStdHandle, in which case GetStdHandle returns the redirected handle. If the standard handles have been redirected, you can specify the CONIN$ value in a call to the CreateFile function to get a handle to a console's input buffer. Similarly, you can specify the CONOUT$ value to get a handle to a console's active screen buffer.

Attach/detach behavior

When attaching to a new console, standard handles are always replaced with console handles unless STARTF_USESTDHANDLES was specified during process creation.

If the existing value of the standard handle is NULL, or the existing value of the standard handle looks like a console pseudohandle, the handle is replaced with a console handle.

When a parent uses both CREATE_NEW_CONSOLE and STARTF_USESTDHANDLES to create a console process, standard handles will not be replaced unless the existing value of the standard handle is NULL or a console pseudohandle.

So, if the parent process's STDIN has NOT been redirected, GetStdHandle(STD_INPUT_HANDLE) will return either NULL or a pseudo-handle that refers to CONIN$. When that value is passed to the child process via STARTUPINFO, the child process will receive a console handle for the STDIN of whichever console it happens to be running in. On the other hand, if the parent process's STDIN has been redirected, GetStdHandle(STD_INPUT_HANDLE) will return an actual handle to a real file/pipe/etc, which the child will inherit and access.

The same applies to STDOUT and STDERR.

So, if you want to redirect the child's STDIN/OUT/ERR handles, you have to set hStdInput/Output/Error to your own handles. If you want the child to receive default handles, use GetStdHandle() and let CreateProcess() decide what kind of handles the child receives based on whether the parent is itself being redirected or not.