c - pthread_exit vs. return

Question

I have a joinable pthread runner function defined as below:

void *sumOfProducts(void *param)
{
...
pthread_exit(0);
}

This thread is supposed to join the main thread.

Whenever I ran my program through Valgrind I would get the following leaks:

LEAK SUMMARY:
   definitely lost: 0 bytes in 0 blocks
   indirectly lost: 0 bytes in 0 blocks
     possibly lost: 0 bytes in 0 blocks
   still reachable: 968 bytes in 5 blocks
        suppressed: 0 bytes in 0 blocks

ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 15 from 10)

I checked the man page for pthreads which said:

  The new thread terminates in one of the following ways:

   * It  calls  pthread_exit(3),  specifying  an exit status value that is
     available  to  another  thread  in  the  same  process   that   calls
     pthread_join(3).

   * It  returns  from  start_routine().   This  is  equivalent to calling
     pthread_exit(3) with the value supplied in the return statement.

   * It is canceled (see pthread_cancel(3)).

   * Any of the threads in the process calls exit(3), or the  main  thread
     performs  a  return  from main().  This causes the termination of all
     threads in the process.

Miraculously, when I replaced the pthread_exit() with a return statement, the leaks disappeared.

return(NULL);

My actual question is three-pronged:

1. Can someone explain why the return statement gave no leaks?
2. Is there some fundamental difference between both statements, in relation to exiting from threads?
3. If so, when should one be preferred over the other?

Answer 1

The following minimal test case exhibits the behaviour you describe:

#include <pthread.h>
#include <unistd.h>

void *app1(void *x)
{
    sleep(1);
    pthread_exit(0);
}

int main()
{
    pthread_t t1;

    pthread_create(&t1, NULL, app1, NULL);
    pthread_join(t1, NULL);

    return 0;
}

valgrind --leak-check=full --show-reachable=yes shows 5 blocks allocated from functions called by pthread_exit() that is unfreed but still reachable at process exit. If the pthread_exit(0); is replaced by return 0;, the 5 blocks are not allocated.

However, if you test creating and joining large numbers of threads, you will find that the amount of unfreed memory in use at exit does not increase. This, and the fact that it is still reachable, indicates that you're just seeing an oddity of the glibc implementation. Several glibc functions allocate memory with malloc() the first time they're called, which they keep allocated for the remainder of the process lifetime. glibc doesn't bother to free this memory at process exit, since it knows that the process is being torn down anyway - it'd just be a waste of CPU cycles.