cuda - Default Pinned Memory Vs Zero-Copy Memory

Question

In CUDA we can use pinned memory to more efficiently copy the data from Host to GPU than the default memory allocated via malloc at host. However there are two types of pinned memories the default pinned memory and the zero-copy pinned memory.

The default pinned memory copies the data from Host to GPU twice as fast as the normal transfers, so there's definitely an advantage (provided we have enough host memory to page-lock)

In the different version of pinned memory, i.e. zero-copy memory, we don't need to copy the data from host to GPU's DRAM altogether. The kernels read the data directly from the Host memory.

My question is: Which of these pinned-memory type is a better programming practice.

Answer 1

I think it depends on your application (otherwise, why would they provide both ways?)

Mapped, pinned memory (zero-copy) is useful when either:

• The GPU has no memory on its own and uses RAM anyway

• You load the data exactly once, but you have a lot of computation to perform on it and you want to hide memory transfer latencies through it.

• The host side wants to change/add more data, or read the results, while kernel is still running (e.g. communication)

• The data does not fit into GPU memory

Note that, you can also use multiple streams to copy data and run kernels in parallel.

Pinned, but not mapped memory is better:

• When you load or store the data multiple times. For example: you have multiple subsequent kernels, performing the work in steps - there is no need to load the data from host every time.

• There is not that much computation to perform and loading latencies are not going to be hidden well