The easy way to achieve copy/compute overlap!
1.Enable Host Mapping*
Runtime: cudaSetDeviceFlags() with cudaDeviceMapHost flag
Driver : cuCtxCreate() with CU_CTX_MAP_HOST
2.Allocate pinned CPU memory
Runtime: cudaHostAlloc(), use cudaHostAllocMapped flag
Driver : cuMemHostAlloc()use CUDA_MEMHOSTALLOC_DEVICEMAP
3.Get a CUDA device pointer to this memory
Runtime: cudaHostGetDevicePointer()
Driver : cuMemHostGetDevicePointer()
4.Just use that pointer in your kernels!
Zero-Copy Guidlines
•Data is transferred over the PCIe bus automatically, but it’s slow
•Use when data is only read/written once
•Use for very small amounts of data (new variables, CPU/GPU communication)
•Use when compute/memory ratio is very high and occupancy is high, so latency over PCIe is hidden•Coalescing is critically important!