Shared Memory Tip

Jeremy — Mon, 07 Feb 2011 15:39:37 +0000

As usual, I’m knee deep in CUDA optimising a fair few algorithms from various papers. Recently, I’ve been implementing the algorithms from this paper with the aim of improving them later/creating my own based from their concepts. The algorithm is an All Pairs Shortest Path algorithm with a nested loop in the kernel. Each time the second loop executes, two values from shared memory are added together and the resulted is evaluated against another variable stored in a register on the appropriate core. For some reason the code was running a lot slower than the results posted in the paper.

My friend here at Durham who is also working with CUDA suggested taking the addition out of the loop and storing the result in a register before the conditional. Much to my surprise, this worked a treat and instantly gave me comparable results with the paper.

Here is the original code before the change:

for (int i = 0; i < gridDim.x; i ++)

{
__shared__ int row[blockWidth][blockHeight];
__shared__ int column[blockWidth][blockHeight];
//Code here fills row and column
__syncthreads();
for(int k = 0; k < blockWidth; k ++)
{
if(row[threadIdx.y][k] + column[k][threadIdx.x] < value)
{
value = row[threadIdx.y][k] + column[k][threadIdx.x];
}
}
}

Here, we can see the change needed to drastically improve the running time of the algorithm:

unsigned int sum;

for(unsigned int k = 0; k < blockWidth; k ++)
{
sum = row[threadIdx.y][k] + column[k][threadIdx.x];
if(sum < value)
{
value = sum;
}
}

Given that shared memory is so quick on CUDA, similar to an L1 cache on CPU, I wouldn’t have thought that it would have made any difference at all. Obviously, I was wrong! So watch out for things like this when using CUDA or any parallel computing platform.

CUDA cuPrintf

Jeremy — Mon, 08 Feb 2010 12:06:08 +0000

I finally got an Nvidia developer account a few days ago which gave me access to a very useful library to use with CUDA.

cuPrintf allows printf equivalent statements to be placed inside CUDA kernels without the need for -deviceemu.

The following example demonstrates a simple use for cuPrintf and displays the current thread ID.

#include
#include "cuPrintf.cu"
__global__ void cuPrintfExample()
{
int tid;
tid = blockIdx.x * blockDim.x + threadIdx.x;
cuPrintf("%d\n", tid);
}
int main()
{
cudaPrintfInit();
cuPrintfExample <<< 5, 2 >>> ();
cudaPrintfDisplay(stdout, true);
cudaPrintfEnd();
return 0;
}

cudaPrintfInit and cudaPrintfEnd only need be called once throughout your entire project.

Output is not automatically displayed on the screen, but stored in a buffer which is cleared and displayed when cudaPrintfDisplay is called. The size of the buffer can be specified with the optional argument cudaPrintfInit(size_t bufferLen).

cudaPrintfEnd simply frees the memory allocated by cudaPrintfInit.

When cudaPrintfDisplay is called, output stored in the buffer is displayed to the console. The second argument in this call either displays the current thread (true) or doesn’t (false). The first arguemnt, specified by stdout in this example, simply defines the descriptor where the cuPrintf log is sent.

On another note, I’ve found that using cuPrintf impacts on the performance of my kernels, presumably due to the data transfer performed every time cuPrintfDisplay() is called.

Jeremy Kemp » CUDA

Shared Memory Tip

CUDA cuPrintf