= Building BOINC applications with Cuda and OpenCL =
[[PageOutline]]

We have built sample BOINC-Cuda and BOINC-OpenCL applications as templates for making your own app. The source files are in the [http://boinc.berkeley.edu/trac/wiki/SourceCodeGit BOINC source tree] in samples/nvcuda and samples/openclapp.

Before starting, you must install the appropriate SDK for the GPU you wish to support: [https://developer.nvidia.com/cuda-downloads NVIDIA SDK], [http://developer.amd.com/tools/heterogeneous-computing/amd-accelerated-parallel-processing-app-sdk/downloads/ ATI / AMD SDK] or [http://software.intel.com/en-us/vcsource/tools/opencl-sdk-2013 Intel Ivy Bridge].  This document assumes version 5 of the NVIDIA SDK or version 2.8 of the AMD SDK.  (Exception: no SDK is required for the OpenCL sample on Mac OS X, which has OpenCL support built in.)

== OpenCL Sample Notes == #openclnotes
The openclapp sample calls boinc_get_opencl_ids(), which reads an init_data.xml file to determine which GPU to use.  Actual project applications must also do this.  Please see the `ReadMe.txt` file for more information.  

This same sample is designed to run with AMD, NVIDIA and Intel Ivy Bridge GPUs.  It is supplied with 3 minimal init_data.xml files, one for each of these 3 vendors (GPU "types".)  Copy the appropriate init_data.xml file into the directory containing the openclapp executable.  

== Windows == #windows
The 32-bit Windows samples compile under Visual Studio.

=== NVIDIA Cuda ===
The VS project file, example_app_nvcuda.vcproj, is in `boinc/win_build/`. Note that NVIDIA Cuda SDK 3.0 or older supports device emulation mode. If your machine doesn't have a CUDA-enabled GPU, then you should build the sample application in the device emulation mode. Do so by setting `Build -> Configuration Manager -> Configuration` to either Emudebug or Emurelease.

The executable file can be found at `boinc/win_build/Build/Win32/build_mode/` (build_mode could be release, debug, emurelease or emudebug).

=== NVIDIA OpenCL ===
The project file, example_app_nvopencl.vcproj, is in `boinc/win_build/`.

The executable file can be found at `boinc/win_build/Build/Win32/build_mode/` (build_mode could be release or debug).

Upon running the executable file, you might encounter the following error:

{{{
This application has failed to start because OpenCL.dll was not found. Re-installing the application may fix this problem.
}}}
To fix this problem, you will need to install the Cuda Developer Driver for Windows,  which can only be done when your machine has an NVIDIA Cuda-enabled GPU.

Please read the [#openclnotes OpenCL Sample Notes] section above.

=== ATI Stream OpenCL ===
The project file, example_app_atiopencl.vcproj, is in `boinc/win_build/`. It uses the same source files as NVIDIA OpenCL.

Please read the [#openclnotes OpenCL Sample Notes] section above.

=== Intel Ivy Bridge OpenCL ===
''Project file to be added.''  It uses the same source files as NVIDIA OpenCL.

Please read the [#openclnotes OpenCL Sample Notes] section above.

== Linux == #linux
The OpenCL samples assume 32-bit Ubuntu.  You may need to modify them for other Linux configurations.

=== NVIDIA Cuda ===
Before running the Makefile, you will need to install gcc 4.3 and g++ 4.3. This is because the NVIDIA Cuda SDK 3.0 has not yet worked with gcc 4.0 and g++ 4.0. There should be no issue compiling cuda files with gcc 4.3 and g++ 4.3 on newer NVIDIA Cuda SDK versions. For a successful compilation, please follow these steps:

1) Install gcc-4.3 and g++-4.3:

{{{
$ sudo aptitude install gcc-4.3 g++-4.3
}}}
2) Go to SDK source directory:

{{{
$ cd ~/NVIDIA_GPU_Computing_SDK/C
}}}
3) Create a directory and create symlinks to gcc-4.3/g++-4.3

{{{
$ mkdir mygcc
$ cd mygcc
$ ln -s $(which g++-4.3) g++
$ ln -s $(which gcc-4.3) gcc
}}}
The Makefile for Linux is made with the assumption that the NVIDIA Cuda SDK is installed at the default location `$(ROOT)/home/USER_NAME/NVIDIA_GPU_Computing_SDK/`. On my Linux machine, for example, it would be `$(ROOT)/home/tuanle/NVIDIA_GPU_Computing_SDK/`. Since the absolute path is used in Makefile, you will probably need to edit file `common.mk` before compiling the sample application. Do so by looking for `/home/tuanle/NVIDIA_GPU_Computing_SDK` in file `common.mk` and replace this path with an appropriate path on your machine.

You will also need to build the necessary Cuda SDK libraries by running makefile in `~/NVIDIA_GPU_Computing_SDK/C/`. The resulting libraries such as `libcutil_i386` are  created in `~/NVIDIA_GPU_Computing_SDK/C/lib/`. Notice that if these libraries are not built, running nvcuda-sample-app makefile could result in the following error:

{{{
/usr/bin/ld: cannot find -lcutil_i386
}}}
If your machine doesn't have an NVIDIA Cuda-enabled GPU, you can compile and run your app in emulation mode. As far as we know, NVIDIA Cuda SDK 3.1 no longer supports emulation mode. Thus, you will need to install an older Cuda SDK version, like version 3.0, on your machine. To build the sample application in emulation mode, simply type:

{{{
make emu=1
}}}
The executable file is created in `/boinc/samples/nvcuda/linux/build_mode/` (build_mode could be release or emurelease). Also, note that you will need to define some environment variables before running the executable file.

{{{
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH
}}}
This will resolve the following error:

{{{
./example_app_nvcuda: error while loading shared libraries: libcudart.so.3: cannot open shared object file: No such file or directory
}}}
=== NVIDIA OpenCL ===
It appears that NVIDIA OpenCL SDK is missing shared library file libOpenCL.so which is needed at compile time. This library file comes with the kernel driver package which can be installed only when your machine has an NVIDIA Cuda-enabled GPU. Thus, any attempt to compile OpenCL sample codes on a non-cuda-enabled GPU machine might result in the following error:

{{{
/usr/bin/ld: cannot find -lOpenCL
}}}
The Makefile for Linux is made with the assumption that the NVIDIA Cuda SDK is installed at the default location `/usr/local/cuda/` with subdirectories `include` and `lib`.  Some versions of Ubuntu have a bug which can prevent the `LD_LIBRARY_PATH` environment variable from being honored as described [https://help.ubuntu.com/community/EnvironmentVariables#File-location_related_variables here], so the Makefile hard codes the path to the standard libraries as '/lib/i386-linux-gnu'.  You may need to modify the Makefile if any of these are at different locations.

Please read the [#openclnotes OpenCL Sample Notes] section above.

=== ATI Stream OpenCL ===
The Makefile for Linux is made with the assumption that the environment variable `AMDAPPSDKROOT` contains the path to the ATI Stream OpenCL SDK. This environment variable should be set automatically when you install the SDK.

Please read the [#openclnotes OpenCL Sample Notes] section above.

=== Intel Ivy Bridge OpenCL ===
''Project file to be added.''  It uses the same source files as NVIDIA OpenCL.

Please read the [#openclnotes OpenCL Sample Notes] section above.

== Mac OS X == #mac
The Mac versions are built using gcc/make (not xcode). 

=== NVIDIA Cuda ===
Unlike Windows and Linux, the 'nvcc' cuda compiler for Mac has some trouble compiling .cu files that contain both BOINC and Cuda code. Any attempt to compile such .cu files might result in errors like these:

{{{
/usr/lib/gcc/i686-apple-darwin9/4.0.1/include/mmintrin.h(55): error: identifier `__builtin_ia32_emms` is undefined
/usr/lib/gcc/i686-apple-darwin9/4.0.1/include/mmintrin.h(68): error: identifier `__builtin_ia32_vec_init_v2si` is undefined
/usr/lib/gcc/i686-apple-darwin9/4.0.1/include/mmintrin.h(111): error: identifier `__builtin_ia32_vec_ext_v2si` is undefined
...
}}}
The solution that we came up with is to compile BOINC and Cuda separately, meaning that compile .c files that have BOINC codes with gcc and compile .u files that have Cuda codes such as kernel definitions with nvcc (This can simply be done by putting .cu and .c file names at corresponding `CUFILES := ` and `CCFILES := ` entries in Makefile_mac). To handle the results computed by the kernels, you will need to write external functions. The function definitions that make kernel calls will be put in .cu file while the function headers with the `extern` prefix are put in .c file and can be called in main function or elsewhere.

If your machine doesn't have an NVIDIA Cuda-enabled GPU,  you can compile and run your app in emulation mode. As far as we know, NVIDIA Cuda SDK 3.1 no longer supports emulation mode. Thus, you will need to install an older Cuda SDK version, like version 3.0, on your machine.

The makefile for Mac is made with the assumption that the NVIDIA Cuda SDK is installed at the default location: `$ROOT/Developer/GPU Computing/` on Mac. If it is installed at a different location on your machine, then you will need to edit file `common_mac.mk`. Do so by looking for `ROOTDIR  ?= /Developer/GPU\ Computing` in file `common_mac.mk` and replace this path with an appropriate path on your machine.

To build the sample application in emulation mode, simply type:

{{{
make -f Makefile_mac emu=1
}}}
The executable file is created in `/boinc/samples/nvcuda/darwin/build_mode/` (build_mode could be release or emurelease).  Also, note that you will need to define some environment variables before running the executable file.

{{{
export PATH=/usr/local/cuda/bin:$PATH
export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH
}}}
=== OpenCL ===
Mac OS X has built in support for OpenCL.  No additional SDK is needed to build and run the OpenCL sample, provided that you have an OpenCL-capable GPU.  Note: at this time, Mac OS X does not yet support OpenCL on Intel Ivy Bridge GPUs.

Use the Terminal application to build the sample application.  In Terminal, `cd` to the `samples/openclapp` directory and type:
{{{
$ make -f Makefile_mac
}}}

Please read the [#openclnotes OpenCL Sample Notes] section above.