ROS + OpenNI2 + NiTE2

After weeks banging my head with OpenNI version 1.5.4 that comes with my ROS fuerte installation, I finally come to the conclusion that OpenNI 1.5.4 is highly frustrating, difficult to use, and has very low code readability. Or maybe it is just me.

It is time to migrate to OpenNI2, the latest version of the library that has been completely hauled with new architecture, with (much) better code readability. And another good thing about this release is that this will not mess the other version of OpenNI, so we can still work with both version in the same time.

The installation process is pretty straight forward. We can get the installation files after registering on OpenNI website. Then just simply run the install script from each folder.

Then, to use OpenNI2 and NiTE2 with ROS, we need to add some parameters to the CMakeLists.txt of our ROS project/package to link them with the libraries. Here’s mine:

cmake_minimum_required(VERSION 2.4.6)

# Change two lines below according to your installation
set(OPENNI2_DIR /home/ariandy/src/OpenNI-Linux-x64-2.2/)
set(NITE2_DIR /home/ariandy/src/NiTE-Linux-x64-2.2/)

#set the default path for built executables to the "bin" directory
#set the default path for built libraries to the "lib" directory



rosbuild_add_executable(testing src/main.cpp)
target_link_libraries(testing OpenNI2 NiTE2)

Now grab any sample program codes from the OpenNI2 or NiTE2 and put it inside our ROS package for testing. It should compile just fine.

There’s still one issue though. NiTE2 uses machine learning method for the human recognition and also skeleton fitting, which relies heavily on training data. It keeps the training data on NiTE2 folder inside NiTE-Linux-*/Samples/Bin folder. And somehow, when NiTE2 initializes it will look for the training data relative to the path (e.g. your executable is at /home/user, then it will look for /home/user/NiTE2/*). That is a bummer, since we can run ROS executable (node) regardless of the path and this NiTE2 thing defeats the purpose. Workaround is by navigating first to NiTE-Linux-*/Samples/Bin/ or NiTE-Linux-*/Redist/ then do rosrun your_package your_node, otherwise it won’t find the training data.

Running roscore and Launching ROS nodes as Background Process

I access my ROS running robot wirelessly using secure shell (SSH). To run my program, I have to first run all the startup scripts such as roscore and my robot’s driver. It’s kind of tedious to open a lot of shells just to keep this services on. And I can’t simply run this on background by using & operator because sometimes I need to check the node’s output.

So to keep it simple and neat, I made simple script to run roscore and robot driver on background, but still be able to check the outputs of this two (e.g. error messages).

First thing first, I like to have my own bin folder, to keep my own scripts.

cd ~
mkdir bin
echo "export PATH=$PATH:/home/ariandy/bin" >> ~/.bashrc
source ~/.bashrc

Then all this scripts will be kept inside this folder. If you need to run roscore and some nodes as a startup script, it will be better if you keep this scripts in /usr/local/bin/ folder.

# file:

source /opt/ros/fuerte/setup.bash
export ROS_PACKAGE_PATH=$ROS_PACKAGE_PATH:/home/ariandy/youbot_driver:/home/ariandy/applications:/home/ariandy/ros_stacks


This is for my node:

# file:

source /opt/ros/fuerte/setup.bash
export ROS_PACKAGE_PATH=$ROS_PACKAGE_PATH:/home/ariandy/youbot_driver:/home/ariandy/applications:/home/ariandy/ros_stacks

# this is my driver
roslaunch youbot_oodl youbot_oodl_driver.launch
# or your own node/launch script
#roslaunch {your_package} {your_thing}

The main tool for our topic is linux’ “screen“. Install it first if you don’t have it.

# file:

echo "running roscore daemon.."
screen -dmS roscore
sleep 2
echo "running youbot oodl daemon"
screen -dmS oodl
echo "done. view with screen -ls"

The format is screen -dmS [any_name] [your_script]. The -dm argument will tell screen to run this session as a daemon. Then we use -S argument to give the session a name, so it will be easier for later. Please note that if your_script is not located on system path folder (e.g. /usr/bin/, /usr/local/bin/, etc.), you have to give the full path for the screen argument.

roscore needs some time (1-2 seconds) to reach its full operational state. That is why I give 2 seconds delay (with sleep 2) before running my node.

With this done, then we can run the things on the background:

running roscore daemon..
running youbot oodl daemon
done. view with screen -ls

To view what are the running sessions, call screen -ls

There are screens on:
	9285.oodl	(05/08/2013 05:01:50 PM)	(Detached)
	9224.roscore	(05/08/2013 05:01:49 PM)	(Detached)
2 Sockets in /var/run/screen/S-ariandy.

The format is [pid].[name]. Use this name if you want to go to each session: screen -r name.

When we’re inside the screen session, Ctrl-C will send a signal to the program we are running inside screen, makes it quitting and then the screen session closes. We don’t want this. What we want is to go back to our original shell session, and put the screen session back as background process. We do that by using key combination Ctrl-A-D, that is while holding Ctrl, press A, then press D.

Getting Raspberry Pi, OpenNI2, and Asus Xtion Pro Live To Work Together

UPDATE Feb 28, 2013:

– It works on my 256MB Raspberry Pi with Asus Xtion Pro Live tested through powered USB Hub from Belkin.
– Camera viewer that is shipped with OpenNI (NiViewer or SimpleViewer) will not work because it’s built with OpenGL. Raspberry Pi doesn’t support OpenGL. So to get camera visualization we have to use OpenCV. In the source above he uses OpenCV from raspbian repository. But since I’m gonna do image processing with OpenCV so I prefer to install it manually.
– Building OpenNI2 from source will take a lot of time. To save the fuss you can grab the pre-compiled Raspberry Pi package from Hirotaka’s website above (OpenNI version 2.0.0), or my package (version 2.1.0, size ca. 1.5MB) here.

For OpenNI2 installation, first install the dependencies:

sudo apt-get install git g++ python libusb-1.0-0-dev freeglut3-dev doxygen graphviz

Please note that doxygen and graphviz needs 600-ish MB to download (5 minutes at ca. 2 MByte/s), and they will take around 900MB of your SD Card space once installed. They are needed to compile the documentation. Once OpenNI2 is built, we do not need this two packages anymore (I think). So if you have limited internet speed, this step itself will take a lot of time, not to mention Raspberry Pi is very slow when it comes to package installation. As mentioned before, you can just download the pre-compiled package and it will work just fine.

Now grab a copy of OpenNI2 source code from github:

git clone

Then there are two files that needed to be altered:
First OpenNI2/ThirdParty/PSCommon/BuildSystem/Platform.Arm. Change or comment this line:

CFLAGS += -march=armv7-a -mtune=cortex-a8 -mfpu=neon -mfloat-abi=softfp #-mcpu=cortex-a8

then replace or add with this:

CFLAGS += -mtune=arm1176jzf-s -mfpu=vfp -mfloat-abi=hard

The second file is OpenNI2/Redist/ Go to line 534 to find this:

compilation_cmd = "make -j" + calc_jobs_number() + " CFG=" + configuration + " PLATFORM=" + platform + " > " + outfile + " 2>&1"

Then duplicate the line, comment the original and change the copied line:

#compilation_cmd = "make -j" + calc_jobs_number() + " CFG=" + configuration + " PLATFORM=" + platform + " > " + outfile + " 2>&1"
compilation_cmd = "make -j1" + " CFG=" + configuration + " PLATFORM=" + platform + " > " + outfile + " 2>&1"

Now let’s build OpenNI2:

cd OpenNI2/

This took ca. 30-40 minutes on my Raspberry Pi.

Then create the OpenNI2 package:

cd Redist/
./ arm

Now you can find the installer package (OpenNI-Linux-Arm-2.1.0.tar.bz2) in the folder OpenNI2/Redist/Final.

To install this package, simply unzip it to somewhere. I chose in /usr/local/src. You might need to change your group into staff so you have write permission in that folder. I’m not sure whether this is “safe” or not.

sudo usermod -a -G staff pi

Or just use sudo while copying.

cd Final/
cp OpenNI-Linux-Arm-2.1.0.tar.bz2 /usr/local/src
cd /usr/local/src/
tar -xjvf OpenNI-Linux-Arm-2.1.0.tar.bz2

Now that we have the installation package, let’s install it:

cd OpenNI-2.1.0-arm/
sudo ./

Nothing will come up if you got it right. Now you can try if it works with your Asus Xtion. First make sure it’s detected in your Raspberry Pi, check the output of lsusb -vv, it should come up somehow like this:

Bus 001 Device 006: ID 1d27:0600  
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x1d27 
  idProduct          0x0600 
  bcdDevice            0.01
  iManufacturer           2 PrimeSense
  iProduct                1 PrimeSense Device
  iSerial                 0 

### DELETED ###

Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0000
  (Bus Powered)

If it’s giving

Bus 001 Device 006: ID 1d27:0600  
Couldn't open device, some information will be missing

unplug and plug in other USB port. My 256MB Raspberry Pi is able to detect the sensor without powered USB hub, but it couldn’t get any data out of it. Some say this is because this RPi version has lower USB bandwidth. But in Hirotaka’s website he’s connecting Xtion directly to his 512MB Raspberry Pi and it works just fine.

Then try to read the sensor data:

cd Samples/Bin

This is my output:

ariandy@raspberrypi /usr/local/src/OpenNI-2.1.0-arm/Samples/Bin $ ./SimpleRead 
Warning: USB events thread - failed to set priority. This might cause loss of data...
[00000000]     3816
[00033369]     3816
[00066738]     3816
[00100107]     3816
[00133477]     3816
[00166846]     3816
[00200215]     3816
[00233584]     3816
[00266954]     3816
[00300323]     3816

If you get the same output, you should get something nice for yourself and celebrate!

Now we just have to make an OpenCV viewer program, because the default SimpleViewer will not compile on Raspberry Pi.

To be continued …

Calculating FPS in OpenCV for Live Capture

cvGetCaptureProperty(CV_CAP_PROP_FPS); simply won’t work for live capture from camera. So following is my workaround, enhanced version of this source:

#include "cv.h"
#include "highgui.h"

#include <stdio.h>
#include <time.h> // to calculate time needed
#include <limits.h> // to get INT_MAX, to protect against overflow

int main(int argc, char** argv){
	CvCapture* capture = cvCaptureFromCAM( 0 );
	if (!capture){
		fprintf(stderr, "ERROR: Capture is null\nPress any key to exit\n");
		return -1;
	// fps counter begin
	time_t start, end;
	int counter = 0;
	double sec;
	double fps;
	// fps counter end
	cvNamedWindow("Original", CV_WINDOW_AUTOSIZE);
		// fps counter begin
		if (counter == 0){
		// fps counter end
		IplImage* origImage = cvQueryFrame(capture);
		cvShowImage("Original", origImage);

		// do your stuff here

		// fps counter begin
		sec = difftime(end, start);
		fps = counter/sec;
		if (counter > 30)
			printf("%.2f fps\n", fps);
		// overflow protection
		if (counter == (INT_MAX - 1000))
			counter = 0;
		// fps counter end

		// will exit when ESC button is pressed
		if ( (cvWaitKey(10) & 255) == 27 ) break;
	return 0;

Raspberry Pi Rasbian + OpenCV


First of all, building OpenCV on Raspbian (Raspberry Pi) will take at least 4 hours. So consider doing this before you sleep so you can leave it compiling overnight.

And also if you’re doing all this through SSH, linux’s “screen” program will definitely very useful. It’s not installed by default, so do this:

sudo apt-get install screen

Click here for short and quick tutorial of how to use screen.

Now let’s cook!

  1. Prepare fresh installed Rasbian on Raspberry Pi.
  2. Run sudo apt-get update, then sudo apt-get upgrade, to make sure everything is updated.
  3. Copy and paste this into the terminal to install all the dependencies (NOTE: Some package names are altered, e.g. libavcodec52 is not available anymore, replaced by libavcodec53):

    sudo apt-get -y install build-essential
    sudo apt-get -y install cmake
    sudo apt-get -y install pkg-config
    sudo apt-get -y install libpng12-0 libpng12-dev libpng++-dev libpng3
    sudo apt-get -y install libpnglite-dev libpngwriter0-dev libpngwriter0c2
    sudo apt-get -y install zlib1g-dbg zlib1g zlib1g-dev
    sudo apt-get -y install libjasper-dev libjasper-runtime libjasper1
    sudo apt-get -y install pngtools libtiff4-dev libtiff4 libtiffxx0c2 libtiff-tools
    sudo apt-get -y install libjpeg8 libjpeg8-dev libjpeg8-dbg libjpeg-prog
    sudo apt-get -y install libavcodec53 libavcodec-dev libavformat53 libavformat-dev libavutil51 libavutil-dev libswscale2 libswscale-dev
    sudo apt-get -y install libgstreamer0.10-0-dbg libgstreamer0.10-0 libgstreamer0.10-dev
    sudo apt-get -y install libxine1-ffmpeg libxine-dev libxine1-bin
    sudo apt-get -y install libunicap2 libunicap2-dev
    sudo apt-get -y install libdc1394-22-dev libdc1394-22 libdc1394-utils
    sudo apt-get -y install swig
    sudo apt-get -y install python-numpy
    sudo apt-get -y install libpython2.6 python-dev python2.6-dev
    sudo apt-get -y install libjpeg-progs libjpeg-dev
    sudo apt-get -y install libgstreamer-plugins-base0.10-dev
    sudo apt-get -y install libqt4-dev libgtk2.0-dev
  4. We need to install other dependencies (x264, ffmpeg, and v4l) manually:
    sudo apt-get remove ffmpeg x264 x264-dev
    tar -xvf x264-snapshot-20120528-2245-stable.tar.bz2
    cd x264-snapshot-20120528-2245-stable/
    ./configure --enable-shared --enable-pic
    sudo make install
    cd ..
    echo "Installing ffmpeg"
    tar -xvf ffmpeg-0.11.1.tar.bz2
    cd ffmpeg-0.11.1/
    ./configure --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --enable-nonfree --enable-postproc --enable-version3 --enable-x11grab --enable-shared --enable-pic
    sudo make install
    cd ..
    tar -xvf v4l-utils-0.8.8.tar.bz2
    cd v4l-utils-0.8.8/
    sudo make install
    cd ..
  5. Download desired version of OpenCV, in this example we’re using version 2.4.3. Unpack it anywhere you like.
    tar -xjvf  OpenCV-2.4.3.tar.bz2
    rm OpenCV-2.4.3.tar.bz2
    cd OpenCV-2.4.3/
    mkdir build
    cd build/
  6. Then to create standard configuration just follow this command:

    The part | sudo tee ./CMAKE.log is used for logging purpose. Normally I used redirection (>>) but it won’t work if you are trying to write inside a folder with no permission.

    You can use cmake-gui if it’s more comfortable for you, but it has to be run from a desktop environment (X Server is running). I did this over SSH so I don’t have the luxury of X.

  7. Then a configuration will be generated, make sure everything you need has no problem. For example see the first lines, it should be able to found the libraries needed, such as gtk+2.0, libavcodec, libavformat, etc.
    In my case cmake couldn’t find linux/videodev.h. According to this source, apparently there’s a change of structure for libv4l but OpenCV didn’t take that into account. In my installation I found the file libv4l1-videodev.h inside /usr/local/include, so we need to make a symlink to the location searched by OpenCV installation:

    sudo ln -s /usr/local/include/libv4l1-videodev.h /usr/include/linux/videodev.h

    TODO: there’s also one missing library, ffmpeg/avformat.h, but I’m not sure whether it’s necessary to fix or not.

  8. Now we’re ready to start the build. This process will take a long time to finish. So as said before, consider doing this before you sleep overnight. I do all this through SSH (I only have 1 keyboard and 1 mouse and I don’t want to back and forth plugging them between my laptop and the Raspberry Pi), so here’s where “screen” program is very useful. You can ignore this if you don’t use SSH:
    screen -S OpenCV_Installation

    Seems like nothing happened, but what actually happens is “screen” starts a new TTY/session, which you can build OpenCV, then you can “detach” it so it will run on background. After that you can quit the SSH session and turn off your laptop, without even bothering the building process.

    Now change user into root

    sudo su -

    then execute this as root

    make && make install

    or if you want some logging

    make | tee make.log && make install | tee make_install.log
  9. Now “detach” the compilation “screen” by pressing Ctrl+A then Ctrl+D consecutively. The building process will run in background. To check the process you have to get back to the “screen”, simply call in any session – either SSH or local
    screen -r

    If you use the logging with tee make.log and tee make_install.log as mentioned above, to check the status you can use tail

    tail -f make.log
  10. Sit back, watch some movie, or sleep. See you in 5 hours!


Now that you have energy from your sleep, let’s continue setting up the system to complete the installation.

  1. We have to tell the system that there are libraries available. Create a file named opencv.conf located in /etc/
    sudo touch /etc/

    Then using your favorite editor add this line into the file:


    After that:

    sudo ldconfig /etc/
  2. Then we have to add this 2 lines into the file /etc/bash.bashrc
    export PKG_CONFIG_PATH


NOTE: Seems like my Raspberry Pi is unable to detect my cheap Logitech C170 yet. This is weird.

OpenCV comes with lots of pre-compiled sample programs, find it in build/bin folder. Some of them are drawing and kmeans. And if you have USB webcam you can check out lkdemo


Press r then it will start tracking objects. Cool huh?

I also found this over here, to test camera capture using python.

import cv2

vc = cv2.VideoCapture(0)

if vc.isOpened(): # try to get the first frame
    rval, frame =
    rval = False

while rval:
    cv2.imshow("preview", frame)
    rval, frame =
    key = cv2.waitKey(20)
    if key == 27: # exit on ESC

Save it and run it with python:


For obvious reasons you can’t do this over SSH.

Internet Connection Sharing in Ubuntu 12.04 Precise Pangolin


The case is when your laptop has internet connection from Wi-Fi, while other PC or device only has ethernet port, and you want to connect the ethernet-only device to internet through the wifi-connected laptop.

Actually it’s quite easy – compared to 5-10 years ago when we have to fiddle with iptables and masquerading. Here’s what we have to do on the laptop:

1. Click on the Network Manager Applet (on top right of the screen), then select “Edit Connections”.

2. On the “Wired” tab, select the “Wired Connection 1” or whatever it might be, then click on “Edit” button on its right side.

3. Move to “IPv4 Settings”, change the “Method” drop-down into “Shared to other computers”.

4. Done.


Then on the other device, just plug the ethernet cable, then it will do the rest automatically, absolute hassle-free.


In case it doesn’t work like that, do this first on the laptop:

$ sudo ifconfig eth0 netmask

then on the device:

$ sudo ifconfig eth0 netmask

Then set the gateway, pointing to the laptop’s address:
$ sudo route add default gw

Making Raspberry Pi SD Card Backup in Linux

$ sudo dd if=/dev/mmcblk0 of=/path/to/image/file.img bs=4M

/dev/mmcblk0 argument depends on your system. In some system this argument could be /dev/sdX where X is the “device number” that points to the SD card. What is X could be known from:

$ sudo fdisk -l

Make sure you point dd to the device (e.g. /dev/mmcblk0, /dev/sdd), not the partition (e.g. /dev/mmcblk0p1, /dev/sdd1).

Making an image of 8GB SD card will take a long time, and dd won’t give any progress update whatsoever. It’s done, when it says it’s done (Walter White, Breaking Bad. LOL). It took 10 minutes (14.3MB/s) with my laptop. The only way to know the progress is by opening other terminal and check what is the size of the image file. The final size of the image file is same as the size of your SD card. OR you can use one front-end (?) of dd, namely dcfldd.

Before unplugging the SD card don’t forget to flush the buffer/cache:

$ sudo sync

Calculate the SHA1Sum of the image for future purposes. For huge file like this there’s a chance that it will get damaged when you do a lot of moving and transfer.

$ sha1sum file.img >> file.img.sha1sum

This will save the sha1sum into a file named file.img.sha1sum. After this you can compress it, transfer it, anything. Then, before restoring this image back into SD card, calculate the SHA1 once again and compare it to the original SHA1.

To compress the image use this:

$ cd /path/to/image
$ gzip file.img

This will “throw” the empty bits of the image and squeeze the file only with bits that actually contains data.

Restoring The Image

To restore the image first decompress the file

$ gunzip file.img.gz

then use dd

$ sudo dd if=/path/to/image/file.img of=/dev/mmcblk0 bs=4M

don’t forget to flush the cache by

$ sudo sync

– sha1sum checking

Getting Started with Raspberry Pi

1. Get a SD Card, consult to compatibility page on

2. Plug it in to linux machine.

3. sudo cfdisk /dev/mmcblk0

4. Delete every partition from there, don’t forget to select “write”.

5. Download Raspbian from here, check the sha1sum, then unzip. You’ll get a image file named something like 2012-12-16-wheezy-raspbian.img

6. Write it into your SD card, make sure you write it into the SD card device (e.g. /dev/mmcblk0, /dev/sdd), not the partition of the SD card (e.g. /dev/mmcblk0s1, /dev/sdd1).

$ sudo dd bs=4M if=./2012-12-16-wheezy-raspbian.img of=/dev/mmcblk0

Wait until it finishes. There will be no progress bar or indicator whatsoever, other than the SD card reader’s LED.

The Raspbian image is about 2GB in size. No matter how big your SD card, when you use it as it is after flashing it won’t be able to utilize the rest of the freespace. Then you need to expand/grow the partition. We do this later on below.

7. Flush the buffer (NOTE: at the first time I skipped this part, then random block reading error pops out everywhere. Segmentation fault, filesystem panic, etc. So make sure you do this.)

$ sudo sync

8. Remove the SD card, put it on the Raspberry Pi and fire it up!

9. First boot it will take some time, after then a blue screen (of life) will come up. It’s actually the “raspi-config” program.

10. Select the second option from the top to expand the free space of your SD card the next time Raspbian reboots. Configure other options as needed (other than expanding the partition, mostly not really necessary).

11. Select finish, then your Raspberry Pi will reboot.

12. Log in with username pi and default password raspberry (if you didn’t change it before in raspi-config). Check out the desktop environment by:

$ startx

Enjoy your Raspberry Pi.