Streaming OpenCV Output Through HTTP/Network with MJPEG

If you want to stream your OpenCV output image to another device which connected to the same network, one way is to write the OpenCV output as MJPEG file, then stream this file through HTTP.

Please note that:
– This might be not the best way (correct me if I’m wrong)
– OpenCV 2.3 cv::VideoWriter has memory leak bug

0. OpenCV Stuff
I assume you already have your OpenCV libraries installed, because several of OpenCV’s dependencies are also MJPEG-Streamer’s, such as libv4l-dev and libjpeg-dev. And make sure your OpenCV application works (it writes the mjpeg file)

1. Install MJPEG-Streamer
Install the dependencies first

sudo apt-get install imagemagick

Then get the latest MJPEG-Streamer from the repository:

git clone
cd mjpg-streamer/mjpg-streamer
make USE_LIBV4L2=true clean all
sudo make DESTDIR=/usr/local install

Then set the environment variable LD_LIBRARY_PATH if you haven’t, either in /etc/bash.bashrc or your own ~/.bashrc, add this line:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib

2. Into The Main Show
First fire up your OpenCV application, this is my application, simple background subtraction and writes the output into mjpg file.

git clone
cd bgsubtract/
make bgsubtract2

Prepare the folder to put the mjpg file into, for example in my home folder:

mkdir /home/ariandy/mjpg

Then run the application:

./bgsubtract2 /home/ariandy/mjpg/out.mjpg -bgs

Leave it running in one terminal, and open another terminal to test the MJPEG-Streamer:

mjpeg_streamer -i " -f /home/ariandy/mjpg" -o " -w /usr/local/www"

The command above is the most basic setting to get the streaming to work. By default, the web server is listening to port 8080. If you want to use port 80, you have to run the command as root (or with sudo).

The default www root of MJPEG-Streamer is located at /usr/local/www. There you can find the demo pages, from which you can get the idea of several ways to display the stream: either direct stream (supported by most Grade-A Desktop Browsers (Sorry, IE!)), using Java MJPEG viewer (cambozola), javascript, etc.

3. Viewing The Stream
On the client, simply point the browser to the server’s IP address with corresponding port (8080). e.g. server’s IP address is, then to access it from the client, point to If you want to omit the :8080 part, then you have to run the server on port 80 (HTTP), as following:

sudo mjpeg_streamer -i " -f /home/ariandy/mjpg" -o " -w /usr/local/www -p 80"

I tried this with Google Chrome, Firefox, Android Browser, and Firefox for Android.



Ubuntu 12.04 SSH Server Problem: Read from socket failed: Connection reset by peer

Problem: couldn’t connect to Ubuntu 12.04 SSH Server. Everytime client wants to connect through ssh, it gives error:

Read from socket failed: Connection reset by peer

Then on the server side, checking the log using tail -f /var/log/auth.log while trying to connect from client gives:

ubuntu sshd[3207]: Server listening on port 22.
ubuntu sshd[3207]: Server listening on :: port 22.
ubuntu sshd[3208]: error: Could not load host key: /etc/ssh/ssh_host_rsa_key
ubuntu sshd[3208]: error: Could not load host key: /etc/ssh/ssh_host_dsa_key
ubuntu sshd[3208]: error: Could not load host key: /etc/ssh/ssh_host_ecdsa_key

According to many sources, this is possibly because the host key files are not exist. To generate the host key files they do this:

sudo ssh-keygen -t rsa1 -f /etc/ssh/ssh_host_rsa_key
sudo ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key
sudo ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key

Then restart SSH server:

sudo service ssh restart

But this doesn’t solve the problem.

To solve this, we need to reinstall the openssh server and wipe all the files:

cd /etc
sudo mv ssh/ ssh.old/
sudo apt-get remove openssh-server openssh-client ssh-import-id

Watch the output and make sure it is uninstalling. Then install it again:

sudo apt-get install openssh-server openssh-client ssh-import-id

The installation process should generate new host key files, if not, check if you have removed/renamed the /etc/ssh/ folder properly.

This is the sign when your problem is about to be solved:

Unpacking openssh-server (from .../openssh-server_1%3a5.9p1-5ubuntu1_amd64.deb) ...
Selecting previously unselected package ssh-import-id.
Unpacking ssh-import-id (from .../ssh-import-id_2.10-0ubuntu1_all.deb) ...
Processing triggers for ureadahead ...
ureadahead will be reprofiled on next reboot
Processing triggers for ufw ...
Processing triggers for man-db ...
Setting up openssh-server (1:5.9p1-5ubuntu1) ...
Creating SSH2 RSA key; this may take some time ...
Creating SSH2 DSA key; this may take some time ...
Creating SSH2 ECDSA key; this may take some time ...
ssh start/running, process 21733
Setting up ssh-import-id (2.10-0ubuntu1) ...

Notice the part where the installation process generating the host keys. Now the problem is solved.

Don’t forget to secure your server by disabling root login in /etc/ssh/sshd_config:

#PermitRootLogin yes
PermitRootLogin no

Getting Raspberry Pi, OpenNI2, and Asus Xtion Pro Live To Work Together

UPDATE Feb 28, 2013:

– It works on my 256MB Raspberry Pi with Asus Xtion Pro Live tested through powered USB Hub from Belkin.
– Camera viewer that is shipped with OpenNI (NiViewer or SimpleViewer) will not work because it’s built with OpenGL. Raspberry Pi doesn’t support OpenGL. So to get camera visualization we have to use OpenCV. In the source above he uses OpenCV from raspbian repository. But since I’m gonna do image processing with OpenCV so I prefer to install it manually.
– Building OpenNI2 from source will take a lot of time. To save the fuss you can grab the pre-compiled Raspberry Pi package from Hirotaka’s website above (OpenNI version 2.0.0), or my package (version 2.1.0, size ca. 1.5MB) here.

For OpenNI2 installation, first install the dependencies:

sudo apt-get install git g++ python libusb-1.0-0-dev freeglut3-dev doxygen graphviz

Please note that doxygen and graphviz needs 600-ish MB to download (5 minutes at ca. 2 MByte/s), and they will take around 900MB of your SD Card space once installed. They are needed to compile the documentation. Once OpenNI2 is built, we do not need this two packages anymore (I think). So if you have limited internet speed, this step itself will take a lot of time, not to mention Raspberry Pi is very slow when it comes to package installation. As mentioned before, you can just download the pre-compiled package and it will work just fine.

Now grab a copy of OpenNI2 source code from github:

git clone

Then there are two files that needed to be altered:
First OpenNI2/ThirdParty/PSCommon/BuildSystem/Platform.Arm. Change or comment this line:

CFLAGS += -march=armv7-a -mtune=cortex-a8 -mfpu=neon -mfloat-abi=softfp #-mcpu=cortex-a8

then replace or add with this:

CFLAGS += -mtune=arm1176jzf-s -mfpu=vfp -mfloat-abi=hard

The second file is OpenNI2/Redist/ Go to line 534 to find this:

compilation_cmd = "make -j" + calc_jobs_number() + " CFG=" + configuration + " PLATFORM=" + platform + " > " + outfile + " 2>&1"

Then duplicate the line, comment the original and change the copied line:

#compilation_cmd = "make -j" + calc_jobs_number() + " CFG=" + configuration + " PLATFORM=" + platform + " > " + outfile + " 2>&1"
compilation_cmd = "make -j1" + " CFG=" + configuration + " PLATFORM=" + platform + " > " + outfile + " 2>&1"

Now let’s build OpenNI2:

cd OpenNI2/

This took ca. 30-40 minutes on my Raspberry Pi.

Then create the OpenNI2 package:

cd Redist/
./ arm

Now you can find the installer package (OpenNI-Linux-Arm-2.1.0.tar.bz2) in the folder OpenNI2/Redist/Final.

To install this package, simply unzip it to somewhere. I chose in /usr/local/src. You might need to change your group into staff so you have write permission in that folder. I’m not sure whether this is “safe” or not.

sudo usermod -a -G staff pi

Or just use sudo while copying.

cd Final/
cp OpenNI-Linux-Arm-2.1.0.tar.bz2 /usr/local/src
cd /usr/local/src/
tar -xjvf OpenNI-Linux-Arm-2.1.0.tar.bz2

Now that we have the installation package, let’s install it:

cd OpenNI-2.1.0-arm/
sudo ./

Nothing will come up if you got it right. Now you can try if it works with your Asus Xtion. First make sure it’s detected in your Raspberry Pi, check the output of lsusb -vv, it should come up somehow like this:

Bus 001 Device 006: ID 1d27:0600  
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x1d27 
  idProduct          0x0600 
  bcdDevice            0.01
  iManufacturer           2 PrimeSense
  iProduct                1 PrimeSense Device
  iSerial                 0 

### DELETED ###

Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0000
  (Bus Powered)

If it’s giving

Bus 001 Device 006: ID 1d27:0600  
Couldn't open device, some information will be missing

unplug and plug in other USB port. My 256MB Raspberry Pi is able to detect the sensor without powered USB hub, but it couldn’t get any data out of it. Some say this is because this RPi version has lower USB bandwidth. But in Hirotaka’s website he’s connecting Xtion directly to his 512MB Raspberry Pi and it works just fine.

Then try to read the sensor data:

cd Samples/Bin

This is my output:

ariandy@raspberrypi /usr/local/src/OpenNI-2.1.0-arm/Samples/Bin $ ./SimpleRead 
Warning: USB events thread - failed to set priority. This might cause loss of data...
[00000000]     3816
[00033369]     3816
[00066738]     3816
[00100107]     3816
[00133477]     3816
[00166846]     3816
[00200215]     3816
[00233584]     3816
[00266954]     3816
[00300323]     3816

If you get the same output, you should get something nice for yourself and celebrate!

Now we just have to make an OpenCV viewer program, because the default SimpleViewer will not compile on Raspberry Pi.

To be continued …

Calculating FPS in OpenCV for Live Capture

cvGetCaptureProperty(CV_CAP_PROP_FPS); simply won’t work for live capture from camera. So following is my workaround, enhanced version of this source:

#include "cv.h"
#include "highgui.h"

#include <stdio.h>
#include <time.h> // to calculate time needed
#include <limits.h> // to get INT_MAX, to protect against overflow

int main(int argc, char** argv){
	CvCapture* capture = cvCaptureFromCAM( 0 );
	if (!capture){
		fprintf(stderr, "ERROR: Capture is null\nPress any key to exit\n");
		return -1;
	// fps counter begin
	time_t start, end;
	int counter = 0;
	double sec;
	double fps;
	// fps counter end
	cvNamedWindow("Original", CV_WINDOW_AUTOSIZE);
		// fps counter begin
		if (counter == 0){
		// fps counter end
		IplImage* origImage = cvQueryFrame(capture);
		cvShowImage("Original", origImage);

		// do your stuff here

		// fps counter begin
		sec = difftime(end, start);
		fps = counter/sec;
		if (counter > 30)
			printf("%.2f fps\n", fps);
		// overflow protection
		if (counter == (INT_MAX - 1000))
			counter = 0;
		// fps counter end

		// will exit when ESC button is pressed
		if ( (cvWaitKey(10) & 255) == 27 ) break;
	return 0;

Raspberry Pi Rasbian + OpenCV


First of all, building OpenCV on Raspbian (Raspberry Pi) will take at least 4 hours. So consider doing this before you sleep so you can leave it compiling overnight.

And also if you’re doing all this through SSH, linux’s “screen” program will definitely very useful. It’s not installed by default, so do this:

sudo apt-get install screen

Click here for short and quick tutorial of how to use screen.

Now let’s cook!

  1. Prepare fresh installed Rasbian on Raspberry Pi.
  2. Run sudo apt-get update, then sudo apt-get upgrade, to make sure everything is updated.
  3. Copy and paste this into the terminal to install all the dependencies (NOTE: Some package names are altered, e.g. libavcodec52 is not available anymore, replaced by libavcodec53):

    sudo apt-get -y install build-essential
    sudo apt-get -y install cmake
    sudo apt-get -y install pkg-config
    sudo apt-get -y install libpng12-0 libpng12-dev libpng++-dev libpng3
    sudo apt-get -y install libpnglite-dev libpngwriter0-dev libpngwriter0c2
    sudo apt-get -y install zlib1g-dbg zlib1g zlib1g-dev
    sudo apt-get -y install libjasper-dev libjasper-runtime libjasper1
    sudo apt-get -y install pngtools libtiff4-dev libtiff4 libtiffxx0c2 libtiff-tools
    sudo apt-get -y install libjpeg8 libjpeg8-dev libjpeg8-dbg libjpeg-prog
    sudo apt-get -y install libavcodec53 libavcodec-dev libavformat53 libavformat-dev libavutil51 libavutil-dev libswscale2 libswscale-dev
    sudo apt-get -y install libgstreamer0.10-0-dbg libgstreamer0.10-0 libgstreamer0.10-dev
    sudo apt-get -y install libxine1-ffmpeg libxine-dev libxine1-bin
    sudo apt-get -y install libunicap2 libunicap2-dev
    sudo apt-get -y install libdc1394-22-dev libdc1394-22 libdc1394-utils
    sudo apt-get -y install swig
    sudo apt-get -y install python-numpy
    sudo apt-get -y install libpython2.6 python-dev python2.6-dev
    sudo apt-get -y install libjpeg-progs libjpeg-dev
    sudo apt-get -y install libgstreamer-plugins-base0.10-dev
    sudo apt-get -y install libqt4-dev libgtk2.0-dev
  4. We need to install other dependencies (x264, ffmpeg, and v4l) manually:
    sudo apt-get remove ffmpeg x264 x264-dev
    tar -xvf x264-snapshot-20120528-2245-stable.tar.bz2
    cd x264-snapshot-20120528-2245-stable/
    ./configure --enable-shared --enable-pic
    sudo make install
    cd ..
    echo "Installing ffmpeg"
    tar -xvf ffmpeg-0.11.1.tar.bz2
    cd ffmpeg-0.11.1/
    ./configure --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --enable-nonfree --enable-postproc --enable-version3 --enable-x11grab --enable-shared --enable-pic
    sudo make install
    cd ..
    tar -xvf v4l-utils-0.8.8.tar.bz2
    cd v4l-utils-0.8.8/
    sudo make install
    cd ..
  5. Download desired version of OpenCV, in this example we’re using version 2.4.3. Unpack it anywhere you like.
    tar -xjvf  OpenCV-2.4.3.tar.bz2
    rm OpenCV-2.4.3.tar.bz2
    cd OpenCV-2.4.3/
    mkdir build
    cd build/
  6. Then to create standard configuration just follow this command:

    The part | sudo tee ./CMAKE.log is used for logging purpose. Normally I used redirection (>>) but it won’t work if you are trying to write inside a folder with no permission.

    You can use cmake-gui if it’s more comfortable for you, but it has to be run from a desktop environment (X Server is running). I did this over SSH so I don’t have the luxury of X.

  7. Then a configuration will be generated, make sure everything you need has no problem. For example see the first lines, it should be able to found the libraries needed, such as gtk+2.0, libavcodec, libavformat, etc.
    In my case cmake couldn’t find linux/videodev.h. According to this source, apparently there’s a change of structure for libv4l but OpenCV didn’t take that into account. In my installation I found the file libv4l1-videodev.h inside /usr/local/include, so we need to make a symlink to the location searched by OpenCV installation:

    sudo ln -s /usr/local/include/libv4l1-videodev.h /usr/include/linux/videodev.h

    TODO: there’s also one missing library, ffmpeg/avformat.h, but I’m not sure whether it’s necessary to fix or not.

  8. Now we’re ready to start the build. This process will take a long time to finish. So as said before, consider doing this before you sleep overnight. I do all this through SSH (I only have 1 keyboard and 1 mouse and I don’t want to back and forth plugging them between my laptop and the Raspberry Pi), so here’s where “screen” program is very useful. You can ignore this if you don’t use SSH:
    screen -S OpenCV_Installation

    Seems like nothing happened, but what actually happens is “screen” starts a new TTY/session, which you can build OpenCV, then you can “detach” it so it will run on background. After that you can quit the SSH session and turn off your laptop, without even bothering the building process.

    Now change user into root

    sudo su -

    then execute this as root

    make && make install

    or if you want some logging

    make | tee make.log && make install | tee make_install.log
  9. Now “detach” the compilation “screen” by pressing Ctrl+A then Ctrl+D consecutively. The building process will run in background. To check the process you have to get back to the “screen”, simply call in any session – either SSH or local
    screen -r

    If you use the logging with tee make.log and tee make_install.log as mentioned above, to check the status you can use tail

    tail -f make.log
  10. Sit back, watch some movie, or sleep. See you in 5 hours!


Now that you have energy from your sleep, let’s continue setting up the system to complete the installation.

  1. We have to tell the system that there are libraries available. Create a file named opencv.conf located in /etc/
    sudo touch /etc/

    Then using your favorite editor add this line into the file:


    After that:

    sudo ldconfig /etc/
  2. Then we have to add this 2 lines into the file /etc/bash.bashrc
    export PKG_CONFIG_PATH


NOTE: Seems like my Raspberry Pi is unable to detect my cheap Logitech C170 yet. This is weird.

OpenCV comes with lots of pre-compiled sample programs, find it in build/bin folder. Some of them are drawing and kmeans. And if you have USB webcam you can check out lkdemo


Press r then it will start tracking objects. Cool huh?

I also found this over here, to test camera capture using python.

import cv2

vc = cv2.VideoCapture(0)

if vc.isOpened(): # try to get the first frame
    rval, frame =
    rval = False

while rval:
    cv2.imshow("preview", frame)
    rval, frame =
    key = cv2.waitKey(20)
    if key == 27: # exit on ESC

Save it and run it with python:


For obvious reasons you can’t do this over SSH.

Internet Connection Sharing in Ubuntu 12.04 Precise Pangolin


The case is when your laptop has internet connection from Wi-Fi, while other PC or device only has ethernet port, and you want to connect the ethernet-only device to internet through the wifi-connected laptop.

Actually it’s quite easy – compared to 5-10 years ago when we have to fiddle with iptables and masquerading. Here’s what we have to do on the laptop:

1. Click on the Network Manager Applet (on top right of the screen), then select “Edit Connections”.

2. On the “Wired” tab, select the “Wired Connection 1” or whatever it might be, then click on “Edit” button on its right side.

3. Move to “IPv4 Settings”, change the “Method” drop-down into “Shared to other computers”.

4. Done.


Then on the other device, just plug the ethernet cable, then it will do the rest automatically, absolute hassle-free.


In case it doesn’t work like that, do this first on the laptop:

$ sudo ifconfig eth0 netmask

then on the device:

$ sudo ifconfig eth0 netmask

Then set the gateway, pointing to the laptop’s address:
$ sudo route add default gw

Making Raspberry Pi SD Card Backup in Linux

$ sudo dd if=/dev/mmcblk0 of=/path/to/image/file.img bs=4M

/dev/mmcblk0 argument depends on your system. In some system this argument could be /dev/sdX where X is the “device number” that points to the SD card. What is X could be known from:

$ sudo fdisk -l

Make sure you point dd to the device (e.g. /dev/mmcblk0, /dev/sdd), not the partition (e.g. /dev/mmcblk0p1, /dev/sdd1).

Making an image of 8GB SD card will take a long time, and dd won’t give any progress update whatsoever. It’s done, when it says it’s done (Walter White, Breaking Bad. LOL). It took 10 minutes (14.3MB/s) with my laptop. The only way to know the progress is by opening other terminal and check what is the size of the image file. The final size of the image file is same as the size of your SD card. OR you can use one front-end (?) of dd, namely dcfldd.

Before unplugging the SD card don’t forget to flush the buffer/cache:

$ sudo sync

Calculate the SHA1Sum of the image for future purposes. For huge file like this there’s a chance that it will get damaged when you do a lot of moving and transfer.

$ sha1sum file.img >> file.img.sha1sum

This will save the sha1sum into a file named file.img.sha1sum. After this you can compress it, transfer it, anything. Then, before restoring this image back into SD card, calculate the SHA1 once again and compare it to the original SHA1.

To compress the image use this:

$ cd /path/to/image
$ gzip file.img

This will “throw” the empty bits of the image and squeeze the file only with bits that actually contains data.

Restoring The Image

To restore the image first decompress the file

$ gunzip file.img.gz

then use dd

$ sudo dd if=/path/to/image/file.img of=/dev/mmcblk0 bs=4M

don’t forget to flush the cache by

$ sudo sync

– sha1sum checking

Getting Started with Raspberry Pi

1. Get a SD Card, consult to compatibility page on

2. Plug it in to linux machine.

3. sudo cfdisk /dev/mmcblk0

4. Delete every partition from there, don’t forget to select “write”.

5. Download Raspbian from here, check the sha1sum, then unzip. You’ll get a image file named something like 2012-12-16-wheezy-raspbian.img

6. Write it into your SD card, make sure you write it into the SD card device (e.g. /dev/mmcblk0, /dev/sdd), not the partition of the SD card (e.g. /dev/mmcblk0s1, /dev/sdd1).

$ sudo dd bs=4M if=./2012-12-16-wheezy-raspbian.img of=/dev/mmcblk0

Wait until it finishes. There will be no progress bar or indicator whatsoever, other than the SD card reader’s LED.

The Raspbian image is about 2GB in size. No matter how big your SD card, when you use it as it is after flashing it won’t be able to utilize the rest of the freespace. Then you need to expand/grow the partition. We do this later on below.

7. Flush the buffer (NOTE: at the first time I skipped this part, then random block reading error pops out everywhere. Segmentation fault, filesystem panic, etc. So make sure you do this.)

$ sudo sync

8. Remove the SD card, put it on the Raspberry Pi and fire it up!

9. First boot it will take some time, after then a blue screen (of life) will come up. It’s actually the “raspi-config” program.

10. Select the second option from the top to expand the free space of your SD card the next time Raspbian reboots. Configure other options as needed (other than expanding the partition, mostly not really necessary).

11. Select finish, then your Raspberry Pi will reboot.

12. Log in with username pi and default password raspberry (if you didn’t change it before in raspi-config). Check out the desktop environment by:

$ startx

Enjoy your Raspberry Pi.


How To Disable Ubuntu 12.04 Precise Pangolin Login Sound

One thing I hate from this release is that there is no easy way to turn off the annoying login sound. Hello ubuntu developer? Is it hard to make a tick box in the settings to enable/disable system sounds?

Anyway, while we’re waiting for the developers to fix this issue, we can work this thing out by renaming or removing the sound file of that login sound. Fire up Terminal, then:
$ cd /usr/share/sounds/ubuntu/stereo
$ sudo mv system-ready.ogg system-ready-OLD.ogg

With this, the system won’t be able to find the login sound file, hence it won’t play any login sound.

We can also use this trick to change the login sound with your own sound file, but first we have to convert our sound file into OGG format. The easiest way is using ffmpeg in terminal:
$ ffmpeg -i cool_sound.mp3 system-ready.ogg

Then copy the newly created system-ready.ogg into the directory /usr/share/sounds/ubuntu/stereo.

VLC’s Dynamic Range Compression Starter Settings


Here’s the Dynamic Range Compression settings to make your movie more fluid, no sudden loud sound but still audible dialogs.

RMS/Peak 0
Attack 50 ms
Release 300 ms
Threshold -20 dB
Ratio 20.0 : 1
Knee Radius 1.0 dB
Makeup Gain 12.0 dB

We can’t change the settings with exact values, so just move the slider at least close to the values.

Slider can be precisely moved by pressing keyboard arrow up and down after clicking on  the slider. Thanks, Austin!