Wednesday, March 20, 2013

WebGL Image Processing or How to Get Consistent gl_FragCoord Values

My QR Code Scanner for FirefoxOS now uses WebGL to speed up image preprocessing.

And this is how the preprocessed video stream looks like (requires Browser with WebGL + getUserMedia support):

I'ts running some multi-pass fragment shaders to estimate a threshold for each pixel and then applies it to the gray value. I'm using a simple quad as described in this article. When I tested my shaders on four different devices, I got a a different result on each of them. I tracked it down to the following issue. The xy-coordinate of gl_Fragcoord is handled differently on all of my devices. According to the OpenGL specs, gl_Fragcoord.xy is supposed to be (0.5, 0.5) for the lower-left pixel. However I've also observed values like (0.0, 0.0), (1.0, 1,0) or (0.0, 0.5) on my Android and FirefoxOS devices. When gl_Fragcoord is used to sample from a texture, this leads to slightly inconsistent results across devices.

This might not be a big problem for a single-pass algorithm, but with multiple passes the error induced by this inconsistency can easily screw up the result completely. To work around this, I detect the device dependent glFragCoord offset and neutralize it. Therefore I render 10x10 canvas with the following fragment shader, that just puts gl_FragCoord.xy on the red/green channel:
void main() {
    vec4 result = vec4(1.0);
    result.rg = gl_FragCoord.xy / 10.0;
    gl_FragColor = result;
I read the values with readPixels():
    function round(val) {
      return Math.round(100 * val / 255) / 10;
    var imgdata = new Uint8Array(4 * canvas.width * canvas.height);
    gl.readPixels(0, 0, canvas.width, canvas.height, gl.RGBA,
      gl.UNSIGNED_BYTE, imgdata);

    var xOffset = imgdata[0];
    var yOffset = imgdata[1];
    // assume 0.1 steps.
    var fragCoordOffset = [round(xOffset), round(yOffset)];
And pass them to my subsequent shader calls. In my shaders I then use the following function instead of gl_FragCoord.xy:
// Fragment coordinate normalized to pixel centers eg: (0.5, 0.5) for bottom-left pixel.
vec2 getNormalizedFragCoord() {
    return (gl_FragCoord.xy - fragCoordOffset) + 0.5;
And voila: same results on all devices.

Thursday, March 7, 2013

QR Code Scanner for FirefoxOS

I just published a very first version of the QR Code Scanner for FirefoxOS that I've been working on lately. It's also available on (without offline support). The GUI is till pretty simple but it does the basic jobs. And this is how it looks like:

The App is AngularJS powered, test-driven by Testacular, scaffolded with Yeoman and built with Grunt. It uses a custom port of the ZXing barcode processing library together with iconv.js and utf8.js to decode QR Codes. To grab images, it uses the getUserMedia API if available and falls back a static image picker else. Currently Opera Mobile is the only Mobile Browser that supports getUserMedia (and only the presto version, the new webkit-based beta does not), but you can also try it with a recent Desktop Browser (Firefox 20+, or Chrome 21+).

The captured images are transferred to a web worker where the real work happens. This works quite smooth, even with Opera on my Android-powered mobile devices. Though I could not test it on any low-end device yet.

Besides obvious GUI improvements, in future versions I'll add support for more barcode types and speed up the detection by offloading some image prepossessing work to the GPU using WebGL.

Monday, December 10, 2012

vGet - Download Videos Directly to your Android Device

Did you ever want to watch a movie on a train trip with bad mobile connectivity on your mobile device? You would like to download the video to your device, but unfortunately it is only embedded in a site and there is no download link?
If this is the point where you usually dig through the HTML source code of the site, extract the URL of the video file and use wget to download it to your desktop computer, vGet might be a handy tool. It basically does the same thing directly on your Android device. No need to turn on your computer.

It works as a minimalist web browser. But instead of playing videos directly, it offers you an option to download them. It works on all websites that use HTML5 videos, which is quite cool. It comes without any site-specific extraction logic.

Unfortunately some sites still use Flash to play videos. As you probably know, Flash is no longer supported on recent versions of Android. Even on older Android versions,it actually never really worked well in my opinion. Today every non-ancient web browser has native video support, so from a technical perspective, there is no reason anymore to use Flash for playing videos at all. However, some some sites seem to know better and still use it. To deal with Flash-ancientness, vGet goes one step further and can even play Flash videos on many sites. It then offers to play them directly (stream) or download them.

For streaming, a media player needs to be installed on your Android device. VLC or MXPlayer both do a good job.

Tip: you don't need to browse to a site using vGet, but instead can send URLs to open with vGet from your regular web browser on your Android device. Just choose "Share -> vGet":

vGet is available on Google Play. If you experience any problems please report them. I hope you find it useful.

Tuesday, November 15, 2011

Vim Clipboard Support on Ubuntu

If recently wondered why clipboard doesn't work in vim on ubuntu (eg. "+y to copy to clipboard). It turned out, that the vim package doesn't ship with clipboard support enabled:

 vim --version | grep clipboard  
 -clientserver -clipboard +cmdline_compl +cmdline_hist +cmdline_info +comments   
  -xterm_clipboard -xterm_save   

You need to install vim-gui-common to get clipboard support:

 vim --version | grep clipboard  
 +clientserver +clipboard +cmdline_compl +cmdline_hist +cmdline_info +comments   
 +xsmp_interact +xterm_clipboard -xterm_save  

Wednesday, November 2, 2011

Ubuntu oneiric on Samsung Series 7 - 700Z5A S03

I recently bought a new Samsung Series 7 Notebook (700Z5A S03) as my replacement for my old Dell Latitude. It ships with a 750 GB hard disk with "8 GB ExpressCache". I replaced the HD by a faster SSD.  In order to replace the disk, one has to open the bottom part of the case. The inside looks like this:

My model came with a Hitachi 7K750-750 inside. That's a usual harddisk. The so called "8 GB ExpressCache" seems to be on the board itself and is still visible as a usual blockdevice on linux after replacing the orignal hard disk. It's identified by hdparm as "Model=SanDisk iSSD P4 8GB, FwRev=SSD 9.14". For an SSD it seems to be quite slow. I use it as a swap device.

# hdparm -t /dev/sdb

 Timing buffered disk reads: 362 MB in  3.02 seconds = 119.98 MB/sec

Ubuntu Oneiric runs out of the box on the Notebook - at least to some extend. The Hybrid  Graphics cannot be used up to now due to lack of driver support. Some folks at are working on that. I also couldn't get the dedicated graphics card disabled by following various instructions on that topic. As soon as the radeon kernel module is loaded (which is required for vgaswitcheroo to be even available) the power consumption of the system is quite high. The lowest power consumption I could achieve so far was by blacklisting the radeon module via modprobe. Idle power consumption, reported by powertop, is about 20-21W. This seems to be quite high as on Windows a idle power consumption of about 15W was reported. This might come from the  dedicated graphic, not being fully disabled by just unloading radeon.
Also the touchpad identified as "PS/2 Elantech Touchpad" by xinput list is not working as a touchpad but just a normal mouse. This is quite annoying as there is no way to disable tab-click. I stuck with  disabling/enabling the touchpad completely so far (xinput set-prop 14 132 0/1 on my device).

The notebook unfortunately cannot be ordered without an operating system. It ships with a Windows 7 Pro DVD labelled "System Recovery Media". Despite the labelling, the DVD acts as a usual Windows installation DVD that can also be installed and activated on other systems (tested on a virtualbox). This at least makes it possible to sell it on Ebay - together with the license sticker from the bottom of the case that can easily be removed by steaming it a bit.

The Touchpad is supported in Kernel 3.2. Even three fingers are supported (can be used to move windows in unity). By default it's quite insensitive though, I had to use my thumb to get the cursor moving. Adding the following lines to /etc/X11/xorg.conf fixes this:

Section "InputClass"
        Identifier "touchpad catchall"
        Driver "synaptics"
        MatchIsTouchpad "on"
        MatchDevicePath "/dev/input/event*"
        Option "FingerLow" "6"
        Option "FingerHigh" "10"
        # Option "VertScrollDelta" "10"

Saturday, October 22, 2011

PDF Mergy - HTML5 PDF Merge Tool

I lately created a simple PDF Merge tool that is now available online. It allows those who are not able or willing to use a command-line to quickly merge PDF files without the need of installing a bloated tool on their commputer.
Files can be added and ordered using a drag and drop interface.

The application is hosted on Google AppEngine, while the actual merging is done on Rackspace CloudServers. After running some benchmarks it turned out that Rackspace CloudServers are much better suited for these kind of Tasks than Amazon EC2 nodes. I'm using a GAE-backed REST queue to distribute merge jobs. To reduce unecessary polling I'm using gae_channel to push queue updates to workers. Using a message system like RabbitMQ would probably be superior but require to run and maintain an additional server - something that I wanted to avoid.
This is the first time I'm using gae_channel on productive enviroment. We'll see if it works well....

Sunday, June 12, 2011

Using Google Appengine Channel API with a Python Client

For a project, I  needed a way to perform long-poll requests with GAE. Just using a normal connection doesn't work because of the 30-Seconds request limit GAE enforces. But starting with Version 1.4 Google released the Google Appengine Channel API which allows long-pull requests with a javascript client library.
It's quite easy to use. create a channel:

from google.appengine.api import channel
token = channel.create_channel(chan_name)

And start sending messages:

channel.send_message(chan_name, msg)

Unfortunately, there is neighter any non-javascript client library for use in desktop applications, nor is there any specifications of the protocol available. So I reverse-engineered the javascript-client and created a python client implementation. Feel free to use it. But be aware that although it seems to work quite well,  it's not well tested yet. I've not yet used it on a real-world application. Be also aware that google could change the underlying protocol without any notice, which would obliviously break this library.

Use it like this:
import gae_channel

channel = gae_channel.Client(token='your channel token')
for msg in chan.messages():
    print msg 
Also have a look at /demo. There's a small demo client available.

$ python 
Your channel name is: fWVwdXvNJP
now run in an other terminal:
python fWVwdXvNJP
I'm now listening for messages, dont close this terminal...

Now in another terminal, run:
$ python fWVwdXvNJP
Enter a message:test

Now you'll see the message appearing in the first terminal.

Download from bitbucket.