Canvas text rendering

23 June 2009, 19:51

Hello again!

The HTML 5 Canvas specification defines two important methods for text rendering: fillText() and strokeText(). You also have a measureText() and the associated text-related properties: font, textAlign and textBaseline.

The Canvas Text API is only implemented in Webkit (Safari and Chrome) and in Gecko 1.9.1+ (Firefox 3.5+).

If you want to render text in your Canvas element in Gecko 1.9.0 (Firefox 3.0) you can use their proprietary Canvas text rendering API which is now, obviously, deprecated in favour of the standardized API. They provided the following methods: mozDrawText(), mozPathText(), mozTextAlongPath() and mozMeasureText(). To style the text you only have the mozTextStyle property.

In PaintWeb I use the mozPathText() method when the standard API is not available.

Opera does not support the Canvas Text API. I learned that the drawImage() 2D context method allows the drawing of SVG document as well - this works only in Opera. I implemented the text tool by adding a new minimal SVG document which contains a <text> element. This was updated and drawn in the Canvas element in sync with user interaction. However, after I completed the implementation I found several bugs:

• SVG redraw issues. When you update the text styling properties, or when you update the text itself, sometimes Opera fails to entirely redraw the SVG document, irrespective of the SVG being visible or not.
• memory leaks (test case). For some unfortunate reason, each drawImage(svgDocument) leaks some amount of memory. Got a crasher with this, and a system freeze after filling my physical memory (1 GB) and the swap (1 GB). ;)
• security violations (test case). Opera considers the SVG document as being an external resource, thus it marks the Canvas as being "dirty" once drawImage(svgDocument) is invoked. You can no longer read pixels using getImageData(), nor can you use the toDataURL() method. This broke the PaintWeb history mechanism, the selection tool, and the "image save" option.

The first two issues I said I can live with, but not with the security violations. Thus, I have disabled the text tool in Opera. I have reported the last two bugs to Opera with the associated minimal test cases.

If you are interested to render text, you can even attempt server-side "hacks". You could make a server-side script which renders text, and you can then draw it in your Canvas with an image element. This, however, defeats the purpose of PaintWeb - I want it to be a "pure" client-side Web application.

Lastly, you can implement "vector drawing" of text using a client-side font definition in some format, and then render it with basic paths in Canvas. Again, this is beyond the purpose of PaintWeb. Opera will implement Canvas Text some day. :)

For further details about attempts at rendering text when the standard Canvas Text API is not available, please read the rendering text blog post wrote by Christopher Clay at the end of 2006.

Published in: bugs, canvas, gecko, gsoc2009, html5, opera, paintweb, svg, text, webkit.

PaintWeb performance

28 May 2009, 11:55

Hello everyone!

This week I have completed my work on performance testing and improvements for PaintWeb on the OLPC XO laptop.

During testing it became obvious that something other than the actual Canvas painting was very slow on the XO. The main performance culprit is that the default Gecko-based browser is configured to render pages using 134 DPI instead of the default 96 DPI. Generally web browsers render pages using 96 DPI. If the XO web browser would do the same the texts and the images would be far too small - the XO display is an odd 200 DPI screen perceived as 134 DPI.

PaintWeb's drawing performance was hugely affected by the bilinear scaling of the Canvas elements being done by the browser on the XO. When I configured the browser to render the page using 96 DPI, the web application became a lot more responsive.

Martin Langhoff, my mentor, got in contact with Robert O'Callahan from Mozilla. He provided us with lots of help in finding a solution for the performance issue.

We did think about having a CSS property to change the DPI only for the Canvas elements, or a different CSS property to disable scaling, or some proprietary API for changing the DPI on a single page. None of these are good ideas, because they allow web developers to start coding for specific DPIs - this is not desired.

Gecko scales pages using integer scaling factors - that's 1, 2, 3, etc - it doesn't use floating point numbers. In a normal Gecko build the scaling factor for 134 DPI is 1 - because 134 / 96 = 1, so you do not get any scaling. You can have a scaling factor of 2 or higher if you go above 192 DPI.

Gecko is patched on the XO in a way that it forces the browser to scale pages using floating-point scaling factors as well. Therefore, for 134 DPI the pages are scaled and they look really good on the XO screen.

The final solution which I implemented into PaintWeb is to simply scale down the Canvas elements in my document. If I accurately scale down the elements, Gecko is sufficiently optimized to cancel any scaling and you do not notice any performance impact. This works really great.

In Gecko 1.9.1 (Firefox 3.5) I can detect the DPI used for rendering the page with CSS 3 Media Queries. I use this in PaintWeb. However, the XO only has Gecko 1.9.0 for now, so I cannot determine the DPI. I am forced to do user agent sniffing to check if the browser runs on the OLPC XO. If it does, then I scale down the Canvas elements using a different way of calculating the scale-down factor - because Gecko is patched - and I always consider the page is render using 134 DPI. Fun, huh? ;)

On Opera, on the XO, I did all my testing using 100% zoom level. It ran much better than Gecko, for obvious reasons (no scaling, yay). Once I fixed the Gecko scaling issue, Opera came second. For some reason Canvas draws much faster in Gecko than in Opera on the OLPC XO.

Opera cannot render pages using different DPI values other than 96. People use zoom, so, for consistency, I use an old trick to measure the zoom level (thanks Arve). Based on this I scale down the Canvas elements. For some zoom levels, like 200%, the scaling is cancelled and PaintWeb works better. Unfortunately, Opera does not allow non-integer pixel values, thus the scaling-down is generally not effective...

Another important performance improvement in PaintWeb is the use of timer-based canvas drawing. This means that mouse move events are either cancelled or coalesced into one. For example, redrawing a polygon with a lot of points for every mouse move is very slow. The tools in PaintWeb use timers to update the canvas every few milliseconds. This approach makes PaintWeb feel faster.

Lastly, I now avoid going into the global scope, for things like Math.round or such. The importance of this change is reduced by the fact the JavaScript that runs is not very intensive - not too much code is executed for each mouse move event. Such changes become more important the more code you run. This will be important for the color space visualization I have.  

The loading performance will improve greatly once I will make a packager for PaintWeb. Additionally, I will continue to constantly check the overall performance of the web application on the OLPC XO.

Go ahead and try PaintWeb from SVN trunk. Lots of thanks to Robert for his great help and to Martin for his assistance and for finding the Gecko patches.

Currently I am working on the new user interface, stay tuned!

Update May 31, 2009: Just published a page on the OLPC wiki about the HTML Canvas performance on the OLPC XO laptops. The page includes code snippets explaining how to work-around the scaling issue.

Published in: canvas, css, dpi, gecko, gsoc2009, moodle, olpc, opera, paintweb, performance.

PaintWeb code refactoring and more

14 May 2009, 11:09

Hello everyone!

I have been working on the PaintWeb code refactoring and now I am nearing completion. The initial PaintWeb 0.5 alpha code was more of a demo - it was all in a single big script. I have now added jsdoc comments almost everywhere and I did split the code into multiple files - per tools, per extensions, per language, and more. I have also made important changes to the API. Now any external code can easily add/remove tools, extensions and keyboard shortcuts.

For more developer-related information please read the latest forum thread I posted on the Moodle forums.

For teachers and potential users of PaintWeb inside Moodle, I have prepared a list of questions on how you would use the paint tool in Moodle.

Martin, my mentor, suggested early in my GSOC application process to also apply for the OLPC Contributors program. So I did, and my project was accepted.

Even if the OLPC XO has a slow CPU by today's expectations, it's only 400 Mhz, the system works quite nicely. It has 256 MB of RAM and 1GB of disk capacity. The Sugar interface and the activities provided are amazing. People who hear about these laptops do not know to appreciate the numerous doors such laptops open, doors to knowledge, for all those children who receive them. They help a lot in learning about computing, maths, music, and more.

The Sugar interface is quite well thought-out. I like the concept of having the neighbourhood, group, home and activity views.

The default browser, is some Python application embedding Gecko - on par with Firefox 3.0. The performance of the browser is lacking. Opera 10 alphas start much faster and feel snappier. The paint tool feels sluggish as well.

The Gnash plugin is more of a problem rather than a solution. I installed Flash Player 10, which is sluggish, but at least it works. The system can play Youtube high-quality videos and even uncompressed DVD videos, with Mplayer over the wireless connection. Flash Player cannot play Youtube videos.

Battery life is good - I can use it about three hours without any problems.

Since last week I have been working on the performance of the PaintWeb application, with the OLPC XO-1 laptop. After several tests, I have managed to improve things sufficiently such that the paint tool is now usable in Opera 10 on the XO. Unfortunately, in Browse.xo it's not, at least not by default.

The main performance culprit affecting PaintWeb on the XO is their use of layout.css.dpi. Gecko allows users to change the DPI used for rendering Web pages, in order to makes fonts and images smaller or bigger. So, on the XO the browser is set to use DPI 134, instead of DPI 96. This makes the fonts and images render bigger - with DPI 96 they would all be way too small. PaintWeb and all the pages feel much slower because Gecko performs bilinear image resampling.

When I set layout.css.dpi to 96, drawing in PaintWeb becomes real-time. I was amazed to see it works so well. It's like on my desktop computer. And ... it's even faster than in Opera 10. ;)

If you want, check out the performance tests yourself. Spoiler: Webkit is the fastest and Gecko is the slowest when running synthetic tests. Obviously, more performance tests will come - these are only limited to the pencil tool and to the main ev_canvas() event handler from PaintWeb.

Next on my of list things to do is a new GUI and a packager for the entire code. Loading PaintWeb is now slower due to the amount of code comments and the increasing number of files. The packager will compress/minify all the files into a single one.

That's all for now. Any feedback is welcome!

Published in: canvas, css, dpi, gecko, gsoc2009, moodle, olpc, opera, paintweb, performance, webkit.