PaintWeb in Internet Explorer

26 June 2010, 20:08

Hello world!

Three days ago we received great news from Microsoft: the third platform preview release of Internet Explorer 9 includes support for the Canvas 2D Context API. Canvas comes into IE 9 together with numerous other platform improvements.

These days I have been working on my open source project, PaintWeb. I am quite excited to see it running quite well in MSIE 9.

Go ahead and try the PaintWeb demo page in Internet Explorer 9!

What works? Almost everything. Known issues:

• The globalCompositeOperation property is not yet supported by MSIE 9. The Color mixer visualization looks a bit weird because of this.
• If you change the canvasElem.style.width/height properties the Canvas element is cleared. This is wrong and breaks the implementation of image load and zoom. Check a minimal test case.
• The PaintWeb integration into TinyMCE demo page fails to work. It seems that TinyMCE 3.2.5 fails to work properly in MSIE 9. I shall upgrade it to the latest version, and thus update the PaintWeb plugin as well.

You may also check the PaintWeb change log, for more details.

Any feedback is welcome!

Published in: canvas, html5, microsoft, msie, paintweb, tinymce.

HTML5 demo: Video and Canvas

12 March 2010, 16:57

Hello everyone!

In the latest article I wrote about SVG and Canvas I included a small HTML5 demo which renders a color histogram using Canvas, analyzing any image element. Knowing that the Canvas API allows developers to also read video frames, I wanted to test how fast can JavaScript and Canvas render a color histogram while the video plays.

I changed the initial script so that now it works with HTML5 videos. Here is the result:

In this demo I include a small video and a big HD trailer - just out of curiosity I wanted to see something that barely runs within the browser. The demo works with Opera 10.5 and Firefox 3.6 - tested on Linux and Windows. Currently, Chromium 5 on Linux crashes when I try to load the page.

I did spend quite some time testing various improvements to the performance of the script. I would note that changing the script to not access the DOM elements and properties directly, does not bring any important improvements in execution speed. However, the change to use bitwise operations did make a difference. Further improvements would come if I would inline functions and remove some of the options. Still, any ideas on how to further improve the performance of the demo would be welcome.

Working on this HTML5 demo I also explored a new API available in Firefox 3.5: Web Workers. As I expected, the result of adding a web worker to this use-case is not something I am happy with. In a web worker I cannot access the video, nor can I access any of the canvas elements. I could move only a really small part of the code into a worker, and the performance penalty of sending the pixels in a message from the main thread to the worker thread is too significant. In the end, it runs quite slower than the script which does not use any web worker.

26th of June update: Opera 10.6 beta 1 was released a few days ago. This release brings support for Web Workers and it is really great to see that my Web Workers demo works in Opera as well.

Any feedback is appreciated.

Published in: canvas, demos, firefox, html5, opera, video.

New article: SVG or Canvas?

17 February 2010, 14:30

Just a quick note to people around here: Opera Software has recently published a new article I wrote: SVG or Canvas? Choosing between the two. You can also read the article on my server. As usual, thanks go to Chris Mills for reviewing and providing feedback for improvements to the article.

In unrelated news, things are going fine around here. A new web site I've been working on will be released to the public pretty soon. More cool stuff will come. ;)

Published in: canvas, opera, svg, tutorials.

Canvas text rendering

23 June 2009, 19:51

Hello again!

The HTML 5 Canvas specification defines two important methods for text rendering: fillText() and strokeText(). You also have a measureText() and the associated text-related properties: font, textAlign and textBaseline.

The Canvas Text API is only implemented in Webkit (Safari and Chrome) and in Gecko 1.9.1+ (Firefox 3.5+).

If you want to render text in your Canvas element in Gecko 1.9.0 (Firefox 3.0) you can use their proprietary Canvas text rendering API which is now, obviously, deprecated in favour of the standardized API. They provided the following methods: mozDrawText(), mozPathText(), mozTextAlongPath() and mozMeasureText(). To style the text you only have the mozTextStyle property.

In PaintWeb I use the mozPathText() method when the standard API is not available.

Opera does not support the Canvas Text API. I learned that the drawImage() 2D context method allows the drawing of SVG document as well - this works only in Opera. I implemented the text tool by adding a new minimal SVG document which contains a <text> element. This was updated and drawn in the Canvas element in sync with user interaction. However, after I completed the implementation I found several bugs:

• SVG redraw issues. When you update the text styling properties, or when you update the text itself, sometimes Opera fails to entirely redraw the SVG document, irrespective of the SVG being visible or not.
• memory leaks (test case). For some unfortunate reason, each drawImage(svgDocument) leaks some amount of memory. Got a crasher with this, and a system freeze after filling my physical memory (1 GB) and the swap (1 GB). ;)
• security violations (test case). Opera considers the SVG document as being an external resource, thus it marks the Canvas as being "dirty" once drawImage(svgDocument) is invoked. You can no longer read pixels using getImageData(), nor can you use the toDataURL() method. This broke the PaintWeb history mechanism, the selection tool, and the "image save" option.

The first two issues I said I can live with, but not with the security violations. Thus, I have disabled the text tool in Opera. I have reported the last two bugs to Opera with the associated minimal test cases.

If you are interested to render text, you can even attempt server-side "hacks". You could make a server-side script which renders text, and you can then draw it in your Canvas with an image element. This, however, defeats the purpose of PaintWeb - I want it to be a "pure" client-side Web application.

Lastly, you can implement "vector drawing" of text using a client-side font definition in some format, and then render it with basic paths in Canvas. Again, this is beyond the purpose of PaintWeb. Opera will implement Canvas Text some day. :)

For further details about attempts at rendering text when the standard Canvas Text API is not available, please read the rendering text blog post wrote by Christopher Clay at the end of 2006.

Published in: bugs, canvas, gecko, gsoc2009, html5, opera, paintweb, svg, text, webkit.

PaintWeb performance

28 May 2009, 11:55

Hello everyone!

This week I have completed my work on performance testing and improvements for PaintWeb on the OLPC XO laptop.

During testing it became obvious that something other than the actual Canvas painting was very slow on the XO. The main performance culprit is that the default Gecko-based browser is configured to render pages using 134 DPI instead of the default 96 DPI. Generally web browsers render pages using 96 DPI. If the XO web browser would do the same the texts and the images would be far too small - the XO display is an odd 200 DPI screen perceived as 134 DPI.

PaintWeb's drawing performance was hugely affected by the bilinear scaling of the Canvas elements being done by the browser on the XO. When I configured the browser to render the page using 96 DPI, the web application became a lot more responsive.

Martin Langhoff, my mentor, got in contact with Robert O'Callahan from Mozilla. He provided us with lots of help in finding a solution for the performance issue.

We did think about having a CSS property to change the DPI only for the Canvas elements, or a different CSS property to disable scaling, or some proprietary API for changing the DPI on a single page. None of these are good ideas, because they allow web developers to start coding for specific DPIs - this is not desired.

Gecko scales pages using integer scaling factors - that's 1, 2, 3, etc - it doesn't use floating point numbers. In a normal Gecko build the scaling factor for 134 DPI is 1 - because 134 / 96 = 1, so you do not get any scaling. You can have a scaling factor of 2 or higher if you go above 192 DPI.

Gecko is patched on the XO in a way that it forces the browser to scale pages using floating-point scaling factors as well. Therefore, for 134 DPI the pages are scaled and they look really good on the XO screen.

The final solution which I implemented into PaintWeb is to simply scale down the Canvas elements in my document. If I accurately scale down the elements, Gecko is sufficiently optimized to cancel any scaling and you do not notice any performance impact. This works really great.

In Gecko 1.9.1 (Firefox 3.5) I can detect the DPI used for rendering the page with CSS 3 Media Queries. I use this in PaintWeb. However, the XO only has Gecko 1.9.0 for now, so I cannot determine the DPI. I am forced to do user agent sniffing to check if the browser runs on the OLPC XO. If it does, then I scale down the Canvas elements using a different way of calculating the scale-down factor - because Gecko is patched - and I always consider the page is render using 134 DPI. Fun, huh? ;)

On Opera, on the XO, I did all my testing using 100% zoom level. It ran much better than Gecko, for obvious reasons (no scaling, yay). Once I fixed the Gecko scaling issue, Opera came second. For some reason Canvas draws much faster in Gecko than in Opera on the OLPC XO.

Opera cannot render pages using different DPI values other than 96. People use zoom, so, for consistency, I use an old trick to measure the zoom level (thanks Arve). Based on this I scale down the Canvas elements. For some zoom levels, like 200%, the scaling is cancelled and PaintWeb works better. Unfortunately, Opera does not allow non-integer pixel values, thus the scaling-down is generally not effective...

Another important performance improvement in PaintWeb is the use of timer-based canvas drawing. This means that mouse move events are either cancelled or coalesced into one. For example, redrawing a polygon with a lot of points for every mouse move is very slow. The tools in PaintWeb use timers to update the canvas every few milliseconds. This approach makes PaintWeb feel faster.

Lastly, I now avoid going into the global scope, for things like Math.round or such. The importance of this change is reduced by the fact the JavaScript that runs is not very intensive - not too much code is executed for each mouse move event. Such changes become more important the more code you run. This will be important for the color space visualization I have.  

The loading performance will improve greatly once I will make a packager for PaintWeb. Additionally, I will continue to constantly check the overall performance of the web application on the OLPC XO.

Go ahead and try PaintWeb from SVN trunk. Lots of thanks to Robert for his great help and to Martin for his assistance and for finding the Gecko patches.

Currently I am working on the new user interface, stay tuned!

Update May 31, 2009: Just published a page on the OLPC wiki about the HTML Canvas performance on the OLPC XO laptops. The page includes code snippets explaining how to work-around the scaling issue.

Published in: canvas, css, dpi, gecko, gsoc2009, moodle, olpc, opera, paintweb, performance.

PaintWeb code refactoring and more

14 May 2009, 11:09

Hello everyone!

I have been working on the PaintWeb code refactoring and now I am nearing completion. The initial PaintWeb 0.5 alpha code was more of a demo - it was all in a single big script. I have now added jsdoc comments almost everywhere and I did split the code into multiple files - per tools, per extensions, per language, and more. I have also made important changes to the API. Now any external code can easily add/remove tools, extensions and keyboard shortcuts.

For more developer-related information please read the latest forum thread I posted on the Moodle forums.

For teachers and potential users of PaintWeb inside Moodle, I have prepared a list of questions on how you would use the paint tool in Moodle.

Martin, my mentor, suggested early in my GSOC application process to also apply for the OLPC Contributors program. So I did, and my project was accepted.

Even if the OLPC XO has a slow CPU by today's expectations, it's only 400 Mhz, the system works quite nicely. It has 256 MB of RAM and 1GB of disk capacity. The Sugar interface and the activities provided are amazing. People who hear about these laptops do not know to appreciate the numerous doors such laptops open, doors to knowledge, for all those children who receive them. They help a lot in learning about computing, maths, music, and more.

The Sugar interface is quite well thought-out. I like the concept of having the neighbourhood, group, home and activity views.

The default browser, is some Python application embedding Gecko - on par with Firefox 3.0. The performance of the browser is lacking. Opera 10 alphas start much faster and feel snappier. The paint tool feels sluggish as well.

The Gnash plugin is more of a problem rather than a solution. I installed Flash Player 10, which is sluggish, but at least it works. The system can play Youtube high-quality videos and even uncompressed DVD videos, with Mplayer over the wireless connection. Flash Player cannot play Youtube videos.

Battery life is good - I can use it about three hours without any problems.

Since last week I have been working on the performance of the PaintWeb application, with the OLPC XO-1 laptop. After several tests, I have managed to improve things sufficiently such that the paint tool is now usable in Opera 10 on the XO. Unfortunately, in Browse.xo it's not, at least not by default.

The main performance culprit affecting PaintWeb on the XO is their use of layout.css.dpi. Gecko allows users to change the DPI used for rendering Web pages, in order to makes fonts and images smaller or bigger. So, on the XO the browser is set to use DPI 134, instead of DPI 96. This makes the fonts and images render bigger - with DPI 96 they would all be way too small. PaintWeb and all the pages feel much slower because Gecko performs bilinear image resampling.

When I set layout.css.dpi to 96, drawing in PaintWeb becomes real-time. I was amazed to see it works so well. It's like on my desktop computer. And ... it's even faster than in Opera 10. ;)

If you want, check out the performance tests yourself. Spoiler: Webkit is the fastest and Gecko is the slowest when running synthetic tests. Obviously, more performance tests will come - these are only limited to the pencil tool and to the main ev_canvas() event handler from PaintWeb.

Next on my of list things to do is a new GUI and a packager for the entire code. Loading PaintWeb is now slower due to the amount of code comments and the increasing number of files. The packager will compress/minify all the files into a single one.

That's all for now. Any feedback is welcome!

Published in: canvas, css, dpi, gecko, gsoc2009, moodle, olpc, opera, paintweb, performance, webkit.

Announcing PaintWeb

6 November 2008, 20:39

Today I finally managed to upload and release my latest project: PaintWeb. Marius helped me with designing the GUI and with other suggestions.

PaintWeb is a client-side Web application which allows users to draw online. It makes use of some newer Web technologies, mainly the HTML 5 Canvas 2D context API. Currently, it's in its infancy, but with lots of work planned ahead.

The major decision for me was to release this project as open-source, under GPL v3. The project is now hosted on the Google Code servers.

Please contribute with feedback, bug reports and even code - volunteers are welcome!

Lots of bugs in the Web application are already known, nonetheless that shouldn't stop you from reporting them.

The Web application works in the latest versions of Opera, Firefox, Safari and Konqueror. Obviously, it also works with any Gecko and WebKit based Web browsers.

Nightly builds of Firefox 3.1 and SVN trunk builds of WebKit have the best support for this Web application.

I'd like to mention that the greatest surprise to me was that the new Konqueror 4 has its own Canvas implemention. They've done a really great job!

I am looking forward to publish more information about the project and to continue work on it.

Update 3 days later: Back in september we presented PaintWeb at a local university-organised seminar. Today we have updated the presentation and we translated it to English as well. Go ahead and download the English or Romanian presentation.

Published in: canvas, gpl, html5, open-source, paintweb, projects, talks.

Linux, Konqueror 4 and more

5 November 2008, 19:44

Sunday I upgraded my Ubuntu installation to the new version: Ubuntu 8.10. Almost three years ago I switched to using Linux exclusively. I haven't reinstalled Ubuntu since then. I always upgrade my system every 6 months, and then I spend a few days fixing post-upgrade issues. It's a pleasure. ;)

I am working on a new painting Web application. The greatest surprise to me was that the new Konqueror 4 implements <canvas>, and it does this theoretically better than Opera. I will make my Web application public and you will see this. Amazing and very quick work done by the Konqueror developers. Congratulations, guys!

Note: I initially thought that the Konqueror implementation of Canvas is just a copy of the code from Webkit, but it's not. They wrote their own code, which is great.

On a related note, I have published a new page with some of my Linux configuration files.

Published in: canvas, configuration, html5, kde, konqueror, linux, ubuntu.