Author: Mihai Şucan (ROBO Design)
In the first part of the article we added keyboard shortcuts to a simple paint application. Now we will implement a way to move the pointer inside the drawing canvas using keyboard shortcuts.
At this point, the web application cannot be used entirely via the keyboard; it still requires a pointing device for the actual drawing. Users who really need keyboard-based drawing will most likely have mouse keys activated in their operating system of choice (check this page out for details of the Windows implementation, for example).
For educational and experimentation purposes, we can implement keyboard-based drawing inside our paint tool. Compared to a system-wide mouse keys implementation, this has the advantage of being tied directly to our web application, opening a range of possible improvements applicable only to our use-case.
To implement mouse keys in the paint application we have to add support for extending the application functionality from external code. We must make sure any extended functionality can be accessed from the keyboard shortcuts configuration file.
To make it easier to follow along with the code walkthrough
presented below, download the
full code example and follow along with it as you read the article.
The code discussed in this article is all found inside the app4
directory.
Let's begin by adding support for removable actions to the paint
application. Here is a snippet from the updated Painter
class:
function Painter () {
// ...
/**
* Holds all the removable functionality from the paint application.
*
* @type Object
* @see Painter#actionAdd Add a new action.
* @see Painter#actionRemove Remove an action.
*/
this.actions = {};
// ...
this.actionAdd = function (id, func, overwrite) {
if (typeof id != 'string' || typeof func != 'function' || (this.actions[id] && !overwrite)) {
return false;
}
this.actions[id] = new func(_self);
return this.actions[id] ? true : false;
};
// ...
this.actionRemove = function (id) {
if (!id || !_self.actions[id]) {
return false;
}
if (typeof _self.actions[id].actionRemove == 'function') {
_self.actions[id].actionRemove();
}
delete _self.actions[id];
return true;
};
// ...
};The new code allows us to add new actions, which are in fact functions
that return an object holding any properties and methods you want. Any new
object is added to the Painter actions with the given ID. When
the action is removed, the actionRemove() method is called, if
the action implements it. This allows you to execute any custom "uninstall"
procedures you like.
There is not much we can do with that, for now. Let's tie things together
by allowing the keyboard shortcuts configuration to call methods from
installed actions. This can be done in the ev_keyhandler()
function:
// ...
var gkey = PainterConfig.keys[ev.kid_];
if (!gkey) {
return;
}
ev.kobj_ = gkey;
// Execute the associated action.
if (gkey.action) {
var action = _self.actions[gkey.action];
if (action) {
var func = action[ev.type];
if (typeof func == 'function') {
func(ev);
}
}
}
// ...This update allows us to include an action property for each
configured keyboard shortcut, just like we have the tool
property for tool activation. When the user presses the key combination, the
method having the same name as the event type will be invoked, from the
action object.
With the above code in place, we can now do something really simple like this:
PainterInstance.actionAdd('test', function (app) {
// ...
this.keydown = function (ev) { ... };
this.keypress = function (ev) { ... };
// ...
});Into the PainterConfig object we can add a keyboard shortcut that invokes the test action:
var PainterConfig = {
keys: {
T: { action: 'test', prop1: 'val1' }
}
};From this code you can see that the key T is associated with
the action with an ID of test. Now we've implemented the change
on the ev_keyhandler() function, the keydown(),
keypress() and the keyup() methods from the test
action are invoked when their respective events are fired. Additionally, by
adding the kobj_ property to the DOM Event object, any method
from the test action can access the keyboard shortcut configuration and any
of its properties. This allows us to pass custom parameters to the action
method.
To implement mouse keys support, we need to always store the last mouse location in an object, to allow keyboard-based movements to continue from the same place.
During typical use, the event flow involves several
mousemove events to the desired position, then
mousedown to start drawing, then several mousemove
events until the destination is reached, and finally a mouseup
event for ending the drawing operation. We need to store the state of the
mouse button - if it is down or not. The mouse button is considered to be
down/active between a mousedown and a mouseup
event. This is needed by a mouse keys implementation in order to determine
and synthetically alter the state of the mouse button and generate any
required mouse events when the user presses a key.
Here are the updates needed in the ev_canvas() function:
function Painter () {
// ...
/**
* Holds the last recorded mouse coordinates and the button state (if it's
* down or not).
*
* @private
* @type Object
*/
this.mouse = {x: 0, y: 0, buttonDown: false};
// ...
function ev_canvas (ev) {
// ...
/*
* If the mouse is down already, skip the event.
* This is needed to allow the user to go out of the drawing canvas, release
* the mouse button, then come back and click to end the drawing operation.
* Additionally, this is needed to allow extensions like mouse keys to
* perform their actions during a drawing operation, even when a real mouse
* is used. For example, allow the user to start drawing with the keyboard
* (press 0) then use the real mouse to move and click to end the drawing
* operation.
*/
if (_self.mouse.buttonDown && ev.type == 'mousedown') {
return false;
}
// Don't overwrite any existing x_ / y_ property.
// These properties might be added by other functions.
if (typeof ev.x_ == 'undefined') {
if (typeof ev.layerX != 'undefined') { // Firefox
ev.x_ = ev.layerX;
ev.y_ = ev.layerY;
} else if (typeof ev.offsetX != 'undefined') { // Opera
ev.x_ = ev.offsetX;
ev.y_ = ev.offsetY;
}
// Update the current mouse position only for mouse events.
// Other events do not provide accurate mouse coordinates.
switch (ev.type) {
case 'mousedown':
case 'mousemove':
case 'mouseup':
_self.mouse.x = ev.x_;
_self.mouse.y = ev.y_;
}
}
if (ev.type == 'mousedown') {
_self.mouse.buttonDown = true;
}
// Call the event handler of the active tool.
var func = _self.tool[ev.type];
if (typeof func != 'function') {
return false;
}
res = func(ev);
if (ev.type == 'mouseup') {
_self.mouse.buttonDown = false;
}
};
// ...
};Here we've added a new property object to the Painter class
- the mouse property - to hold information about the mouse
coordinates and the button state. The updated ev_canvas()
function tries to determine the mouse coordinates if the DOM Event object
does not already have the two x_ and y_
properties. Both properties will be added by the mouse keys implementation.
The new ev_canvas() function updates the mouse coordinates and
the button state, depending on each event type.
The above completes the updates necessary to the main code - check out the fourth index.js update.
In the current tools, we have set an inner variable named
started to true on mousedown, to track if the mouse
button is being clicked, so their mousemove event handler can
perform the drawing operation. We can drop that now, and reuse the new
mouse.buttonDown property from the application object. Check out
the minor changes in the fourth tools.js
update.
The Painter API provides us with sufficient pointer
information, and an extension mechanism for adding custom keyboard shortcuts
- we can now start implementing mouse keys support. Let's create a new mousekeys.js file, which is added into the index.html file. Here is the basic code structure for
mouse keys:
/**
* @class The MouseKeys action.
*
* @param {Painter} app Reference to the main paint application object.
*/
function PaintMouseKeys (app) {
var canvas = app.buffer.canvas,
mouse = app.mouse;
/**
* Holds the current mouse movement speed in pixels.
*
* @private
* @type Number
*/
var speed = 1;
/**
* Holds the current mouse movement acceleration, taken from the
* configuration.
*
* @private
* @type Number
* @see PainterConfig.mousekeys_accel The mouse keys acceleration setting.
*/
var accel = PainterConfig.mousekeys_accel;
if (!canvas || !canvas.parentNode) {
return false;
}
/**
* Holds a reference to the DOM element representing the pointer on top of the
* canvas element.
*
* @private
* @type Element
*/
var pointer = document.createElement('div');
if (!pointer) {
return false;
}
pointer.id = 'mousekeysPointer';
pointer.style.display = 'none';
canvas.parentNode.appendChild(pointer);
function pointerMousemove (ev) { ... };
function dispatch (type, ev) { ... };
this.keydown = function (ev) { ... };
this.keypress = function (ev) { ... };
this.actionRemove = function () { ... };
canvas.addEventListener('mousemove', pointerMousemove, false);
};The PaintMouseKeys class object holds the main methods:
keydown(), keypress() and
actionRemove(). The two private variables, speed
and accel, hold information about the speed and the
acceleration of the mouse pointer. The longer the user holds down a key, the
faster the pointer will move. While a key is being held down the
speed value increases based on the accel
value.
The pointer DOM element is added in the parent node of the
<canvas> element because it is needed for providing the
user with a visual indicator of where the pointer is moving. Since this
script cannot move the real mouse pointer, which might not even exist, we
need our own virtual pointer. The pointerMousemove() updates
the position of the pointer element when the mouse moves:
/**
* Track the virtual pointer coordinates, by updating the position of the
* pointer element. This allows the keyboard users to see where
* they moved the virtual pointer.
*
* @param {Event} ev The DOM Event object.
*/
function pointerMousemove (ev) {
if (typeof ev.x_ == 'undefined' || !ev.kobj_ || !ev.kobj_.action || ev.kobj_.action != 'mousekeys') {
if (pointer.style.display == 'block') {
pointer.style.display = 'none';
}
return;
}
pointer.style.top = ev.y_ + 'px';
pointer.style.left = ev.x_ + 'px';
};The function above will only update the pointer location if the
mousemove event contains the x_ and
y_ coordinates, and only if the event was generated by the
mousekeys action. If the event is not a synthetic one, the
pointer element is hidden. We only want it visible when the
keyboard is used.
You need to decide how you want the users to interact with the drawing canvas using their keyboard. I decided for this demo to use the number keys. Therefore, key 2 should move the cursor to the south (towards the bottom of the image), key 4 to the west (left), key 6 to the east (right), and key 8 should move the pointer towards the north (top of the image). The rest of the keys, 1, 3, 7 and 9, should move the pointer in diagonal directions (eg 1 for south-west). If the user also holds down the Shift key in addition to a direction, the mouse pointer moves faster.
Having defined what we want for the pointer movement (the
mousemove event) we need a key to generate the
mousedown and the mouseup events. For this demo,
when the user presses the 0 key the synthetic
mousedown event is dispatched. Pressing it again would generate
the mouseup event. As such, this key allows the user to toggle
the "mouse is down" state to true or false, by alternating the two events.
When the mouse is down, any mousemove event will perform the
drawing operation associated with the active tool. The drawing tools do not
need any code change since the mouse events are all fired synthetically.
Another important technical point is that we do not want to modify the
script when we change the keyboard shortcuts. Here is the modification for
the PainterConfig object:
var PainterConfig = {
/**
* The mouse keys movement acceleration.
*
* @type Number
* @see PaintMouseKeys The MouseKeys extension.
*/
mousekeys_accel: 0.1,
/**
* Keyboard shortcuts associated to drawing tools and other actions.
*
* @type Object
* @see PaintTools The object holding all the drawing tools.
*/
keys: {
0: { action: 'mousekeys', param: 'Toggle' },
1: { action: 'mousekeys', param: 'SouthWest' },
2: { action: 'mousekeys', param: 'South' },
3: { action: 'mousekeys', param: 'SouthEast' },
4: { action: 'mousekeys', param: 'West' },
6: { action: 'mousekeys', param: 'East' },
7: { action: 'mousekeys', param: 'NorthWest' },
8: { action: 'mousekeys', param: 'North' },
9: { action: 'mousekeys', param: 'NorthEast' },
L: { tool: 'line' },
P: { tool: 'pencil' },
R: { tool: 'rect' }
}
};We have added the virtual pointer acceleration we want, and the new
keyboard shortcuts for the mousekeys action. The
param property specifies the action what to do. This is where
adding the kobj_ property to the DOM Event object in the
ev_keyhandler() becomes useful. The keydown() and
keypress() methods from the mouse keys implementation can use
the param property to determine the mouse movement direction
associated to the keyboard shortcut. As such, we do not include any of the
keyboard shortcuts at all in the implementation, only in the configuration
file.
When the Shift key is held down, the system generates different keys for the numbers on the number pad. We need to take them into consideration in the keyboard shortcuts configuration:
lib.extend(PainterConfig.keys, {
'Shift Insert': PainterConfig.keys['0'],
'Shift End': PainterConfig.keys['1'],
'Shift Down': PainterConfig.keys['2'],
'Shift PageDown': PainterConfig.keys['3'],
'Shift Left': PainterConfig.keys['4'],
'Shift Right': PainterConfig.keys['6'],
'Shift Home': PainterConfig.keys['7'],
'Shift Up': PainterConfig.keys['8'],
'Shift PageUp': PainterConfig.keys['9']
});We use the extend() function to add the new keys, as
duplicates of the numbers. With this we have completed the update needed for
the configuration file.
Now we will continue with the MouseKeys action implementation. Here is
the code for the keydown() event handler:
this.keydown = function (ev) {
speed = 1;
accel = PainterConfig.mousekeys_accel;
if (pointer.style.display == 'none') {
pointer.style.display = 'block';
pointer.style.top = mouse.y + 'px';
pointer.style.left = mouse.x + 'px';
}
if (!ev || !ev.kobj_ || ev.kobj_.param != 'Toggle') {
return false;
}
var type = mouse.buttonDown ? 'mouseup' : 'mousedown';
dispatch(type, ev);
return true;
};The keydown() method always resets the speed and the mouse
acceleration to the default values. This ensures that the speed gained from
the previous key press is not reused in another key press for mouse
movement. Otherwise, the mouse movement would become too fast quite
quickly.
The pointer element is always made visible, such that the
user can see where the virtual pointer is located when he/she starts using
the keyboard for drawing.
The kobj_ property object is used to determine which action
needs to be performed. If the keyboard shortcut object holds the
param property with the value Toggle, we perform
the action of the key 0, as discussed above. We dispatch
a mouseup event if the mouse button is down, otherwise we
dispatch the mousedown event. The mouse.buttonDown
boolean property is the one being updated by the ev_canvas()
function as previously described.
The implementation for the rest of the number keys used for dispatching
synthetic mousemove events is held in the
keypress() event handler in the mousekeys
action:
this.keypress = function (ev) {
if (!ev || !ev.kobj_ || !ev.kobj_.param) {
return false;
}
if (ev.shiftKey) {
speed += speed * accel * 3;
} else {
speed += speed * accel;
}
var w = canvas.width,
h = canvas.height,
x = mouse.x,
y = mouse.y,
step = Math.ceil(speed);
switch (ev.kobj_.param) {
case 'SouthWest':
x -= step;
y += step;
break;
case 'South':
y += step;
break;
case 'SouthEast':
x += step;
y += step;
break;
case 'West':
x -= step;
break;
case 'East':
x += step;
break;
case 'NorthWest':
x -= step;
y -= step;
break;
case 'North':
y -= step;
break;
case 'NorthEast':
x += step;
y -= step;
break;
default:
return false;
}
if (x < 0) {
x = 0;
} else if (x > w) {
x = w;
}
if (y < 0) {
y = 0;
} else if (y > h) {
y = h;
}
mouse.x = x;
mouse.y = y;
dispatch('mousemove', ev);
return true;
};The implementation for the movement keys has been put in the
keypress() event handler because this event is always repeated
while the user holds down a key. The keydown event is
dispatched only once.
The code is rather simple: increase the movement speed based on the
acceleration (with a variation if the Shift key is down), and
update the mouse coordinates based on the param property of the
keyboard shortcut object. At the end, the synthetic mousemove
event is dispatched. The mouse coordinates are those automatically updated
by the ev_canvas() function.
Here is the code that performs the actual synthetic event dispatching:
function dispatch (type, ev) {
var ev_new = document.createEvent('MouseEvents');
ev_new.initMouseEvent(type,
ev.bubbles, ev.cancelable,
ev.view, 0,
0, 0,
0, 0,
ev.ctrlKey, ev.altKey,
ev.shiftKey, ev.metaKey,
0, ev.relatedTarget);
// Make sure the new coordinates are passed to the event handlers.
ev_new.x_ = mouse.x;
ev_new.y_ = mouse.y;
// Make sure the event handlers can check this is a synthetic event.
// This is needed by the pointerMousemove() function.
ev_new.keyCode_ = ev.keyCode_;
ev_new.key_ = ev.key_;
ev_new.kid_ = ev.kid_;
ev_new.kobj_ = ev.kobj_;
canvas.dispatchEvent(ev_new);
};This function creates a new mouse event of the given type.
The new mouse event will share several properties (eg the active key
modifiers) with the current keyboard event. Several new properties are
added: the new mouse coordinates and the keyboard-related properties. The
new event is dispatched to the buffer <canvas>
element.
When the real mouse is used, typically the following execution chain is activated:
ev_canvas() event handler is invoked, which determines the
mouse coordinates on the <canvas> element and updates
the mouse object to hold them and the button state.When the mouse keys are used, typically the following execution chain is activated:
KeyboardEventListener class instance from the minimal
JavaScript library.ev_keyhandler() function from the
paint application, which in turn invokes the event handler within the
mousekeys action.ev_canvas() function and by the active tool.mousemove events, the
pointerMousemove() function updates the location of the
pointer element.To wrap it all up, we include the actionRemove() method and
perform the actual addition of the action in the Painter
instance:
function PaintMouseKeys (app) {
// ...
/**
* Handles action removal. This will remove the pointer DOM element and the
* canvas event listener.
*/
this.actionRemove = function () {
canvas.parentNode.removeChild(pointer);
canvas.removeEventListener('mousemove', pointerMousemove, false);
};
};
window.addEventListener('load', function () {
// Add the MouseKeys action to the Painter instance.
if (window.PainterInstance) {
PainterInstance.actionAdd('mousekeys', PaintMouseKeys);
}
}, false);That's all! Go ahead and test the updated application. Also, make sure you check out the updated documentation, generated from the source code.
In the third and final part of the article we will look into the cross-browser compatibility layer we use for dealing with the browser differences in the DOM keyboard events handling.
If you want to learn more, stick around at DevOpera and also check the upcoming changes to PaintWeb - the open-source web application this article is based on.