Alp Toker

Improved HTML5 canvas support

Brent Fulgham has been merging Cairo graphics backend features from the Adobe Apollo/AIR branch of WebKit (#16558, #16577, #15382). The Adobe developers have been cooperative and their code is well-written — hopefully they’ll start merging their own work soon. This puts the graphics backend a couple of weeks ahead of schedule (the original target was GNOME 2.24):

Acid2

Luca Bruno has provided the last (#16365) in a series of fixes to get the Acid2 smiley face rendering correctly. This should help dispel rumours that the WebKit/GTK+ team is dropping acid.

Image decoder enhancements

Google engineers have contributed a handful of improvements (#15974, #16169) to WebKit’s image decoders, which will be shared between the GTK+ and Android ports.

Mobile features designed for Android can also now be easily enabled in the GTK+ port (eg. LOW_BANDWIDTH_DISPLAY support, r28960). It’s great to see cooperation on features like this.

Trunk open for Maemo/Hildon

The Maemo/Hildon mobile platform (used in Nokia internet tablets and Ubuntu Mobile) is now an official component of the GTK+ port. This means that these libraries can be used directly in WebKit instead of being maintained out of tree.

The JavaScript engine has seen recent optimizations which bring it further ahead of the stock browser shipped in OS2008 for the N800/N810 devices. Check out any of the freely available JS/AJAX benchmarks if you’re interested in performance.

GtkPrint

Initial printing support (#15576) has landed. Cairo’s paginated surface API lacks some features we need to implement this fully. I’ve posted a proposal for new API that will be useful in matching the print functionality of the Mac and Win ports.

What do you get when you take WebKit/GTK+, add GStreamer and finish off with a sprinkling of code from Clutter?

Pierre-Luc Beaudoin of Collabora has been working on a GStreamer-based media backend for WebKit.

Last week, we landed his work (#16145), which adds support for the WHATWG HTML5 video/audio specification allowing streaming media to be embedded in web pages without the need for plug-ins.

The last couple of days, I’ve been integrating this media backend with our Cairo-based graphics pipeline (#16356). The result is that streaming web video can now be CSS transformed, embedded in SVG, bounced around the page and generally manipulated with JavaScript.

And the screenshot? It’s a video (one of many GUADEC flicks rescued by Thomas Wood) of Federico talking about Sabayon, played back natively in Epiphany at an angle of 45° and with an alpha value of 0.6.

The semi-transparent user controls superimposed on the video are defined entirely in HTML/CSS complete with animated transition effects (WebKit-only right now, other browsers will fall back gracefully).

Exciting stuff!

In the last month, we’ve started to look more seriously at the WebKit public API and how it fits with GLib/GTK+ principles. Some of the fixes have been obvious:

• #16174: Use “URI” not “URL” in public API
• #15691: The main widget should be called WebView, not Page

Policy changes have been made to ensure consistency:

• Any public entry point, property or signal that does not match GTK+ conventions is now considered a Major bug alongside security issues and crashers (with some exceptions like the new portable JavaScript engine C API (#15687) — more on this in my next post.)
• Any new API contributions without corresponding GTK-Doc documentation will be rejected. Enough said.

The Hacker’s guide to WebKit/GTK+ now covers topics such as API design, patch contribution and optimisation (including Cairo/SVG/canvas performance and memory reduction).

Sources of inspiration

GtkTextView

GtkTextView has a solid design and well-understood data model thanks to its heritage, being a synthesis of the venerable Tk text widget. Some applications like Tomboy are starting to push the limits of GtkTextView as they look towards integrating rich media and Web content synchronisation.

Challenges:

• Can we provide a GtkTextIter implementation that works with the W3C DOM (and our upcoming GObject DOM binding), or is the concept of GtkTextTags too limiting to make this worthwhile?
• How faithfully can we bridge between Markup and HTML?

GtkMozEmbed

GtkMozEmbed is a thin binding around a stable and mature browser engine, Gecko. The primary consumers here are Epiphany, Devhelp and Yelp. To make serious use of the underlying engine, these applications have to resort to C++ and Gecko’s internal APIs which can be inconvenient.

GtkMozEmbed seems to be awkward on non-X backends since it just uses the Mozilla port for that platform, not a GTK+ backend. WebKit/GTK+ on the other hand supports all GTK+ backends natively including DirectFB, OS X and Windows. Looks like there is nothing much worth keeping from the GtkMozEmbed API, but we’ve gained valuable insight from the ways in which developers have struggled to use it.

Challenges:

• Accessibility: We have several accessibility features, but no formal accessibility using ATK. Contributions here are welcome, otherwise I’ll look into adding support in a few weeks.
  • #16135: We’re adding caret-based browsing support straight into WebCore.
• Patches already available but not yet integrated:
  • #15610: International/complex text rendering support using Pango. The Imendio guys are working to complete and integrate my initial work on this feature.
    
  • #14120: Input method support using GTK+ input methods
  • Inline search API (Update: Adam Roben from Apple has pointed out that this can be worked around by calling into window.search using the JS C API for now)
  • #15576: GtkPrint support using Cairo

GtkHTML

GtkHTML is used primarily in Evolution. Again, there is nothing much worth keeping from the GtkHTML API as far as I can tell. WebKit now has excellent editable content support and exposes necessary public API for basic editing. Complex modifications will soon be possible using the GObject DOM, but until then, the DOM can be easily manipulated using the JavaScript C API (more on which soon).

Challenges:

• emacs keybinding support using GtkBindingSet (see also #15911). Perhaps an emacs user can provide this?

Is your application hitting limitations in these text viewing/editing components? Any that I’ve missed? Do speak up now so we can cater for your use-case in WebKit!

In my talk at LCE 2007, I touched on the issue of proprietary branches and their effect on developer morale.

On the WebKit/GTK+ team, we’ve had to deal with this issue a few times.

• A year ago, Adobe promised great new graphics features from their AIR fork of WebKit that never materialised. We chose to redouble efforts to improve the open Cairo graphics backend instead of sitting about waiting for their contributions, a decision that has paid off.
• More recently, a company has been working on a largely proprietary fork of the browser called OWB. It is remarkable that, looking at some of their recent code drops (which cannot be easily merged back to trunk due to refactoring), they still have bugs and performance issues that we fixed in WebKit trunk several months ago. It’s sometimes unpleasant to implement features that others are working on behind closed doors, but we do it in an open forum with peer review. What’s more, we do it better.

Fast forward, November 2007

Today, Google announced that it’s developing a WebKit-based browser as part of its Linux-based Android mobile platform.

In the hours since this news came about, contributors have already started to ask whether their work is about to be obsoleted or replaced by a Google code drop. It’s inevitable that there will be some overlap.

I’d like to make it clear that we’ve already discussed this scenario, and it’s life as usual for the GTK+ port. If there are contributions to components of the GTK+ port, they will be reviewed through the same processes as any other contribution.

Furthermore, it’s likely that WebKit/GTK+ already has superior graphics capabilities to any forked browser since we’ve been tracking the very latest features including recent SVG updates, CSS transformation and CSS animation support. Another WIP feature of particular interest to mobile devices is the upcoming full page scaling support.

With current efforts to bridge Web applications and the underlying platform and research into features like bridging JavaScriptCore to D-Bus and other frameworks, it’s likely that we are also ahead of the game in the Web-as-a-platform department.

All that said, I look forward to working together with the developers of the Android platform towards a more open and flexible mobile Web!

A friend pointed out a slightly confused post by a Mozilla developer regarding WebKit/GTK+.

I don’t really care much for browser wars, but since this post touches on so many aspects of the browser that I’m working on, like Cairo graphics, native widget styling, transparency and complex script support, I thought I’d give a reply.

• WebKit/GTK+ doesn’t really attempt to compete with Mozilla or Firefox in the browser space. It instead targets application developers who want a full-featured browser engine with a fun, powerful GTK+-style API. gtkhtml and gtkmozembed have proved to be inadequate, too heavy or un-portable.
• In the mobile and embedded space, WebKit/GTK+ is mostly competitive with Opera and NetFront rather than Gecko. I can think of only once that a vendor has come along and asked how it compares to Gecko on mobile devices.
• We’re working with the developers of general purpose browsers like Epiphany and Midori, as well as domain-specific Web applications like the GNOME documentation browser Devhelp, and mobile platforms like the Maemo browser EAL and OpenMoko’s browser and feed reader to make sure WebKit can provide all the features they need. It turns out they don’t need a UI toolkit like XUL since GTK+ provides the functionality they need, so we aren’t attempting to bundle a toolkit with our browser engine.
• WebKit/GTK+ is not tied to Linux. It’s portable to OS X and Windows. Behaviour on those platforms is intended to be similar to that on Linux, and the API on all platforms is identical.
• It can render that “Hello” button correctly. It also supports a bunch of fancy features from HTML5 and CSS3 that Gecko does not and loads pages faster before optimisation work has even started, but I don’t think we need to go into a feature-for-feature debate here.

An application hosting OS X Dashboard widgets (widgets can store data locally using HTML5 client-side database storage):

An implementation of Freedesktop notifications using Growl WebKit styles (incidentally, work is ongoing to provide integration with various Freedesktop specifications where appropriate):

An example of native widget styling:

glyph-pixmaps

Carl’s glyph-pixmaps work has been merged to xserver. This provides a speedup for text rendering by hashing and caching glyph data to avoid constant re-uploads. Eric Anholt breaks it down for us.

Cairo and XShm

Shared memory has had a bad rap when it comes to X11, and it’s been found in the past that it doesn’t provide a significant performance benefit over sockets at the transport level (SMT).

However, when it comes to image data, XShm can provide a “sensible performance optimization” by avoiding gratuitous copies. To take advantage of this, you need to do a little work to ensure that the data can be shared directly with the X server. Chris Wilson has just proposed a patch to support this in Cairo.

WebKit and complex text

I’ve started work on support for complex text in WebKit/Gtk+ using Pango. This includes support for non-Western writing systems.

To continue today’s performance theme, it’s worth noting that WebKit provides two code paths for text rendering:

1. Blazing flast: Known as the CG path on OS X, this path caches glyph data and avoids expensive computations wherever possible. In the GTK+ port, this code path requires direct use of the font-system on every supported platform. Luckily, it turns out the WebKit code base already has optimised text implementations for several platforms, which we can just borrow and plug in to our own port.
2. Fast: The ATSUI path on OS X, this path is capable of complex text rendering, while still providing opportunities for caching layout information from previous text runs. Here, we use Pango, along with a few clever tricks.

A heuristic is applied to detect which code path can be used. Since the capabilities of Pango are slightly different to ATSUI, this may yet need some tweaking to avoid more fallbacks then are necessary.

WebKit/Gtk+ gets a reviewer

I am now an official WebKit reviewer. This should help speed up development of the Gtk+ port. Thanks go to Apple and the WebKit team!

Canvas support using Cairo

A few hours ago I landed support for the Canvas HTML element, rendered using Cairo (just like everything else in the browser engine, including ordinary content and SVG graphics). This feature paves the way for significant bling, with WebKit having been the originator of the Canvas standard for use in the Mac OS X Dashboard.

This screenshot demonstrates some dynamic frame/rotation/shadow effects applied to an image on the Web.

Public API refactoring

I’ve now re-factored WebKit/Gtk+’s public API so that the types are named WebKitPage, WebKitFrame etc. This looks and feels a lot better than the old naming convention, and is part of an ongoing effort to provide an elegant and seamless GObject C API for the browser engine. I’ve also updated the Epiphany browser and the EAL wrapper to track these changes so disruption to these projects should be minimal. If you are not having fun embedding WebKit/Gtk+ in your application we’ve done something wrong!

A special thanks

Mark Rowe (who works at Apple) can be considered godfather of the Gtk+ port. He has provided unfaltering enthusiasm and patch review since I started hacking on the browser over a year ago, and last night stayed up till 5AM making sure the Gtk+ port worked on the upcoming feature branch. Earlier that day, he had been hacking on GdkPixbuf image and animation loading support (which has unfortunately stalled since the GdkPixbufAnimation API is something of a black box).

Moving on

In other news, I’ve left Collabora Ltd. for something completely different. Do try to avoid sending email to my old work address. I believe they are hiring someone to continue their WebKit/Gtk+ contracting business. (Other aspiring GNOME companies that have started to contribute to WebKit/Gtk+ include Imendio and Igalia.)

P.S. I left a little treat for you guys in the office

Yesterday’s talk on WebKit/Gtk+ at LinuxConf Europe was well received, and I was particularly impressed by the turnout considering that it was scheduled against Linus’ git BOF next door. The audience had more questions than there was time for, but those that I was able to answer were split mostly between questions about mobile performance/footprint and standards compliance.

The talk covers some of the history of both the WebKit browser engine and the WebKit/Gtk+ port, looks at the ethos of the developer community and explores current and planned features, such as SVG, Canvas and audio/video. It goes into detail about work involving the Nokia N800 internet tablet and Maemo as well as a brief look at integration work being done by the OpenMoko project, the state of play with OLPC and, of course, GNOME’s Epiphany web browser and Devhelp documentation browser.

Download WebKit/Gtk+ talk slides

I’ve been working with George Wright the last few days to integrate the WebKit/Gtk+ browser engine into Nokia’s recently announced EAL (Engine Abstraction Layer).

WebKit/Gtk+ will soon be joining Opera and Mozilla as a well-supported alternative browser for the Maemo platform. I believe we are the first to provide SVG support:

It’s exciting to hear the Red Hat guys have been experimenting with WebKit/Gtk+ on the OLPC platform — J5 is a competent engineer and his comments have helped to give us direction for further development.

Given also the recent activity on Epiphany/WebKit, these are good signs that we’re working towards a usable browser engine both for mobile devices and the desktop, but there are still features to be implemented and bugs to fix, and I encourage those interested to get stuck in to the WebKit bug tracker, whether filing bugs or contributing patches.

Great credit goes also to Holger Freyther of the OpenMoko project, who is co-maintaining WebKit/Gtk+ upstream in Apple’s SVN repository along with me (we completed paperwork with Apple to become SVN committers about a month ago) and has been dealing with some particularly involved parts of the port involving scrolling, frames and native widgetry.

Sunday came and went in Birmingham. This was an opportunity to meet many of my colleagues at Collabora face to face for the first time.

Managed D-Bus talk went very well. Huge interest from the usual suspects, but also from the wider GNOME community. Packed room, engaging audience.

Sundays are usually quite boring, but this one was an exception. After talks, lead a jazz crawl across the pubs and bars of the city — it seems we caught the end of the 23rd Birmingham International Jazz Festival 2007.

Download Managed D-Bus talk slides