Info

Ben is Vice President of Mobile Engineering at Walmart.com, where he works closely with his long-time friend Dion Almaer.

Web 2.0 Expo and web2Open

Tomorrow, Dion and I are giving a talk at O’Reilly’s Web 2.0 Expo conference in San Francisco at the Moscone Center: “Web Developer Tools: How to Be Productive Building for the Web” (Wed. Apr 1, 10:50 am). While normally Web 2.0 Expo costs money to attend, our session is free; all you need do is register for the free web2Open program. As part of our session we’ll be releasing something small; we’d love to see you there and get your feedback.

Following the presentation, we’ll be hosting a web2Open session at 12:40 pm–also in Moscone–to host a discussion about the state of Developer Tools for the Open Web and explore their future. If you’ve an interest in the subject and find yourself in town, won’t you drop in?

Marc Andreessen

They say Marc Andreessen, co-founder of Netscape and co-author of the Mosaic browser, once said:

[An operating system] is just a bag of drivers.

People have been fantasizing about the web as application platform for as long as we’ve had it. Nearly a decade later, we’re really just getting started at realizing this vision–of truly reproducing the power of traditional operating system APIs inside of the browsers.

While some have had this vision of browser-as-application-runtime since the beginning, most of us have traditionally viewed the browser as a web page renderer. It’s only been in the past few years that some have begun to push hard on changing this status quo. Google stands out in this group both with the creation of boundary-pushing “desktop-quality” applications like Gmail and in describing Google Chrome as an application run-time, not a page viewer. [1]

The Chrome Comic

Here in the Mozilla Developer Tools Lab, we’ve been pondering the various gaps in the tool-chain when you treat the browser as a serious, OS-grade application run-time. We’ll talk more about the landscape of tools and what’s available in a different post. In this one, we’d like to talk about one of the gaps we’ve found: memory tools.

The Memory Problem

It’s a rare application developer indeed who doesn’t wish their GUI to be “snappy”. In technical terms, Jakob Nielsen defines snappy as responding to user input within a tenth of a second. To put that in perspective, that’s shorter than it takes the average person to blink their eye.

Jakob Nielsen

If an application’s appetite for memory crosses over into gluttony, it can put a developer’s snappiness ambitions at risk. There are at least a couple of reasons why.

First, applications have a finite amount of memory available to them. When the operating system runs out of memory, a cool trick lets them supplement disk space for memory, but when this happens, performance hits the floor–hard drives, being mechanical, are orders of magnitude slower than memory.

While web applications don’t directly interact with the operating system to obtain memory, the browser does both for its own internal functions and as a proxy to the appetite of the web applications it is displaying, and as a web application’s memory consumption grows, so does that of the browser.

Therefore, if an individual web application’s memory needs grow sufficiently large, it can force the operating system to start dipping into disk space to provide sufficient memory, and when this happens, kiss any semblance of responsiveness goodbye.

Since there’s no way for a web application to know how much memory is available before this performance doomsday occurs, its good behavior to make your memory footprint as svelte as possible.

Garbage Collection and You

But there’s another, much more important reason why small web application memory footprints are good. It has to do with the way memory is handled in a browser. Like Java and pretty much any scripting language, JavaScript manages memory allocation for developers. This frees developers from having to deal with the tedious bookkeeping associated with manual memory management, but it comes at a cost.

That cost is embodied by the garbage collector. As a web application executes, it is constantly creating new objects, most of which are fairly transient–they are part of a transaction that has completed, like creating some short-lived jQuery objects to look-up some DOM elements. These objects consume memory. Eventually, the web application has created enough objects and is therefore consuming enough memory that the collector needs to wade through all the objects to see which ones are no longer being used and therefore represent memory that can be released.

This is where the performance implication comes in. To do its work, the collector stops the web application’s execution. Typically, this happens so fast that the user doesn’t notice. But when a web application creates lots and lots of objects, and these objects aren’t transient, the collector has a lot of work to do–it must go through all of these objects to ferret out the ones that are no longer used. This is turn results in delays that the user can perceive–and impairs the application’s responsiveness.

Leaks

To be clear, most web pages and web applications don’t push the browser’s memory limitations enough to cause performance problems related to either of the scenarios above. As stated at the outset, this blog entry is about those web applications that need to treat the browser as a high-performance run-time, which in the context of this entry means that they have much-larger-than-average memory requirements.

However, these issues apply to more than just those web apps that are designed to use large amounts of memory; they can also apply to long-running applications which, over time, gradually consume small amounts of memory until the footprint grows to be quite large. When an application consumes more memory than its designers intended, it is said to leak memory. [2]

And this leads in turn to a third way in which memory can give the shift to performance: when the browser itself leaks memory. It turns out that mere mortals have created web browsers, and every so often they’ve made mistakes which can trigger either of the two scenarios described above.

Diagnosing the Problem

So how do you as a developer go about troubleshooting these sorts of problems? Today, there’s really only one way good way to do it: use the operating system’s tools. Unfortunately, this option doesn’t provide the right level of detail; you can either see how much memory the browser is consuming in aggregate (which is fine to let you know that your memory use is increasing, but doesn’t tell you why) or you can see which data structures in the browser itself are consuming the memory (which is fine if understand the guts of the browser, but it’s pretty hard for anyone else to understand how this maps into the web application they’ve developed).

What’s missing is a tool targeted at web developers that makes it easy to understand what’s happening with their application’s memory usage. We propose to create such a tool.

Start Small, Start Focused

Our plan is to start small and address two key needs that are presently unmet by any of the existing, developer-friendly, easy-to-use tools we’ve seen on any browser. These needs are:

  1. Understand the memory usage of an application
  2. Understand the garbage collector’s behavior

While here in the Developer Tools Lab we’re most interested in creating developer tools for the entire web community (i.e., not just Firefox users), in this case because the tool will need some pretty deep integration with the browser, we’re going to start with Firefox (because we sit close to the engineers who work on it).

We plan on the initial implementation of this tool to be simple. For memory usage, we want to introduce the ability to visualize the current set of non-collectible JavaScript objects at any point in time (i.e., the heap) and give you the ability to understand why those objects aren’t collectible (i.e., trace any object to a GC root). For the garbage collector, we want to give you a way to understand when a collection starts and when it finishes and thus understand how long it took.

Help Us!

This is obviously a small step into a large world. Is it a good first step? What do you think we should do differently? We’d love to hear from you, and thanks for reading!

[1] Of course, Firefox does a fine job of acting as application run-time; my point is that Google was the first to call out web applications as a distinct class of web content and to talk in terms of supporting these for their mainstream browser. Incidentally, Mozilla Labs’ Prism project sought to pioneer this idea years before.

[2] I’m using the term “leak” in a much more general way than is common in most developer communities. Traditionally, the term is applied to an application that allocates memory and then neglects to deallocate it when done. Because a language like JavaScript doesn’t allow developers to manually allocate or deallocate memory, it is impossible to leak at the JS level in this sense. But in my broader sense, any time a developer unintentionally creates memory footprint (e.g., by continuously storing objects in a hash in a mis-designed cache, etc.), I consider it a leak. This broader definition is borrowed from the Java community.

[3] Image of Marc Andreessen taken from http://images.businessweek.com/ss/08/05/0520_valleyboys/source/4.htm

[4] Image of the Google Chrome comic book taken from http://www.google.com/googlebooks/chrome/

[5] Image of Jakob Nielsen taken from http://www.useit.com/

Jackie

When my first child starting talking, the way she said “baby” was comical: it was basically two grunts, a high-pitch followed by a low-pitch, which when concatenated sounded roughly like the word itself.

As time went on, she learned how to speak with greater precision, but the funny bit was that for a while, whenever she would say “baby” she would revert back to her earliest versions of the word. So you’d hear fair renditions of words and then the two grunts.

Eventually, she worked it out.

Tonight as I spent time with my fourth child, I noticed the same behavior with a different word. And then it occurred to me. We don’t reproduce the sounds that we hear. Instead, we learn how to produce sounds and, independently, map the sounds we hear to the sounds we’ve learned how to produce. Hence, most of us have really bad accents when we learn new languages–we are mapping a new set of different sounds to the set of existing sounds that we already know how to make. Mismatches are inevitable.

I doubt this will be a revelation to anyone but me, but it did get me wondering: are there some people who can reproduce the sounds they hear with anything near fidelity? Such people would be amazing at learning new languages, among other things.

Is this a skill I can learn? It seems so much more efficient than having to map what I hear to what I know how to say.

Does anyone know?

OmniFocus and Things

I’ve been using OmniFocus on OS X since the betas and have never really enjoyed it. I find the UI quirky and am still in awe of how tracking to-do items can become as complicated and unintuitive as the Omni team have made it. As someone else once said about something else, it’s like I step into a 747 cockpit full of all these daunting controls when all I really wanted was a Cessna. But with time, its concepts have sunk in and I’m able to be productive with the sucker. I don’t enjoy using it, but I appreciate it.

But, don’t get me started on the mobile version. For the longest time, you couldn’t actually interact with the iPhone OmniFocus app until >30 seconds after launching it–sometimes not for minutes after launch. Even in the latest version, after a ~5 second start-up delay, I’m sometimes able to enter new to-do items, but only in a special mode where I can’t categorize them. So then I have to remember to go into this special mode out of my normal workflow in the desktop application where I can see them–otherwise, they are lost forever.

OmniFocus on start-up

Of course, that’s only sometimes. Most of the time after I launch the iPhone version, the UI just sits there completely non-responsive while I watch a spinning circle. I’ve watched the circle spin for minutes with no indication of what’s happening (just last night, in fact). You can re-launch the app, but it’ll out-smart you and re-spin the circle when you go back in.

So it was with no small amount of enthusiasm that I finally tried out OmniFocus’ much sexier competitor: Things. Much has been said by many about how beautiful Things is, how Mac-like, how clueful the designers behind Things are, how lives are being improved dramatically by it, and so forth. I’ve tracked some of the praise for the past year and frankly, found my expectations were really high. “Finally,” I thought, “a to-do program I can enjoy using.”

To my surprise, that turned out not to be the case. While the UI appeals to me much more than OmniFocus, I found the quirks unpalatable. For example, when entering a new project with the keyboard, there are four fields. The third field is the due date. When this field gets focus, it is instantly populated with today’s date. If you wanted to get to the fourth field on the way and just happened to pass through the third field on the way? Tough. You’ve now got to delete the date with the mouse. Hmm…

Then take the “Areas of Responsibility”, something like what other to-do managers call a “context” (but, don’t they advocate tags for contexts?). There is no way to assign to-do items to an area without using the mouse, and areas don’t sync to the iPhone version of Things.

As I explored deeper, I found other bothersome limitations. For example, OmniFocus has a notion of projects with to-do items that can be completed sequentially, in parallel, or that are just catch-alls for items. In Things, projects just contain to-do items. Oh, and in Things items in projects can’t be set to repeat. (The Things developers promise to rectify both of these limitations.) I use both of these features. Duration of a to-do is implemented as a tag? Really?

I need to stop myself; if I continue to compare features, OmniFocus will win over and over again. I knew that heading into Things, but I expected to catch the zen of Things’ fantastic design and use that to overcome the missing features. But in the end, I find I am an OmniFocus kind of guy after all–maybe not to the surprise of those who know me.

This all ties into a meme that I’ve been pondering for a few years. More on that in another post.

Ben at QCon

On-stage at QCon London this week, Dion and I gave a demo of Bespin. After spending a few minutes building up to the demo, emphasizing how fast Bespin performs, how it scales to tens of thousands of lines, beats DOM-based editor approaches–the usual litany of claims–I proceeded to mash on the keyboard and demonstrate that Bespin’s performance was… decent on a hundred-line file and, as it turned out, “slow as molasses” on thousand-line files. (Still, the most embarrassing moment of the talk was to come later, when Dion talked of his hurrying home after school every day as a youth to “twiddle his paddle”; that’s another story.)

Dion's Paddles

Just days before, Bespin was wicked fast. What happened?

I thought it might be interesting to explain this little performance regression.

Tabs

Shortly before launching Bespin, we decided to cheat on supporting tabs in the editor. Our initial code didn’t support properly displaying tabs so we decided to convert any tabs in a file to a fixed amount of whitespace when such files were imported. Obviously, this is unacceptable and we knew we had to get to real tab support in the future.

To be clear, I hate tabs. They are evil, loathsome little control characters, and the programmers who use them are the sorts of people who go out on weekends to weird nightclubs where they hang out with the hippies who put curly braces on new lines.

Jamie Zawinski

Speaking of nightclubs, Netscape alumnus and nightclub owner Jamie Zawinski has written an interesting piece that discusses tab support in editors. Essentially, he concludes that tab characters should be removed entirely from files in favor of spaces–that editors should transparently remove them on save.

As much as I agree with this perspective, and as much as I don’t want to add tab support to Bespin, one of our goals with the project is to be a great editor for all kinds of programmers; even the tab-using ones. So, I have to stop whining and support them.

There are two basic ways to add this support. Tabs can either be present in the editor and rendered as a certain amount of white space, or they can not be in the editor but instead be inserted when the file is written to disk as a sort of compression scheme (i.e., one tab character is used to represent one or more spaces).

Of those two approaches, the latter would be quite easy for us to implement. We’d simply scan each line in the file for spaces when saving and when one or more spaces occurs immediately before a tab stop, replace it with a tab character in the file. Ten minutes work.

But would you believe that there are some people who, in addition to wanting real tab characters in their files, actually mix tabs and spaces? Yes, it’s true. There are even file formats which depend on this sort of thing. Horrors!

So I can’t do the easy thing and just deal in terms of spaces at run-time in the editor. I have to support putting real tab characters in the file during the edit session, and distinguishing such characters from spaces.

Implementing Tab Support

Bespin backs each row in the editor with a string array; each character is represented by an element in the array. Because Bespin is limited to fixed-width fonts, we’ve been able to get away with easy measuring operations, like calculating the pixel width of a line by multiplying the size of the line’s array by a standard character pixel width:

var lineWidthInPixels = lineArray.length * characterWidthInPixels;
Pseudo-code for measuring a line’s desired width

We then use this measurement to determine whether the line will entirely fit in the editor without horizontal scrolling.

As it turns out, we need to measure the width of a line quite often. In fact, we have to know the width of every line in the editor, regardless of how few lines are currently visible in the window, in order to properly render the horizontal scrollbar. That’s because every editor I’ve seen sizes the horizontal scrollbar according to the widest line in the entire file, regardless of what’s currently visible.

Because obtaining the length of an array and multiplying it by some value is so cheap, we’ve been able to get away with measuring each line in the file very, very often.

The Performance Regression Explained

And, this is what drove our vaunted performance right into the floor. You see, with tabs, measuring the width of a line is no longer quite so cheap. We now have to scan each character in the line to see if it is a tab character and then work out how much whitespace the tab character represents. Here’s roughly how we do that now:

var length = 0;

// check for tabs and handle them
for (var i = 0; i < lineArray.length; i++) {
    // check if the current character is a tab
    if (lineArray[i].charCodeAt(0) == 9) {
        // since the current character is a tab, we potentially need to
        // insert some blank space between the tab character
        // and the next tab stop
        var toInsert = tabstop - (i % tabstop);

        // increment the length to correspond to the new space
        length += toInsert;
    } else {
        length++;
    }
}

var lineWidthInPixels = length * characterWidthInPixels;
Modified extract from Bespin for measuring lines with tabs

Hey, wouldn’t you know, executing this code once per every line on every repaint is considerably more expensive than the previous code sample.

Making It Fast Again

While perhaps we could find more efficient ways to measure the length of a tab-riddled line, even when the algorithm was based on the length of the line’s backing array this represented a bottleneck that would have manifested itself eventually. Performance was degrading linearly in inverse proportion to the length of the file; tabs makes the degradation exponential, but either way, it’s unacceptable. We didn’t care about the degradation at 50,000 lines, but at 5,000,000 lines, we probably would have.

Maybe a better answer would be to measure each line once and then re-measure the line only when it changes. Genius! But, in reviewing our current code that marks lines as “dirty”, I discovered that it’s incomplete; there are many line mutations that merrily skip the dirty codepath. I didn’t want to get distracted by fixing this bit of infrastructure (which might have introduce yet more side-effects), so I moved on past this option and searched for another accelerator.

The next obvious approach would seem to be to only measure those lines that are presently visible. The disadvantage is that the horizontal scrollbar will size itself only according to what you can see; so if all of the visible lines fit in the editor window but somewhere a few pages down there are some super-long lines, you’d see no horizontal scrollbar at all. Better this than our present slower-than-CRuby-on-a-good-day behavior (har har), so I implemented it.

Dion and I were a bit intrigued by the result. We expected to be a bit off-put by a horizontal scrollbar whose width changes as you scroll vertically, but because Bespin’s scrollbars are a bit subtle with the transparency and all, it actually seems more cool than jarring. And, in fact, as we think about it, we can imagine some users preferring this behavior over the traditional behavior.

Horizontal Scrollbar 1

Horizontal Scrollbar 2

Horizontal Scrollbar 3

What do you think?

(The public Bespin instance doesn’t yet have this behavior; it will in the next day or so. By the way, all the inflammatory statements in this post, such as tabs are evil, etc., are just jokes. Mostly.)

UPDATE

I modified the section on Jamie Zawinski after Ted Mielczarek rightly pointed out that I completely misquoted him. I wrote this post on a plane without access to Jamie’s piece, and my memory was obviously faulty. Still, the spirit was the same–we both hate tabs.

[1] Pictures of Ben and Dion cribbed from the JAOO blog (http://blog.jaoo.dk/2009/03/11/sirtonyhoareatqcon/)

[2] Picture of Jamie Zawinski credited to Yetta Howard/Salon.com (http://archive.salon.com/tech/feature/2000/02/10/zawinski/)

One of the disadvantages of writing a blog post on little sleep late at night in a hotel room while watching a movie and talking with Dion and doing your email is that you inevitably forget to write something you meant to.

In this case, one of the big drivers for writing the previous Bespin and Canvas post was to discuss accessibility–which I then neglected entirely to mention. Let me rectify that here and add a few other miscellaneous bits.

Accessibility

Rendering all of our content to a giant bitmap is obviously not the best way to integrate with all kinds of technologies designed to make content more accessible in various ways. But getting past that initial observation, we’re actually really interested in finding ways to make Bespin very accessible.

One of our initial ideas is to embed semantic content inside of the <canvas> element itself, as in:

<canvas>
  The content inside the Bespin editor
  could be embedded inside here as text nodes
</canvas>

It might not be expensive (in UI latency terms) to keep these text nodes synchronized with the canvas editor since the UI thread doesn’t need to re-render anything as a result of the constant updates. If this approach were to incur a performance penalty, Bespin could have an “accessibility mode” that would optionally turn this on.

Bespin Dashboard

Bespin Dashboard

The Bespin dashboard also uses canvas, but for components instead of just text. We’re actually thinking of killing two birds with this one and having the hierarchy of the components defined inside of the <canvas> element:

<canvas>
  <splitPanel>
    <list/>
    <horizontalTree/>
  </splitPanel>
</canvas>

We could then build the UI using this hierarchy rather than by using the entirely programmatic approach we have in place now:

// this is pseudo-code not pulled directly from Bespin, but
// Bespin's dashboard code looks similar
var splitPanel = new SplitPanel();
var list = new List();
var tree = new HorizontalTree();
splitPanel.add([ list, tree]);
...

You would style and layout these components using separate artifacts (CSS and a layout-specific artifact).

We’ll talk more in a future blog post about this concept of using canvas to render components; it’s really more about supplementing DOM-rendered components with canvas-rendered components and laying out the combination with JavaScript (rather than CSS / HTML)–this is what we briefly mentioned as “Thunderhead” in the announcement video.

An advantage to this approach for accessibility is that we can provide a clean, fully-semantic hierarchy of the content, rather than status quo of HTML which runs the spectrum from style-and-content-mashed-together-seamlessly on one end to almost-pure-semantic-content-with-a-few-compromises-to-deal-with-CSS-realities on the other.

Let me again emphasize that Thunderhead is an experiment at what this kind of thing would look like; we’re not sure if we like it and we may scrap it entirely at some future point.

SVG?

We’re frequently asked why we didn’t use SVG for the editor. To be honest, the biggest reason is that I have a lot of experience creating custom components with Java2D, and Java2D strongly resembles the canvas API. It was pretty natural to start hacking away with canvas and JS.

But there are a few reasons we felt good about going down that road to begin with. First, based on experimenting with DOM rendering, we felt we had good reason to suspect that SVG wouldn’t perform as quickly as the canvas implementation since SVG has its own DOM API and thus is likely to mirror all of the performance issues to be had with HTML’s DOM.

On top of that fundamental concern, we had another: while it’s straight-forward to implement our own painting optimizations in canvas (and we have), with SVG it’s much more subtle and involves understanding the implementation details of SVG run-times and whatever optimizations have been implemented under their covers (again, the same as with HTML DOM performance optimization).

Also, as a fairly large specification, we were concerned about compatibility headaches as differing subsets of SVG are implemented in the browsers that support it. We didn’t have the energy to fight DOM compatibility issues during the day and SVG compatibility headaches at night.

Having said that, we’d love to see an SVG version of the Bespin editor emerge at some point and compare its performance and codebases. There’s certainly a good chunk of JS code that could go away if an SVG version worked out.

Stand-alone Editor?

We’ve also been asked whether we’d like to make it easy for folks to embed Bespin’s canvas editor in their own non-Bespin projects. Absolutely we do. Bespin will eventually spin out three projects:

  1. The Bespin project: an on-line code editor (i.e., the project we’ve currently announced)
  2. Thunderhead: a canvas/JS GUI toolkit
  3. (To Be Named Editor): a stand-alone canvas editor that you can embed

To make it seem less like I’m making this up just now, here’s what the Thunderhead logo looks like:

Thunderhead

Canvas versus Firefox

Some folks have also wondered why we didn’t just enhance Firefox to do the things that we achieved by using canvas and JavaScript. The answer to that is simple: we’re not interested in making Bespin only run on Firefox. We want as broad an audience as possible to use it.

However, we’re already talking with standards folks and browser folks about how best to pull some of these ideas back into the browsers. This isn’t canvas versus the browser, it’s about using canvas to experiment with new features ahead of the browser, and applying those “learnings” (is that really a word? WTF) back to the browsers.

More to come soon, thanks for reading.

Bespin

It’s been thrilling to finally release Bespin to the world. There’s nothing quite like seeing feedback start pouring in via IRC, blog comments, Twitter, and email within minutes of pushing “Post” on the first announcement blog entry. I’m very grateful to everyone who has taken some time to engage with us in any of the various media out there. We’ve also seen some interesting community efforts at extending Bespin; check out our blog post on the Labs website for details.

Culling through everything, it’s clear that a few aspects of the project are controversial. I’d like to address one of them: our choice of canvas as the rendering engine for the editor.

Bespin and Canvas

Bespin takes advantage of a relatively new feature of browsers to power its text editor: the <canvas> element. Canvas gives developers something the intriguing ability to render arbitrary two-dimensional graphics in a web page; it’s basically an <img> element you can draw into with JavaScript commands. The API looks something like this:

// all drawing performed via the context
var ctx = document.getElementById("canvasId").getContext("2d");

ctx.fillStyle = "black";  // accepts same values as CSS "background" property

ctx.fillRect(0, 0, 50, 50); // draws a black 50x50 rectange in the canvas

Canvas is available in the currently released versions of Safari, Opera, Firefox, and Chrome; it has been a part of the HTML 5 draft specification for a long time. It is not available in Internet Explorer.

Bespin builds on this API to recreate a text editor from scratch. That’s right; rather than use the native facilities of every modern operating system that give you word processing features for free, Bespin rolls them from scratch, including such basic constructs as a blinking cursor and text selection.

The Bespin Editor

The Bespin Editor

One of the the features we didn’t have to re-implement is text rendering; the HTML 5 specification includes an API to render text on a canvas. Unfortunately, this portion of the canvas API has only been implemented by Firefox, though it’s already in WebKit nightly builds (and thus will very likely be in a future version of Safari) and Chromium nightlies (from which Chrome versions are derived). We expect Opera will soon as well.

But Does It Scale?

We weren’t actually sure this would work when we set out to do it. You wouldn’t think a JavaScript interpreter in a browser could render enough frames on a large, modern 32-bit image display to make the experience anything close to what you get effortlessly in Notepad on Windows XP running on 10-year-old hardware.

To be sure, for some users, the performance does indeed suck–we’re looking into exactly why (it appears that certain video drivers cause performance problems on Linux; Google Chrome’s multi-process architecture appears to slow down performance on that browser). The interesting thing is that for large classes of users, performance is really, really good. There’s a lot to say about topic; we’ll touch on that more in a future blog post.

Why?

The first time we demoed Bespin to someone who understand these implementation details, the reaction was “Wow, you’re crazy!” followed by “That’s really cool.” Pretty much everyone else I’ve spoken with has the same first reaction (the crazy bit) but the follow-up is not always positive (ranging from the aforementioned to “WTF!?”).

Surely there’s a higher-level abstraction we could have built on? Something above pixel painting?

Well, let’s get one thing out of the way first-up: there may very well be a more effective way to do what we’ve done than with canvas. I’m not saying this is the best way to do it; I just want to explain why we did it.

Control, Control, We Must Have Control

One of our goals with Bespin is to be able to (at some point in the future) deliver a fantastically compelling user experience for editing code on the Web. We don’t want Bespin to be the editor you use when you can’t get to your desktop; we want it to be your editor of choice in any context.

It turns out that’s a tall order. Some users like word-wrapping; some don’t. Some users actually like column selection modes while other (more sane) users like line-oriented selection (I kid, I kid). Developers are notoriously finicky with their tools–especially their code editor–and we didn’t want to re-educate the world on how our editor worked; we want to make Bespin adaptable to as wide an audience as we could.

We felt that if we built Bespin on something like <textarea> or the “content editable” feature of HTML, we’d have a built-in set of limitations over which we’d have very little or no control.

The Alternatives

To build on the last paragraph, we considered three different alternatives to canvas:

  1. textarea
  2. contentEditable
  3. “line blocks”

The “control” argument above is one reason we dismissed textarea and contentEditable; further, we’ve seen a large number of other web editor implementations based on these approaches and based on playing with those and by the many horror stories floating around the Web from their authors, we were concerned that we couldn’t overcome performance and compatibility problems, especially when editing large files.

Line Blocks

“Line blocks” are a different story. What if we had a bunch of <div> elements stacked vertically and treated each one as a rendering surface for a line of text in the document being edited? We played with that approach a bit but couldn’t get as much performance out of it as we could with canvas, especially for wide files (>80 columns).

Compatibility

We’ve taken some flak for claiming that Bespin is based on “Open Web technologies” but only supporting Firefox and WebKit nightly. Earlier in this post, I explained the state of canvas support in the browsers and how we’re hopeful that our canvas approach will work in future versions of Chrome, Safari, and Opera.

However, we have a plan in case browsers don’t implement the proper text rendering APIs: bitmap font rendering. By pre-rendering fonts into images, we can then render the images on the canvas. There are a number of proof-of-concept implementations floating around that demonstrate this concept (though unlike some, we wouldn’t render the fonts as vectors–there’s no way it would perform to our needs).

What about Internet Explorer? It’s hard to say what the future of canvas support is for IE; it certainly isn’t coming in IE8. However, there are a number of projects that emulate canvas by mapping it to VML (an abandoned IE proprietary rendering technology), Flash, and Silverlight. So far, none of these bridges offers adequate performance for us to seriously considering supporting it. We’re aware of promising developments in this area and we’ll experiment with IE support as these technologies evolve.

Mobile Support: iPhone, Android, Fennec, and More

We’re also very excited to get Bespin working on mobile devices with fully-featured browsers. The iPhone’s browser–”mobile Safari”–does support canvas, but does not yet have the text rendering APIs. We may implement the aforementioned bitmap font rendering just to get on the iPhone. We are also looking at other mobile platforms, such as Android, Fennec, Palm, and so forth.

How cool would it be to change your code and redeploy it from the beach?

Obviously, it will take some interface changes for Bespin to be usable on a mobile device. We have some ideas here and we look forward to working with the community on developing the concepts further.

What do you think?

What do you think of our use of canvas in Bespin’s editor?

Note: I realized the morning after that I had omitted some of what I meant to include in this post; I’ve retroactively renamed this post “Part 1” and added a “Part 2”.

Some time ago I found myself on a United Airlines flight and flipped through their on-flight magazine, Hemispheres:

hemispheres

I found this particular ad immensely amusing:

myvu1

There are so many things to love about this ad, but the lower left-hand corner is the best part:

myvu2

Yes, they do.

Follow

Get every new post delivered to your Inbox.