Bespin and Canvas: Part 1
It’s been thrilling to finally release Bespin to the world. There’s nothing quite like seeing feedback start pouring in via IRC, blog comments, Twitter, and email within minutes of pushing “Post” on the first announcement blog entry. I’m very grateful to everyone who has taken some time to engage with us in any of the various media out there. We’ve also seen some interesting community efforts at extending Bespin; check out our blog post on the Labs website for details.
Culling through everything, it’s clear that a few aspects of the project are controversial. I’d like to address one of them: our choice of canvas as the rendering engine for the editor.
Bespin and Canvas
Bespin takes advantage of a relatively new feature of browsers to power its text editor: the <canvas> element. Canvas gives developers something the intriguing ability to render arbitrary two-dimensional graphics in a web page; it’s basically an <img> element you can draw into with JavaScript commands. The API looks something like this:
// all drawing performed via the context
var ctx = document.getElementById("canvasId").getContext("2d");
ctx.fillStyle = "black"; // accepts same values as CSS "background" property
ctx.fillRect(0, 0, 50, 50); // draws a black 50x50 rectange in the canvas
Canvas is available in the currently released versions of Safari, Opera, Firefox, and Chrome; it has been a part of the HTML 5 draft specification for a long time. It is not available in Internet Explorer.
Bespin builds on this API to recreate a text editor from scratch. That’s right; rather than use the native facilities of every modern operating system that give you word processing features for free, Bespin rolls them from scratch, including such basic constructs as a blinking cursor and text selection.
The Bespin Editor
One of the the features we didn’t have to re-implement is text rendering; the HTML 5 specification includes an API to render text on a canvas. Unfortunately, this portion of the canvas API has only been implemented by Firefox, though it’s already in WebKit nightly builds (and thus will very likely be in a future version of Safari) and Chromium nightlies (from which Chrome versions are derived). We expect Opera will soon as well.
But Does It Scale?
We weren’t actually sure this would work when we set out to do it. You wouldn’t think a JavaScript interpreter in a browser could render enough frames on a large, modern 32-bit image display to make the experience anything close to what you get effortlessly in Notepad on Windows XP running on 10-year-old hardware.
To be sure, for some users, the performance does indeed suck–we’re looking into exactly why (it appears that certain video drivers cause performance problems on Linux; Google Chrome’s multi-process architecture appears to slow down performance on that browser). The interesting thing is that for large classes of users, performance is really, really good. There’s a lot to say about topic; we’ll touch on that more in a future blog post.
Why?
The first time we demoed Bespin to someone who understand these implementation details, the reaction was “Wow, you’re crazy!” followed by “That’s really cool.” Pretty much everyone else I’ve spoken with has the same first reaction (the crazy bit) but the follow-up is not always positive (ranging from the aforementioned to “WTF!?”).
Surely there’s a higher-level abstraction we could have built on? Something above pixel painting?
Well, let’s get one thing out of the way first-up: there may very well be a more effective way to do what we’ve done than with canvas. I’m not saying this is the best way to do it; I just want to explain why we did it.
Control, Control, We Must Have Control
One of our goals with Bespin is to be able to (at some point in the future) deliver a fantastically compelling user experience for editing code on the Web. We don’t want Bespin to be the editor you use when you can’t get to your desktop; we want it to be your editor of choice in any context.
It turns out that’s a tall order. Some users like word-wrapping; some don’t. Some users actually like column selection modes while other (more sane) users like line-oriented selection (I kid, I kid). Developers are notoriously finicky with their tools–especially their code editor–and we didn’t want to re-educate the world on how our editor worked; we want to make Bespin adaptable to as wide an audience as we could.
We felt that if we built Bespin on something like <textarea> or the “content editable” feature of HTML, we’d have a built-in set of limitations over which we’d have very little or no control.
The Alternatives
To build on the last paragraph, we considered three different alternatives to canvas:
- textarea
- contentEditable
- “line blocks”
The “control” argument above is one reason we dismissed textarea and contentEditable; further, we’ve seen a large number of other web editor implementations based on these approaches and based on playing with those and by the many horror stories floating around the Web from their authors, we were concerned that we couldn’t overcome performance and compatibility problems, especially when editing large files.
“Line blocks” are a different story. What if we had a bunch of <div> elements stacked vertically and treated each one as a rendering surface for a line of text in the document being edited? We played with that approach a bit but couldn’t get as much performance out of it as we could with canvas, especially for wide files (>80 columns).
Compatibility
We’ve taken some flak for claiming that Bespin is based on “Open Web technologies” but only supporting Firefox and WebKit nightly. Earlier in this post, I explained the state of canvas support in the browsers and how we’re hopeful that our canvas approach will work in future versions of Chrome, Safari, and Opera.
However, we have a plan in case browsers don’t implement the proper text rendering APIs: bitmap font rendering. By pre-rendering fonts into images, we can then render the images on the canvas. There are a number of proof-of-concept implementations floating around that demonstrate this concept (though unlike some, we wouldn’t render the fonts as vectors–there’s no way it would perform to our needs).
What about Internet Explorer? It’s hard to say what the future of canvas support is for IE; it certainly isn’t coming in IE8. However, there are a number of projects that emulate canvas by mapping it to VML (an abandoned IE proprietary rendering technology), Flash, and Silverlight. So far, none of these bridges offers adequate performance for us to seriously considering supporting it. We’re aware of promising developments in this area and we’ll experiment with IE support as these technologies evolve.
Mobile Support: iPhone, Android, Fennec, and More
We’re also very excited to get Bespin working on mobile devices with fully-featured browsers. The iPhone’s browser–”mobile Safari”–does support canvas, but does not yet have the text rendering APIs. We may implement the aforementioned bitmap font rendering just to get on the iPhone. We are also looking at other mobile platforms, such as Android, Fennec, Palm, and so forth.
How cool would it be to change your code and redeploy it from the beach?
Obviously, it will take some interface changes for Bespin to be usable on a mobile device. We have some ideas here and we look forward to working with the community on developing the concepts further.
What do you think?
What do you think of our use of canvas in Bespin’s editor?
Note: I realized the morning after that I had omitted some of what I meant to include in this post; I’ve retroactively renamed this post “Part 1” and added a “Part 2”.
It sounds a little crazy…but then again all ambitious project do. Congratulations.
Edwin Khodabakchian
February 18, 2009 at 12:15 am
I’m not very well informed on , but all I know was that this was originally an Apple product/language correct? My question is why was canvas chosen over let’s say SVG being that this project is housed with Mozilla? Also, I’m curious as to why you chose which is a web technology that is not identifiable through any DOM method and isn’t semantic?
I’m curious mostly because I’m still completely lost on how you did all this in the first place! I would have never thought of using the element and to rebuild everything including scroll bars and cursors and you did it well.
You guys have definitely given me inspiration to try out the element though! thanks again!
Oscar Godson
February 18, 2009 at 12:20 am
Crap, it stripped out everytime i did canvas as a tag… so the odd empty spaces i wrote canvas their as a HTML tag
Oscar Godson
February 18, 2009 at 12:21 am
Bespin is sexy cool. I look forward to seeing what comes next.
Eric Shepherd
February 18, 2009 at 12:31 am
Ben, you and Dion are doing a great job. I’m looking forward to the first public version of Bespin, because… if I’m using the first testing version and I’m already excited… I don’t know when the first public version comes!
I think using Canvas is a good idea, as well as difficult, and I only have to say something about that, be careful not to do Bespin a monster, because it runs a bit slowly on computers not so old (iBook G4 late 2005).
I have already tested it on Webkit Nightly Build and it runs very smoothly (Webkit Javascript Engine is gorgeus) But it has some issues with keybindings.
Keep this good work going!
Oscar C.
February 18, 2009 at 12:40 am
@Oscar: Canvas makes more sense for Ben and Dion then SVG, as it gives them a lower level, immediate-mode API for what they are doing. They wouldn’t get anything out of using SVG in this case — SVG allows you to have graphical markup, which is good for search engines, accessibility, etc. but makes no sense in this case. Both SVG and Canvas are standards, so using Canvas is totally fine (HTML 5 actually documents Canvas, as well as provides a standard way to embed SVG into normal HTML, so moving forward both are fine options).
Brad Neuberg
February 18, 2009 at 12:58 am
Ben, have you seen the amazing work Ray Cromwell has been doing around speeding up Canvas with a Flash backend, using some GWT features? You should touch base with him; the performance he’s been able to get sounds pretty amazing.
Brad Neuberg
February 18, 2009 at 12:59 am
Canvas is a good choice for many reason, however I see a problem with canvas : accessibility. How is your editor rendered to screen readers ? Perhaps it can be resolved by storing the text content in a DOM element ?
I think a better solution would to implement a real text editor in the browser (Yes, I dream
), a sort of , where javascript.xml is an xml file which describes the syntax of the language for the syntax coloring, the autocompletion etc..
Laurentj
February 18, 2009 at 1:17 am
sorry, your blog eats my exemple of html tag. So I said, a sort of
textarea type=”rich” language=”javascript.xml” options=”selectionmode=columns;indent=spaces;indent-length=4″
Laurentj
February 18, 2009 at 1:20 am
FYI, ‘{‘ and ‘}’ don’t work in Bespin when using
OSX and Finnish keyboard. To get ‘{‘ one has to press
alt+shift+8 and for ‘}’ alt+shift+9.
smaug
February 18, 2009 at 1:22 am
Brad, thanks for the mention. I do have a Flash implementation of the Canvas API, but there is a big caveat to text rendering. While I support text rendering in my GWT API (emulated in various ways, see this old post http://timepedia.blogspot.com/2007/06/gwt-canvas-rendering-rotated-text.html for how I emulate rotated text drawing on canvas using drawImage() and font-book images generated by Java2D), in Flash9 atleast, there is no equivalent of canvas.fillText(). Here is Flash’s equivalent to CanvasRenderingContext2D: http://livedocs.adobe.com/flash/9.0/ActionScriptLangRefV3/flash/display/Graphics.html
The way I handle text rendering then is to essentially overlay Text widgets onto the Flash scene. It was the low hanging fruit. There may be a way to do it for real in Flash, but getting access to vector font information and actually drawing the vectors, but this may be inefficient.
For IE, it’s probably the only solution, unless you want to use a Silverlight or Java Applet canvas implementation.
-Ray
Ray Cromwell
February 18, 2009 at 1:23 am
http://www.glazman.org/weblog/dotclear/index.php?post/2009/02/18/Surprised
Daniel Glazman
February 18, 2009 at 1:44 am
@smaug: We are talking about it in the Google group
. I’m testing a textArea test case and it works fine at least on Spanish (Spain) keyboard layout.
Oscar C.
February 18, 2009 at 2:06 am
Bespin is incredible, well done Ben! I can see why you chose Canvas and it’s fine for the next year or so. In the long term, however, we should improve contentEditable so it can deliver what you need, because it’s way more accessible e.g. to search engines and internationalisation.
Could you list the CSS extensions (e.g. for text-wrapping, text-selection), bug fixes and performance enhancements that would mean you could start using contenteditable? Then, if they were prioritised in Firefox, everyone on the web could benefit from them.
Chris
February 18, 2009 at 2:47 am
[...] The following is a lightly edited repost from an entry on my personal blog. [...]
Ajaxian » Canvas for a Text Editor?
February 18, 2009 at 5:00 am
I also had the “Wow, you’re crazy” first reaction. But I can see how an Open Web API of a Scintilla type editor would be nice.
I still have the “Wow, you’re crazy” reaction to Thunder Head – Why recreate widgets in canvas? Couldn’t you use standard widgets. I’ll wait for the post explaining the rationale
Mark Finkle
February 18, 2009 at 6:18 am
I think the using of canvas in Bespin is just the beginning, this must inspire us to use canvas in bigger projects like 3D rendering or something like that.
Congrats!!
Felix Vargas
February 18, 2009 at 6:34 am
I really liked the project.
The canvas element give you the ability to make really cool stuff with the text editing. I have make several changes on a local copy. Starting from scratch(leaving OS stuff) is power to make really non-web text editor. For example there was ( as far as know ) problem with newline that don’t save spaces for the new line correctly, but with 4 lines it was fixed
Radoslav Stankov
February 18, 2009 at 7:29 am
Ben – thanks for the writeup, and thanks for the suggestion for us to use canvas on our site too. I found it very easy to use, and like you, particularly liked the control it gives you over all the pixels in the element.
Had a few issues getting it to work in IE (especially
– but google’s excanvas (and some hacks) helped here. Do you have any plans to use an excanvas-like solution to get Bespin working on IE?
Look for some canvas graphs in a week or two on SAC.
Brendan Gibson
February 18, 2009 at 8:42 am
auto-emoticons …
meant to say especially IE8
Brendan Gibson
February 18, 2009 at 8:43 am
Have you guys looked at what the folks over at EtherPad are doing? They have the most interesting ideas I’ve seen on how to tackle online editing I’ve seen. You can listen to an interview on Phil Windley’s technometria podcast with Aaron Iba to get an idea of just how cool their app is: http://itc.conversationsnetwork.org/shows/detail3985.html
Mike Farmer
February 18, 2009 at 9:27 am
[...] this case, one of the big drivers for writing the previous Bespin and Canvas post was to discuss accessibility–which I then neglected entirely to mention. Let me rectify that [...]
Bespin and Canvas: Part 2 « Ben Galbraith’s Blog
February 18, 2009 at 9:49 am
As an experiment, I think what you’re doing is great. However, I’d echo some other comments above that essentially rolling your own contentEditable implementation in JS+canvas is the wrong way to go. I’m sure it makes your life a lot easier in the implementation, but in the long run it doesn’t benefit other users of the web. If it’s not possible to do what you want using contentEditable and other web features that browsers implement natively, then you should file bugs or make proposals for changes that would let you do what you want. It’s certainly not the ideal way to get something working now, but if you can convince browser makers to implement the things you need, then others can benefit from your work. As I’m sure you’ve seen, you’re not the only people to think about writing a web-based editor, and you won’t be the last. Bespin is a cool prototype, and the canvas-based approach may be the way to go for prototyping, but I’d urge you to find out how we can make existing web technology (not just canvas) suitable for what you’re doing.
Ted Mielczarek
February 18, 2009 at 10:46 am
[...] Bespin and Canvas: Part 1 Source: Ben Galbraith’s Blog Excerpt: [...]
Bespin and Canvas: Part 1 | Switch on the Code
February 18, 2009 at 1:10 pm
I don’t think contentEditable can really be fixed for this. First, any fix is likely to take a long long time to make it into browsers to be useful.
Secondly, the contentEditable spec seems to assume you are building a rich-text HTML editor, inserting , and other elements into the DOM without giving you full control. If you’re writing a code editor, do you really want all of these extra DOM elements auto-inserted by the UA, which I might add, seems to have considerable freedom in what it actually does?
Third, if you want a really high performance editor capable of dealing with large documents, you need full control over rendering in order to implement regional updates/repaint-on-dirty. This would essentially force the “stack of DIVs” scenario, with all of them set to contentEditable, but with numerous filters to prevent the UA from inserting junk DOM nodes. Is this really much better?
Maybe I’m wrong, and there’s an easy way to use contentEditable for this, I’ll confess I’ve never used it, only perused the spec. But if I were building a Bespin-like editor, I’d want to take full control over rendering and event handling, like just about every other IDE out there. If you look at platform IDEs like IntelliJ, NetBeans, Eclipse, etc they don’t use the bog-standard widgets provided by the operating system or platform SDK, but rather they roll their own, and for good reason.
Ray Cromwell
February 18, 2009 at 1:30 pm
Thanks to all for the kind words.
@Oscar Godson: Yes, canvas originated as a Safari-specific extension to HTML but is now a part of the HTML 5 specification. See “Bespin and Canvas Part 2” for more information on accessibility–the short answer is that we think there might be ways to make a canvas approach accessibility-friendly.
@Oscar C: Yes, WebKit does run wicked fast.
@Brad, Ray: We are very keen to try out Ray’s work on Canvas-on-Flash.
@Laurentj: Please see the Part 2 blog post for more information accessibility. We definitely look forward to some help in this area.
@smaug: Key handling needs a good rework.
We’ll get it right.
@Mark Finkle: Yeah, we’ll go into detail on all that.
@Brendan Gibson: Glad you’re having fun with canvas.
I talk about IE plans in the Part 2 post.
@Chris, Ted Mielczarek: In Part 2 I talk a small bit about our efforts to push functionality into the browsers themselves. We’d obviously love to see the web run-time itself enriched.
Ben Galbraith
February 18, 2009 at 1:42 pm
I think replacing contentEditable with canvas and handling all text manipulation with js, will be good in the long run. The first things I thought when I was looking the Baspin source was “Can I implement multiple column editing ?! Hm .. I can … super”
Radoslav Stankov
February 18, 2009 at 1:45 pm
I disagree with some of the other posters here, I am totally stoked about your from-scratch implementation. This is much more flexible and powerful than simply using some standard browser component.
I’m hoping it’ll ultimately get to something like the Java Swing text components…nicely factored out so if you want to, say, replace the text model, it’s easy to do, with just a half dozen or so methods to implement.
I’ve had a project in mind for a while that needs exactly this, and being able to implement it in the browser without much difficulty would be way cool
Dennis
February 18, 2009 at 2:26 pm
[...] Bespin and Canvas: Part 1 It’s been thrilling to finally release Bespin to the world. There’s nothing quite like seeing feedback [...] [...]
Top Posts « WordPress.com
February 18, 2009 at 4:15 pm
This is good news for those of us who have been frustrated by the thought tha HTTP was never going to become fast and responsive enough to feel like an desktop application.
So now I think I may have figured out the “You’re crazy” reaction: many web people have never gone through the experience of programming dots on a screen. From that perspective, writing an editor from the dots up might sound pretty insane.
But that’s what the vast majority of us are looking at right now: text characters built up from dots in software, courtesy, in a sense, of PARC and Steve Jobs. (Okay, and also Bill Gates, who always got there last and thereby made the most money.)
The original Apple II or PC technology, on the other hand, displayed text on the monitor by way of an Ascii screen buffer hooked and a ROM character generator, all built into the computer’s hardware. As the electron beam scanned across the TV tube, the character generator… but no, that would be too much detail for this message.
From a purely webby point of view, the browser and the PC’s operating software (Windows, Mac OS, X11, etc.) now perform the invisible job of the 40-year-old character generator video hardware. Only by means of Java, Javascript, Flash, etc., can a web programmer directly manipulate the dots. That’s why the canvas tag looks like a bold leap forward, even though in a sense it brings the web environment back to the introduction of the Macintosh in 1984, when the idea of the Mac seemed kind of crazy to me for a few minutes: wow, everything is made of dots! The idea of programming an interactive computer without a character generator was something like being asked to create a painting with only a toothpick dipped into some oil paint… like the crazy pointillists.
Ralph
February 18, 2009 at 5:10 pm
Housekeeping note: Your link to Part 2 is broken.
Tim McCormack
February 18, 2009 at 7:03 pm
[...] Bespin and canvas in depth http://benzilla.galbraiths.org/2009/02/17/bespin-and-canvas/ [...]
WebDevGeekly » Blog Archive » Episode 3
February 19, 2009 at 5:36 pm
The never-ending story goes on an on — that’s amazing! Every widgets system defines at a moment a *custom* widget and there are always people that start to create their own widgets system inside it. Their reasons always make sense: performance, usability, extensibility and so on. And the end is always the same: users prefer _native_ widgets.
And please read (reread?) about AWT vs. Swing vs. SWT — it’s the same story.
Dmitry Stadnik
February 20, 2009 at 1:42 am
Some users might prefer native widgets. I prefer IntelliJ IDEA’s editor over Eclipse’s StyledText widget. I much prefer Java2D over the SWT graphics API.
Of course, your comment is part of the whole Swing vs SWT flamewar. If users always preferred the native look and feel, then why are websites with custom UIs so popular? All of the UI widgets people create for the web, draggable regions, message boxes, trees, menus, etc are totally custom, abusing the DOM to create what wasn’t there natively.
The SWT flamewar in the Java community really has nothing to do with Bespin.
Ray Cromwell
February 20, 2009 at 1:53 am
And the end is always the same: users prefer _native_ widgets.
What do you mean by “native”? I’m guessing you mean something like the following: “implemented by means of system calls and environment-specific libraries which encapsulate common OS functionality and which therefore pertain only to the specifically targeted operating system under discussion.” Also I suspect you are concerned about speed of screen refresh, fast reaction to clicks and other performance-related issues.
I think you’re right when it comes to menus, buttons, scroll bars, window frames and similar objects. But, except with respect to speed, I’m not sure your point is applicable to a text editor pane. I’m pretty sure most word processors and text editors implements their own text panes using code written in, say, C++. Aside from speed, how does that approach differ from doing the same in Javascript? I anyone complaining that Google Maps is not “native” enough? Their map pane, as far as I know, is all implemented in Javascript.
Ralph
February 20, 2009 at 8:04 am
This is pretty amazing. Look forward to what you can do with it.
But the disturbing thought hit me: isn’t this just re-inventing NeWS, this time with Javascript instead of Postscript, and Canvas as the new Postscript-like rendering surface?
You even have to solve the same problem of building GUI toolkits for your Canvas-based “window server.”
Chris Ryland
February 20, 2009 at 4:25 pm
Hey, NeWS was a great for it’s time, and the idea of Postscript-like rendering surfaces lived on in DisplayPostscript on NeXT, and Quartz on OSX.
I don’t really see much different between building GUI toolkits for the canvas, vs building them for the DOM, the exception of the browser handling hit detection/event propagation, and CSS handling layout decisions. Otherwise, people already build completely custom widgets today which override tons of native browser behavior.
Ray Cromwell
February 20, 2009 at 4:34 pm
[...] Why Canvas? [...]
Supporting the system clipboard in your Web applications; What a pain! on Dion Almaer's Blog
February 26, 2009 at 9:02 am
Hi Ben,
an interesting and deep thread to be sure, thanks for pointing me in it’s direction, it’s nice to see controversy produce some hard hitting results and dispelling some myths.
I’ve responded to your message here:
http://www.trisis.co.uk/blog/?p=113#comments
I’d be curious to know what lessons you guys have learn’t from doing this in Canvas, whether if speed and flexibility weren’t an issue and you could have done this successfully in XHTML/CSS/JS would you have?
Simon Kenyon Shepard
March 17, 2009 at 12:01 pm
[...] une balise HTML 5, mais ne paniquez pas !), tels Bespin, qui ne definit qu’un aspect visuel. Le choix est largement justifié, mais ça ne change pas la problématique : sans JS, point de salut. Evidement, on peut toujours [...]
HTML 5 VS AJAX : l’amputation sera nécessaire | E-vidence
March 22, 2009 at 10:55 am
[...] Bespin and Canvas: Part 1 « Ben Galbraith’s Blog (tags: canvas javascript) [...]
Knowtu » links for 2009-08-16
August 16, 2009 at 5:05 pm
[...] [...]
OnlineEditor (Syntax-Highlighting) - AJAX (Asynchronous JavaScripting and XML) Forum
August 26, 2009 at 1:18 am
[...] created using JavaScript (only 62kb compressed, apparently) and the HTML5 canvas element (update: more on that implementation). Seeing other examples of what its capable of, I really need to start learning more about [...]
Bespin, the cloud, and Canvas | Broken Links
October 5, 2009 at 4:25 am
I was wondering if we could get bespin and the palm ares toolkit working on the palm pre browser. It would definitely be cool.
bespin.mozilla.org loads up on the pre browser, but I can’t log in as the log in bar is in a div that is not rendered on the pre browser.
I’m going to try a self hosted version of bespin next.
The next step is to make a pre app – just embed bespin as a general purpose code/text editor for the palm pre!
Jonathan Lin
January 19, 2010 at 11:58 am