Introducing 'Usertext' - a plugin for Rails

I was at BarCamp Sheffield at the weekend, which was both fun and interesting, especially as it was a crowd of people I mostly hadn't met before.

One of the principles of BarCamp is that everyone is encourage to lead a session. At BathCamp I gave a short talk about Lego, and the various fan and developer communities that have sprung up around it. For BarCamp Sheffield, I thought I'd do something a bit different and actually show off some code I'd written. I'm not really a heavy web developer - I spend more time thinking about what websites should do now - but I still like to tinker.

The thing I've been tinkering with recently is a plugin for Ruby on Rails. This started out as a specific need for another project I'm working on (which isn't ready to be released yet), but I decided it was interesting enough to be released as a plugin that anyone can use.

The plugin fills a particular itch of mine, which is the way that user input gets 'marked up' (turned into HTML) on various websites. Currently, there are a few different approaches to this. Some website just convert line breaks into <br> tags and ignore everything else. Other websites might use some form of markup language like Textile or MarkDown or BBCode. These are all designed to be as simple as possible, but require users to learn special codes like [i]this[/i] or _this_ or *this* in order to emphasise words so that they appear in italicised text. Which isn't a big deal if it's a site you're spending a while on, and are prepared to invest the effort required to learn the codes, but it's pretty annoying for websites where you want to just leave a quick comment or message.

The Wired article A Million Little Markups recognises this problem, but the solution they suggest is just using 'plain old HTML', under the assumption that 'if your users don't know yet they will need to learn at some point'.

You can't actually just output whatever HTML your users submit, as that leaves all sorts of security holes, so you normally end up having to say 'some HTML accepted' and then list what tags are available. This is what a lot of blogs currently do, and is the default behaviour of some of the most popular blogging platforms.

For me though, asking people to learn a markup language, whether its BBCode or HTML, is asking too much. I wanted users to just be able to type a message fairly naturally, and then have the text appear as they expected. In short, I wanted to turn the problem of designing a simple markup language that's easy to use into a problem of interpreting and marking up the text that people already use.

So that was the aim for this plugin. I can't say it's neccessarily 'there' yet, but I think it's a pretty good start. And the philosophy of this approach means continually revising and honing the algorithm as you learn more about how people write.

The plugin is called Usertext (because it's for marking up the text of 'users', not publishers), and it's licenced under the MIT Licence, which means you can do with it what you want, without restriction. To install it in a Rails project, you just type script/plugin install http://usertext.googlecode.com/svn/trunk/usertext -x.

If you want to just try it out, visit www.usertext.org, and just start typing into the box on the left. The box on the right shows the output, and can be toggled between showing the HTML or the formatted text.

Try it out first, and let me know how intuitive it seems. Does it detect your bulleted and numbered lists or blockquotes?

Once you've done that, you can try out some of the more subtle typographic features, such as the way 'straight quotes' get turned into curly quotes or apostrophe characters, dashes get turned into hyphens or em dashes, and three full stops gets turned into an ellipsis (…). Other little features include converting currency references to the appropriate symbol (YEN25 becomes ¥25) and turning emoticons into smiley face symbols (:-) becomes ☺).

As this is the first bit of code that I've been released, I'm really interested in feedback, bug reports and feature requests. You can either leave a comment below, e-mail me, or submit an issue on the Google Code page. I've already had some useful suggestions from BarCamp Sheffield attendee David A. Black, so I'm pleased I did the presentation. Perhaps sharing your code isn't so scary after all...

Update: I've uploaded the slides from my session to Slideshare. I can't promise they make a huge amount of sense on their own, but they might be useful...