Henri Bergius - Oscom

Forget about HTTP when building microservices

henri.bergius@iki.fi (Henri Bergius) — Thu, 13 Apr 2017 00:00:00 +0000

Microservices — an architectural pattern we recommended in our 2012 International PHP Conference keynote — is pretty popular these days. There are many benefits to consider:

Independent development and release lifecycle for each microservice
Ensuring clear API boundaries between systems
Ability to use technologies most applicable for each area of a system

In an ideal world, microservices are a realization of the Unix philosophy as applied to building internet services: writing programs that do one thing, and do it well; writing programs that work together.

Just use a message queue

However, when most people think about microservices, they think systems that communicate with each other using HTTP APIs. I think this is quite limited, and something that makes microservices a lot more fragile than they could be. Message queues provide a much better solution. This is mentioned in Martin Fowler’s Microservices article:

The second approach in common use is messaging over a lightweight message bus. The infrastructure chosen is typically dumb (dumb as in acts as a message router only) - simple implementations such as RabbitMQ or ZeroMQ don’t do much more than provide a reliable asynchronous fabric - the smarts still live in the end points that are producing and consuming messages; in the services.

When we were building The Grid, we went with an architecture heavily reliant on microservices communicating using message queues. This gave us several useful capabilities:

Asynchronous processing
If you have heavy back-end operations or peak load, a microservice might be swamped with work to be done. If your web server only needs to send work to a queue and not wait for result immediately, you have a lot more freedom on how to organize the work. Furthermore, you can split your service along different scalability characteristics — some systems may be network-bound, others CPU-bound etc
Autoscaling
Since the work to be performed by your microservices is kept in message queue, you can use the combination of the queue length and typical processing times to automatically determine how many instances of each service you need. Ideally you can use this to ensure that your processing times stay consistent regardless of how many users are on your system at a given time
Dead lettering
It is possible to configure RabbitMQ to place any failed operations into a dead letter queue. This gives you a full record of any failing operations, making it possible to inspect them manually, replay later, or produce new tests based on real-world failures
Rate limiting
Sometimes you’re dealing with external HTTP APIs that are rate limited. Once a microservice starts hitting rate limits, you can keep new requests in a queue until the limit lifts again
In-flight updates
A message queue can be configured so that non-acknowledged messages go back into the queue. This means that if one of your services crashes, or you deploy a new version of it, no work gets lost. When the service is back up again, it can pick up right where the previous instance left off

Better tooling

HTTP is something all backend developers are familiar with. With message queues you have to deal with some new concepts, and hence new tools are also needed.

MsgFlo

Client libraries for talking with common message queues exist for pretty much every language. However, this still means that you’ll have to handle things like message pre-fetch limits, acknowledging handled messages, and setting up queues yourself. And of course keeping track of what service talks to what can become a burden.

We developed the MsgFlo tool to solve these problems. MsgFlo provides open source client libraries for bunch of different programming languages, providing a simple model to handle message-related workflow.

To give you an overview of the dataflow between services, MsgFlo also provides a way to define the whole system as a flow-based programming graph. This means you’ll see the whole system visually, and can change connections between services directly from graphical tools like Flowhub.

If you’re using Heroku, MsgFlo can also generate the Procfile for you, meaning that the services you’ve defined in your graph get automatically registered with the platform.

Guv

As mentioned above, queue-based microservices make autoscaling quite easy. If you’re on Heroku, then our Guv tool can be used to automate scaling operations for all of your background dynos.

I wrote more about GuvScale in a recent blog post.

Conference talk videos

henri.bergius@iki.fi (Henri Bergius) — Fri, 31 Mar 2017 00:00:00 +0000

As I’m preparing for a NoFlo talk in Bulgaria Web Summit next week, I went through some older videos of my conference talks. Sadly a lot of the older ones are not online, but the ones I found I compiled in playlists:

Some highlights

NoFlo, Managing Workflows with JavaScript, my first NoFlo talk, Wakanday 2011:

Decoupling Content Management with Create and PHPCR from SymfonyLive Paris 2012:

The Internet is your Application Blueprint, keynote with Lukas at the International PHP Conference 2012 where we talked about microservices:

Introduction to Flow-Based Programming and NoFlo at c-base in 2013:

If you find any I’ve missed, please let me know!

The Grid: Web Design by Artificial Intelligence

henri.bergius@iki.fi (Henri Bergius) — Wed, 17 Feb 2016 00:00:00 +0000

As mentioned last year, I’m working on a Artificial Intelligence that can do web design. It is called The Grid. Last week I gave a talk at Lift Conference explaining how it all works.

If you can’t see the video embed above, it is also available on the Livestream site and YouTube.

Why did I reimplement Jekyll using NoFlo

henri.bergius@iki.fi (Henri Bergius) — Mon, 12 Aug 2013 00:00:00 +0000

Jekyll is a delightful piece of software. A Ruby application that turns your Markdown and HTML files to a nicely constructed static website. Since the generated site is static, you can deploy and serve it from anywhere with no security or performance concerns. As a matter of fact, this site is built with Jekyll.

For websites that don’t need to offer dynamic functionality this is in many ways the culmination of Decoupled Content Management — you can write the post using whatever editor, use GitHub as the content repository, and deploy the pages to any web server you want.

If you’ve been to our Kickstarter project you probably know that I’ve reimplemented Jekyll using the NoFlo flow-based programming environment. If I’m so happy with Jekyll as it is, why would I do such a thing?

Introducing noflo-jekyll

Before I go to the reasons, let me briefly introduce the project itself: noflo-jekyll

Just like any flow-based program, the NoFlo reimplementation of Jekyll is built out of a graph. Here is how the main graph of the application looks like when loaded via the NoFlo UI prototype:

Quite a few of the boxes you see in that graph are actually subgraphs. Since the UI is still under work, the subgraphs have been defined in the more text editor friendly .fbp syntax.

In total at the current version the graphs contain 107 nodes. Most of these running a component from the extensive NoFlo component libraries, but there are also 4 custom components built for this project. These components put together are under 500 lines of code, and everything else is using 100% reusable code from existing libraries. Not bad compared to the size of the original Jekyll code base at over 16k LoC!

You can grab the first release of noflo-jekyll right now from NPM. Please refer to the README on how to use it.

Where this is coming from

When I started working with the rest of The Grid team last year, Jekyll development had practically stopped. It looked like nothing was happening in that community. Since Jekyll was something we very much wanted to utilize for various projects, the situation looked a little bit risky.

Since we’re very much a NoFlo shop, it felt natural to take Jekyll as sort of a specification, and reimplement it using FBP techniques. Initially this was an internal effort, but then we decided, very much in the nature of GitHub’s “open source (nearly) everything” philosophy, that this should be opened as well.

I got noflo-jekyll into a running state in about a week, and then moved my attention to other things we needed to build. And so the finishing touches and a public release were postponed to a better day.

As it happens, Jekyll development resumed pretty soon after that. But there are still many reasons why noflo-jekyll can be quite cool.

Benefits of the NoFlo port

Here are some reasons why especially Node.js developers should care about this project:

Pure JavaScript, no need for Ruby or other runtimes in your environment. Especially handy if you’re using Grunt for site generation
Other data sources, in NoFlo everything is just a flow of data. You could easily plug in other data sources than the file system. For example, database query results
Different converters, don’t want to use Markdown? Just plug in your own mark-up processor component
Different template engines, don’t want to use Liquid? Just plug in your own template processor component
Use as library or executable, this Jekyll implementation is just a NoFlo graph. You can use it in other NoFlo applications, as a Node.js module, or as a command-line executable

However, as with any reimplementation of a application being actively developed, there are also some potential caveats to observe.

A NoFlo example

Most of the existing NoFlo applications out there are dealing with various business processes, and so very little of that is available out in the open. As such, noflo-jekyll can probably show how to build bigger systems out of flow-based graphs, and also how to connect them with the rest of the Node.js ecosystem.

noflo-jekyll is now available under the MIT license for your static site generation needs. But as it happens, it wasn’t the only interesting NoFlo example to be released this week. You may want to also check out how to handle HTTP requests in NoFlo with the Woute module. There is even an example project available.

Keeping up with Jekyll

To realize the benefits of flow-based static site generation, it is quite important to keep the NoFlo reimplementation up-to-speed with things changing in Jekyll-land.

Because of this, the most important part of our automated tests is comparing results between a site generated by the original Jekyll program, and the NoFlo version. This means that when new features are added to Jekyll, we can follow the red-green-refactor style of test-driven development, and add these features to the test site.

I’ve been happily building my blog and the NoFlo site with this for a while, and except for some minor templating and Markdown handling differences between the Node.js and Ruby versions of those libraries, things are looking solid. If you find something that doesn’t work with the site or templates you have, please let us know!

The mobile-first Web

henri.bergius@iki.fi (Henri Bergius) — Fri, 07 Jun 2013 00:00:00 +0000

The growth of mobile web users is staggering. While some of us have been browsing the web on mobile devices for nearly ten years, most of the world population is only now getting there.

The number of mobile web users is already at 1.5 billion, which happens to be quite close to the total number of Internet users back in 2009.

And it is growing rapidly. In 2015 there will be an estimated 2 billion smartphone users which is quite close to the total number of Internet users currently.

In the developed world, this is likely to be a mixture of tablets, smartphones, and traditional desktop computers, with most users having at least two different web-capable devices. In the developing world, the smartphone is the computer.

Considering these statistics, it is insanity to design websites and services PC-first, with mobile only as an afterthought.

How to prepare

Just some years ago, the mobile web was a slum. Instead of getting full-featured websites, many sent us out to poorly-built and featureless m. sites. Now more and more sites go with responsive web design that makes the site itself adapt to different screen sizes and resolutions.

But even with responsive design, it is easy to go overboard. Tools like WordPress Jetpack and jQuery Mobile oversimplify the site itself by trying to make it look and feel like a native app. In the mobile-first world this is not the right way to go.

In The Rise of the Mobile-Only User content strategist Karen McGrane makes a valid point (emphasis added):

Mobile users should get the same content. It’s frustrating and confusing for them if you only give them a little bit of what you offer on your “real” website. If you try to guess which subset of your content the mobile user needs, you’re going to guess wrong. Deliver the same content as your desktop user sees. (If you think some of your content doesn’t deserve to be on mobile, guess what — it doesn’t deserve to be on the desktop either. Get rid of it.)

There is no pixel-perfect

In this new world users will access your content or software using a wildly varying set of devices. And each of them has the reasonable expectation of being able to access the full experience and the full set of features you’re providing.

This changes web design substantially. Even in the old world of different PC browsers, pixel perfect web design was rarely that. With responsive design, it is even less so.

Instead:

Think in visual components instead of full pages. The composition of a page out of these components can vary for different screen sizes
Design the compositions always for at three screen form factors: a full-sized desktop or tablet screen, a smartphone screen, and the 7” tablet in between
Make your user interface elements big enough to be used on inaccurate touch screens
Never, ever require a plugin to access some content or functionality

CSS Media Queries make responding to different form factors quite easy. And besides that, they also make it easy to optimize for the different screen densities we now have. This way your images will look sharp on anything the users have, from the “retina-class devices” to the lowest-specced Chinese smartphone, while requiring the user to only download the assets that their device can utilize.

The devices people use to access the services you provide will vary greatly not only in their display capabilities, but also in the ways you can do input. Some will have mice and physical keyboards, but an increasing amount will instead have a touchscreen. For these users, it is a big service to use the correct HTML5 input types so that the on-screen keyboards and widgets can adapt to the content being entered.

The web is not native

The web is its own platform, and as such it is foolish to try and mimic traditional desktop applications. It will never feel quite right whatever you do.

It is a lot better to accept this and fully embrace the unique advantages of the web platform:

The web is an universal runtime that works on 100% of the computing devices your users have
There are no gatekeepers telling what you can publish, and no middlemen taking a cut of whatever you sell online
The web is built out of URLs that users can easily share with each other, and continue using when they switch devices
URLs also allow any application on the web to link to any screen or state of another application
It is just as easy to provide content as it is to provide functional applications on the web

Every major software company on the planet has their own web browser, and the competition between these is fierce. This will ensure that over time, the web will keep on getting better and faster. Compare this to traditional software platforms that can easily stagnate or get abandoned. Thanks to the standard protocols it also allows you to use any technology of your choosing for the server side of your software.

Paul Graham put it well in his The other road ahead from 2001:

And you don’t have to know if you bet on Web-based applications. No one can break that without breaking browsing. The Web may not be the only way to deliver software, but it’s one that works now and will continue to work for a long time. Web-based applications are cheap to develop, and easy for even the smallest startup to deliver.

Things are getting better

As somebody who has been developing for the web for nearly twenty years the rate the web developer experience keeps improving is sometimes dizzying.

We get used to some limitations in the stack, and then suddenly something comes along and removes that shortcoming. We’re still exploring the new kinds of designs and visual experiences technologies like Media Queries and WebGL make possible, just like it took years for the community to find best practices around things like AJAX.

And yet new amazing things keep pouring in. My personal favourite recently has been Web Components which gives a standard way to provide reusable widgets on the web, and to do things like data binding and templating. This alone will make a lot of the popular frameworks and libraries obsolete.

Just watch this video and see how much easier web development is becoming:

The current work on standardizing Web Payments together with more grassroots efforts like Bitcoin promise to add better ways to do business on the web. This should remove the last big advantage of native applications in that they’re easier (but very expensive) to monetize via app stores and in-app payments.

Linked Data on the web and new features like Yandex Islands make it easier to connect web applications and data on the web together. Tools like my Create.js make the web easier to edit, and Web Intents promise even closer integration between web apps.

Each of these will make the web richer and better. Each of them will allow new startups to be built, and new meaningful connections to happen over the internet, many of these using mobile devices.

It is an exciting time to be a web developer.

Automated linking with rich text editors

henri.bergius@iki.fi (Henri Bergius) — Tue, 30 Apr 2013 00:00:00 +0000

The web is built of links, of pages linking to other resources on the internet. But making those links manually is tedious. This is another area where modern inline editors could do better.

Yesterday on Hacker News, there was a thread about Wikidata, Wikimedia Foundation’s new knowledge base. This comment struck me especially:

I was using Wikipedia the other day and it occurred to me how primitive it is to have all the inner links to other Wikipedia articles defined manually, surely these should have been automated by now (i.e., marking a word or two would link you to the relevant article).

And indeed, this is a usability problem that can already be fixed with the Semantic Interaction stack underneath Create.js.

Introducing Annotate.js

Annotate.js is a VIE widget built by Szaby Grünwald. It works very similarly to a spell checker in traditional text editors — you write text, and it highlights the potential entities you might want to link to. You then can either accept or decline these link suggestions by clicking them. In case of multiple potential matches, you can also disambiguate between them by selecting from an offered list.

Here is a quick video of Annotate.js in action:

You can also try it yourself with an online demo.

Currently Annotate.js has been integrated with the Hallo rich text editor, but it would be easy to do the same with other popular editors like Aloha and CKEditor.

Connecting to entities

The big question with automatic linking is where the entities come from. There are services like OpenCalais that can provide these suggestions for your content, but most of them are focused only on shared knowledge bases of big companies, famous people, and major cities.

Unless you’re running a newspaper, it is unlikely that these are the things your content is about.

Apache Stanbol is an open source engine that can provide the enhancements for you. Out of the box it provides suggestions based on the Wikipedia knowledge repository. But more importantly, you can feed it with your own entities.

This way the enhancements you get for your content can be tuned to be meaningful to your content and your audience. If you write about medicine, they could be about symptoms and diseases, or if you’re writing about technology, they could be specific open source projects and their contributors. With Stanbol, the choice is yours.

The current downside of Stanbol is that you’ll have to run it yourself, but there may be solutions coming for that as well.

Beyond editing

Like never losing your content or managing collections, Annotate.js shows what we can do to improve the editing experience when we interact with it in a semantically meaningful manner.

What Annotate.js does is not merely creating links, but it also marking the machine-readable relationship between them and the HTML content being edited. This can then be used by yet another set of tools — like search engines — to understand and organize the content better.

It is easy to see Create.js (like Drupal did, unfortunately) as just an easy way to add nice inline editing features to your CMS. However, while that is a good initial step, the addition of being able to interact with your content on the semantic level can do a lot more. Automated linking is just another demonstration of that.

As the ecosystem around Create.js and VIE matures, and it ships in more systems, there will be things that we can’t even imagine now built on the stack.

If your CMS is properly decoupled, you can benefit from that immediately.

Why WordPress needs to get Decoupled

henri.bergius@iki.fi (Henri Bergius) — Fri, 22 Mar 2013 00:00:00 +0000

Couple of days ago there was an interesting post on the Dire State of WordPress, talking about the issues developers have when working with this hugely popular content management system:

When you learn PHP from WordPress as I (and probably many other people) did, you just assume that all the idiosyncrasies and illogicalities are par for the course when building a CMS-driven site. It’s taken me several years to realise that it doesn’t have to be like this. In fact, it wasn’t really until I started learning more about OO principles and working with frameworks that I started looking at WP in a different light. Why is every instance of every model treated like a post? Why can’t I extend the functionality of a post type without writing a whole mess of hooks, filters and functional code into a file that isn’t semantically related to that post in any way?

These may sound like the gripes of someone whose needs will never be met by something like WordPress, but the lack of fundamental logic that gives rise to the above issues is what makes WordPress problematic for developers at all levels. And because the crossover on the venn diagram of ‘people who know good application development practises’ and ‘people who build sites in WordPress’ is such a thin sliver, it just trundles along in its current hokey state, baffling both the unseasoned and the proficient developer alike.

As expected, this generated a lively discussion on Hacker News.

Technical debt

Now, I’ve used WordPress on-and-off ever since it was still called b2 cafelog, but I wouldn’t consider myself a WordPress developer. But still, looking at it mostly from the outside, it is clear that many of the issues WordPress users and developers face today result from technical debt accruing from bad architectural decisions done a decade ago, and the missing will to correct those.

WordPress is still undergoing a transition from a blogging platform to a general-purpose CMS. The changes this requires mean that without a sound architecture, the problems with the underlaying engine will only become exacerbated.

This is not something unique to WordPress. Many popular CMSs, including Drupal and TYPO3 have been suffering from the same problem. We all in the open source CMS community have been there. The difference however is that these projects have identified this issue, and have taken steps to correct it.

Start decoupling

The reason why CMSs become so messy is because they all try to be so many things at once. A CMS is a web framework, a CMS is a web editing tool, and a CMS is a content store. But each of these are in reality their own, specialized areas, and so the principle of separation of concerns tells us that they should be handled by separate pieces of software.

I wrote a still-popular post titled Decoupling Content Management two years ago to argue just that. Projects like PHPCR and Create.js have since then sprung up to move that from idea to practice, and have already been adopted by many CMSs.

In nutshell, the transition I want CMSs — WordPress included — to make is this:

Here is my talk on the subject from JS.everywhere in Paris late last year:

We also talk about the cross-CMS collaboration this enables in the TYPO3 Neos launch video.

Think about it, developers from Midgard CMS and TYPO3 working together and sharing substantial parts of the user interface codebase, while still enabling each project to retain a unique look-and-feel!

Non-participation

Over the past two years I’ve given tens of talks about Decoupled Content Management in various developer and CMS conferences. In these events I’ve talked with a lot of developers from various different CMSs from both the open source and proprietary worlds. But despite its popularity, WordPress core developers have been absent.

Symfony activist Lukas Smith has noticed the same:

I think the first step would be that the WordPress core developers start to mingle with the PHP community. I simply have not met a single WordPress core developer at any PHP conference and I tend to go to 6-10 a year in various countries around the world.

I have also not seen any WordPress core dev participate on an php-src internals discussion let alone something like the Framework Interoperability Group that outs out the PSRs.

But without communication the chances of collaboration are low and without collaboration imho its going to be hard for WordPress to identify and assess the potential for new directions.

Moving forward

While WordPress is still the top dog of the CMS world, it is a precarious position that shouldn’t lull them into complacency. The web moves forward, things change, and all technical debt must be paid eventually. And that is what can fatally slow down a project.

Solving the architectural issues is a big amount of work, and so I would start by examining how other projects like Drupal have made their transition. The part Symfony components to the rescue of your PHP projects of my Symfony Live Paris liveblog is a good place to start.

For WordPress, like for any PHP content management system, my secret agenda for PHP CMSs should apply. If you’re a WordPress core developer, I’d love to chat about how to move forward.

Create.js in 2013

henri.bergius@iki.fi (Henri Bergius) — Sun, 06 Jan 2013 00:00:00 +0000

It is now 2013, and the IKS project, started back in 2009 to improve content management systems through semantic technologies, has ended. Alongside Apache Stanbol and VIE.js, the Create.js inline editing toolkit was one of the major outcomes of this European Union funded effort.

This post outlines the current state of Create, and some of the things that will be happening around it in 2013.

Increased CMS adoption

Thanks to being picked up by major CMSs like Drupal 8, TYPO3 Neos, Midgard, Symfony CMF and OpenCms for their inline editing needs, Create.js is experiencing very strong adoption. Many of these CMSs will be shipping a new major version during 2013.

These releases will bring Create.js into millions of sites out there, giving users a friendlier and faster way to edit the web. By being built on the semantic interaction framework of VIE, this also means a significant expansion of the RDFa Linked Data base.

This amount of new users will be staggering, meaning that we will need to put a lot of focus into stability and ease-of-use of the toolkit. To make life for new integrators and contributors easier, we will also need to improve the documentation and test coverage. All of this is shown also by the fact that Create.js on GitHub has climbed from few hundred watchers a year ago to nearly 1400 today.

Any editor you want

The first versions of Create.js were written exclusively with the Aloha Editor in mind. Later on we added support for Hallo.js in order to provide a simpler, MIT-licensed alternative. However, the needs of CMSs change and new promising editors are released.

With this in mind, we received a contribution from the OpenCms team to have a more generic editor abstraction allowing integration of any new editors that CMSs may want to integrate. With it, the latest versions of Create.js support also Redactor and CKEditor 4 alongside Aloha and Hallo.

The utility of this abstraction layer was proven recently when Drupal switched from Aloha to CKEditor.

Hallo.js:

Aloha Editor:

CKEditor:

Redactor:

In addition to the editors we now support, integrators can use the reasonably simple property editor API to implement their own editors when needed. For example, Drupal provides a custom editor widget that utilizes forms rendered on the server side for editing more complex field types.

New default UI

As many of you probably know, Create.js can be used on two levels: as a full inline editing interface, or as a set of components for building your own editing UI.

In the latter part of 2012, much of focus was put into the components themselves, as that is the way most CMSs leverage Create. But the default user interface is also important, and so I have been collaborating with the Berlin-based designer Marie Schweiz to improve the user experience of stock Create.js. This work is still in the concept stage, but I’m quite happy with the direction we’re headed.

One important concern to tackle with the new UI is handling of non-visible metadata and the other concerns raised by Lullabot’s excellent critique of inline editing. Here you can see some of that in action:

Other important parts is to use colored indicators for showing what can be edited, and its state (published, untranslated, etc.), and to make the process of adding new content clearer:

Clear visuals and bigger touch targets should also make the new UI friendlier for tablets and other new touchscreen devices.

New editing tools

The main purpose of Create.js is to enable inline editing of textual content, but even in the original versions we had additional features like image handling and versioning that most CMSs need.

Adding more functionality related to non-textual content is highly useful, and something that ought to go hand-in-hand with the new UI effort. In the new default UI we have a large overlay (seen in blue in the geolocation editor) that can be used for new interesting editing features, like better image pickers, or even editing images and videos to be inserted to a page.

Another idea I’ve been toying around has been a more semantic way of handling tables. Instead of editing tables in the traditional WYSIWYG way, how about making queries to your CMS system and then presenting the data in tabular manner? Aloha’s Blocks feature can give a lot of inspiration here.

Getting involved

In 2013 the development of Create.js should happen in a lot more community-driven manner. Given the massive adoption of the library, this shouldn’t be a problem.

If you’re interested in getting involved, take a look at the repository on GitHub, file issues, and send pull requests. The most immediate concern is getting the 1.0.0 stable out of the door, which should be helped greatly by the recently-released VIE 2.1.0.

For discussing both development and integration of Create.js, the Create.js Google Group and the #iks IRC channel on FreeNode are the best options. The Create.js change log is also an important thing to watch.

Interview: Create.js and VIE in CMSs

henri.bergius@iki.fi (Henri Bergius) — Fri, 30 Nov 2012 00:00:00 +0000

Create.js and VIE were recently added to the core of Drupal 8. Just like with TYPO3 Neos, I’ll write a longer post on how things went later.

The German PHPmagazin was already fast enough to interview me on that at VIE und Create.js: Warum In-place Editing so erfolgreich ist.

While there were many interesting CMSs already using Create – including OpenCms, Symfony CMF, Midgard, and Neos – I can understand why the integration with Drupal has received so much notice. After all, the system is estimated to have a 7% market share of all CMSs, and so to run more than 2% of the entire web. In 2010 this was calculated to mean about 7 million websites. These numbers are big.

The PHPmagazin interview was published in German. For the more international audience, I received permission from the author to publish the English version here on my blog:

First of all congratulations to the success of your libraries: Symfony CMF and Drupal seem to like you very much.

Thanks! I’m really happy that major CMSs like Drupal and TYPO3 have joined the Create.js effort, as well as some of the newer projects like Symfony CMF. We’re still sadly missing some of the big names like WordPress and Joomla. I’d be happy to help them get started as well.

It should be noted however that the libraries have been a team effort. While I started and do maintain them, many others from the IKS consortium and the various CMS projects have contributed significantly.

How come, they consider your libraries over others?

I’d imagine it is the combination of:

Timing (they all have been thinking about inline editing in this development cycle)
Standards (VIE and Create work with the web standards, not against them)
Focus (the Create library is really only intended for inline editing for CMSs, nothing more, nothing less)
Licensing (MIT license allows them all to integrate this code without any legal hassles)

I could expand a little bit on the timing point. The HTML5 suite of standards is now out with useful features like localStorage, contentEditable, and RDFa, and these already have decent support in mainstream browsers. There are also great tools like Aloha Editor and my Hallo.js utilizing them. This all means the time is right for bringing direct manipulation interfaces into content management systems.

What does it mean for you libraries to be part of a project? Do we have to fear fragmentation into many builds for each CMS?

Obviously versioning will be an issue. I hope the CMSs will track versions of my libraries as closely as their release cycle allows, but in any case backwards compatibility is something that we will have to take seriously, as this library will be installed on millions of servers.

As for fragmentation, I think the fact that VIE and Create encourage CMS projects to consider and adopt standards like RDFa and JSON-LD will actually reduce fragmentation between systems and increase possibilities for interoperability and shared code.

What kind of issues are there to be fixed when implementing vie/create into a CMS?

Inline editing is quite closely connected to two areas in a CMS:

We need to be able to tell in CMSs output templates where the editable things are, and what they are (usually via RDFa annotations)
We need to be able to save changes back to the server (usually via a RESTful API)

Both of these are things where most CMSs have had to do some server-side work to make Create.js possible, and there have even been libraries like CreatePHP built to make that easier.

At the same time, adding support for the different capabilities of the different CMSs has meant that Create and VIE have become better with each new system we integrate with.

Inline editing: you have to do it right

henri.bergius@iki.fi (Henri Bergius) — Mon, 05 Nov 2012 00:00:00 +0000

In a curious turn of events, the Plone team is considering to remove their inline editing feature around the same time when similar features are being added to popular CMSs like TYPO3 and Drupal.

While the reason why Ploners are considering to remove the feature is technical, wanting to remove an old library dependency, the discussion thread contains quite a lot of people complaining about inline editing. Some examples:

No problem for us, we always disabled the feature. It confused users more than it helped them.

I positively hated that feature and it was always one of the first we switched of in a new site. So to my mind it can be removed.

What went wrong?

At the same time we’ve been getting a lot of positive feedback for Create.js, the inline editing toolkit many CMSs are now using. Are Plone users to different in their preferences? Turns out this might not be the case, and the issue with Plone’s inline editing feature is just poorly thought-out UX:

inline edit was a great feature. People was not complaining against inline editing itself and most of them appreciated it a lot.

What was the real problem with end users? Users just hated opening the inline edit when trying to click on a link on a document. This could be solved very easily and I’d like a future reimplementation of inline edit keeping in mind these usability issues.

Create.js could help

So, the important lesson here is that UX matters. This is why in Create you have a separation between the editing state (where clicking on something allows you to edit it), and browsing state where all content behaves normally.

In general we’re putting quite a lot of effort into the user experience, for instance by helping users to never lose content or to manage collections. We ship Create with a UI that we feel works well, but still allow projects to build their own user experience on top of the common infrastructure.

During the course of the IKS Project, I’ve worked closely with several CMS communities, even participating in their hackathons and developer events. I’d be more that happy to help the Plone community to improve their editing experience the same way.

If you’re interested in a better inline editing experience, the Create.js integration guide is a good place to start. My post Decoupling Content Management gives more background on the approach. There are also videos of Decoupled CMS talks available.