Web Content Repositories are more than just plain old relational databases. In fact, the requirements that arise when managing web content have led to a class of content repository implementations that are comparable on a conceptual level. During the IKS community workshop in Rome we got together to compare JCR (the Jackrabbit implementation) and Midgard's content repository. While in some cases the terminology might be different, many of the underlying ideas are identical. So we came up with a list of common traits and features of our content repositories. For comparison, there is also Apache CouchDB.

So, why use a Content Repository for your application instead of the old familiar RDBMS? Repositories provide several advantages:

• Common rules for data access mean that multiple applications can work with same content without breaking consistency of the data

• Signals about changes let applications know when another application using the repository modifies something, enabling collaborative data management between apps

• Objects instead of SQL mean that developers can deal with data using APIs more compatible with the rest of their desktop programming environment, and without having to fear issues like SQL injection

• Data model is scriptable when you use a content repository, meaning that users can easily write Python or PHP scripts to perform batch operations on their data without having to learn your storage format

• Synchronization and sharing features can be implemented on the content repository level meaning that you gain these features without having to worry about them

In addition to being a GObject-powered content repository for PHP, Python and Objective-C, the Mjolnir release provides several significant goodies on top of the older Midgard2 series:

• MgdSchema Views provide a way to define joined data types
• MgdSchema types can be extended with subclasses and additional information
• Transactions are available for storage engines that support them
• New authentication API provides support for different types like OpenID and OAuth
• ...and there are a lot more unit tests, helping to safeguard against regressions

We've been testing running the Qaiku microblogging service with Mjolnir. The exactly same PHP code that we used with Midgard 8.09 LTS performs 20-60% better when running on Mjolnir.

Get Midgard2 9.09 Mjolnir while it is hot! Builds for various Linux distributions are already starting to hit OBS repositories...

There are many examples of this happening, from Facebook's framing of web to netbooks systems like the JoliCloud not having an address bar. Certainly many companies are looking at Mozilla's search engine revenue and Apple's app store model and want to emulate that, moving the web into silos of their own control. But at the same time, we're thinking of Linked Data and open, interoperable web standards.

Web indeed is at new crossroads.

Chris Messina predicts the death of URLs:

a future without URLs and without the infinite organicity of the web frightens me. It’s not that I know what we’ll lose by removing this artifact of one of the most generative periods in history — and that’s exactly the point! The URL and the ability for anyone to mint a new one and then propagate it is what makes the web so resilient, so empowering, and so interesting! That I don’t need to ask anyone permission to create a new website or webpage is a kind of ideological freedom that few generations in history have known!

Tim O'Reilly presents a call to arms:

It could be that everyone will figure out how to play nicely with each other, and we'll see a continuation of the interoperable web model we've enjoyed for the past two decades. But I'm betting that things are going to get ugly. We're heading into a war for control of the web. And in the end, it's more than that, it's a war against the web as an interoperable platform. Instead, we're facing the prospect of Facebook as the platform, Apple as the platform, Google as the platform, Amazon as the platform, where big companies slug it out until one is king of the hill.

And it's time for developers to take a stand. If you don't want a repeat of the PC era, place your bets now on open systems. Don't wait till it's too late.

At base, these products represent two polar opposite ends of the spectrum: Apple prefers to hide complexity within the technology whereas the open source approach puts the complexity on the surface of the device in order to expose advanced functionality and greater transparency into how to directly manipulate the device. Put another way, the reason that people would buy the $69 Apple MagicMouse is because they want Apple’s designers to just “figure it out” for them, and provide them with an instantly-usable product. The reason why someone would pay $75 for this mouse is because it strictly keeps all the decision-making about what the mouse does in the hands (pun intended?) of the purchaser.

What I worry about, however, is that pockets of the open source community continue to largely be defined and driven by complexity, exclusivity, technocracy, and machismo. While I do support independence and freedom of choice in technology — and therefore open source — I prefer to do so inclusively, with an understanding that there are many more people who are not yet well served by technology because appropriate technology has not been made more usable for them.

More focus on usability and clarity would be needed with most Open Source projects, but there are already some bright spots. GNOME, for example, has good tradition in simple interfaces. I recently made the jump from Mac OS X to Ubuntu Netbook Remix, and have generally been quite happy with it.

But for Open Source projects the challenge is more difficult than for proprietary products. On the other hand we want to provide a nice, usable experience, but we also want to let our users delve deeper into the functionality, to make changes and maybe even become contributors to the project. This is where new tools like OLPC's View Source button can do wonders if implemented more widely.

Mystery books have been received by many prominent Qaiku members before. They are beautifully handcrafted notebooks personalized for the recipient, often containing hints about Qaiku involvement, like having the inner covers made in printed version of that user's profile background. There is no information about the sender or the reason for making them. All are sent from random Turku post offices.

My copy of the mystery book is a mobile notebook, a bit in Moleskine-like style. The inner covers have a map of Europe from 1810, fitting my interest in history and geography spot-on. The book came with a pen, and had been sent from Turku 10 at 12:02 yesterday.

Several Qaiku members have posted pictures of their books on Flickr with tag "mysteerikirja", and there is a Qaiku channel about it. It remains to be seen whether the books are some viral marketing campaign, or have been made by some individual with Amélie-like tendencies. Anyway, quite a delightful surprise!

Microfeed is a specification and a reference implementation of client-server architecture providing access to various information sources that have a feed-type interface. Examples of those feed sources include micro-blogging services, such as Twitter, Facebook, Jaiku, Qaiku, and Laconi.ca. By utilizing Microfeed architecture, a client application can focus on user interface, while the actual feed fetching is done in the background independently. The communication between a local Microfeed server publishing information about feeds and a client application displaying that information to an user is done with the D-Bus messaging following the publisher-subscriber principle.

Microfeed service already is the power behind Henrik Hedberg's new Mauku microblogging interface for Maemo 5. Here you can see a stream of updates from both Qaiku and Twitter:

If you're implementing a tool that deals with microblogging services, please consider using microfeed for it. Advantages from this include:

• User accounts to various services need to be entered only once and can be reused
• You don't need to concern yourself with the particular features or quirks of a microblogging service API, just use the D-Bus interfaces provided by Microfeed
• Twitter? Qaiku? StatusNet? Facebook? You can let your users choose what services they want to use, without overhead of having to implement the protocols for each of them

More information from http://microfeed.org/

Um, hello? Folks, the whole point of doing open-source code is to let it fork. The idea is to accelerate evolution by encouraging weird mutations. Creating an open source program and hoping it won't fork is like decorating your house with a zillion Christmas lights and a forty-foot inflatable Santa and hoping nobody stops to look at it.

This is an interesting way to look at Open Source. Traditionally freedom to fork has been seen as a safeguard against dead projects or vendors, as a way to hand maintainership over to parties that are still interested.

But what FSJ is talking about is forks being beneficial by themselves. This is the model that Distributed Version Control Systems like git also promote: every developer has their own fork of the software, and merges to "blessed" repositories happen under the watchful eye of a maintainer.

This is quite a different model than the traditional centralized way of working with projects. Merging between forks has its costs, but if we embrace this model we gain lots of new developer flexibility and possible new workflows. DVCSs haven't been with us for a long time yet and so it takes some time for this new distributed way of working to take root.

• Midgard Weekly Summary #75: October 2nd 2009

MWS has been running before, with 66 issues released between 1999 and 2002, and 8 issues in 2007. This time we follow the idea of a Collaborative MWS.

Notices about new published summaries will be sent to the Midgard user mailing list, Qaiku #midgard channel, Twitter @MidgardProject and are available via RSS. Enjoy!

Technorati Tags: midgard

• September 30th - October 1st: OpenMind and MindTrek in Tampere, Finland
  I'll give a talk about the Midgard project and how our company has evolved together with it. In addition Linux-tekijä awards, where I was in the awards committee, will be announced
  
  
• October 9th - 11th: Maemo Summit in Amsterdam, the Netherlands
  While I'm not giving a talk in the event it will be great to meet the rest of the maemo.org team, and the community around the project
  
  
• October 17th: Finhack in Forssa, Finland
  A free-form hacking event for the Finnish free software community. Expect some progress with Midgard's XMPP capabilities
  
  
• October 23rd - 25th: Midgard Gathering Helsinki, Finland
  Midgard users and developers get together to learn about the new capabilities of Midgard 9.09 Mjolnir and discuss the roadmap for the next release
  
  
• October 29th: The Open Web in Ede, the Netherlands
  NLUUGs conference on new web technologies where I'll be speaking both about GeoClue and the Midgard Content Repository
  
  
• November 11th - 13th: Semantic Search workshop in Rome, Italy
  Together with a general meeting of IKS Project participants
  
  
• November 13th - 15th: FSCONS in Gothenburg, Sweden
  The annual Nordic free software gathering where I'll be speaking about Midgard2

Looking forward to all the interesting discussions and ideas that will surely come up from these events. If you will be around in one of those, make sure to look me up and we can chat. The events will also be covered in my Qaiku stream.

"See — the problem with a full scale Content Management System is that it has too many opinions. Those opinions were though of by somebody other than you and the needs of your organization. The more developed a content management system (or any piece of software, really) the more “opinions” it has. And the more “opinions” it has, the more likely one of them is going to really tick you off."

I can relate to this. We work with one system in particular that makes an astonishing array of presumptions about how you’re going to use it, and if you try to step outside those presumptions, demons fly out of the abyss and try to suck your eyeballs out.

This goes back to a previous discussion we had about Content Management as an API. In that post, we had a great experience with hand-rolling a CMS...

The term they are looking for is Content Repository, a service that provides common APIs for content storage, retrieval, signaling and so forth. With Midgard we're following this approach, providing content retrieval and web functionality APIs first, and then building some reusable user interfaces on top of that.

In addition to Midgard some content repositories to look at include Apache CouchDB and Jackrabbit. All of them allow you to stop worrying about storage and retrieval methods and focus on the actual end user functionalities, while keeping the whole system accessible and scriptable for integration purposes.

Technorati Tags: couchdb, jcr, midgard

Midgard2 is a more universal content repository where CMS is just one application:

Please note that more choice in databases and web servers is not the only goodie provided by Midgard2. You also get things like a completely rewritten MVC framework, database views, transactions and native datetime objects. And all of this for multiple programming languages, not just PHP.

Technorati Tags: midgard

Tonight we're taking the ferry to Gdynia, and then the idea is to head south through Poland, Slovakia, Hungary, Serbia and Bulgaria towards Asia Minor and the Caucasus. There are no strict plans or schedules, except that we must be back in Finland by the beginning of September.

You can follow our trip on Qaiku (feed) and Flickr. If you are somewhere on that prospective travel route, feel free to send me an SMS and maybe we can have a beer :-)

Technorati Tags: adventure, motorcycle

Content repositories allow you to separate the actual front-end of your application from background processing tools. More than just their underlying databases, they impose common rules for data access, and keep multiple applications up-to-date on data changes through signaling. Midgard2 provides a flexible content repository that avoids the restrictions of the traditional ORM approach. And not only your PHP web application, but also to possible Python, Objective-C and C# tools you use.

This enables you to split applications into smaller, easily maintainable and scalable pieces that can be run on different systems and platforms as needed. In addition to web, the Midgard2 library can be used for desktop and mobile application development, building software that synchronizes with web services. It is based and engineered fully on the top of the desktop (GNOME) software stack. Being highly modular and having very little dependencies it scales from a note taking application to a full-blown CMS system. Combined with advanced replication capabilities it allows you to synchronize data between offline and online instances of your service.

Unlike shown in the program, the talk will be given by Arttu Manninen this time, as I will be off motorcycling somewhere around Asia Minor. Arttu? Yep, this guy:

Technorati Tags: froscon, midgard, php

Sure, the applications still give you a way to share things and take them out of the storage. You can export a contact out of your address book as a vcard file. But the role of The File here is slowly being reduced to a role of an intermediate storage medium. The business card is temporarily put in the .vcf file before it gets injected into somebody else’s database (another address book?).

As more and more applications operate on databases, the computer is becoming a monolithic black-box that “has things”. How exactly (and where) the data is stored is becoming less clear. The application and the interface becomes united with the user data. It becomes one.

This echos the sentiments of Alex Payne when he warned against what he calls Everything Buckets:

Computers work best with structured data. Everything Buckets discourage the use of structured data by providing a convenient place to commingle “structureless” data like RTF and PDF documents. Rather than forcing the user to figure out the rhyme and reason of their data (for example, by putting receipts in a financial management application and addresses in an address book), Everything Buckets cry: “throw it all in here! Search it! Maybe I’ll corrupt my proprietary database, but maybe I won’t and you’ll have the joy of sifting through a mire of RTF documents. Doesn’t that sound great?”

And yes, I agree that obscure application-specific databases are not really better than obscure proprietary file formats.

This is exactly why I've been talking about content repositories, services like Midgard2 and CouchDb that not only can provide superior content storage and organization, but do it in a way that multiple applications can share. You can easily write your own scripts to perform batch operations on the data, and receive D-Bus notifications when something changes.

And good repositories also provide easy synchronization tools so you can have your data available on all of your computers, and even on the web. If they can also do peer-to-peer sharing, we're close to achieving the fully free cloud.

Technorati Tags: couchdb, gnome, midgard

"Immersing myself in a book or a lengthy article used to be easy," he wrote. "Now my concentration often starts to drift after two or three pages. I get fidgety, lose the thread, begin looking for something else to do. I feel as if I'm always dragging my wayward brain back to the text." For my own part, I now find it challenging to sit still on my sofa through the length of a feature film. The urge to, for example, jump up and check the IMDB filmography of a supporting actor is well-nigh irresistible, and once I'm at the computer, why not check e-mail? Most of the time, I'll wind up pausing the DVD player before the end of the movie and telling myself I'll watch the rest tomorrow.

Exactly the same symptoms I'm having. This is the reason I've written some of my best code while offline at the countryside or on a road trip, and why it was so relaxing to be without a phone for a week recently.

How to solve the issue of constant distractions? Maybe we'll need to be sometimes offline. And even while connected, we need attention profiling and better user interfaces. Something for the developers of the future free desktop to consider.

Confession: I must've switched browser tabs a dozen time while reading the Salon article. Concentration indeed...

Technorati Tags: attention

Early versions of Gnome and KDE were pretty much just clones of the Microsoft Windows UI. They’ve diverged since then, and I’d say Ubuntu’s default Gnome desktop is in most ways better from a design and usability standpoint than Windows Vista. But it’s still fundamentally a clone of Windows — menu bars within the window, minimize/maximize/close buttons at the top right of the window, the ugly single-character underlines in menu and button names. At a glance it looks like Windows with a different theme. The idea being that if you want Windows users to switch to Gnome or KDE, you’ve got to make it feel familiar. But that’s not how you get people to switch to a new product. People won’t switch to something that’s just a little bit better than what they’re used to. People switch when they see something that is way better, holy shit better, wow, this is like ten times better.

So I think Gnome and KDE are stuck with a problem similar to the uncanny valley. By establishing a conceptual framework that mimicks Windows, they can never really be that much different than Windows, and if they’re not that much different, they can never be that much better. If you want to make something a lot better, you’ve got to make something a lot different.

This is a good point to consider as GNOME moves towards 3.0 with the promising GNOME shell. KDE's Plasma has taken some steps also, but really, the examples we should look at here are Moblin 2.0 and the Sugar desktop. Both of them have largely discarded the traditional "Windows model" in favor of more contextual and information-oriented user interfaces.

Technorati Tags: gnome, kde, moblin, sugar

If you want to contribute to a freely available map of the world, download OSM2Go to your tablet and start mapping! My slides are available on SlideShare.

See also my Qaiku notes for SoTM day 1 and SotM day 2. Really amazing to see how far the project has advanced since the 2007 conference. Much of Western countries is already mapped, and many NGOs are working to get the developing world mapped, in many places for the first time ever in digital format.

Technorati Tags: maemo, openstreetmap, sotm09

I gave my Midgard2: Content repository for desktop and the web talk yesterday in GCDS. The slides are available on SlideShare. The main idea was that any application that deals with structured data could benefit from using a content repository like Midgard2 or CouchDB.

So, what is a content repository? It is a service that sits between an application and a data store. It provides several advantages:

• Common rules for data access mean that multiple applications can work with same content without breaking consistency of the data
• Signals about changes let applications know when another application using the repository modifies something, enabling collaborative data management between apps
• Objects instead of SQL mean that developers can deal with data using APIs more compatible with the rest of their desktop programming environment, and without having to fear issues like SQL injection
• Data model is scriptable when you use a content repository, meaning that users can easily write Python or PHP scripts to perform batch operations on their data without having to learn your storage format
• Synchronization and sharing features can be implemented on the content repository level meaning that you gain these features without having to worry about them

Midgard2 is a content repository library that is built on top of glib, libgda and dbus, making it fit the general free desktop infrastructure very well. You can use it in any application that is written in C, Objective-C, Python, PHP, or soon Mono. Learn more from the slides!

Technorati Tags: couchdb, gnome, midgard, mono

I gave a quick talk about adventure motorcycling, speaking about my 2004 trip around the Black Sea, and briefly mentioning the Death Monkey trip on 50cc mopeds from Helsinki to Gibraltar.

You can find pictures from the Caucasus trip on Flickr and my travel log on Routa MC.

Technorati Tags: motorcycle, adventure

Here in Gran Canaria Desktop Summit CouchDb has been somewhat a hot topic, as the Ubuntu project is planning to use it as the content repository for desktop applications.

We had a lunch with Jan Lehnardt today and discussed how to make Midgard2 and CouchDb interoperate better, and as it happens, it is actually very easy: CouchDb has a replication protocol that we can support also in Midgard, making the two repositories able to synchronize content with each other.

There is now a first test implementation of Midgard-to-CouchDb synchronization support, with better Midgard integration and CouchDb-to-Midgard coming soon. Check out the Midgard MVC component on Github. Anyway, already pretty cool!

Setting up replication on CouchDb admin UI:

Midgard record replicated successfully into CouchDb:

I'll talk more about this and repository-oriented application development in my Midgard2: Content repository for desktop and the web talk tomorrow at 16:45. Be there!

Technorati Tags: couchdb, midgard, replication