Here in Gran Canaria Desktop Summit CouchDb has been somewhat a hot topic, as the Ubuntu project is planning to use it as the content repository for desktop applications.

We had a lunch with Jan Lehnardt today and discussed how to make Midgard2 and CouchDb interoperate better, and as it happens, it is actually very easy: CouchDb has a replication protocol that we can support also in Midgard, making the two repositories able to synchronize content with each other.

There is now a first test implementation of Midgard-to-CouchDb synchronization support, with better Midgard integration and CouchDb-to-Midgard coming soon. Check out the Midgard MVC component on Github. Anyway, already pretty cool!

Setting up replication on CouchDb admin UI:

Midgard record replicated successfully into CouchDb:

I'll talk more about this and repository-oriented application development in my Midgard2: Content repository for desktop and the web talk tomorrow at 16:45. Be there!

Technorati Tags: couchdb, midgard, replication

The Midgard Project has finally released the first public version of Midgard Application Server Suite. The new release contains Midgard core libraries, a PHP3-based web application server for the Apache platform and the needed web-based administration tools.

May 8th 2009:

It's time for celebration - Midgard CMS turns ten in May! The decade of Midgard will be celebrated with a guest gala that is to be held at restaurant Ostrobotnia in Helsinki, Finland.

Looking forward to seeing as many of you in the party as possible! Please register ASAP as we need to tell the restaurant how many tables are needed.

Last few weeks have been a great example of this. Nemein is the company which originally built the OpenPsa2 suite of project management and CRM tools for consulting companies. In last two years there have been some shifts in strategy, and so we ceased working on the product.

Luckily for our users, OpenPsa has been out under the GNU General Public License since 2004. This meant that while commercial development had stopped, they were able to step in and continue where we left off. It took a while, but during this fall Andreas Flack from Content Control started maintaining the software.

Now the effort is starting to bear fruit. Midgard world has recently settled into a new synchronized release model, and OpenPsa is swiftly catching up with the rest of the architecture. We expect that it may be released with the Ragnaroek 8.09.2 revision.

Good work, Andreas!

The recent direction of business applications has been centralization to web-based systems, easing deployment, upgrades and management of application security. However, at same time centralization provides new risks like the introduction of a single point of failure for application usage and in most cases inability to work offline.

At the same time, working culture has started so shift more towards networked individuals, or "web workers" forming ad-hoc coalitions to work on various projects. For them, centralized applications might not be desirable due to data ownership and infrastructure requirements.

Moving the applications from centralized web servers to a peer-to-peer network allows web workers to be in control of their own data, stay productive even in unstable connectivity situations, and collaborate easily with their colleagues in an ad-hoc way.

This talk outlines some ways to move forward in building peer-to-peer networked business applications. There are many open source frameworks targeting the problem, including replicated databases like CouchDb and Prophet, and application-oriented P2P networks like Swallow/DBE. These will be discussed together with some real-world examples of business applications built with them:

Ajatus - a "Personal CRM" built on top of CouchDb, a RESTful replicated object database
OpenPsa - project management system doing resourcing over the DBE P2P network
Simple Defects - a P2P bug tracking system built on top of the Prophet replicated database

The first phase of P2P business applications will likely be services operating in closed networks of users' social networking or instant messaging contacts. But the P2P model can also provide opportunities for wider networking, making it possible to find new project partners or collaborators anywhere in the world. This wider-ranging business network will introduce new challenges like security and reputation management. Some ideas related to this will also be discussed.

Other interesting projects in this sphere I did not mention are Telepathy Tubes and F2F. We have also had some ideas for how Midgard could do this...

I had some goals:

• Keeping status of different projects up-to-date with more accuracy
• Ensuring our sales and project management knows if some project is being blocked by missing materials
• Enabling a more distributed, web working culture

We're a small company of less than 10 people, and as such most project management methods have not been very successful for us. In general they have been made for situations where same person or team keeps on working on a project for several months, whereas in our situation a person typically works on several projects every day.

I discussed this over some beers with Tero Heikkinen from Rohea, and he told me how they were implementing Scrum in their small company. While their number of different projects running at the same time is a bit smaller, their situation otherwise is quite similar: same technologies used, Ajatus for work tracking, etc.

We had a company sauna evening and I presented Tero's ideas there: we would partially implement the Scrum model, and keep tuning it to our needs. At the first phase this means:

• Every morning we have an all-hands 15-20min meeting ("the daily Scrum") where everybody goes over what they have been doing the previous day, and what they were planning to do today. If they are being blocked by something missing: a software bug, missing information or other materials, this is also brought forward
• Every project has a file in Google Docs where we keep the project status and task list (Backlog). This task list is updated based on what comes up in the morning meeting
• Work hours are reported with Ajatus. Rohea also uses it for project burn-down charts, and once their add-on for that is finished we may do the same
• Emilia, the project manager (or Scrum Master) is responsible for resolving possible impediments and maintaining the per-project status files

The approach we have taken should be quite pragmatic and low-tech. Instead of fancy project management software we use simple word processing for status data. And thanks to Google Docs the documents produced are accessible and editable from anywhere.

Similarly the actual meetings are quite easy to manage. The people who are at the office attend there, and others attend either via a Skype or mobile phone conference call, depending on network availability. We decided to have them at 10am so that everybody will be able to participate. Even if there is a Sprint or meeting scheduled for the same time, the short time needed for our all-hands meeting means it can be held over a "cigarette break".

Ajatus is the only more experimental piece of software in our puzzle. In our company, we use it for hour reporting, expense tracking and keeping meeting minutes. For these it works quite well, although more reporting tools are definitely needed. The alpha status of CouchDb, the database software powering Ajatus has bit us a few times by database corruption (caused by OSX-specific erlang bug) or simply difficult installation procedure, but these problems will hopefully improve over time.

We're now in the second week of this model, and at least the gut feel is that this has improved coordination inside the company. The next challenge then is to let the customer get involved in the process. This can mean just sharing the project status files, or even giving them access to actual meetings or the Ajatus data.

This post contains very early ideas, but we would be happy to get some feedback on them.

Basic idea

Each Midgard or Ajatus server runs a "synchronization daemon" which is connected to a XMPP server using some JID identity.

The sync daemon registers a set of pub/sub nodes corresponding to the content structure on the Midgard or Ajatus instance:

• in Midgard: Sitegroups, MgdSchema types, paths (possibly a regexp), approval state
  • /midgard_article/all, /midgard_article/approved
• in Ajatus: tags

The pub/sub nodes can be set up with some access control rules. For example, Ajatus tags would by default require "whitelist" authorization to subscribe.

When content is changed, the sync daemon gets notified about it (via D-Bus signals in Midgard, and via CouchDb external indexer API in Ajatus). The content object (and immediate children like parameters and file attachments) is serialized into the syndication format and sent onward as a XMPP "pub" leaf. If the object appears on multiple nodes (multiple Ajatus tags for instance), the "pub" leaf is sent to all of them.

XMPP server and the federated network will then handle notifying the subscribers of the nodes about the new leaf.

The subscribers will receive the leaf, and unserialize it to the Ajatus or Midgard database.

Communication between sync daemon and application

The sync daemon should exist as an entity separate from the actual user application. Communication between the sync daemon and Midgard or Ajatus should happen via the application database.

This means that XMPP pub/sub whitelists for Ajatus tags would be maintained in the ajatus_db (non-replicated) database, and the sync daemon would read them from there. The Jabber server credentials would also be stored in the same database.

Information about resources from jabber buddies

User's replication partners are stored as local contacts in the application database. In Ajatus this means "Contacts" and in Midgard "midgard_person" objects. Both storage models have optional JID field.

If JID is marked for a contact, the sync daemon should at startup (or at an interval) try to discover if the contact has pub/sub resources available.

Content transformation in replication stage

While mainly intended for Midgard-to-Midgard and Ajatus-to-Ajatus replication, the same mechanisms could work across different systems.

For this, the sync daemon on the subscribing end should support XSLT transformations before content is unserialized into the system. The XSLT transformation templates should be configurable per subscription.

Hermod / Hermóðr (Midgard replication)

• "in Norse mythology, messenger of the gods. He was a son of the principal god, Odin, and his wife, Frigg. Known as Hermod the Swift, he was called upon by the other gods when they had a task requiring speed and urgency." - Encyclopedia Britannica
• Written in Python
  • Using twisted framework or x60br XMPP library
  • using standard Midgard MgdSchema object serialize/unserialize methods
• How to proof-of-concept before D-Bus? Some watcher in MidCOM? (watcher touches spool file)

PillowTalk (Ajatus and CouchDb replication)

• Written on Erlang
• Extendable through plugins
  • Content parsers (Midgard2Ajatus, Ajatus2Midgard)
  • Security
  • CDATA JSON block or convert to XML? (Decision of the content parser?)

Technorati Tags: ajatus, midgard, jabber, replication, synchronization, xmpp

Ideas may come to you when you're sitting in a bus, boarding an airplane or visiting a hospital. For a CRM to work the data must be available and editable in any situation.

Having now dogfooded Ajatus for almost two months, I have to say this has been an important aspect. It is powerful to have all your customer and project data with you at all times, and still be able to use it via the familiar web UI.

Offline is useful. You can write your notes in every meeting, update them in train, report expenses as they incur. And most importantly, as long as your computer is running, the application is never down. And still, though replication your data will be safe with your peers or the corporate server.

This is aspect even more important when you start running the app on ultra-portable devices, or go outside the industrialized world.

Technorati Tags: ajatus, offline, web, replication

• Feb 15th - 17th: Midgard developer meeting in Linköping, Sweden. Lots of hacking and talking about Midgard 2 and MidCOM 3
• Feb 22nd - 24th: FOSDEM in Brussels, Belgium. I'm going for Open Business Organisations of Europe (OBOOE) meeting on behalf of COSS and Jerry will give a lighting talk on Ajatus
• Apr 11th - 12th: Helsinki Media Conference in Helsinki, Finland. I will be talking about attention profiling.
• Apr 17th - 19th: FISL 9.0 in Porto Alegre, Brazil. I will be talking about GeoClue
• Jun 3rd: ServOSS seminar in Helsinki, Finland. I will be talking about interaction between a company and a free software community

Looking forward to meeting many of you in these events! The July conferences of GUADEC in Istanbul and State of the Map in Limerick sound interesting as well.

It is possible to follow my movements through Dopplr:

Technorati Tags: ajatus, geoclue, obooe

The means to that is CouchDb's integrated replication feature. With it you can easily share your Ajatus data with anybody in the network, or just keep your laptop and desktop system in sync.

At the moment we only support full replication over HTTP, but the plan is to support replicating only data with specific tags (for example, only data related to a specific project), and to do it over XMPP.

As things are, here is how you set up Ajatus replication on a Mac OS X system:

First allow CouchDb to talk to the outside world (note: it is a good idea to keep the firewall up when not replicating!):

Then access the CouchDb management console in http://localhost:5984/_utils/ and go to "Replication". Set up the replication paths there. The database you're interested in is "ajatus_db_content" (by default, see advanced setup):

Once you have entered the paths just click "Replicate" and wait a sec. Replication is one way, so remember to also replicate back from the other server!

When replication is done your Ajatus system should have data entered on multiple systems:

My company is now dogfooding Ajatus. At the moment we use it in full peer-to-peer mode, but there are some plans to install a central server for security and backup purposes. I will post some notes on our experiences later this month.

In related news: CouchDb lead developer Damien Katz has been hired by IBM to work full-time on the database. This is great news for both CouchDb and Ajatus. Congratulations!

Technorati Tags: ajatus, couchdb, replication

Read the installation instructions to get started. Once CouchDb 0.7+ has proper installers available things will be much easier.

Now for champagne... Happy new year from the Nemein team!

Technorati Tags: ajatus, couchdb

As we're nearing first release, the plan is to start dogfooding Ajatus quite soon. For me personally this has been cause of some concern, as Ajatus is quite browser-intensive and my Firefox 3 beta has proven to be crashy. So, a Web Worker Daily story on site-specific browsers came at a very good time: in a couple of minutes I had Fluid installed and was having a separate Ajatus window launchable straight from the OS X Dock.

Fast, nice, and separate from whatever I may be doing in my actual browsing sessions. I like it.

Setting up Ajatus in Fluid

Ajatus as a Mac desktop app!

Note: Mozilla Lab's Prism does the same with a more cross-platform, but I like Fluid for the WebKit speed and simplicity.

Technorati Tags: ajatus, crm, fluid, osx, prism

Main changes in 0.7.0 were:

• JSON instead of XML
• Javascript view engine
• Easier build system
• ...and a nice administrative UI

With the new release Ajatus is a lot easier to set up as CouchDb can be installed straight from release packages.

Technorati Tags: couchdb

What makes Ajatus so special is the approach we're taking with it. Having with OpenPsa found the traditional, hierarchical CRM approach unworkable we wanted to solve the problem in a different way:

• Local, rich AJAX client everybody can run on their laptop or internet tablet
• Replication to allow sharing data with partners, customers and the employer
• Simple base data types (note, event, contact, ...) that users can customize and extend
• Possibility to build integration tools and plug-ins in almost any language (with CouchDb's restful JSON interface)
• Speed

To help us stay on the right path we even wrote an Ajatus Manifesto to guide ourselves.

Currently the software already runs and does pretty much all the basic things needed. Once we get it into state where we can dogfood it (in interoperation with the company OpenPsa) we will make the first release. Until then, stay tuned, check the Git repository and join the talk!

Technorati Tags: ajatus, couchdb, openpsa

The database is available for various different platforms, and provides a RESTful JSON API, which makes it very easy to access basically from any programming languages. To make things even simpler, access libraries have emerged for various languages like PHP and Ruby.

This weekend Jerry Jalava released a jQuery CouchDB access library. This is an important step for us as Midgard is already making the jQuery migration, and we are considering to use CouchDB for a project.

So, why CouchDb? First of all, it is something new and interesting. And it is multiplatform and replicated, meaning that we should be able to get our data everywhere, from a web server to Mac desktop to an Internet Tablet. Besides regular database usage, some of the PHP code deployment ideas are also promising.

Technorati Tags: couchdb, jquery

The idea with the WiFi bunny is to connect it as an ambient information display for our OpenPsa management suite. Some use cases we have so far envisioned include:

• Displaying status of the support ticket queue via the belly lights
• Alerting us of new tickets
• Reporting the number of invoiceable hours reported at the end of the day
• Alerting of calendar events happening at the office
• Generally handling status messages based on our Plazes locations

Technologically this should be quite simple using OpenPsa's integrated notification framework and the PEAR Services_Nabaztag library.

I will report when we have some concrete results with this.

Technorati Tags: nabaztag, erp, openpsa

Today I added support for populating Microformatted contact information pulled using the nice hKit library. To use this feature simply create or edit a company and add their website URL:

OpenPsa will check on the background whether the site contains hCards, and populate any information gained to the contact entry.

For example, Bang & Bonsomer provides their address as a hCard in their site footer:

OpenPsa finds this information and adds it to the entry:

Couldn't be much easier. Now if only more sites provided hCards...

Technorati Tags: hcard, microformats

Trip to LatinoWare and OpenBeach 2005, and the Nivea Sun sailing race in Brazil

My slides are now available in PDF format (6MB). I also have most of the talk in cell phone video format that I will try to process and YouTube at some point.

After the talk I've been mostly hanging out in the conference cafeteria, discussing free software with other speakers and participants.

The conference is held in the technology center of Itaipu, world's largest hydroelectric plant. On thursday evening we went to see the light show on the massive dam wall, and after that to a Open Bar student party.

Friday morning, slightly hung over, we crossed the Friendship Bridge to Ciudad del Este, Paraguay. The city had the strongly commercialized feeling of a border town, closely resembling Vyborg for example. But that was quickly forgotten by getting to try a motorcycle taxi on the way back.

Updated 2006-11-20: Dave Neary has a good write-up on the conference.

As I've already given two earlier talks on DBE in Brazil, I will this time skip most of the politics and focus on how DBE can make everybody in the business of free software happier.

In the other news, here is the first look at the new version of the Maemo application catalog:

Notable changes include switching to a more easily browse-able format from the old giant wiki page, using the new single-click install format and automatically linking all wiki pages tagged with application's name to the application page.

Updated 2006-11-15: My talk has been rescheduled to 3pm.

The idea of the Project Broker is to be a DBE service that automatically looks for potential subcontractors in the Fada P2P network. Matching criteria include competence tags, schedule and endorsements given by other members of the network. In addition to DBE, the same searches also work inside a single OpenPsa installation enabling automatic scheduling and resourcing of tasks:

Another change we rolled in was a reworked hour reporting system to the OpenPsa front page. There are now two ways to report hours: by manually entering them using a composite widget, or by letting OpenPsa keep the time for you selecting what you are working on: