Reflecting on the Forward25 Data Mashup in Exeter, April 1st – 4th 2011

From the 1st to the 4th of April 2011, “Forward 25” hosted a Data Mashup event to look into new and innovative ways to visualise data from the UNFPA’s Upcoming State of the World Midwifery Report.  “Forward 25” Careers is a social enterprise promoting creative action on future work and careers; focussing on young persons aged 9 to 25.

I had never attended a mashup event before so this seemed like a really good opportunity, right on my doorstep!

Experiencing the event…

The Data Mashup was a great success overall, but I cannot take much credit for that.  Other commitments meant that I could only assist the team for the Sunday.  Despite this, I found the event exhilarating, and a real insight into both the local talent and enthusiasm that exists for events such as these.

The event was very well organised by the team at Forward 25, and likely to be one of many.  Before joining the team, my expectations were that I would be the odd one out and others would have had been there for the duration.  This however wasn’t the case at all.  There was a core team who had dedicated their time to the event, but many others had come for only a day or two.

I wasn’t expecting to get involved in the development so late in the event.  The more dedicated developers at the event would already be battling against the clock to deliver functional apps for the following day – and likely would not appreciate needing to take time out to bring me up to speed.  I would have preferred to add value by to the event as a tester benefitting from the fact that I had no prior knowledge of the developments.  Unfortunately the development had not advanced to this stage yet.  Completing an end to end app in just over 3 days was always going to be challenging.

In the end I thought I would attempt to setup a JSR-286 portlet development environment to offer an additional delivery approach for the midwifery data.  Unfortunately this was a futile attempt because installing Liferay (preferred Portal server), Eclipse IDE, Liferay IDE plugin for Eclipse, and carry out the actual portlet development all in one afternoon was simply not possible.  The attempt did however attract some interest from the fellow Java developers at the event who were new to portlet development; so I feel the effort was not wasted.

Can the event be improved?

As mentioned earlier, the event was organised very well by Forward 25, but as a first of its kind in Exeter, it would seem sensible to reflect on the experience with a view to hopefully making the next mashup event will be even better!

So here are some of my observations / suggestions:

  • It would be useful if some preliminary information about the dataset feeding into the mashup was made available before the event.  Maybe as a separate presentation & registration event.  This would also provide an opportunity to identify which technical skills can be called upon before the actual event starts.
  • There should be an emphasis on documenting the high level design of the app(s) at the end of the design phase.  This document could be considered an induction document for developers who arrive later in the event.
  • As this is a data mashup, it would be useful if the raw data is readily available via APIs or easily accessible databases; rather than the identification & setup of suitable services taking part during the event and deducting from the time available for the development of the apps.

Reference

If you’re interested in reading more about this mashup event, check out the following links:

Advertisements

4 thoughts on “Reflecting on the Forward25 Data Mashup in Exeter, April 1st – 4th 2011

  1. I would imagine it would be impossible to build anything substantial in 3 days using the Java platform even if you used multiple Java frameworks like Hibernate+Spring+PrimeFaces. Given the time constraints using enterprise scale tech like JSR-286 portlets would seem even less suitable to me.
    I think in such a rapid prototyping sprint, you have to use the more Agile and dynamic languages like Ruby, LiftWeb, XQuery/XRX or dare I say it… PHP 😉

  2. That’s a fair comment. Most of the development that took place was PHP based.
    However, I wouldn’t rule Java out for events such as these because as you mentioned, there are many frameworks available which aim to both speed up development and make it more reliable.
    JSR-286 portlets are merely a lightweight wrapper (though many will claim the opposite!) for other frameworks accessed via bridges. I did have a deployable JSR-286 portlet after just the 5 hours or so I spent there, and most of the time was actually spent getting access to and understanding the dataset, and identifying innovative ways to visualise it. We were intending to use Raphael (raphaeljs.com) for the visualisation.

    My thoughts were that midwifery data visualisation would benefit from integration with other data visualisation. JSR-286 seems an ideal technology for achieving this as it allows the portlets to be loosely coupled, yet cooperative. The midwifery dataset was centred around the countries that the data had been collected for, and hence provided a natural key (the name of the country) to facilitate loosely coupled JSR-286 IPC (Inter-portlet communication) with any other portlets developed in isolation from this mashup event that has an interest in collecting/visualising data at a country level. With all the open data movements that are happening at present, this doesn’t seem far fetched?

    I believe all the code written at the mashup event was intended to form the basis for an open source project.

  3. My point is that even with huge amounts of frameworks Java is still too big and cumbersome for rapid prototype development. Not to mention the fact that it will take you an hour or so to build the scaffolding that you need to glue these Java frameworks together and support any sort of cohesive simple web app views by convention methodology.
    Conversely Ruby lets you build a prototype that will do CRUD using full MVC conventions out of the box with a few minutes… imagine what you could do in 5 hours!

    I think that Inter-Portlet Communication is an interesting topic, so I am not disputing that, but I am disputing its worth and weight in a project of such a short timeframe.

    IMHO if you want the most use out of your data in terms of 3rd parties producing visualisations and apps etc, then you should just provide RDF (if its linked data you want), XML and JSON representations.
    If you have a good base dataset then this can be done almost transparently within an hour or so. People who want to do fast reuse, like simple structured data, or really simple protocols like unAPI. Im not sure that IPC is a simple or reusable protocol for anything other than IPC.

  4. Spending an hour building the scaffolding of a Java deployment seems like no time at all to me. I would suggest that determining/understanding the user requirements is where the majority of time is spent. Though I can appreciate the benefits of frameworks such as Ruby for RAD.

    You raised a really interesting point about RDF. I didn’t think at the time to suggest it for the data mashup just gone but I think you’re right. One of the main reasons why I pursued developing a portlet is to enable my deliverable to form a component part of larger web apps developed in isolation of the data mashup event. i.e. steer away from developing a rigid web app which displays information in a predetermined way; Needing a developer to contribute directly to the code base of the web app in order to enhance/manipulate the information further.

    RDF arguably takes this to the next level and opens it up to a wider audience of developers than those experienced in Java dev or a technology with an implementation of WSRP 2.0.

    Leaving the discussion about mashups behind for a minute –

    Some time ago I was thinking about if RDF could be utilised for portlet IPC to take loose coupling of portlets to the next level. My experience with RDF isn’t vast (something I will be addressing when time allows!), I think I understand the concepts well enough to derive use cases and also potential issues. Please correct me if I’m wrong, but one topic which doesn’t seem to have been addressed yet with RDF is establishing a trust framework for assertions?

    Assuming this concern is justified, its use within the enterprise could be compromised. Though my research into portal technology could offer a solution. I was thinking that if RDF was used to describe portlet IPC events information then one would implicitly gain trust in RDF assertions. This is because only trusted portlets would be deployed/consumed into the enterprise’s portal in the first place.

    Not sure what your thoughts are on this?

    I guess one could question the real usefulness/practicality/ROI of RDF in the context of enterprise scale (i.e. relatively small in comparison to the www) systems generally.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s