It has been eight years since Tim Berners-Lee threw up his hands and said "it's all crap, lets do it over" and set off to create the semantic web. We've got very little to show for it so far. I firmly believe the work semantic web technologists are pursuing is important and the concepts will inevitably be realized and I very much want to see this research become viable. But things are not moving fast enough and the tack semantic researchers are taking simply isn't working.
Semantic web technology is marred in a chicken/egg paradox. The technologies are generally not useful unless they are adopted and implemented on a large scale and people are not willing to invest in implementing them unless they are useful. This is exacerbated by the fact that there are very high technology, business, and social barriers to implementing the semantic web.
- Technology Barriers: Even today, implementing RDF parsers is complex and difficult and the best tools are hopelessly slow. These are the most basic and fundamental tools the semantic web needs to operate and we still can't get them to work.
- Business Barriers: If the semantic web is implemented the current web industry will be intensely disrupted. Ebay, Google, Amazon - virtually all mainstays of web-business will have to significantly adjust their business and technology models. Because of this web-businesses are trepidacious when it comes to investing, adopting, and promoting the semantic web.
- Social Barriers: The way in which we use the web will be greatly changed when the semantic web is implemented. Just look at the current state of usability in feed aggregation for a hint of what will be required for users to adopt the newly realized functionality.
These barriers are far from insurmountable, but the tack the current researchers are taking simply won't cut it.
- Researchers are not finding adequate use-cases for implementing compelling functionality, instead they are creating widgets. There are a great many of organizations out there with real-world needs that would be greatly served by implemented semantic web-technology but researchers are for the most part turning a blind eye and working in a vacuum.
- Researchers are not picking their battles. Instead they are building generic tools with little real world applicability.
- Researchers are not keeping up with the web and web-publishing software. It seems that in an effort to remain neutral towards the current web-publishing industry semantic web researches choose to build their own tools in isolation. This means that anyone wanting to reuse these tools in a real world application has to re-implement them within their own web-publishing environment which due to the high technology barriers simply isn't happening. This is a shame because it would actually save the researchers time, effort, and money if they simply implemented their tools within web-publishing environments such as Drupal and it would allow adopters to implement the tools at zero cost.
- Researchers are not moving at the pace the web is currently developing, instead they are attempting to leap-frog it. A good example of this is the Structured Blogging and Microformats initiatives. Why are semantic web researchers not collaborating with the teams pursuing these projects?
So what can we do about it?
- Researchers need to stop thinking of themselves as researchers and start thinking of themselves as implementors.
- Research institutes need to join forces with emerging businesses looking to adopt semantic technology. This breaks the current model of business / research institute collaboration since startups do not have money to contribute to fund research, but tough noogies.
- Researchers need to build their tools in real-world development environments, i.e. as modules for LAMP web-publishing tools such as Drupal and Wordpress. They need to find more organizational partners to deploy their solutions. They need to do something other than build widgets.
Comments
Re: RDF Semantic web research isn't working
If you are willing to look around, there are already a good number of applications that are operationally deployed. I have captured just a few of these here:
http://lsdis.cs.uga.edu/projects/semdis/2006-03-23__Sheth-Apps-2006.ppt
http://lsdis.cs.uga.edu/~amit/blog/
Re: RDF Semantic web research isn't working
Amit, I skimmed through your powerpoint. It exampled many interesting RDF applications, but they all seem to be projects sponsored in an academic setting, not 'real world' applications in the wild. Am I wrong?
Re: RDF Semantic web research isn't working
In the Technology Barrier section, you mentioned that the RDF parsers are hard to write and are slow. Why do you want to write an RDF parser when there are bunch of them already - e.g. RIO and ARP. Also, what do you mean when you say the parsers are slow. Do you have some numbers on that?
Re: RDF Semantic web research isn't working
Zack.
I tend to agree and have more here
Re: RDF Semantic web research isn't working
Hopefully I'll have time to respond at length later, but just re. microformats and Structured Blogging: there is work under way in the GRDDL Working Group connecting microformat data to the Semantic Web. Structured Blogging is also covered because they support microformats (and there is conversion available between their XML format and RDF/XML).
SW in web-publishing environments
... if they simply implemented their tools within web-publishing environments such as Drupal
Foregoing the grand discussion, I was wondering if you looked at this module for Drupal. Likewise there is an proposal to implement rdf in Plone -- something which would make me very happy.
Re: Semantic web research isn't working
I've been reading your posts off of SIMILE which I also subscribe to. I think you need to remember that researchers and real-world implementors are two different people. The goal of research is to advance the state of the art, to solve problems. If researchers want to then implement their technologies or solutions, that's when they start knocking on industry or even start their own company. The ideal world would be that researchers work together with implementors in industry, but it's very difficult to do so, since each have their own agenda. A corporate research lab would be the right place where employees here (like IBM Research for example) do research but it's for the benefit of the company. I think this is where you'll see the results that you're looking for.
This model is failing the semantic web
The 'industry' is putting up some money to fiddle around with semantic web technology but won't pull the trigger on implementing it until it is proven. This is for good reason, you bear a huge burden by being the earliest adopter of newly created standards. So the burden to prove the technology remains upon the University. The problem is the Universities are unable yet to descend on their own to real-world problems, they need partners cutting big checks to fund researchers. But the only adequate partners (adopters) of semantic web technology are the little guys, open-source tinkerers and entrepreneurs. So far the Universities continue on ignoring them.
Re: This model is failing the semantic web
It sounds like you're really passionate on getting this to work and I don't blame you, because I used to work in industry and I got frustrated about how research to implementation takes such a long time. But instead of ranting about it in discussion, try to determine what it is that you could do to improve on this. Maybe volunteer to join the semantic web working group at W3C and make your voice known. The guys at SIMILE and in other research groups are doing their best, but they don't know everything, they're focussed on their immediate goals. Instead of criticizing, invoke some action on your part, and take the initiative would be my suggestion.
Well..
I agree, SIMILE is probably not the appropriate venue to be having this debate - but then again what is? I don't think it is the W3C, their job is to just create the standards.
Really it is TimBL and the Universities job to make these decisions. At some point it may make sense to appeal directly to them.