Social Glass

1994: WebCrawler and Lycos became the first widely adopted search engines.  They flattened the Internet giving all resources an equal chance of being discovered.  Before search engines, people were meant to find content by going to “what’s new” web pages where they’d find hyperlinks to web pages that had been recently added.  From those pages they’d find links to other pages and so on.  The Internet, at least at its conception, was meant to be surfed from site to site, not flattened.  But search engines proved there was a much more efficient, comprehensive way to find information.

And I continue, bewildered, that companies follow the pre-Lycos model when constructing their intranets.  For some reason they always try to consolidate information into knowledge management “silos” like Documentum.  They set policies that change the homepage of their worker’s browsers to be the landing page for the new intranet portal which has “everything an employee will ever need to know” buried deep within.  And then they wonder why nobody uses these new solutions.

It’s because they can’t find anything.

Data will always be federated, scattered across many information systems, just as it is on the Internet.  Instead of investing in building a one size fits all intranet, invest in procuring and deploying an enterprise search engine.  Let it crawl the file share.  Let it crawl Lotus Notes.  Let it crawl the shiny new Documentum application.  Then set policies changing the worker’s homepage to the new enterprise search application if you must.  And make that search page simple.  Nobody understands taxonomies nor will they spend the time to.  Forget faceting.  Follow Google’s example making search dead simple. See what happens then.  A search for “jury duty” might actually produce a result that tells the worker how much compensation he’s entitled to if summoned.

And I’ll bet your workers will be much less pissed off that you changed their browser’s default homepage.

Recent discussions at work have prompted me to re-iterate something very fundamental that often gets overlooked when it comes to Enterprise 2.0. An organization will never adopt a single social productivity tool. Knowledge will ALWAYS be scattered. We’ve come to accept this on the Internet where search engines make information on a myriad sites searchable, but for some reason organizations think they can get everybody to use “wiki X”, and that the search feature in “wiki X” will be good enough.

Stop.

As Dion Hinchliffe says (and as I have written before),

“Discoverability isn’t an after thought , it’s the core”

Organizations need to embrace the fact that their data will be federated. Sure, workers will put their documents in “wiki X”, but they’ll also put them on the file share, in content management systems, and on email servers. Data that cannot be found is useless. Enterprise search will unlock data and increase the propensity for information (and the knowledge workers who create it) to be discovered. Discoverability leads to recognition, and recognition leads to increased participation. Enterprise 2.0 must be approached holistically.

Clearspace doesn’t do this. Thoughtfarmer doesn’t do this. Mindtouch doesn’t do this. There is no “Enterprise 2.0 in a box” solution. Period.

I came across a post from Dion Hinchcliffe recently that talked about ideas for SOA architects in 2007 (ok, so it’s a year old). Dion talks about how Web 2.0 philosophies can be leveraged to enhance the effectiveness of SOA. The recommendation that stood out to me was “Deeply Embracing URI Addressability”, where Dion argues:

The hyperlink is the fundamental unit of thought on the Web and it should be in your service designs and (hopefully granular) schemas as well. Giving each discrete piece of information, every service, and all content a globally addressable URI instantly gives a service, and the data it carries back and forth across its interface, access to countless new consumption and reuse scenarios.

To illustrate how unique addressability might work, consider an organization that sells insurance to customers. Each customer is given an account number. The organization has an SOA strategy and decides to make use of REST (Representational State Transfer) to expose its information assets, including “customer” in an interoperable manner.

REST “strictly refers to a collection of network architecture principles that outline how resources are defined and addressed” (Wikipedia). It also makes use of the HTTP verbs POST, GET, PUT and DELETE (similar to database CRUD operations) to define what action to take on a given resource.

The insurance company develops a REST interface that sits on top of its CRM system. Customers become uniquely addressable by account number, as follows:

http://insuranceco.corp.com/customer/9886574

where “9886574″ represents the account number for a particular customer. A “GET” request (similar to READ on a database) to this URI might return the customer profile and claim history.

This is what Dion means when he says information should be uniquely addressable. Each customer has his own URI, which means behind the firewall other applications can reference a customer using this address. From here the possibilities are endless, where the intranet starts behaving like the internet. Information assets begin having inbound links, and this “democracy” so famously publicized by Google’s PageRank algorithm (where an inbound link counts as a vote) starts to apply not just to web pages, but to entities abstracted by services in an SOA (like “customer”). In this way information can be:

• Bookmarked in a social bookmarking system. Users include comments, like “Harry Young [customer 9886574] calls in every week and I’m tired of dealing him”. Users also tag these resources in the social bookmarking system, i.e. “disgruntled”, “customer”.
• Searched through an enterprise search engine. Adding “http://insuranceco.corp.com/customer/” to the seed of our enterprise search engine causes all customer information to be natively crawled and indexed.
• Referenced from wiki and blog posts.

As a blanket statement I’d say that any SOA architect should strongly consider exposing resources in this manner so that he can leverage the network effects and unintended uses that are achievable through unique addressability.  Good stuff Dion.

No, this is not another post about combining social bookmarking and enterprise search. Instead I wanted to highlight how the social dimension of enterprise search – the search terms people use and the links they click as a result – can add to the effectiveness of an enterprise search solution.

The idea is simple:

1. Capture “fruitful” search queries – queries that cause a user to actually click a search result.
2. Capture the link, title, query and timestamp for each search result a user clicks. This can be done by re-writing the URLs to post to the search application first before redirecting to the original link.
3. Display the number of times a document has been viewed (“views:5″, as is the case with the first document in the screenshot). Documents that are viewed more than others tend to be more helpful, and this information can be useful when determining what to look at.
4. Massage the search statistics so that popular search terms are displayed in a “Search tag cloud”. This lets the user knows what the enterprise is looking for and helps create stickiness.

“Rabble” is a simple proof of concept that we developed in Ruby on Rails, and we could probably do a lot more with it. It integrates to a Google Search Appliance on the backend, and I’ve discussed how to do this integration before.

Interestingly, much of the information captured by Rabble exists inside the search appliance, and yet there’s no API to access it. Hopefully such an API is on Google’s roadmap, as information about user behavior can go a long way toward adding a valuable social dimension to enterprise search.

Over at the FASTForwardBlog, Bill Ives highlights the value of combining enterprise search with social bookmarking.  Tom Mandel recently commented on my blog about this as well, and I’ve written about this topic a few times before.

I’ve recently posted a position paper about the business value of fusing search and social bookmarking, borrowing ideas from Andrew McAfee’s post on weak ties.  You can read it over at mike2.openmethodology.org by clicking here.