Monday, August 11, 2008

Assertions, sentences, and ratings

Ken mentioned that we're in the middle of reorganizing the database. I'll fill in some more details about what we're doing.

Currently, our users give their ratings to assertions, the things that make up the links of ConceptNet. Many sentences can yield the same assertion: for example, "dogs are mammals" and "a dog is a kind of mammal" both turn into an assertion that can be expressed as IsA(dog, mammal). The ratings on these assertions are useful to representations such as AnalogySpace.

The problem is that the OMCS web site doesn't want to show you IsA(dog, mammal), it wants to show you something in natural language. And some of the natural language we've collected is of

What should matter to OMCS isn't just how good the abstracted assertions are, it's how good the sentences are.

So we're reorganizing the database. After this, your ratings will apply to the sentences you see, and the scores on assertions will come from aggregating those ratings. We'll display each assertion using its highest-rated sentence. This puts our users in charge of which sentences show up, instead of arbitrary decisions by the computer.

The hard part of the reorganization is that we have to take all of the existing ratings and find out where they came from. If they came from a user on the new site, for example, we need to know what sentence they were looking at when they gave the rating. The database generally has this information, but not necessarily recorded in a smart way.

So what we've really been doing is cleaning up messes in the database while we track down where the ratings should go. And most of which were created by me a couple of days before giving a demo. Sorry about that.

3 comments:

AKSHAY BHAT said...

open mind commons seems to be down.
as well as csc.media,mit.edu .

Ken Arnold said...

Thanks for the update. The whole Media Lab infrastructure is having some problems that are affecting our sites. Hopefully it'll get resolved within a day or two.

Anonymous said...

from http://www.media.mit.edu/research/23

Collecting Common Sense Henry Lieberman, Marvin Minsky, Jason Alonso, Kenneth C. Arnold, Catherine Havasi, Jayant Krishnamurthy, Dustin A. Smith, Robert H. Speer and Luis von Ahn

Wow when did Luis Von Ahn joined your team?

Can we then hope to see the verbosity data soon?