Tuesday, December 27, 2005

Finding patterns...

So now that Rick has put up the FoxBlogs tag cloud, some may be asking "what's a tag cloud?"

Basically it's running the content analysis service from Yahoo (who knew they had one? Everytime Google talks about something, everyone's on it but Yahoo? Needs to do better job of marketing to developers- or maybe I should just listen better )

What does that do? Wow - no wonder no one likes to listen to developers ramble on...they talk and talk and talk but don't really say what it does... (this is because I just went through 5 minutes of links without a good "stand on its own" description.)

Essentially, it attempts to put things into context automatically. For example, when Ken Levy talkes about Headphones, , the entire set of threads are tagged with that term. Now how does it recognize that term? That's the trick.

From Yahoo's own Y!Q (context query), "Y!Q analyzes the context you provide and determines automagically the most important keywords to use." - so you can provide the context but it determines the keywords.

Uh - "automagically?"

In short, it must look for patterns. Patterns are really quite interesting. I'm just starting to read as opposed to listen, to Freakonomics by Steven Levitt and Stephen Drubner.

(it was on my Audible wish list but I ended up getting the hard copy version)

In one chapter, they describe how they identified that some teachers in the Chicago Public System were cheating on standardized tests (yes, teachers, not students - read the book - very cool chapter on why a teacher would be motivated to cheat) - and they did it through patterns ( teachers "fixed" certain exams with the correct answers in a "pattern").

Now patterns aren't necessarily keywords but it would be very cool to take the TagCloud data at the end of, say, a year, and see if there are any patterns in the way people blog - kind of like a VFP ZeitGeist

I love finding patterns in data - it's one of the things I love about using FoxPro interactively. While the TagCloud application doesn't do the analysis automatically, it is very cool the way it highlights keywords that everyone is talking about (obvious ones like Microsoft, vfp and visual foxpro) and ones that have only been mentioned a few times or only on a particular site (like that all important word, "argh!")

No comments: