1.05 and Update

November 7, 2007

Fact200 still remains largely in maintenance mode as there are other more pressing projects higher up in the priority list these days. For the most part this update consists of updates to the parsing code used to gather results from google and yahoo.

Updated Google/Yahoo Sources

One of the principles of Fact200 is to appear mostly transparent (Fact200 does identify itself in the UserAgent field) to search engines so that it receives the exact same results as a person accessing the search engine directly through a browser. The exception to this is live.com’s search which goes through an RSS interface. The downside of this “same as website” approach is that it’s completely non-standardized and when search engines update their pages, Fact200 needs an update to stay in sync. This only affects natively parsed search like Google, Yahoo, and AltaVista. Searches and sources that go through standardized XML feeds like RSS or Atom will always work because of the fixed standards.

Minibrowser Images

Another new feature in this release fixes one of the major annoyances I had: images in the minibrowser. This has always been controversial to me because images (obviously) add a lot to web pages and it’s what I think most users expect. Unfortunately though, they are not cached by Fact200 and thus they cause a major delay in displaying data (in the minibrowser only) that is otherwise is completely cached on the client side; all style links are parsed and placed inline. This turns near instantaneous loading into sometimes perpetually slow spinning, depending on the website.


A large feature that’s been on the todo list for a long time is library sharing within a LAN. The use case is that a user primarily uses his laptop, but is at home at his desktop wanting to search through his library on the laptop. A second use case is an office environment where people have a common set of searches or news.


A second feature that’s always been around, but is more abstract, is an interface overhaul. I’ve always been aware that the interface is a little quirky in two ways: the main interface supports too many modes and the configuration of libraries/categories/sources is complex.

The configuration portion is lower on the todo list because I consider it an advanced item.

I’d like to revamp the main interface, but am at a loss of ideas of exactly how to go about it. The main problem is that there are 3 primary modes of operation: meta-searching (gather data from different sources), library searching (filtering through data that’s already been found), and news reading (gathering data from regularly updated sources).

Right now, the meta-searching interface is the most direct and prominent whereas the other 2 modes are masked by it. Instead of splitting the two modes off (which would be much easier from an interface standpoint), I’m trying to figure out a way to keep them integrated in a more seamless fashion, the common subject point being search.

There’s also a few more minor interface issues like the minibrowser. The pull in one direction is to make it more like a real browser that accurately matches the captured preview images. The pull in the other is to make a quick method for accessing the textual data in web pages. This release pushes it towards the latter.

But in this end it boils down to time and priorities. Fact200 receives maybe 4 hours of development time a month these days.

Been a long time

March 9, 2007

At long last, the 1.0 version has been released! In true “web 2.0” fashion, there isn’t much difference between the release and the betas. In fact, only a day after releasing 1.0 I found a few bugs. Its amazing how the bugs always show up after the testing cycle. For the most part, the major emphasis for this non-beta release was stability and cleanup. Major memory leaks were cleaned up, and many edge case crashes were fixed.

Anyway, this brings the first stage of Fact200’s development to a close. There will, of course continue to be updates, but I will excercise the utmost restraint before adding new features; even small ones. My original concept was to have a 1.0 and a 2.0 release, each centered around a central feature. 1.0’s central feature is its library building, searching, and retrieval functionality. 2.0’s will, in true 2.0 fashion, emphasize more of a social angle.

2.0 should be much simpler than 1.0 which had the burden of being the initial release with many, many, MANY extra features added along the way to help fill in the cracks. Hopefully its in fair shape and 2.0 won’t be a major effort. Actually, instead of calling it 2.0, I’ll probably go with a 1.5 moniker.

Things have gotten busy in other areas of work as well. Enough that time for Fact200 features is getting pushed to the bottom of the queue for now. Bugs still have high priority so if you find any bugs, please post them on the forums site! Actually, I’m still really flexible, so post your feature ideas too! Depending on the difficulty and the utility of the feature, I may just try and squeeze it in.


November 29, 2006

The continuous struggle for building something actually useful.
Project Management on Flickr – Photo Sharing!


November 13, 2006

Been a while since the last post. Fact200 continues forward in development. The last release (1.0b15) mostly saw some bug fixes and convergences towards standards support and integration. Integration mainly comes in the form of using RSS for import/export as well as supporting fact200 specific extensions to preserve sources and to allow anyone to show info in the sponsors area. In addition to extended RSS support, OPML support has been added to allow for the easy import/export of RSS/RDF/Atom sources to and from other newsreaders.

Additional startup script support has been added in the form of //seed and //seedall commands which allow a library to be seeded with urls from a source.  The difference between the two is that //seed obeys the maximum number of result restrictions to help ease the load (in the even all the urls are from the same domain) whereas //seedall attempts to do everything at once. There is a subtle difference in that //seedall will only be performed when the library is created whereas //seed re-checks the source every time. All seeded results are added with the permanent attribute to ensure their addition to the Library.

In other news, I found a replacement for my crappy Compaq x1030 laptop which constantly failed with video card problems.  I went the opposite extreme of the x1000 and bought one of the new C2D MacBook Pros. I spent the last week getting familiar with OS X and configuring the system to run both Bootcamp and Parallels. Even without spending the $$$ for a full retail version of XP Home and Parallels, this sucker cost more than any other system I’ve ever bought. But, at least so far, it seems to have been worth it.

As long as it doesn’t crap out on me in a couple years, the Max OS X/Parallels/Bootcamp combo has exceeded my expectations and with the addition of an Apple Wireless keyboard and Logitech Bluetooth mouse, works as an unexpectedly good desktop replacement for my aging Dell 5100 system which is around 4 years old. I had originally thought of getting a Quad core setup to replace the desktop system, but I’ll probably put that off indefinitely or at least until 8 core systems start coming out.

The only negative I’ve been able to find with the laptop so far is the Battery life. The old Compaq started out at around 5+ hours with the screen dimmed down. With the MBP the longest I’ve been able to get is around 4+ hours doing mostly word processing with the screen dimmed and bluetooth and Wifi turned off. I get closer to 3 hours with the screen dimmed, Wifi on and doing development in Parallels which puts a heavier average load on the CPU. For the MacBook nerds out there: Yes, I did calibrate the battery.

Interesting links of the day:

Croatia – Plitvicka Jezera National Park – waterfalls, Plitvicka, Croatia
Review: iPod Shuffle 2


October 12, 2006

Yahoo released their Internet time capsule today. It’s an interesting property of the Internet that just about anything accessible by a searchbot can be considered as part of permanent web history. I would guess that even pages marked as ‘no-cache’ are still being crawled and saved by ‘bots posing as IE or Firefox.

So it seems that once you hit that post, send, or upload button, it’s good to keep in mind that you’re making Internet history, for better or worse. As Scott Adams found out, retractions don’t really work in the age of the Internet. In a way, the Intenet monitors and records (for what may seem like an eternity) all web activity.

It’s not limited to public Internet spaces either. The beauty of ideas is that they take up no space or materials and are thus easy to disseminate.  Rep. Foley learned that instant messages are not so private, and many Craigslist users found that e-mails can be <gasp> re-posted.

Wow, actual applications?

October 8, 2006

As the author, I guess it shouldn’t amount to a huge surprise that I actually found a real life, honest to goodness use for Fact200. One that would have been extremely difficult if not nearly impossible to do without.

Arriving as a dutiful employee, I arrived at work to check my e-mail. Only there wasn’t any in my Inbox. Odd. Hmm, my IM was down too and my connections to external servers were down. As it turns out, the office DSL was down and in the end it turns out the DSL modem went kaput.

But most of my work could be done on internal servers so it didn’t seem like a big deal…Until I needed to look something up. Blame it on the old age, beer, or just plain laziness: I just don’t memorize things like I used to and the Internet makes a whole lot of information accessible. However, no DSL meant no Internet. So I typed my search query into Fact200 anyway and ‘lo and behold several answers popped up, complete with previews. I clicked on the preview to switch to the minibrowser and indeed, it was a complete reference page for the function call I needed.

I was a little lucky because these particular pages weren’t part of the “Permanent” collection of the library and thus could have been flushed out before I found them. Of course, I immediately marked them as permanent and finished off the little function I was writing.
Break time! Opening up my news feeds library in Fact200, I meandered my way through unread articles. It was yesterday’s news, of course, but it served its purpose quite well.


October 6, 2006

After reading The Design of Everyday Things (DOET) by usability guru Don Norman, this version reflects some surprising and radical changes.

Contrary to modern design fads (spearheaded by spartan Google UI), the Fact200 UI has become slightly more complex. On his site, Mr. Norman has a short critique of Google’s interface that states why he doesn’t think it’s a great design. The gist of it is that if it only does ONE thing, designing an interface is trivial. Indeed, Google does more than one thing these days, and navigating to any of these other things is actually made more difficult due to the interface’s adherance to its old design.

Well, Fact200 was initially modeled in the vein of Google’s interface, trying to make things look as simple as possible. But this came at a price of hiding functionality. One of the principles in DOET is that of visibility: if a users wants to do xyz with your product, it should be readily apparent 1) that it can do it and 2) how to do it.

So, in the end, the mode button which used to switch between loaded internet results and Library filtered results has been converted to a dropdown menu with 4 modes:Loaded results,Filtering Loaded results, Filtering the Library, and Viewing Load History.

The first and third mode are the modes that the old button used to toggle through. Filtering loaded results was formerly a hidden mode that allows sub-searching through recently loaded results only, instead of through the entire library. This has an added advantage over library searching in that it does not rely on the global library indexer. The last mode is new and provides a way for users to retrace their steps.

One of the potential problems of Fact200 in the past was that in indexing all loaded content, it would gradually bog the user’s system down in disk activity and loads would take longer and longer to fully complete. I also surmise that most of the indexing is wasted because 95% of what is loaded is unwanted by the user over the long term. Thus this recent release allows users to turn off the global library indexer.

Un-indexed content cannot be retrieved from the library through search, thus negating Fact200’s original goal of an independant library that is searchable in real time. To help alleviate this void, the aforementioned ‘Filter through loaded results’ feature provides sub-searching functionality and items marked as permanent are still indexed. This keeps most of the functionality intact while drastically improving performance and efficiency. For those who really need to index everything, the option still exists to do so.

Other notable features in this release:

  • The ability to turn off comprehensive image preview generation when doing batch loads. This feature was originally for offline news reading, but Fact200 functions quite well online. When  rendering for online use, every single preview isn’t necessary and helps save lots of time.
  • Wrapping. It’s now possible to see the title/content snippets of search results without having to maximize or resize the window frame
  • Removal of compact tabs option. I used to like the wide tabs, but ultimately, I stopped using them and they really just waste space. So all tabs are now compact and an option is no longer available for them to be wide.

BeforeNavigate2 Woes

September 23, 2006

There’s a bug I discovered a couple days ago where the Minibrowser sometimes displays the wrong URL and will jump to the wrong url if the external browser action is invoked (either by clicking the button in the Find pane or by clicking return in the Minibrowser).

This is caused by frame URLs invoking the BeforeNavigate2 event causing the minibrowser to think that its navigating somewhere else when in reality it’s probably just an Iframe loading an ad. The prototype for the function is:

afx_msg void CMiniBrowser::OnBeforeNavigate2(LPDISPATCH lpDisp,BSTR url,short flags,BSTR target,VARIANT FAR* post,BSTR headers,BOOL *cancel) ;

At first, I thought that just looking to see if the target length equaled 0 would be sufficient to detect/distinguish a frame. This is unfortunately not a safe assumption. After combing the internet (with Fact200 no less) and Usenet newsgroups, I was only able to find other people with the same question but no answers. After a whole bunch of trial and error, including using the TopLevelContainer method (which didn’t work!), I settled on the following:

afx_msg void CMiniBrowser::OnBeforeNavigate22(LPDISPATCH lpDisp,BSTR url,short flags,BSTR target,VARIANT FAR* post,BSTR headers,BOOL *cancel) {

IWebBrowser2 *wb;

LPDISPATCH container;

bool isFrame=false;

if (SUCCEEDED(lpDisp->QueryInterface(IID_IWebBrowser2,(void**)&wb)) && wb!=NULL) {

if (SUCCEEDED(wb->get_Parent(&container)) && container!=NULL) {







Side note: WordPress’s editor is absolutely horrendous for formatting code!

1.0b12 Released!

September 19, 2006

Following my semi-normal 2 week cycle, 1.0b12 has been released featuring mostly bug fixes. The major bug fix in this release is to Google Groups which changed their search result format thus causing Fact200 to return no results. Google has also changed the output of their Sponsored Listings which is why they don’t show up, but fixing this is way, way down on the list of priorities.

The most distinct new feature of this  release is the ability to adjust the text size in the results page using the ctrl+ and ctrl- combos. For us young folks (although I think I might be halfway into the middle-ground), having small microscopic type enables us to see more, yet for the older folks like my Dad, large type (we’re talking 24pt here) makes things much more comfortable.

In addition to adjusting the type size, you can also adjust thumbnail sizes from the keyboard. This was and still is customizable from the preferences dialog, but adjusting it from the keyboard is much faster and gives instant feedback (a big theme of Fact200).

There are various other subtle changes to help the UI be a little more intuitive and visible.

I’m about finished reading  Don Norman’s “The Design of Everyday Things”. Although it doesn’t contain any stunning revelations (considering Fact200’s UI, I’m sure this is a surprise to some of you), it codifies UI principles I’ve already read and known about for some time. I’m not sure where in the time-line it will occur, but I’ll probably do a systematic evaluation of the UI and try to iron out some of the obvious UI gaffes that are present in Fact200.

The emphasis of Fact200’s UI is to get people using it as easily as possible and I think by providing a typical search/Google look, it has accomplished this just fine. However, the ease of the initial features comes at a price of making the more advanced features harder to use, largely because they’re invisible.  The initial features also look a little too similar to traditional search results and thus there’s no apparent benefit to Fact200 vs. just using Google, although I think the hover previews are an exception to this. I’m currently thinking of averaging the difficulty out as much as possible so that the initial simple actions might become a little more difficult (or daunting), but the more advanced features will become more apparent and easier to use. Of course, ultimately this will all depend on user feedback.


September 15, 2006

Even the mighty G is not immune to javascript exploits and phishing scams.
Phising Exploit Discovered in ‘Google Public Search Service’