As I found out the hard way captchas are a must for sites that allow anonymous users to post comments to blog. Spammers have gotten pretty smart and can easily defeat simple methods of preventing spam as I had implemented previously. Previously I have simply asked the user to enter "5+5" into one of the form fields. Spammers did not take long to script for this simple check and next thing I know I got 500 comment that where links to some link farm. Hopefully with the new captcha mechanism I implemented and the fact that there is an extra step in the form submission this is the last of this type of spam that I am going to see.

I restarted development on the php blog that runs this site. I realize that WordPress is pretty bad under the hood. So far I have lots of cool ideas for the directions to take the project into. In the immediate future I need to at least implement the basic features that most people would expect. After that I can start to developing my new ideas for better ways to blog.

Today I read the discussion on slashdot about Wikipedia 2.0 - now with added trust article. Apparently the new version of wikipedia will include some sort of trust model beyond what is available right now. This was not surprising to me since I have for a while thought that wikipedia needs some more tools to help in ensuring quality of its content.

Recently I have found myself finding the site indispensables to simply satisfy my curiosity from time to time. I like watching the history channel and if I hear something that I want to know about it well just type it into google and if its a well known topic the first result is usually for a wikipedia article. At work I find myself going to wikipedia articles sometimes when I was to get the first idea about some technology I want to look into. I got a bad case of Poison Ivy last week and although I was not able to diagnose it by looking at the list of skin diseases (too bad no visual search exist yet) I was able to find out everything I needed to know to confirm my diagnose from a doctor and even get a pretty good idea which over the counter meds can help me. So as it is obvious the site is important. Yes the reliability is not guaranteed and yes I find it necessary sometimes to read the second sources to get a clearer idea sometimes (especially in the Poison Ivy situation) but overall the site is taking on an ubiquity where most trust in its contents will only benefit the general public. It should be in everyones interest that tools are being worked on to help find vandalism more efficiently and score editors and most importantly facilitate better research!

As far as the technology of verifying trust goes things are not clear cut. Its going to be harder at developing helpful tools then the article makes it sound. To me the stuff is basically in the research lab stages. I watched the a google tech talk by one of the Wikipedia chief programmers. The sense I got from that talk is that technologically wikipedia has bigger technical issues to spend its resources on. For one they need better editing functionality. They need scalability beyond what their current architecture can provide. And they need better UI improvements in many places. If anything they need to start structuring their articles in a more structured form then the hodge podge of wiki tags (although I am not sure if their current approach is actually more effective overall). Anyway if anything comes out of the metawiki related to the trust model at best it will be experimental at first.

One of the reasons why I think integrating a trust model is hard is as follows. The computers will have too much say at deciding what is trustworthy and what is not. Although this kind of work for Google at deciding which web pages are relevant it wont work as good for wikipedia. If Google misinterprets a webpage as relevant there is not much of an implication but if the wikidedia makes mistakes many people will lose trust in the system. As the article suggested another side effect will be that certain rouge users will build up trust ratings for the sole purpose of increasing trust in some edit they they would be say paid for. In effect the rating system will become a means of getting higher ratings in itself. Instead of trying to do really hard statistical analysis on the data and protecting against all the potential spammers that would try to profit from it somehow a few simple policy tweaks can go a long way.

Here is a quick list of tweaks that I think can go a long way on increasing wikipedia trust without trying do an overly complex system:

1. Stop allowing anonymous edits. The IP trails have become kind of a irrelevant distraction form the issue at hand. It does not matter that the IP of the edit belonged to whoever company. The IP does not represent that any company and too many words have been written on blogs about this pointless metric. Take out the IP tracking for anonymous users and only allow individual users to take responsibility for their edits. Of course the IP can still stick around in the log files and those can still be provided to everyone as they are now but thats about it.
2. Better automation for people who want to do fact checking and reviews of changes to articles. Right now there is no clearing house for this. There is a raw feed of edits but that is less useful because those come at a rate thats impossible to track. Perhaps the edits should be distributed in a collaborative way to different reviewers and put into an inbox for them to handle. An edit can go to two people and if both people disagree the software can escalate the review of some edit. This feature would be in addition to all the watch lists that people use.
3. Put people into expert groups and provide them the necessary materials that they would need to do proper research. This is kind of hard too. The internet is not always the best research source. My poison ivy case is a perfect example. I found very little scientific information on the web on the topic of "contact dermatitis". Apparently medical researchers have little interest in a condition that generally can be considered an annoyance; never mind that it generates about a million ER visits annually in the US. For the few facts that had sources cited in the wikipedia article I found the sources to be no more than first page google results and being particularly weak in their authority. For one of the section I suspected that the manufacturer of the product may have played a role it its edit although it was impossible for me to verify. (Good luck at letting AI do it when even a human cannot)

I been guilty of neglecting my blog in recent times so I thought I should just sit down and write something.

I changed jobs recently. I used to work at Zerve as a PHP programmer and now I work for another company as a Java programmer. Since my new employer is in Brooklyn I conveniently moved to Brooklyn to be closer to work. So far I like the new job. Technically the job is pretty easy with normal working hours. When I get home there is time to think about other big ideas.

The cable company. I don't have much good to say except that it all works now finally. Cablevision should really invest in the training of their technicians. But not even so much as training as at least develop proper installation procedures. The executives at this company should read this as a case study to use to fix some basic problems with their customer service.

There was no reason that 3 visits by their technicians had to happen when it all could have been done in the first installation visit. For some reason the cabling in my apartment building was done improperly by the original contractors who installed it years ago. At least that is what one of their guys told me which I thought was a smooth way to blame someone else. "Oh it was not us it was the contractor" -- like that makes a difference.

In any case the original guys who done the cabling years ago crimped the cable going to my apartment incorrectly in no less then two places. Tech 1 who was doing the installation, hooked up to the bad cable coming into the apartment. He did not check the signal strength nor did he bother to even bring the checker device they all have. After confirming that the TV did turn on and that some channels worked, he just left. Had he did his signal check or even bothered to note that one of the crimps on the cable outside my window was done poorly he could have went ahead and fixed the cable. Not surprisingly I found some channels did not work some of the time. After a lot of wasted time on the phone I had the second tech come over. Tech 2 also failed to see the bad crimping job outside my window but assumed that the problem is probably at the other end of cable around the building. He replaced the crimp on the cable around the building saying it was a bad contracting job. He at least tested the signal after the fix and said it was bare minimum. Some time later I found my cable completely not working. I gave an angry call to tech support and scheduled another appointment. A short time later I discover that the crimp outside my window has failed and the cable was completely dislodged. In any case the 3rd guy fixed that problem. I don't see why all this trouble could not have been avoided has the original tech done some preventative maintenance and fixed the cabling. Also none of the two technicians who knew there was bad wiring in the building made any attempt to fix all the remaining badly crimped cables for the rest of the tenants. I mean why bother doing it now when they can come countless times over in the future to fix these same problems later?

I want to bring up some points about subject of database optimization based on my collective experience in dealing in this area.

• Optimizing sql queries requires effort which in business world translates into money. A business may want efforts spend on development of features which would bring the company revenue and hence more money to spend on development. So efforts spend on optimization may well be spend on things that are more important at any one time. Off-course if the application is so slow that it is unusable that's a problem that is worth spending the effort on optimizing.
• Normalizing the data model is good in general but its great for performance. I noticed that joining tables is almost always efficient if the data model is normalized properly. Also I found it is helpful to give consistent naming convention to key columns. For example Post( Post_Id , ... ), Comment( Comment_Id, Post_Id, ... ).
• Optimize only when it is required based on real world profiling. Each application may have different set of requirement but in general new application do not have a lot of data and hence will not see performance issues that would show up if a lot of data is present. So in the beginning developing the critical features is more important than making the application snappy. Sure it may take 4 seconds to submit some form and the fix may be trivial but still the effort required may still divert the scarce developer resource. In my experience those initially badly written part of the application are better of being rewritten completely when the data-model is re-factored. At that time it is OK since the parts being rewritten are the ones that matter.

I got to say first of all I like my iPod and the click wheel user interface is nothing but revolutionary. But there is one user interface bug that is still in the iPod even after all these years that is the biggest quirk for me. This is not a bug in a sense that something works incorrectly but a bug in a sense that it should be done differently because it is so obvious.

Here is how I most commonly encounter this problem. I start playing say a song. In the middle of the song I decide to browse somewhere else for whatever reason. Now I decide not to play anything else but to change the volume. This is where things get difficult. Right now I am in a menu item of some other album or artist. In order to change the volume I need to be on the screen of the currently playing song. I can either wait for about 5 seconds for the menu screen to timeout or I can browse to the top of the menu and click on "Now playing". I don't know about you but waiting for 5 seconds when I want something now is way too long. Now I browse to the "Now playing" item and change the volume. At this point it is impossible to easily get back to the album of the currently playing song. What if I want to change to some other song in the same album but because I clicked on the "Now playing" item and because it is right under the top level menu of the iPod, I would have to browse to my album all over again.

Technically this is not a bug. The iPod is actually operating correctly and consistently. It treats the "Now playing" menu item as if it was a song in itself. Clicking on it changes the position of the iPods memory of where you are in the menu. I think this is where Apple has made a big mistake. The iPod should ignore the "Now playing" item as a memorable menu position. Even if it is useful to remember this position (you want to look at the song you are playing and immediately go back to the top to continue browsing other songs) it should only keep this memory for a perceivably short time such as less than a second.

If someone from Apple is reading this please FIX this! Its too easy.

Finally got the site back up. I originally hosted the site on phpwebhosting.com then I rewrote my blog. I wrote the blog in php5 using mysql5. Turned out phpwebhosting did not host those versions of the software on the server I was on. So I figured I move the site to my own server at home at least I would be satisfied with all the support I would need. That is from myself. Turns out I was wrong. I turned out to be the worst person to support my own server. Now I am back with phpwebhosting and on a new server that has all the software I need. I turned my old AMD machine I was using as a server int my new desktop machine running opensuse which is pretty nice. More stuff later...

A friend of my asked me to help her pick out some tech book at Barns & Nobles. I was somewhat disappointed at the selection of books that where on the shelves. Out of the hundreds of tech books only three or four I would consider worth owning and I already own like three of those. I guess that makes me somewhat biased. Lets consider the books that where good.

• The Pragmatic Programmer. This book is great because the authors talk about issues like code duplication, refactoring, testing. All the things that real programmers face every day.
• C Programming Language. Most people who buy tech books there probably don't even know C is a language. This is a weird one to have on shelves where "Learn Asp.net in 5 minutes now" books in the vicinity.
• JavaScript The Definitive Guide. I always love those O'Reilly books that are the definitive guides because they usually are.
• Apache Cookbook. Another O'Reilly book and probably the best tech books possible at to buy at Barns & Nobles.
• The Mythical Man-Month. Another classic that does not fit in with the rest of them. I read it way back in college. The premise is simple. Can't just throw people at a software development project and expect it to work. At least that was the revelation way back in the 70's at IBM. Fast forward thirty years and its still true. Software development still is too hard so you can't just chop it up into smaller chunks of work and had it out to people.

I guess if I needed to leard C# in 10 minutes I would get one of the other books. Although all the information is online anyway that a book is not necessary but nice. Generally O'Reilly, Apress, and Wrox are tech books that are any good the rest is just crap.

For a while I came to a realization that it is pretty much impossible for a Mac person like me to talk and video chat with someone on a PC until today that is. First I thought iChat would do it but not so. There is no iChat client on Windows. Second I figured oh well surely multi-billion dollar Yahoo has a solution. Not so. Even though the Yahoo Beta client for the Mac claims to have video chat it is missing voice. Thats a pretty big oversight if you ask me. But than video does not seem to work with it either as I tried it and it did not work. So Skype is the magic bullet here. Skype just released a Mac client which supports both voice and chat!

At work I continue to work on some involved javascript using some ajax along the way. My general experience is that javascript stuff is harder to do then normal php coding. Ajax just adds complexity. The reason why doing user dynamic user interface in javascript is due to asynchronous nature of the browser. Everything can happen out of order. Dealing with events is also a problem. For example on any div you may get a mouse over event fired after the mouse out event has already fired. Things are generally chaotic in nature. If you try to use ajax using it asynchronously makes things even more chaotic and not much easier. This is why in the few places that I need to retrieve data from the server I just go with a synchronous request. Even though it may freeze the browser for a sec or two at least it will lead to less bugs in the code.

In browser news. Microsoft announced that they will distribute IE7 as a high priority update to Windows XP users. This is actually good news. I wish they done it as a critical upgrade. Its critical for me dammit. How much money have companies wasted in development effort just working around IE6 bugs. Must add up to billions. But that is how it goes. Firefox 1.0 has not been an angel in that regard either. I have come across plenty of bugs in that browser that caused me plenty of grief. Sometimes I just wish it was never released in the first place, or at least not without the automatic upgrade feature. So far i had not come across any major bugs in Firefox 1.5.

Coming up I got to talk about APC and some page caching javascript I will put releasing soon.

I was listening to the security now podcast today which was about denial of service attack. Steve Gibson called it "web weaponry". That guy just loves to coin terms. Anyway the only defense that Steve mentioned was the fact that some company has to pay alot of money for bigger pipes. This way it can actually have a chance of operating and handling some of the legit traffic until the attack can be stopped. I had an idea for another defense which seems kinda obvious if you think about it. Its called "obscurity". Yea they taught us in college that security by obscurity is a sin but this is a different. Any company that relies on internet connectivity for their livelihood should not be telling the public about it. Only the people who need to know about it should know about it. Ofcourse if the clients are the public then there is no choice but otherwise keep your mouse shut!