Blog Not Needed

The development is progressing wonderfully and the site is shapping up rather nicely, so, having said that I have a firm launch date of 20th May, 2006.

On 20th May, which is a Saturday, the new site will be going live. The site’s data will be just as it was in December 2005. There will be a lot of dead links from long dead sites in the database and over the next several months the database will be cleansed of old dead links, and to keep things fresh, I’ve implemented a more rigerous indexing process. One that validates URL’s and also does GET tests to see that they don’t return 404 errors.

I’m pretty excited to get this up and running and the new site just feels better. Things work better (which isn’t to say it will be bug free, but one of things I’m going to do, starting around the 13th of May is having beta testers use the application before it goes live.) Yes, there is an extensive project management infrastructure in place for me to manage this new roll out, and as I stated before, there is a bug reporting application in place to capture all of the problems during testing. Development is nearly complete now, so testing is about ready to start, if you want to help test, let me know.

Heh, um yeah. Because of the entire re-design and re-write of everything, I want to make sure those that care enough to report bugs/defects/oopses can do so in an effective manner. So, to that end I set up a nice open source bug tracking system. I was considering bugzilla, but that interface is way too complex for the non-technical (hell, even for the technical it is way too complicated). So, I opted to go with Mantis.

When the site launches, I will link in the bug defect reporting site, that has a generic “reporter” user account, so folks won’t have to create a mantis account. Following is the access information:

URL: http://mantis.diysearch.com
Username: general
Password: diysearch

I’d like to get as many people as possible to report problems, it just makes for a better, more well-run site.

diysearch “celebrates” its tenth year in existence. Yeah, that’s right. It was in 1996 when I created a sub-site off of my UPState zine website that was a basic search interface on top of a text file that listed links from other zine editors. Basically I Posted to usenet letting folks know about this little side project. Well, the site has gone through a lot in those ten years. A decade! gah!

Yeah, if I spent more time the site would probably be much more established than it is, which brings me to this post. Over the last many months, while my life was in turmoil, I was considering just getting rid of the site, selling it, passing it off… anything. It wasn’t until my sweetheart convinced me to actually build a new version of the site, from the bottom up and put forth a concerted effort to make it more, to make it satisfying again. I’ve been working with this project for a decade, and has never returned a penny. What I’ve come to realize is that in order to make this project more satisfying it has to be less of a burdon on myself, and to do that is to get this place to start paying its own way.

Its important to note that the changes I’ll be getting into do NOT have anything to do with making adding links a pay service. Hell NO! What I’m alluding to here is a series of “value added” things we’ve been coming up with, following google to some respect. Hey, google, to date, has got it right so far. I’m not prepared to go into details about the value added services quite yet in that they are still forming up, and will be rolled out post launch (post 20th May). I have decided that this project needs to make it easier on me to continue it.

What’s in it for you? Well, to put it bluntly, we’ve got a whole (marketing bullshit term approaching) brand equity built up in this thing, and not to mention a bit of a legacy, so the better we do, the better your project does. But you already know that. You know, that whole symbiotic relationship thing?

Another change is the bringing on of an editor-in-chief. The last version of the site was content heavy, but because of the system in place, it opened up huge security holes, so when the old site was destroyed, all of that content went with it (*sigh*). At the same time, I was getting email from users whose main complaint was that it was difficult to navigate the actual core functional piece of the site, the search engine! Heh, yeah, I understand. With the new site, the content is being broken off into a separate site (the Wiki) and will keep the main site (the search engine) completely clear. The Wiki will play an integral role in the main site, and will, hopefully, stand on its own. Because of this, I’ve turned ownership of the Wiki to Solomon Grey, our editor-in-chief. She is going to spend her time seeding the site, managing the content (as well as making sure the stuff I write is up to snuff). She’s a techincal writer and was looking for a project to further her technical writing chops and agreed to volunteer for the site. I’m damned excited over this turn of events. She’s got skillz.

Last, but not least (so far) is the new look. You may have noticed the new simpler look and feel. Yes, the old site was too complicated. This new one is simple, dare I say elegant? Keeping the graphics down to about zero, a newer more simple logo/id and a much lighter weight UI will just make the user experience better.

There we have it. There are other changes in the works, and I’ll be using the blog to document them, but yeah, things are looking up!

I’ve been making a lot of noise about the new development and testing of the new site. I thought I might take a minute here and describe the architecture. The new site is basically based on two core technologies. PHP and Java. The front end is basically all written using PHP using a basic form of an MVC design pattern. I was considering using PHP Trax, but frankly I didn’t need or want a framework with so much “stuff” involved.

I am using my own form of web services (again, I needed something more specific rather than SOAP) in order to integrate the front end with the back end. The front end never actually connects directly to the database, instead there’s an extensive middle tier. That is what is written using Java. Basically there’s a servlet, called the service proxy, which handles all requests from the front end. The idea is that the servlet takes XML over HTTP, translates it into java objects, then invokes an engine which handles the specific action, then translates the response (result) objects back into XML and passes that back to the front end. The entire engine mechanism is rather dynamic in that when ever any new functionality needs to be developed, a new engine can be plugged in without effecting any of the existing code. Everything is driven off of a configuration file and a basic set of POJOs (plain old java objects).

The result will be a highly flexible application that can be modified and adapted without impacting the site in any meaningful way. There is nearly 10,000 lines of code so far and with the development about 85% complete, this new site will be one that can finally achieve my original goal from about five years ago, have an infrastructure that can actually grow.

Well, in an attempt to keep the pressure on me to finish all of the development and testing, I am setting a tentative go live date of 20th May. Basically, this is going to be a phased launch. What that means is, the search engine will go live on 20 May (Saturday) and following that the Wiki will go live. The Wiki is already in place, and will be available on the 20th, but there won’t be a lot of content. My main priority is to get the search engine working (that includes all of the functionality, like user management, indexing, user account management etc.)

So, that’s the plan so far. Its less than a month away. So I should stop writing about it and get back to work.