Friday, December 21, 2012

Favourite Quotes


Let each man exercise the art he knows.
Aristophanes

Judge each day not by the harvest you reap but by the seeds you plant.
Robert Louis Stevenson

In a gentle way, you can shake the world.
Mohandas Gandhi

Ideas shape the course of history.
John Maynard Keynes

Change your thoughts and you change your world.
Norman Vincent Peale

Great hopes make great men.
Thomas Fuller

You have to dream before your dreams can come true.
Abdul Kalam

Wherever you are - be all there.
Jim Elliot

Always bear in mind that your own resolution to succeed is more important than any other.
Abraham Lincoln

Be sure you put your feet in the right place, then stand firm.
Abraham Lincoln

Give me six hours to chop down a tree and I will spend the first four sharpening the axe.
Abraham Lincoln

I do the very best I know how - the very best I can; and I mean to keep on doing so until the end.
Abraham Lincoln

I like to see a man proud of the place in which he lives. I like to see a man live so that his place will be proud of him.
Abraham Lincoln

I will prepare and some day my chance will come.
Abraham Lincoln

I'm a slow walker, but I never walk back.
Abraham Lincoln

If there is anything that a man can do well, I say let him do it. Give him a chance.
Abraham Lincoln

That some achieve great success, is proof to all that others can achieve it as well.
Abraham Lincoln

The things I want to know are in books; my best friend is the man who'll get me a book I ain't read.
Abraham Lincoln

To sin by silence when they should protest makes cowards of men.
Abraham Lincoln

Whatever you are, be a good one.
Abraham Lincoln

When I do good, I feel good. When I do bad, I feel bad. That's my religion.
Abraham Lincoln

You cannot build character and courage by taking away a man's initiative and independence.
Abraham Lincoln

You have to do your own growing no matter how tall your grandfather was.
Abraham Lincoln

The truest wisdom is a resolute determination.
Napoleon Bonaparte

Thursday, December 20, 2012

Worth Reading..


A professor stood before his philosophy class and had some items in front of him. When the class began, he wordlessly picked up a very large and empty mayonnaise jar and proceeded to fill it with golf balls. He then asked the students if the jar was full. They agreed that it was.

The professor then picked up a box of pebbles and poured them into the jar. He shook the jar lightly. The pebbles roll ed into the open areas between the golf balls. He then asked the students again if the jar was full. They agreed it was.

The professor next picked up a box of sand and poured it into the jar. Of course, the sand filled up everything else. He asked once more if the jar was full.. The students responded with a unanimous ‘yes'.

The professor then produced two Beers from under the table and poured the entire contents into the jar effectively filling the empty space between the sand.The students laughed.

Now,said the professor as the laughter subsided, ‘I want you to recognize that this jar represents your life. The golf balls are the important things—-your family, your children, your health, your friends and your favorite passions—-and if everything else was lost and only they remained, your life would still be full. The pebbles are the other things that matter like your job, your house and your car.The sand is everything else—-the small stuff.

‘If you put the sand into the jar first,’ he continued, ‘there is no room for the pebbles or the golf balls. The same goes for life.

If you spend all your time and energy on the small stuff you will never have room for the things that are important to you.

Pay attention to the things that are critical to your happiness.

Spend time with your children. Spend time with your parents. Visit with grandparents. Take your spouse out to dinner. Play another 18. There will always be time to clean the house and mow the lawn.

Take care of the golf balls first—-the things that really matter. Set your priorities. The rest is just sand.

One of the students raised her hand and inquired what the Beer represented. The professor smiled and said, ‘I’m glad you asked.’The Beer just shows you that no matter how full your life may seem, there’s always room for a couple of Beers with a friend.

Monday, September 24, 2012

Design of web search engine

USING C :The design of the search engine can be basically done in C using the data structure PATRICIA TREE for indexing the terms.The md5 hash function and porter stemming algorithms can be found on net.The complexity lies in the type of documents indexed.If the search engine is indexing real world documents like web pages containing html data,the design would become  very complex in C.The design can be done in the following steps:
  • Remove the stop words and apply porter stemming for the documents need to be indexed.
  • The clean documents are indexed using the data structure patricia tree which internally uses the md5 hash function for calculating the 128 bit hash.(2^128 unique entries can be indexed)
  • Using the hash values generated by MD5 algorithm index the terms in each document maintaining the document ID,document frequency,term frequency which will be helpful in identifying the search terms while indexing.[For decreasing the index size without compromising with the performance of search engine these details are necessary]
  • The question arises at this juncture is...What is the index size?Is the index fit to  main memory?
  • The answer  lies in the number of documents we are indexing.The main memory is of course not sufficient if we are indexing large collection of data.
  • The solution is...writing the patricia tree ie., index to hard disk(secondary memory) and reading it from disk for reconstructing the tree while performing search or re-indexing the modifying documents based on different measures[time,page history].This similar to writing a tree to secondary memory ie., file and reading the file for reconstruction.
  • The search function includes traversing the tree and checking for posting lists of the search term.
        The efficiency of the design is depicted by the time taken to show the search results.We can include several features including snippets and pagination for SERP.  
Useful links:
MD5 hash function:gcc.gnu.org/svn/gcc/branches/cilkplus/libiberty/md5.c
Porter stemming function:www.cs.cmu.edu/~callan/Teaching/porter.c


USING PYTHON:Alternatively, you can use the Whoosh library in python for indexing the documents parsed by beautiful soup also a python library.You need to just take care of the documents to be indexed.

Firstly parse the html documents through BEAUTIFUL SOUP and obtain the schema ie title,body and relavant text need to be indexed.Then use the whoosh documentation for indexing the text parsed by beautiful soup.You can perform stemming,and all other preprocessing using the library functions available in WHOOSH.

If several indexes are available ie.,LUCENE(search engine index constructed using lucene) can be integrated with the WHOOSH index using HAYSTACK.Many real-world websites are using HAYSTACK to search the indices for a given query.

Pros an Cons:
As the index size grows the efficiency of the search engine falls drastically.An increase of 50% index size may slows down by factor of 0.8 of performance.This is very advantageous for research purposes and for comparing performances of search terms used in indexing of search engine.

http://packages.python.org/Whoosh/indexing.html/
http://pypi.python.org/pypi/Whoosh/
http://pypi.python.org/pypi/BeautifulSoup/

Useful links for integrating several search indexes for having a front end search engine:
http://haystacksearch.org/
http://groups.csail.mit.edu/haystack/