i need a program that is a distributed web search engine.
below are my requirements.
The search engine must use a crawler to build a database and the search should be conducted on the database.
Your system must be distributed, which means that it should have subsystems that run in different computers --- but this nature should be transparent to users as users will use your system in the same way as an ordinary search engine such as Google. If the programming environment of your lab does not allow you to implement subsystems in different computers of the network, you should implement the subsystems (multiple processes) on the same computer.
Your system should be able to crawl at least TXT files (those with extension ".txt"). All searches are with regard to the contents (rather than names) of files. For instance, suppose a file "[login to view URL]" includes a string "xyz", then your system should return the file "[login to view URL]" when users enter a query for "xyz".
Your system should support advanced search operators, namely ' ' (that is, a space) for "and", 'OR' for or, '-' for exclusion, and double quotation marks for exact-word search. The combination of these operators should also be supported, using brackets (that is, '()') to define the sequence of operations. For instance, the following is a valid query:
(ABC ("DEF" OR GHI)) - JKL
If no brackets are included, then the sequence of all operation is from left to right. Please note that there is a space in front and after each operator.
Your software should be written in a programming language that is available in your lab, so that your tutor can compile and run your program in the lab.
you can integrate some open source search engine but it cannot borrow everything from the open source software. please state which portion belongs to the open source search engine.