C# Multithreaded App Shell for Web Spider / Bot / Scrape
$30-250 USD
En curso
Publicado hace más de 15 años
$30-250 USD
Pagado a la entrega
Hello everyone,
I am looking for an experience .NET programmer to help me out with a learning project. I would prefer C#, but will accept VB.NET...but I must insist that the code is .NET 2.0 ([login to view URL] 2005). As the heading implies, this project is a multi-threaded spidering/scraping/bot application. I threw together some C# code that retrieves the page source of one web page, but need to know how to do many pages at a time, in a multi-threaded manner.
I am looking for someone that has done this before and has access to the source code... the reason for this is because I want it for a rather inexpensive rate (i.e. CHEAP)! If you could just plug the multi-threaded portion of your code (not using the backgroundworker object) into mine, that's all I want. I'd like one variable to determine how many threads run at a time, with the initial value being 10. There is a list<string> variable that will contain all the web source code, a listbox that contains the page URLs, a RichTextBox to contain source, and a WebBrowser object. I'd really like two quotes from everyone: one that just returns the list<>, and one that works along with the WebBrowser object (once each page finishes loading, it goes to the next). I'm pretty sure I won't need this second portion, but I want to know how it's done. So really just give me a quote on filling the list<> object.
There are a few things that I'd like to add, along with just the retrieval of some strings with my existing function.
*** ProgressBar - I'd like the progresbar to update, depending on how many pages have been retrieved (example: 20/25 pages finished would show the "image" of 80%). Easy stuff...
*** Ability to change number of threads
*** The form close event must be handled. For example, if there are threads running and the user clicks exit, it must kill all threads and exit the program.
*** TextBox Update on the Main form: I'd like to know how to SAFELY modify objects/elements on the main form while multi-threading. Please add a textbox that shows the status of the process (example: each time a new page source is retrieved, increment a counter on the textbox by one.
That's it. I am including the sample project file that has the 'getSource' function. It has all I mentioned except for the 'counter'/'status' textbox.
As for the URLs, just fill in the ListBox with a listing of like 80 different (or the same) URLs for testing.
Thanks and remember I don't want to pay too much b/c this is for my own learning. That's why I'd prefer someone who is either an expert at multithreading with WinForms, has a project(s) with the source that they can just plug into my app, or has the ability to quickly copy examples from other pages into my code.
*** This is a .NET 2.0 Winforms application ONLY; please do not reply with something like "I can do this in Perl"...
Best Regards,
Panuvin