Hi All,
There are 4 separate tasks that need to be accomplished:
1) Web App Script (using Django/boto or similar) - The script needs to be able to execute the following tasks:
- When file is loaded from website, automatically run a pre-processing script (this would require 1 small-instance)
- Launch multiple spot cluster compute instances automatically if pre-processing script exits cleanly, and then run our HPC application. We are planning on using Rocks ([login to view URL]) or Starcluster ([login to view URL]) for resource management (I have Starcluster configured as of now)
- Terminate all instances if there are no more simulations to run
I was recommended by Amazon to set up a web tier of instances on EC2 to handle the web requests, a processing tier for the initial pre-processing and orchestration, and SQS for the message queues. Amazon can scale out our tiers depending on usage, either by the number of web requests, or the size of the message queues using the CloudWatch metrics service.
2) User Registration and Authentication - Django or similar
3) Large file upload to servers using jQuery and simple web page design for landing page and file upload page (Similar to [login to view URL] . Once the user has logged into our website we would like to allow them to upload multiple 0.5GB files.)
4) Database setup for storage of user data on Amazon persistent storage.
Let me know if you have any questions.
Thanks!