I have dealt with optimizing several drupal sites. Many times it is because lack of resources on the server, poor service provider for the server, or more often it is because the number of modules / poorly optimized code.
There is a rare case that the issue you are seeing is because of the media you are loading. In this case it would mean there would be a need to change how the page loads. We can create lazy loaders for the content below the fold(off screen) to load shortly after the initial page load.
Let me know if you have any specific questions. Additionally we should specify what connection speed you have that you are wanting to target a 2 sec load time, I think a good mark is the 3G connection of a cell phone as it is one of the more tricky marks to hit.