File download access to path denied

I have two servers on which web app is running on one server and the other one is doc server. Both are under the same domain.
When I try to download a file programmatically I get the error access to path denied.
Here is the authentication set in IIS:
Anonymous - IUSR ASP.Net Impersonation Windows Authentication
AppPool - Network Service
web.config
<system.web> <identity impersonate="true" /> </system.web>
Looks like permission issue, what permission to be given to the download directory?!