Renaming foder error

We are using the code below to rename a customer folder in a virtual directory.  If we try it shortly after accessing the folder (e.g. reading/displaying photos) we get the error below.
Access to the path '\\lifedevr2\Apps\shareddocs\Photos\HARRISON, LINDSEY & MATT' is denied.
If we go back 20 minutes later everything works fine.  Apparently asp.net holds file system stuff for some time.  We are using impersonation so that access to the file system is ok.  Below is the code we are using to rename the file folder.
                    Dim strFolderPathNew As String = "~/Photos/" & strNewFolderName                     Dim strPathPhyNew As String = HttpContext.Current.Server.MapPath(strFolderPathNew)     &nb ...