The problem is the size of the file (in MB), unfortunately it just takes longer to process and transmit larger file sizes.
Non-secure version of the file loads more quickly because there isn't an encryption overhead and it is possible to transmit in chunks since it does not need to be encrypted.
The possible solutions are:
- Optimize the PDF itself so that it is smaller. Perhaps try a service such as Smallpdf.
- Fine-tune the web server (and/or upgrade it). It might be possible to upgrade the memory/processing capabilities of your server to ensure processing happens more efficiently. All of this would need your web host's involvement.
- There is the option to cache the encrypted version of the PDF so only the first time the PDF is viewed will it be need to be encrypted. The 'Cache Encrypted PDFs' option can be checked on the Secure tab of the plugin settings.
Ultimately, if you find secure PDFs are too slow for your site, you may prefer to try uploading the PDFs in non-secure mode instead. It is a trade-off for how important you think the 'secure' functionality is in your case.