Ok I figured out what the problem was. So I opened Terminal and used the codesigning app to sign the .app file using my developer certificate. After that I was able to double-click the .app file and launch it successfully.
This is the command I used in Terminal:
codesign -s "Your Company, Inc." /path/to/MyApp.app