Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macOS (and iOS?) exports missing provisioning profile required for Apple App Store distribution #73876

Closed
lostminds opened this issue Feb 24, 2023 · 11 comments · Fixed by #74644

Comments

@lostminds
Copy link

Godot version

4.0rc4

System information

macOS 13.2

Issue description

I've been experimenting with trying to get a Godot project onto the Apple App Store (or at least into the Test Flight testing part) and I've run into an issue: Basically, when codesigning code with certain entitlements (like the now required App Sandbox) this also seems to require a provisioning profile to be embedded. However, it seems that Godot does not do this currently, and there is no option in the export dialogue to pick a provisioning profile to use.

In the case of iOS exports I think this problem could be the same, but there Godot generates an XCode project where you can fix the code signing issues and let XCode handle the signing and uploading, so there you can work around it. For macOS there is no way to generate an XCode project where you can fix the signing and upload it to the app store, so you need to create a pkg installer package and use the Transporter app to upload to app. And while I've managed to create a signed .pkg that Transporter accepts and uploads (using this information) I get these two cryptic errors back from Apple by e-mail (APP_NAME replaced):

We identified one or more issues with a recent delivery for your app, "APP_NAME" 0.9.1 (0.9.1). Please correct the following issues, then upload again.

ITMS-90238: Invalid Signature - The main app bundle APP_NAME at path APP_NAME.app has following signing error(s): valid on disk /Volumes/workspace/app_data/SWValidationService/mz_6836161176781981326dir/mz_4826098586216996361dir/com.lostminds.APP_NAME.pkg/Payload/APP_NAME.app: satisfies its Designated Requirement test-requirement: code failed to satisfy specified code requirement(s) . Refer to the Code Signing and Application Sandboxing Guide at http://developer.apple.com/library/mac/#documentation/Security/Conceptual/CodeSigningGuide/AboutCS/AboutCS.html and Technical Note 2206 at https://developer.apple.com/library/mac/technotes/tn2206/_index.html for more information.

ITMS-90889: 'Cannot be used with TestFlight because the bundle at “APP_NAME.app” is missing a provisioning profile. Main bundles are expected to have provisioning profiles in order to be eligible for TestFlight.'

The first of these seems to indicate that there is something wrong with the code signing or signature and I don't really know what since the signature seems valid and there's no issue running the game. But it could be something that is not signed correctly during export.

The second one is however more straight forward and indicates that the provisioning profile is missing as stated in the title. It seems there is a way to embed these (as describe here under "Embed Distribution Provisioning Profiles") but it seems like this has to be done by adding it to the bundle before signing, so it can't be fixed after the export as I understand it.

Steps to reproduce

  • Export a signed (and notarized) macOS app
  • Create a signed installer app store package for this (here's how)
  • Upload the package to the App Store using the Transporter app
  • Observer signing and provisioning profile errors coming back from Apple

Minimal reproduction project

This is I think more of an export process omission/issue and not specific to a certain project.

@akien-mga akien-mga added this to the 4.x milestone Feb 24, 2023
@lostminds lostminds changed the title macOS (and iOS?) exports missing required provisioning profiles required for Apple App Store distribution macOS (and iOS?) exports missing provisioning profile required for Apple App Store distribution Feb 24, 2023
@lostminds
Copy link
Author

As the Apple signing and distribution process has become increasingly convoluted and cryptic to manage outside of XCode it's also perhaps worth thinking about if it would be possible to add an XCode-based macOS export just like with iOS. So that just like with iOS Godot would export an XCode project you can then use to actually build/sign/distribute the game on macOS.

@bruvzg
Copy link
Member

bruvzg commented Feb 24, 2023

Can you provide output of codesign --display --requirements - --verbose=4 APP_NAME.app?

We can add a provisioning profile to the export options, but I not sure what to do with the first cryptic message.

As a workaround, you can try exporting it without signing, manually add provisioning profile, and sign / notarize it from the command line.

@lostminds
Copy link
Author

Here's the codesign output. (Replaced APP_NAME and the actual team id with MY_TEAM_ID) as far as I can tell it looks valid.

Executable=/Applications/APP_NAME.app/Contents/MacOS/APP_NAME
Identifier=com.lostminds.APP_NAME
Format=app bundle with Mach-O universal (x86_64 arm64)
CodeDirectory v=20500 size=555807 flags=0x10000(runtime) hashes=17358+7 location=embedded
VersionPlatform=1
VersionMin=658432
VersionSDK=851968
Hash type=sha256 size=32
CandidateCDHash sha256=ff2de94f93fe53cd8c99b85c5c33d27281c58246
CandidateCDHashFull sha256=ff2de94f93fe53cd8c99b85c5c33d27281c5824645c0af1fae145a9f999a5f53
Hash choices=sha256
CMSDigest=ff2de94f93fe53cd8c99b85c5c33d27281c5824645c0af1fae145a9f999a5f53
CMSDigestType=2
Executable Segment base=0
Executable Segment limit=64655360
Executable Segment flags=0x1
Page size=4096
Launch Constraints:
	None
CDHash=ff2de94f93fe53cd8c99b85c5c33d27281c58246
Signature size=8924
Authority=Developer ID Application: Lars Gafvert (MY_TEAM_ID)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=24 Feb 2023 at 14:48:31
Info.plist entries=18
TeamIdentifier=MY_TEAM_ID
Runtime Version=13.0.0
Sealed Resources version=2 rules=13 files=2
designated => identifier "com.lostminds.APP_NAME" and anchor apple generic and certificate 1[field.1.2.840.113635.100.6.2.6] /* exists */ and certificate leaf[field.1.2.840.113635.100.6.1.13] /* exists */ and certificate leaf[subject.OU] = MY_TEAM_ID

Right, I forgot you could still export without codesigning, that's a good idea to try. If you have any hints on how to manually add the provisioning profile and then code sign it that would be helpful. Otherwise I'll go try to figure it out myself. I guess it could also be a good idea to test and see it signing it that way will also fix the signature issue.

@bruvzg
Copy link
Member

bruvzg commented Feb 24, 2023

If you have any hints on how to manually add the provisioning profile and then code sign it that would be helpful.

AFAIK, if enough to copy it to the APP_NAME.app/Contents/embedded.provisionprofile before signing, but I'm not sure, I do not have Apple dev account to check.

codesign --timestamp --options runtime --entitlements emtitlements.entitlements -s APPLE_ID_CODE -f -v APP_NAME.app to sing.

For the sandboxed app, entitlements.entitlements will look like:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.security.app-sandbox</key>
	<true/>
</dict>
</plist>

If you need some other entitlements, take a look at this part of exporter code which is generating it:

ent_f->store_line("<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
ent_f->store_line("<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">");
ent_f->store_line("<plist version=\"1.0\">");
ent_f->store_line("<dict>");
if (Engine::get_singleton()->has_singleton("GodotSharp")) {
// These entitlements are required to run managed code, and are always enabled in Mono builds.
ent_f->store_line("<key>com.apple.security.cs.allow-jit</key>");
ent_f->store_line("<true/>");
ent_f->store_line("<key>com.apple.security.cs.allow-unsigned-executable-memory</key>");
ent_f->store_line("<true/>");
ent_f->store_line("<key>com.apple.security.cs.allow-dyld-environment-variables</key>");
ent_f->store_line("<true/>");
} else {
if ((bool)p_preset->get("codesign/entitlements/allow_jit_code_execution")) {
ent_f->store_line("<key>com.apple.security.cs.allow-jit</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/allow_unsigned_executable_memory")) {
ent_f->store_line("<key>com.apple.security.cs.allow-unsigned-executable-memory</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/allow_dyld_environment_variables")) {
ent_f->store_line("<key>com.apple.security.cs.allow-dyld-environment-variables</key>");
ent_f->store_line("<true/>");
}
}
if (lib_validation) {
ent_f->store_line("<key>com.apple.security.cs.disable-library-validation</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/audio_input")) {
ent_f->store_line("<key>com.apple.security.device.audio-input</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/camera")) {
ent_f->store_line("<key>com.apple.security.device.camera</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/location")) {
ent_f->store_line("<key>com.apple.security.personal-information.location</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/address_book")) {
ent_f->store_line("<key>com.apple.security.personal-information.addressbook</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/calendars")) {
ent_f->store_line("<key>com.apple.security.personal-information.calendars</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/photos_library")) {
ent_f->store_line("<key>com.apple.security.personal-information.photos-library</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/apple_events")) {
ent_f->store_line("<key>com.apple.security.automation.apple-events</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/debugging")) {
ent_f->store_line("<key>com.apple.security.get-task-allow</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/app_sandbox/enabled")) {
ent_f->store_line("<key>com.apple.security.app-sandbox</key>");
ent_f->store_line("<true/>");
if ((bool)p_preset->get("codesign/entitlements/app_sandbox/network_server")) {
ent_f->store_line("<key>com.apple.security.network.server</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/app_sandbox/network_client")) {
ent_f->store_line("<key>com.apple.security.network.client</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/app_sandbox/device_usb")) {
ent_f->store_line("<key>com.apple.security.device.usb</key>");
ent_f->store_line("<true/>");
}
if ((bool)p_preset->get("codesign/entitlements/app_sandbox/device_bluetooth")) {
ent_f->store_line("<key>com.apple.security.device.bluetooth</key>");
ent_f->store_line("<true/>");
}
if ((int)p_preset->get("codesign/entitlements/app_sandbox/files_downloads") == 1) {
ent_f->store_line("<key>com.apple.security.files.downloads.read-only</key>");
ent_f->store_line("<true/>");
}
if ((int)p_preset->get("codesign/entitlements/app_sandbox/files_downloads") == 2) {
ent_f->store_line("<key>com.apple.security.files.downloads.read-write</key>");
ent_f->store_line("<true/>");
}
if ((int)p_preset->get("codesign/entitlements/app_sandbox/files_pictures") == 1) {
ent_f->store_line("<key>com.apple.security.files.pictures.read-only</key>");
ent_f->store_line("<true/>");
}
if ((int)p_preset->get("codesign/entitlements/app_sandbox/files_pictures") == 2) {
ent_f->store_line("<key>com.apple.security.files.pictures.read-write</key>");
ent_f->store_line("<true/>");
}
if ((int)p_preset->get("codesign/entitlements/app_sandbox/files_music") == 1) {
ent_f->store_line("<key>com.apple.security.files.music.read-only</key>");
ent_f->store_line("<true/>");
}
if ((int)p_preset->get("codesign/entitlements/app_sandbox/files_music") == 2) {
ent_f->store_line("<key>com.apple.security.files.music.read-write</key>");
ent_f->store_line("<true/>");
}
if ((int)p_preset->get("codesign/entitlements/app_sandbox/files_movies") == 1) {
ent_f->store_line("<key>com.apple.security.files.movies.read-only</key>");
ent_f->store_line("<true/>");
}
if ((int)p_preset->get("codesign/entitlements/app_sandbox/files_movies") == 2) {
ent_f->store_line("<key>com.apple.security.files.movies.read-write</key>");
ent_f->store_line("<true/>");
}
}
ent_f->store_line("</dict>");
ent_f->store_line("</plist>");

xcrun notarytool submit --apple-id APPLE_ID_EMAIL --password APP_PASSWORD --team-id TEAM_ID ZIP_WITH_THE_APP_NAME.zip to notarize.

At least that's what Godot exporter do.

@lostminds
Copy link
Author

Thanks for the help. After a couple of tries I managed to correctly sign the package and upload it to the app store without any errors during upload or processing. However, even though it now no longer complained about the missing profile it's still listed as not available for testing in the Test Flight interface. I've contacted test flight support to try and get some information on what is still missing. I'm unsure if this means it would also not be able to publish the game, or if it's just not available for Test Flight testing.

Here's a breakdown of what I did in steps (some the same as your description, with some modifications)

Step 1: Export the game from godot with codesigning disabled
As an .app bundle APP_NAME.app in the examples below

Step 2: Download an Apple Mac appstore provisioning profile
You can generate and download one from the apple developer website certificates and profiles

Step 3: Place a copy of your distribution provisioning profile in the app bundle
APP_NAME.app/Content/embedded.provisionprofile as described here

Step 4: Set up an entitlements file
For example APP_NAME.entitlements, minimally specifying App Sandbox as it's required. Here's my minimal file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.security.app-sandbox</key>
	<true/>
</dict>
</plist>

Step 5: Codesign the app bundle using terminal
codesign --timestamp --options runtime --entitlements APP_NAME.entitlements -s "Apple Distribution: TEAM_NAME (TEAM_ID)" -f -v APP_NAME.app
Note here that you have to use the same signing ID (usually "Apple Distribution: TEAM_NAME (TEAM_ID)") as you used for the provisioning profile, otherwise it will be considered invalid. So it's not the same as the signing ID "Developer ID Application: TEAM_NAME (TEAM_ID)" that you use when you sign for distribution outside the app store.

Step 6: Create an appstore package for upload using terminal
productbuild --component APP_NAME.app /Applications --sign "3rd Party Mac Developer Installer: TEAM_NAME (TEAM_ID)" APP_NAME.pkg
Note here that we need to explicitly state the /Applications install path and that we need to sign it again using a different identity for creating installer packages.
Other things to note is that the package will also be rejected if the icon .icns file doesn't contain the largest 1024 size appstore icons. It will also be rejected it you've already uploaded a package with the same build number ("version" in the export dialogue, not short version), so that needs to be unique for each upload.
Also the pkg generation process adds a minimum required macOS version that seems to be missing from the original godot export?

Step 7: Upload the package to the App Store using the Transporter app
You can get the Apple Transporter app from the app store, sign in with your developer ID and then upload your APP_NAME.pkg file.

As for the bug report above I think the steps that would be required to do this in Godot would be to select in the export panel codesigning if you're signing it for developer ID distribution outside the app store (including the notarization upload step), or if you're exporting for App Store distribution and in that case include the provisioning profile, sign with the appstore distribution ID and maybe even generate the pkg you need for Transporter but skip the notarization.

@lostminds
Copy link
Author

lostminds commented Feb 28, 2023

Some further follow-up:
It seems that while it's unclear if these entitlements are required for publishing there are two additional (undocumented) entitlements required for Test Flight testing via the app store: com.apple.application-identifier and (probably) com.apple.developer.team-identifier. The latter should just be the team ID and the former is a combination of the team ID and the app identifier. So for example if your teamID is "W12345678" and your app identifier is "com.company.game" the com.apple.application-identifier should be "W12345678.com.company.game"

This means that the minimal entitlements file required would look like this (with placeholders)

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.security.app-sandbox</key>
	<true/>
	<key>com.apple.developer.team-identifier</key>
	<string>TEAM_ID</string>
	<key>com.apple.application-identifier</key>
	<string>TEAM_ID.APP_IDENTIFIER</string>
</dict>
</plist>

With this addition I finally got the build up on App Store Connect and working in Test Flight.
(Edit: Spoke a little too soon here. While this got it working for internal testing trying to submit it for review to allow external testing got me a new different error about using a beta xcode version that is not allowed. Again, waiting for apple support to see if they can help with this.)

@bruvzg
Copy link
Member

bruvzg commented Feb 28, 2023

Some further follow-up

I was thinking about adding these two, but app won't work if you use them without provisioning profile. I'll update my pr to add it if profile is selected.

@bruvzg
Copy link
Member

bruvzg commented Feb 28, 2023

different error about using a beta xcode version that is not allowed

Official export templates are compiled using OSXCross, so this might be the reason, and require template rebuild with Xcode. Or it might be looking for the DT* keys in the Info.plist (which are usually created by Xcode, but not set in the export template).

@lostminds
Copy link
Author

Official export templates are compiled using OSXCross, so this might be the reason, and require template rebuild with Xcode.

Ok, perhaps a separate issue should be opened for this then if you think you know what the issue might be? Otherwise I think it will block any mac app store distribution since you can't submit the builds for review even if you can now upload them.

@lostminds
Copy link
Author

It seems like the missing DT* keys in the info.plist were indeed the issue. Good catch! I've done some tests and opened a separate issue for this #74154

@bruvzg
Copy link
Member

bruvzg commented Mar 1, 2023

It seems like the missing DT* keys in the info.plist were indeed the issue.

That's good, we can add them easily (I guess as a pre-filled export options to allow users with custom templates setting different versions).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment