Add --debug option to briefcase run

[timrid]

This is an first attempt for adding (remote) debugging support for bundled apps. This is not ready yet but i would like to share this anyways.

It supports:

• pdb
• debugpy (for VSCode)

I have tested the following platforms:

• Windows
• Android (on Windows)
• macOS
• iOS Simulator (on macOS)

The following points are missing:

• Test this on an real iPhone (unfortunately I don't have one)
• Currently you have to add import __briefcase_startup__ in your __main__.py. Instead of this it is possible to use .pth files. On windows it is working, but this creates at least problems on Android... I dont have tested iOS and macOS yet.
• As suggested in Add --debug option to briefcase run #2147 (comment) pyproject.toml is probably not in the right place for the configuration. @freakboy3742 suggested to add user-level configuration. But this is totally out of scope for this PR. So I used the pyproject.toml even if it is not ideal.
• Adding pycharm is probably also possible, but it makes some problems, so it can be added later (External Packages with data_files are not correctly working (eg. with pydevd-pycharm) #2152).
• Enhance the documentation.
• Add unit-tests

Fixes #2147

PR Checklist:

• All new features have been tested
• All new features have been documented
• I have read the CONTRIBUTING.md file
• I will abide by the code of conduct

[freakboy3742]

I know this is marked as a draft - but it was sufficiently exciting that I've taken a quick look - it's really impressive work! I've dropped a couple of questions and notes inline.

The biggest note is about the broad structure of the debugger module itself. Whenever Briefcase adds a feature like this that involves an interface and one or more implemenations, we try to do so in a way that allows a third party to use the same interface to provide their own implementations. For example, there's an interface for defining GUI bootstraps - we provide one for Toga, and for PySide; but third parties can provide their own implementations of that bootstrap.

A similar approach is needed here. This PR is adding 2 implementations - PDB and DebugPy - of a generic "debugger" interface. Those two implementations should be registering against a generic registration interface; and Briefcase itself should be loading possible debuggers from that interface. There will only be 2 for now; we might add others into Briefcase's codebase in future (like a PyCharm implementation); but any third party should be able to write their own.

This will likely also lead to some simplification of code - the debugger.py module currently has a bunch of "if PDB" type logic - that's really "call the X method on the debugger interface". That helps to isolate the PDB logic into a PDB module; and makes the debugger itself easier to write, because you can have a "dummy" debugger that exists for testing purposes.

Some other high level notes, based on comments you've left in the PR, and the PR description:

• Resolving the Android issue, and moving to a .pth based implementation will likely be a pre-requisite for merging this. If you can elaborate on what the problem is, we might be able to work out what is going on - it's possible it might be an edge case in site handling that we're not handling correctly for the Android case.
• Don't worry about physical iPhone testing - if it works on simulators, that's more than enough to warrant merging; and I'm happy to take on any additional testing needed to verify something works on iOS devices.
• Regarding configuration - I agree that this PR doesn't need to solve the "user-level configuration" problem. However, I think the better "workaround" solution in this case is command line options. If we add this to pyproject.toml, it's difficult to walk that back later; but regardless of whether there is a user-level configuration mechanism, there will be a need to manually handle command line overrides to that configuration.

[freakboy3742]

I suspect what you're looking for is subprocess.check_output rather than subprocess.run. That will capture all the output of the command that is executed, returning the command output so it can be parsed/handled (if needed), but only surfacing the output to the user if an error is raised by the command.

[freakboy3742]

Two notes here:

1. There's a typo on startup_module
2. I'd be inclined to simplify this to just __briefcase__.
3. Although you're using it for debug content here, it could also be used for a remote control testing feature that we're potentially looking at adding. To that end, the code to install/remove the debugger bootstrap should probably be on base.py as a generic capability that an app can invoke under different circumastances.

[mhsmith]

I'd be inclined to simplify this to just __briefcase__.

Double-underscore names are reserved for the Python language, so _briefcase would be better.

[freakboy3742]

Good point - and it would also match the broad pattern of .pth files that are installed by venv etc as part of "environment setup."

[timrid]

To the problem with .pth files on Android: When processing the .pth file I get the following error before the main module is even started. On windows executing the startup code via .pth is no problem. I guess that java.chaquopy.import_override modifies some behavior, so that _socket is not loaded when executing my startup code... @mhsmith Do you have an idea?

...
W/python.stderr: Error processing line 1 of /data/data/com.example.helloworld/files/chaquopy/AssetFinder/app/__briefcase_startup__.pth:
W/python.stderr: 
W/python.stderr:   Traceback (most recent call last):
W/python.stderr:     File "<frozen site>", line 206, in addpackage
W/python.stderr:     File "<string>", line 1, in <module>
W/python.stderr:     File "import.pxi", line 60, in java.chaquopy.import_override
W/python.stderr:     File "/data/data/com.example.helloworld/files/chaquopy/AssetFinder/app/__briefcase_startup__.py", line 4, in <module>
W/python.stderr:       import socket
W/python.stderr:     File "stdlib/socket.py", line 52, in <module>
W/python.stderr:   ModuleNotFoundError: No module named '_socket'
W/python.stderr:
W/python.stderr: Remainder of file ignored
D/MainActivity: Running main module helloworld
...

[mhsmith]

Thanks, this is indeed a bug in the .pth initialization order on Android (chaquo/chaquopy#1338).

Until this is fixed in Chaquopy, the only workaround I can think of is to import the startup module manually in the app code. So the module could catch the ModuleNotFoundError, and raise its own exception giving instructions for how to do this.