Add Language Server Protocol for GDScript #29780

Geequlim · 2019-06-14T14:55:05Z

It basicly works but not finished yet.

Add a plugin for godot editor to provide a LSP Server for GDScript language.

More features would be added in the future by @ankitpriyarup who's the selected student of this slot of the GSoC this year.

This PR contains a commit of #29626 for the code completion usage.

I submit the PR to master. Once it was meged @ankitpriyarup could add other features by submit other PRs based on this to master branch.

UPDATE: 2019-06-30

The LSP implementation is works correctly both the server and client side.

Current supported features:

document symbols
workspace symbols
diagnostics
completion
completion resolve
hover
definition
Smart resolve mode: to resolve symbols for dynamic typed variables to seed up coding.
Script API dump

Planning features:

signatureHelp
documentLink

Testers are wellcomed. Here is the client for Visual Studio Code extension
godot-tools-0.9.0.zip

groud · 2019-06-14T16:22:16Z

core/script_language.h

@@ -200,6 +200,34 @@ class ScriptInstance {
 	virtual ~ScriptInstance();
 };

+struct ScriptCodeCompletionOption {
+	enum Kind {


As for you other PR, "Kind" is not the correct word here but "Type".

I'm not really good at English.
I just followed the LSP CompletionItem https://code.visualstudio.com/api/references/vscode-api#CompletionItem

Geequlim · 2019-06-16T06:08:00Z

Note for reviewer:
This PR is ready to review but not ready to be merged before #29781.
It contains the commit of #29626 so it need to be rebased after #29626 got merged.

Jason0214

Just did a simple review based on the code style stuff. I have tested this branch and functionally works well.

Jason0214 · 2019-06-17T04:30:33Z

modules/gdscript/language_server/gdscript_extend_parser.cpp

+		lsp::Position pos;
+		int line = get_error_line() - 1;
+		const String &line_text = get_lines()[line];
+		pos.line = line;


This error position resolving code is so similar with the warning position resolving one below, maybe we can make a static function to do the work to convert gdscript position to lsp position? And the document symbol position resolving is similar as well

That sounds good.

Jason0214 · 2019-06-17T04:30:45Z

modules/gdscript/language_server/gdscript_extend_parser.cpp

+		lsp::Position pos;
+		int line = warning.line - 1;
+		const String &line_text = get_lines()[line];
+		pos.line = line;


See comment above

Jason0214 · 2019-06-17T04:31:00Z

modules/gdscript/language_server/gdscript_extend_parser.cpp

+		symbol.detail = m.data_type.to_string();
+		symbol.deprecated = false;
+		const int line = m.line - 1;
+		symbol.range.start.line = line;


See comment above

Jason0214 · 2019-06-17T04:31:32Z

modules/gdscript/language_server/gdscript_extend_parser.cpp

+		symbol.kind = lsp::SymbolKind::Event;
+		symbol.deprecated = false;
+		const int line = signal.line - 1;
+		symbol.range.start.line = line;


See comment above

Jason0214 · 2019-06-17T04:31:39Z

modules/gdscript/language_server/gdscript_extend_parser.cpp

+		symbol.kind = lsp::SymbolKind::Constant;
+		symbol.deprecated = false;
+		const int line = E->get().expression->line - 1;
+		symbol.range.start.line = line;


See comment above

Jason0214 · 2019-06-17T04:46:38Z

modules/gdscript/language_server/gdscript_language_protocol.h

+#include "modules/websocket/websocket_peer.h"
+#include "modules/websocket/websocket_server.h"
+
+class GDScriptLanguageProtocol : public JSONRPC {


I think Protocol is not a good name for this class from the perspective of English, I would name it like GDScriptLanguageServerImplementation (definitely too long.. I am really bad at giving variable name🙃

I think the best name would be GDScriptLanguageServer, the Implementation isn't necessary, alternatively GDScriptLSP

Jason0214 · 2019-06-17T04:48:17Z

modules/gdscript/language_server/gdscript_language_protocol.cpp

+}
+
+void GDScriptLanguageProtocol::on_client_disconnected(int p_id, bool p_was_clean_close) {
+	clients.erase(p_id);


Should we close the WebSocketPeer before removing it from client list?

Jason0214 · 2019-06-17T04:52:00Z

modules/gdscript/language_server/gdscript_language_protocol.cpp

+	}
+}
+
+String GDScriptLanguageProtocol::format_output(const String &p_text) {


How about rename it as packup_output?

Jason0214 · 2019-06-17T04:56:47Z

modules/gdscript/language_server/lsp.hpp

@@ -0,0 +1,1293 @@
+/*************************************************************************/


Is this specification header file comes from https://github.com/Microsoft/language-server-protocol/blob/gh-pages/specification.md, if so, should we make it a thirdparty?

Also we may need to rewrite non-static member initializer to constructor as Godot not support c++11 yet

I don't want the lsp.hpp contains so much constructors I'd like remove the default value assignments.

The lsp.hpp was written in godot code by myself. Some of the ducumentations are coped fromhttps://github.com/Microsoft/language-server-protocol/blob/gh-pages/specification.md. So I don't want to add more thirdparty files and messup the build scripts.

Jason0214 · 2019-06-17T05:01:01Z

modules/jsonrpc/jsonrpc.cpp

+	BIND_ENUM_CONSTANT(InternalError)
+}
+
+Dictionary JSONRPC::make_response_error(int p_code, const String &p_message, const Variant &p_id) const {


I would make function can be rewrite to a class like JSONRPC_Error_Response, and are the dict keys are made as class member and has a member function to convert it a dict or directly serialize it. Anyway just some subjective opinions

akien-mga · 2019-07-04T10:09:35Z

Should be rebased now that #29626 is merged.

Geequlim · 2019-07-04T10:39:26Z

@akien-mga Removed the first commit and rebased to master branch.
I'm not sure the commits are really correct.

henriquelalves · 2019-08-11T04:41:20Z

I'm testing this PR and it's overall working pretty well, but I'm having a weird crash using auto-complete when writing a "_init" constructor. To reproduce, just start writing "_init" on a newly created script and use auto-complete; it will write an extra "(" at the end (not sure if it's relevant for the crash), than state a "Expected '(' after identifier (syntax: 'func ([arguments]):'" error on VSCode, and saving the file will crash the engine. Godot will return this error on console: https://gist.github.com/henriquelalves/594746c6df44540567552eb7b8294c09.

Also, I'm not sure if it was supposed to be working, but contextual node auto-completion (e.g. "get_node([..]" is only working for singletones, and F2 (renaming a symbol) doesn't work (apparently it's a Language Server method that has yet to be implemented, but I wasn't sure).

I'm going to test this PR a lot and I would love to help as much as I can with feedback, but is here the most appropriate channel? The branch this PR comes from doesn't have the "Issues" tab enabled, and I'm unsure if those are errors already being taken in consideration.

Implement hover Implement completion documentation resolve Implement hover documentation Implement go to definition

The smart resolvaion can guess most symbols but it might be slow so disabled by default users can turn on it in the editor setting

Improved uri and workspace path translatation on windows platform. The smart resolvation is much faster than builtin's in the server side. The smart resolve mode is still disabled as default as the clients might be slow with a planty of completion items.

Only level one inner classes would be resolved currently but it sould cover most real world use case Improve documation parseing for const values Improve documation format for native symbols

Drop test initialize message sent to client Remove unused code property for the parser class

Expose GDScriptLanguageProtocol singleton and classes for editor plugins (Not visiable in class tree) Fix minor bug in symbol resolve

This action will show help for target symbol in godot editor and bring the godot editor window to foreground Improved markdown documentation for symbols.

Geequlim · 2019-08-11T06:22:04Z

@henriquelalves Thanks for your testing.
The issue with virtual method completion is fixed now, you need update the code and recompile the engine.

Also, I'm not sure if it was supposed to be working, but contextual node auto-completion (e.g. "get_node([..]" is only working for singletones

The LSP server doesn't resolve node pathes as I think I not really neccessary because we code in external text editor with LSP client and it is really often we open diffrent script files in the external editor which is not attached to the opening scene in the godot editor so the node pathes from would be wrong.
I suggest you copy node pathes from the godot editor scene tree dock with Ctrl + C with target node.

F2 (renaming a symbol) doesn't work (apparently it's a Language Server method that has yet to be implemented, but I wasn't sure).

GDScript is a dynamic typed language so it is hard to track references for symbols. So I'm not going to implement this feature as it may cause bugs randomly.
I suggest you use search and replace to do this kind of operations manually to avoid errors.

Any more feedbacks are welcomed. You can pin me here or in IRC.

henriquelalves · 2019-08-14T01:47:23Z

The LSP server doesn't resolve node pathes as I think I not really neccessary because we code in external text editor with LSP client and it is really often we open diffrent script files in the external editor which is not attached to the opening scene in the godot editor so the node pathes from would be wrong.
I suggest you copy node pathes from the godot editor scene tree dock with Ctrl + C with target node.

I still believe that this would be a useful feature for really big scenes, so instead of going back and forth from the editor to Godot, you could just focus on the editor itself - but I understand it's not necessary to make the feature go live.

The issue with virtual method completion is fixed now, you need update the code and recompile the engine.

I just recompiled with the last commit and it's still happening tho! But after I autocomplete any virtual method, it now finishes with a "()" (e.g. "func _input(event: InputEvent) -> void:()") and the editor crashes in the same way as before.

I'll keep testing as much as I can, as I'm already having a lot of fun coding in VSCode again! I just missed my plugins so much.

Jason0214 · 2019-08-14T04:58:14Z

The LSP server doesn't resolve node pathes as I think I not really neccessary because we code in external text editor with LSP client and it is really often we open diffrent script files in the external editor which is not attached to the opening scene in the godot editor so the node pathes from would be wrong.

Probably we can make this feature dispatchable at runtime, for example, let the plugin parsing some .tscn file in the workspace, if it can build a tree(subtree) of the script file, we can make use of it, otherwise we just don't care about node path. It could be very useful.
@ankitpriyarup has a implementation for renaming, it's actually promising, but also encounter the issue of external reference through node path.

@henriquelalves thanks for the testing! It is glad to see people interested in this work😄

@Geequlim maybe we should push it a little bit to get more reviews on this PR. I will also tell @ankitpriyarup to merge his work into this branch (he has a doc support, partly working renaming and some fixes to autocompletion).

vnen · 2019-08-26T13:14:05Z

modules/gdscript/language_server/gdscript_extend_parser.cpp

+
+String ExtendGDScriptParser::get_text_for_completion(const lsp::Position &p_cursor) const {
+
+	String longthing;


The StringBuilder class might be more suitable for this kind of string concatenation.

vnen · 2019-08-26T13:20:29Z

modules/gdscript/language_server/gdscript_workspace.cpp

+
+void GDScriptWorkspace::list_script_files(const String &p_root_dir, List<String> &r_files) {
+	Error err;
+	DirAccessRef dir = DirAccess::open(p_root_dir, &err);


Traversing the file system might be really slow on large project. I think it would be better to get the EditorFileSystem cache.

This is running in a seperated thread so I think it would be ok for performance.
I'm not sure the EditorFileSystem cache is the latest version of the script code.

vnen · 2019-08-26T13:49:04Z

Took a glance at the code and it looks mostly okay, just pointed out a couple of things that I commented.

Since this is mostly new code I believe it's okay to be merged, as it shouldn't affect any current feature. We can fix potential issues with LSP itself later on.

Schroedi · 2019-08-28T10:38:22Z

Thanks for tackling this important task! When it works, it is fantastic. It sometimes seems to crash though and has some minor annoyances. As the bugs in the vscode client repository are disabled, I post them here as well.

Symptoms/Problems

Cyclic references disable completions

Sometimes godot does not detect the cyclic reference or the message is only shown later. It is obvious when the error message is there, but this is not always the case. No completion will be shown afterward. In I guess any file that is affected? The message is only shown in one file for me in a larger project, but the completion seems to be broken everywhere. In the attached sample, the completion works in unaffected files.
LspTest.zip

Sometimes godot hangs while closing

I am unable to reproduce it reliably. It seems to happen when the lsp crashed. I cannot reproduce it in the minimal sample.

Syntax highlighting is for CamelCase variables and functions is strange

Starting godot from vscode fails on Linux

In godot-tools.ts:78 I had to replace windows by linux:
let shell = vscode.workspace.getConfiguration("terminal.integrated.shell").get("linux", "");
And explicitly set terminal.integrated.shell.linux to a value (bash and zsh both worked).
Otherwise, godot open workspace in editor and godot start project would fail.
Without the modification I get this error from code --verbose when clicking on the open editor message in the failed to connect popup.

[1868:0828/122706.909802:INFO:CONSOLE(240)] "%c[Extension Host] %crejected promise not handled within 1 second: TypeError: Cannot read property 'endsWith' of null %c(at t.onMessage.process.on.t.catch.t (/usr/lib/code/out/vs/workbench/services/extensions/node/extensionHostProcess.js:759:23))", source: file:///usr/lib/code/out/vs/workbench/workbench.main.js (240)
[1868:0828/122706.910095:INFO:CONSOLE(240)] "%c[Extension Host] %cstack trace: TypeError: Cannot read property 'endsWith' of null
	at escape_command (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:60:31)
	at run_godot (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:70:34)
	at run_editor.Promise (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:96:17)
	at new Promise (<anonymous>)
	at GodotTools.run_editor (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:55:16)
	at open_workspace_with_editor.Promise (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:45:22)
	at new Promise (<anonymous>)
	at GodotTools.open_workspace_with_editor (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:38:16)
	at retry_connect_client.vscode.window.showErrorMessage.then.item (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:144:22) %c(at t.onMessage.process.on.t.catch.t (/usr/lib/code/out/vs/workbench/services/extensions/node/extensionHostProcess.js:759:99))", source: file:///usr/lib/code/out/vs/workbench/workbench.main.js (240)
[1868:0828/122706.910784:INFO:CONSOLE(3267)] "[[object Object]]Cannot read property 'endsWith' of null", source: file:///usr/lib/code/out/vs/workbench/workbench.main.js (3267)
[1868:0828/122706.910815:INFO:CONSOLE(3267)] "TypeError: Cannot read property 'endsWith' of null
	at escape_command (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:60:31)
	at run_godot (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:70:34)
	at run_editor.Promise (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:96:17)
	at new Promise (<anonymous>)
	at GodotTools.run_editor (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:55:16)
	at open_workspace_with_editor.Promise (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:45:22)
	at new Promise (<anonymous>)
	at GodotTools.open_workspace_with_editor (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:38:16)
	at retry_connect_client.vscode.window.showErrorMessage.then.item (/home/ascadian/.vscode-oss/extensions/geequlim.godot-tools-0.9.0/out/godot-tools.js:144:22)", source: file:///usr/lib/code/out/vs/workbench/workbench.main.js (3267)

This issue seems similar:
microsoft/vscode-maven#337
With this fix:
https://github.com/microsoft/vscode-maven/pull/381/files

Starting the project closes the editor

When I start the editor via the lsp dialog stating that godot is not running and I start the game from vscode afterward, the editor is closed.

My configuration

I compiled godot from this commit:
ankitpriyarup@1491805
My VS Code is:

Version: 1.36.1
Commit: 2213894ea0415ee8c85c5eea0d0ff81ecc191529
Date: 2019-07-22T07:38:21.043Z
Electron: 4.2.8
Chrome: 69.0.3497.128
Node.js: 10.11.0
V8: 6.9.427.31-electron.0
OS: Linux x64 5.1.15-1-surface

Extension version:
ankitpriyarup/godot-tools@46ba5a1

akien-mga · 2019-08-28T11:04:24Z

modules/gdscript/language_server/gdscript_language_protocol.h

+#include "lsp.hpp"
+#include "modules/jsonrpc/jsonrpc.h"
+#include "modules/websocket/websocket_peer.h"
+#include "modules/websocket/websocket_server.h"


Note that this will fail building if either the jsonrpc or websocket modules are disabled. We don't have a good system to define module inter-dependencies, but this should be addressed eventually.

Especially since building Godot without websocket but with gdscript might be a valid use case, so it should be made possible to build gdscript without the LSP in this case.

Solved by #31741.

akien-mga · 2019-08-28T11:05:35Z

I'll merge this PR as is so that @ankitpriyarup can submit their additional (re)work.

We should then review if @ankitpriyarup's work solves the comments which have been raised in this PR or not, and if not, we should ensure that all of them are handled further on (and marked as resolved when they are).

akien-mga · 2019-08-28T11:06:59Z

Thanks a lot!

Follow-up to godotengine#29780.

Geequlim · 2019-08-28T14:10:29Z

@Schroedi Thanks for your testing.
I tested your project with the master LSP server the code completion doesn't work with s1.gd and s2.gd which contain cyclic reference errors. But works for s3.gd which does not contains errors.

The bug about shell should be fixed in the coming days.

Thanks for your test. You can report related issuses here godotengine/godot-vscode-plugin

Follow-up to godotengine#29780.

Geequlim requested review from bojidar-bg, karroffel and vnen as code owners June 14, 2019 14:55

Faless mentioned this pull request Jun 14, 2019

Allow setting websocket buffers sizes internally. #29781

Merged

groud reviewed Jun 14, 2019

View reviewed changes

akien-mga mentioned this pull request Jun 15, 2019

Compilation warnings not reported by CI jobs #29033

Closed

2 tasks

akien-mga added feature proposal topic:editor topic:gdscript labels Jun 15, 2019

akien-mga added this to the 3.2 milestone Jun 15, 2019

Geequlim changed the title ~~[WIP] Add Language Server Protocol for GDScript~~ Add Language Server Protocol for GDScript Jun 16, 2019

Jason0214 reviewed Jun 17, 2019

View reviewed changes

Geequlim mentioned this pull request Jul 10, 2019

Cannot debug godot with gdb/lldb #30464

Closed

Calinou mentioned this pull request Jul 16, 2019

Autocompletion Service Support for external text editors #2245

Closed

Geequlim added 10 commits August 11, 2019 13:30

Add JSONRPC-2.0 implementation as a module

61ed6ef

Add GDScript Language Protocol plugin

f58560a

Add a symbol pool to cache all native symbols and workspackes symbols.

37aafaa

Implement hover Implement completion documentation resolve Implement hover documentation Implement go to definition

Add optional smart resolve sulotion

fa6d6a3

The smart resolvaion can guess most symbols but it might be slow so disabled by default users can turn on it in the editor setting

Improve symbol resolve for inner classes

b2f0231

Only level one inner classes would be resolved currently but it sould cover most real world use case Improve documation parseing for const values Improve documation format for native symbols

Check client workspace directory is valid

9618b0c

Drop test initialize message sent to client Remove unused code property for the parser class

Add generate script api to dictionary support

666ed89

Expose GDScriptLanguageProtocol singleton and classes for editor plugins (Not visiable in class tree) Fix minor bug in symbol resolve

Add optional goto definition support for native symbols

72d11cd

This action will show help for target symbol in godot editor and bring the godot editor window to foreground Improved markdown documentation for symbols.

Improve code compeletion for virtual methods with signatures

e5b91a1

vnen reviewed Aug 26, 2019

View reviewed changes

akien-mga reviewed Aug 28, 2019

View reviewed changes

akien-mga merged commit 46ad603 into godotengine:master Aug 28, 2019

akien-mga mentioned this pull request Aug 28, 2019

GDScript: Disable LSP if either jsonrpc or websocket are disabled #31741

Merged

akien-mga added a commit to akien-mga/godot that referenced this pull request Aug 28, 2019

GDScript: Disable LSP if either jsonrpc or websocket are disabled

b994124

Follow-up to godotengine#29780.

Geequlim deleted the gdscript-lsp branch October 3, 2019 05:56

pchasco pushed a commit to pchasco/godot that referenced this pull request Oct 23, 2019

GDScript: Disable LSP if either jsonrpc or websocket are disabled

862a76e

Follow-up to godotengine#29780.

howdoicomputer mentioned this pull request Jan 29, 2020

Add support for GDScript emacs-lsp/lsp-mode#1361

Closed

atomicptr mentioned this pull request Feb 18, 2020

Fix/Add new LSP support exigow/intellij-gdscript#43

Open

rushsteve1 mentioned this pull request Jul 14, 2020

Godot Engine Support joaotavora/eglot#511

Closed

Faless mentioned this pull request Mar 22, 2021

Implement DAP-compliant middleware to interact with Editor Debugger godotengine/godot-proposals#2494

Closed

Calinou mentioned this pull request Jul 9, 2023

Documentation unclear. Can't find "Enable Smart Resolve" godotengine/godot-vscode-plugin#482

Closed

akien-mga mentioned this pull request Sep 13, 2024

LSP: Omit some values based on specification version 3.17 #96725

Merged

		@@ -0,0 +1,1293 @@
		/*************************************************************************/


		String ExtendGDScriptParser::get_text_for_completion(const lsp::Position &p_cursor) const {

		String longthing;

Add Language Server Protocol for GDScript #29780

Add Language Server Protocol for GDScript #29780

Conversation

Geequlim commented Jun 14, 2019 • edited Loading

UPDATE: 2019-06-30

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Geequlim commented Jun 16, 2019

Jason0214 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Geequlim Jun 18, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akien-mga commented Jul 4, 2019

Geequlim commented Jul 4, 2019

henriquelalves commented Aug 11, 2019

Geequlim commented Aug 11, 2019

henriquelalves commented Aug 14, 2019

Jason0214 commented Aug 14, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Geequlim Aug 27, 2019 • edited Loading

Choose a reason for hiding this comment

vnen commented Aug 26, 2019

Schroedi commented Aug 28, 2019

Symptoms/Problems

Cyclic references disable completions

Sometimes godot hangs while closing

Syntax highlighting is for CamelCase variables and functions is strange

Starting godot from vscode fails on Linux

Starting the project closes the editor

My configuration

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akien-mga commented Aug 28, 2019

akien-mga commented Aug 28, 2019

Geequlim commented Aug 28, 2019

Geequlim commented Jun 14, 2019 •

edited

Loading

Geequlim Jun 18, 2019 •

edited

Loading

Geequlim Aug 27, 2019 •

edited

Loading