Improvements to memory handling for WorkerThreadPools

[BrianBHuynh]

Describe the project you are working on

I am working on holding thursday sessions where I teach students in university (juniors and seniors) how to use github, as well as collaborate on a project in godot (https://github.com/BrianBHuynh/GameDev-lesson-team-project) which is sanctioned by the computer science department head for my university.

I am also working on a cute but super early stage vpet / social game (currently in accessibility stages) https://github.com/BrianBHuynh/Mieu-CatChat

I am also a frequent contributor to the Godot docs.

Because of the first and 3rd projects ease of use and documentation is quite important to me.

Describe the problem or limitation you are having in your project

Currently the implementation of the WorkerThreadPool is slightly unintuitive and hard to teach/use, especially from a documentation standpoint as the names of functions in the object can be misleading or uninformative. Currently memory for task is not automatically managed, and each task needs to manually be cleared/acknowledged/merged through a "wait" function.

There was an issue (godotengine/godot#84888, godotengine/godot#84899) posted around a year ago which was resolved by a documentation change due to lack of knowledge on how Worker ThreadPools work at the time, however even though the issue was resolved, I think implementation of threadpools can be more intuitive.

Currently the documentation officially states that

Warning: Every task must be waited for completion using wait_for_task_completion or wait_for_group_task_completion at some point so that any allocated resources inside the task can be cleaned up.

However this can be confusing as you must wait a task even if it is completed for memory to be cleared, which is unintuitive as for the WorkerThreadPool, the wait functions do not solely freeze the thread it is called from, but instead act as the main form of resource clearing as well!

Some issues to show that people do run into problems (before and after the documentation change): godotengine/godot#79069
godotengine/godot#84888
(after)
godotengine/godot#84899
godotengine/godot#99316 (mine)

As can be seen in 84899 there has been complaints about the current solution as having to call waits to clear resources "pretty much eliminate the ease of use of WorkerThreadPool (fire and forget)." and requires some active effort to manage memory and some people would expect task to happen automatically as there is no function which's name implies memory/resource management so a reading of the docs is needed to use this object intended to simplify the process of multi threading.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

I have multiple solutions which may make memory management via threads easier (and which I've gotten some progress into implementing)

1. Add a function which has a simpler use, acting to merely clear resources and return true if the task is completed (Add clear_task_if_completed function to WorkerThreadPool godot#99316)
   OR
2. Roll this functionality into the current is_task_completed and is_group_task_completed functions

Both of these solutions would make it so that there is a way to acknowledge the task, and also clear memory without calling a wait function (which may risk freezing a thread, or the main thread, if implemented badly). Also the second name would have a slightly more intuitive name and use case without the need for documentation reading (and it's existence implies that task need to be resource cleared).

3. Add a auto clear function / bool which would automatically remove the task from memory once it is completed.

By implementing this in the engine, it will make it easier for programmers to manage memory if there is no need for acknowledgement. In the case when the completion of code does not need to be acknowledged (for example in the case of a save system which is both intensive and also should not be waited on).

4. Rename wait to join, as suggested by the main programmer for WorkerThreadPools @RandomShaper which may put it more in line with the function of Joining threads. (brought up as a solution here Add clear_task_if_completed function to WorkerThreadPool godot#99316)

My suggested solution:
I think that a hybrid of the 3 solutions would make the WorkerThreadPools far more intuitive to use. By combining the is_task_completed checks with resource clearing (like in the wait functions) to allow for waitless resource management, adding a method to automatically clear memory for less essential task, and renaming the wait function, the worker threadpool would be far more intuitive to use.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

As the core engine should be kept as lean as possible, I have included the amount of lines AND loops included in each solution in theory (only for singular task, I have not looked at group task yet but they should be similar )

Implementing the 3 suggested solutions is estimated to add anywhere from 25 lines of extra code, to a low of 10 extra lines, although this is only for single task, and assuming similar implementation both group and single task will take anywhere from 20 to 50 extra lines of code.

solution 1 (lines added 27):
godotengine/godot#99316

solution 2 (lines added 6):
replace the current is_task_completed function with the clear_task_if_completed function code in solution 1, this adds only 6 more lines onto the current function that's already implemented.

solution 3 (aprox lines added 19):
Add these lines to the end of the _process_task internal function

if (p_task->auto_complete && p_task->completed && p_task->waiting_pool == 0 && p_task->waiting_user == 0) {
	tasks.erase(p_task->self);
	task_allocator.free(p_task);
}

Add this function

bool WorkerThreadPool::auto_clear_task(TaskID p_task_id) {
MutexLock task_lock(task_mutex);
Task **taskp = tasks.getptr(p_task_id);
if (!taskp) {
ERR_FAIL_V_MSG(false, "Invalid Task ID"); // Invalid task
}
Task *task = *taskp;
if (task->completed) {
if (task->waiting_pool == 0 && task->waiting_user == 0) {
tasks.erase(p_task_id);
task_allocator.free(task);
}
return true;
}
task->auto_complete = true;
return task->auto_complete;
}

NOTE: 14 of those lines for solution 3 can be removed IF willing to add a argument into the add task function, although this more intensive solution will work with all current code without breaking it.

Solution 4 (0 lines added): Just renaming function from wait to join which may be more accurate.

If this enhancement will not be used often, can it be worked around with a few lines of script?

This enhancement can be worked around in a few lines of code, however the current workaround can be clunky and hard to simply write/explain in the docs to someone without coding knowledge, and the usage of WorkerThreadPools is ideally to simplify the multithreading process for people not familiar and or unwilling with implementing it themselves with the Threads class.

Workaround for checking to see if a task is completed, and then clearing resources:
If is_task_completed(tid):
wait_for_task_completion(tid)

Note how if you want to prevent memory leaks, and no other code relies on acknowledging that your task is finished you must check to see if a task is completed, and then WAIT on said task completing (although it's already completed).

You also need to keep an array or list of active task as well, as task management is not automatically done, and there is no way to get a list of all the currently monitored tasks.

Is there a reason why this should be core and not an add-on in the asset library?

In worse case this could expand the core by ~50 lines in return for easier use of the worker threadpool and overall some performance improvements all around when using the worker threadpool. (Allows automatic memory management, or a way of managing memory without freezing the thread it is called from (which also calls 2 functions instead of 1)).

[AlyceOsbourne]

I always found the phrasing rather unintuitive, and always thought it should be join, much like Python. It's not too much work to wrap into a static utility function, but as you say, it's a bit hard to tell from the documentation that you should probably consider to do that.

[BrianBHuynh]

Yeah that was one of the main reasons I first started looking into a solution, it's a little odd that the wait functions aren't purely just wait functions as their names would suggest but instead also the main memory management method

[Bitlytic]

I do agree that there should be some way to just auto free resources once the task is done, as it would be nice to just send this off and forget about it. I don't think it should be renamed to join though, since that's more of a convention of older languages whereas wait makes it clearer to newer people that you are actively waiting for the task to finish.

[RandomShaper]

I'd like to see a real-world case where manually awaiting the tasks is so inconvenient. Moreover, where you don't care about the task being finished or not, if that's the case.

That would help shaping any potential fire-and-forget additions to the API. The current API is built around the idea of using tasks in the pool the same way you'd use threads, that is, being in control of what you submitted and its lifetime. That's why "join" would fit the picture.

[BrianBHuynh]

As one of the goals is to eventually run as many things as possible on the WorkerThreadPool to take advantage of the performance benefits, and short lived tasks are ideal for usage in the WorkerThreadPool, I’ve identified the kinds of functions (in accordance to the docs, as well as some real world testing and peer review) that can easily be offloaded onto the WorkerThreadPool as long as certain conditions are met. By having a auto clear / resolve system in place however I believe that the barrier of entry for a developer to take advantage of multithreading can be lowered to achievable levels even for the inexperienced.

The conditions which are needed for a function to be multithreaded without any resolution outside of the task (fire and forget) are that a task must:

Not need a return, although this is also true for task in the current system as well
Not have a need for code in the main (or another thread) to be finished before it can be completed, and even then said code can be made sequentially/in line inside of the task at the risk of potential performance losses from the task no longer using multiple threads (but being performed on a separate thread still).
The other code that depends on the task being completed cannot be waiting on a group task if you want to fire and forget it, unless you move the entire group task to a single thread (task instead of group task).
As long as these three conditions are true, most (if not all) functions should be able to be offloaded to the WorkerThreadPool as long as some effort is made to make it ThreadSafe (Using deferred calls, mutexes, etc etc)

From the docs for the WorkerThreadPool we have this kind of example code

var task_id = WorkerThreadPool.add_group_task(process_enemy_ai, enemies.size())
# Other code...
WorkerThreadPool.wait_for_group_task_completion(task_id)
# Other code that depends on the enemy AI already being processed.

Since this is a group task, the other code that depends on the enemy AI already being processed would prevent it from being set and forget. It COULD be though if the group task was turned into a normal task (although that has performance downsides compared to using group tasks), or if the Other code that depends on the enemy AI already being processed wasn’t there.
Code that lends itself better to the fire and forget mentality however is nearly any non group task that you’d want to offload off of the main thread (and then deferred if it needs to be made thread safe, ideally at the end due to performance penalties from syncing up to the main thread).

Going back to the previous example, if we made process_enemy_ai into process_enemies_ai (where now all of the enemies are processed on a single thread / normal task) we can get rid of the wait entirely. As all of the code that would have waited can just trigger once the first part of the task is finished.
var task_id = WorkerThreadPool.add_task(process_enemies_ai)
# Other code...
This is of course not ideal for performance compared to using a group task performance wise, however it is preferred over single threading on the main thread, and it is also easier to implement (for someone unfamiliar with multithreading) than the group task, and it also means that most if not all tasks should be able to be fire and forgot in the context of multithreading.

The same is also the case for if process_enemies_ai had no code which relies on all of them being finished as well, in which case the wait function would have been unneeded and the group task can be safely cleared once all the individual tasks were finished.

Now onto real world examples:

Following the save dictionary system of saving. It is fairly trivial to offload most of the overhead to a separate thread. By using mutexes while reading the save dictionary (and or making a copy of the dictionary), the relatively heavy process of saving the file to storage can be pushed to a separate thread without any wait operations being needed. In the case when you would want to display a message telling the user to not shut off the game as it is saving, call_defered and set_defered can be used without ever needing to acknowledge the task’s completion outside of the thread.
For procedural generation, the task of generating a section of the map / a chunk (or whatever you want to call it) can be fairly heavy, the thing about that however is that according to the Thread Safe APIs doc, one may add a child node / scene using call deferred. As using call deferred allows you to add a node to the scene tree (syncing the thread up to the main thread), a explicit wait outside of the task, or acknowledgement outside of the task is not required (being perfect for fire and forget).
Loading regions: Since loading a file from disk can be fairly intensive at times (compared to accessing data already loaded in memory), the task of loading an area can be offloaded to a separate thread and then added like in #2 with a call_defered. The official docs show how a node can be added to another node outside of the scene tree, and then said node can be added to the scene tree. This is a useful example as the docs themselves (although written for Thread and not WorkerThreadPool) would never require the usage of any code after wait_for_task_completion. This can be done for regions of the game as well as…
loading scenes/areas/objects/furniture: As with the above you can also load individual objects or multiple objects in the same way, for example in the docs an enemy was loaded using multithreading and then a defered_call was used to add it to the scene tree.
Changing variables and calling functions: A task can be made to change a variable or call multiple functions and then put onto a separate thread as long as it is used in a thread-safe way. If you need to do a complex calculation which would take alot of resources, or you want to use any of the Global Scope singletons (which are threadsafe), that can be offloaded to another thread (and into the WorkerThreadPool as a task). As long as the task in hand you don’t expect any returns from (like with other multithreaded functions), most things can be offloaded to the WorkerThreadPool as a set and forget, however things which don’t have to sync up to the main thread as often will of course give more performance benefits.

Performance benefits:
There is no signal sent out when a task is finished, this means you have to manually clear the resources for tasks as the docs say, this can be done for tasks by iterating over an array every set interval, checking a task (or a list of task) when certain conditions are met, or through other means. Overall though this causes more overhead, and also causes more unwieldiness than if there was a way to auto clear those tasks in which that is unnecessary.
Warning: Every task must be waited for completion using wait_for_task_completion or wait_for_group_task_completion at some point so that any allocated resources inside the task can be cleaned up.
If we were looking at the code we previously were looking at as examples above, when putting the reliant code in the same task (and thus removing the need for a wait), that reliant code can immediately be called once the original code is finished, which could be useful in some cases, and could lead to more performance as there is little chance of the main thread halting or unnecessary checks.
Auto clear task are cleared immediately after they are finished, which helps prevent memory leaks and the off chance that you may have to iterate over a large array of tasks at one time (in the case where uncleared tasks build up due to them being cleared periodically, potentially concentrating the performance load of clearing out all the tasks).

Usability benefits of auto clear:
If made into its own function, it is optional to use and will not break any current code (or require compatibility methods).
Having it exists in general (as either a function or a bool in add_task) will make it more apparent that memory/resources do need to be cleared (as currently you’d need to pull up the documentation or search up the function in the editor to realize that tasks need to be cleared).
Having an auto clear function allows out of the box multithreading. Since most functions as shown above) can be turned into a multithreaded function as long as they use call_defered and set_defered when necessary, as well as other thread safe practices, this allows people to more easily implement multithreading without having to worry about memory management.
Since some multithreaded functions already don’t require acknowledgement (i.e a wait), and don’t have any code dependent on them being finished (like adding the enemy in thread-safe api docs), they shouldn’t need to have to be manually cleared from memory once finished anyways.
If tasks each have code which is dependent on them, then normally it is easier to put the dependent code within the task itself than create a loop or listener for once they’re finished. If I have 8 different types of functions which all have different dependent code which resolves them, and I put that dependent code behind a wait, I need to create a condition or loop or array for each task type which can bloat up the game’s code more than if I just resolved each task within it’s own task (and then auto cleared it).
BIG benefit for newbies: Since godot is so user friendly, and does try to warn you when you do something that is not thread-safe within a thread (set_thread_safety_checks_enabled) and that is automatically true, having a method to auto manage memory for functions will get rid of one of the common problems which don’t come with a in editor warning, memory leaks. In most cases anyways the memory handling in godot is not a big worry, so giving a easy solution here should help anyone who intends on taking advantage of multithreading in their games.

[BrianBHuynh]

Currently I'm working on adding multiplayer to my game and I have just realized that the send_p2p_packet function listed on godot steam (as well as some other functions listed) would also benefit from the inclusion of auto_clear. They currently send out data through the steamworks api but there is no return or anything else that relies on the function/task being completed at all since it is just sending out data (and as such not interacting with anything that would be thread unsafe). This would work perfectly with the auto_clear function since it never interacts with the scene tree or any resources after being called at all!

For fully self contained functions like this (taken from https://godotsteam.com/tutorials/p2p/):

func send_p2p_packet(this_target: int, packet_data: Dictionary) -> void:
	var send_type: int = Steam.P2P_SEND_RELIABLE
	var channel: int = 0
	var this_data: PackedByteArray
	this_data.append_array(var_to_bytes(packet_data))
	if this_target == 0:
		if Steamlobbies.lobby_members.size() > 1:
			for this_member in Steamlobbies.lobby_members:
				if this_member['steam_id'] != Steamworks.steam_id:
					Steam.sendP2PPacket(this_member['steam_id'], this_data, send_type, channel)
	else:
		Steam.sendP2PPacket(this_target, this_data, send_type, channel)

You can just toss the send_p2p_packet into the worker threadpool and forget about it

I did so here: BrianBHuynh/Mieu-CatChat@3b609c0

I've also added the save_game function into my gdscript implementation of the code from saving your game (more or less) from the docs without any problem as well!

[BrianBHuynh]

I wanted to find more in world examples of when auto_clear would be useful, so I searched up "workerthreadpool language:GDScript" on github, and within the first 21 results, a good 6 of them implemented auto clear in their own way, although most of the time it uses either a async wait, or a loop which goes through an array/dict to clear out task once they're done, and out of these 6 none really benefit from the wait other than for memory management reasons

Auto clear implemented in these repos:
https://github.com/jacob-grahn/platform-racing-4/blob/0f063a9c07cb6162d1c57f0e939cfa08ca6a5a20/client/lib/AsyncWorkerThreadPool.gd#L41
https://github.com/BenjaTK/Gaea/blob/23561ce3ac7b7514f0b68451ba04e997a0ad2485/script_templates/Node/thread_wrapper.gd#L13
https://github.com/the-mirror-gdp/the-mirror/blob/de23a9630a556da25e747e2847b2b7387078e849/mirror-godot-app/gameplay/space_object/heightmap/heightmap.gd#L199

https://github.com/Khaligufzel/Dimensionfall/blob/a1cdf0b0f5ee74db3e772af67edff1df929c6c01/Scripts/Helper/task_manager.gd#L23
https://gist.github.com/mashumafi/fced71eaf2ac3f90c158fd05d21379a3

https://github.com/EIRTeam/Project-Heartbeat/blob/f9196c3c8fd22e77615d866a0db701c7c8949db1/menus/song_list/FilterSongsTask.gd#L81

https://github.com/HotariTobu/gd-data-binding/blob/16ac8f4f30949bd27879dcc68eb2ac2ec7a234e7/addons/gdscript_formatter/gdscript_formatter.gd#L122

Out of the front page 28% of the uses of worker threadpool included some kind of manually done auto clear task which is more resource intensive than just having a in built auto clear task function which shows that there is atleast some demand for such a feature.

[BrianBHuynh]

I just did a little bit more research and found that roughly
24/195 files containing "workerthreadpool" in gdscript (12.3%) include a self made auto clear function
94/195 files that contain workerthreadpool in gdscript include WorkerThreadPool.wait_for (48%)
27/191 files that contain workerthreadpool.add in gdscript do NOT track the TID, meaning they never use wait or any similar functions at all (14%)

This clearly shows that memory is not being managed sufficiently, as the wait function is currently being used in less than half of all files which use workerthreadpool, AND over 10% of all implementations end up creating the auto_clear function for no reason other than to clear memory.

At the same time ontop of that 12.3% that have auto clear, another 14% "set and forget" as they never even track the TID

Here is the list of implementations of auto clear
https://gist.github.com/mashumafi/fced71eaf2ac3f90c158fd05d21379a3
https://github.com/BenjaTK/Gaea/blob/23561ce3ac7b7514f0b68451ba04e997a0ad2485/script_templates/Node/thread_wrapper.gd#L13
https://github.com/BrianBHuynh/Mieu-CatChat/blob/145dc79d8783c5d8e2519fb28029dfbb2dcc6952/current/scripts/globals/autoloads/multithreading.gd#L13
https://github.com/cospan/delaunay-visualizer/blob/1d15aa74ad3f8adc269fb2ebab73305c2494ae35/Autoload/task_manager.gd#L16
https://github.com/EIRTeam/Project-Heartbeat/blob/f9196c3c8fd22e77615d866a0db701c7c8949db1/menus/song_list/FilterSongsTask.gd#L81
https://github.com/elmarhoppieland/DemonCrawl/blob/b725154ac638cd0e12c3ca95d918595dec943379/Resources/AssetManager.gd#L165
https://github.com/HotariTobu/gd-data-binding/blob/16ac8f4f30949bd27879dcc68eb2ac2ec7a234e7/addons/gdscript_formatter/gdscript_formatter.gd#L122
https://github.com/IvanSchroeder/Project-Breachers/blob/1027d3951f6270256d4353fdde8a467e6fc45f97/addons/gaea/renderers/2D/threaded_tilemap_gaea_renderer.gd#L7
https://github.com/jacob-grahn/platform-racing-4/blob/0f063a9c07cb6162d1c57f0e939cfa08ca6a5a20/client/lib/AsyncWorkerThreadPool.gd#L41
https://github.com/Khaligufzel/Dimensionfall/blob/a1cdf0b0f5ee74db3e772af67edff1df929c6c01/Scripts/Helper/task_manager.gd#L23
https://github.com/LinoBigatti/Project-Heartbeat/blob/5ba3beb989011a93f72f152d3d3ef5b5ee958aab/menus/song_list/FilterSongsTask.gd#L80
https://github.com/molarmanful/bited/blob/cb060d49ac41d59decd3d08bd4fa5e17af4d0baf/components/bdf_parser.gd#L78
https://github.com/Monday-nr/GreekBibleVocabularyApp/blob/c53866f55314b844286fecb71e8dd519777ea7dd/Scripts/ResourceHandlerScript.gd#L33
https://github.com/Pet2Ant/Platformer/blob/7a22a5c654f11d240f2974ae543d59360c3df6e9/addons/gdscript_formatter/gdscript_formatter.gd#L134
https://github.com/R4Qabdi/outermension/blob/518f705aa745fefdea67d8d2ce26cc30e742acc8/addons/gaea/others/threaded_chunk_loader_2d.gd#L18
https://github.com/saltern/GearStudio/blob/2a93def4ad510f1530bf81dea5d4e01064970d1e/AUTOLOADS/sprite_import.gd#L70
https://github.com/the-mirror-gdp/the-mirror/blob/de23a9630a556da25e747e2847b2b7387078e849/mirror-godot-app/gameplay/space_object/heightmap/heightmap.gd#L199
https://github.com/uzkbwza/smileygame/blob/48da77f560948dcd4d0046384b5e6febfc1bb1fa/addons/worstconcept-spawnpool/WCSpawnPool.gd#L106
https://github.com/VaibhavRaina/DaaProject/blob/5802b407928ac72e2d3146dc7197d2105234d9f5/addons/gaea/others/threaded_chunk_loader_2d.gd#L14
https://github.com/VaibhavRaina/DaaProject/blob/5802b407928ac72e2d3146dc7197d2105234d9f5/addons/gaea/others/threaded_chunk_loader_3d.gd#L14
https://github.com/VaibhavRaina/DaaProject/blob/5802b407928ac72e2d3146dc7197d2105234d9f5/addons/gaea/renderers/2D/threaded_tilemap_gaea_renderer.gd#L5
https://github.com/VaibhavRaina/DaaProject/blob/5802b407928ac72e2d3146dc7197d2105234d9f5/addons/gaea/renderers/3D/threaded_gridmap_gaea_renderer.gd#L5
https://github.com/ywmaa/My-Touch/blob/636d56e779d391118adb99a6b83a5c2e227d3ad6/task_manager.gd#L18

[BrianBHuynh]

(TD:LR based on all the usages of worker threadpool in godot on github, roughly 52% of people on github are not clearing memory, and atleast 24% of people are just setting and forgetting/fire and forgetting although that's the rough estimate for both based on github users)

[KromulusTheBlue]

When I first discovered the WorkerThreadPool, I understood from the documentation that every add_task method must have a corresponding wait_for_task_completion, but it did not occur to me that failing to include the wait_for_task_completion step would be prevent system resources from being freed after task completion. Now that I'm aware of that potential, I do see that it is mentioned, but I did not fully comprehend it when I first read through the documentation.

Lately I've been working to refactor my code to take advantage of the WorkerThreadPool, and I can definitely imagine a scenario in which I could have added a task without a corresponding wait_for_task_completion method if the main thread method that added the task didn't require any further steps after the task was added.

[radiantgurl]

I feel like this would be a valid solution for a memory leak issue, especially how developers have tried to patch it themselves as said above. I'd love to see this inside Godot.

[cliffy987]

I am a member of the project mentioned at the beginning of the proposal,

As someone who is still studying computer science and learning Godot, many of the programs I write are relatively basic. If I were to use WorkerThreadPools, I would likely not need to access a task once it has been finished (I would mainly use it to run basic functions, like save_game(), in parallel to allow the program to run faster). I think having the tasks be cleared automatically by default would make Godot more accessible to amateur developers like myself.

[RandomShaper]

From the meeting:

• Idea of having user tags assignable to tasks so you can do WorkerThreadPool.await_for_all(tags);.
• Internal auto-clear: completed tasks get freed even if the map still contains an entry; i.e., task_id -> nullptr.
• We also need that the finish() function complains about unclaimed tasks in the high-prio queue. We have to confirm first that the risk of a task being there at that point is true (looks like it).

[clayjohn]

To add to Pedro's comment, we had some concerns about auto-clearing/merging the task in the meeting:

1. It encourages users to treat worker threads as "fire and forget" threads, even if they need to access the results of the work in the future. This is a recipe for hard to track down race conditions
2. It leaves users unable to check for task completion (unless they check soon enough)
3. It creates an inconsistency where if users wait on the task it will either wait and stall, or throw an error because the task is already completed and has been cleaned up automatically.

We agree that the memory leaks noticed by Brian are a problem, but we want to ensure that our solution to the problem isn't worse than the original problem itself. I suspect that Pedro's second and third points should be enough to address the original problem (clearing the task memory without clearing the task_id from the map, and complaining about unfinished tasks on close). Combined, those two items will eliminate the memory leaks as the memory will get cleaned up right away, and they will help educate users about the need to claim claim tasks once they are finished.

The final piece is the usability for cases where you want a truly fire and forget task because you don't care about the result. Pedro's first point might help with those cases. But we might want to also consider something else. Personally, I think Pedro's suggestion would be ideal. We can suggest to users to tag their fire and forget threads with a common tag, then just wait on all of them at once during shutdown to avoid the warning. This approach is nice since it doesn't encourage beginners to create race conditions in their code.

[BrianBHuynh]

I think I might have a potential solution which might work with most if not all of these concerns, but at the same time I have to ask if there is any way to access the results of a threaded task like described in your first point @clayjohn, at the same time is there any point in keeping the task in memory other than something which list if its completed or not? I was looking over the worker threadpool class and it seems like you can give each task a description, however I can't see anything which would allow you to access said description or return values (or other values within the task) which would mean that the description argument is wholly unneeded.

Since there's no way to access said return value / description as it currently is, I think it would be a great idea to turn all task into solely their task ID after being completed in memory like @RandomShaper suggest on his first point, which would mean that the memory leaked from each uncleared task is a lot smaller.

Also as I was thinking, could we potentially have a completely "fire and forget" solution with a somewhat threatening name to dissuade people from using it without knowing what they are doing? For example "Async task" (although I'm not sure if it is an accurate name, it was used by someone previously from the listed commits above). Having a more technical name and a big enough warning should dissuade people from using it without knowing the risks and at least reading up on it first, and at the same time it would help in the cases where someone wouldn't need to access the results or confirm completion in the future.

Sorry for the wall of text, just jotting down my thoughts for now so we can discuss them in the next meeting! I should have a more organized document for thursday.