Improve performance for when queue is large in size

[levibostian]

From my experience with using multiple iterations of Wendy, I have noticed that once the queue size gets to around 800+ tasks in the queue, performance of the code really matters.

I have not profiled the Wendy code base in a while and therefore, I am not able to pinpoint lower performance points at this time. However, I would love to have a goal with Wendy where each component (reading/writing tasks, running tasks) can support tens of thousands of tasks. By making Wendy have a very low overhead, it can make a greater impact on battery life, for example.

Tasks

• Write tests for Wendy to run against a large queue.
• Run tests, with profiling, to get an idea of performance today.
• .....? From these results, we will decide what we can do to improve. Without data, it's hard to say.

[levibostian]

Resource: https://github.com/dfed/CacheAdvance, could give some inspiration for how to make writing to disk more performant.