Investigate file descriptor use with tokio 0.2

[lukesteensen]

As explained in #1142, we want to upgrade our underlying runtime to tokio 0.2 for better performance, more modern interfaces, etc. The tokio-compat runtime gives us an incremental path for that upgrade, so we'd like to use that as a first step.

In some preliminary experiments switching to tokio-compat, @LucioFranco found an issue where the new runtime caused significantly increased use of file descriptors, particularly during test runs. Before we merge any changes to master, we should try to reproduce and measure this effect so we can evaluate its impact both for users and contributors.

Here's a rough plan of attack:

1. In a branch off of master, do the minimal transition to tokio-compat (see Upgrade to tokio v0.2 and std::future::Future #1142) and get tests passing.
2. Compare fd use for cargo test --features docker across master and the new branch.
3. Do some simple manual tests of fd use while running vector normally for short-ish periods of time (e.g. 5-10 mins) to see if there's any evidence of a leak.

Once we have some data to compare, we can evaluate the impact and see how we want to proceed.

[LucioFranco]

And to clarify a few things, I found that when I disabled the tokio 0.2 IO driver that the high fd usage went away. So this would be another area to check.

[LucioFranco]

Another note, one thing I found was if we disabled the io driver here https://github.com/tokio-rs/tokio-compat/blob/master/src/runtime/threadpool/builder.rs#L172 it seemed to fix the issue for me. Leading me to believe this was with the 0.2 io driver. I would suggest cloning tokio-compat locally, and patching it into vector. If you have questions feel free to reach out.

check https://docs.rs/tokio/0.2.11/tokio/runtime/struct.Builder.html#method.enable_io docs and the general tokio 0.2 builder docs.

[MOZGIII]

Solved as part of #1922 via tokio-rs/tokio-compat#28

[binarylogic]

Nice work