Make serde dependency optional

[leighmcculloch]

What

Make serde dependency optional. Also run tests and builds against the serde feature.

Why

The serde dependency shouldn't be included in the dependency tree unless the serde feature is enabled.

[leighmcculloch]

This change includes a TODO to remove the --optional-deps usage once weak dependencies are recognized by cargo-hack (taiki-e/cargo-hack#147) and once the bug with first-class features that have the same name as an optional dependency not being seen as a feature to test is addressed (taiki-e/cargo-hack#153).