server: Begin accepting auth header and have it override callback URL values

[thomshutt]

What does this pull request do? Explain your changes. (required)

• Begin accepting a new Livepeer-Transcode-Configuration header in the HTTP Push segment endpoint.
• If present, this header should have the same format as the response from the "HTTP Auth Callback URL".
• If present and the Manifest ID from the path is not referring to an existing Manifest ID, then a call is still made to the callback URL as before (and fails if this call returns an error) but the fields it returns are overridden by the ones in the header
• If present and the Manifest ID in the URL already exists, this field is ignored

Specific updates (required)

• Create a new method for parsing the header and associated unit tests
• Add logic to override response from callback URL in the case outlined above. Unit test for this override behavior.

How did you test each of these updates (required)
✅ Wrote new unit tests
❌ TODO: Manual testing

Does this pull request close any open issues?

• Handle Livepeer-Transcode-Configuration header in HTTP push requests #2332

Checklist:

• Read the contribution guide
• make runs successfully
• All tests in ./test.sh pass
• README and other documentation updated
• Pending changelog updated

[thomshutt]

@yondonfu @iameli Here's my first pass at this, would appreciate you guys taking a quick look (possibly at just the description in this PR) and checking it lines up with the behavior you were expecting

[yondonfu]

The description of the flow sounds right to me, but just had a few comments on these bits:

If present and the Manifest ID from the path is not referring to an existing Manifest ID, then a call is still made to the callback URL as before (and fails if this call returns an error) but the fields it returns are overridden by the ones in the header

• Just confirming that if there is no callback URL that auth will be skipped and Manifest ID will still be created which I believe is also the current behavior?
• I think we had originally discussed returning an error if the profiles configuration in the header do not match the ones returned by the callback URL, but I don't feel strongly about returning an error vs. overriding the profiles with whatever is returned by the callback URL. Seems like either option works as long as there is a way to avoid accepting the profiles in the header if there is a mismatch with what is returned by the callback URL

If present and the Manifest ID in the URL already exists, this field is ignored

This sounds right as we're explicitly not supporting changing the profile configuration mid-stream right now so if the Manifest ID already exists then the profiles originally configured for the stream should continue to be used.

[thomshutt]

Just confirming that if there is no callback URL that auth will be skipped and Manifest ID will still be created which I believe is also the current behavior?

That's correct, for a new Manifest ID in the URL:

• If there's no callback URL and no header then it behaves as before (allow the request, create new Manifest ID)
• If there's no callback URL but there is a header then it acts the same as if it had got a successful callback response in the old flow
• If there's a callback URL and no header then it behaves identically to before

I think we had originally discussed returning an error if the profiles configuration in the header do not match the ones returned by the callback URL, but I don't feel strongly about returning an error vs. overriding the profiles with whatever is returned by the callback URL. Seems like either option works as long as there is a way to avoid accepting the profiles in the header if there is a mismatch with what is returned by the callback URL

I think that rejecting probably does make more sense in terms of simplicity / being able to reason about what's going on for the first iteration of this. I'll update that now.

[thomshutt]

Closing this in favour of a simpler change (no logging refactoring etc.) since a lot has changed on master in the meantime #2357