Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move back to ChannelMonitor RemoteTxCache #610

Merged
merged 25 commits into from
May 28, 2020

Conversation

ariard
Copy link

@ariard ariard commented Apr 28, 2020

#561 & consorts were moving some remote tx data in OnchainTxHandler, after a lot of back-and-forth on API between ChannelMonitor and OnchainTxHandler, it has been laid out that OnchainTxHandler should be scoped to transaction-finalization and fee-bumping, transaction construction or templating should be provided by ChannelMonitor. Therefore, RemoteTxCache, as channel dynamic data belong to ChannelMonitor.

Copy link
Collaborator

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just looked over the full diff pretty quickly, but this looks pretty nice - not much diff for a lot of functionality :). Some comments, mostly simple docs/naming/etc stuff, but this is looking good :).

@@ -245,12 +247,24 @@ pub trait ChannelKeys : Send+Clone {
/// return value must contain a signature.
fn sign_local_commitment_htlc_transactions<T: secp256k1::Signing + secp256k1::Verification>(&self, local_commitment_tx: &LocalCommitmentTransaction, local_csv: u16, secp_ctx: &Secp256k1<T>) -> Result<Vec<Option<Signature>>, ()>;

/// Signs a justice transaction.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These need real documentation, a number of the parameter names don't appear to be correct, etc. But I'm sure you knew that :)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I knew it. But was expecting better documentation to be part of your key-interface cleanup but it got merged first ;)

@@ -198,7 +198,7 @@ pub(super) fn derive_public_key<T: secp256k1::Signing>(secp_ctx: &Secp256k1<T>,
/// Derives a revocation key from its constituent parts.
/// Note that this is infallible iff we trust that at least one of the two input keys are randomly
/// generated (ie our own).
pub(super) fn derive_private_revocation_key<T: secp256k1::Signing>(secp_ctx: &Secp256k1<T>, per_commitment_secret: &SecretKey, revocation_base_secret: &SecretKey) -> Result<SecretKey, secp256k1::Error> {
pub fn derive_private_revocation_key<T: secp256k1::Signing>(secp_ctx: &Secp256k1<T>, per_commitment_secret: &SecretKey, revocation_base_secret: &SecretKey) -> Result<SecretKey, secp256k1::Error> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs here should likely be improved if we're making it pub.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pub (crate) fn is enough for now

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strongly disagree - you can't easily implement a ChannelKeys without it. Lets just write a comment :).

#[derive(PartialEq)]
pub(super) enum InputDescriptors {
#[derive(PartialEq, Clone, Copy)]
pub(crate) enum InputDescriptors {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be dropping the "s"? This doesn't seem to be plural, which I find confusing.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Planning to move InputDescriptors to better-documented/generic TxTemplate as described in #606

@@ -823,13 +842,10 @@ impl<ChanSigner: ChannelKeys> PartialEq for ChannelMonitor<ChanSigner> {
self.funding_info != other.funding_info ||
self.current_remote_commitment_txid != other.current_remote_commitment_txid ||
self.prev_remote_commitment_txid != other.prev_remote_commitment_txid ||
self.their_htlc_base_key != other.their_htlc_base_key ||
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the new fields?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Damn I new I forgot it at some point but never committed... Corrected

their_htlc_base_key: &PublicKey, their_delayed_payment_base_key: &PublicKey,
their_to_self_delay: u16, funding_redeemscript: Script, channel_value_satoshis: u64,
remote_htlc_base_key: &PublicKey, remote_delayed_payment_base_key: &PublicKey,
on_remote_csv: u16, funding_redeemscript: Script, channel_value_satoshis: u64,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, on_remote_csv definitely makes more sense than their_to_self_delay, but maybe call it remote_tx_to_remote_csv, or so? Characters don't cost us much :)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on_remote_tx_csv ? Imo remote_tx_to_remote_csv doesn't make sense because even if are processing a remote tx we are doing so from the viewpoint of local, this csv matters us as local.

Overall, either is fine but to avoid confusion, it's more important to be consistent on this with every variable names.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with that, though I prefer something that indicates that its the CSV on the output to our counterparty on a remote tx.

@TheBlueMatt
Copy link
Collaborator

Needs rebase on master.

@ariard ariard force-pushed the 2020-04-cache-in-monitor branch 2 times, most recently from 8b8dd01 to c7761cf Compare April 30, 2020 01:25
@ariard
Copy link
Author

ariard commented Apr 30, 2020

Rebased at c7761cf, with more renaming commits for killing their_to_self/our_to_self in ChannelMonitor, confusion by their names was reason of test failure.

@ariard ariard force-pushed the 2020-04-cache-in-monitor branch from c7761cf to d9ab6f6 Compare April 30, 2020 01:42
@codecov
Copy link

codecov bot commented Apr 30, 2020

Codecov Report

Merging #610 into master will increase coverage by 0.03%.
The diff coverage is 95.32%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #610      +/-   ##
==========================================
+ Coverage   91.29%   91.32%   +0.03%     
==========================================
  Files          35       35              
  Lines       20776    20952     +176     
==========================================
+ Hits        18967    19134     +167     
- Misses       1809     1818       +9     
Impacted Files Coverage Δ
lightning/src/ln/onchaintx.rs 93.94% <90.32%> (-0.91%) ⬇️
lightning/src/chain/keysinterface.rs 94.86% <91.17%> (-2.11%) ⬇️
lightning/src/ln/channelmonitor.rs 95.71% <98.48%> (+0.20%) ⬆️
lightning/src/ln/chan_utils.rs 97.17% <100.00%> (ø)
lightning/src/ln/channel.rs 86.63% <100.00%> (+<0.01%) ⬆️
lightning/src/ln/functional_tests.rs 97.43% <100.00%> (+0.11%) ⬆️
lightning/src/util/enforcing_trait_impls.rs 100.00% <100.00%> (ø)
lightning/src/util/macro_logger.rs 89.28% <100.00%> (ø)
lightning/src/util/test_utils.rs 78.21% <100.00%> (+0.37%) ⬆️
lightning/src/ln/reorg_tests.rs 98.94% <0.00%> (-0.04%) ⬇️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d4ad57b...81e358c. Read the comment docs.

@ariard ariard force-pushed the 2020-04-cache-in-monitor branch from d9ab6f6 to 9eb0620 Compare May 1, 2020 02:39
@ariard
Copy link
Author

ariard commented May 1, 2020

Updated 9eb0620, see IRC conv, I would prefer to move forward with this and address witnessScript dry-up in a follow-up. This PR is already big enough and touch to some critical path.

@TheBlueMatt
Copy link
Collaborator

see IRC conv, I would prefer to move forward with this and address witnessScript dry-up in a follow-up. This PR is already big enough

Not sure which IRC conversation this is in reference to, but I don't get why - such a change does not change the size of this PR at all and its a comment specific to the new code, not something related to cleaning up old code, so I don't think this is a reasonable excuse.

@ariard
Copy link
Author

ariard commented May 4, 2020

Not sure which IRC conversation this is in reference to, but I don't get why - such a change does not change the size of this PR at all and its a comment specific to the new code, not something related to cleaning up old code, so I don't think this is a reasonable excuse.

Okay updated at cf18536, turned out every element was available and didn't have to touch to ChannelMonitor.

@@ -64,8 +65,8 @@ pub enum SpendableOutputDescriptor {
DynamicOutputP2WSH {
/// The outpoint which is spendable
outpoint: OutPoint,
/// The secret key which must be used to sign the spending transaction
key: SecretKey,
/// Per commitment point to derive delayed_payment_key by key holder
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

derive how? Please point the user to ln::chan_utils::derive_private_key, make it pub, and describe exactly which inputs should be passed to it. Based on the docs here I don't know how to sign for this transaction, whereas previously I easily could. Note that the docs also, above, say "The private key which should be used to sign the transaction is provided" which is also wrong now.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, also made public get_revokable_redeemscript, you do want to regenerate script for verification.

@@ -4088,7 +4088,7 @@ fn test_manager_serialize_deserialize_inconsistent_monitor() {
}

macro_rules! check_spendable_outputs {
($node: expr, $der_idx: expr) => {
($node: expr, $der_idx: expr, $chan_signer: expr) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmm...This is not ok - the point of this check is to check that the user can sign for outputs we notify them of, not that we can sign for outputs we notify them of. Making them dig into private fields of the ChannelMonitor to get the chan_signer back and then call methods on it is not ok.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay changed to TestKeysInterface::get_descriptor_base_keys()

///
/// Per_commitment_point is the dynamic point corresponding to the channel state
/// detected onchain.
fn sign_payment_transaction<T: secp256k1::Signing>(&self, spend_tx: &Transaction, input: usize, amount: u64, per_commitment_point: &PublicKey, network: Network, secp_ctx: &Secp256k1<T>) -> Result<Signature, ()>;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused - we never call this outside of tests, why is it in a public interface?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

/// Per_commitment_point is the dynamic point corresponding to the channel state
/// detected onchain. It has been generated by us and is used to derive our
/// delayed_payment_key.
fn sign_delayed_transaction<T: secp256k1::Signing + secp256k1::Verification>(&self, spend_tx: &Transaction, input: usize, on_local_tx_csv: u16, amount: u64, per_commitment_point: &PublicKey, secp_ctx: &Secp256k1<T>) -> Result<Signature, ()>;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just in general I think this is the wrong way to go about it - removing the secret key is great, but there's no reason to require that a ChannelKeys has this function - the user can chose to sign the delayed transaction any way they want and we never actually need it. We should obviously provide a way to get at the relevant keys from InMemoryChannelKeys so that the user can sign correctly, but if its eg a hardware wallet, its not our job to tell them how to implement the API.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes should add somewhere keys don't have to reside on same device.

@ariard ariard force-pushed the 2020-04-cache-in-monitor branch from cf18536 to 59e20d4 Compare May 6, 2020 01:43
@ariard ariard force-pushed the 2020-04-cache-in-monitor branch from 59e20d4 to def3da3 Compare May 6, 2020 22:24
/// confirms).
/// <BIP 143 signature> <empty vector> (MINIMALIF standard rule) <provided witnessScript>
///
/// Note that the nSequence field in the spending transaction input must be set
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you chane this? the previous copy "mentions in the input" which you removed, but sequence is per-input. Also note that both the old and the new copy are missing a "to" before "to_self_delay.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I modified back to "spending input", IMO CSV semantic are sometimes unclear to a lot of people so found better to explain that's the input in the spending transaction. Added missing to.

/// These are generally the result of a "revocable" output to us, spendable only by us unless
/// it is an output from us having broadcast an old state (which should never happen).
///
/// WitnessScript may be regenerated by passing the revocation_pubkey, to_self_delay and
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you mention somewhere that these are the things that are given by the users' ChannelKeys if they're using a custom ChannelKeys? I dont understand how a user using KeysManager/InMemoryChannelKeys can get this info, though? AFAICT, using those its entirely impossible to spend funds from such an event.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, added a commit commenting this far better, and from where you should get each piece of the witnessScript. Tell me if you think it's good enough.

@ariard ariard force-pushed the 2020-04-cache-in-monitor branch 2 times, most recently from 71d1faf to 8ad1ff6 Compare May 9, 2020 01:54
@ariard
Copy link
Author

ariard commented May 9, 2020

As discussed offline, updated with a hallmark for StaticOutputRemotePayment/DynamicOutputP2WSH descriptors.

@@ -345,6 +421,7 @@ impl ChannelKeys for InMemoryChannelKeys {
fn htlc_base_key(&self) -> &SecretKey { &self.htlc_base_key }
fn commitment_seed(&self) -> &[u8; 32] { &self.commitment_seed }
fn pubkeys<'a>(&'a self) -> &'a ChannelPublicKeys { &self.local_channel_pubkeys }
fn hallmark(&self) -> &u32 { &self.hallmark }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If its smaller than 2ish u64s, dont use a reference.

Sha256::from_engine(sha).into_inner()
}

fn derive_channel_keys(&self, channel_value_satoshis: u64, hallmark: u32) -> InMemoryChannelKeys {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're going to rely on users being able to regenerate channel keys, we can't use self.unique_start in the generation of them - if you serialize the keys and then read them back from disk they'll be different!

Also, without unique_start here I'm pretty uneasy - in general we try to protect the user from accidentally using the same keys twice on reload even if they never saved the copy after the original use. Thats the point of unique_start - after restarting you'll always get new keys even if you didn't save the old one's channel_child_index counter.

The easy solution to this is to just include unique_start in the "hallmark" - instead of a u32, which doesn't carry much entropy, make it a u64 pair - the first contains the child_index and unique_start_nanos, the second unique_start_secs.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay updated while storing seed, staarting_time_nanos, starting_time_secs in KeysManager to return (chan_id | starting_time_nanos, starting_time_secs) as part of ChannelKeys.

These key state is now part of SpendableOutputDescriptor and therefore can be passed to derive_channel_keys, seed is already required to be well stored between KeyManager serialization.

Let me know if you find names compelling and documentation clear enough for user.

@ariard ariard force-pushed the 2020-04-cache-in-monitor branch from 8ad1ff6 to 42f5077 Compare May 12, 2020 02:02
@ariard ariard force-pushed the 2020-04-cache-in-monitor branch from 42f5077 to e3f9454 Compare May 12, 2020 21:43
@@ -125,15 +151,16 @@ impl Readable for SpendableOutputDescriptor {
}),
1u8 => Ok(SpendableOutputDescriptor::DynamicOutputP2WSH {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somewhat unrelated to the diff itself, but as this keeps getting changed, how will we start ensuring backwards compatibility or migration capabilities in the future?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it would be great to have them more standardized, ideally as https://bitcoinops.org/en/topics/output-script-descriptors/

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Part of points to address in the future

@@ -394,6 +473,45 @@ impl ChannelKeys for InMemoryChannelKeys {
local_commitment_tx.get_htlc_sigs(&self.htlc_base_key, local_csv, secp_ctx)
}

fn sign_justice_transaction<T: secp256k1::Signing + secp256k1::Verification>(&self, justice_tx: &Transaction, input: usize, amount: u64, per_commitment_key: &SecretKey, htlc: &Option<HTLCOutputInCommitment>, on_remote_tx_csv: u16, secp_ctx: &Secp256k1<T>) -> Result<Signature, ()> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nesting here, as in the method below, is pretty deep. Could this be changed to either using ? with corresponding From for () implementations, or using consecutive (non-nested) match clauses for the Options that either return the value to a top-level-scope variable or return from the method?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, took the consecutive (non-nested) match clauses option.

@@ -449,6 +569,8 @@ impl Readable for InMemoryChannelKeys {
InMemoryChannelKeys::make_local_keys(&secp_ctx, &funding_key, &revocation_base_key,
&payment_key, &delayed_payment_base_key,
&htlc_base_key);
let user_id_1 = Readable::read(reader)?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the fact that the derivation params are, in fact, user IDs should probably be documented a bit more widely

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact they can be user_id but they can whatever else you want, changed names to params_1/params_2

@ariard ariard force-pushed the 2020-04-cache-in-monitor branch from cd3581f to 0e4e5f4 Compare May 20, 2020 23:37
@ariard
Copy link
Author

ariard commented May 20, 2020

@arik-so Thanks for review, addressed most of your comments at 0e4e5f4!

Copy link
Collaborator

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One Real Bug and a few more comments just having looked through keysinterface.rs.

Antoine Riard and others added 17 commits May 28, 2020 04:21
A dynamic-p2wsh-output like `to_local` on local commitment/HTLC txn
require a signature from delayed_payment_key to be spend. Instead of
sending private key in descriptor, we ask for spender to derive again
the corresponding ChannelKeys based on key state, uniquely identifying
a channel and encompassing its unique start data.

Descriptor modification is done in next commit.
Add sign_delayed_transaction in ChanSigner to be able to spend
SpendableOutputDescriptor in test framework.
Add sign_payment_transaction in ChanSigner to be able to spend
SpendableOutputDescriptor in test framework

Extend ChannelKeys with remote_pubkeys to access remote revocation
basepoint for witnessScript construction.
Dry-up remote pubkeys tracking in one struct.

This introduce a duplicate of RemoteTxCache, which is going
to be removed in next commit when OnchainTxHandler version is
removed.
RemoteTxCache was providing all data needed at transaction
signature for any remote HTLC transaction or justice transaction.
This move was making the API between OnchainTxHandle akward and
scope of responsibilites with ChannelMonitor unclear.

Instead scope OnchainTxHandler to transaction-finalization, fee-bumping
and broadcast only.
on_remote_tx_csv is the CSV delay encumbering remote transactions
revokable outputs as required by local.

on_local_tx_csv is the CSV delay encumbering local transactions
revokable outputs as required by remote.

Local/remote is here defined from a code processing viewpoint,
process running this code is "local".
Instead of blindly signing provided witnessScript, signer must derive
channel keys corresponding to the provided per-commitment-point and
regenerate templated witnessScript to ensure its syntax correctness.
Instead of blindly signing provided witnessScript, signer must derive
channel keys corresponding to the provided per-commitment-point and
regenerate templated witnessScript to ensure its syntax correctness.
`to_local` output or remote output on remote commitment transaction
needs a channel keys to be spent. As per-channel keys are derived from
KeysManager seed and per-channel secrets those must be backed up by
any descriptor bookmarking for latter spend. We test that generating
a new KeysManager loaded with such backed-up seed/per-channel secrets
return the correct keys for spending a `to_local` output.
@ariard ariard force-pushed the 2020-04-cache-in-monitor branch from 0e4e5f4 to 81e358c Compare May 28, 2020 08:22
@TheBlueMatt TheBlueMatt merged commit 9dbce1c into lightningdevkit:master May 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants