feat: indexer resolution #476
Conversation
behaviour is now merged into the FullResolver
|
This is largely reviewed-and-vetted (but not well tested) code written by @dignifiedquire that we don't want to let get too stale. Feel free to put as much or as little time into review as necessary. I can provide context on indexers & whatnot if needed |
|
It is weird for a p2p system to do such things: call gateways and indexer nodes via http. Might as well put the data on s3. But it seems to do what it advertises... |
iroh-resolver/src/indexer.rs
Outdated
| { | ||
| let mut bytes = unsigned_varint::encode::u32_buffer(); | ||
| unsigned_varint::encode::u32(*t as u32, &mut bytes); | ||
| serializer.serialize_str(&base64::encode(&bytes)) |
There was a problem hiding this comment.
probably want to make stable clippy happy here and remove the &
iroh-api/src/api.rs
Outdated
| FullLoaderConfig { | ||
| http_gateways: config | ||
| .http_resolvers | ||
| .clone() |
There was a problem hiding this comment.
.iter()
.flatten()instead of
.clone()
.unwrap_or_default()
.into_iter()would avoid having to clone things.
iroh-gateway/src/bad_bits.rs
Outdated
| let loader_config = FullLoaderConfig { | ||
| http_gateways: config | ||
| .http_resolvers | ||
| .as_ref() |
There was a problem hiding this comment.
This is just a test, but
.iter()
.flatten()
.map(|u| GatewayUrl::from_str(u).unwrap())would look less weird.
iroh-gateway/src/core.rs
Outdated
| let loader_config = FullLoaderConfig { | ||
| http_gateways: config | ||
| .http_resolvers | ||
| .as_ref() |
| .map(|s| &s[..]) | ||
| .unwrap_or(&[][..]) | ||
| .iter() | ||
| .map(|u| u.parse().unwrap()) |
There was a problem hiding this comment.
Can we collect into an anyhow::Result and do the unwrap outside? If the strings are unparseable, this whole method will panic, right?
Ah, right, this is only used from tests. Then it is OK I guess. Although I usually also use anyhow::Result<()> in tests.
async fn fetch_car_recursive() -> anyhow::Result<()> {will fail the test if an error is returned.
iroh-gateway/src/main.rs
Outdated
| FullLoaderConfig { | ||
| http_gateways: config | ||
| .http_resolvers | ||
| .clone() |
iroh-one/src/main.rs
Outdated
| http_gateways: config | ||
| .gateway | ||
| .http_resolvers | ||
| .clone() |
Probably not the place to discuss this, but anyway: I don't see IPFS primarily as a p2p system, but as a content based addressing stack. The beauty of that is that you can rely on whatever transport and should use the most efficient one in all situations, which p2p often is not... |
supersedes #416. Using a separate PR to keep things comparable. Updating this branch changed a lifetime parameter that has me somewhat concerned.