Skip to content

Commit 5ddbe21

Browse files
warpforkwarpfork
authoredAug 27, 2018
Merge pull request #70 from ipfs/rsrch
cid implementation research
·
v0.5.0gx/v0.9.0
2 parents afcde25 + 2cf56e3 commit 5ddbe21

File tree

8 files changed

+700
-0
lines changed

8 files changed

+700
-0
lines changed
 

‎_rsrch/cidiface/README.md

+144
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
What golang Kinds work best to implement CIDs?
2+
==============================================
3+
4+
There are many possible ways to implement CIDs. This package explores them.
5+
6+
### criteria
7+
8+
There's a couple different criteria to consider:
9+
10+
- We want the best performance when operating on the type (getters, mostly);
11+
- We want to minimize the number of memory allocations we need;
12+
- We want types which can be used as map keys, because this is common.
13+
14+
The priority of these criteria is open to argument, but it's probably
15+
mapkeys > minalloc > anythingelse.
16+
(Mapkeys and minalloc are also quite entangled, since if we don't pick a
17+
representation that can work natively as a map key, we'll end up needing
18+
a `KeyRepr()` method which gives us something that does work as a map key,
19+
an that will almost certainly involve a malloc itself.)
20+
21+
### options
22+
23+
There are quite a few different ways to go:
24+
25+
- Option A: CIDs as a struct; multihash as bytes.
26+
- Option B: CIDs as a string.
27+
- Option C: CIDs as an interface with multiple implementors.
28+
- Option D: CIDs as a struct; multihash also as a struct or string.
29+
- Option E: CIDs as a struct; content as strings plus offsets.
30+
31+
The current approach on the master branch is Option A.
32+
33+
Option D is distinctive from Option A because multihash as bytes transitively
34+
causes the CID struct to be non-comparible and thus not suitable for map keys
35+
as per https://golang.org/ref/spec#KeyType . (It's also a bit more work to
36+
pursue Option D because it's just a bigger splash radius of change; but also,
37+
something we might also want to do soon, because we *do* also have these same
38+
map-key-usability concerns with multihash alone.)
39+
40+
Option E is distinctive from Option D because Option E would always maintain
41+
the binary format of the cid internally, and so could yield it again without
42+
malloc, while still potentially having faster access to components than
43+
Option B since it wouldn't need to re-parse varints to access later fields.
44+
45+
Option C is the avoid-choices choice, but note that interfaces are not free;
46+
since "minimize mallocs" is one of our major goals, we cannot use interfaces
47+
whimsically.
48+
49+
Note there is no proposal for migrating to `type Cid []bytes`, because that
50+
is generally considered to be strictly inferior to `type Cid string`.
51+
52+
53+
Discoveries
54+
-----------
55+
56+
### using interfaces as map keys forgoes a lot of safety checks
57+
58+
Using interfaces as map keys pushes a bunch of type checking to runtime.
59+
E.g., it's totally valid at compile time to push a type which is non-comparable
60+
into a map key; it will panic at *runtime* instead of failing at compile-time.
61+
62+
There's also no way to define equality checks between implementors of the
63+
interface: golang will always use its innate concept of comparison for the
64+
concrete types. This means its effectively *never safe* to use two different
65+
concrete implementations of an interface in the same map; you may add elements
66+
which are semantically "equal" in your mind, and end up very confused later
67+
when both impls of the same "equal" object have been stored.
68+
69+
### sentinel values are possible in any impl, but some are clearer than others
70+
71+
When using `*Cid`, the nil value is a clear sentinel for 'invalid';
72+
when using `type Cid string`, the zero value is a clear sentinel;
73+
when using `type Cid struct` per Option A or D... the only valid check is
74+
for a nil multihash field, since version=0 and codec=0 are both valid values.
75+
76+
### usability as a map key is important
77+
78+
We already covered this in the criteria section, but for clarity:
79+
80+
- Option A: ❌
81+
- Option B: ✔
82+
- Option C: ~ (caveats, and depends on concrete impl)
83+
- Option D: ✔
84+
- Option E: ✔
85+
86+
### living without offsets requires parsing
87+
88+
Since CID (and multihash!) are defined using varints, they require parsing;
89+
we can't just jump into the string at a known offset in order to yield e.g.
90+
the multicodec number.
91+
92+
In order to get to the 'meat' of the CID (the multihash content), we first
93+
must parse:
94+
95+
- the CID version varint;
96+
- the multicodec varint;
97+
- the multihash type enum varint;
98+
- and the multihash length varint.
99+
100+
Since there are many applications where we want to jump straight to the
101+
multihash content (for example, when doing CAS sharding -- see the
102+
[disclaimer](https://github.com/multiformats/multihash#disclaimers) about
103+
bias in leading bytes), this overhead may be interesting.
104+
105+
How much this overhead is significant is hard to say from microbenchmarking;
106+
it depends largely on usage patterns. If these traversals are a significant
107+
timesink, it would be an argument for Option D/E.
108+
If these traversals are *not* a significant timesink, we might be wiser
109+
to keep to Option B, because keeping a struct full of offsets will add several
110+
words of memory usage per CID, and we keep a *lot* of CIDs.
111+
112+
### interfaces cause boxing which is a significant performance cost
113+
114+
See `BenchmarkCidMap_CidStr` and friends.
115+
116+
Long story short: using interfaces *anywhere* will cause the compiler to
117+
implicitly generate boxing and unboxing code (e.g. `runtime.convT2E`);
118+
this is both another function call, and more concerningly, results in
119+
large numbers of unbatchable memory allocations.
120+
121+
Numbers without context are dangerous, but if you need one: 33%.
122+
It's a big deal.
123+
124+
This means attempts to "use interfaces, but switch to concrete impls when
125+
performance is important" are a red herring: it doesn't work that way.
126+
127+
This is not a general inditement against using interfaces -- but
128+
if a situation is at the scale where it's become important to mind whether
129+
or not pointers are a performance impact, then that situation also
130+
is one where you have to think twice before using interfaces.
131+
132+
### one way or another: let's get rid of that star
133+
134+
We should switch completely to handling `Cid` and remove `*Cid` completely.
135+
Regardless of whether we do this by migrating to interface, or string
136+
implementations, or simply structs with no pointers... once we get there,
137+
refactoring to any of the *others* can become a no-op from the perspective
138+
of any downstream code that uses CIDs.
139+
140+
(This means all access via functions, never references to fields -- even if
141+
we were to use a struct implementation. *Pretend* there's a interface,
142+
in other words.)
143+
144+
There are probably `gofix` incantations which can help us with this migration.

‎_rsrch/cidiface/cid.go

+48
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
package cid
2+
3+
import (
4+
mh "github.com/multiformats/go-multihash"
5+
)
6+
7+
// Cid represents a self-describing content adressed identifier.
8+
//
9+
// A CID is composed of:
10+
//
11+
// - a Version of the CID itself,
12+
// - a Multicodec (indicates the encoding of the referenced content),
13+
// - and a Multihash (which identifies the referenced content).
14+
//
15+
// (Note that the Multihash further contains its own version and hash type
16+
// indicators.)
17+
type Cid interface {
18+
// n.b. 'yields' means "without copy", 'produces' means a malloc.
19+
20+
Version() uint64 // Yields the version prefix as a uint.
21+
Multicodec() uint64 // Yields the multicodec as a uint.
22+
Multihash() mh.Multihash // Yields the multihash segment.
23+
24+
String() string // Produces the CID formatted as b58 string.
25+
Bytes() []byte // Produces the CID formatted as raw binary.
26+
27+
Prefix() Prefix // Produces a tuple of non-content metadata.
28+
29+
// some change notes:
30+
// - `KeyString() CidString` is gone because we're natively a map key now, you're welcome.
31+
// - `StringOfBase(mbase.Encoding) (string, error)` is skipped, maybe it can come back but maybe it should be a formatter's job.
32+
// - `Equals(o Cid) bool` is gone because it's now `==`, you're welcome.
33+
34+
// TODO: make a multi-return method for {v,mc,mh} decomposition. CidStr will be able to implement this more efficiently than if one makes a series of the individual getter calls.
35+
}
36+
37+
// Prefix represents all the metadata of a Cid,
38+
// that is, the Version, the Codec, the Multihash type
39+
// and the Multihash length. It does not contains
40+
// any actual content information.
41+
// NOTE: The use -1 in MhLength to mean default length is deprecated,
42+
// use the V0Builder or V1Builder structures instead
43+
type Prefix struct {
44+
Version uint64
45+
Codec uint64
46+
MhType uint64
47+
MhLength int
48+
}

‎_rsrch/cidiface/cidBoxingBench_test.go

+71
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
package cid
2+
3+
import (
4+
"testing"
5+
)
6+
7+
// BenchmarkCidMap_CidStr estimates how fast it is to insert primitives into a map
8+
// keyed by CidStr (concretely).
9+
//
10+
// We do 100 insertions per benchmark run to make sure the map initialization
11+
// doesn't dominate the results.
12+
//
13+
// Sample results on linux amd64 go1.11beta:
14+
//
15+
// BenchmarkCidMap_CidStr-8 100000 16317 ns/op
16+
// BenchmarkCidMap_CidIface-8 100000 20516 ns/op
17+
//
18+
// With benchmem on:
19+
//
20+
// BenchmarkCidMap_CidStr-8 100000 15579 ns/op 11223 B/op 207 allocs/op
21+
// BenchmarkCidMap_CidIface-8 100000 19500 ns/op 12824 B/op 307 allocs/op
22+
// BenchmarkCidMap_StrPlusHax-8 200000 10451 ns/op 7589 B/op 202 allocs/op
23+
//
24+
// We can see here that the impact of interface boxing is significant:
25+
// it increases the time taken to do the inserts to 133%, largely because
26+
// the implied `runtime.convT2E` calls cause another malloc each.
27+
//
28+
// There are also significant allocations in both cases because
29+
// A) we cannot create a multihash without allocations since they are []byte;
30+
// B) the map has to be grown several times;
31+
// C) something I haven't quite put my finger on yet.
32+
// Ideally we'd drive those down further as well.
33+
//
34+
// Pre-allocating the map reduces allocs by a very small percentage by *count*,
35+
// but reduces the time taken by 66% overall (presumably because when a map
36+
// re-arranges itself, it involves more or less an O(n) copy of the content
37+
// in addition to the alloc itself). This isn't topical to the question of
38+
// whether or not interfaces are a good idea; just for contextualizing.
39+
//
40+
func BenchmarkCidMap_CidStr(b *testing.B) {
41+
for i := 0; i < b.N; i++ {
42+
mp := map[CidStr]int{}
43+
for x := 0; x < 100; x++ {
44+
mp[NewCidStr(0, uint64(x), []byte{})] = x
45+
}
46+
}
47+
}
48+
49+
// BenchmarkCidMap_CidIface is in the family of BenchmarkCidMap_CidStr:
50+
// it is identical except the map key type is declared as an interface
51+
// (which forces all insertions to be boxed, changing performance).
52+
func BenchmarkCidMap_CidIface(b *testing.B) {
53+
for i := 0; i < b.N; i++ {
54+
mp := map[Cid]int{}
55+
for x := 0; x < 100; x++ {
56+
mp[NewCidStr(0, uint64(x), []byte{})] = x
57+
}
58+
}
59+
}
60+
61+
// BenchmarkCidMap_CidStrAvoidMapGrowth is in the family of BenchmarkCidMap_CidStr:
62+
// it is identical except the map is created with a size hint that removes
63+
// some allocations (5, in practice, apparently).
64+
func BenchmarkCidMap_CidStrAvoidMapGrowth(b *testing.B) {
65+
for i := 0; i < b.N; i++ {
66+
mp := make(map[CidStr]int, 100)
67+
for x := 0; x < 100; x++ {
68+
mp[NewCidStr(0, uint64(x), []byte{})] = x
69+
}
70+
}
71+
}

‎_rsrch/cidiface/cidString.go

+161
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
package cid
2+
3+
import (
4+
"encoding/binary"
5+
"fmt"
6+
7+
mbase "github.com/multiformats/go-multibase"
8+
mh "github.com/multiformats/go-multihash"
9+
)
10+
11+
//=================
12+
// def & accessors
13+
//=================
14+
15+
var _ Cid = CidStr("")
16+
var _ map[CidStr]struct{} = nil
17+
18+
// CidStr is a representation of a Cid as a string type containing binary.
19+
//
20+
// Using golang's string type is preferable over byte slices even for binary
21+
// data because golang strings are immutable, usable as map keys,
22+
// trivially comparable with built-in equals operators, etc.
23+
//
24+
// Please do not cast strings or bytes into the CidStr type directly;
25+
// use a parse method which validates the data and yields a CidStr.
26+
type CidStr string
27+
28+
// EmptyCidStr is a constant for a zero/uninitialized/sentinelvalue cid;
29+
// it is declared mainly for readability in checks for sentinel values.
30+
const EmptyCidStr = CidStr("")
31+
32+
func (c CidStr) Version() uint64 {
33+
bytes := []byte(c)
34+
v, _ := binary.Uvarint(bytes)
35+
return v
36+
}
37+
38+
func (c CidStr) Multicodec() uint64 {
39+
bytes := []byte(c)
40+
_, n := binary.Uvarint(bytes) // skip version length
41+
codec, _ := binary.Uvarint(bytes[n:])
42+
return codec
43+
}
44+
45+
func (c CidStr) Multihash() mh.Multihash {
46+
bytes := []byte(c)
47+
_, n1 := binary.Uvarint(bytes) // skip version length
48+
_, n2 := binary.Uvarint(bytes[n1:]) // skip codec length
49+
return mh.Multihash(bytes[n1+n2:]) // return slice of remainder
50+
}
51+
52+
// String returns the default string representation of a Cid.
53+
// Currently, Base58 is used as the encoding for the multibase string.
54+
func (c CidStr) String() string {
55+
switch c.Version() {
56+
case 0:
57+
return c.Multihash().B58String()
58+
case 1:
59+
mbstr, err := mbase.Encode(mbase.Base58BTC, []byte(c))
60+
if err != nil {
61+
panic("should not error with hardcoded mbase: " + err.Error())
62+
}
63+
return mbstr
64+
default:
65+
panic("not possible to reach this point")
66+
}
67+
}
68+
69+
// Bytes produces a raw binary format of the CID.
70+
//
71+
// (For CidStr, this method is only distinct from casting because of
72+
// compatibility with v0 CIDs.)
73+
func (c CidStr) Bytes() []byte {
74+
switch c.Version() {
75+
case 0:
76+
return c.Multihash()
77+
case 1:
78+
return []byte(c)
79+
default:
80+
panic("not possible to reach this point")
81+
}
82+
}
83+
84+
// Prefix builds and returns a Prefix out of a Cid.
85+
func (c CidStr) Prefix() Prefix {
86+
dec, _ := mh.Decode(c.Multihash()) // assuming we got a valid multiaddr, this will not error
87+
return Prefix{
88+
MhType: dec.Code,
89+
MhLength: dec.Length,
90+
Version: c.Version(),
91+
Codec: c.Multicodec(),
92+
}
93+
}
94+
95+
//==================================
96+
// parsers & validators & factories
97+
//==================================
98+
99+
func NewCidStr(version uint64, codecType uint64, mhash mh.Multihash) CidStr {
100+
hashlen := len(mhash)
101+
// two 8 bytes (max) numbers plus hash
102+
buf := make([]byte, 2*binary.MaxVarintLen64+hashlen)
103+
n := binary.PutUvarint(buf, version)
104+
n += binary.PutUvarint(buf[n:], codecType)
105+
cn := copy(buf[n:], mhash)
106+
if cn != hashlen {
107+
panic("copy hash length is inconsistent")
108+
}
109+
return CidStr(buf[:n+hashlen])
110+
}
111+
112+
// CidStrParse takes a binary byte slice, parses it, and returns either
113+
// a valid CidStr, or the zero CidStr and an error.
114+
//
115+
// For CidV1, the data buffer is in the form:
116+
//
117+
// <version><codec-type><multihash>
118+
//
119+
// CidV0 are also supported. In particular, data buffers starting
120+
// with length 34 bytes, which starts with bytes [18,32...] are considered
121+
// binary multihashes.
122+
//
123+
// The multicodec bytes are not parsed to verify they're a valid varint;
124+
// no further reification is performed.
125+
//
126+
// Multibase encoding should already have been unwrapped before parsing;
127+
// if you have a multibase-enveloped string, use CidStrDecode instead.
128+
//
129+
// CidStrParse is the inverse of Cid.Bytes().
130+
func CidStrParse(data []byte) (CidStr, error) {
131+
if len(data) == 34 && data[0] == 18 && data[1] == 32 {
132+
h, err := mh.Cast(data)
133+
if err != nil {
134+
return EmptyCidStr, err
135+
}
136+
return NewCidStr(0, DagProtobuf, h), nil
137+
}
138+
139+
vers, n := binary.Uvarint(data)
140+
if err := uvError(n); err != nil {
141+
return EmptyCidStr, err
142+
}
143+
144+
if vers != 0 && vers != 1 {
145+
return EmptyCidStr, fmt.Errorf("invalid cid version number: %d", vers)
146+
}
147+
148+
_, cn := binary.Uvarint(data[n:])
149+
if err := uvError(cn); err != nil {
150+
return EmptyCidStr, err
151+
}
152+
153+
rest := data[n+cn:]
154+
h, err := mh.Cast(rest)
155+
if err != nil {
156+
return EmptyCidStr, err
157+
}
158+
159+
// REVIEW: if the data is longer than the mh.len expects, we silently ignore it? should we?
160+
return CidStr(data[0 : n+cn+len(h)]), nil
161+
}

‎_rsrch/cidiface/cidStruct.go

+164
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
package cid
2+
3+
import (
4+
"encoding/binary"
5+
"fmt"
6+
7+
mbase "github.com/multiformats/go-multibase"
8+
mh "github.com/multiformats/go-multihash"
9+
)
10+
11+
//=================
12+
// def & accessors
13+
//=================
14+
15+
var _ Cid = CidStruct{}
16+
17+
//var _ map[CidStruct]struct{} = nil // Will not compile! See struct def docs.
18+
//var _ map[Cid]struct{} = map[Cid]struct{}{CidStruct{}: struct{}{}} // Legal to compile...
19+
// but you'll get panics: "runtime error: hash of unhashable type cid.CidStruct"
20+
21+
// CidStruct represents a CID in a struct format.
22+
//
23+
// This format complies with the exact same Cid interface as the CidStr
24+
// implementation, but completely pre-parses the Cid metadata.
25+
// CidStruct is a tad quicker in case of repeatedly accessed fields,
26+
// but requires more reshuffling to parse and to serialize.
27+
// CidStruct is not usable as a map key, because it contains a Multihash
28+
// reference, which is a slice, and thus not "comparable" as a primitive.
29+
//
30+
// Beware of zero-valued CidStruct: it is difficult to distinguish an
31+
// incorrectly-initialized "invalid" CidStruct from one representing a v0 cid.
32+
type CidStruct struct {
33+
version uint64
34+
codec uint64
35+
hash mh.Multihash
36+
}
37+
38+
// EmptyCidStruct is a constant for a zero/uninitialized/sentinelvalue cid;
39+
// it is declared mainly for readability in checks for sentinel values.
40+
//
41+
// Note: it's not actually a const; the compiler does not allow const structs.
42+
var EmptyCidStruct = CidStruct{}
43+
44+
func (c CidStruct) Version() uint64 {
45+
return c.version
46+
}
47+
48+
func (c CidStruct) Multicodec() uint64 {
49+
return c.codec
50+
}
51+
52+
func (c CidStruct) Multihash() mh.Multihash {
53+
return c.hash
54+
}
55+
56+
// String returns the default string representation of a Cid.
57+
// Currently, Base58 is used as the encoding for the multibase string.
58+
func (c CidStruct) String() string {
59+
switch c.Version() {
60+
case 0:
61+
return c.Multihash().B58String()
62+
case 1:
63+
mbstr, err := mbase.Encode(mbase.Base58BTC, c.Bytes())
64+
if err != nil {
65+
panic("should not error with hardcoded mbase: " + err.Error())
66+
}
67+
return mbstr
68+
default:
69+
panic("not possible to reach this point")
70+
}
71+
}
72+
73+
// Bytes produces a raw binary format of the CID.
74+
func (c CidStruct) Bytes() []byte {
75+
switch c.version {
76+
case 0:
77+
return []byte(c.hash)
78+
case 1:
79+
// two 8 bytes (max) numbers plus hash
80+
buf := make([]byte, 2*binary.MaxVarintLen64+len(c.hash))
81+
n := binary.PutUvarint(buf, c.version)
82+
n += binary.PutUvarint(buf[n:], c.codec)
83+
cn := copy(buf[n:], c.hash)
84+
if cn != len(c.hash) {
85+
panic("copy hash length is inconsistent")
86+
}
87+
return buf[:n+len(c.hash)]
88+
default:
89+
panic("not possible to reach this point")
90+
}
91+
}
92+
93+
// Prefix builds and returns a Prefix out of a Cid.
94+
func (c CidStruct) Prefix() Prefix {
95+
dec, _ := mh.Decode(c.hash) // assuming we got a valid multiaddr, this will not error
96+
return Prefix{
97+
MhType: dec.Code,
98+
MhLength: dec.Length,
99+
Version: c.version,
100+
Codec: c.codec,
101+
}
102+
}
103+
104+
//==================================
105+
// parsers & validators & factories
106+
//==================================
107+
108+
// CidStructParse takes a binary byte slice, parses it, and returns either
109+
// a valid CidStruct, or the zero CidStruct and an error.
110+
//
111+
// For CidV1, the data buffer is in the form:
112+
//
113+
// <version><codec-type><multihash>
114+
//
115+
// CidV0 are also supported. In particular, data buffers starting
116+
// with length 34 bytes, which starts with bytes [18,32...] are considered
117+
// binary multihashes.
118+
//
119+
// The multicodec bytes are not parsed to verify they're a valid varint;
120+
// no further reification is performed.
121+
//
122+
// Multibase encoding should already have been unwrapped before parsing;
123+
// if you have a multibase-enveloped string, use CidStructDecode instead.
124+
//
125+
// CidStructParse is the inverse of Cid.Bytes().
126+
func CidStructParse(data []byte) (CidStruct, error) {
127+
if len(data) == 34 && data[0] == 18 && data[1] == 32 {
128+
h, err := mh.Cast(data)
129+
if err != nil {
130+
return EmptyCidStruct, err
131+
}
132+
return CidStruct{
133+
codec: DagProtobuf,
134+
version: 0,
135+
hash: h,
136+
}, nil
137+
}
138+
139+
vers, n := binary.Uvarint(data)
140+
if err := uvError(n); err != nil {
141+
return EmptyCidStruct, err
142+
}
143+
144+
if vers != 0 && vers != 1 {
145+
return EmptyCidStruct, fmt.Errorf("invalid cid version number: %d", vers)
146+
}
147+
148+
codec, cn := binary.Uvarint(data[n:])
149+
if err := uvError(cn); err != nil {
150+
return EmptyCidStruct, err
151+
}
152+
153+
rest := data[n+cn:]
154+
h, err := mh.Cast(rest)
155+
if err != nil {
156+
return EmptyCidStruct, err
157+
}
158+
159+
return CidStruct{
160+
version: vers,
161+
codec: codec,
162+
hash: h,
163+
}, nil
164+
}

‎_rsrch/cidiface/enums.go

+76
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
package cid
2+
3+
// These are multicodec-packed content types. The should match
4+
// the codes described in the authoritative document:
5+
// https://github.com/multiformats/multicodec/blob/master/table.csv
6+
const (
7+
Raw = 0x55
8+
9+
DagProtobuf = 0x70
10+
DagCBOR = 0x71
11+
12+
GitRaw = 0x78
13+
14+
EthBlock = 0x90
15+
EthBlockList = 0x91
16+
EthTxTrie = 0x92
17+
EthTx = 0x93
18+
EthTxReceiptTrie = 0x94
19+
EthTxReceipt = 0x95
20+
EthStateTrie = 0x96
21+
EthAccountSnapshot = 0x97
22+
EthStorageTrie = 0x98
23+
BitcoinBlock = 0xb0
24+
BitcoinTx = 0xb1
25+
ZcashBlock = 0xc0
26+
ZcashTx = 0xc1
27+
DecredBlock = 0xe0
28+
DecredTx = 0xe1
29+
)
30+
31+
// Codecs maps the name of a codec to its type
32+
var Codecs = map[string]uint64{
33+
"v0": DagProtobuf,
34+
"raw": Raw,
35+
"protobuf": DagProtobuf,
36+
"cbor": DagCBOR,
37+
"git-raw": GitRaw,
38+
"eth-block": EthBlock,
39+
"eth-block-list": EthBlockList,
40+
"eth-tx-trie": EthTxTrie,
41+
"eth-tx": EthTx,
42+
"eth-tx-receipt-trie": EthTxReceiptTrie,
43+
"eth-tx-receipt": EthTxReceipt,
44+
"eth-state-trie": EthStateTrie,
45+
"eth-account-snapshot": EthAccountSnapshot,
46+
"eth-storage-trie": EthStorageTrie,
47+
"bitcoin-block": BitcoinBlock,
48+
"bitcoin-tx": BitcoinTx,
49+
"zcash-block": ZcashBlock,
50+
"zcash-tx": ZcashTx,
51+
"decred-block": DecredBlock,
52+
"decred-tx": DecredTx,
53+
}
54+
55+
// CodecToStr maps the numeric codec to its name
56+
var CodecToStr = map[uint64]string{
57+
Raw: "raw",
58+
DagProtobuf: "protobuf",
59+
DagCBOR: "cbor",
60+
GitRaw: "git-raw",
61+
EthBlock: "eth-block",
62+
EthBlockList: "eth-block-list",
63+
EthTxTrie: "eth-tx-trie",
64+
EthTx: "eth-tx",
65+
EthTxReceiptTrie: "eth-tx-receipt-trie",
66+
EthTxReceipt: "eth-tx-receipt",
67+
EthStateTrie: "eth-state-trie",
68+
EthAccountSnapshot: "eth-account-snapshot",
69+
EthStorageTrie: "eth-storage-trie",
70+
BitcoinBlock: "bitcoin-block",
71+
BitcoinTx: "bitcoin-tx",
72+
ZcashBlock: "zcash-block",
73+
ZcashTx: "zcash-tx",
74+
DecredBlock: "decred-block",
75+
DecredTx: "decred-tx",
76+
}

‎_rsrch/cidiface/errors.go

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
package cid
2+
3+
import (
4+
"errors"
5+
)
6+
7+
var (
8+
// ErrVarintBuffSmall means that a buffer passed to the cid parser was not
9+
// long enough, or did not contain an invalid cid
10+
ErrVarintBuffSmall = errors.New("reading varint: buffer too small")
11+
12+
// ErrVarintTooBig means that the varint in the given cid was above the
13+
// limit of 2^64
14+
ErrVarintTooBig = errors.New("reading varint: varint bigger than 64bits" +
15+
" and not supported")
16+
17+
// ErrCidTooShort means that the cid passed to decode was not long
18+
// enough to be a valid Cid
19+
ErrCidTooShort = errors.New("cid too short")
20+
21+
// ErrInvalidEncoding means that selected encoding is not supported
22+
// by this Cid version
23+
ErrInvalidEncoding = errors.New("invalid base encoding")
24+
)

‎_rsrch/cidiface/misc.go

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
package cid
2+
3+
func uvError(read int) error {
4+
switch {
5+
case read == 0:
6+
return ErrVarintBuffSmall
7+
case read < 0:
8+
return ErrVarintTooBig
9+
default:
10+
return nil
11+
}
12+
}

0 commit comments

Comments
 (0)
Please sign in to comment.