Skip to content
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Commit 2692d0b

Browse files
jasnellruyadorno
jasnell
authored and
ruyadorno
committedJan 21, 2021
buffer: introduce Blob
The `Blob` object is an immutable data buffer. This is a first step towards alignment with the `Blob` Web API. Signed-off-by: James M Snell <jasnell@gmail.com> PR-URL: #36811 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
1 parent d97917f commit 2692d0b

13 files changed

+1031
-0
lines changed
 

‎doc/api/buffer.md

+114
Original file line numberDiff line numberDiff line change
@@ -287,6 +287,119 @@ for (const b of buf) {
287287
Additionally, the [`buf.values()`][], [`buf.keys()`][], and
288288
[`buf.entries()`][] methods can be used to create iterators.
289289

290+
## Class: `Blob`
291+
<!-- YAML
292+
added: REPLACEME
293+
-->
294+
295+
> Stability: 1 - Experimental
296+
297+
A [`Blob`][] encapsulates immutable, raw data that can be safely shared across
298+
multiple worker threads.
299+
300+
### `new buffer.Blob([sources[, options]])`
301+
<!-- YAML
302+
added: REPLACEME
303+
-->
304+
305+
* `sources` {string[]|ArrayBuffer[]|TypedArray[]|DataView[]|Blob[]} An array
306+
of string, {ArrayBuffer}, {TypedArray}, {DataView}, or {Blob} objects, or
307+
any mix of such objects, that will be stored within the `Blob`.
308+
* `options` {Object}
309+
* `encoding` {string} The character encoding to use for string sources.
310+
**Default**: `'utf8'`.
311+
* `type` {string} The Blob content-type. The intent is for `type` to convey
312+
the MIME media type of the data, however no validation of the type format
313+
is performed.
314+
315+
Creates a new `Blob` object containing a concatenation of the given sources.
316+
317+
{ArrayBuffer}, {TypedArray}, {DataView}, and {Buffer} sources are copied into
318+
the 'Blob' and can therefore be safely modified after the 'Blob' is created.
319+
320+
String sources are also copied into the `Blob`.
321+
322+
### `blob.arrayBuffer()`
323+
<!-- YAML
324+
added: REPLACEME
325+
-->
326+
327+
* Returns: {Promise}
328+
329+
Returns a promise that fulfills with an {ArrayBuffer} containing a copy of
330+
the `Blob` data.
331+
332+
### `blob.size`
333+
<!-- YAML
334+
added: REPLACEME
335+
-->
336+
337+
The total size of the `Blob` in bytes.
338+
339+
### `blob.slice([start, [end, [type]]])`
340+
<!-- YAML
341+
added: REPLACEME
342+
-->
343+
344+
* `start` {number} The starting index.
345+
* `end` {number} The ending index.
346+
* `type` {string} The content-type for the new `Blob`
347+
348+
Creates and returns a new `Blob` containing a subset of this `Blob` objects
349+
data. The original `Blob` is not alterered.
350+
351+
### `blob.text()`
352+
<!-- YAML
353+
added: REPLACEME
354+
-->
355+
356+
* Returns: {Promise}
357+
358+
Returns a promise that resolves the contents of the `Blob` decoded as a UTF-8
359+
string.
360+
361+
### `blob.type`
362+
<!-- YAML
363+
added: REPLACEME
364+
-->
365+
366+
* Type: {string}
367+
368+
The content-type of the `Blob`.
369+
370+
### `Blob` objects and `MessageChannel`
371+
372+
Once a {Blob} object is created, it can be sent via `MessagePort` to multiple
373+
destinations without transfering or immediately copying the data. The data
374+
contained by the `Blob` is copied only when the `arrayBuffer()` or `text()`
375+
methods are called.
376+
377+
```js
378+
const { Blob } = require('buffer');
379+
const blob = new Blob(['hello there']);
380+
const { setTimeout: delay } = require('timers/promises');
381+
382+
const mc1 = new MessageChannel();
383+
const mc2 = new MessageChannel();
384+
385+
mc1.port1.onmessage = async ({ data }) => {
386+
console.log(await data.arrayBuffer());
387+
mc1.port1.close();
388+
};
389+
390+
mc2.port1.onmessage = async ({ data }) => {
391+
await delay(1000);
392+
console.log(await data.arrayBuffer());
393+
mc2.port1.close();
394+
};
395+
396+
mc1.port2.postMessage(blob);
397+
mc2.port2.postMessage(blob);
398+
399+
// The Blob is still usable after posting.
400+
data.text().then(console.log);
401+
```
402+
290403
## Class: `Buffer`
291404

292405
The `Buffer` class is a global type for dealing with binary data directly.
@@ -3389,6 +3502,7 @@ introducing security vulnerabilities into an application.
33893502
[UTF-8]: https://en.wikipedia.org/wiki/UTF-8
33903503
[WHATWG Encoding Standard]: https://encoding.spec.whatwg.org/
33913504
[`ArrayBuffer`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer
3505+
[`Blob`]: https://developer.mozilla.org/en-US/docs/Web/API/Blob
33923506
[`Buffer.alloc()`]: #buffer_static_method_buffer_alloc_size_fill_encoding
33933507
[`Buffer.allocUnsafe()`]: #buffer_static_method_buffer_allocunsafe_size
33943508
[`Buffer.allocUnsafeSlow()`]: #buffer_static_method_buffer_allocunsafeslow_size

‎lib/buffer.js

+5
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,10 @@ const {
115115
createUnsafeBuffer
116116
} = require('internal/buffer');
117117

118+
const {
119+
Blob,
120+
} = require('internal/blob');
121+
118122
FastBuffer.prototype.constructor = Buffer;
119123
Buffer.prototype = FastBuffer.prototype;
120124
addBufferPrototypeMethods(Buffer.prototype);
@@ -1210,6 +1214,7 @@ if (internalBinding('config').hasIntl) {
12101214
}
12111215

12121216
module.exports = {
1217+
Blob,
12131218
Buffer,
12141219
SlowBuffer,
12151220
transcode,

‎lib/internal/blob.js

+238
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,238 @@
1+
'use strict';
2+
3+
const {
4+
ArrayFrom,
5+
ObjectSetPrototypeOf,
6+
Promise,
7+
PromiseResolve,
8+
RegExpPrototypeTest,
9+
StringPrototypeToLowerCase,
10+
Symbol,
11+
SymbolIterator,
12+
Uint8Array,
13+
} = primordials;
14+
15+
const {
16+
createBlob,
17+
FixedSizeBlobCopyJob,
18+
} = internalBinding('buffer');
19+
20+
const {
21+
JSTransferable,
22+
kClone,
23+
kDeserialize,
24+
} = require('internal/worker/js_transferable');
25+
26+
const {
27+
isAnyArrayBuffer,
28+
isArrayBufferView,
29+
} = require('internal/util/types');
30+
31+
const {
32+
customInspectSymbol: kInspect,
33+
emitExperimentalWarning,
34+
} = require('internal/util');
35+
const { inspect } = require('internal/util/inspect');
36+
37+
const {
38+
AbortError,
39+
codes: {
40+
ERR_INVALID_ARG_TYPE,
41+
ERR_BUFFER_TOO_LARGE,
42+
ERR_OUT_OF_RANGE,
43+
}
44+
} = require('internal/errors');
45+
46+
const {
47+
validateObject,
48+
validateString,
49+
validateUint32,
50+
isUint32,
51+
} = require('internal/validators');
52+
53+
const kHandle = Symbol('kHandle');
54+
const kType = Symbol('kType');
55+
const kLength = Symbol('kLength');
56+
57+
let Buffer;
58+
59+
function deferred() {
60+
let res, rej;
61+
const promise = new Promise((resolve, reject) => {
62+
res = resolve;
63+
rej = reject;
64+
});
65+
return { promise, resolve: res, reject: rej };
66+
}
67+
68+
function lazyBuffer() {
69+
if (Buffer === undefined)
70+
Buffer = require('buffer').Buffer;
71+
return Buffer;
72+
}
73+
74+
function isBlob(object) {
75+
return object?.[kHandle] !== undefined;
76+
}
77+
78+
function getSource(source, encoding) {
79+
if (isBlob(source))
80+
return [source.size, source[kHandle]];
81+
82+
if (typeof source === 'string') {
83+
source = lazyBuffer().from(source, encoding);
84+
} else if (isAnyArrayBuffer(source)) {
85+
source = new Uint8Array(source);
86+
} else if (!isArrayBufferView(source)) {
87+
throw new ERR_INVALID_ARG_TYPE(
88+
'source',
89+
[
90+
'string',
91+
'ArrayBuffer',
92+
'SharedArrayBuffer',
93+
'Buffer',
94+
'TypedArray',
95+
'DataView'
96+
],
97+
source);
98+
}
99+
100+
// We copy into a new Uint8Array because the underlying
101+
// BackingStores are going to be detached and owned by
102+
// the Blob. We also don't want to have to worry about
103+
// byte offsets.
104+
source = new Uint8Array(source);
105+
return [source.byteLength, source];
106+
}
107+
108+
class InternalBlob extends JSTransferable {
109+
constructor(handle, length, type = '') {
110+
super();
111+
this[kHandle] = handle;
112+
this[kType] = type;
113+
this[kLength] = length;
114+
}
115+
}
116+
117+
class Blob extends JSTransferable {
118+
constructor(sources = [], options) {
119+
emitExperimentalWarning('buffer.Blob');
120+
if (sources === null ||
121+
typeof sources[SymbolIterator] !== 'function' ||
122+
typeof sources === 'string') {
123+
throw new ERR_INVALID_ARG_TYPE('sources', 'Iterable', sources);
124+
}
125+
if (options !== undefined)
126+
validateObject(options, 'options');
127+
const {
128+
encoding = 'utf8',
129+
type = '',
130+
} = { ...options };
131+
132+
let length = 0;
133+
const sources_ = ArrayFrom(sources, (source) => {
134+
const { 0: len, 1: src } = getSource(source, encoding);
135+
length += len;
136+
return src;
137+
});
138+
139+
// This is a MIME media type but we're not actively checking the syntax.
140+
// But, to be fair, neither does Chrome.
141+
validateString(type, 'options.type');
142+
143+
if (!isUint32(length))
144+
throw new ERR_BUFFER_TOO_LARGE(0xFFFFFFFF);
145+
146+
super();
147+
this[kHandle] = createBlob(sources_, length);
148+
this[kLength] = length;
149+
this[kType] = RegExpPrototypeTest(/[^\u{0020}-\u{007E}]/u, type) ?
150+
'' : StringPrototypeToLowerCase(type);
151+
}
152+
153+
[kInspect](depth, options) {
154+
if (depth < 0)
155+
return this;
156+
157+
const opts = {
158+
...options,
159+
depth: options.depth == null ? null : options.depth - 1
160+
};
161+
162+
return `Blob ${inspect({
163+
size: this.size,
164+
type: this.type,
165+
}, opts)}`;
166+
}
167+
168+
[kClone]() {
169+
const handle = this[kHandle];
170+
const type = this[kType];
171+
const length = this[kLength];
172+
return {
173+
data: { handle, type, length },
174+
deserializeInfo: 'internal/blob:InternalBlob'
175+
};
176+
}
177+
178+
[kDeserialize]({ handle, type, length }) {
179+
this[kHandle] = handle;
180+
this[kType] = type;
181+
this[kLength] = length;
182+
}
183+
184+
get type() { return this[kType]; }
185+
186+
get size() { return this[kLength]; }
187+
188+
slice(start = 0, end = (this[kLength]), type = this[kType]) {
189+
validateUint32(start, 'start');
190+
if (end < 0) end = this[kLength] + end;
191+
validateUint32(end, 'end');
192+
validateString(type, 'type');
193+
if (end < start)
194+
throw new ERR_OUT_OF_RANGE('end', 'greater than start', end);
195+
if (end > this[kLength])
196+
throw new ERR_OUT_OF_RANGE('end', 'less than or equal to length', end);
197+
return new InternalBlob(
198+
this[kHandle].slice(start, end),
199+
end - start, type);
200+
}
201+
202+
async arrayBuffer() {
203+
const job = new FixedSizeBlobCopyJob(this[kHandle]);
204+
205+
const ret = job.run();
206+
if (ret !== undefined)
207+
return PromiseResolve(ret);
208+
209+
const {
210+
promise,
211+
resolve,
212+
reject
213+
} = deferred();
214+
job.ondone = (err, ab) => {
215+
if (err !== undefined)
216+
return reject(new AbortError());
217+
resolve(ab);
218+
};
219+
220+
return promise;
221+
}
222+
223+
async text() {
224+
const dec = new TextDecoder();
225+
return dec.decode(await this.arrayBuffer());
226+
}
227+
}
228+
229+
InternalBlob.prototype.constructor = Blob;
230+
ObjectSetPrototypeOf(
231+
InternalBlob.prototype,
232+
Blob.prototype);
233+
234+
module.exports = {
235+
Blob,
236+
InternalBlob,
237+
isBlob,
238+
};

‎node.gyp

+3
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,7 @@
111111
'lib/internal/assert/assertion_error.js',
112112
'lib/internal/assert/calltracker.js',
113113
'lib/internal/async_hooks.js',
114+
'lib/internal/blob.js',
114115
'lib/internal/blocklist.js',
115116
'lib/internal/buffer.js',
116117
'lib/internal/cli_table.js',
@@ -609,6 +610,7 @@
609610
'src/node.cc',
610611
'src/node_api.cc',
611612
'src/node_binding.cc',
613+
'src/node_blob.cc',
612614
'src/node_buffer.cc',
613615
'src/node_config.cc',
614616
'src/node_constants.cc',
@@ -707,6 +709,7 @@
707709
'src/node_api.h',
708710
'src/node_api_types.h',
709711
'src/node_binding.h',
712+
'src/node_blob.h',
710713
'src/node_buffer.h',
711714
'src/node_constants.h',
712715
'src/node_context_data.h',

‎src/async_wrap.h

+1
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ namespace node {
3838
V(ELDHISTOGRAM) \
3939
V(FILEHANDLE) \
4040
V(FILEHANDLECLOSEREQ) \
41+
V(FIXEDSIZEBLOBCOPY) \
4142
V(FSEVENTWRAP) \
4243
V(FSREQCALLBACK) \
4344
V(FSREQPROMISE) \

‎src/env.h

+1
Original file line numberDiff line numberDiff line change
@@ -450,6 +450,7 @@ constexpr size_t kFsStatsBufferLength =
450450
V(async_wrap_object_ctor_template, v8::FunctionTemplate) \
451451
V(base_object_ctor_template, v8::FunctionTemplate) \
452452
V(binding_data_ctor_template, v8::FunctionTemplate) \
453+
V(blob_constructor_template, v8::FunctionTemplate) \
453454
V(blocklist_instance_template, v8::ObjectTemplate) \
454455
V(compiled_fn_entry_template, v8::ObjectTemplate) \
455456
V(dir_instance_template, v8::ObjectTemplate) \

‎src/node_blob.cc

+336
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,336 @@
1+
#include "node_blob.h"
2+
#include "async_wrap-inl.h"
3+
#include "base_object-inl.h"
4+
#include "env-inl.h"
5+
#include "memory_tracker-inl.h"
6+
#include "node_errors.h"
7+
#include "node_external_reference.h"
8+
#include "threadpoolwork-inl.h"
9+
#include "v8.h"
10+
11+
#include <algorithm>
12+
13+
namespace node {
14+
15+
using v8::Array;
16+
using v8::ArrayBuffer;
17+
using v8::ArrayBufferView;
18+
using v8::BackingStore;
19+
using v8::Context;
20+
using v8::EscapableHandleScope;
21+
using v8::Function;
22+
using v8::FunctionCallbackInfo;
23+
using v8::FunctionTemplate;
24+
using v8::HandleScope;
25+
using v8::Local;
26+
using v8::MaybeLocal;
27+
using v8::Number;
28+
using v8::Object;
29+
using v8::Uint32;
30+
using v8::Undefined;
31+
using v8::Value;
32+
33+
void Blob::Initialize(Environment* env, v8::Local<v8::Object> target) {
34+
env->SetMethod(target, "createBlob", New);
35+
FixedSizeBlobCopyJob::Initialize(env, target);
36+
}
37+
38+
Local<FunctionTemplate> Blob::GetConstructorTemplate(Environment* env) {
39+
Local<FunctionTemplate> tmpl = env->blob_constructor_template();
40+
if (tmpl.IsEmpty()) {
41+
tmpl = FunctionTemplate::New(env->isolate());
42+
tmpl->InstanceTemplate()->SetInternalFieldCount(1);
43+
tmpl->Inherit(BaseObject::GetConstructorTemplate(env));
44+
tmpl->SetClassName(
45+
FIXED_ONE_BYTE_STRING(env->isolate(), "Blob"));
46+
env->SetProtoMethod(tmpl, "toArrayBuffer", ToArrayBuffer);
47+
env->SetProtoMethod(tmpl, "slice", ToSlice);
48+
env->set_blob_constructor_template(tmpl);
49+
}
50+
return tmpl;
51+
}
52+
53+
bool Blob::HasInstance(Environment* env, v8::Local<v8::Value> object) {
54+
return GetConstructorTemplate(env)->HasInstance(object);
55+
}
56+
57+
BaseObjectPtr<Blob> Blob::Create(
58+
Environment* env,
59+
const std::vector<BlobEntry> store,
60+
size_t length) {
61+
62+
HandleScope scope(env->isolate());
63+
64+
Local<Function> ctor;
65+
if (!GetConstructorTemplate(env)->GetFunction(env->context()).ToLocal(&ctor))
66+
return BaseObjectPtr<Blob>();
67+
68+
Local<Object> obj;
69+
if (!ctor->NewInstance(env->context()).ToLocal(&obj))
70+
return BaseObjectPtr<Blob>();
71+
72+
return MakeBaseObject<Blob>(env, obj, store, length);
73+
}
74+
75+
void Blob::New(const FunctionCallbackInfo<Value>& args) {
76+
Environment* env = Environment::GetCurrent(args);
77+
CHECK(args[0]->IsArray()); // sources
78+
CHECK(args[1]->IsUint32()); // length
79+
80+
std::vector<BlobEntry> entries;
81+
82+
size_t length = args[1].As<Uint32>()->Value();
83+
size_t len = 0;
84+
Local<Array> ary = args[0].As<Array>();
85+
for (size_t n = 0; n < ary->Length(); n++) {
86+
Local<Value> entry;
87+
if (!ary->Get(env->context(), n).ToLocal(&entry))
88+
return;
89+
CHECK(entry->IsArrayBufferView() || Blob::HasInstance(env, entry));
90+
if (entry->IsArrayBufferView()) {
91+
Local<ArrayBufferView> view = entry.As<ArrayBufferView>();
92+
CHECK_EQ(view->ByteOffset(), 0);
93+
std::shared_ptr<BackingStore> store = view->Buffer()->GetBackingStore();
94+
size_t byte_length = view->ByteLength();
95+
view->Buffer()->Detach(); // The Blob will own the backing store now.
96+
entries.emplace_back(BlobEntry{std::move(store), byte_length, 0});
97+
len += byte_length;
98+
} else {
99+
Blob* blob;
100+
ASSIGN_OR_RETURN_UNWRAP(&blob, entry);
101+
auto source = blob->entries();
102+
entries.insert(entries.end(), source.begin(), source.end());
103+
len += blob->length();
104+
}
105+
}
106+
CHECK_EQ(length, len);
107+
108+
BaseObjectPtr<Blob> blob = Create(env, entries, length);
109+
if (blob)
110+
args.GetReturnValue().Set(blob->object());
111+
}
112+
113+
void Blob::ToArrayBuffer(const FunctionCallbackInfo<Value>& args) {
114+
Environment* env = Environment::GetCurrent(args);
115+
Blob* blob;
116+
ASSIGN_OR_RETURN_UNWRAP(&blob, args.Holder());
117+
Local<Value> ret;
118+
if (blob->GetArrayBuffer(env).ToLocal(&ret))
119+
args.GetReturnValue().Set(ret);
120+
}
121+
122+
void Blob::ToSlice(const FunctionCallbackInfo<Value>& args) {
123+
Environment* env = Environment::GetCurrent(args);
124+
Blob* blob;
125+
ASSIGN_OR_RETURN_UNWRAP(&blob, args.Holder());
126+
CHECK(args[0]->IsUint32());
127+
CHECK(args[1]->IsUint32());
128+
size_t start = args[0].As<Uint32>()->Value();
129+
size_t end = args[1].As<Uint32>()->Value();
130+
BaseObjectPtr<Blob> slice = blob->Slice(env, start, end);
131+
if (slice)
132+
args.GetReturnValue().Set(slice->object());
133+
}
134+
135+
void Blob::MemoryInfo(MemoryTracker* tracker) const {
136+
tracker->TrackFieldWithSize("store", length_);
137+
}
138+
139+
MaybeLocal<Value> Blob::GetArrayBuffer(Environment* env) {
140+
EscapableHandleScope scope(env->isolate());
141+
size_t len = length();
142+
std::shared_ptr<BackingStore> store =
143+
ArrayBuffer::NewBackingStore(env->isolate(), len);
144+
if (len > 0) {
145+
unsigned char* dest = static_cast<unsigned char*>(store->Data());
146+
size_t total = 0;
147+
for (const auto& entry : entries()) {
148+
unsigned char* src = static_cast<unsigned char*>(entry.store->Data());
149+
src += entry.offset;
150+
memcpy(dest, src, entry.length);
151+
dest += entry.length;
152+
total += entry.length;
153+
CHECK_LE(total, len);
154+
}
155+
}
156+
157+
return scope.Escape(ArrayBuffer::New(env->isolate(), store));
158+
}
159+
160+
BaseObjectPtr<Blob> Blob::Slice(Environment* env, size_t start, size_t end) {
161+
CHECK_LE(start, length());
162+
CHECK_LE(end, length());
163+
CHECK_LE(start, end);
164+
165+
std::vector<BlobEntry> slices;
166+
size_t total = end - start;
167+
size_t remaining = total;
168+
169+
if (total == 0) return Create(env, slices, 0);
170+
171+
for (const auto& entry : entries()) {
172+
if (start + entry.offset > entry.store->ByteLength()) {
173+
start -= entry.length;
174+
continue;
175+
}
176+
177+
size_t offset = entry.offset + start;
178+
size_t len = std::min(remaining, entry.store->ByteLength() - offset);
179+
slices.emplace_back(BlobEntry{entry.store, len, offset});
180+
181+
remaining -= len;
182+
start = 0;
183+
184+
if (remaining == 0)
185+
break;
186+
}
187+
188+
return Create(env, slices, total);
189+
}
190+
191+
Blob::Blob(
192+
Environment* env,
193+
v8::Local<v8::Object> obj,
194+
const std::vector<BlobEntry>& store,
195+
size_t length)
196+
: BaseObject(env, obj),
197+
store_(store),
198+
length_(length) {
199+
MakeWeak();
200+
}
201+
202+
BaseObjectPtr<BaseObject>
203+
Blob::BlobTransferData::Deserialize(
204+
Environment* env,
205+
Local<Context> context,
206+
std::unique_ptr<worker::TransferData> self) {
207+
if (context != env->context()) {
208+
THROW_ERR_MESSAGE_TARGET_CONTEXT_UNAVAILABLE(env);
209+
return {};
210+
}
211+
return Blob::Create(env, store_, length_);
212+
}
213+
214+
BaseObject::TransferMode Blob::GetTransferMode() const {
215+
return BaseObject::TransferMode::kCloneable;
216+
}
217+
218+
std::unique_ptr<worker::TransferData> Blob::CloneForMessaging() const {
219+
return std::make_unique<BlobTransferData>(store_, length_);
220+
}
221+
222+
FixedSizeBlobCopyJob::FixedSizeBlobCopyJob(
223+
Environment* env,
224+
Local<Object> object,
225+
Blob* blob,
226+
FixedSizeBlobCopyJob::Mode mode)
227+
: AsyncWrap(env, object, AsyncWrap::PROVIDER_FIXEDSIZEBLOBCOPY),
228+
ThreadPoolWork(env),
229+
mode_(mode) {
230+
if (mode == FixedSizeBlobCopyJob::Mode::SYNC) MakeWeak();
231+
source_ = blob->entries();
232+
length_ = blob->length();
233+
}
234+
235+
void FixedSizeBlobCopyJob::AfterThreadPoolWork(int status) {
236+
Environment* env = AsyncWrap::env();
237+
CHECK_EQ(mode_, Mode::ASYNC);
238+
CHECK(status == 0 || status == UV_ECANCELED);
239+
std::unique_ptr<FixedSizeBlobCopyJob> ptr(this);
240+
HandleScope handle_scope(env->isolate());
241+
Context::Scope context_scope(env->context());
242+
Local<Value> args[2];
243+
244+
if (status == UV_ECANCELED) {
245+
args[0] = Number::New(env->isolate(), status),
246+
args[1] = Undefined(env->isolate());
247+
} else {
248+
args[0] = Undefined(env->isolate());
249+
args[1] = ArrayBuffer::New(env->isolate(), destination_);
250+
}
251+
252+
ptr->MakeCallback(env->ondone_string(), arraysize(args), args);
253+
}
254+
255+
void FixedSizeBlobCopyJob::DoThreadPoolWork() {
256+
Environment* env = AsyncWrap::env();
257+
destination_ = ArrayBuffer::NewBackingStore(env->isolate(), length_);
258+
unsigned char* dest = static_cast<unsigned char*>(destination_->Data());
259+
if (length_ > 0) {
260+
size_t total = 0;
261+
for (const auto& entry : source_) {
262+
unsigned char* src = static_cast<unsigned char*>(entry.store->Data());
263+
src += entry.offset;
264+
memcpy(dest, src, entry.length);
265+
dest += entry.length;
266+
total += entry.length;
267+
CHECK_LE(total, length_);
268+
}
269+
}
270+
}
271+
272+
void FixedSizeBlobCopyJob::MemoryInfo(MemoryTracker* tracker) const {
273+
tracker->TrackFieldWithSize("source", length_);
274+
tracker->TrackFieldWithSize(
275+
"destination",
276+
destination_ ? destination_->ByteLength() : 0);
277+
}
278+
279+
void FixedSizeBlobCopyJob::Initialize(Environment* env, Local<Object> target) {
280+
v8::Local<v8::FunctionTemplate> job = env->NewFunctionTemplate(New);
281+
job->Inherit(AsyncWrap::GetConstructorTemplate(env));
282+
job->InstanceTemplate()->SetInternalFieldCount(
283+
AsyncWrap::kInternalFieldCount);
284+
env->SetProtoMethod(job, "run", Run);
285+
env->SetConstructorFunction(target, "FixedSizeBlobCopyJob", job);
286+
}
287+
288+
void FixedSizeBlobCopyJob::New(const FunctionCallbackInfo<Value>& args) {
289+
static constexpr size_t kMaxSyncLength = 4096;
290+
static constexpr size_t kMaxEntryCount = 4;
291+
292+
Environment* env = Environment::GetCurrent(args);
293+
CHECK(args.IsConstructCall());
294+
CHECK(args[0]->IsObject());
295+
CHECK(Blob::HasInstance(env, args[0]));
296+
297+
Blob* blob;
298+
ASSIGN_OR_RETURN_UNWRAP(&blob, args[0]);
299+
300+
// This is a fairly arbitrary heuristic. We want to avoid deferring to
301+
// the threadpool if the amount of data being copied is small and there
302+
// aren't that many entries to copy.
303+
FixedSizeBlobCopyJob::Mode mode =
304+
(blob->length() < kMaxSyncLength &&
305+
blob->entries().size() < kMaxEntryCount) ?
306+
FixedSizeBlobCopyJob::Mode::SYNC :
307+
FixedSizeBlobCopyJob::Mode::ASYNC;
308+
309+
new FixedSizeBlobCopyJob(env, args.This(), blob, mode);
310+
}
311+
312+
void FixedSizeBlobCopyJob::Run(const FunctionCallbackInfo<Value>& args) {
313+
Environment* env = Environment::GetCurrent(args);
314+
FixedSizeBlobCopyJob* job;
315+
ASSIGN_OR_RETURN_UNWRAP(&job, args.Holder());
316+
if (job->mode() == FixedSizeBlobCopyJob::Mode::ASYNC)
317+
return job->ScheduleWork();
318+
319+
job->DoThreadPoolWork();
320+
args.GetReturnValue().Set(
321+
ArrayBuffer::New(env->isolate(), job->destination_));
322+
}
323+
324+
void FixedSizeBlobCopyJob::RegisterExternalReferences(
325+
ExternalReferenceRegistry* registry) {
326+
registry->Register(New);
327+
registry->Register(Run);
328+
}
329+
330+
void Blob::RegisterExternalReferences(ExternalReferenceRegistry* registry) {
331+
registry->Register(Blob::New);
332+
registry->Register(Blob::ToArrayBuffer);
333+
registry->Register(Blob::ToSlice);
334+
}
335+
336+
} // namespace node

‎src/node_blob.h

+137
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
#ifndef SRC_NODE_BLOB_H_
2+
#define SRC_NODE_BLOB_H_
3+
4+
#if defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS
5+
6+
#include "async_wrap.h"
7+
#include "base_object.h"
8+
#include "env.h"
9+
#include "memory_tracker.h"
10+
#include "node_internals.h"
11+
#include "node_worker.h"
12+
#include "v8.h"
13+
14+
#include <vector>
15+
16+
namespace node {
17+
18+
struct BlobEntry {
19+
std::shared_ptr<v8::BackingStore> store;
20+
size_t length;
21+
size_t offset;
22+
};
23+
24+
class Blob : public BaseObject {
25+
public:
26+
static void RegisterExternalReferences(
27+
ExternalReferenceRegistry* registry);
28+
static void Initialize(Environment* env, v8::Local<v8::Object> target);
29+
30+
static void New(const v8::FunctionCallbackInfo<v8::Value>& args);
31+
static void ToArrayBuffer(const v8::FunctionCallbackInfo<v8::Value>& args);
32+
static void ToSlice(const v8::FunctionCallbackInfo<v8::Value>& args);
33+
34+
static v8::Local<v8::FunctionTemplate> GetConstructorTemplate(
35+
Environment* env);
36+
37+
static BaseObjectPtr<Blob> Create(
38+
Environment* env,
39+
const std::vector<BlobEntry> store,
40+
size_t length);
41+
42+
static bool HasInstance(Environment* env, v8::Local<v8::Value> object);
43+
44+
const std::vector<BlobEntry> entries() const {
45+
return store_;
46+
}
47+
48+
void MemoryInfo(MemoryTracker* tracker) const override;
49+
SET_MEMORY_INFO_NAME(Blob);
50+
SET_SELF_SIZE(Blob);
51+
52+
// Copies the contents of the Blob into an ArrayBuffer.
53+
v8::MaybeLocal<v8::Value> GetArrayBuffer(Environment* env);
54+
55+
BaseObjectPtr<Blob> Slice(Environment* env, size_t start, size_t end);
56+
57+
inline size_t length() const { return length_; }
58+
59+
class BlobTransferData : public worker::TransferData {
60+
public:
61+
explicit BlobTransferData(
62+
const std::vector<BlobEntry>& store,
63+
size_t length)
64+
: store_(store),
65+
length_(length) {}
66+
67+
BaseObjectPtr<BaseObject> Deserialize(
68+
Environment* env,
69+
v8::Local<v8::Context> context,
70+
std::unique_ptr<worker::TransferData> self) override;
71+
72+
SET_MEMORY_INFO_NAME(BlobTransferData)
73+
SET_SELF_SIZE(BlobTransferData)
74+
SET_NO_MEMORY_INFO()
75+
76+
private:
77+
std::vector<BlobEntry> store_;
78+
size_t length_ = 0;
79+
};
80+
81+
BaseObject::TransferMode GetTransferMode() const override;
82+
std::unique_ptr<worker::TransferData> CloneForMessaging() const override;
83+
84+
Blob(
85+
Environment* env,
86+
v8::Local<v8::Object> obj,
87+
const std::vector<BlobEntry>& store,
88+
size_t length);
89+
90+
private:
91+
std::vector<BlobEntry> store_;
92+
size_t length_ = 0;
93+
};
94+
95+
class FixedSizeBlobCopyJob : public AsyncWrap, public ThreadPoolWork {
96+
public:
97+
enum class Mode {
98+
SYNC,
99+
ASYNC
100+
};
101+
102+
static void RegisterExternalReferences(
103+
ExternalReferenceRegistry* registry);
104+
static void Initialize(Environment* env, v8::Local<v8::Object> target);
105+
static void New(const v8::FunctionCallbackInfo<v8::Value>& args);
106+
static void Run(const v8::FunctionCallbackInfo<v8::Value>& args);
107+
108+
bool IsNotIndicativeOfMemoryLeakAtExit() const override {
109+
return true;
110+
}
111+
112+
void DoThreadPoolWork() override;
113+
void AfterThreadPoolWork(int status) override;
114+
115+
Mode mode() const { return mode_; }
116+
117+
void MemoryInfo(MemoryTracker* tracker) const override;
118+
SET_MEMORY_INFO_NAME(FixedSizeBlobCopyJob)
119+
SET_SELF_SIZE(FixedSizeBlobCopyJob)
120+
121+
private:
122+
FixedSizeBlobCopyJob(
123+
Environment* env,
124+
v8::Local<v8::Object> object,
125+
Blob* blob,
126+
Mode mode = Mode::ASYNC);
127+
128+
Mode mode_;
129+
std::vector<BlobEntry> source_;
130+
std::shared_ptr<v8::BackingStore> destination_;
131+
size_t length_ = 0;
132+
};
133+
134+
} // namespace node
135+
136+
#endif // defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS
137+
#endif // SRC_NODE_BLOB_H_

‎src/node_buffer.cc

+6
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
#include "node_buffer.h"
2323
#include "allocated_buffer-inl.h"
2424
#include "node.h"
25+
#include "node_blob.h"
2526
#include "node_errors.h"
2627
#include "node_external_reference.h"
2728
#include "node_internals.h"
@@ -1199,6 +1200,8 @@ void Initialize(Local<Object> target,
11991200
env->SetMethod(target, "utf8Write", StringWrite<UTF8>);
12001201

12011202
env->SetMethod(target, "getZeroFillToggle", GetZeroFillToggle);
1203+
1204+
Blob::Initialize(env, target);
12021205
}
12031206

12041207
} // anonymous namespace
@@ -1239,6 +1242,9 @@ void RegisterExternalReferences(ExternalReferenceRegistry* registry) {
12391242
registry->Register(StringWrite<UCS2>);
12401243
registry->Register(StringWrite<UTF8>);
12411244
registry->Register(GetZeroFillToggle);
1245+
1246+
Blob::RegisterExternalReferences(registry);
1247+
FixedSizeBlobCopyJob::RegisterExternalReferences(registry);
12421248
}
12431249

12441250
} // namespace Buffer

‎test/parallel/test-blob.js

+186
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
'use strict';
2+
3+
const common = require('../common');
4+
const assert = require('assert');
5+
const { Blob } = require('buffer');
6+
7+
{
8+
const b = new Blob();
9+
assert.strictEqual(b.size, 0);
10+
assert.strictEqual(b.type, '');
11+
}
12+
13+
assert.throws(() => new Blob(false), {
14+
code: 'ERR_INVALID_ARG_TYPE'
15+
});
16+
17+
assert.throws(() => new Blob('hello'), {
18+
code: 'ERR_INVALID_ARG_TYPE'
19+
});
20+
21+
assert.throws(() => new Blob({}), {
22+
code: 'ERR_INVALID_ARG_TYPE'
23+
});
24+
25+
assert.throws(() => new Blob(['test', 1]), {
26+
code: 'ERR_INVALID_ARG_TYPE'
27+
});
28+
29+
{
30+
const b = new Blob([]);
31+
assert(b);
32+
assert.strictEqual(b.size, 0);
33+
assert.strictEqual(b.type, '');
34+
35+
b.arrayBuffer().then(common.mustCall((ab) => {
36+
assert.deepStrictEqual(ab, new ArrayBuffer(0));
37+
}));
38+
b.text().then(common.mustCall((text) => {
39+
assert.strictEqual(text, '');
40+
}));
41+
const c = b.slice();
42+
assert.strictEqual(c.size, 0);
43+
}
44+
45+
{
46+
assert.throws(() => new Blob([], { type: 1 }), {
47+
code: 'ERR_INVALID_ARG_TYPE'
48+
});
49+
assert.throws(() => new Blob([], { type: false }), {
50+
code: 'ERR_INVALID_ARG_TYPE'
51+
});
52+
assert.throws(() => new Blob([], { type: {} }), {
53+
code: 'ERR_INVALID_ARG_TYPE'
54+
});
55+
}
56+
57+
{
58+
const b = new Blob(['616263'], { encoding: 'hex', type: 'foo' });
59+
assert.strictEqual(b.size, 3);
60+
assert.strictEqual(b.type, 'foo');
61+
b.text().then(common.mustCall((text) => {
62+
assert.strictEqual(text, 'abc');
63+
}));
64+
}
65+
66+
{
67+
const b = new Blob([Buffer.from('abc')]);
68+
assert.strictEqual(b.size, 3);
69+
b.text().then(common.mustCall((text) => {
70+
assert.strictEqual(text, 'abc');
71+
}));
72+
}
73+
74+
{
75+
const b = new Blob([new ArrayBuffer(3)]);
76+
assert.strictEqual(b.size, 3);
77+
b.text().then(common.mustCall((text) => {
78+
assert.strictEqual(text, '\0\0\0');
79+
}));
80+
}
81+
82+
{
83+
const b = new Blob([new Uint8Array(3)]);
84+
assert.strictEqual(b.size, 3);
85+
b.text().then(common.mustCall((text) => {
86+
assert.strictEqual(text, '\0\0\0');
87+
}));
88+
}
89+
90+
{
91+
const b = new Blob([new Blob(['abc'])]);
92+
assert.strictEqual(b.size, 3);
93+
b.text().then(common.mustCall((text) => {
94+
assert.strictEqual(text, 'abc');
95+
}));
96+
}
97+
98+
{
99+
const b = new Blob(['hello', Buffer.from('world')]);
100+
assert.strictEqual(b.size, 10);
101+
b.text().then(common.mustCall((text) => {
102+
assert.strictEqual(text, 'helloworld');
103+
}));
104+
}
105+
106+
{
107+
const b = new Blob(
108+
[
109+
'h',
110+
'e',
111+
'l',
112+
'lo',
113+
Buffer.from('world')
114+
]);
115+
assert.strictEqual(b.size, 10);
116+
b.text().then(common.mustCall((text) => {
117+
assert.strictEqual(text, 'helloworld');
118+
}));
119+
}
120+
121+
{
122+
const b = new Blob(['hello', Buffer.from('world')]);
123+
assert.strictEqual(b.size, 10);
124+
assert.strictEqual(b.type, '');
125+
126+
const c = b.slice(1, -1, 'foo');
127+
assert.strictEqual(c.type, 'foo');
128+
c.text().then(common.mustCall((text) => {
129+
assert.strictEqual(text, 'elloworl');
130+
}));
131+
132+
const d = c.slice(1, -1);
133+
d.text().then(common.mustCall((text) => {
134+
assert.strictEqual(text, 'llowor');
135+
}));
136+
137+
const e = d.slice(1, -1);
138+
e.text().then(common.mustCall((text) => {
139+
assert.strictEqual(text, 'lowo');
140+
}));
141+
142+
const f = e.slice(1, -1);
143+
f.text().then(common.mustCall((text) => {
144+
assert.strictEqual(text, 'ow');
145+
}));
146+
147+
const g = f.slice(1, -1);
148+
assert.strictEqual(g.type, 'foo');
149+
g.text().then(common.mustCall((text) => {
150+
assert.strictEqual(text, '');
151+
}));
152+
153+
assert.strictEqual(b.size, 10);
154+
assert.strictEqual(b.type, '');
155+
156+
assert.throws(() => b.slice(-1, 1), {
157+
code: 'ERR_OUT_OF_RANGE'
158+
});
159+
assert.throws(() => b.slice(1, 100), {
160+
code: 'ERR_OUT_OF_RANGE'
161+
});
162+
163+
assert.throws(() => b.slice(1, 2, false), {
164+
code: 'ERR_INVALID_ARG_TYPE'
165+
});
166+
}
167+
168+
{
169+
const b = new Blob([Buffer.from('hello'), Buffer.from('world')]);
170+
const mc = new MessageChannel();
171+
mc.port1.onmessage = common.mustCall(({ data }) => {
172+
data.text().then(common.mustCall((text) => {
173+
assert.strictEqual(text, 'helloworld');
174+
}));
175+
mc.port1.close();
176+
});
177+
mc.port2.postMessage(b);
178+
b.text().then(common.mustCall((text) => {
179+
assert.strictEqual(text, 'helloworld');
180+
}));
181+
}
182+
183+
{
184+
const b = new Blob(['hello'], { type: '\x01' });
185+
assert.strictEqual(b.type, '');
186+
}

‎test/parallel/test-bootstrap-modules.js

+1
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,7 @@ const expectedModules = new Set([
101101
'NativeModule internal/vm/module',
102102
'NativeModule internal/worker/io',
103103
'NativeModule internal/worker/js_transferable',
104+
'NativeModule internal/blob',
104105
'NativeModule path',
105106
'NativeModule stream',
106107
'NativeModule timers',

‎test/sequential/test-async-wrap-getasyncid.js

+1
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,7 @@ const { getSystemErrorName } = require('util');
6969
delete providers.ELDHISTOGRAM;
7070
delete providers.SIGINTWATCHDOG;
7171
delete providers.WORKERHEAPSNAPSHOT;
72+
delete providers.FIXEDSIZEBLOBCOPY;
7273

7374
const objKeys = Object.keys(providers);
7475
if (objKeys.length > 0)

‎tools/doc/type-parser.js

+2
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,8 @@ const customTypesMap = {
4040
'WebAssembly.Instance':
4141
`${jsDocPrefix}Reference/Global_Objects/WebAssembly/Instance`,
4242

43+
'Blob': 'buffer.html#buffer_class_blob',
44+
4345
'BroadcastChannel':
4446
'worker_threads.html#worker_threads_class_broadcastchannel_' +
4547
'extends_eventtarget',

0 commit comments

Comments
 (0)
Please sign in to comment.