Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve documentation on how to proxy requests formdata to another fetch #2227

Closed
quentincaffeino opened this issue Aug 17, 2021 · 13 comments
Closed
Labels
documentation Improvements or additions to documentation p3-edge-case SvelteKit cannot be used in an uncommon way

Comments

@quentincaffeino
Copy link

quentincaffeino commented Aug 17, 2021

Describe the problem

When FormData is received by the endpoint inside sveltekit I can't pass it to fetch as is, it isn't sent.

Describe the proposed solution

  1. implement this functionality
  2. document on how to use fetch with correct typings, because it accepts initial FormData but does not send it. But when I use ReadableStream VSCode tells me that Type 'Readable' is not assignable to type 'BodyInit'. which is correct for browser fetch, but isn't for node-fetch.

Alternatives considered

Currently I'm doing this by parsing and creating new FormData object.

import { Readable } from 'stream';
import { FormData } from 'formdata-node';
import { FormDataEncoder } from 'form-data-encoder';

/**
 * @type {import('@sveltejs/kit').RequestHandler}
 */
export async function post({ params, headers, body }) {
	let form = new FormData();
	for (const [key, value] of body.entries()) {
		form.append(key, value);
	}

	const encoder = new FormDataEncoder(form);

	const res = await fetch(url, {
		method: 'POST',
		headers: Object.assign(
			headers,
			Array.from(Object.entries(encoder.headers)).reduce(
				(pv, cv) => ({ ...pv, [cv[0].toLocaleLowerCase()]: cv[1] }),
				{}
			)
		),
		body: Readable.from(encoder.encode()),
		credentials: 'include',
	}).catch(console.error);

	...
}

Importance

nice to have

Additional Information

No response

@JeanJPNM
Copy link
Contributor

Have you tried: body: new URLSearchParams([...body.entries()])?

@quentincaffeino
Copy link
Author

Have you tried: body: new URLSearchParams([...body.entries()])?

No I haven't, thanks, I'll try it out tomorrow. However I'm afraid it might not work since I'm passing request to an older backend which might not accept other type than form/multipart

@quentincaffeino
Copy link
Author

@JeanJPNM thanks it worked. Small note you have to convert URLSearchParams object to string so it now looks like this:
body: new URLSearchParams([...request.body.entries()]).toString()

@quentincaffeino
Copy link
Author

I found one case where URLSearchParams won't work: uploading files.

@bluwy
Copy link
Member

bluwy commented Dec 4, 2021

Besides the file upload caveat (which is tracked at #70), does @JeanJPNM's suggestion solve this issue @quentincaffeino? If so, we can close this.

@quentincaffeino
Copy link
Author

@bluwy, My issue is a bit different. I'm using some endpoints as a proxy to some other api. You can see that in my example. I'm proxying what user have sent with another fetch request. But if you think you would not work on this and I'd have to develop some kind of workaround to pass formdata then you can close it.

@JeanJPNM
Copy link
Contributor

JeanJPNM commented Dec 4, 2021

I just realized you can put as never at the assignment, since this is only a typing issue, of course there should be a better solution, but I think this is the best workaround.

@bluwy
Copy link
Member

bluwy commented Dec 4, 2021

Thanks for the update. This looks like a node-fetch limitation rather than something that can be fixed in sveltekit. So a workaround may actually be the right way for this. Regarding the types, I don't think that can be fixed, as TypeScript can't differentiate between client and server environments. But I do agree that it's weird to be hitting these issues, maybe we can document this regarding node-fetch's limitation. Feel free to send a PR!

@bluwy bluwy added documentation Improvements or additions to documentation p3-edge-case SvelteKit cannot be used in an uncommon way labels Dec 4, 2021
@quentincaffeino
Copy link
Author

Wouldn't having an access to request.body stream solve this? This way it could be passed to next fetch right away. I guess it is somewhat related to #1563 but instead of streaming response, I think we need to stream a request. Does this make sense?

@bluwy
Copy link
Member

bluwy commented Dec 5, 2021

Unfortunately that's unfamiliar territory for me, so I can't quite comment on that. But if the feature is node-specific, you could use extra middlewares and handle that special case.

@quentincaffeino
Copy link
Author

quentincaffeino commented Dec 6, 2021

Thanks, @bluwy, will look at middlewares.

But that's not only node, that could be useful for any non-static adapter. Request streaming comes handy where you have very large requests and you dont want to clutter your ram with whole request and wait for it to fully upload, this way you can start processing it right away.

I found this example for nodejs http server which might give you a better idea of how this could be used:

http.createServer(function(r, s) {
    console.log(r.method, r.url, r.headers);
    var body = "";
    r.on('readable', function() {
        body += r.read(); // here instead of concatenation we could start processing data
    });
    r.on('end', function() {
        console.log(body);
        s.write("OK"); 
        s.end(); 
    });
}).listen(42646);

Not sure yet how it could be done with the current sveltekit server implementation.

I guess old issue title is no longer representative so I'm going to update it.

@quentincaffeino quentincaffeino changed the title Ability to pass FormData inside endpoints Streaming client Request Dec 6, 2021
@quentincaffeino quentincaffeino changed the title Streaming client Request Improve documentation on how to proxy requests formdata to another fetch Dec 6, 2021
@quentincaffeino
Copy link
Author

Or even better, I will open another issue

@Rich-Harris
Copy link
Member

SvelteKit's approach to request bodies changed substantially since this issue was raised — endpoints receive a standard Request object, so you can do this sort of thing:

export async function post({ request }) {
  const res = await fetch('/other', {
    method: 'post',
    body: await request.formData();
  });

  // ...
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation p3-edge-case SvelteKit cannot be used in an uncommon way
Projects
None yet
Development

No branches or pull requests

4 participants