Added ability to use websockets transport over unix sockets

[mdevaev]

This is useful if you are building a single http entrypoint from multiple sockets using nginx, and you want to restrict access to the API from the localhost. To restrict access from entrypoint side, external nginx auth_request is used.

[januscla]

Thanks for your contribution, @mdevaev! Please make sure you sign our CLA, as it's a required step before we can merge this.

[mdevaev]

@januscla accepted

[lminiero]

Thanks, this does sound like an interesting feature! Pinging @alexamirante as he may be interested in this for our own deployments as well. I'll have a look at the code later to do a review: in case we wanted to test this, any quick way to do that?

[mdevaev]

The simplest way is to configure a couple of nginx sections and force it to use a unix socket to connect to the janus backend. Next, you will be able to communicate with the api via nginx just as if it was exposed by janus itself via tcp. Here the small example for nginx config (run janus on ws_unix="/run/janus.sock", make sure that nginx has rights to this socket, for simplicity, you can do after running chmod 777 /run/janus.sock)

Upstream for the serve sectionr:

upstream janus {
    server unix:/run/janus.sock fail_timeout=0s max_fails=0;
}

Location:

location /janus {
    proxy_pass http://janus;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Scheme $scheme;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Port $server_port;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_connect_timeout 7d;
    proxy_send_timeout 7d;
    proxy_read_timeout 7d;
}

Maybe websocat can be used to access a unix-socket, but I haven't tried.

[mdevaev]

Let me ask a detached question :) I was a little surprised by the functionality of pfunix transport and thought it was pure http, not a custom protocol. I didn't find any examples with its application, maybe I did not search well? Is it used only for your internal needs?

[lminiero]

Let me ask a detached question :) I was a little surprised by the functionality of pfunix transport and thought it was pure http, not a custom protocol. I didn't find any examples with its application, maybe I did not search well? Is it used only for your internal needs?

I don't remember how that came out, but it looked like an easy way to control a Janus instance locally. Notice it only works with SOCK_SEQPACKET and SOCK_DGRAM, which explains why we didn't need to add a new framing protocol on top of that: we could just treat each message as a separate request/response/event, which felt more lightweight. Not sure if anyone is using it in production, though (we aren't).

[mdevaev]

I see. Thanks for the explanation. I think using web sockets over unix socket will make this even easier. By the way, the same can be done for rest transport. Http over unix socket is not a very common thing, but it has its own specific niche. For example, docker, when used locally, opens a unix socket instead of tcp for its http api through which the docker cli works.

It can even be used together with curl out of the box: curl --unix-socket blabla.sock http://localhost.

[lminiero]

@januscla accepted

@mdevaev I don't see your name in the list. Did you just authenticate with Github? After that there's a form to compile at the end of the page. Please let me know if you're encountering issues there.

I plan to make some tests today, but won't be able to merge unless the CLA is signed.

[mdevaev]

@lminiero looks like there was a glitch last time. I did it again.

[lminiero]

Thanks, I see it now!

That said, I tried launching Janus with one of your new settings, uncommenting:

ws_unix = "/run/ws.sock"		# Use WebSocket server over UNIX socket instead of TCP

but it fails:

[FATAL] [transports/janus_websockets.c:janus_websockets_init:679] Error creating vhost for WebSockets server...

which is the result of the lws_create_vhost. I think my libwebsockets version does support the feature, as otherwise I'd be shown one of the warnings you added. Any idea on what may be failing here?

[mdevaev]

Any other errors? What is your libwebsockets version?
Try enabling all libwebsockets messages including debug

[lminiero]

Nope, that's the only error I see, no other warning either. I'm using libwebsockets 4.1.2, the version available in Fedora 33. Pinging @atoppi to see if he gets the same behaviour (I think he also has lws 4.x but manually compiled).

[lminiero]

Try enabling all libwebsockets messages including debug

Good point, that printed:

[libwebsockets][ERR] ERROR on binding fd 19 to "/run/ws.sock" (-1 13)
[libwebsockets][ERR] VH default: iface /run/ws.sock port 0 DOESN'T EXIST
[libwebsockets][ERR] init server failed

which made me realize it was a permission error (I always run Janus as my user, not root). Setting a different path in /tmp fixed that. Sorry for the noise! Now to make some actual tests 😄

[mdevaev]

👌

[lminiero]

Seems to be working nicely in the nodejs test that I made (the ws package supports unix sockets). I noticed that disconnecting from the unix socket reported the disconnection in Janus as well, which answered one of the questions I had. I couldn't test secure websockets with unix sockets, because apparently ws can't do those (or at least I couldn't find a way to do that in my 2 minutes search), but I don't see why they shouldn't work for those clients that do support them.

Thansk, merging! 👍

[mdevaev]

Great!