Amazing how the old is new again. Welcome back, CGI!
I hope everyone's aware that this is just a toy and should never be used to do any real work, because most command line tools were never written to be exposed to the internet at large.
tcpserver does not pipe input. It sets certain environmental variables, closes fd's 0 and 1 and runs another command line program. It has no quivalent to QUERY_STRING. Very different from CGI or this Websocket contraption. Much safer.
But the parent says that "most command line tools were never written to be exposed to the internet at large", and tcpserver is a command line tool that runs other command line tools, so is tcpserver OK to use for "real work" by his standards? Maybe we should use a GUI tool "written to be exposed to the internet at large"?
Whats so toy-like about this? Its no toy but rather a useful method of tying differing software systems together .. I can think of many productive uses for this.
Problem is that it requires that the app handle all the security aspects of talking to the Internet, because STDIN/STDOUT is an unrestricted protocol. Normally when you write a webapp you have a well-tested parser to validate the HTTP request for you, and you have safe APIs to access parameters, and your templating language auto-escapes dangerous characters for you, and the response goes through middleware to prevent XSRF attacks, and the whole thing is written in a safe language where there's no possibility of buffer overruns.
With this, you have to handle all of that in your application binary. You can certainly do this. However, if you are asking why you need to do this, you are almost certainly not ready to do this.
I agree that this is a pretty useful method of tying software systems together. However, one of the the systems you're tying together is the public Internet, where all sorts of danger lurks. Writing secure parsers is hard. I've written an HTML parser [1], one of the most well-tested ones out there (I spent the better part of a year doing nothing but testing & debugging, and ran literally billions of documents through it), and Google's security team still found 2 bugs in it with about a day's work.
> Normally when you write a webapp you have a well-tested parser to validate the HTTP request for you
The only HTTP in WebSockets is the handshake, which is what this daemon handles for you.
> and you have safe APIs to access parameters,
There are no "parameters"; WebSockets transport a stream of opaque packets.
> and your templating language auto-escapes dangerous characters for you,
There are no "unsafe characters" in a binary protocol. At best you could claim NUL and LF, the former because it has special meaning to C, and the latter because it's used as a message delimiter by this daemon. The worst either of those will cause is reading less of a message instead of more.
> and the response goes through middleware to prevent XSRF attacks
You have to do this in any language you use. This has nothing to do with communicating via Unix pipes.
> With this, you have to handle all of that in your application binary.
Most HTML parsing engines are available as libraries.
I think you're missing the forest of his argument through the trees of the details you quote.
It's oftentimes really dangerous to expose the raw stdin/stdout to the web. The point is that this by default has none of the usual safeties associated with making a program web-accessible. It's dangerous.
This doesn't make it a bad tool. It just means you need to be aware of the downsides.
If he said, "it's dangerous to connect a non-Web-aware program to the Internet", I'd agree.
Except he didn't, he said, "Problem is that it requires that the app handle all the security aspects of talking to the Internet", which is not unique to applications using this daemon, or even to applications using WebSockets.
Thanks for the reply, you are probably right. I've cleaned up the snark. Authoritatively stated incorrect information is somewhat of a pet peeve of mine; my day job often requires that I patiently remediate such information, so I have developed a bit of a short fuse for these things off the job.
nostrademons's comment is for me the most important comment here in this discussion. Thanks nostrademons!
Just as an illustration, from the linked page:
"Security. Gumbo was initially designed for a product that worked with trusted input files only. We're working to harden this and make sure that it behaves as expected even on malicious input, but for now, Gumbo should only be run on trusted input or within a sandbox."
Nostrademons doesn't know what he's talking about. See bct's comment, and my heavily-downvoted comment. Security has nothing to do with using STDIO and everything to do with how you code your program.
If you have corrections, post them, which you did up above. My post is being upvoted because it makes people think about considerations that they didn't think about before, which is a good thing. Further details are also a good thing, but I've found - through numerous engineering projects - that if you think you've found the "right" solution and someone is just "wrong", there's usually a trade-off you missed somewhere and a concern they're considering that you're not.
Anyway, I don't think that your conclusion ("Security has nothing to do with using STDIO and everything to do with how you code your program") and my conclusion ("You can do this, but you better consider the attack surface you're opening your program up to first") are all that incompatible. My point is that most programs with an interface on STDIN are not designed out of the box to be run on untrusted, potentially malicious input, and so you will have to spend time hardening yours that far exceeds the time you spend hooking up the network protocol to STDIN.
> My point is that most programs with an interface on STDIN are not designed out of the box to be run on untrusted, potentially malicious input, and so you will have to spend time hardening yours that far exceeds the time you spend hooking up the network protocol to STDIN.
Yes, that is a valid point. However that is not what you said, which was:
> Problem is that it requires that the app handle all the security aspects of talking to the Internet
which is not unique to applications using this daemon, or even to WebSocket applications in general. That's borderline FUD. Combined with the inaccurate statements you made to back it up, your original post is simply incorrect and misleading.
> if you think you've found the "right" solution and someone is just "wrong", there's usually a trade-off you missed somewhere and a concern they're considering that you're not.
No, not if the "right" solution is backed by facts and research, and the "wrong" solution is based on demonstrably incorrect half-remembered mistruths from ten years ago. There's not really a tradeoff there.
There is no reason not to think that anyone who has written an app to use STDIO may be more likely, in fact, to have a safe string-parsing/protocol handler setup. I mean, there is no guarantee that just because an app uses STDIO, they have exposed their strlen() and its ilk to the world ..
.. and the thing is, its not the 90's any more. There are a lot of great tools that can be exposed on this interface, if you think about it, quite safely ..
tl;dr: sqlite.org and fossil-scm.org run using a simple HTTP server via inetd and some parts of the website are CGI scripts. In 2010 they served over a quarter million requests per day on a tiny VPS with only 3% CPU load.
So I've been curious about this; Nimrod has some SCGI libraries and the like built in, but I didn't know whether that would be good enough for real world usage without building a concurrent application server into my App itself. How does it handle multiple requests at once without impacting performance badly? I know that's a noob question, I struggled to find good information on SCGI/CGI for today's use cases, so I sort of wrote it off as too hard basket...
CGI (when used with suexec) is still the tool of choice for your usual low-coat shared webhosting provider, and it's still good enough if all your customers want to do is deploy Wordpress, phpBB and maybe some custom PHP and Perl CGI scripts.
Take a look at how the Leap Motion works. You connect the Leap Motion to your usb port, a demon/service interacts with the Leap Motion and opens a websocket server on localhost so that websites can interact with the hardware device without browser plugins.
Thanks lhaussknecht That's also what I thought! I don't understand the negativity here!
"Oh, this is a toy. Oh, this doesn't meet my Enterprise needs. Oh, this isn't EAL7¹ Certified, I can't run my Atombomb defense system with it."
Dear Ladies and Gentlemen, you have to admit that this is ueber useful for so many scenarios, that are out of the box, you're thinking in.
I can now hack a firebase clone, just for fun and my prototypes don't require the setup of dozens or a hundred npm packages. Besides the coolness of NodeJS, there are still some use-cases for hackers with this.
