I love the idea, I've also been frustrated, and there has been chicken and egg problems which this may help with. That said, the quote:
> "And lastly, because we're fed up with having to install half-an-universe's worth of Ruby, Python and Perl to bootstrap our servers or run CI jobs"
If this takes off, some will use it some won't, and we'll be stuck installing a "half-an-universe's worth of Ruby, Python, Perl, and Shellfire scripts to bootstrap our servers or run CI jobs"
Perhaps - I hadn't yet even thought of dreaming the similitude of it taking off. Thanks (;-). But shellfire wouldn't be as invasive. Each shellfire script stands alone, a bit like a statically-compiled C executable. Of course, there are still call outs to third-party binaries - but that ordinarily shouldn't be much more than busybox or coreutils, grep, awk and sed, with may be curl. And one can, with shellfire snippets, even embed binaries in the shell script... although whether that's wise is another discussion.
The proof in the pudding will be trying to make libertine linux use it - a not even started project of mine to build a minimal, net-bootable linux with an immutable file system (upgrade == reboot a new image).
Yeah, I'm ambivalent about this. (For some background, I used to hate Perl until I had to write some non-trivial shell scripts, then I realized Perl was a godsend compared to shell).
We've long been at the point where you can expect Perl and Python to be installed (Perl is required by the Debian's base tools, and Python by Redhat's).
You may not have the libraries you need at the version you need, and it's a shame that neither system has a convenient way to address that (Python virtualenv remains a hacky solution IMHO).
I was initially enthused by Shellfire's offering but, come to think of it, all it's doing is bringing SH scripting to some level with Perl or Python... but, as you point out, we have those already.
> If the job requires one page of code or less, use shell. If it requires more than one page of code, use Python or whatever. Simple as that.
The problem with this simple guideline is that tools grow, so what was well less than a page becomes much more—and each small increment is too small to justify a complete re-write, so eventually you wind up with a huge shell-script monster.
That's mostly my heuristic as well, except there's a sweet spot for Perl somewhere in the middle.
Python has a upgrade problem. We're still running a bunch of CentOS 5.5 machines which depend on Python 2.4. In my org, we still have scripts littered around that expect a later version of python, but are executed from just "#!/usr/bin/python" instead of "python2.7". which fails on those old CentOS machines, on which you can't just set a newer version of Python as default because that'll break many system utilities.
Also, Perl's syntactic sugar for regular expressions turns out to be very useful (and I say this after years of grumbling against it).
It's not path-relative: I can't copy an env from one dir to another. It's an external tool that you need to enable separately from your execution, when really it should've been part of Python's base binary (resolving libs from the CWD to user libpath to system libpath), to name just a couple issues.
virtualenv appears to be a giant, epic work-around for... what exactly? Somehow, perl managed to achieve the same thing with the PERL5LIB envar. And did it in a far more portable, reusable and repeatable way.
Exactly. I love Python for many other reasons, but how they still haven't managed to fix local vs user vs system libraries, when pretty much every other language has, and think virtualenv is an acceptable solution, is beyond me.
Of course, since they have many geniuses working on their codebase, I assume the problem must actually be pretty fundamental and tricky.... right?
What I'm worried about this is that it builds on shell precisely. So to debug it / understand how it behaves, you need to understand shell (I suppose bash for most people) + shellfire. bash is already full of pitfalls, I'm not very enthused of adding a layer on top. While it certainly helps when you know what you're doing / it works; it obscures the cause of some already obscure behaviour even further.
I also don't like the "framework" approach. I actually thought this would be actual "shell functions" to do all kinds of common shell tasks; a copy-paste library of sorts. I was actually pretty excited about that as it is something I see myself using on a regular basis.
But don't let my negativity bring anybody down, I'm sure it can be useful; and if it's useful for you, just use it and be happy.
Thank you. It's almost impossible to have a set of 'shell functions' that are entirely independent of one another, once you move beyond a few special cases. That said, this is as near to copy and paste as you can get - it's Git based, and the idea is you just add groups of functions as git submodules. You can test and deploy as you go, and then, when you're ready, fatten the script and release it.
The shell needs to be understood. There's no doubt it's an old looking-language to most people, but that's why we should treat it with the same discipline as anything else. I'd agree that the shell can be obscure. Wherever possible, shellfire tries to make things explicit, not obscure through good naming; hence the reams of functions on core/variable to do simple things, eg core_variable_startsWith rather than having to figure the weird looking ${var?/:} syntax.
This is pretty cool. It faciliates the use of what I've seen as a number of sore omissions from bash. I dislike the "why aren't you using a real language" mentality - if we can make bash better, why not do it?
Why not? Because building on a solid foundation makes for more reliable and robust programs. Things like Shellshock and the Valve bash bug that wiped out all your files demonstrate that the shell scripting languages have fundamental design flaws which make bugs more likely than in other languages.
I wouldn't suggest writing user-facing shell scripts, and bugs (even big ones) happen with other languages all the time. I don't want to see large pieces of software written entirely in bash, but scripting where it's useful is fine with me.
I love shell scripts because they let you distribute zero-setup portable programs without dependencies. But if you're already going to be depending on shellfire, why not instead depend on a VM for a more appropriate language?
I think posix shell is quite a beautiful language, and I enjoy hacking together a medium size application with it just because it's fun, but it has a lot of disadvantages relative to a more modern scripting language. If you use perl/python/ruby/javascript/lua/lisp instead, your application will perform better, be easier to maintain and be able to run on windows without cygwin.
I get the impression that this project is trying to advertise itself for making real applications, but I don't think that's realistic. As a toy, though, it's really cool and I'll probably play around with it at least a little.
In a more controlled environment where you are writing shell scripts all of the time, it seems like this could be something useful. For example, you could distribute it with your "dotfiles" to use in your .bashrc / .zshrc files. Or you could make sure it's installed on all of your Linux/*BSD servers.
It's obviously not so great for distributing around to random places where it probably won't be installed.
shell script and a modern language like Python or Ruby have different use cases. Both have expressiveness, and power, and all, including the shell, very different applicability.
This project is far from a toy; there's a complete MQTT client written in it, bish-bosh[1], and swaddle (see comment below). I wouldn't build a webserver in it, but I would build command line wrappers of all those REST APIs out there so I don't have to install gems and eggs and whatever else to just get stuff done. shellfire is minimally dependent.
shellfire exists because installing Ruby and Python and all their dependencies is simply not an option for minimal bare metal servers and routers. Because there are far too many 'toy' shell scripts out there that aren't robust. Because the shell is the glue we need for out big ticket applications. Because I really loathe having to depend a VM configured and setup and snapshotted.
Performance is not a goal, not should it be for any scripting language. If you need performance, write in C or perhaps Java, although V8 is getting closer all the time.
And there is not dependency. shellfire deliberately discourages the model of 'install framework in /usr/lib'. It just doesn't work. But it does support working nicely with git-based dependencies as you develop; you can even deploy as nothing more than a git export, if you so choose.
I'm a big fan of removing dependencies and very partial to distributing single dependency free executables, but don't you just need to install all the shellfire dependencies instead of Python/Ruby etc. Appreciate that you can 'fatten' apps but this is true for the other languages too (pyinstaller etc.).
No, it's not like Python/Ruby/etc dependencies. You use your own git; dependencies are submodules in your git repo. shellfire deliberately is not designed to be installed in `/usr/lib` or wherever. In this sense, it's more like Go. Going down the git route gives fine-grained control over which dependency versions one uses.
This makes development completely independent of the `/usr`, etc of the machine your own. If you clone your repo from your laptop to someone else's desktop, no need to to install deps, no need even for a working network env - valuable if you're in one of those places that breaks Ruby because they uses a Windows proxy (eg most UK Gov setups).
You don't have to fatten; you can just deploy by exporting your git repo. That sort of route would be perfectly acceptable for most enterprise work. Still no need to install deps, as everything is relatively-pathed.
It looks cool, but pretty complicated (as shell script always seems to be in real application). I'll have to come back and see what I can come up with or replace using this though.
Thank you, that's appreciated. I want it to not be complicated in use - but like anything new, there's always that wall of initial confusion. It's early days for all of it, and this is taking a very different approach with the shell: it treats it like a proper language. Arguably, one that needs much more discipline than first appears.
Everyone has a different learning style - perhaps the tutorial will help? Or just looking at a real use case - try bish-bosh at https://github.com/raphaelcohn/bish-bosh. Let me know if there's a better way of explaining it.
If there's enough folks interested, maybe we could run an interactive tutorial or IRC session?
> "And lastly, because we're fed up with having to install half-an-universe's worth of Ruby, Python and Perl to bootstrap our servers or run CI jobs"
If this takes off, some will use it some won't, and we'll be stuck installing a "half-an-universe's worth of Ruby, Python, Perl, and Shellfire scripts to bootstrap our servers or run CI jobs"