He said he considered java (there have been attempts to write drivers in java) and python. Python didn't make it because of high ram usage (java wasn't any better). He didn't look into ruby, but "not for any particular reason". Probably because embedding ruby is quite hard compared to python or lua.
As someone who once threw together a Lua kernel module for FreeBSD 4 just for fun, I approve of the idea -- although I don't know how useful it is.
Being able to rapidly prototype kernelspace things is probably the main advantage. I'd think twice about actually running garbage-collected code in the kernel in production. And the slide that touts that it's easy for users to modify -- yes, well -- you could easily argue that it shouldn't be easy for users to modify what's in the kernel :)
I've seen this on an embedded product that has been sold for years.
The license is liberal, it's remarkably easy to work with Lua, is can basically call any of your C functions. Great for debugging stuff, prototyping, certain kinds of monkey patches. Great for doing unit test like experiments too; you can have some Lua put the system in to a particular state and then do your experiment.
How would you compare it with python for things like unit testing c code? Here lua is chosen because of it's small footprint, but for testing that doesn't really matter.
Yea, I agree that it's neat, but probably of limited practical value.
If you need something like this, it's probably best to either host a kernel in userspace (like UML), or else have a lightweight kernel interface that allows you to do your heavy lifting in userspace. I've never done any Plan9 driver development, but my understanding is that Plan9 has userspace equivalents of most of the internal driver API calls, so that a driver module can almost seamlessly be developed as a userspace driver and then (if it turns out to be performance critical) run as an in-kernel module in production.
Once you've got good userspace APIs for the kernel APIs you need to use, then all kinds of rapid prototyping possibilities open up.
Lua sometimes drives me nuts, but it's such a humble little language that you can't hate it for too long. There are alternatives I would have preferred, like the Lua-inspired Squirrel (http://http://squirrel-lang.org ), but Lua is the most well-known and is written in plain C, which is likely appealing to a BSD project. I'm interested to see if this gains any popularity and if the other BSDs show interest.
Arrays based on 1. The odd and slightly under-documented stack that you have to use when interfacing it with C. Global variable scope by default. Tables (.. are both good and bad). Not standard named operators like '=~' and ':'.
Having said that, it's not a bad language. It has an excellent community, who really went to great lengths to help me writing the Lua libguestfs bindings.
The 1-based arrays are theoretically annoying (I'm a C guy), but in practice I find they simply aren't an issue, I guess due to the differences in typical programs for each language. "~=" is unusual (I too wish it used "!=") but other than that, Lua's pretty much like everybody else (":" doesn't really count because it's an operation that doesn't exist in most languages).
"Global by default" can be annoying, but at least it's better than "local by default" like python etc. [As a programmer, who'd rather find bugs easily than save some typing, I think better than either would be "nothing by default, you gotta declare your variables", but that might not be popular for "dash off a quick script" usage; maybe make it configurable based on the load context or something?]
But this:
> The odd and slightly under-documented stack that you have to use when interfacing it with C
... seems almost bizarre. Lua's C interface is probably the most elegant and usable I've ever seen. A stack is the classic abstraction to use for this application, Lua's implementation is well-designed, and Lua's API documentation is both comprehensive and very well written.
It's annoying building a class system by hand. I've got C functions I wrote to do it for me, but I wish I didn't have to.
There are certain behaviors I dislike even more. One example is a historical quirk in which all non-nil or non-false expressions are true. This means inconsistent behavior like:
while 0 do print("Loops forever") end
while not 1 do print("Does nothing") end
and:
while 1 do print("Loops forever") end
while not 0 do print("Does nothing") end
I also don't like how undefined variables return nil. This can cause a host of subtle errors, including accidentally creating sparse arrays if you make a typo in a variable name, which will mess up the behavior of the length operator.
The syntax, which is what most people complain about in Lua, really doesn't bother me. It's the sometimes unconventional behavior of the language and its minimal feature set.
The only things that evaluate to false are nil and false, so applying the "not" operator to a true value (non-nil, non-false) will evaluate to false. Hence, "not 1" is false, as is "not 0".
There are operating systems written in system programming languages with GC. Check Native Oberon and A2 for desktop operating systems used at Zurich's university.
Smalltalk was originally developed as OS. Only the so called primitives were done in Assembly, with the remaining parts in Smalltalk itself.
Sun played around with Java in Solaris for device drivers and Squawk is a VM that runs directly on hardware with device drivers written in Java, as an example.
It's very small, fast, and uses little memory. It's easy to learn and easy to interop with C. It's got two great implementations, with one (LuaJIT) probably holding the top spot among all dynamic, garbage-collected language currently in existence, measured by performance (although the creator -- `mikep` on HN -- is very careful to avoid making this sort of claim).
It's also got one of the coolest GUI toolkits around in IUP. If you've ever wondered by a cross-platform GUI toolkit that uses native widgets has to be so huge and complex, IUP's what you're looking for. Unfortunately, IUP doesn't yet have a native OSX driver, but you can use motif graphics driver to build a program that will run on OSX. Like Lua, IUP's in portable C, so a number of other languages have built bindings, notably go and perl. Like Lua, IUP has a very liberal license license (MIT), so you can use it for commercial projects without worrying about dynamic or static linking.
Thanks for the correction. I should have verified the HN username for Mike; not sure where I pulled `mikep` out of since I always enjoy reading his comments here and elsewhere. It's too late for me to correct it in my original comment, so thanks again.
IUP seems cool in theory, but in practice, I've never even been able to compile/install it, despite trying multiple times over the years! Unfortunately the build system seems to be mostly of the "Works great on the author's system!" sort... ><
I always try a bit to hack around the issues, but the problems have always exceeded my "oh this is too annoying, I'll try again next year" threshold (part of the problem being that it's an unusual build system, not just a few Makefiles, so it's less familiar).
I just noticed your comment, so sorry for the belated reply. I just wanted to mention that the tecgraf, makers of IUP, has thankfully simplified their build system -- no more custom build system. Also, they've started releasing pre-built binaries for all major OSs, including Linux and OSX. So now it's as easy and downloading a file and running sudo ./install. And if you want to build it yourself, it's as simple as running a few make files (make; make install each for the IUP, CD, and IM libraries).
But absolutely, the first time I tried to use IUP a few years ago, I couldn't get it build due to the complexity of the build system. Fortunately, that's all changed now.
Small, fast, and easily embeddable in C programs. It's been popular in videogames for years but hasn't seen much usage outside that industry, so it's always interesting to see it used in different ways.
It's not the language itself. It's its implementation. Embedding a Lua interpreter in a C program is straightforward. See the 'Python' and 'Java' slides in the OP: it's all about integration.
Javascript is popular outside the browser not because it's a great programming language, but because it gets us back to where we were before the browser became the dominant app delivery platform: it's now again possible to write your entire client-server app in one language.
If you're talking Node.js, sure, but that was a stepping stone. It's now being used for all sorts of weird things, like WinRT, and as GNOME's flagship desktop app language. These need not have any server-side components.
A large part is obviously that JavaScript is now well known and many developers have been exposed to it, but also because a minimal prototype language with first class functions is often all you need or want, just like Lua. (I assume this has been realized after exposure to JavaScript, or maybe I'm giving JS devs too much credit and they're not fans of minimal and clean languages.)
This is a tweaked/updated version of a stackoverflow answer I wrote (see end for link) to someone wanting to know what language they should use for embedding:
Lua, and Lua's standard implementation(s), capture a wonderful balance of features/smallness/speed/usability, particularly for embedded use.
Lua:
1. Is very small, both source and binary, an order of magnitude or more smaller than many more popular languages (Python etc). Because the Lua source code is so small and simple, it's perfectly reasonable to just include the entire Lua implementation in your source tree, if you want to avoid adding an external dependency.
2. Is very fast. The Lua interpreter is much faster than most scripting languages (again, an order of magnitude is not uncommon), and LuaJIT2 is a very good JIT compiler for some popular CPU architectures (x86, arm, mips, ppc). Using LuaJIT can often speed things up by another order of magnitude, and in many cases, the result approaches the speed of C. LuaJIT is also a "drop-in" replacement for standard Lua: no application or user code changes are required to use it.
3. Has LPEG. LPEG is a "Parsing Expression Grammar" library for Lua, which allows very easy, powerful, and fast parsing, suitable for both large and small tasks; it's a great replacement for yacc/lex/hairy-regexps. [I wrote a parser using LPEG and LuaJIT, which is much faster than the yacc/lex parser I was trying emulate, and was very easy and straight-forward to create.] LPEG is an add-on package for Lua, but is well-worth getting (it's one source file).
4. Has a great C-interface, which makes it a pleasure to call Lua from C, or call C from Lua. For interfacing large/complex C++ libraries, one can use SWIG, or any one of a number of interface generators (one can also just use Lua's simple C interface with C++ of course).
5. Has liberal licensing ("BSD-like"), which means Lua can be embedded in proprietary projects if you wish, and is GPL-compatible for FOSS projects.
6. Is very, very elegant. It's not lisp, in that it's not based around cons-cells, but it shows clear influences from languages like scheme, with a straight-forward and attractive syntax. Like scheme (at least in it's earlier incarnations), it tends towards "minimal" but does a good job of balancing that with usability. For somebody with a lisp background (like me!), a lot about Lua will seem familiar, and "make sense", despite the differences.
7. Is very flexible, and such features as metatables allow easily integrating domain-specific types and operations.
8. Has a simple, attractive, and approachable syntax, which can be important if you intend to have end-users write scripts.
9. Is designed for embedding, and besides its small size and fast speed, has various features such as an incremental GC that make using a scripting language more viable in such contexts.
10. Has a long history, and responsible and professional developers, who have shown good judgment in how they've evolved the language over the last 2 decades.
Let me add my favorite hobby horse to your point 4: LuaJIT's C FFI is a tremendous and undervalued advantage compared to any other language/implementation I've used. Its core value proposition:
0) It can parse C header files and automatically construct bindings.
1) You can write Lua code that works directly on C data types with C memory layout and is JIT compiled. It makes it very easy to achieve performance within ~2x of C while still maintaining the expressiveness and extensibility of Lua. And if you miss writing OO C++, you can easily add methods to your C structs and they will be JIT compiled. See here http://wiki.luajit.org/Allocation-Sinking-Optimization and search for "Point Class with FFI cdata Struct" and then "Point Class Benchmarks".
2) Because the memory layout matches C data types, it is trivial to swap out a Lua implementation of a core operation with a C implementation if that becomes necessary.
3) It is easy to write cross-platform systems code without conditional compilation because the native calls through FFI can all be constructed dynamically through Lua (so instead of #ifdef WIN32 all over your socket code, you just have a single line checking for ffi.os == "windows" and then if necessary call the winsock initializer as ffi.C.WSAStartup(foo,bar)).
Indeed LuaJIT's FFI is a very cool and usable, but it does one huge drawback: it basically ties you to LuaJIT.
I'm not trying to diss LuaJIT—it's a wonderful and impressive JIT implementation, and for some applications, irreplaceable—but compared to the standard Lua implementation, LuaJIT is (1) less portable, (2) more complicated, and (3) has various implementation drawbacks.† In addition, of course, there are other alternate Lua implementations which have their own set of tradeoffs.
Essentially, it's very nice to be able to switch Lua implementations depending on the circumstance. With normal usage, LuaJIT is carefully designed to make this possible (indeed, easy: one need only change the library one links against, it's not even necessary to recompile one's app), but once you start using things like FFI, that ability is lost.
There's a port of LuaJIT's FFI interface to standard Lua, but it's also not portable, and because it will have very different performance characteristics than FFI in LuaJIT (which can "compile out" FFI accesses), it's not really a drop-in replacement for it, and I think not nearly as compelling.
† The one that's bitten me in the past is that LuaJIT has a much smaller limit on addressable memory than standard Lua because of the particular design of its nan-encoded object representation. There are some input files for my Lua-using app that will only work when I compile with standard Lua, because using LuaJIT will run out of memory; those are times when I'm very glad I didn't commit to a LuaJIT-only design...
He said he considered java (there have been attempts to write drivers in java) and python. Python didn't make it because of high ram usage (java wasn't any better). He didn't look into ruby, but "not for any particular reason". Probably because embedding ruby is quite hard compared to python or lua.
Very interesting talk!