The more I see of what goes on under the covers, the more surprised I am that anything works at all. As an example, every gtk+ program I run spews thousands of errors to the console. Why? Are they bad? The programs seem to run anyway, never crashing more than one would expect. So … why the console spam?
I think not a single actual application is completely well formed. They all work on luck. Especially in the embedded world. Btw, my android just rebooted.
Although from hearing about the Apollo Guidance Computer, it's just as much of a miracle they were able to get it to work without any debugging or compilers, with basically one shot to write the program to ROM.
Well they put in an effort of 1400 man years for just the software. They even built an interpreter, because (flight/trajectory) requirements were changing all the time.
Also: "The highest level of testing was performed with a high fidelity digital simulation of the computer, spacecraft hardware, and mission environment."
So? If they indicate a problem or potential problem, they should be fixed in the software. The fact that there is a "warning" message means that someone, somewhere thought it serious enough to write home about. If they don't indicate a potential problem, then why even print them?
Think about how you're supposed to treat compiler warnings: You don't ignore them--they are there for a reason. Quite the opposite--you should use your compiler to treat warnings as errors, enforcing good development practice from the start.
The grandparent's comment was about the surprise and concern you feel when you see all of these warnings yet things still evidently "work". When I sit down to a new project, pull down the latest code, start to work with it and see thousands of compile-time and run-time warning messages, the alarm bells start going off in my brain: This project is barely held together.
It makes launching Gtk+ apps from the console a major annoyance since you get the warning spam at random times while you may have been doing something more important. It would be nice if they'd just quiet up and add a verbose warnings option for the devs.
I get some compiler warnings when I compile SQLite. Isn't this a problem?
Doesn't it indicate poor code quality?
Quality assurance in SQLite is done using full-coverage testing, not by
compiler warnings or other static code analysis tools. In other words, we
verify that SQLite actually gets the correct answer, not that it merely
satisfies stylistic constraints. Most of the SQLite code base is devoted
purely to testing. The SQLite test suite runs tens of thousands of separate
test cases and many of those test cases are parameterized so that hundreds
of millions of tests involving billions of SQL statements are run and
evaluated for correctness prior to every release. The developers use code
coverage tools to verify that all paths through the code are tested.
Whenever a bug is found in SQLite, new test cases are written to exhibit
the bug so that the bug cannot recur undetected in the future.
During testing, the SQLite library is compiled with special instrumentation
that allows the test scripts to simulate a wide variety of failures in
order to verify that SQLite recovers correctly. Memory allocation is
carefully tracked and no memory leaks occur, even following memory
allocation failures. A custom VFS layer is used to simulate operating
system crashes and power failures in order to ensure that transactions are
atomic across these events. A mechanism for deliberately injecting I/O
errors shows that SQLite is resilient to such malfunctions. (As an
experiment, try inducing these kinds of errors on other SQL database
engines and see what happens!)
We also run SQLite using Valgrind on Linux and verify that it detects no
problems.
Some people say that we should eliminate all warnings because benign
warnings mask real warnings that might arise in future changes. This is
true enough. But in reply, the developers observe that all warnings have
already been fixed in the builds used for SQLite development (various
versions of GCC, MSVC, and clang). Compiler warnings usually only arise
from compilers or compile-time options that the SQLite developers do not
use themselves.
Thanks for posting that, very interesting. I don't think it's entirely fair. No one should care about warnings about style (spacing, case, etc), but much more commonly warnings about correctness are important.
> Here is one example of an ironic piece of waste: Sam Leffler's graphics/libtiff is one of the 122 packages on the road to www/firefox, yet the resulting Firefox browser does not render TIFF images. For reasons I have not tried to uncover, 10 of the 122 packages need Perl and seven need Python; one of them, devel/glib20, needs both languages for reasons I cannot even imagine.
This is a better read than the linked article, really.
That's rich coming from PHK, whose software requires a full C compiler as a runtime dependency.
The reason Firefox pulls libtiff is simply because it uses a generic image manipulation library (libgdk-pixbuf), which in my system it's also used by many other applications - which do require TIFF support. I guess if you have a desktop with only FF installed that may be a waste, but how many people really have FF and not any kind of image viewer installed?
That's a very sensible requirement. It says in the docs exactly what they use it for. Not like they put in a bunch of other dependencies and some 3rd degree dependency pulled in a C compiler.
The C compiler in question pulls in a bunch of libraries for dealing with math problems (libgmp, libisl, libmpc, libmpfr) and zlib. Does their config language need support for "integer points bounded by linear constraints"? And why are they pulling zlib, when they also bundle it?
I don't see the difference. Instead of including a specific compiler for their needs, they are bazaar-ing a full generic C compiler (along with irrelevant libraries) just like Mozilla is doing with a generic image manipulation library, which pulls libtiff since many packages that also use it need TIFF support.
I mean I guess they could bundle tiny C compiler but your point about the C compiler is more about how modern distributions bundle and distribute compilers more than about varnish depending on a C compiler. You can choose to install the smallest subset of the libraries required to enable varnish to perform its job. Not like varnish explicitly requires libgmp to do bignum arithmetic. It's how your distribution of choice installs gcc and pulls in all the stuff with it.
No, gcc requires libgmp, it's not just a choice by the distribution. But even if it was, that is exactly why Firefox pulls libtiff - it's not like it explicitly requires the library, it's just how the distribution installs libgdk-pixbuf. Yet PHK uses that as an example of the waste created by the development model.
Possibly. The article is from August 2012, and Firefox started including PDF.js sometime in 2012, according to Wikipedia. I have no idea when PDF.js started supporting embedded TIFFs though.
There’s an argument to be made that silly error messages are better than crashing browsers
There's an argument, but it's a really lousy one. If you're passing NULL as a string to printf, your code is broken. A crash which results in someone tracking down and fixing the bug is far better than quietly doing something which is 100% guaranteed to be wrong.
I agree with this 100%. Crashes are a way of not-so-subtly reminding you that your code is still buggy, and exactly where it crashed is usually a good indicator of where to start debugging.
Rule of Repair: Repair what you can — but when you must fail, fail noisily and as soon as possible.
To take this example to the extreme, imagine if the default state of the runtime/OS were configured so that invalid memory accesses didn't cause segfaults, but e.g. yielded -1 or 0 or some other value and simply let the code continue... by the time you see something wrong, it's probably very far away from what originally caused it.
It's funny to see OpenBSD's printf doing this explicit null-checking and that they are considering to remove it, when AFAIK Microsoft's C library printf has always done the IMHO sane thing of just reading and writing to whatever addresses it's been given --- and if they turn out to be invalid, like null, then it crashes as it should.
Rule of Real World Developmen: Repair what you can and keep it running with kludges, because if your program fails noisily and soon, it gets uninstalled, and you get fired.
> "A crash which results in someone tracking down and fixing the bug is far better than quietly doing something which is 100% guaranteed to be wrong."
Not necessarily. Wrong is relative and can have either small or large costs, and crashing also has a cost to the user (and to the developers if they lose users.)
As an example, I play a very old video game (Descent). One of the major problems with one of the modern source ports is that the game crashed under certain conditions -- like if it received a malformed data packet. This is a game that's already fairly tolerant to lost data, so just dropping a bad packet is an acceptable result. By comparison, crashing in the middle of a competitive match is a much worse result, because it affects things like momentum and where various powerups have been left around the map.
It would be great to fix the bug that was resulting in malformed packets -- but the cost of directly affecting a high-stakes match is not worth it. I'd rather tolerate a fraction of a percent of lost data every game for the next 20 years than ever have a high-stakes match where the result was tainted by a badly-timed crash.
That's looking at it the wrong way. The bug is not that it received a malformed packet: any network-aware program must expect to receive bad data and avoid crashing or other undesirable behavior when receiving bad data.
Sure, the player's software on the other end should not have sent the malformed packet, but crashing on bad untrusted input is a bug of its own, and I'd say a worse one.
(And who knows, maybe the malformed packet wasn't caused by a bug on the other side, but by something malicious, or perhaps some sort of network-caused data corruption.)
There were several other conditions that would cause the game to crash, which were more reasonable to just play through, aside from the malformed packet handling.
The overall point here is that there is a real cost to making your software crash. Most of your users want software that is stable, and crashing is often worse than doing the wrong thing and continuing to run in a slightly janky state.
There are two use cases there that end up with a conflict of interest: One is the UX of having a crash during an important play session(high stakes match) and the other is the developer ease of tracing and debugging errors, which favors crashing over letting erroneous behavior taint the whole system.
In practice this means that during development you want the game to fail fast and loudly so that the biggest bugs are taken care of, but gradually add features to fail more quietly later, e.g. by moving more errors into logs and traces.
This is a very good strategy for getting dangerous security flaws. BIND's security has improved dramatically since they went all-out on assertions; at least 1/3 of the "crash" bugs would have been remote code execution without the protective crashing.
If it were possible to gain an advantage in a competitive match by having your system deliberately drop a packet, the game would quickly become impossible to play competitively - that kind of cheating would be basically impossible to detect because there's no way to tell it apart from a genuine bad internet connection. If you've carefully thought through what the game should do when a packet is malformed that's one thing, but if you haven't thought about it then cleanly crashing is probably the best option.
That isn't true at all: Often competitive players are required to use the tournament organizer's hardware anyways.
And often competitive players form groups to play in, so everybody knows each other even while practicing.
You'd probably want to be on a LAN to minimize the risk of this happening accidentally.
So in practice, I don't think it would matter too much even if money was on the line. You would just have to take appropriate precautions, and watch to disqualify people for cheating.
Except that it's also possible to gain an advantage by causing a game crash.
Ultimately, "crash the game" is among the worst possible options. (Also ultimately, it's a very old open-source game in which it's relatively easy to compile your own client software, so "make the game crash as an anti-cheat method" only works on honest players anyway.)
Yeah, and as browser developers have learned, process isolation is the right way to handle these errors anyway. A process crashing doesn't mean your browser has to crash.
The bogus UI message showing "(null)" is a pretty obvious flag that a bug needs to be tracked down and fixed too - potentially better than a crash, because this way you'll have a pretty decent idea where to start looking for the bug before you even fire up the debugger.
I always thought low-level programmers (C, C++, assembly, etc.) cared more about minimal abstractions and minimizing dependencies. But it turns out it's the same story no matter what level of the stack you are on. Currently working on a project with 200+ requirements in requirements.txt and growing. Before anyone says microservices just realize that you have taken the same programmers that made this mess and shoveled the problem elsewhere. These people will find a way to introduce 100+ dependencies on other services.
I always thought low-level programmers (C, C++, assembly, etc.) cared more about minimal abstractions and minimizing dependencies.
There are two types of "low-level programmers" I've encountered: there those who truly "think low-level" and enjoy it, the ones who will avoid huge amounts of abstraction and general inefficiency in favour of small, efficient and usually very simple solutions; and those who may be using a low-level language, but are actually using it from the mindset of someone who loves abstraction and libraries more than anything else. I'm in the former group.
That `requirements.txt` was probably generated by `pip freeze > requirements.txt`, which would yield all the installed packages and _their_ dependencies.
Though, if 200+ packages were used in the codebase explicitly, that requires some sort of wizardry!
> That `requirements.txt` was probably generated by `pip freeze > requirements.txt`, which would yield all the installed packages and _their_ dependencies.
And also various development or test packages installed in the venv at the time e.g. third-party debuggers, profilers, etc...
> Currently working on a project with 200+ requirements in requirements.txt and growing.
That's incredible. I've worked on some very large python projects before, but 200? That's pure insanity. Can I ask what that project does? I just can't imagine a project where you'd need 200 libraries.
Somewhere along the lines, someone might have just done a pip freeze and pulled in all the dependencies needed, recursively.
For example, my project has 66 dependencies after cleaning up pip-freezes' output to only keep secondary dependencies if they were depended on by more than one thing (e.g. chardet is a dependency of every package that works with character processing).
If we're talking about only primary dependencies, we have 13 or so.
If we're talking about the entire dependency network... its easily above 200.
Rails itself is broken up into smaller packages, so you can only load parts of Rails if you want. Very few apps actually do. (and that's only like 8 of the 80)
I work on a abnormally large Django codebase. It has 30 dependencies on the requirements.txt file. I've worked on other Django codebases and they usually have about 15. Python has a big standard library which cuts down on dependencies. Not that Django > Rails. Just different.
I can't either. I'm surprised every day. The app is just another generic CRUD enterprise thing. I guess in that regard it's not bad at all. I've seen Java projects with way more requirements.
There is low-level code that is as you describe, and low-level programmers who write it. Probably if printf is involved, it's not that "low level".
Maybe this needs printf for a test suite, and talloc doesn't call it itself? Maybe there is also a way to run talloc in some verbose mode? I'm not familiar with it.
