Reliability, determinism, and security vulnerabilities are a good start for the purchaser. For the reviewer, we already know what methods [1] historically improved the assurance of software. Every method they added, the more bugs they found. That most proprietary and FOSS software use little rigor is why they're insecure. Only a few proprietary or academic offerings, not community driven, had the rigor for the B3/A1/EAL6/EAL7 process. I give examples here [2] for those that want to see the difference in software lifecycle.
Can you name one FOSS product designed like that? Where every state, both success and failure, is known via design along with covert channels and source-to-object code correspondence? I've never seen it. Although, it has happened for a number of proprietary products whose claims were evaluated by NSA & other professional reviewers for years straight without serious flaws found. So, for high security, the "proprietary niche" that does that has beaten FOSS by far and mainstream FOSS is comparable to mainstream proprietary in quality (i.e. priorities of provider matters most).
FOSS can potentially outdo proprietary in highly assured systems given they have free labor. In practice, they do whatever they feel like doing and so far that's not using the best software/systems engineering practices available. So, I don't trust FOSS any more than proprietary except in one area: less risk of obvious subversion if I verified transport of the source and compiled it myself. Usually plenty of vulnerabilities anyway, though. Would love to see more high assurance efforts in FOSS.
Can you name one FOSS product designed like that? Where every state, both success and failure, is known via design along with covert channels and source-to-object code correspondence? I've never seen it. Although, it has happened for a number of proprietary products whose claims were evaluated by NSA & other professional reviewers for years straight without serious flaws found. So, for high security, the "proprietary niche" that does that has beaten FOSS by far and mainstream FOSS is comparable to mainstream proprietary in quality (i.e. priorities of provider matters most).
FOSS can potentially outdo proprietary in highly assured systems given they have free labor. In practice, they do whatever they feel like doing and so far that's not using the best software/systems engineering practices available. So, I don't trust FOSS any more than proprietary except in one area: less risk of obvious subversion if I verified transport of the source and compiled it myself. Usually plenty of vulnerabilities anyway, though. Would love to see more high assurance efforts in FOSS.
[1] http://web.archive.org/web/20130718103347/http://cygnacom.co...
[2] https://www.schneier.com/blog/archives/2014/04/friday_squid_...