I started a company that was asked to write a proposal to make what became the app that failed last night. I declined because it was outside of our core skillset. You can read my prior comments or look at my profile to validate this. If you want my perspective as someone who started a Democratic software company, you should keep reading.
This app emerged from a mandate to make the caucus more accessible and transparent. It was well-intentioned but underfunded and lacked comprehensive organizational buy-in. Introducing tech can help but you have to spend tons of money to make it reliable and usable, then you have to spend more to train everyone in using it. This is a problem organizations of all sizes and shapes face when making massive IT changes.
Shadow is a firm that makes custom software for Democrats and progressives. It has an unnecessarily sinister name. There are not a lot of companies that make software for Democrats because it’s an awful job. You make very little money. Everyone hates you when things go wrong, which they will, because the product testing cycle and margins are nonexistent. Then everyone will assume things went wrong because you are some combination– you choose– of secretly evil, secretly working for Bernie, secretly working for The Establishment/Hillary (per someone's unpersuasive Imgur post below), or secretly working for Buttigieg.
Others have noted that Shadow also made software for the Buttigieg campaign. If you take my claims above as true, this should be unsurprising to you: a hard market where everyone hates you and no one has money to pay you is not attractive to enterprising software engineers, so there are few firms available to choose.
I don't know whether I heard correctly or if the number was accurate, but I heard someone throw out $60k as the amount paid for the app development this morning on the radio.
That sounds like a lot of money to a lay-person, but assuming $100/hr, which seems like at least a reasonable ballpark rate, that breaks down to 600 man-hours, or 15 man-weeks.
That is a very tight deadline to turn out a critical app like this. Especially since I assume it wasn't just a single person doing the coding.
People don't realize that software is expensive and a custom built application is probably not the correct choice for a single-use product. I'm sure whoever was in charge felt that $60k was a quite generous offer to build this. In reality it's basically nothing.
I used to have acquaintances approach me and say, "Hey I've got this great app idea you could help me with."
The response I found shut them down real quick was, "Great. Do you have $100k lying around to get the first beta out the door?"
60k for an app that runs results collections for the Iowa Democratic Caucus sounds absurdly low. By comparison, just to get an app like that assessed for security issues --- and my assumption is that this is not an especially complicated app --- you'd already in the neighborhood of 20k.
By being an app, they can easily advise the user "download it in advance and register in advance" by that verify they have it installed. Then they can make sure the app will work to collect the data in case of wifi/mobile network issues so that the user enters the data on the caucus site and if that building for whatever reason has no network go outside to upload.
Making sure that average Joe preloads a webapp and to make sure it will be configured a day later isn't easy.
Properly syncing from the web app - as discussed in these threads - adds complexity though.
Why is downloading a native app easier for Average Joe than clicking a link to a web app that's served with appropriate Cache-Control headers and uses localStorage?
Cache-Control gets ignored when browser tabs are "paged out", especially on mobile. It's egregious on Mobile Safari, where effectively only two or three pages are "live" and the remainder have to be re-grabbed. No connection? No re-grab. The version fetched might be any version from any point in history, including your "does this work?" flight that you pushed (because of course you pushed one). There's also very little mechanism to tell ahead-of-time whether a network request will work - you'll have to essentially pre-flight all server communication, which is doable, but not easily accessible the way that a "save locally, upload in background once internet" would work as a native app.
As an alternative to having them download a separate app, have them download a browser that supports the stale-if-error Cache-Control directive. It's fair to ask people doing serious work not to use Mobile Safari.
It's also fair to ask them to keep checking the native or web app to make sure the data was sent, so one doesn't have to rely on web workers staying alive in the background. Just save to localStorage, catch network errors and keep retrying.
But these are precinct officials, who took the time to become volunteers in this. They should be able to spend 1/100 of the effort that would take them to file taxes and preload a webapp.
Absolutely yes. It's true of all organizations, but I've found that with non-profits especially, just figuring out the need you are trying to address is quite difficult.
It makes me think of all that uproar for some iPad app for the TSA that just randomly showed a left or right arrow to send tell people which line to get in. Surely easy to code. But I bet it was months or years of interviews to determine that's all they actually needed.
The added complexity of having to build an app and all the boilerplate can significantly increase the complexity from just a simple website.
And while an app not hard, a https website with login and post capability is on the _extremely simple_ territory. As long as precinct official are trustable, I don't see any reason why they have to use an app.
I guess for the wow factor? As the entire political process is essentially becoming.
I think the argument of $60k being not that much still stands, and you would still be left with a sub-par result regardless of whether its an app or a website.
Many caucus workers are more senior than the average citizen. I know lots of older folks who can go to a website if you text them the address, but who can't install an app. Many apps are apps instead of PWAs for no defensible reason.
Why does it have to be an app? This is the part that confuse d me the most. Do it from an web page, use cloud, heck, it could even be cheaper.
You only had 1000 precincts, all you had to deal with is 1000 calls - and if one of them went wrong, you'll let that person know through some kind of reviewing mechanism. I understand that $60K is nothing but this kind of failure is absurd.
For clarification sake I suspect nobody is talking about a stand alone, run-on-the-mobile-OS, never to communicate via internet application. Idk for sure (wasn’t worth the investigation), but I’m betting their “app” is just shorthand for a web service that looks like a stand alone app through the UI.
But regardless of the architectural nature of their (Shadow Inc’s) product it’s worth noting that high performing, mission critical software is challenging to do correctly. That is to say doing it with feasible, adequate solutions for security, availability, performance, testing, and usability from the end users and administrators perspectives is a whole world of responsibility for an engineering team, even if the objective seems trivial in casual conversation.
People like to characterize software projects as “easy” and almost never know the scope of what they’re talking about. Idk anything about Shadow. But judging by their lack of effective consulting on this project, and consequently allowing such a terrible product to be released, they are bad at what they do. This whole incident should go down as a lesson to folks to not screw around with silly budgets, inexperienced teams, and unrealistic timelines when looking for quality software. That the DNC or whoever doesn’t know any better is a sign of the complete cultural ignorance that otherwise functional adults in this world have about the seriousness of software as any solution to any real-world problem. Everyone is a failure in this case: Shadow, the DNC, even the commenters still misunderstanding things on this thread.
I agree with you. Why an app, heck this could be done with a private Google Sheet. If they wanted call in tool, I could set this up in Twilio in 5-10 minutes.
In a way it was. If you're building something that is going to function on one night and one night only you really, really don't want to choose tech that won't let you do an on-the-fly update if something goes wrong.
One of the issues was that the app needed to be installed outside of the App Store. It seems that by being an app and not a website another hurdle was introduced.
$60K would probably be enough to create a landline boiler room phone bank staffed with volunteers to answer the phones from people reporting results. It wouldn't have the "wow" factor, but it would be a lot more secure.
"...Iowa Democrats, on the advice of the national party, abandoned plans to have caucus results called in by phone because of security concerns and instead build an app..."
No kidding - the instructions for the app itself came with a "secret pin code" printed out on paper for the volunteers to use to install it. Why not just give them a "secret pin code" to call in with to verify who they are, its just as secure.
The story when it fully comes out will be quite interesting. The assumptions that we all are making are almost certainly to turn out wrong in some way - I'm looking forward to trying to compare the ways in which I was wrong on my thinking today given the relatively sparse reporting thus far.
$100/hr is remarkably low in agency mobile work. When I was doing agency work five years ago, even our Bulgarian developers were billed out at more than that. I was billed out at 3x that back then and it would be more now. It's really challenging to find skilled developers who want to perform agency work and to keep them from leaving for a competing firm.
There would be endless queues if those Bulgarian devs would get even a fraction of that rate. They were lucky if they were getting more than $10-15/h. I also doubt there would be huge shortage of US devs willing to do the work if they were to get $120-150/h. The other problem is modus operandi of a standard software agency- it just suck your sole out for those extra lines of code.
The developers don’t get all the money they’re billed out for. The billable rate goes to the company to pay business expenses, of which developer salary is a minor fraction.
It's worth adding to this that turning it into a Web App doesn't make it dramatically cheaper.
Most of the time here (and hence money) goes in gathering requirements and stakeholder engagement.
I'd imagine there is probably at least 200 hours of meetings, travel and phone calls just getting to the point of working out what they need. And yes, a lot of that is going to be physically sitting down with volunteers in some town and seeing how things are done.
I mean that's the kind of money you spend funding two interns for a few months. Is it a surprise that this app looks like it was designed by a few interns on a tight schedule?
70K is the median programmer income in the US. (Based on a very thorough let's look at the first thing Google regurgitates!)
It should have been enough to fund a business analyst and an app developer for a few months. Plus Shadow Inc's overhead + profit.
It's absolutely bonkers how shitty the industry is. A kid can throw together an app in a few days, but somehow 60K USD is not enough.
That said I work in "consulting", I know how fast time, money, billable hours can be burned through, when there's no real drive, motivation, vision and delegation of responsibilities and control.
There's no way 60k can pay for multiple people for a few months: your 60k median excludes overheads, insurance, etc.
To be honest, I'm not sure if that a few people for a few months is even enough to avoid something like this from happening: this is crazy stress launch in gymnasiums with very high device variety. It probably demands a couple months just of serious dedicated QA and dry runs if you really wanted to avoid this outcome.
Sure, the point isn't that it isn't possible to have an app created for that price, the point is that at that price you can only create an app that leaves a solid risk of exactly what happened.
> It should have been enough to fund a business analyst and an app developer for a few months
Probably. But that doesn't matter if no one advised the client that they needed a realistic, production-like field test (most of the cost for which wouldn't have been part of the contract, since it would have been client side; the developer-side support for that is a small fraction of the cost.) Errors in report definitions that exclude some data in circumstances that don't come up in developer-constructed testing scenarios is, like, not at all uncommon.
> It should have been enough to fund a business analyst and an app developer for a few months. Plus Shadow Inc's overhead + profit.
No. The median software developer makes ~$50/hr. Overhead is 2-3x (usually higher for a smaller company but we'll be generous). So company is charging $100/hr. At 60k that's a single developer working 15 weeks (10 if we're being realistic). But we know that it is at least two developers so that's 7 weeks (2 on our realistic scale).
> A kid can throw together an app in a few days,
Sure, but it doesn't matter if flappy bird crashes occasionally. Flappy bird doesn't need to communicate with other users. I know this app they built wasn't that complicated, but the kids that are pumping out apps in a few days are above average talent and probably working more than 8hrs in a day. You're comparing above average to what is already suspected to be below average (considering they are recent bootcamp graduates and have little experience).
I guess the question is why does it have to be an app? If you've got a tight budget and deadline why add all that complexity when some cgi scripts will do and may even be overkill? It looks like it's just running a webview or something anyway.
I think you are 100% right, it is a very tight deadline with very few manhours. I did not advance far enough in the proposal process to learn their budget but that sounds plausible to me.
I think the plan was clearly that they would reuse the software for every caucus during this primary, so the $60K was one of a series of payments for the same product.
> I think the plan was clearly that they would reuse the software for every caucus during this primary, so the $60K was one of a series of payments for the same product.
No. The plan was that it would be used in the IA and NV caucuses, which is why the IA and NV Democratic Parties jointly purchased it.
The other states and territories using caucuses were never planning on using it. (And mostly, the other caucuses are structured differently with more primary-like features, so it probably wouldn't make sense to use shared software tailored to the IA and NV caucuses for them.)
CO cancelled all caucuses after 2016 when 10's of thousands of people got turned away because most of the gathering places were over capacity. Sanders supporters especially felt extremely underrepresented since they were young and showed up less early than others
Counterpoint, the primary use environment of the application will be on a smartphone. Web applications are not even second class citizens on iOS. In a situation where the user will be entering data all night, opening and closing and refreshing, variable internet connection quality and so on, all else being equal I would imagine the ideal platform would be a native application.
I lean towards agreeing with you that a simple website plus cheap laptop would the real world best case, but I can understand why you would want a native app.
You could use local storage and make an offline web app to avoid data loss.
I agree a web app isn't great for low friction data entry, but they only enter the totals for each candidate x each round, so that'd be a dozen or two dozen short numbers for the whole night per user.
The only extra thing I see is a photo upload. Which is not rocket science from a web app either. At least on iOS you can take people right to the photo gallery or the camera.
I would really like to believe this isn't a flask app I couldn't whip up in an afternoon and polish over a week in my free time... but I'm having a hard time.
In my experience it actually means less. SWE who cut their teeth at google are spoiled with amazing tooling and processes to support everything they do.
My personal experience with google engineers is 20% code and 80% complaining that our issue tracking system isn't as perfect as googles was.
How many unpaid hours did they put in? How much stock did they give to people who also put unpaid hours in? Never mind that they could launch, fix, release, fix release, etc...
A venture with a long development tail after release is absurdly different from this. They had to get it right the first (and only) time, and no one is anticipating even a one in a million chance of getting rich off it.
Because they weren't seriously paying themselves. Nobody builds an Iowa Caucus app in the hopes of selling to a big tech company for billions of dollars.
> That is a very tight deadline to turn out a critical app like this. Especially since I assume it wasn't just a single person doing the coding.
One screenshot showed an SQL error.
I think if you're doing an one man job, under pressure, making an app that has limited scalability, the last thing you should be doing is not using some kind of ORM or something similar (especially at a login page)
So it looks like they made it unnecessarily hard on themselves.
The problem didn’t come from making software for the Buttigieg campaign but the CEOs wife was on the Buttigieg campaign and he was a supporter.
Also Shadow was openly hostile to the Sanders campaign in numerous encounters.
It’s not as bad as the CEO of a major voting software company saying he’ll “bring home the votes for Republicans” (Diebold, 2004) a but this is a clear case of conflict of interest.
>There are not a lot of companies that make software for Democrats because it’s an awful job. You make very little money. Everyone hates you when things go wrong, which they will, because the product testing cycle and margins are nonexistent. Then everyone will assume things went wrong because...
In other words it's exactly the kind of business that attracts activist types who have non-monetary motivations and it could therefore present an opportunity for tampering, particularly since any discrepancies can be dismissed as bugs from underfunding and rushed development.
Whatever is actually going on, the system is clearly insanely broken.
>Shadow is a firm that makes custom software for Democrats and progressives.
If I can ask, why the specific nature for their clientele? Isolating your solutions to a specific political party seems strange to me especially if the emphasis on making money. If anything, money normally supersedes political leanings at the end of the day which makes the whole fiasco even stranger to me. Wouldn't you want to provide a solution for both parties by default? Not doing so seems nefarious on it's own.
It sounds like pairing products with political leanings is not a good business decision? Which may explain where all the 'hate' is coming from - it's just bad business and sows the seeds of doubt in their credibility in the minds of many.
Hey, I'm really glad you asked, and I'm happy to share. The reason that most people start companies in politics is because they have strong political leanings. You do it because you believe strongly in something or someone and want to support it, even if it pays less or sucks.
Even if you wanted to cross the aisle, it would be hard. If you work for Democrats, Republicans will not work with you. If you work for Republicans, Democrats will not work with you.
Going to especially second that last bit. Not everybody is expressly data science, but most companies start running into some sort of data at one point or another.
Campaigns aren't wrong to be concerned about who has their data. There is good historical reason for them to be skeptical that security on your system will be strong enough to prevent customers from accessing other customer's data. https://time.com/4155185/bernie-sanders-hillary-clinton-data...
Now imagine getting buy-in for using customer data to improve your models.
I can't blame campaigns for being paranoid about vendors potentially going across the aisle. NationBuilder is one of the only big names I know that does it. They are in a unique and unenviable position given the death grip NGP VAN has, and they pay a big price for it on nearly every front.
Really interesting point here. Although, there is a little more nuance needed and this is one of the big issues plaguing the campaigning software space - not all are created for the same goals and not all offer the same services.
If you take your example of NationBuilder, a company I know well as I led their European expansion, it only sold software not data. Unlike most American campaign software, NationBuilder does not sell data as part of their package. NationBuilder does not believe that you can create a sustainable and powerful community by using data that you have purchased, and thus only sells the platform.
So yes I agree with you that while campaigns can (and should) be paranoid about who has access to their data, and even how it is being used, this data brokerage model isn't the model that all companies adopt and it's a really important distinction to make.
While I am here, I will add the following, tech companies who want to democratise democracy (and I'm not saying that's the mission for everyone) and help lower the barrier to entry, cannot claim to do so if they alone decide who has access to these tools. Crucial decisions, such as which parties have access to the latest technology, should not be in the hands of a few tech titans of Silicon Valley - where the power they already wield is already unmeasurable.
All of the above is a fascinating and important debate and also a very American one. Campaigning and political technology is incredibly partisan in America, compared to Europe where what we fear most are monopolies.
because you are a non-partisan software? I agree with the 'getting into trouble piece' I think. I don't see how being non-partisan gets you into more trouble or would increase those chances.
Customers will probably be less upset if something leaks internal to a party during a primary than they would be if something leaked to a different party.
I agree with all this. To your point, NationBuilder has been nearly shut out of the Democratic Party at anything above the local level as a result of serving Trump in 2016.
So when Republicans and Democrats walk from the Capitol to their respective fundraising call centers, is your claim that that there is a different software stack in each building to handle the call automation?
I have no idea how to find out whether those tech stacks are indeed different. But if someone knows the answer I'd definitely find it instructive.
AFAIK the fundraising done by actual elected officials is not highly automated, but tools for phone banking, canvassing, and that sort of organizing are definitely partisan (see for example https://act.ngpvan.com/paid-phones on the Dem side).
Whenever I've done calling or canvassing for progressive causes or democratic candidates we've used some version of NGP VAN (https://www.ngpvan.com/about).
I wouldn't be surprised if there are some cross-party solutions (especially the closer you get to core infrastructure—I'm sure both parties use AWS).
You are right, in the deepest recesses there is crossover. Stripe, media companies/FB for ad buys, Microsoft Azure, AWS. It is very rare anywhere else.
I don't know how strong political beliefs correlate with technical ability, but from my anecdotal evidence, the people I work with are highly competent and not very political.
While highly political people I know don't seem to work in tech.
Even if many people think that they "aren't very political" and are incorrect, there are some people that aren't very political and this is justified. For example, if you are a 5 year old and you believe you are not very political, you are probably correct. If you have been living in the woods for the last 30 years, don't know anything about modern politics, and believe that you are not very political, it would be weird for us to say "the thought you are not political is itself quite political" rather than "that sounds like a justified true belief."
So assuming it's possible in principle to be "not very political," the real question is how disengaged, uninformed, or lacking in understanding do you have to be for your statement to be true in itself, rather than a "quite political" statement. For myself, I think if someone is disengaged enough that they don't care who will become the next president, and there are no local political issues they will argue with someone about, they are "not very political."
You don't think there's anything political about removing yourself from society for 30 years and living in the woods?
> I think if someone is disengaged enough that they don't care who will become the next president, and there are no local political issues they will argue with someone about
What this is saying is that they are happy with the status quo, and they see no room for improvement, or they're not bothered if things are improved one way or another. You can be not interested in politics, but I think doing that is just supporting the current politics.
For many people, just existing is pretty political, like when an entire country has a debate and a vote on whether you should be afforded basic human rights.
> You don't think there's anything political about removing yourself from society for 30 years and living in the woods?
I think there's something political about it, but I also believe that it could be done for non-political reasons (spiritual reasons, practical reasons, mental health reasons). As a pure narrative, it would most likely be politically motivated, but in hypothetical-land it wouldn't necessarily be. Even if there is something "political" about it, it doesn't really map to what people mean when the use the term "political." It may map to opinions about how the state should be governed, the broadest definition of political, but it's much more frequently used to refer to a particular subset of debates which are at least minimally polarizing in some way.
> What this is saying is that they are happy with the status quo, and they see no room for improvement, or they're not bothered if things are improved one way or another.
All I'm trying to point out is that it's a legitimate spectrum. When I say "someone disengaged enough that they don't care who will become the next president" it's very easy for the reader to imagine someone in shoes very like theirs and making that choice somewhat actively to not be engaged.
But in the wide world, you can be simply unable (or unwilling) to devote any energy to understanding how the current political environment relates to your life, and thereby form no opinions one way or the other. Perhaps you work 16 hours a day to support your family, or you live in a country only temporarily and don't even understand the language. This would imply the fact that you lack the resources to form a political opinion is "supporting the status quo."
Support doesn't have to be voluntary, and often isn't. If you are coerced into supporting the status quo, that doesn't mean you aren't supporting it, it just means you lack the freedom to make this decision.
One underlying question is why there was an app for this in the first place, when election process activists basically say do it on paper and by hand with people and audits.
Anecdotally, nations doing it that way seem to report faster than when software and electronics become involved.
Can you comment on whether the proposal was part of the the SBIR / STTR program? Just curious as to the scope of companies that were participants in the proposal process.
As an aside - these instances of underfunded and underdeveloped services, specifically in the political sphere, are really painful reminders of the flip side to the idealistic technocratic future that I think a lot of folks around here assume to be an inevitability.
Caucuses are run by the parties, not the government; it went through whatever podunk procurement process the Iowa Democratic Party has, not anything federal or even governmental.
Curiously... Any take on Michael Slaby, Timshel (defunct), The Groundwork (defunct), the dozens (up to 70 at one point, many from SV/NYC) of engineers that worked on it for roughly two years in close contact with HFA 2016 (The Groundwork was right down the street from HFA 2016 HQ), and the $700,000 that HFA 2016 paid for that work? More generally, do you think there is a problem with for-profits which focus on political organizations under the guise of "focusing on non-profit"? How often do you think illegal in-kind contributions take place through these companies to political campaigns?
I worked with Michael Slaby in 2008 and 2012. He's an incredible technology manager. He's great at punching up and fighting for his engineer's perspectives and helping upper management understand limitations and then getting out of the way.
Since that time he's probably had his own epiphanies from working in industry but one of mine was that we did shockingly little QA and everything was built as fast as humanly possible. Everything was building the airplane in the air and it was largely due to constraints of the political or legal kind. Timeline management and allocating resources to testing was always an afterthought.
Testing usually comes from a feedback loop of launching broken features, incident response, manager gets in hot water, engineer gets in hot water, engineer proposes testing plan, manager uses incident as primary source reference to secure additional budget, testing implemented.
In politics no service, company, initiative, or team lasts long enough to complete the cycle.
"In politics no service, company, initiative, or team lasts long enough to complete the cycle."
This is systemic corruption. A billionaire can dump as much money into a for-profit company to build a platform for a specific campaign and then when the campaign is over then can claim that there isn't enough business to keep the platform solvent for the next 2 to 4 years and do it all again with the benefit of the previous code base. It's better than a Super PAC because the for-profit company can work directly with the campaign and no one will bat an eye. The company can even have foreign financing. All the while, their losses amount to a huge tax write-off and the public pays the bill.
In the case of The Groundwork, it likely cost upwards of $10 million for the people, building lease, AWS costs, etc. But where did all of the money come from? Certainly Hillary didn't pay for this. I'd love to see Eric Schmidt's 2016 Tax Returns.
Why is any sort of "Democratic software company" even remotely necessary? Why can't they pay for white-labeled solutions? Or pay an established consulting agency to build things?
Please educate me on something. I fail to understand why "making custom software for <insert party here>" is something anyone would want to do in the context of owning and growing a real business. This seems fraught with all kinds of issues.
First of all (and it seems I've been using this word a lot lately) this turns software development into a religion rather than a business with solid business strategy. The equivalent in other businesses, like the entertainment industry, are people who do work for stars for nothing or nearly nothing because they think it will lead somewhere. All they get is a weird and abusive form of servitude (I've seen this first hand) and no future.
Why would someone not write software for politics instead and make part of their value proposition that they are not biased (and, of course, take the steps necessary to actually deliver that).
We have a close relative in South America who owns a company that makes and markets voting hardware and software. They are not aligned with any specific political party, movement or cult. They sell to everyone, in and outside South America and have a very nice business.
Hey, I appreciate your comment and I'm happy to share. It is very hard to make into a real business, very few people have succeeded. You do it because you believe in a cause or a candidate or a party.
As far as why people don't sell to everyone, the above explains a lot. Another explanation is that political sales (which is different from true election tech, which is purchased by governments) are driven by trust and personal relationships. Once you work for one party in the US, the other will not trust you.
This is interesting and, yes, I do understand. Yet, another take on this is that groups (parties) would then rob themselves from perhaps working with or searching for the most capable of providers.
I am not trying to diminish anyone at all, but I would think these kinds of party line attachments mean that not everyone ends-up with the most capable developers, particularly so if what drives this is passion rather than a skilled software shop that with proven capabilities and experience across a range of relevant domains with an equally proven track record of delivering quality, well tested bug-free product.
Of course, the parties would have to we willing to pay more than hobby money for what they need.
> but I would think these kinds of party line attachments mean that not everyone ends-up with the most capable developers
This is a real phenomenon, but not in the way you think. Democrats have historically had a tech edge, because the real talent leans left politically. Obama kind of killed that, but for a good cause - he diverted a lot of those politically-driven types into working directly for the federal government under the umbrella of USDS and 18F.
Unless they're A) very large, and B) selling commodities, a company that works with one party will not hired by the other party. The two parties have two completely different sets of infrastructure.
Maybe what I am thinking is that, if it is a real business with products that are agnostic any user would benefit from a much higher quality product. I don't know enough about software in politics to understand just how ridiculous this idea might be.
Perhaps an example might serve as a better explanation of what I am thinking:
Trello is a real company with a product that is agnostic. Politicians of any school of though can use their products to great effect. They benefit from everyone using this product, including, perhaps, their competitors and opposition.
If, on the other hand, each political party hires a passionate coder to create their own kanban board software they are very likely to end-up with a bunch of suboptimal solutions.
In other words, if they can, if the nature of the software allows, it would benefit them to look for universal providers rather than party-aligned providers. If you can trust Trello with your political party data you can then trust a similar company with other data you might produce with their software.
I guess it comes down to the software. What kinds of things are they doing that requires custom tools that have to be written specifically for one party?
* There are types of software that are politics-specific, and therefore do not exist until a party pays for it to be made.
* Politics is a zero-sum game: if you're funding software (either institutionally, or by being a tech worker who works below market wage For The Cause), you don't want it to be used by your political opponents.
So commodity stuff that already exists is bipartisan. Any application that is specific to political campaigning is highly partisan.
[1] There's also a secondary issue of data security for cloud products; parties maintain extensive voter files with things like past voting history, up-to-date contact information, and past contacts [1a]. They do not want this information falling into the hands of the opposing party, and so don't want their data touching any organization that might potentially have cross-partisan ties, or godforbid might have cotenancy for D and R data.
[1a] This is basically the king of all CRMs, and these days they might be able to reimplement it as a bunch of plugins on top of a commercial provider like Salesforce, but those only became available relatively recently, data migration would be nontrivial, and there's a substantial amount of politics-specific functionality that would need to be rewritten as plugins.
What is personally interesting to me as a classical liberal is to watch how both main parties engage with followers. I generally sign up for updates from both parties from local all the way up to federal level. I do this with a dedicated email address. It is interesting to watch the approach, intensity and messaging coming out of each camp. It is also interesting to watch how far and wide your email address and info is shared. Definitely do not do this with an email account you care about.
Can you share any more about the proposal process? Where was the RFP posted? What were the evaluation criteria? I was kind of assuming that this was more of a handshake deal or a cold sell or something, surprised to hear it went through a structured bid of some kind.
I'll do my best, but I'll probably disappoint! A friend of mine who works in politics told someone in the IDP that I might be helpful because my company makes software for Democratic political campaigns. I had a conversation with the IDP staffer, learned more, and told them very quickly we wouldn't be able to help with this. They might have ran a more formal RFP beyond that, but I have no knowledge of it, because we did not advance beyond that conversation.
I realize this is HN but let's not be so milquetoast about the critical infrastructure of our democracy. Non-free, closed-source software DOES NOT belong in the machinery of our democratic processes.
Did they make the text-banking app for Buttigieg, or merely sell them SaaS? Biden, Clinton, other Dem campaigns, and the DNC have all used Shadow's software.
> I declined because it was outside of our core skillset.
Um, isn't this app allowing people to enter data, and sending that to an API? How could that be out of anyone's core skillset? Almost every single app must do that.
Checked their LinkedIn and the employees who work there. The founder is non-technical and there are two developers: one who is a "back-end intern" and the other is a front-end developer (both are fresh from bootcamps).
For most software development work a bootcamper is just as likely to be a solid team member as someone with a CS degree after a few years of good industry experience. They key issue isn't that they are bootcampers but that they are fresh bootcampers with no more senior people around to guide the project or mentor them. (similarly a fresh CS grad is fairly worthless in serious production work and will not improve much if they don't get the opportunity to work on serious projects with great sr devs and leadership invested in mentoring them)
A fresh bootcamp graduate can be a good developer, but I would argue that someone fresh off of bootcamp is way less prepared than a CS grad to directly start working in a small team with mostly non technical people. A CS grad is usually way more knowledgeable when it comes to software architecture, handling data, optimization etc.
Bootcamps usually focus on specific stacks and tools, which can be great. But that means almost no skill/knowledge outside of that specific stack
My experience has been the same with fresh grads from everywhere. There's a non negligible amount of work to get them up to speed with business communication and task definition.
Basically schools give you assignments or projects that are well defined and have clearly defined goals/results and cover solved problems. While that's true in the generic sense of business problems it's not how we work, we have to be able to take initiative to define and adjust those goals as reality shifts and goals are more loosely defined. You really have to put it the work to get new folks up to speed with defining their tasks to reach the goals.
Not for nothing but it's tough when you don't have a senior or two on hand to bring people up to speed and mentor them for a month or two before you can really get a new grad productive. And it's difficult with time constraints to do that well, I have to define my own tasks and can't spend as much time defining things for a junior as well as I'd like or train them as much as I should.
College grads may be more blank a slate but I haven't noticed a big difference in terms of actual onboarding. For what it's worth I think boot camps are the next trade schools and we're going to see a division in labor between software techs and engineers just like we have for electrical engineers and electricians or MechEs and mechanics. Tale as old as time, all the work is valuable, just more nuanced.
That might be true, but if I get to choose between someone with no industry experience, and another person with no industry experience whose mind has been poisoned by a certificate mill, I’m taking the former. Those boot camps are, literally, worse than useless.
Yeah I guess I would choose a CS grad also but "poisoned by a certificate mill" is pretty dramatic. I don't have any illusion fresh bootcamp grads are awesome sr devs but seriously you can, again surrounded by more senior mentors, put a high potential bootcamp grad on a project and they will be far from worse than useless almost immediately and certainly after a few months be quite valuable because their albeit very minimal and focused training got them a toehold and some skills they could apply and expand on over time. Sounds like you had a bad experience.
it failed to reliably report on one of the traditionally pivotal elections in the American election cycle. Which is pretty bad in an era when trust in political institutions is low.
Given the numerous safety standards and qualifications we require of engineers in the most mundane jobs I'm not sure why interns get to write software that underpins the electoral process.
And putting literally any failure at the feet of a jr. developer is completely wrong.
State party officials who made the plans to use the app without clearly any vetting, the management of Shadow who must have been making insane promises on a shoestring budget, the DNC who allowed the app anywhere near an election, and even pushed the state party officials in the direction of using it.
In big tech areas, there's usually a firm assumption of liberal-leaning employees and technologists. Maybe all the democrats who can code are too busy in the advertising industry.
For a company of its size, it certainly got a good amount of C-titled people.
CEO, COO, CTO...
Here's the thing I don't understand about them, I thought startups were averse to this? Am I wrong about startups or am I wrong about them being a startup?
Titles are a form of compensation.
When you are a cash-strapped startup, especially in a domain that's not particularly lucrative, you pay people with titles.
If this incident hadn't happened, these folks could have proudly claimed that they were the CTO/COO/CPO etc of Shadow Inc on their LinkedIns and whatever article BuzzFeed/Forbes wrote about them.
With the exception of CTO, none of them seem to be engineers.
If I'm on the hiring end, this is not something that will help. Instead, if someone's going to sell me the importance of their title at their 6 people startup, it's going to negatively affect their chances.
It's the other way around - if you're trying to recruit an engineer or ops person from FAANMG, you can tell them
"Well, I know you're an SWE 2 and make $300K/year, and we can't match that, but guess what, you can be the VP of Engineering at our 2-person startup and when we go public, you'll be a VP or SVP of a public company." You'd be surprised how many people fall for that.
The COO majored in Music Technology at Oberlin. That's quite a bit more technical than most people realize. TIMARA (the music tech program at the Oberlin Conservatory) involves a decent amount of programming and/or audio engineering. To put that in perspective, the founder of Macromind/Macromedia (Marc Canter) is also an alumnus of TIMARA.
The Verge report says that the Android version was distributed using TestFairy beta testing free version, and that this is limited to 200 users. Assume 25% of the 1600 sites used Android and half of the users would be rejected.
But looking at the TestFairy pricing page, the free version is actually limited to 200 sessions per month. I would read this as every time a user logs on - definitely not going to work for 400 users logging on multiple times in one night.
The company who produced this app didn't just fail on development but on any procurement/contractual sense. How could they have imagined that relying on a free tier of a beta testing platform could be acceptable in a real life production environment?
One time I was working on a Splunk thing at an old job. We had an engineering contact there, and when a question came up about how something in Splunk was tested, the guy literally just replied back to our email with that meme and no other details.
I'd like to introduce you to my friend, the state of Georgia, for the past 20 years. Only this year, by court order, are they reintroducing a paper trail. While you're correct that Iowa has a paper trail for their caucus, it's not a concept that is ubiquitous, and has severely damaged the integrity of elections in the US, and especially in the state of Georgia.
IKR? And for all options they may end up with, they chose an APP. WHY? Does laptop not exist anymore? Does phones lack the ability to open webpages anymore?
Hasn't logging in been one of the things we perfected over 30 years?
Yes, if you want to have a small tool for volunteers without logistics of shipping machines, laptops don't exist anymore. Developing desktop software for such purpose is way more complex, especially when considering updates etc. Which Windows version are you targeting? And Mac? What libraries you need?
For app development, even if for two platforms (Android and iOS) it is a lot simpler these days, than desktop.
The GP asked "Don't laptops exist anymore" to that I answered. And no - a webapp isn't an alternative for those crappy laptops, filled with viruses, outdated browsers, and/or weird privacy plugins the (grand-)son installed are not an alternative. (We could argue about requirements of integrity here ... an app can offer a little bit more than a webapp on an arbitrary windows PC ... But integrity can be guaranteed by having the second channel (other people watching and then comparing published results) as there is no secrecy involved (for one we talk about summaries, for second caucuses are public anyways))
What did this app do exactly? It was just a reporting app, I'm assuming that the source data collected, albeit perhaps in different formats, lived somewhere else and that there wasn't a lot of data or a lot of variations. From my understanding it seems there was an Auth0 redirect issue, but why wasn't this just a night of taking the source data and doing scripting then shipping out the reports via a secure DropBox type service? We've all be there where an ETL job fails, since it isn't critical (e.g., financial transaction), it wasn't tested every which way and we just had to some scripting.
Even if it is, say, 500TB of data, in 300 different formats usually those formats aren't drastically "different." Maybe I'm not understanding what the application was supposed to do to not understand why this wasn't solved every quickly. Or maybe given the timeline it was solved quickly once the right people got involved and figured out what needed to be done.
Well, at least if they paid a billion dollars they could have had an expectation that the app would work, and they could sue IBM if it didn't. Indeed at that $$$$$ they would have been ripped off though!!
At $50k they certainly weren't going to get what they wanted delivered, and they have no-one to blame but themselves.
to be fair, they needed to scale to roughly a thousand concurrent users (where concurrent is used very loosely here). so I wouldn't say that scaleability was really a requirement. 1000 users is nothing.
I’m truly astounded that anyone thought this was a good name and took the time to write this marketing copy:
Why Shadow?
When a light is shining, Shadows are a constant companion. We see ourselves as building a long-term, side-by-side “Shadow” of tech infrastructure to the Democratic Party and the progressive community at large.
I'm all about Hanlon's razor but naming the ticking time-bomb of a voting software startup "Shadow Inc." seems a bit too on the nose to be pure stupidity.
Everything about this reeks of a publicity stunt to "ruin" electronic voting in the public eye.
...says the greenbean account. Blumenthal is an excellent reporter, and all of his reporting is extensively sourced. The article linked above is liberally peppered with links to sources, several per paragraph. If there's anything there you doubt, you can just click through.
Sorry for tooting the horn again, but this is really a solved problem. You go with paper ballots and vote-counting machines. The result is available pretty fast, and when there's concern you can always re-count everything by hand.
Caucuses are great. High information voters who have to publicly stand by their candidate, and persuade other members of their community to support their candidate. More on the “democracy” side of the equation than the “republic” side.
What about the Iowa Caucuses makes you believe these are "high information" voters? They often have nonsensical positions and ranked choices. They're committed voters, but that is not at all the same thing.
That's the obvious and vivid example, but more broadly and tellingly are people's stated preferences and the outcomes of realignments, where people are oscillating between progressive and moderate candidates. I see no evidence that Iowa voters are "high-information"; I think most of them see participatory politics as a sport, just like voters in every other state.
The OP wasn't being metaphorical in any sense, they were literally describing how caucuses work. If your first choice doesn't meet a qualifying threshold in your precinct, then you can switch your vote in the second round, and supporters of other candidates can literally try to convince you before that round. It all happens in the open with people literally crossing to different parts of the room to caucus for different candidates. The app -- and its failure -- didn't change any part of this process.
But the Iowa caucuses also has a delegate apportionment system that privileges rural voters over those in large cities, similar to the electoral college. It strains meaning to describe something designed to produce an undemocratic outcome as being more democratic than republican.
That's a real issue -- and one that seems like it's going to have an actual effect this time around, giving Buttigieg slightly more delegates than Sanders even though Sanders actually has slightly more support. I think the Iowa Caucus worked as designed, despite the collapse of the app, but just like the Electoral College, that doesn't mean the actual design is good!
We don't know that they were fine. Until Sanders campaign urged them to make a change in transparency, only the final delegate count was reported. So it could've been a dumpster fire every other time and they just obscured the discrepancies with "final delegate count"
> The result is available pretty fast, and when there's concern you can always re-count everything by hand.
Good old progressive enhancement. Not sure why people want to rip out the old functionality (paper) until the new functionality (e-voting) has a solid track record on it's own.
I admittedly dont know the specifics of this process, but to break it down very abstractly, there seems to be a challenge and a misconception.
The challenge is: is the person submitting the results who they say they are, and are they authorized to submit results for said precinct, did their data submission transform on transit. The misconception seems to be that any of the data is private.
Issuing 1700 instagram accounts and having each precinct post and tag a photograph would have been both more secure, and instantly verifiable. Anybody could go back later and go "yep thats the photo I posted." "Yep that is the account that we expected the photo to come from." "Yep, the account wasn't improperly accessed."
Everyone in the public can see the live results real time, and hand verify the summarization. A worst case scenario would be someone doctoring one of the Math Sheets, printing out a near perfect duplicate, and creating confusion over which document is the "real" one. Breaking into a physically secure area with a fake piece of paper may cause a situation with an unverifiable truth. Having a public, timestamped photograph minimizes counterfeiting and forgery.
Providing a csv/xlsx file, and having a whitelisted list of senders would have worked, along with a bot ingesting the files as they came. A onedrive with 1700 shared spreadsheets would have worked, each precinct manager would have just needed a copy of mobile excel. The technology that generates my fantasy football cheat sheets seems more advanced.
The goal here should be to get a definitive answer posted into the public record, timestamped and proven to be the appropriate sender.
The "pin" nonsense is the scary part of all this. Did anybody who shouldnt have download the app, and submit the wrong results?
>he had to sign in with an email and password, provide a two-factor authentication, and enter a one-time password generated by the Google Authenticator app.
What is the point of 2fa/authenticator, if the person is registering for the first time the night of the primary? Wouldnt 2fa be for preventing access to the account youve already made, not to prevent someone from creating your account? Unless somehow the phone numbers were already on file, and a sanity check took place before the 2fa step, which I doubt. Is 2fa anything more than security theater in this application?
Where I live we use paper ballots that are both scanned and tallied on sight by a machine when submitted, then hand counted by volunteers at a central location that anyone can observe.
This is a caucus not an election. The democrats have never used paper ballots (as far as I know - my history is short though). The republicans still use hand counted ballots.
Either way in a caucus you are NOT electing a president, you are just choosing a delegate who goes on to the national convention where the president is chosen. There are rules so that the caucus results feed who is supported at the national convention, but there are ways around those rules if you want to go to the national convention and then support someone who wasn't even running in the first place - though you pretty well better be assured of a win before you think about trying those tricks as the media will go nuts. (the same rules exist in primary states, technically there is a caucus somewhere that chooses the delegate to the national convention who can then vote for someone else)
Last election cycle, the Bernie Sanders campaign insisted that the Iowa caucus should be changed so that more data was reported. Back then only the final number of delegates were reported. So it is entirely possible that the caucus always was a mess but that it only got discovered this year.
I don't know. I feel this may be a great application of blockchain and public ledger tech. Of course it would need to be open source... And... You know.. Obviously written in Rust.
With strong voter ID. Paper ballots do you no good if you can't verify that they were cast by a real person that was eligible to vote in that area, and that they only voted once.
Courts found that the North Carolina voter ID laws were designed to be discriminatory[1]. Same in Texas[2]. So it's definitely true in at least two cases that voter ID laws were provably designed for voter suppression.
Something being provably true is a higher bar than it being explicitly stated (because an explicit statement isn't enough to prove something to be true). I agree that voter suppression was the primary intent, but the legislation wasn't called "The Illegal Voter Suppression Act 2013".
That is election fraud, perpetrated by the people running the elections themselves, not voter fraud. If you can't trust the people running the elections, then no amount of voter ID is going to matter.
In 2016 there were only four confirmed cases where people voted who should not have and would have been stoped with voter ID.
All four voted for Trump, but that's not super relevant. What's relevant is that it was four out of 127,000,000 votes. Even if the problem were 1,000 times worse then we know it still would have had no effect on the election, even if they were all in Wisconsin.
Paper ballots with scanned results in the booth. No scan means redo your vote. That way you have a track record + instant results.
Folks can take home the stub to ensure non-repudiation etc..
They can look up the serial number on the stub they take home to see for sure it was recorded.
FYI we should also have basic ID requirements for voting. The arguments against it are kind of ridiculous, and federal or state ID should be free.
There should be no 'voter registration' required either: the state knows your address and that's that.
For lighter elections like city alderman, you can use your ID + passcode to SMS or email your vote.
Maybe make a law requiring that any tech used for legal voting has to get the thumbs up from the NSA or some super amazing agency that is also secular in nature. None of this 'quickie app for voting' rubbish.
At this point, I hate politics so much, I think every nation needs a 'caretaker' Pres/PM from the bureaucracy - someone who can clean out the cobwebs, get rid of a bunch of old laws, clean up the tax code, take on the public unions (I mean reform not destroy, I mean teachers probably need higher salaries) and basically do a big, operational house cleaning that has nothing to do with politics.
This is a caucus not a primary. From what I understand there are no ballots at all - you physically walk around the room until you find the group you want to support, if your group is large enough to choose delegates (generally there are more slots than people to fill them) to the next convention. The delegates are in theory (in practice it is different) not required to vote for you, so when your group chooses people you need to make sure they are like minded.
Once everything has been decided, an iron clad paper trail IS produced. There’s nothing anonymous about the results document and it is signed in front of witnesses. AIUI, the paper doc isn’t discarded until after the party convention.
I didn't say there was no paper trail. I said there no ballots. That is a subtle difference that if you don't go to democrat caucus in Iowa you probably don't understand.
Theoretically a group can win delegates in a caucus with just an issue not a candidate - for example you can all agree that the most important issue is Oranges (a fruit that doesn't grow in Iowa...) and get a delegate without committing to a president. Of course in practice most people show up only for the president poll and leave as soon as that is over.
At this point, I hate politics so much, I think every nation needs a 'caretaker' Pres/PM from the bureaucracy - someone who can clean out the cobwebs, get rid of a bunch of old laws, clean up the tax code, take on the public unions (I mean reform not destroy, I mean teachers probably need higher salaries) and basically do a big, operational house cleaning that has nothing to do with politics.
You realize that every single thing you said there is political, right?
Yes, I agree, 'everything' is politicized, but some much more than others.
'Simplifying' the tax code could be ideological, but it's only overtly political given the interest groups of those who profit from complicated taxes.
But by that virtue - every little policy of every government agency implies 'politics' and so we'll get nowhere.
There's quite a lot of things that Americans widely support, even on some of the heavy 'litmus' issues, it's surprising how much Americans have in common.
So yes, you're right, everything would be pulled apart by Fox and MSNBC, but there's a lot that would be less divisive.
I'm going to assume good faith here, and answer it seriously.
> "clean out the cobwebs"
This isn't specific enough to respond to.
> "get rid of a bunch of old laws"
Which ones, and why? Sure there are probably some like where people can tie up horses or something that aren't especially relevant, but having them there causes no harm because they aren't used.
> "clean up the tax code"
There is no way to do this without creating winners and losers. Choosing who wins and who loses is a political act.
> "take on the public unions"
Taking on unions is one of the most political acts you can do. Whether or not it is "right", unions are a powerbase of some political groups.
> "operational house cleaning that has nothing to do with politics"
What - specifically - do you mean? The two examples you give here are highly political.
This app failed because no one did enough testing and enough training. What I saw was a relatively simple app to input some data. The problem seems to have been that they never tested it at scale and they assumed that all the users would be able to download it and use it.
My experience is that customers hate to pay for testing once they see the product running. They assume that it's done. I had a customer tell me that if I did the programming right it should always work and testing should be minimal. I had to explain to him that that's not the case with software and testing is one of the most important parts of the software development cycle. He felt it was a waste and that I was looking to add extra costs for no reason.
I bet there was not enough money. People seem to feel that a few grand will cover the costs. They figure 10k is an outrageous amount. What they don't seem to understand is that it will barely cover the costs of planning the app.
Both the developers and the people that approved the app for use need to take responsibility. Too bad since it could have saved a lot of money and time in the long run.
BTW, this could have been a Google Form with a spreadsheet as a back end. But user training would have still been an issue. You can't get around that.
This should really be getting more attention, here is an alternative breakdown: https://imgur.com/gallery/ycOC0HX.
The jest of it is one of the most important institutions in the United States (the Democratic National Committee) uses a highly nepotistic and incompetent system for managing IT which leads to colossal failures in marketing, canvassing, and security. Not to mention massive PII violations as millions of emails, phone numbers and SSNs, are passed around in plain-text via CSV files.
The reason this happens is because hundreds of millions of dollars are lit on fire during election season and all the sharks, including former Google employees, come out to swim. Even well intentioned projects get slammed by the crunch of the election season (seriously try shipping a well scaled app in < 2 months with terrible product direction) and ultimately fail - failing the needs of the entire citizenship of the country.
After the success of Obama's 2008 and 2012 campaigns even more money was funneled into IT as a sort of perceived silver bullet. But in 2016 it wasn't, and yet no analysis was done to correct the problems for the 2020 cycle - because the decision makers (all these "CTOs") are clueless fucks who are just there for the money and could care less about the integrity of our democratic system.
- in 2016 I worked for one of the companies in this niche and saw the bidding/sales/engineering processes first hand. FWIW I am a life long democratic voter and this makes me sick to my stomach.
There's no reason why it has to be this way other than DC careers being an absolute rat race. Elections happen literally like clock work. There is no need to get an app out in 2 months; there will always be the next election if the app isn't ready to ship. Plenty of elections have happened smoothly without your app; it isn't necessary.
However, if you are only a campaign manager for six months or whatever and have to prove your worth or else move back in with your parents and spend the rest of your working life fetching coffee and slowly paying for your $200k georgetown political science degree, then yeah that app better be out in 2 damn months.
Rat races don't lead to quality, they lead to desperate people taking shortcuts and making mistakes, and fodder for managers to point to on a slide when they have to appear in front of superiors. Things that woudn't otherwise happen if pressure was lowered and we thought about things like burnout and well being rather than whatever meaningless work-related metric is hot right now. This sort of failure is inevitable in fields operating like this.
Well, you aren't technically wrong here, but I disagree with the sentiment. The RNC leveraged a the same kind of data used by the Obama team in '12 to squeak out an Electoral College win in `16.
I think it's easy to get overly ambitious with civic tech and deliver expensive garbage. However, when people stay focused and deliver then I think the results speak for themselves. While the '12 and '16 elections didn't turn on candidates tech strategies alone, it's clear that they contributed.
Absolutely not. There remains 0 evidence that the microtargeting strategies employed by both the Obama and Trump campaigns did anything. It gets lots of press, but literally no one has presented anything to demonstrate that any of it moved the needle at all.
I personally worked on digital marketing campaigns for years as a data scientist in a previous job, and have become entirely disillusioned with the entire notion (and I am not alone[0]). Its largely snake oil, and its effectiveness is incredibly overblown. But literally billions of dollars, large marketing teams embedded in every company, and some of the largest tech giants on the planet all have an incentive to drink the Kool-Aid.
"advertising doesn't work" didn't work for RC Cola... There is more to it than clickthrough. It's saturation. You don't have to believe in the system for the system to believe in you.
I think its a general reference to losing the popular vote but winning the electoral college and otherwise having nothing to do with the electoral college margins.
There is some analysis that boils down to the below:
“He concluded, with help from The Cook Political Report, that the election hinged not on Clinton's large 2.8 million overall vote margin over Trump, but rather on about 78,000 votes from only three counties in Wisconsin, Pennsylvania, and Michigan (by the same logic, Obama won in 2012 due to three counties in Florida, Ohio, and Pennsylvania)
That's one way of looking at it. Another way, is to break out how many of those electoral college points were won at razor thin margins. The reality is that the election was VERY close. 107,000 votes in three states decided the whole darn thing. WI, MI, PA, and FL were won by Trump with less than 2% margins each.
Conspiracy theorists will be out in force, but it's important to remind people that the State parties run the elections and their processes.
Edit: Seeing downvotes on this, but it's in support of tptacek's comment. The DNC and State democratic parties are separate and the latter run the elections. You might believe that the DNC is fixing the process, but the reality is that they have little influence in how the States choose to run their election/nomination processes.
In all honesty, the Iowa caucases are likely a huge pain in the ass for them and this failure is probably driving a lot of DNC folks crazy right now.
There's no such thing as an "Iowa DNC". You presumably mean the IDP, the Iowa Democratic Party. They're separate organizations.
The DNC and the state parties are often at odds with each other. There's a good series of "Reply All" episodes about this, focusing on the Alabama Democratic Party.
Since I heard those episodes I've been hoping that the Alabama situation isn't typical. Do you have any special insight about Iowa? Is there an Iowa kingmaker running everything from behind the scenes?
Most notably, the IDP has a strong interest in maintaining their first-to-vote status, which the DNC hates. Hence the eternal drift of primaries earlier and earlier - the DNC will try to move non-Iowa/NH primaries earlier, and those two will in response go even earlier to preempt them. (To the point where New Hampshire even passed a law saying that they have to be the first primary by at least 7 days.)
Given that the failed caucus app CEO is married to a Buttigeig staffer and was partying with the Iowa Democratic Party head and other members of the Buttigeig campaign right before the caucuses, I would say so:
"HuffPost, citing multiple Democratic sources and campaign finance records, reported amid the caucus chaos late Monday that Shadow Inc. was behind the app. ACRONYM, a Democrat-linked technology and campaign consulting nonprofit group, owns Shadow Inc."
"Shadow Inc.’s website has scant information about the firm and its employees. But a review of social media accounts shows that three of the Washington, D.C.-based firm’s top executives worked on the 2016 Hillary Clinton campaign.
The CEO of Shadow Inc., Gerard Niemira, was the director of product on Clinton’s campaign, his LinkedIn profile says."
This is a lot of stuff that has nothing to do with my comment or with the DNC, the subject of the parent comment, which, again, does not run the Iowa Caucuses.
This seems overly paranoid to me - wouldn't we want people with experience in the civics tech space to be leaders at a company working in the civics tech space? Besides, wouldn't most of the good Bernie tech people... go back to work for Bernie? There's a reason why the Hillary people need new jobs.
This is not even close to paranoid. Conflicts of interest are an important issue for nonprofits because it can cause them to lose tax exempt status, and the IRS makes a whole todo about it, and therefore so do competent boards. Trying to brush this under the rug because of politics is disgusting and would not be tolerated at ethical and competent organizations.
There is a very small pool of engineers who work full-time in politics because it is a small and highly cyclical industry with extraordinarily small margins. I have personally worked with many competing campaigns and nearly all vendors in the space do. Many organizations grapple with these conflicts of interest in and outside of politics, and do so successfully. From my perspective, it is quite paranoid.
Your sentiment is highly commendable, I wish I could offer guidance. However, do note that the IDP (Iowa Democratic Party) had the app made, not the DNC.
The Democratic Party must be destroyed to save it. This is a material manifestation of underlying shit, just a symptom at several degrees of remove from any more-than-superficial cause.
The only thing that could conceivably provide a systematic opening to fix things (ie, not just fixing specific things, after they've become apparent, because they've already catastrophically failed), in the realm of badly-written science fiction and fantasy, would be:
Bernie Sanders (or another equally popular candidate, in the near-future) is ratfucked so badly and obviously (which this could be the start of the perception of, even if it is just sufficiently advanced incompetence to be indistiguishable from malice), then the nominee loses to Trump so convincingly, that it completely shatters the party -- like it is no longer able to function at a basic level, and will clearly, even to the most delusional, never be able to win another election again -- and then... something, I don't even know how this would work. I am not creative enough to write this laughable science fiction any further.
There might also be a route involving Bernie Sanders winning both the primary and the general by a landslide, but that comes with its own and different set of systematic problems that I don't think would make the party's competence at dealing with technical problems, at least, any better.
But in either case, the deepest underlying rot is still the toxic interests of its financial backers, which are reproduced in the very bones of the party, and occasionally pop up in especially ridiculous and very indirect ways like this (and many more that don't get frontpaged as a problem on HN, or the New York Times for that matter). There's no fixing anything without somehow getting rid of them. And there is no realistically traversable path to that.
If they ratfuck him this time anywhere near as bad as they did last time, he'll run independent. In that case, Trump will win even more easily than I've been expecting since February 2017. Of course, Trump has a sort of death wish, so he could invade Canada or something and then Bernie would win without owing anything to the other face of the status quo party.
The most recent results (at 62% reporting, somehow not including urban areas, for many hours now) show Bernie winning the popular vote but trailing in convention votes. I hope I'll hear Democrats complaining about the Electoral College again soon...
Like all thinking adults, Bernie views Trump as a symptom of a deeper disease. Bernie has fought to fix USA's underlying problems for over four decades. Why would he care in a personal sense about the latest plutocratic figurehead? Trump's policies aren't significantly different than those of his predecessors; Bernie's policies will be.
Besides, Bloomberg is more of a Republican than Trump is. Why would Bernie care to see him elected?
tmpz22, where do you work in this space? I started Tuesday Company. I would love to talk more with you and hear about your experiences, I am sure we have a lot in common.
I don’t agree with the use of an app here for many reasons. That said, such an app would basically need secure authentication and a form to upload some pre-templated numbers.
It seems pretty hard to screw that up so badly but clearly it’s quite easy to make a complete dumpster fire from those requirements.
they used the a free tier of TestFairy that limited the number of test users to 200 and on iOS used TestFlight and did not use either the app store for the final version or make an enterprise version, and possibly could not distribute the app to the approximate 1700 users. They still would have needed to get the app through the app store approval process for general release to fix the app at least on the iOS side as far as I understand it. To fix it quickly would have just required them to go back to calling in results by voice as in past years. Don't know why they did not just do that.
> To fix it quickly would have just required them to go back to calling in results by voice as in past years. Don't know why they did not just do that.
They did that - but apparently the phone number for calling in the results was the same as for "help my app doesn't work", so the phone line was flooded. Unfortunately I don't have the source link to hand, so feel free to treat that as hearsay.
However, CNN had a live cross to someone trying to call in their results, and after an hour on hold, when they finally got through, the other side hung up on them while they were broadcasting live on air:
"It took an hour and a half for Shawn Sebastian to get hung up on in 10 seconds. And it unfolded on national television Monday, making the moment, in the words of some, “one of the most ridiculous things I’ve seen on cable TV.”"
According to multiple reports, the vote counts entered into app and sent were not what was received by HQ. How do we know that the company wasn't paid to change the numbers on the backend? How would the numbers change by themselves? I'll be called a tinfoil-hatter for assuming malice.
This is why we need to start using Byzantine Fault Tolerant distributed systems to vote via our mobile phones. No need for a voting holiday or standing in line. Bigger turnout, too. If it's secure enough for banking apps, why not for opt-in voting via app.
Maybe it's too hard to move to electronic voting nationwide. But every organization has governance and could use an electronic voting system based on BFT consensus of mutually distrusting parties. Vote using an app, it gets stored "on-chain", then you can check it on another app.
Wow! That provides some great info on this. Still leaves me wondering if the end result wasn't by design though. It's kind of hard to buy into the "Opps" explanation.
It certainly could've been done by design but I think more likely this was a doomed project from the start. They rushed this app out with too little time for testing.
I don't want to speculate too much on that screenshot but it looks like they have an error in their Auth0 integration. When you use a service like Auth0 you give it a redirect url to send authenticated users to. It looks like they passed in an invalid url for that redirect. Maybe they only tested this 2fa step locally and something was different when they deployed to users' phones, I can't say for sure, but it doesn't look good.
In particular it looks like Firefox Fenix/Preview (or possibly Reference Browser), not Fennec (regular Android). Only Fenix and Reference have bottom URL/tab bars. Fenix has a purple "Try Again" button, I don't know about Reference Browser, and Fennec has a gray button.
One possible issue is that Fennec/Fenix doesn't open http URLs in external apps by default (unsure about custom protocols), whereas Chrome does.
The jest of it is one of the most important institutions in the United States (the Democratic National Committee) uses a highly nepotistic and incompetent system for managing IT which leads to colossal failures in marketing, canvassing, and security. Not to mention massive PII violates as millions of emails, phone numbers and SSNs, are passed around in plain-text via CSV files.
The reason this happens is because hundreds of millions of dollars are lit on fire during election season and all the sharks, including former Google employees, come out to swim. Even well intentioned projects get slammed by the crunch of the election season (seriously trying shipping a well scaled app in < 2 months with terrible product direction) and ultimately fail - failing the needs of the entire citizenship of the country.
After the success of Obama's 2008 and 2012 campaigns even more money was funneled into IT as a sort of perceived silver bullet. But in 2016 it wasn't, and yet no analysis was done to correct the problems for the 2020 cycle - because the decision makers (all these "CTOs") are clueless fucks who are just there for the money and could care less about the integrity of our democratic system.
- in 2016 I worked for one of the companies in this niche and saw the bidding/sales/engineering processes first hand. FWIW I am a life long democratic voter and this makes me sick to my stomach.
Please don't post duplicate comments to HN. In addition to lowering the signal/noise ratio of the site, it creates a big pain when we go to merge threads (as happened here).
If you want to refer to something you said elsewhere, please use a link.
My bad, I got emotional over the content discussed and just copy-pasted the content because it seemed like the original thread would be eclipsed. I'll make sure not to do it in the future and in general have a great appreciation for the work you do moderating HN - I imagine it is quite stressful at times.
The owners and staffers at Shadow INC have a severe conflict of interest. The company is loaded with former Clinton staffers and the CEO sent a tweet in support of Buttigeig, who recently paid tens of thousands to the company.
Other suspicious dealings are a premature victory announcement by Buttigeig's campaign and a leak of a picture of paper tallies which included a PIN allegedly used to login to the tally app. Looking for the tweet now...
Aren't you though? I mean, your post is essentially "CLINTON" "ALLEGED" "SEVERE" "LEAK" . Lot's of words meant to imply some grand conspiracy. The reality is almost certainly that the developer (who has an unfortunate name) probably just didn't have enough time/budget to put out a decent product. Not everything is an actual conspiracy.
Is it really difficult to believe that a group of people in charge of something may be biased?
Calling it a "conspiracy" theory is just dismissive slander at this point. People have conspired successfully to achieve far greater and more malicious goals.
That would be an awful lot of work for such a tiny gain. And it doesn't scale. AT ALL. What moron would go to the trouble? Iowa picks corn, not presidents.
What hard work? How many devs do you have to pay off to write a handful of lines of code manipulating a secret ballot, especially when they may be willing because of personal bias? We know electronic voting in its current implementation is severely vulnerable to tampering, both external and internal.
>I think Georgia might be the most notorious for the behavior you describe.
I don't know if you're baiting me but personally I don't think corruption is limited to Democrats. Particularly if both sides start to believe the other is secretly cheating.
It feels like the American system works when a critical majority are playing with good faith - and that good faith is increasingly lacking.
Apparently, the app wasn't wasn't released until Feb 1. Also, The New York Times reported that the Iowa Democratic Party was secretive about using the app, and that it asked that Shadow’s name be withheld from the public. "According to a person familiar with the app, its creators had repeatedly questioned the need to keep it secret, especially from the Iowa precincts where it would be used."
This doesn't necessarily mean there is a huge conspiracy, but the optics -and the reality- of the party machine functioning properly is terrible, and the choice is between incompetence or malfeasance. Either way, they shot themselves in the foot, and it's not a good look.
As someone is not a huge fan of Democratic party - no conspiracy but total incompetence by Shadow -
https://www.theverge.com/2020/2/4/21122737/iowa-democractic-...
It's like they had no one on their staff who actually released an app through the Android or iOs app stores.
Should a company's employees/CEO have to be apolitical? I'd be surprised if there is such a company where none of its employees are involved in politics in any way. Especially if its a company whose sole purpose is developing software for a political party.
No, but until recently it was professional to keep your politics private and, more importantly, when there are inconsistencies disproportionately favoring an otherwise less popular candidate, public support from employees and especially the CEO, in conjunction with a money trail, these personal indicators of bias become relevant for justification of suspicion.
There doesn’t seem to be any indication this was anything but a regular screwup. Throwing around suggestions of a vague conspiracy looks unwarranted.
You’re suggesting that to avoid suspicion companies providing services to political campaigns must be apolitical and to have never previously provided services to a candidate in the race or any previous opponent of a candidate in the race. How could that even be possible?
What? In the case of the hypothesis, he gets more delegates. He also gets to take them from the person he is screwing (likely Sanders). Both of those immediate rewards have their own payoffs down the line (eg, in Buttigieg receiving an appointment to a grateful Warren administration or what have you) even if he does not win.
The idea that the Democratic nominating process would be rigged is not ipso facto crazy given that it is publicly acknowledged to have actually happened last election cycle.
The count has been frozen at "62%" for six hours now. That just happens to be the first/only level at which "Mayor Pete" had a delegate lead over Bernie, even though Bernie still leads the popular vote. Several days from now, when the count ticks up to 100%, Bernie will lead both measures by large margins. Still, in the meantime, the corporate media can pretend like the Mayor is ahead. They know he's going to bomb in SC worse than Biden bombed in Iowa, but they're hoping that somehow they can get him to "Super Tuesday" when the other Mayor can take over for him due to his 9-figure ad spend.
Of course, the idea that the perfect candidate to beat a vain old NY rich guy is another vain old NY rich guy who also banned soft drinks the last time he held elected office is risible, but they don't care about that. The point is to ratfuck any candidate who might plausibly do fewer stupid wars.
But your scenario doesn’t actually help Buttigieg. Also the app people can’t control the timing of the manual reporting, so there would somehow be a completely new element of the conspiracy. And you’re switching from a conspiracy orchestrated by Buttigieg to one orchestrated by the corporate media, Bloomberg, and/or the industrial military complex.
Buttigieg is 38 years old. His previous jobs were with McKinsey and "military intelligence" and his previous election total was 8,000 votes. He isn't trying to win the presidency. Iowa itself is a tiny state with a tiny number of convention delegates. The whole point of Iowa is that the winner gets some momentum. In this case, the winner won't get any momentum. That's the point.
Some people like to bore us with the "once is a coincidence" maxim. Some of them might like to pretend that this caucus is "once" and we'll have to wait a week or two to get upset about the ratfucking. Normal voters can remember four years ago, however, and this isn't even just the fourth time the national "Democratic" apparatus has ratfucked Bernie.
No, I'm saying those in support of ostensibly neutral third parties should not be making tweets indicating strong bias for one of the parties they are charged with treating neutrally.
Is it really so difficult to keep your public persona neutral and vanilla if you're a damn CEO? Wasn't this standard practice until recently? What, because Trump vomits on Twitter 24/7 everyone else needs to as well?
>You think it’s OK for CEOs of companies providing services to political campaigns to have political opinions as long as they keep them secret?
No, I think it's questionable when a CEO publicly discloses a strong political bias, receives private funds from the personally favored candidate, has cordial photos with the candidate at social events, and then is hired by the DNC and charged with being unbiased in a secret, electronically controlled ballot.
There are tens of millions of dollars changing hands in the ramp up to the election. This is not a complex scheme involving millions of mouths to keep shut and olympic mental gymnastics to justify. A handful of politically leaning and/or paid off devs add a package to the code and you have whatever results you want.
Ask yourself, do you really think corruption only exists outside of the United States? The bottom line is there are many discrepancies which happen to align toward a possibly calculated malicious intent. This feels like a "conspiracy theory" because there are so many tenuously linked components and no hard proof - but that's how any real life unfolding event looks until people have poured over it for months post facto and sorted through everything.
In any case what I'm saying is that there's plenty of incompetence and bias here to at least be prudently suspicious.
Good thing they are in the tally business and not the counting business, then. What is the difference, you might ask? One failed last night while the other one is occurring right now.
I'm not really feeling the difference between counting and tallying, but even keeping that distinction, it still sounds to me like they're mostly not in the tally business for lack of performance. If they had performed, it would still be a conflict of interest, just one none of us would likely have heard about. I feel that doesn't make anything better.
It is not a controversial statement to say that an election shouldn't be run with closed software made by a private company with financial, personal, and professional ties to one of the candidates. If something like this happened elsewhere in the world the US would use it as pretext for intervention.
If there was some grand conspiracy here, wouldn't it make more sense to release a working front end, and do whatever evil machinations / rigging behind the scenes...
Intentionally releasing a broken app is an idiotic excuse for a conspiracy.
If there was some grand conspiracy there wouldn't even be a news story. We'd have read Iowas results in the early hours of the morning and moved on with our lives.
"The app was not deployed through traditional app stores or even sideloaded using an enterprise certificate. Instead, it was deployed through the TestFairy testing platform, which is similar to Apple’s TestFlight and used predominantly for Android and iOS apps that are not yet finalized."
An app like this should not be on any app store ever. You want only pre-chosen people to have access to the app. It does no good for anyone else to have access to this and it can hurt.
4 years ago I had the Republican app on my phone. I was the backup reporter just in case the main reporter had problems. Since the main reporter used the app correctly I didn't use it, but I had access to it.
I agree, and there was more security. However access to the app implies the ability to DoS the remote with too many login attempts... Restricting access to the app was just one layer in the security scheme making it a little harder for someone who wanted to mess with the system.
Is it just me, or is the money just draining away from the economy right now? It's like we were all splashing around in the ocean and suddenly everyone is noticing that their toes are touching the bottom, and there's a few people standing on the steps, floaties on, looking around at us in horror.
Let me get this straight. A company deeply embedded in the Democratic establishment (0) that has worked directly with Buttigieg — the candidate with close ties to Facebook aka the company undermining democracy since 2016 — managed to totally screw up, potentially undermining the campaign of Sanders, the anti-establishment candidate.
If you are arguing a conspiracy of some kind it is probably best to rule out incompetence before moving on to active malfeasance. This is after all the same political party that managed rollout of the non-functioning Obamacare website.
Thats thinking in circles. If these hypothetical conspirators are so hyper-competent as to deliberately feign incompetence, there would be no evidence possible that could dissuade you, as all evidence against it would just be more evidence of a conspiracy.
Eventually you have to step back, take stock of your own experience with people, software, and their interactions, and make your own judgement. If you have no experiecne with software and people, then ask someone you trust who does. I don't see a whole lot of people who write software who are that surprised at what happened given the makeup of both the timing and the resources put into this app.
Well, there is long and detailed thread at the start of this page that begins with a comment from someone who started a different Democratic political software company and who turned down working on this app.
If you want a tl;dr, though, you might consider that (a) there's a paper trail, (b) the app was always intended to be optional and some precincts weren't using it in the first place, and especially (c) caucuses are not secret votes. That last one is how different campaigns had their own estimates before the official counts started coming out in haphazard fashion. Whatever bad things one can say about the idea of caucusing in general -- and we're hearing an awful lot of bad things -- they're actually pretty damn difficult to rig.
Buttigieg overperformed polling because polling only captures people's first choices. A lot of voters whose first choice candidates didn't meet the 15% qualifying point in the first round switched allegiance to him in the second, and the net effect was that a lot of moderates ended up in his camp rather than Biden's. This end result is probably an "only in a caucus" thing, but it doesn't require nefarious intent.
If you're to sabotage the caucuses and you're writing the software that tracks it, there's a thousand far more effective ways to do so and many of them wouldn't immediately shine a spotlight on you directly.
For malfeasance to remain undiscovered you need to be smart and clever.
For the kind of malfeasance that concatenates an SQL query with user input and then shows the whole thing to the user in an error message, you don't need that.
Chances are whatever software developers are used by the rnc also work for trump or whichever Rs campaign. If you need something done, you will probably be asking your buddy what they did to do that rather than digging through the local classifieds for someone with zero connection to your personal network at all. If your boss tells you to find a dev tomorrow, you are probably just gonna text your fellow intern buddy what company their group went with and go with that to make your life easier.
That being said, it's important to grasp what actually happened with the app rather than give in to misinformation and political apathy surrounding the news. Iowa voted by paper, so we are all just sitting tight while the hand count occurs as is traditionally done.
I'm pretty sure caucus votes are public anyway, it's not a secret ballot.
You can't really rig votes even with an app. Hanlon's razor probably applies here; never attribute to malice that which is adequately explained by stupidity
This is ridiculous. Paper. Use a Paper as the true vote. Cross check after that.
I have read so many comments on here up-selling the ability of technology but no one in this thread has the answer. It's apparent.
Do the paper vote - as it was always done. Let our technology analyze it afterwords. Why are we all trying to put our technology in front of this simple device. of paper.
You all know this rule = "make a single application do one thing well."
Again, this is ridiculous from a Credibility sense. Paper + networks, Paper + mobile, forget the paper, Mobile only, Mobile + scale. Where are we at? No where and much less. Just take a paper vote and tally it. Is that so hard?
I think the author's take on this is a bit heavy with hyperbole, but there's no doubt Iowa was a mess last night.
My son was there working as volunteer, and so was his ex-girlfriend, who's still a close family friend. I was chatting with them both throughout the evening and into the wee hours this morning.
They worked different locations in the Des Moines metro area and the results they reported to me were pretty close to the same in both.
I'll say this about it, it's hard to imagine that less than 4500 calls to a server over the course of an hour or two would "crash" it, or even ten times that number. And that I think an SQL db is a poor choice for an app like that.
CouchDB, or most any open source "nosql" db, would've been a better choice and, really, so would a dead simple flat file db to store the data for each precinct in and those are both very easy to build for something as simple as this.
And they didn't even need an "app" per se. All they needed was a password protected web page with a dead simple form.
What happened makes it a bit difficult to avoid pondering if the delays weren't deliberate. Whatever the case may be, the DNC is who ended up taking it on the chin.
I'm not sure what the db has to do with this, you need to structure the data somehow to be able to tally and breakdown results. SQL databases can certainly handle multiple magnitudes more transactions per second so it doesn't strike me as prima faciae bad decision making.
Database? Heck, I would think each precinct could have uploaded their own csv file in a common format. Concatenate all of those, pop it into excel, and pivot table away. But what do I know.
Any database should work just fine for this application. 1678 precincts. 11 candidates. 2 rounds. That's 37 thousand records to fully represent everything.
You can fit this in a google sheet with two tabs. 1678 rows, 11 columns, two tabs. Your central tabulators can create filter views for the precincts they are responsible for recording, and it includes full auditing of who changed what cell and when. Where can I collect my $60k?
> I have a feeling you haven't used SQL databases very often.
No, I've not. I just never liked the way it worked so I have to honest and open about my prejudice.
> This is exactly the kind of thing that [...] noSQL databases do poorly.
That's really not true. I've built web apps that do pretty much exactly what that voting apps does and I use CouchDB to do it because it was built specifically for this kind of thing, and it makes it very easy.
That's not the same as saying SQL won't work. Or even that it's a bad choice by design if it's what you're familiar with and good at using it.
How much time have you spent with CouchDB?
I do know that those coming from a long history of using SQL dbs often have a hard time using CouchDB. I had a hard time with SQL, so maybe it's a right brain/left brain kind of thing. Or maybe it's just not really wanting to learn a different way to do what you already know how to do with a different tool.
I chose to use a hand rolled flat file database early on (around 2000) as a backend for my web apps. When CouchDB hit v1.4 I switched to it because it worked very much the same as my db and offered a lot of advantages. It's gotten a lot better over the years since too.
So yeah, my view comes from a different perspective.
I've used CouchDB on and off for some years[1], although I've used SQL databases for longer. I've also used NoSQL databases ranging from MongoDB to Cassandra, Hadoop, BDB (and variants), DynomoDB, etc.
CouchDB (or any other NoSQL database) just isn't the best choice here. The application is literally tables of relational data with multiple simultaneous users which is pretty much the perfect use case for a SQL Database Server.
"The application is literally tables of relational data with multiple simultaneous users"
How is this data "relational"?
Each precinct has 3 numbers to submit for each candidate, that's not a big task for one person, and none of those precinct's candidates numbers are related to anything else in the db except the grand total of all precincts numbers and that's a one way relationship. The grand totals never change any of the data they're derived from.
If you want to give each candidate's precinct representative access to the precinct's db in CouchDB than you just limit their access to one file for their specific candidate in a db for their specific precinct, so by design it's purposely not related to any other candidate or precinct.
That's why CouchDB is a good choice for this kind of app.
You're right, I have not used CouchDB, but from my understanding it shines on unstructured data, where you might need to store slightly different things from different places.
But in a voting application structuring the data is actually very very helpful, it keeps bad data out, and helps with auditing.
Your response implies that structured data, including numbers, are a somehow a problem for CouchDB, but that's just not the case.
Validating data is something you can and should do before saving it in the DB and with CouchDB that would be done with a "design document" using javascript.
In this case validating that user input is a positive number or a zero should be done on the client side before even submitting the data as well.
What happens when two people share authentication (which will happen!) and access the app at the same time. Maybe you are going to try and implement file-locking, yourself, badly?
SQL has literally been used for this kind of app since the 1980s. It's simple to use, and easy to find developers who use it.
>>What happens when two people share authentication (which will happen!) and access the app at the same time.
If they both come from a different district and log into CouchDB they'll either only write to their file (one db, multi-user) or write to a file (or files) in their db (db per user).
If two users write to same file in same the db at the same time the DB will note the conflict and decide which data gets stored in the file.
Yes, this is how I remember CouchDB working. It leaves the hard parts of conflic resolution up to the client app:
Once you have retrieved all the conflicting revisions, your application can then choose to display them all to the user. Or it could attempt to merge them, write back the merged version, and delete the conflicting versions - that is, to resolve the conflict permanently.
You do realize that SQL databases handle this automatically via locking schemes and transaction, right?
The window is pretty small to create a conflict in a well designed CouchDB interface because of the way revisions are used. You should get the latest revision when you load the data for editing. And it's trivial to set a "locked" flag in a file being edited to prevent others from overwriting it if you have it configured so others can do that.
In this particular use case there really should only be one authorized user who can edit the data file for each precinct so race conditions like you describe just don't exist.
You can also use CouchDB's live sync feature to push the latest revision to everyone viewing the file in close to real time.
So conflicts are really a non-issue here. By design they just don't happen.
I'm not sure that proposing alternate solutions is so enlightening. Even so, they could have just had the person in charge of each location post the results to twitter. Then they wouldn't have had to do any tabulation, because the public would have done the tabulation for them...
In what way is a sql db a poor choice for this? You do know that there are tens of thousands of businesses that run strictly on postgres/mysql with more throughput than that?
My coworker has an alternate theory that frankly makes a lot more sense than “the app broke.”
The Iowa caucus is not a paper ballot. People stand in groups in a large multipurpose room and raise their hands for who they vote for. Depending on the viability threshold, people move to new groups and are recounted.
The exact rules are complicated and the process is run by unpaid humans. Moreover, in 2020 everyone has smartphones to post embarrassing mistakes on social media. And this year is a very crowded primary for the democrats, raising the chance for error.
Under this theory, the app is a convenient scapegoat to hide the fact that the process is inaccurate and bad. For politicians this seems pretty convenient. Who would you rather blame, and decades old tradition or an app contractor?
I don't think that's a better theory. The process may be 'bad' but it has been that way, and worked, for a long time. As far as we know, the new ingredient was the app.
There were some confounding factors, like an apparent day-of decision to require the caucuses to report the results of every vote cast, not just the last one. This led to considerably higher data requirements in precincts where multiple votes were necessary to reach consensus. The new requirements were apparently not reported to the staff until around noon on the day of the election.
As anybody can tell you, nothing sinks projects faster than changing requirements late in the development cycle.
The app also sucked for another reason. Lots of Iowa is rural and cell coverage can be spotty. The app was designed with the all too common assumption in this day and age that the Internet works everywhere.
The extra information has been known since 2016. Bernie (or his campaign) asked for it as a means of providing additional transparency into the process.
> The process may be 'bad' but it has been that way, and worked, for a long time.
The process has never worked. I attended a couple caucuses myself between becoming 17 [0] and leaving Iowa. I still hear about them from family. They have various flaws. The most severe one IMHO is that due to the time commitment, they only reflect the voice of die-hards and/or those with few responsibilities. That's true for choosing the candidates and even more so for choosing the platform.
However, I imagine you mean this specific failure mode (unable to produce an authoritative count within X hours for some smallish X) is new. AFAIK, that's correct.
I would love it if this failure inspires them to change to a standard primary process. Probably won't, though.
[0] iirc you can participate if you will be 18 by the general election.
Example of rule confusion: "No new voters are permitted to join the caucus after the first alignment. But in at least 70 precincts, more than 4 percent of the total, there are more tabulated total votes on final alignment than on first alignment."
That's just silly. The process has a lot of faults, but they're mostly around their inclusiveness. If anything, the fact that a bunch of people can visibly see the results of their respective caucuses, can report (e.g. tweet) on them real-time means that the ability to manipulate them is much, much harder.
Picture it, there's 100 people in a room who all have to move themselves around to their respective candidates and people count them and shout out the numbers.
All of the candidates are represented (they're reporting the numbers) and can hear the same numbers, all of the voters can hear them too.
It's messy, but it's probably one of the most difficult systems to "hack" in terms of real numbers since so many people know the results immediately.
I don't think the hacking malicious. People just make mistakes.
Random example: if candidate is below the viability threshold, that group is supposed to disband and join another group. You could have a mistake where, on the 2nd round, two non-viable groups merge and form a viable group.
For example, maybe Biden is at 14% and Klobuchar is at 2%. Under rules, those groups are both non-viable. But you could easily see a mistake where the caucus operators allow the 2% Klobuchar group to "merge" with the Biden group.
This app emerged from a mandate to make the caucus more accessible and transparent. It was well-intentioned but underfunded and lacked comprehensive organizational buy-in. Introducing tech can help but you have to spend tons of money to make it reliable and usable, then you have to spend more to train everyone in using it. This is a problem organizations of all sizes and shapes face when making massive IT changes.
Shadow is a firm that makes custom software for Democrats and progressives. It has an unnecessarily sinister name. There are not a lot of companies that make software for Democrats because it’s an awful job. You make very little money. Everyone hates you when things go wrong, which they will, because the product testing cycle and margins are nonexistent. Then everyone will assume things went wrong because you are some combination– you choose– of secretly evil, secretly working for Bernie, secretly working for The Establishment/Hillary (per someone's unpersuasive Imgur post below), or secretly working for Buttigieg.
Others have noted that Shadow also made software for the Buttigieg campaign. If you take my claims above as true, this should be unsurprising to you: a hard market where everyone hates you and no one has money to pay you is not attractive to enterprising software engineers, so there are few firms available to choose.