wow, great experiment. I'm amazed the whole team went through with duplicating everything for that long. Nice work :)
I resorted to feels. After decades of programming, I know when I'm being productive, and I can reasonably estimate when a colleague is being productive. I extrapolate that to the LLM, too. Absolutely not an objective measure, but I feel that I can get the LLM to do in a day a task that would take me 2-3 weeks (post-Nov 25 and using parallel agents).
I did too also generally resorted to feels. However, as a team we decided that we need to convince ourselves that LLMs are both an accelerator as well as not introducing flaws in our process (regressions, performance issues, security issues...). My team is pretty awesome and we generally try not to "fall for FOMO" type deals, like "everyone is using ____, surely we must as well."
just like HN, we had team members that were hesitant to say the least in the beginning as well as team members that were "convinced" a lot earlier that LLMs can be a great accelerator to what we are doing. I would venture a guess that similar situation exists(ed) in many places. so we tried to figure out a way where it is not just portion of the team "putting the foot down" so-to-speak but more like "OK, lets see if this can be measured so that everyone (or lets say overwhelming majority) is on board." I wish I would read that more teams at least attempted this approach...
I resorted to feels. After decades of programming, I know when I'm being productive, and I can reasonably estimate when a colleague is being productive. I extrapolate that to the LLM, too. Absolutely not an objective measure, but I feel that I can get the LLM to do in a day a task that would take me 2-3 weeks (post-Nov 25 and using parallel agents).