I don't I'm afraid, for starters I contract and no longer work on the project, plus there's IP invovled etc. Also I'm no data scientist, just a hacker really :)
The main difficulty was getting access to the data (and ensuring it was valid, as with all ML projects), lucking we managed to get that from several sources (councils, water companies etc). The flow data was in TimescaleDB, pandas dataframes so we could use varying levels of frequency of data and we used HDF5 iirc as well (detail is hazy, it was a few years back now).
We did demo it to the Met Office here in the UK too. They were interested but already had their own thing cooking up so the project never really got out of prototype, but it was making accurate predictions. I think there were some other areas that might turn out to be flaky over time using this method (such as rapid changes to catchment areas) but that could maybe be factored in someway with more thought on the model and verification on a larger set/timeframes. Feel free to hit me up if you want any more detail on the tech side, but don't ask me about stats, maths fu I ain't :)
The main difficulty was getting access to the data (and ensuring it was valid, as with all ML projects), lucking we managed to get that from several sources (councils, water companies etc). The flow data was in TimescaleDB, pandas dataframes so we could use varying levels of frequency of data and we used HDF5 iirc as well (detail is hazy, it was a few years back now). We did demo it to the Met Office here in the UK too. They were interested but already had their own thing cooking up so the project never really got out of prototype, but it was making accurate predictions. I think there were some other areas that might turn out to be flaky over time using this method (such as rapid changes to catchment areas) but that could maybe be factored in someway with more thought on the model and verification on a larger set/timeframes. Feel free to hit me up if you want any more detail on the tech side, but don't ask me about stats, maths fu I ain't :)
joel [at] smashthesystems.com