WinDbg Preview's Time Travel Debugger can do this: https://docs.microsoft.com/en-us/windows-hardware/drivers/de... The great advantage of such a debugger is that you can use it to follow data flow: if you see an incorrect value somewhere, you can set a memory breakpoint and then run the program backwards to jump to the point in time where the value was written.

I've only used it with C++ applications so far, but the WinDbg blog claims that it also works when debugging managed code with the SOS debugging extension.

Visual Studio confusingly has the somewhat-similar "Intellitrace", but AFAIK that only records previous states collected for certain events (breakpoints, first-chance-exceptions, ...). It can't rewind the program an arbitrary point like the time-travel debugger can, so it's not useful to follow data flow.