And to be more exact, GPT-4chan is based on GPT-J (same architecture as GPT-3 whose weights were never released) which only had 6B params and that was back in 2021-2022.