Those look all programming related tokens, and I think they should relate to the...

		imrehg on June 29, 2023 \| parent \| context \| favorite \| on: XGen-7B, a new 7B foundational model trained on up... Those look all programming related tokens, and I think they should relate to their focus of improving this model's code generation capabilities, and adding quite a bit of data related to that (BigCode Starcoder in the Second stage in pre-training https://blog.salesforceairesearch.com/xgen/#pre-training-dat... )