I've always found training models on code to be interesting – there's large amounts of structured data and text is a fairly simple input/output interface that can be automatically be checked for syntax correctness.
LLMs for Code
LLMs for Code
LLMs for Code
I've always found training models on code to be interesting – there's large amounts of structured data and text is a fairly simple input/output interface that can be automatically be checked for syntax correctness.