I've always found training models on code to be interesting – there's large amounts of structured data and text is a fairly simple input/output interface that can be automatically be checked for syntax correctness.