The Go compiler was initially written in C but is now entirely written in Go. The Rust compiler is written in Rust (initially in OCaml). These compilers are capable of compiling themselves. Linux is compiled and developed on Linux. PyPy is a self-hosted Python interpreter.
This makes sense as folks pursue chatgpt 4.0 as the gold standard. But it will not lead to step wise improvements. OpenAI hints larger models are reaching diminishing returns. Where does the next big leap come from if everyone is tuning and chasing and asking opengpt which output is better? Can these algorithms search / evolutionary optimize to get to the next level? Is there enough gpu / toys in the world for that?