Sakana AI　Can LLMs Invent Better Ways to Train LLMs?

Sakana AI 2024-06-13

Sakana AI’s research leverages evolutionary algorithms to improve LLM training methods. The new loss function “DiscoPOP,” discovered via LLM-driven processes, outperforms existing methods. The study demonstrates using LLMs to propose and implement new optimization algorithms autonomously.

Source (Published: June 13, 2024)