In his latest talk at the CUDA event, Andrej showcased his work on replicating the GPT-2 LLM using C and CUDA, effectively eliminating reliance on PyTorch and all dependencies except one.

The key takeaway is profound: PyTorch, once considered a massive and indispensable package for LLM and AI programming, is essentially a crutch for when AI isn’t yet powerful enough. In the near future, the mainstream approach will likely shift to ‘naked’ code built directly on C or CUDA. This is because advanced AI won’t require the high-level language crutches, such as Python.

Witness this transformation and embrace the future!