Open Source Models Outpace Google and OpenAI: The Cost of Free Public Involvement
- Google and OpenAI have been seen as leaders in AI, but open source models are outpacing them
- Open Source models are faster, more customizable, more private, and pound-for-pound more capable
- The success of open source is due to the low cost public involvement enabled by cheaper fine-tuning mechanisms
- Competing with open source is a losing proposition for large institutions
- Facebook benefited from the leaked model since it garners an entire planet of free labor when most innovation builds on top of their architecture
- No one will have a moat and open source alternatives can eclipse them.
Revolutionary AI Efforts Spark Rapid Innovation
- In February of 2023, a language model known as “Llama” was launched
- A few days later it leaked to the public, and shortly after that Artem got the model working on a Raspberry Pi
- Stanford quickly followed with “Alpacat”, which allowed fine tuning within hours on an Nvidia card, driving the cost of AI-related efforts down significantly
- Then, “Vicuna” was released and eventually an open source GPT3 was created using the Llama leak, making it no longer dependent on Facebook or Metamind.