Symbolica, an artificial intelligence startup focused on developing foundational models to rival OpenAI, the creator of ChatGPT, has disclosed securing a $31 million series A funding round led by Khosla Ventures. Joining in the funding were General Catalyst, Abstract Ventures, and Buckley Ventures, as stated by the company.
George Morgan, CEO of Symbolica and former Tesla employee working on its self-driving system, revealed that the company has devised a framework to create alternatives to the widely-used “transformer” deep learning architecture. This innovation was outlined in a paper co-authored with Google’s AI subsidiary DeepMind. Transformers serve as the core technology behind ChatGPT and the current race in generative AI.
Morgan emphasized, “The transformer is not the ultimate solution in AI.” He highlighted the prevalent trend in the industry of building upon existing structures through successive modifications.
There’s a vigorous debate within the AI community regarding whether further advancements in performance can be achieved merely by increasing compute power and training data. While some, like Microsoft-backed OpenAI, are heavily investing in scaling up compute, others, such as Symbolica, believe that different architectures of foundational models could yield superior results compared to scaling transformers.
Christopher Kauffman, a partner at General Catalyst, remarked, “The industry at large is grappling with the limitations of transformers, whether it’s the escalating costs associated with scaling or their reliability.”
Symbolica’s initial offering will be a coding assistant, slated for launch in early 2025. However, Morgan noted that the company requires time to recruit and train its model before its release.
