In an effort to promote greater collaboration, ChatGPT-like open source models have been made available to the research and business community by artificial intelligence chip startup Cerebras Systems on Tuesday.
Seven models, from smaller 111 million parameter language models to a larger 13 billion parameter model, were all made available by Silicon Valley-based Cerebras. They were all trained on its Andromeda AI supercomputer.
According to Andrew Feldman, founder and CEO of Cerebras, “there is a big movement to close what has been open sourced in AI…not it’s surprising as there’s now huge money in it.” “It has been so open, and that has contributed greatly to the excitement in the community and the progress we’ve made.”
Models with more parameters can carry out more intricate generative operations.
For instance, ChatGPT, an OpenAI chatbot that was released late last year, has 175 billion parameters, can produce research and poetry, and has helped to generate significant interest in and funding for AI more generally.
While larger models run on PCs or servers, Cerebras claimed that smaller models can be used on smartphones or smart speakers, although complex tasks like summarizing lengthy passages call for larger models.
Bigger isn’t always better, though, according to Cambrian AI chip consultant Karl Freund.
If you give a smaller model more training, it can become accurate, according to some interesting papers that have been published, said Freund. Therefore, there is a trade-off between size and skill.
Due to the design of the Cerebras system, which includes a chip the size of a dinner plate designed for AI training, Feldman claimed that his largest model only required a little more than a week to train. This is a significant improvement over the several months that it typically takes.
The majority of AI models in use today were developed using Nvidia Corp. (NVDA.O) chips, but more and more startups, including Cerebras, are vying for market share.
According to Feldman, the models developed on Cerebras machines can also be customized or further trained on Nvidia systems.
