The New Foundation Models of AI Lack “Responsible” Paperwork
If you’ve seen photos of a teapot shaped like avocado or read a well-written article that veers off on slightly weird tangents, you may have been exposed to a new trend in AI . Machine learning systems called DALL-E, GPT and PaLM are making a splash with their incredible ability to generate creative work. These systems are known as “foundation models” and are not all hype and party tricks. Foundation models are models trained on broad data (generally using self-supervision at scale) that can be adapted to a wide range of downstream tasks. These models, which are based on standard ideas in transfer learning and recent advances in deep learning and computer systems applied at a very large scale, demonstrate surprising emergent capabilities and substantially improve performance on a wide range of downstream tasks. Given this potential, foundation models are seen as the subject of a growing paradigm shift, where many AI systems across domains will directly build u...
