This is the great cover of the world of AI. Anthropic is a startup founded by former OpenAI engineers who were unhappy with it. Its advances have attracted the interest of Google, which has invested $400 million in it, and crystallized with Cloud, a chatbot that is now dusting off ChatGPT and GPT-4. As? Reading a lot
The more tokens, the better. Current conversational AI models are better the more information they are able to absorb in each query. This is the so-called context (or context window), and is measured in tokens, the basic units into which the information “captured” by the model is divided. A token may be equivalent to a single word, but it is also often equivalent to a smaller set of words. Google Bard has a 2K token limit – which also seems to be the cap for Bing Chat – GPT-3 and ChatGPT support 4K, and GPT-4 works out of the box with an 8K limit, but there is a version of the latter Which goes to 32K tokens.
Cloud 1, GPT-4 0. However, in Anthropic they have made a brutal leap forward in this section. Cloud’s “context window” was 9K tokens, but they have now made it possible to support a window of 100K tokens, which according to the company allows for approximately 75,000 words to be entered in a single prompt. What does it mean?
“Claude, read this novel in one sitting.” The easiest way to understand this is to point out that Claude is able to read a novel in seconds, and from there answer any question about that novel. With ChatGPT we have to present the novel in small pieces, and it is not at all trivial to treat all those pieces as a text in order to ask questions later. This is what GPT-4 makes easy with that 32K upper limit, but goes much further with Anthropic Cloud.
Companies shake hands. This is especially interesting for companies, who will be able to use the cloud to present (boring?) documents of tens (or even hundreds) of pages with the multitude of data on which we all Kinds of requests can be made. Claude will be able to summarize them, make tables and draw conclusions or generate ideas from it. It would be like being a robotic analyst or consultant ready to consume data and extract ideas, information and conclusions.
It also consumes audio. Not only that: that reference window is also valid for audio. Those 100K tokens means that the cloud can “consume” about 6 hours of podcasts. This assembly was done by AI, which aired a very long episode of 58K words and which was fully voiced by Cloud.
And, of course, the code. The same advantage holds in the case of developers. For example, programmers can give an entire software project as input, or have the cloud read 240 pages of documentation for an API, and then start asking questions about it.
