Discussion about this post

User's avatar
Markus Kohler's avatar

There is still for the time being the question of performance and cost even if the context size gets bigger. Therefore I wouldn’t be do sure that simple implementations which just send everything would be always competitive. Just sending what is needed might result in dramatic cost savings and at the same time improve performance.

Expand full comment
Richard Guinness's avatar

Really interested to see how "well" these very long contexts are "used" (looking at paper "Lost in the middle: how language models use long contexts")

Expand full comment
2 more comments...

No posts