Why doesn't Pro include longer context windows? I'm a Plus member, and the biggest limitation I am running into by far is the maximum length of a context window. I'm having context fall out of scope throughout the conversion or not being able to give it a large document that I can then interrogate. So if I go from paying $20/month for 32,000 tokens, to $200/month for Pro, I expect something more akin to Enterprise's…
The longer the context the more backtracking it needs to do. It gets exponentially more expensive. You can increase it a little, but not enough to solve the problem. Instead you need to chunk your data and store it in a vector database so you can do semantic search and include only the bits that are most relevant in the context. LLM is a cool tool. You need to build around it. OpenAI should start shipping these other…
Isn't that kind of what Anthropic is offering with projects? Where you can upload information and PDF files and stuff which are then always available in the chat?