21–30 of 43 posts
Hey Nick and Ben, congrats to launch! I really like that you're going in the TS way by default. I personally think there will me more AI Engineers (devs building LLM apps/agents) working in TS than in Python. I wanted to ask if you accept PRs for integrations? I'm a co-founder of E2B [0]. We give private sandboxed cloud envs to any agent. We're building two things: - [1] Agent Protocol - it's an open protocol that de…
I've come to the conclusion that anything that "abstracts" the openai complete/chat complete API call is just bad practice and to stay away from the entire framework, with the exception of microsoft guidance. Just because you can, doesn't mean you should. And if you do abstract the completion API, then it must either reduce friction or increase capabilities over just calling openai with http fetch/axios. Which micros…
We chose to do this for our first module to take a stab at integrating RAG pipelines in a coherent manner, but we don't plan on following this pattern in all modules within our framework. There is possibly one exception here, which is that an interface that allows composable middleware for things like logging, error handling, or redirecting of requests may justify wrapping in some places.
The next steps for us involve lower-level functionality. One need we see again and again is more robust data extraction and processing. Most people we talk to who use other community projects (e.g., langchain or llama) find that data loading and chunking are among the most valuable parts of those libraries. We agree, but would like more robust functionality for these tasks, so this is one thing we're working towards next.
Beyond that, we're working on infrastructure. Easy model serving from Node (for OSS or proprietary models), monitoring, and pipelines for fine-tuning based on production inference results.
We use GPT-4 pretty heavily in a Typescript project, but have noticed lag from the TS versions of popular libraries (OpenAI’s npm lib, Langchain TS, etc.). This framework is exciting to see. Even though Python is the “language of AI” most foundational models just sit behind an HTTP endpoint, making the web (and thus JS/TS) a perfect fit, as you’ve called out. It’d be neat to see a caching layer (maybe similar API to…
We absolutely want to add a caching layer. Actually, we think middleware is where a lot of the value of the framework will come: it enables a whole bunch of features, e.g. sending errors to datasets for labeling, caching, user throttling, analytics, A/B tests, ...
We're likely going to build the serving module next which will cover this.
I've come to the conclusion that anything that "abstracts" the openai complete/chat complete API call is just bad practice and to stay away from the entire framework, with the exception of microsoft guidance. Just because you can, doesn't mean you should. And if you do abstract the completion API, then it must either reduce friction or increase capabilities over just calling openai with http fetch/axios. Which micros…
Now's the time to learn how LLMs work from the ground up not being a framework chaser. (watch Karpathy's GPT from scratch video and read through huggingface's LLM documentation from RLHF to PEFT fine-tuning)
We use GPT-4 pretty heavily in a Typescript project, but have noticed lag from the TS versions of popular libraries (OpenAI’s npm lib, Langchain TS, etc.). This framework is exciting to see. Even though Python is the “language of AI” most foundational models just sit behind an HTTP endpoint, making the web (and thus JS/TS) a perfect fit, as you’ve called out. It’d be neat to see a caching layer (maybe similar API to…
Glad to hear this, indeed we think there's opportunity for some more cutting edge tooling in the TS ecosystem. We absolutely want to add a caching layer. Actually, we think middleware is where a lot of the value of the framework will come: it enables a whole bunch of features, e.g. sending errors to datasets for labeling, caching, user throttling, analytics, A/B tests, ... We're likely going to build the serving modu…
We're open to trying more TS-focused libraries, but definitely more hesitant after our initial experiences with other libs. The less magic the better (no hidden prompts, etc.).
Still, I'm a huge fan of TypeScript and will give it a try anyway :)
I'm gonna be that guy who will probably show up sooner or later anyway, but... I can't imagine performance can compete with other languages? What were your findings or experience with that? Still, I'm a huge fan of TypeScript and will give it a try anyway :)
Before I share some thoughts on this, let me just say that our primary motivators for Axilla have much more to do with bringing better AI tooling to an otherwise flourishing ecosystem rather than shaving milliseconds off an arbitrary task or request. Given that, I'm not sure how fruitful a performance discussion will be.
If by performance you meant maturity of third party packages for AI-related functionality, then yes JS/TS is lacking. This is what is motivating us :). We want better tooling for AI applications in TS.
If you're referring to performance for CPU-bound tasks, then yes JS would not be as good as lower-level languages like Rust or Go. If you're referring to JS compared to Python, then I don't know how true that is. Python doesn't have a great concurrency story either (at least not today). JS may be single threaded for the most part, but with web workers and WASM (+ WebGPU!), we now have tools at our disposal for dramatically speeding up CPU-bound tasks while not blocking the main thread. Assuming we get the interfaces right, we can swap out a subset of the implementation with a WASM-based implementation later if justified.
There is nothing about Python the language that makes it especially well-suited for AI/ML-related functionality. It is just the language whose ecosystem has the most maturity when it comes to that functionality. We hope to chip away at that over time.
Hey Nick and Ben, congrats to launch! I really like that you're going in the TS way by default. I personally think there will me more AI Engineers (devs building LLM apps/agents) working in TS than in Python. I wanted to ask if you accept PRs for integrations? I'm a co-founder of E2B [0]. We give private sandboxed cloud envs to any agent. We're building two things: - [1] Agent Protocol - it's an open protocol that de…
I'm gonna be that guy who will probably show up sooner or later anyway, but... I can't imagine performance can compete with other languages? What were your findings or experience with that? Still, I'm a huge fan of TypeScript and will give it a try anyway :)
Hey Chris , can you further qualify performance? Before I share some thoughts on this, let me just say that our primary motivators for Axilla have much more to do with bringing better AI tooling to an otherwise flourishing ecosystem rather than shaving milliseconds off an arbitrary task or request. Given that, I'm not sure how fruitful a performance discussion will be. If by performance you meant maturity of third pa…
But now that I'm looking at the actual code samples, I'm not even sure JavaScript is doing any of the actual heavy lifting? (I see you use OpenAI's embedding) so this tool is more of the glue connecting all the parts? Again, I'm out of my wheelhouse here.