31–40 of 63 posts
Earlier quoted context omitted.
That's absolutely not what they said. Let's keep hn better than that.
It's obviously tongue-in-cheek, but it's also not not what they said.
> Python 3.12 improves the language perf by 4% A bit shy of the original target of 50% improvement per release, totaling 5x speedup across four releases. Hope it catches up in the next couple releases.
Python 3.12 also does not exist, because it has not been finished and released yet. So there's time.
hmmm
I just tried some pre-trained models to classify images, using a CPU, it takes about 10s to 15s per image. I did not know it was so inefficient.
Would you share some details? This indeed sounds very slow, so I suppose there should be some easy ways to speed things up. Just to give you an idea of what's possible: A couple years ago I worked on live object recognition & classification (using Python and Tensorflow) and got to about ~30 FPS on an Nvidia Jetson Nano (i.e. using the GPU) and still ~12 FPS on an average laptop (using only the CPU).
using:
cfg.apply_low_vram_defaults()
interrogate_fast()
I tried lighter models like vit32/laion400 and others etc all are very very slow to load or use (model list: https://github.com/mlfoundations/open_clip)
I'm desperately looking for something more modest and light.
Are there any that allow a user to run an LLM with GPT-3-ish capabilities on a single pc w/ Where "reasonable" is something like no more than a few minutes to get the output. A little longer wouldn't be too bad either since you could script something that submits prompts automatically and let things run in the background or something.
Trying to search for such a thing-- if it exists-- is nearly impossible right now with so much being done & talked about. Some content talks about running something on only 16GB GPU ram (!!!) which is far beyond many discrete cards. a 3080 has up to 16, some only 8 (I think 16 is the max?) and fairly decent entry level+ cards top out at 4GB.
Any options out there?
Earlier quoted context omitted.
Python has exploded in popularity over the past ~7 years because of AI. Python’s ever increasingly relevance is directly tied to AI.
And Flask, and FastAPI, and pandas, and pola.rs, and numba, and as glue code, and.... "AI" is pretty narrow, but the python ecosystem is massive, which a strong incumbency in the data processing space. Weird, since it i̶s̶n̶'̶t̶ ̶a̶ ̶t̶y̶p̶e̶d̶ is a dynamically language, where sometimes placeholders like pandas `object` result in unexpected behavior, but it sure is simple to write something quick.
Having used it for over 15 years, I assure you Python is a strongly typed language.
>Most new Python projects are still about AI Are there any that allow a user to run an LLM with GPT-3-ish capabilities on a single pc w/ Where "reasonable" is something like no more than a few minutes to get the output. A little longer wouldn't be too bad either since you could script something that submits prompts automatically and let things run in the background or something. Trying to search for such a thing-- if…
Earlier quoted context omitted.
And Flask, and FastAPI, and pandas, and pola.rs, and numba, and as glue code, and.... "AI" is pretty narrow, but the python ecosystem is massive, which a strong incumbency in the data processing space. Weird, since it i̶s̶n̶'̶t̶ ̶a̶ ̶t̶y̶p̶e̶d̶ is a dynamically language, where sometimes placeholders like pandas `object` result in unexpected behavior, but it sure is simple to write something quick.
> "AI" is pretty narrow, but the python ecosystem is massive, which a strong incumbency in the data processing space. Weird, since it isn't a typed language . . . . Having used it for over 15 years, I assure you Python is a strongly typed language.
Earlier quoted context omitted.
So weird - you really think AI/ML is a flash in the pan when it comes to Python? You realize pytorch, tensorflow, pandas, numpy, scikit-learn are all approaching 10 years on pypi?
Yeah all the stuff I was seeing seemed to be variations and light wrappers around openai APIs and such. We don't need a million versions of those things, one or two clear 'winners' should emerge. You can see them here, expand it to 50 per page and start browsing down after the usual suspects: https://awesomepython.org/
Also I don't know how this list is built but it seems to be literally measuring current popular things - so by definition no matter when i visit this list whatever is it at the top will probably wane in popularity shortly after I've visited.
Yet what I am missing in the news is some hint of where Python is going longer term. What will 4.0 look like? Which of the know limitations will be addressed? Will it ever be performant on a standalone basis? Will it ever be native on mobile? Etc.
I just tried some pre-trained models to classify images, using a CPU, it takes about 10s to 15s per image. I did not know it was so inefficient.