101–110 of 135 posts
Also: I simply asked to the 7B model "hi, who are u": Thought for 1 seconds Okay, the user is asking "Who are u?" with a casual tone ("u" instead of "you", exclamation mark). Let me break this down. First, I need to address the question itself. They want to know who I am as an AI assistant. My standard response is that I'm DeepSeek R1, designed to be helpful and handle reasoning step by step. But maybe they're not fa…
This is how the future of "AI" has to look like: Fully-traceable inferences steps, that can be inspected & adjusted if needed. Without this, I don't see how we (the general population) can maintain any control - or even understanding - of these larger and more opaque becoming LLM-based long-inference "AI" systems. Without transparency, Big Tech, autocrats and eventually the "AI" itself (whether "self-aware" or not) w…
I'm just now moving my main workflows off openai over to local models and I'm starting to find that these smaller models main failure mode is that they will accept edgecases with the goal of being helpful. Especially in extraction tasks. This appears as inventing data or rationalizing around clear roadblocks. My biggest hack so far is giving them an out named "edge_case" and telling them it is REALLY helpful if they…
Just curious - are you using Open WebUI or Librechat as a local frontend or are all your workflows just calling the models directly without UI?
Lots of optimization left there, but the systems are pinned most of the time so not focused on that at the moment as the gpus are the issue not the queuing.
I'm just now moving my main workflows off openai over to local models and I'm starting to find that these smaller models main failure mode is that they will accept edgecases with the goal of being helpful. Especially in extraction tasks. This appears as inventing data or rationalizing around clear roadblocks. My biggest hack so far is giving them an out named "edge_case" and telling them it is REALLY helpful if they…
I'm just now moving my main workflows off openai over to local models and I'm starting to find that these smaller models main failure mode is that they will accept edgecases with the goal of being helpful. Especially in extraction tasks. This appears as inventing data or rationalizing around clear roadblocks. My biggest hack so far is giving them an out named "edge_case" and telling them it is REALLY helpful if they…
Can you describe this more? Is “edge_case” a key in the structured output schema?
I got excited by reading the article about releasing the training data, went to their HF account to look at the data (dolma3) and first rows? Text scraped from porn websites! https://huggingface.co/datasets/allenai/dolma3
That said I like to think of it was my dataset I would have shuffled that part down in the list so it didn’t show up on the hf preview
I got excited by reading the article about releasing the training data, went to their HF account to look at the data (dolma3) and first rows? Text scraped from porn websites! https://huggingface.co/datasets/allenai/dolma3
Isn’t this before any curation has happened? I looked at it, I can see why it looks bad, but if they’re really being open about the whole pipeline, they have to include everything. Giving them a hard time for it only promotes keeping models closed. That said I like to think of it was my dataset I would have shuffled that part down in the list so it didn’t show up on the hf preview
Earlier quoted context omitted.
Isn’t this before any curation has happened? I looked at it, I can see why it looks bad, but if they’re really being open about the whole pipeline, they have to include everything. Giving them a hard time for it only promotes keeping models closed. That said I like to think of it was my dataset I would have shuffled that part down in the list so it didn’t show up on the hf preview
Hard time? What value does adult videos description, views and comments add to small (7,32B) models?
However, the use of this text for training might be transformative enough to constitute fair use, in which case a license from the authors would be unnecessary. For now this is an unsettled legal question, but it's not going to stay unsettled for long, at least not in the US. In fact, we've already seen two judges address the question in summary judgement rulings and reach roughly opposite conclusions [1]. One of those cases has since been settled, but inevitably, some of the many ongoing AI copyright cases will make their way to appeals courts, and probably the Supreme Court.
In the long run, I suspect that this will be allowed one way or another. Either courts will make a finding of fair use, or Congress will step in and create some kind of copyright carveout. Both have their limitations: court rulings tend to draw fuzzy lines around what conduct is allowed and what isn't, while legislation draws sharp lines that tend to be too sharp (with random restrictions and carveouts based on negotiations).
If so, what happens next? Some free software purists will never accept this type of use, and they'd have reasonable grounds for not doing so (legal uncertainty in the rest of the world, or moral/ethical grounds). But I think it would be a mistake for the free-software world broadly to reject it. This type of model is as open as is physically possible, and represents a real improvement in user agency compared to mere open-weights models, let alone compared to the closed models that seem to be getting increasingly dominant.
Anyway, we'll see.
[1] https://www.skadden.com/insights/publications/2025/07/fair-u...
Earlier quoted context omitted.
I'm only superficially familiar with these, but curious. Your comment above mentioned the VL model. Isn't that a different model or is there an a3b with vision? Would it be better to have both if I'd like vision or does the vision model have the same abilities as the text models?
This has been my question also: I spend a lot of time experimenting with local models and almost all of my use cases involve text data, but having image processing and understanding would be useful. How much do I give up (in performance, and running on my 32G M2Pro Mac) using the VL version of a model? For MOE models, hopefully not much.
i think the model itself is actually "smarter" because they split the thinking and instruct models so both modalities become better in their respective model
i use it almost exclusively to OCR handwritten todo lists into my todo app and i don't think it's missed yet, does a great job of toolcalling everything