In my benchmark Deepseek-v4-flash did much better than Qwen 3.8 27B at reverse engineering.
I gave Qwen 3.8 27B a reverse-engineering job and it finished in 30 minutes
41–50 of 159 posts
Re: I gave Qwen 3.8 27B a reverse-engineering job and it finished in 30 minutes
#42Local models would be even better if they did not ship with all the refusal shenanigans built-in. You can safely bet organized crime has access to the best models without these hoops, which makes the case that the average user (=non-criminal) should have access too. As I understood from an ex-Anthropic employee, some orgs got access to Mythos based on their high enough spending level, not on other grounds. Either we…
I’d expect these shenanigans to get much worse over time for the average Joe. Imagine a world where any random person can run a super-capable model on their own hardware with no limitations and no one to pull the plug. Information has always been power and those who already have power won't just allow everyone else having the same tools as them
> Imagine a world where any random person can run a super-capable model on their own hardware with no limitations and no one to pull the plug.
It's an arms race. You have to run increasingly capable model partly because others can or do.
Re: I gave Qwen 3.8 27B a reverse-engineering job and it finished in 30 minutes
#43i'm not good with paper work, in fact, i'm horrible with anything that's paperwork related. for the past few days, i ran this model on my rtx 4090 + rtx 3070 and told it to check all the bills, invoices, contracts for me and my small company. i used pi with llama and the pi-llama plugin. oh, boy - i hooked it to my email, told it to download all of the invoices and bills i had for both me and my company and organize…
Pro tip: usually you can download your emails to an offline collection. That way there's no risk. Hooking it up to your email sounds very very risky.
Re: I gave Qwen 3.8 27B a reverse-engineering job and it finished in 30 minutes
#44[flagged]
In my experience xhigh very often produces better results on the first try to the extent that the less cogitation-enthused settings actually waste more in the long run.
Re: I gave Qwen 3.8 27B a reverse-engineering job and it finished in 30 minutes
#45Earlier quoted context omitted.
Completely coincidentally, we're just about to launch a service that does exactly this (API access to uncensored open models)! We have a waitlist at the moment but will be live very soon! https://violentdelights.ai
Given the context, your domain name is 'chefs kiss', perfect. I am completely curious what your legal defense would be though. "Come do things with AI that are probably illegal!" What?! We had no idea people would do things that are illegal!
I guess we'll burn that bridge when we get to it!
Re: I gave Qwen 3.8 27B a reverse-engineering job and it finished in 30 minutes
#46Local models would be even better if they did not ship with all the refusal shenanigans built-in. You can safely bet organized crime has access to the best models without these hoops, which makes the case that the average user (=non-criminal) should have access too. As I understood from an ex-Anthropic employee, some orgs got access to Mythos based on their high enough spending level, not on other grounds. Either we…
Hugging face is filled with uncensored versions of your favorite local models, so in a way they are shipped without the refusal stuff, via the magic of fine tuning or however they get this stuff out of models.
Re: I gave Qwen 3.8 27B a reverse-engineering job and it finished in 30 minutes
#47> The first attempt at recovering the key was wrong in a very specific way; it produced a working key and the signature check passed, but a hash the binary computes as an integrity check didn't match. In my experience, most models would have called it done and left it at that, but Qwen 3.8 27B didn't do that. Instead, it highlighted the mismatch, went back to the drawing board, and kept going until the value matched…
Well, it seems that Linus doesn't use those: > And this was a debug session from hell, enormously helped by an AI doing much of the grunt-work. > I'd like to call it my tireless helper, but the AI several times stated flat out that this was impossible and unsolvable and that we should just write a report about it. > I suspect those things have been trained by people who may not be quite as stubborn as I am. https://g…
Both things can be true. I’ve noticed both the same thing the parent posted and what Linus posted and my vibe on the split (I haven’t kept detailed notes) is that on greenfield code they tend to maybe over-verify and on brownfield code or data analysis they sometimes give up too early or… I’m not sure, need a bit of encouragement to keep pulling at threads.
On the data analysis side, something specific I’ve noticed is an (understandable) bias towards computing numerical statistics, which they do very well and reading the post-analysis report has significantly improved my own “statistical thinking” approach overall. Numerical statistics are cool and understandably what a text-based LLM is going to want to work with, but asking the model to produce time-domain and frequency-domain plots of, say, specific events has multiple times resulted in “trying to plot this out has shown the opposite of what I concluded numerically… recalculating…” There’s still a pretty significant review and critically assess step for me, especially since the actions I take as a result of the analysis are pretty expensive, especially if they steer the next data collection run in a useless or harmful direction.
Re: I gave Qwen 3.8 27B a reverse-engineering job and it finished in 30 minutes
#48> The first attempt at recovering the key was wrong in a very specific way; it produced a working key and the signature check passed, but a hash the binary computes as an integrity check didn't match. In my experience, most models would have called it done and left it at that, but Qwen 3.8 27B didn't do that. Instead, it highlighted the mismatch, went back to the drawing board, and kept going until the value matched…
I believe this is part of the complaints of new models taking longer/requiring higher spend - they go the extra mile on verification, regardless of whether their change is correct already or not. So on problems that an earlier model one-shotted an answer to and did some lighter verification, the newer models might take longer to come back to the user due to running all the tests for your software they could find.
I noticed that too. I'm thinking about adding a prompt to disable those tests. We have the unit and feature tests anyway: add to them. I'm OK with the syntax checks: I work with interpreted languages, Ruby, JS, Python.
Re: I gave Qwen 3.8 27B a reverse-engineering job and it finished in 30 minutes
#49> I gave it the hardest real task that fits on one machine: reverse-engineering a commercial app's license check...
Respectfully, tasks that allow for explicit straightforward true/false or done/not-done tests are not the "hardest real task[s]." In fact, those are the ones that see the most gains from AI-assisted coding.
Testable tasks are where the largest opportunity is.
Re: I gave Qwen 3.8 27B a reverse-engineering job and it finished in 30 minutes
#50Earlier quoted context omitted.
Given the context, your domain name is 'chefs kiss', perfect. I am completely curious what your legal defense would be though. "Come do things with AI that are probably illegal!" What?! We had no idea people would do things that are illegal!
I guess we'll burn that bridge when we get to it!
More seriously though, I think we should be fine: we don't host any content, and what people do with the models is their own responsibility (legally speaking, in our jurisdiction, at least according to Claude -- we're talking to a real lawyer next week). Like any other provider, we offer no guarantees of sane, safe, or accurate results.