Mmm, a world where a defender-LLM is essentially required is great news for people selling inference.
We have a year to fix security everywhere
11–20 of 371 posts
Re: We have a year to fix security everywhere
#12> On September 22, Apple is releasing the M5 Mac Studio with 256 GB of unified memory [..] it will probably [..] enough to write this snippet of code in 3 seconds The author has obviously never ran an LLM on a mac! In 3 seconds, it will have possibly started to think about maybe scheduling a date to contemplate the planning timeline for processing the second token in your prompt.
The author put in the numbers, but maybe you didn’t read them.
45 t/s a second is perfectly respectable especially with no limits and 24/7 uptime with very little power draw on the Studio.
Luna is at around 100 t/s for comparison, but it’s a worse model than 5.3 Flash
Re: We have a year to fix security everywhere
#13It's just the same advice as ever: be extremely, exceedingly careful in what you expose to any network. When I set up machines for production, they don't respond to pings and they don't even have an SSH port open without knocking. There are also ways to eschew the need for an SSH port entirely.
People who never took that seriously will never take this seriously either, and that's their loss. (And loss of the commons, unfortunately.)
There's just also new advice: you can't afford to expose an unsecured system to the internet even for a moment. Think of those IPv4 address space scanners, except this time any one of them could be capable of developing individualized attacks in mere minutes. They don't sleep, they don't take breaks.
Re: We have a year to fix security everywhere
#14> On September 22, Apple is releasing the M5 Mac Studio with 256 GB of unified memory [..] it will probably [..] enough to write this snippet of code in 3 seconds The author has obviously never ran an LLM on a mac! In 3 seconds, it will have possibly started to think about maybe scheduling a date to contemplate the planning timeline for processing the second token in your prompt.
The difference is memory bandwidth. The M5 Ultra that's coming out on 22nd September can do 1,200GB/s. The M5 Max you can buy today only has 614GB/s.
So, that gets us to about where nVidia was with Ampere in 2020. Let's hope the M7 catches us up with at least Hopper.
Re: We have a year to fix security everywhere
#15[flagged]
Re: We have a year to fix security everywhere
#16Maybe all these vital infrastructure companies should not have spent the past decades in a race to the bottom of cybersecurity. There is going to be a reckoning.
Re: We have a year to fix security everywhere
#17Or we could just dump Linux and Windows and switch to a microkernel operating system, which is much more secure. These endless patching cycles are simply not going to work in the long run. Operating systems get orphaned all the time, especially the ones in cheap Chinese stuff.
Got any leads on good tutorials on how to use a microkernel operating system on a VPS somewhere to host a website?
Just checked with Google Gemini on how one might be able to do the above. It pointed to Minix3/seL4/Genode and vps providers who either support custom ISOs or run it within an emulator like QEMU.
You can also look at using Unikernels for this purpose. Here is an article Unleashing Extreme Speed and Security: Deploying Unikernels with NanoVMs on VPS to Eliminate the Linux OS - https://xylentis.com/blog/unleashing-extreme-speed-and-secur...
Re: We have a year to fix security everywhere
#18Here's an idea: as a first step, simplify everything, and make sure you're aware how your stack works, and what it imports. As an example: WordPress is a horrible thing, but the core has been through so much, that it's suprisingly secure. Then plugins and themes come, and whoosh, the security is gone. We need a new KISS: keep it simple, stupid, secure.
The "surprisingly secure" WordPress just had a unauthenticated RCE earlier this year. Just simplifying isn't going to be enough. https://nvd.nist.gov/vuln/detail/cve-2026-63030
Plus, how secure are the plugins?
Re: We have a year to fix security everywhere
#19> On September 22, Apple is releasing the M5 Mac Studio with 256 GB of unified memory [..] it will probably [..] enough to write this snippet of code in 3 seconds The author has obviously never ran an LLM on a mac! In 3 seconds, it will have possibly started to think about maybe scheduling a date to contemplate the planning timeline for processing the second token in your prompt.
The author put in the numbers, but maybe you didn’t read them. 45 t/s a second is perfectly respectable especially with no limits and 24/7 uptime with very little power draw on the Studio. Luna is at around 100 t/s for comparison, but it’s a worse model than 5.3 Flash
The joke is that macs are famously slow at prompt prefill and you are not getting anything back in 3 seconds, or probably even 30. Once they get generating, it can be acceptable, but the TTFT is horrendous.
There's a ton of well-understood things Apple can and hopefully will do to massively accelerate every stage of this pipeline and hopefully they're hard at work implementing most of them for m7.
Re: We have a year to fix security everywhere
#20Just like Cryptolocker, this will be the "Finding Out" phase for everyone who has been putting off best practice security.
But, lets be clear, Best Practice will save you. We can engineer assuming there are zero days in path. Go to your CTO now cap in hand and ask for overlapping controls, wafs, application monitoring, backups and all the other shit you haven't been doing.
Because when you find out, I will laugh, it will be very very very funny to me.