Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Autopilot on Cars for $999

comma.ai

Post reply on HN
Print view View original

271–280 of 298 posts

Re: Autopilot on Cars for $999

#271

Post by bri3d »

throwaway0a5e wrote: post #238
kevin_thibedeau wrote: post #178

Earlier quoted context omitted.

Seems like it could have disastrous results if the OEM components start talking on the bus, sharing a completely different state.

Canbus is designed to be an unreliable electronic shouting match. The modules on it expect it and can handle it.

CANbus relies on message identifiers to prioritize communication. Spamming messages of equal arbitration ID at the same time, as is possible in an override system without careful design consideration, produces undefined behavior, as both messages can win the arbitration process.

Furthermore, corrupting command streams by "racing" valid messages which are sequence-checked is expecting a fault state to produce valid "normal" behavior. That's a lot of trust in OEM control modules - good thing they're usually written with safety in mind!

Re: Autopilot on Cars for $999

#273

Post by Torq_boi »

bri3d wrote: post #265
Torq_boi wrote: post #260

Earlier quoted context omitted.

When you say not everything is open source. I assume you mean the training code is not open source? I'm curious what you would want to learn from that? You wouldn't be able to actually train a model since you wouldn't have access to the data. The end-user can inspect, audit and understand the decisions their vehicle is making. All you have to do is see how the neural network behaves for different inputs. That's the c…

Comma don't even say _how_ the model works! What layers are there? What learning strategies are they using? What do they do? It's literally a black box! "All you have to do is see how it behaves for different inputs" is just black box reverse engineering! Machine Learning is NOT a magic black box. Comma have constructed a "stack" of models, just as you would connect a series of functions to make a kernel in the mathe…

The models are now saved in ONNX format. Which is the most readable format available. You can view the architecture of the model with a basic neural network viewer.

Again, I'm curious what you want to learn from the training code?

Chengyao's medium post is great, but it is only possible because the models, the code that runs them and the code that parses the outputs is fully open source.

Re: Autopilot on Cars for $999

#274

Post by bri3d »

Torq_boi wrote: post #273
bri3d wrote: post #265

Earlier quoted context omitted.

Comma don't even say _how_ the model works! What layers are there? What learning strategies are they using? What do they do? It's literally a black box! "All you have to do is see how it behaves for different inputs" is just black box reverse engineering! Machine Learning is NOT a magic black box. Comma have constructed a "stack" of models, just as you would connect a series of functions to make a kernel in the mathe…

The models are now saved in ONNX format. Which is the most readable format available. You can view the architecture of the model with a basic neural network viewer. Again, I'm curious what you want to learn from the training code? Chengyao's medium post is great, but it is only possible because the models, the code that runs them and the code that parses the outputs is fully open source.

My binary is saved in a PE format. Which is the most readable format available. You can view the architecture of the software by opening it in the basic Ghidra pseudocode decompiler. All Windows software is now "fully open source."

Chengyao's Medium post is advanced reverse-engineering work requiring a detailed knowledge of the appearance of specific ML algorithms saved in a binary format. And even with this knowledge, Chengyao was only able to _speculate_ about the behavior of the model and the desired response to certain inputs.

What would satisfy me from Comma, if they were aspiring to some kind of "open" label, would be a detailed document explaining each layer of the ML system and what its goals are - like Chengyao's Medium post, but without the need to reverse-engineer the system and attempt to infer its behavior!

Now, maybe Comma don't aspire to be truly open, in which case, that's fine - In that case, Comma is a closed model with an open-source CAN interceptor on top. So essentially, crowd-sourcing the tedious and high-liability parts (vehicle integration, driving video) while owning the valuable parts (training data and model architecture). Very cool!

Re: Autopilot on Cars for $999

#275

Post by gonehome »

buran77 wrote: post #252
gonehome wrote: post #143

Earlier quoted context omitted.

> "Tesla is a real example that passed this test. Their marketing language brands AP as "fully self driving, some features unavailable due to local laws"." This isn't true, FSD has always been a 'coming soon' feature you can prepay for distinct from autopilot. Autopilot has always been advanced lane assist. "Autopilot" in planes just holds the same flight pattern and doesn't really do anything sophisticated, autopilo…

German authorities did find Tesla's claims about the AP as misleading [0]. Tesla's AP page was updated (globally) to reflect "full self driving in the future" but until recently it just said "Tesla cars come equipped with all the necessary hardware for full self driving" with a footer claiming some features are unavailable due to local laws. Regardless, Tesla was not found guilty for anything other than misleading ad…

I don't agree, but I know this is a position a reasonable person could hold.

I just don't think autopilot means autonomous driving and I think that's clear to people.

If someone thinks cruise control means you can crawl into the backseat of the car, is that the fault of the car manufacturer?

I also think Germany may have a bias given their own car industry.

What comma.ai is doing I think is categorically different.

I'm okay with people hacking on their own stuff, I just don't like the "This is Alpha don't use it" when they obviously intend you to buy and use it that way.

Re: Autopilot on Cars for $999

#276

Post by DesolationJones »

bri3d wrote: post #183
stjo wrote: post #96

Earlier quoted context omitted.

Sell directly to manufacturers? Nearly no one (except Tesla) has anything coming in the near future. Why anyone they pay billions to develop the technology in-house if they can just pay Comma.ai to integrate a sub $1000 thingy in their cars.

I don't think Comma would pass compliance as any control-related automotive system, much less a self-driving one, at any established automotive manufacturer in a regulated market (US, EU). The code isn't written to any commercial audit standard that I can tell, the hardware is COTS mobile phone hardware, and change management and testing (in a formal sense) seems pretty much non-existent. Most manufacturers demand co…

This is what Geohot said in a reddit post.

"We have done most of the ISO26262 analysis, we're hiring someone right now to get it written up nicely and open sourced. (those interested can find the job posting) It's one of our goals for openpilot 1.0"

They have a slightly more in depth explanation of their safety model in the "Background — safety architecture" section of this post.

https://medium.com/@comma_ai/how-to-write-a-car-port-for-ope...

Re: Autopilot on Cars for $999

#277

Post by bri3d »

DesolationJones wrote: post #276
bri3d wrote: post #183

Earlier quoted context omitted.

I don't think Comma would pass compliance as any control-related automotive system, much less a self-driving one, at any established automotive manufacturer in a regulated market (US, EU). The code isn't written to any commercial audit standard that I can tell, the hardware is COTS mobile phone hardware, and change management and testing (in a formal sense) seems pretty much non-existent. Most manufacturers demand co…

This is what Geohot said in a reddit post. "We have done most of the ISO26262 analysis, we're hiring someone right now to get it written up nicely and open sourced. (those interested can find the job posting) It's one of our goals for openpilot 1.0" They have a slightly more in depth explanation of their safety model in the "Background — safety architecture" section of this post. https://medium.com/@comma_ai/how-to-w…

Thank you!

This is the first place I've seen this acknowledged - and, bizarrely, I read most of the PRs and commits to the `safety` part of Panda, and didn't find a single reference to the checklist in that Medium post (and, only some of the requirements seem to be implemented in most cars). It feels really late to me to be doing this, and it seems like they could use a good docs person and some, well, leadership in the project.

One thing I noticed in general was that it seems like most Comma communication is side-channeled - most commits and PRs do not have much of anything in terms of description or documentation, and code review is really sparse, it feels like there's a back-room discussion happening rather than GitHub, presumably on Discord? This makes it almost impossible to understand the safety constraints and reasoning, or to audit changes to the system.

But, it also sounds like they could very well be on the right track for 1.0, provided they hire the right person and they're able to clean things up.

Thank you for pointing this out, cheers!

Re: Autopilot on Cars for $999

#278

Post by rbrajdic »

coddle-hark wrote: post #122
delightful wrote: post #18

Is it correct Comma.AI sees this as the following statement below appears to say, or am I missing something? If so, why would anyone be using this product outside of a test environment that’s fully controlled? — “Any user of this software shall indemnify and hold harmless comma.ai, Inc. and its directors, officers, employees, agents, stockholders, affiliates, subcontractors and customers from and against all allegati…

The latest commit in the repo [0] right now is "should work" (34ff295). Filtering by "bug" in the issue tracker gives: Comma two freeze and reboot while engaged . I recently had an incident on the interstate where my comma two froze completely (while engaged) and rebooted. The video froze, Comma's steering torque turned off, then after about five seconds in this state, the device rebooted. Zygote restarting while OP…

The latest commit in the repo [0] right now is "should work" (34ff295)

That's a quick UI fix from looking at the code. Not sure how that makes the product less safe.

Re: Autopilot on Cars for $999

#279

Post by commaai »

3327 wrote: post #114
cbracketdash wrote: post #26

Earlier quoted context omitted.

It is still very early in its development stage so they do not want much liability.

I remember the founder interviewed me to be CEO, when he hit the investment and publicly insulted Papa Elon for kudos and bad assness. The guy was a jerk on the phone and 10 mins in I told Him to piss off and thought to myself “wow - who would work with this guy” he’s been at it since 2016 so glad it didn’t flop, but looks like he ate his words to Papa Musk. Nothin’ like some good old humble startup pie.

I suspect you are thinking of a different startup. We have never interviewed anyone to be CEO.

Re: Autopilot on Cars for $999

#280

Post by commaai »

bri3d wrote: post #274
Torq_boi wrote: post #273

Earlier quoted context omitted.

The models are now saved in ONNX format. Which is the most readable format available. You can view the architecture of the model with a basic neural network viewer. Again, I'm curious what you want to learn from the training code? Chengyao's medium post is great, but it is only possible because the models, the code that runs them and the code that parses the outputs is fully open source.

My binary is saved in a PE format. Which is the most readable format available. You can view the architecture of the software by opening it in the basic Ghidra pseudocode decompiler. All Windows software is now "fully open source." Chengyao's Medium post is advanced reverse-engineering work requiring a detailed knowledge of the appearance of specific ML algorithms saved in a binary format. And even with this knowledg…

What format would you rather the model be saved in? ONNX is the most cross platform and standard as far as I know, and it's also what we use internally.

It's not like a PE format which is compiled from something else higher level.

Post reply on HN