Live data from Hacker News

SceneScript, a novel approach for 3D scene reconstruction

ai.meta.com

11–20 of 32 posts

Re: SceneScript, a novel approach for 3D scene reconstruction

#11
Want more data? Create an mobile app to cartograph new rental appartments. I know in other countries you often rent with furniture in it. We in germany for example take everything with us and have to check if/how the furniture fits.

With a system like the one in the video you walk through the old place, and walk through the new place, and output for example 2 Sweet Home 3d plan's.

No need to have 3d glasses for that.

Re: SceneScript, a novel approach for 3D scene reconstruction

#12
post #10
post #9

Earlier quoted context omitted.

They still have to solve for: * How uncomfortable the headsets are to wear for long periods if time * How useless they are (what are they solution for?) * How ridiculous they look * How expensive they are * How expensive it is to produce content for After they solve these issues then I will worry about the dystopia we are in. VR headsets may look cool and futuristic in the movies but in reality they are just screens…

All of these (except maybe the "useless" part) are almost always solved by time (Moore's law)

I'd think that all of them would be solved by time. I'm just saying they are nothing to be concerned with until they are (and we are still far way from that point in time). They are just fancy demos to make a company look attractive to investors.

Re: SceneScript, a novel approach for 3D scene reconstruction

#13
This was a very important missing piece for mixed/augmented and virtual reality.

The semantic understanding of the objects around you (without upload to a server).

A detailed set of actual meshes of the objects in your living and working spaces.

So much can be built on this key building block, to actually help people with their daily lives, low vision people are only one group that will see massive benefits.

I also think this technique could and should be extended generally into computer vision applications, understanding semantic meaning, volumes and masses via the often highly structured environments around us.

Starting from a principled position of meaning, volumes and meshes and then training to assign gaussian splats to those meshes seems like something clever people could solve a lot of problems with.

Re: SceneScript, a novel approach for 3D scene reconstruction

#14
My issue with this approach is that the synthetic data, and the language used to describe it, necessarily encode a viewpoint - a bias - on the part of the creators. I am firmly behind the need to use language as an intermediary signifier, not just for utility purposes, but because it unlocks the ability to use language as an anchor for arbitrary AR data. My problem with this particular approach is that it requires us all to agree on what things are called, and requires the training dataset to cover everything users might come across.

My feeling is the better approach is to train a generalized segmentation model, derive multi-parameter representations of those segmented ‘things’, derive embeddings from those things for compression purposes, then allow users to ‘tag’ or ‘label’ or ‘describe’ those things as a service to other users. Then we could choose whose language about the world we ‘subscribe’ to, and anyone could attach any digital payload to any real thing/person/place without anyone else’s help or permission.

Re: SceneScript, a novel approach for 3D scene reconstruction

#15
post #3

I'm sure the tech is neat. But I get a real sinking feeling from how hard tech giants like "Meta" are trying openly to create dystopia. The "Metaverse" is not aspirational! > That understanding would let AR glasses tailor content to you and your individual context, like seamlessly blending a digital overlay with your physical space Horrifying, if it ever takes off in the way that "Meta" wants it to. Take the modern s…

> The Metaverse is not aspirational!

Wait, Snowcrash was horror for you? I thought the whole point of post-cyberpunk was to not assume all tech would be dystopian.

Re: SceneScript, a novel approach for 3D scene reconstruction

#16
post #3

I'm sure the tech is neat. But I get a real sinking feeling from how hard tech giants like "Meta" are trying openly to create dystopia. The "Metaverse" is not aspirational! > That understanding would let AR glasses tailor content to you and your individual context, like seamlessly blending a digital overlay with your physical space Horrifying, if it ever takes off in the way that "Meta" wants it to. Take the modern s…

[deleted]

Re: SceneScript, a novel approach for 3D scene reconstruction

#17
post #3

I'm sure the tech is neat. But I get a real sinking feeling from how hard tech giants like "Meta" are trying openly to create dystopia. The "Metaverse" is not aspirational! > That understanding would let AR glasses tailor content to you and your individual context, like seamlessly blending a digital overlay with your physical space Horrifying, if it ever takes off in the way that "Meta" wants it to. Take the modern s…

> The Metaverse is not aspirational! Wait, Snowcrash was horror for you? I thought the whole point of post-cyberpunk was to not assume all tech would be dystopian.

Snowcrash is a dystopian libertarian world cleverly hidden by the viewpoint characters treating it as completely normal.

Re: SceneScript, a novel approach for 3D scene reconstruction

#18

This was a very important missing piece for mixed/augmented and virtual reality. The semantic understanding of the objects around you (without upload to a server). A detailed set of actual meshes of the objects in your living and working spaces. So much can be built on this key building block, to actually help people with their daily lives, low vision people are only one group that will see massive benefits. I also t…

A huge problem is privacy

> without upload to a server

Doesn't matter. If the app has to render them then it can still send them to the server. If the app does not render directly it can still infer them by doing collision testing against them. If it can't even check for collisions then there isn't much point in having the data in the first place.

It gets worse. The more you want AR/MR to understand the scene the more fine grained data you need. The camera sees things like: the underwear you left on the floor, the medicines/pills on your table, the posters on your wall, the bank account browser tab still on your computer screen, the adult toys you forgot to put away, your undressed body, your dressed body, your political baseball cap or t-shirt, your naked roommate/spouse/kids who just walked into the room.

I don't see how this can be solved in a trust-less way. In other words, you'll have to trust each and every individual AR/MR app you use, that they aren't spying on you. The platform/device/OS itself can not prevent this and still allow the majority of the AR/MR uses that people imagine.

Re: SceneScript, a novel approach for 3D scene reconstruction

#20
post #18

This was a very important missing piece for mixed/augmented and virtual reality. The semantic understanding of the objects around you (without upload to a server). A detailed set of actual meshes of the objects in your living and working spaces. So much can be built on this key building block, to actually help people with their daily lives, low vision people are only one group that will see massive benefits. I also t…

A huge problem is privacy > without upload to a server Doesn't matter. If the app has to render them then it can still send them to the server. If the app does not render directly it can still infer them by doing collision testing against them. If it can't even check for collisions then there isn't much point in having the data in the first place. It gets worse. The more you want AR/MR to understand the scene the mor…

Companies don't spy on you because they don't like you, they spy on you because they make money out of it. Outlaw making money out of personal data (by outlawing targeted ads, data purchase for insurance, and data brokers) and the incentives to spy on you disappear.

You can't be probusiness and pro-privacy at the same time though, you need to limit businesses in order to protect the people.

Post reply on HN