Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Berkeley Deep Drive Dataset

bdd-data.berkeley.edu

Post reply on HN
Print view View original

41–50 of 54 posts

Re: Berkeley Deep Drive Dataset

#41

Post by jamesblonde »

web007 wrote: post #36
nerdponx wrote: post #35

Earlier quoted context omitted.

How would one go about creating a torrent for it? Or uploading to IPFS?

I already emailed the creator a couple weeks back to request / offer a torrent, but haven't heard anything back. The problem here is that both of your suggestions involve a 2-step process: 1. Download the file 2. Create a torrent from it, or upload it to IPFS Since step 1 is already a 2TB download, getting to either version of step 2 is untenable. I agree with one of the other posters in this thread, the default for…

Subject to licensing, we intend to make the dataset available (along with loads of other big datasets for ML) using a bit-torrent like program called Dela for the Hops Hadoop platform. Maybe in 3 weeks or so, it will all be released - with this dataset. Dela integrates with HDFS/S3/GCS backends, and it supports NAT traversal, and a delay-based congestion control over UDP - good for high bandwidth/high latency networks. See http://www.hops.io and our paper - https://ieeexplore.ieee.org/document/7980225/

Re: Berkeley Deep Drive Dataset

#42

Post by cr4zy »

itchyjunk wrote: post #29

I am curious, does training the AI on other driving data sets help? What I means is, not just sedan data set. Trucks, buses, maybe 2 wheelers? Would this help the model generalize more and make better prediction of how other vehicles work or would it just add noise?

It helps a lot in my experience. So in simulation I tried this with imitation learning by training on a hood camera, a camera at the height of what a semi-truck hood would be, and another camera offset to the left 1.5m along with steering and throttle for labels. I also added random noise to the position (less than a meter), rotation (less than degree), fov (less than a degree), capture height (< 1%), and capture width (< 1%). The result was a 3x higher average score on a driving benchmark where the score was meters driven minus seconds taken, second-meters of lane deviation, and seconds where acceleration surpassed 0.5g forces (to measure comfort). The dataset, training code, and sim are at deepdrive.io - a different entity with the same name :)

Re: Berkeley Deep Drive Dataset

#43

Post by codetrotter »

mehrdadn wrote: post #25
ghaff wrote: post #22

Earlier quoted context omitted.

The GPL absolutely falls under the open source definition. You can ship GPL programs and code alongside programs that may have different licenses. What you can't do is combine GPL code with code that has incompatible licenses. Copyleft imposes some requirements on redistribution. It does not impose restrictions on usage at all.

> Copyleft imposes some requirements on redistribution. It does not impose restrictions on usage at all. I wasn't saying copyleft imposes restrictions on usage. The first "open-source" criterion says the following (and note that, like you said, this is a restriction on redistribution and not usage): > The license shall not restrict any party from selling or giving away the software as a component of an aggregate soft…

> namely: that it must be with source code

GPL does not say that. What it does say is that you must provide the source upon request.

> The fact that you cannot redistribute GPL software without source (whether bundled with other software or otherwise) is a restriction on whether/how you can redistribute GPL software, hence it goes against the "shall not restrict" requirement. And there's no exception carved out for "restrictions that require source code to be included".

So all of what you said there is simply incorrect, because like I said, you absolutely can distribute GPL software without including the source code alongside it. And that is what is done by everyone 99% of the time.

You only need to provide the source upon request to the people that ask you for it.

I encourage you to take the time to read the GPL FAQ. Even though GPL is not my preferred license I think it is important to have a good understanding of it. https://www.gnu.org/licenses/gpl-faq.en.html

Re: Berkeley Deep Drive Dataset

#44

Post by mehrdadn »

codetrotter wrote: post #43
mehrdadn wrote: post #25

Earlier quoted context omitted.

> Copyleft imposes some requirements on redistribution. It does not impose restrictions on usage at all. I wasn't saying copyleft imposes restrictions on usage. The first "open-source" criterion says the following (and note that, like you said, this is a restriction on redistribution and not usage): > The license shall not restrict any party from selling or giving away the software as a component of an aggregate soft…

> namely: that it must be with source code GPL does not say that. What it does say is that you must provide the source upon request. > The fact that you cannot redistribute GPL software without source (whether bundled with other software or otherwise) is a restriction on whether/how you can redistribute GPL software, hence it goes against the "shall not restrict" requirement. And there's no exception carved out for "…

>> namely: that it must be with source code

> GPL does not say that. What it does say is that you must provide the source upon request.

Yes, I was being brief. I'm well aware. [1]

[1] https://news.ycombinator.com/item?id=17202439

-------------

To your edit:

> So all of what you said there is simply incorrect, because like I said, you absolutely can distribute GPL software without including the source code alongside it. And that is what is done by everyone 99% of the time. You only need to provide the source upon request to the people that ask you for it.

No, it makes no difference at all. You cannot redistribute the software unless you are willing and able to redistribute the source code as well. That is very clearly a restriction on your redistribution of the software. The fact that we happen to be talking about the software's own source code makes no difference as to whether it's a restriction or not. It'd be a restriction whether we're talking about "source code", or "$100,000", or anything else. The simple fact that you have to be willing and able to provide {something} before you can redistribute the software is obviously a restriction on your redistribution of the software.

Re: Berkeley Deep Drive Dataset

#45

Post by singularity2001 »

koolba wrote: post #28
derefr wrote: post #20

Earlier quoted context omitted.

It's a really big download (1.8TB). One where both you and they would be really rather perturbed if the download failed at 90%. In fact, they'd probably be perturbed by the bandwidth costs even if everyone who wanted the dataset was only downloading it once. Maybe it uses WebTorrent? (Not sure why it couldn't just fall back to giving you a .torrent file in that case, though.)

Why even fall back? The default for something like this should be a torrent as this is exactly what BitTorrent was supposed to solve!

"only hackers use torrent"

Re: Berkeley Deep Drive Dataset

#46

Post by hprotagonist »

derefr wrote: post #20
MBCook wrote: post #11

Earlier quoted context omitted.

It’s in DOWNLOAD button. How does it not work in EVERY browser? A link would. A form submission would. Ultra simple JavaScript would. It’s not a question of “why doesn’t chrome work“ but more a question of “how is this even an issue“.

It's a really big download (1.8TB). One where both you and they would be really rather perturbed if the download failed at 90%. In fact, they'd probably be perturbed by the bandwidth costs even if everyone who wanted the dataset was only downloading it once. Maybe it uses WebTorrent? (Not sure why it couldn't just fall back to giving you a .torrent file in that case, though.)

>It's a really big download (1.8TB).

COCO (and friends) provide either cloud-backed rsync tools or curl snippets for this reason.

Re: Berkeley Deep Drive Dataset

#47

Post by syntaxing »

Before I register to download the data, is there a smaller dataset for you to play with on the portal? I've been itching to do something fun after taking my SDC from Udacity. Nut1.8TB is way more than I can handle right now. Can someone upload a portion of this? (<10GB)?
Post reply on HN