Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Quilt (YC W16) – A versioned data portal for S3

news.ycombinator.com

Post reply on HN
Print view View original

61–65 of 65 posts

Re: Launch HN: Quilt (YC W16) – A versioned data portal for S3

#61

Post by akarve »

codetrotter wrote: post #52

> Try searching for anything on https://open.quiltdata.com/ and let us know how search works for you. I suggest adding the possibility of searching for exact matches with quotation marks, and also to ensure that it works with the quotation marks that the default keyboard on iOS has. For example, I want to search for “Irish Setter” and only see results that include those two words next to each other like that.

Here is an example of the "Irish Setter" query: https://open.quiltdata.com/search?q=irish%20%2B%20setter

You would type "irish + setter" in the search box.

Re: Launch HN: Quilt (YC W16) – A versioned data portal for S3

#62

Post by raghava »

akarve wrote: post #14
lichtenberger wrote: post #12

So you basically store S3 Buckets in Elastic Search and you're using Git for versioning a hierarchy of buckets, right? It's interesting that versioning now finally seems to be getting some traction in mainstream database systems (even though they are not really optimal in these systems my opinion) and for instance also in your data store. You position this as a Dropbox or Google Drive replacement, right? :-) I'm aski…

Not quite ;) S3 is the primary data and metadata store, so that the rest of the stack is a pure function of S3 data (including Elastic). We don't use git at all yet. We use S3 object versioning and then capture the version, SHA-256, etag, etc. in a JSONL-based manifest https://open.quiltdata.com/b/quilt-example/tree/.quilt/packa... . Said JSONL manifest is simply a "locked list" of all the S3 objects in that package.…

Neat. But would this not build dependency on s3s versioning and make it hard for getting this portable across other clouds?

Re: Launch HN: Quilt (YC W16) – A versioned data portal for S3

#63

Post by akarve »

raghava wrote: post #62
akarve wrote: post #14

Earlier quoted context omitted.

Not quite ;) S3 is the primary data and metadata store, so that the rest of the stack is a pure function of S3 data (including Elastic). We don't use git at all yet. We use S3 object versioning and then capture the version, SHA-256, etag, etc. in a JSONL-based manifest https://open.quiltdata.com/b/quilt-example/tree/.quilt/packa... . Said JSONL manifest is simply a "locked list" of all the S3 objects in that package.…

Neat. But would this not build dependency on s3s versioning and make it hard for getting this portable across other clouds?

Not quite. Abstraction layers like min.io support versioning. More importantly, Quilt manifests only require a "fully qualified physical key" that points to the data. In theory, the manifest can work with any URI: S3, local disk, etc.

Re: Launch HN: Quilt (YC W16) – A versioned data portal for S3

#64

Post by codetrotter »

akarve wrote: post #61
codetrotter wrote: post #52

> Try searching for anything on https://open.quiltdata.com/ and let us know how search works for you. I suggest adding the possibility of searching for exact matches with quotation marks, and also to ensure that it works with the quotation marks that the default keyboard on iOS has. For example, I want to search for “Irish Setter” and only see results that include those two words next to each other like that.

Here is an example of the "Irish Setter" query: https://open.quiltdata.com/search?q=irish%20%2B%20setter You would type "irish + setter" in the search box.

Thank you :)
Post reply on HN