Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Glue – Pandas as a DAG (Now a web app)

gluedata.io

Post reply on HN
Print view View original

11–20 of 30 posts

Re: Show HN: Glue – Pandas as a DAG (Now a web app)

#11

Post by 0xbadcafebee »

gthompson1 wrote: post #7
dsmmcken wrote: post #5

What is the expected inputs and outputs? How do you see this fitting into a users workflow?

I'm not totally sure what the question is but currently these are the inputs: delimited, excel, json, feather, parquet, sqlite, postgresql, mysql, googleSheets Outputs are the same (excluding databases currently). This is probably the most time consuming piece of the project if I was to expand this set dramatically given the quantity of possible connections, although here is where open source could help. It's hard to…

Target data scientists and you'll have a very large segment of users. They aren't programmers but they deal with data a lot.

Re: Show HN: Glue – Pandas as a DAG (Now a web app)

#12

Post by gthompson1 »

remram wrote: post #10

This site makes my phone slow down to a crawl, to the point that my music app stopped. And it's just the landing page. This is insane. What is the resolution on those auto-playing videos?

Way too high. I haven't updated the landing page in a while. I need to. Sorry.

Re: Show HN: Glue – Pandas as a DAG (Now a web app)

#13

Post by gthompson1 »

hokkos wrote: post #9

I guess it targets non programmer users and also without math knowledge, so why use lambda and sigma symbols ?

If you have other suggestions for the tool symbols let me know. With this project the right balance between technical and non technical is hard to hit.

Re: Show HN: Glue – Pandas as a DAG (Now a web app)

#14

Post by mousetree »

0xbadcafebee wrote: post #11
gthompson1 wrote: post #7

Earlier quoted context omitted.

I'm not totally sure what the question is but currently these are the inputs: delimited, excel, json, feather, parquet, sqlite, postgresql, mysql, googleSheets Outputs are the same (excluding databases currently). This is probably the most time consuming piece of the project if I was to expand this set dramatically given the quantity of possible connections, although here is where open source could help. It's hard to…

Target data scientists and you'll have a very large segment of users. They aren't programmers but they deal with data a lot.

I think that is OP's intent. Data Scientists, Data Engineers, Data Analysts, Data X

Re: Show HN: Glue – Pandas as a DAG (Now a web app)

#15

Post by programmarchy »

gthompson1 wrote: post #7
dsmmcken wrote: post #5

What is the expected inputs and outputs? How do you see this fitting into a users workflow?

I'm not totally sure what the question is but currently these are the inputs: delimited, excel, json, feather, parquet, sqlite, postgresql, mysql, googleSheets Outputs are the same (excluding databases currently). This is probably the most time consuming piece of the project if I was to expand this set dramatically given the quantity of possible connections, although here is where open source could help. It's hard to…

Would you be able to leverage singer taps / targets?

https://www.singer.io

Re: Show HN: Glue – Pandas as a DAG (Now a web app)

#16

Post by gthompson1 »

mousetree wrote: post #14
0xbadcafebee wrote: post #11

Earlier quoted context omitted.

Target data scientists and you'll have a very large segment of users. They aren't programmers but they deal with data a lot.

I think that is OP's intent. Data Scientists, Data Engineers, Data Analysts, Data X

Yeh I still appreciate the callout / opinion though. It's not obvious to me who is best to target. As a full stack engineer with analytics experience one of the main uses of the tool is scheduling, running pipelines remotely vs the UI for data munging which I can write pretty easily. But for non technical users the data munging piece might be really helpful (e.g. no code). Its a little hard to serve both a marketer and a developer given the needs are so different. But then on the other hand I assume a marketer is more likely to pay for the tool (and thus keep it going) vs a dev who might be a harder sell. I built this tool with the hope of being able to serve both which is where maybe I bit off more than I could chew.

Re: Show HN: Glue – Pandas as a DAG (Now a web app)

#17

Post by gthompson1 »

programmarchy wrote: post #15
gthompson1 wrote: post #7

Earlier quoted context omitted.

I'm not totally sure what the question is but currently these are the inputs: delimited, excel, json, feather, parquet, sqlite, postgresql, mysql, googleSheets Outputs are the same (excluding databases currently). This is probably the most time consuming piece of the project if I was to expand this set dramatically given the quantity of possible connections, although here is where open source could help. It's hard to…

Would you be able to leverage singer taps / targets? https://www.singer.io

Oh this is very interesting, I had never heard of this. Yeh I could potentially use this. I do think using a third party library or provider makes sense to massively expand the input / output options. Seems like there is a standard schema for the different tap configs that I could pull and wrap in UI forms in a generalized fashion.

I don't know if you think these are fair comment about the state of the project? - https://www.youtube.com/watch?v=TBrSOPNEg-g&ab_channel=Resta...

Re: Show HN: Glue – Pandas as a DAG (Now a web app)

#18

Post by gthompson1 »

0xbadcafebee wrote: post #8
gthompson1 wrote: post #2

Hey guys, I posted this maybe a year ago. It was originally an electron based desktop app which was cool but pretty hard to maintain / get people to download and test it out. Given that there would likely always need to be a cloud component regardless I decided that it would be easier to port it over into a web app and maintain it that way. Since doing that I took a job and am pretty burned out on the project. I was…

If you think you were burned out before, becoming an open source maintainer probably won't help :) If I were you I would put out a call for maintainers, and maybe consider a license that keeps modifications open source. That way a company that modifies it will have to contribute back to the project. If you do open source, I recommend moving your roadmap to a GitHub Project, put together a contributors guide/agreement…

"If you think you were burned out before, becoming an open source maintainer probably won't help" - Ha yeh I believe that.

"That way a company that modifies it will have to contribute back to the project. If you do open source, I recommend moving your roadmap to a GitHub Project, put together a contributors guide/agreement." - Thanks for the advice

Re: Show HN: Glue – Pandas as a DAG (Now a web app)

#19

Post by austinjp »

gthompson1 wrote: post #2

Hey guys, I posted this maybe a year ago. It was originally an electron based desktop app which was cool but pretty hard to maintain / get people to download and test it out. Given that there would likely always need to be a cloud component regardless I decided that it would be easier to port it over into a web app and maintain it that way. Since doing that I took a job and am pretty burned out on the project. I was…

Just wanted to say: nice job! :)

I can fully understand you burning out on something like this. How about declaring a time-out period where development is paused while you take a break? Maybe triage bug reports and feature requests in the meantime, but take some time to yourself to recover and reset your perspectives.

Just a thought.

Re: Show HN: Glue – Pandas as a DAG (Now a web app)

#20

Post by 0xbadcafebee »

gthompson1 wrote: post #16
mousetree wrote: post #14

Earlier quoted context omitted.

I think that is OP's intent. Data Scientists, Data Engineers, Data Analysts, Data X

Yeh I still appreciate the callout / opinion though. It's not obvious to me who is best to target. As a full stack engineer with analytics experience one of the main uses of the tool is scheduling, running pipelines remotely vs the UI for data munging which I can write pretty easily. But for non technical users the data munging piece might be really helpful (e.g. no code). Its a little hard to serve both a marketer a…

I'd say you should still work toward whatever your own interest with the tool is. If it seems like too much work, make a list of all the functionality you want, figure out the easiest way to implement each, and work on the least-effort/highest-reward things first. And do whatever feels fun!
Post reply on HN