Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: A tool to seed your dev database with real data

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 79 posts

Re: Show HN: A tool to seed your dev database with real data

#11

Post by toomuchtodo »

osrec wrote: post #9
kodah wrote: post #8

Earlier quoted context omitted.

Credentials in a config file can be mistakenly checked into a repository. They're easy to exfiltrate from files, say I write a script with well known configuration locations for thousands of applications and just dumbly pull them all from a compromised system. I now have little bits of access to the wider system where I can now jump from system to system. The best way to store ephemeral secrets is in an environment v…

How do you automate setting that environment variable?

Injection into the environment (container config, Docker, k8s, etc) or the execution context. The actual mechanics are highly dependent on where your secrets are and where they’re going for use. On a dev workstation, could just be some bash or Python using the AWS cli, for example.

Re: Show HN: A tool to seed your dev database with real data

#12

Post by tehlike »

nicoburns wrote: post #10

This does sounds kind of useful. On the other hand I performed a similar task just yesterday using the native pg_dump and pg_restore commands, and it only took a couple of hours to setup (and now I have a repeatable script), so this’ll need to be implemented really well to provide value.

I am on the same boat but couple hours is terrible still. The best is probably copying the data directory straight which should cut it down to seconds, but i have yet to automate that + there are production credentials/sensitive data problems that needs to be tackled too...

Re: Show HN: A tool to seed your dev database with real data

#14

Post by dvt »

kodah wrote: post #8
herpderperator wrote: post #3

Earlier quoted context omitted.

Why?

Credentials in a config file can be mistakenly checked into a repository. They're easy to exfiltrate from files, say I write a script with well known configuration locations for thousands of applications and just dumbly pull them all from a compromised system. I now have little bits of access to the wider system where I can now jump from system to system. The best way to store ephemeral secrets is in an environment v…

I don't get this logic, that's what .gitignore is for. I've been using .env files for years and never mistakenly checked one into a repo.

Re: Show HN: A tool to seed your dev database with real data

#15

Post by atonse »

mdaniel wrote: post #2

Please don't require static AWS credentials: https://github.com/Qovery/replibyte/blob/v0.4.4/replibyte/sr... or at least either include "AWS_SESSION_TOKEN" in that setup (if it is present) in order to allow "aws sts assume-role" to work, or allow `AWS_PROFILE`, or just use the aws-sdk's normal credential discovery mechanism which at least on their "main" SDKs is a fallback list of them, but I couldn't follow the docs…

Should work out of the box if they’re using the rust AWS library right?

Re: Show HN: A tool to seed your dev database with real data

#16

Post by dboreham »

dvt wrote: post #14
kodah wrote: post #8

Earlier quoted context omitted.

Credentials in a config file can be mistakenly checked into a repository. They're easy to exfiltrate from files, say I write a script with well known configuration locations for thousands of applications and just dumbly pull them all from a compromised system. I now have little bits of access to the wider system where I can now jump from system to system. The best way to store ephemeral secrets is in an environment v…

I don't get this logic, that's what .gitignore is for. I've been using .env files for years and never mistakenly checked one into a repo.

Still a bad idea though.

Re: Show HN: A tool to seed your dev database with real data

#17

Post by kodah »

dvt wrote: post #14
kodah wrote: post #8

Earlier quoted context omitted.

Credentials in a config file can be mistakenly checked into a repository. They're easy to exfiltrate from files, say I write a script with well known configuration locations for thousands of applications and just dumbly pull them all from a compromised system. I now have little bits of access to the wider system where I can now jump from system to system. The best way to store ephemeral secrets is in an environment v…

I don't get this logic, that's what .gitignore is for. I've been using .env files for years and never mistakenly checked one into a repo.

idk what to tell you. I work on a security team, one of the tools the team built finds and identifies secrets already checked into VCS or ones at the pre-commit stage. It's certainly not a seldomly used tool.

Re: Show HN: A tool to seed your dev database with real data

#18

Post by dboreham »

osrec wrote: post #9
kodah wrote: post #8

Earlier quoted context omitted.

Credentials in a config file can be mistakenly checked into a repository. They're easy to exfiltrate from files, say I write a script with well known configuration locations for thousands of applications and just dumbly pull them all from a compromised system. I now have little bits of access to the wider system where I can now jump from system to system. The best way to store ephemeral secrets is in an environment v…

How do you automate setting that environment variable?

For a specific example of one way how : https://github.com/99designs/aws-vault

Re: Show HN: A tool to seed your dev database with real data

#19

Post by atonse »

fedeb95 wrote: post #13

interesting, however couldn't it detect tables and columns automatically instead of having to specify them in the configuration file? If I understand correctly each table is to be specified by hand. Say I have nearly a hundred tables...

Seems like you only specify transformers

Re: Show HN: A tool to seed your dev database with real data

#20

Post by mdaniel »

kodah wrote: post #17
dvt wrote: post #14

Earlier quoted context omitted.

I don't get this logic, that's what .gitignore is for. I've been using .env files for years and never mistakenly checked one into a repo.

idk what to tell you. I work on a security team, one of the tools the team built finds and identifies secrets already checked into VCS or ones at the pre-commit stage. It's certainly not a seldomly used tool.

I believe GP's comment is at the intersection of "the chain is only as strong as its weakest link" and "defenders have to be correct every time, attackers just once"
Post reply on HN