Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning from a Year of Security Breaches

medium.com

Post reply on HN
Print view View original

41–50 of 53 posts

Re: Learning from a Year of Security Breaches

#41

Post by BuuQu9hu »

haser_au wrote: post #31
mag00 wrote: post #16

Hi, I wrote this! To continue a discussion: - How does your engineering team track new "debt" after releasing code? (if at all, and why not) - Do you pay anyone for centralized logging, or wish you didn't? Are you making it useful? - Do you feel like your company is good at managing access when hiring / firing people? Otherwise thanks for any feedback, I enjoy writing these!

Can only speak about my corner of a very large organisation; - Technical debt of custom coded solutions is a known issue across our organisation. New strategy is to move to market solutions, therefore outsourcing the risk to organisations with (hopefully) better code management than we have. For my corner, we don't have technical debt measured accurately enough for my liking. - Yes, we pay for an use centralised logg…

> (hopefully)

I hope you are auditing the code of those external orgs.

Re: Learning from a Year of Security Breaches

#42

Post by haser_au »

BuuQu9hu wrote: post #41
haser_au wrote: post #31

Earlier quoted context omitted.

Can only speak about my corner of a very large organisation; - Technical debt of custom coded solutions is a known issue across our organisation. New strategy is to move to market solutions, therefore outsourcing the risk to organisations with (hopefully) better code management than we have. For my corner, we don't have technical debt measured accurately enough for my liking. - Yes, we pay for an use centralised logg…

> (hopefully) I hope you are auditing the code of those external orgs.

When you use SaaS products, auditing the code is not a service they offer. You have to rely on certifications from independent certifying organisations, etc.

Re: Learning from a Year of Security Breaches

#43

Post by haser_au »

tptacek wrote: post #32
haser_au wrote: post #31

Earlier quoted context omitted.

Can only speak about my corner of a very large organisation; - Technical debt of custom coded solutions is a known issue across our organisation. New strategy is to move to market solutions, therefore outsourcing the risk to organisations with (hopefully) better code management than we have. For my corner, we don't have technical debt measured accurately enough for my liking. - Yes, we pay for an use centralised logg…

Which logging systems did you like/not like?

Alientvault: Ok...we probably didn't get full potential here HP ArcSight: Extremely powerful, especially the normalizing of logs across similar system. Requires a team to manage though. Splunk: Our business isn't ready for cloud based hosting of centralised logs. Otherwise, we'd be on this already. From my perspective, purely from a reduction in complexity to pull useful information (not just Security).

Re: Learning from a Year of Security Breaches

#44

Post by emfree »

lvh wrote: post #26
tptacek wrote: post #25

Earlier quoted context omitted.

I've had good luck with Cloudwatch and, if you're on AWS, I'd recommend it over any other hosted log system (with the possible suggestion of a more elaborate ELK setup that you build yourself). The trick to Cloudwatch is --- like most AWS services --- never using the web UI.

That's a good point! If you have someone consuming it that wants a (shared) web UI, you want Kibana. If they prefer to consume their text in a terminal and are fine with typing `aws logs` a bunch, CloudWatch is fine (and probably a little less twiddly than ELK).

These are such great comments, thanks for sharing your insights. For folks looking for other options, I'd also mention https://honeycomb.io, perhaps the most promising newcomer in this space. It's essentially Facebook's Scuba for the rest of us.

Re: Learning from a Year of Security Breaches

#45

Post by jakozaur »

acidbaseextract wrote: post #10

Where do I start on centralized logging? I'm primarily an application developer, deployment isn't my strong suit. My hair is on fire at my current startup. There's a ton to do, we're trying to launch several new major efforts in January. What's a good plug and play solution that I don't have to think about? Are there hosted installs of Elasticsearch/Logstash/Kibana? Is ELK even what I want? Every time I start looking…

I would use some SaaS solution. ELK can be cheaper, but can take tons of time to configure and maintain if you run it. For early startup paying under $100 / month for some logging solution is no brainer vs. spending time configuring.

Disclaimer: I work at Sumo Logic I would recommend: https://www.sumologic.com On top of grep like searches, you can do analytical searches (SQL on text data).

Re: Learning from a Year of Security Breaches

#46

Post by darekkay »

acidbaseextract wrote: post #10

Where do I start on centralized logging? I'm primarily an application developer, deployment isn't my strong suit. My hair is on fire at my current startup. There's a ton to do, we're trying to launch several new major efforts in January. What's a good plug and play solution that I don't have to think about? Are there hosted installs of Elasticsearch/Logstash/Kibana? Is ELK even what I want? Every time I start looking…

Here are some SaaS choices:

* Sentry: https://sentry.io/welcome/

* Logentries: https://logentries.com/

* Loggly: https://www.loggly.com/

* Opbeat: https://opbeat.com/

* Papertrail: https://papertrailapp.com/

Sentry is open source and there is even an official up-to-date docker image: https://hub.docker.com/_/sentry/

Loggly published an "Ultimate Guide to Logging": https://www.loggly.com/ultimate-guide/

Re: Learning from a Year of Security Breaches

#47

Post by kevinr »

acidbaseextract wrote: post #10

Where do I start on centralized logging? I'm primarily an application developer, deployment isn't my strong suit. My hair is on fire at my current startup. There's a ton to do, we're trying to launch several new major efforts in January. What's a good plug and play solution that I don't have to think about? Are there hosted installs of Elasticsearch/Logstash/Kibana? Is ELK even what I want? Every time I start looking…

It depends a LOT on how many machines and services on the machines you're dealing with. There's a remarkable amount of stuff at the small end which is good, cheap, and fast to deploy.

I've been using Loggly for my personal machines (~8, mostly cloud VPSes). On the plus side, it's free at my scale, and the analysis and reporting tools are nice at least in theory. On the minus side, I can't get my logs past 7 days archived to S3 without paying $150/month, which I really want since my main use-case is longer-term analysis and forensics.

I'm planning to switch to Papertrail, which for the princely sum of $7/mo will give me a simpler UI and a year's archiving to S3.

Loggly and Papertrail both use the same deployment strategy (you hook them up to syslog and/or your app's logging package), and I had Loggly up and running and providing useful feedback in solidly under four hours.

Re: Learning from a Year of Security Breaches

#48

Post by kevinr »

rokosbasilisk wrote: post #4

Im surprised credential theft is still the lowest hanging fruit. I thought Banks seem to have solved alot of that.

Banks are in the business of managing financial risk for their customers, and they have enough money to eat a lot of risk before it becomes a problem for them. Other business models with less money in them do not have the same kind of resources.

Re: Learning from a Year of Security Breaches

#49

Post by hobofan »

kevinr wrote: post #47
acidbaseextract wrote: post #10

Where do I start on centralized logging? I'm primarily an application developer, deployment isn't my strong suit. My hair is on fire at my current startup. There's a ton to do, we're trying to launch several new major efforts in January. What's a good plug and play solution that I don't have to think about? Are there hosted installs of Elasticsearch/Logstash/Kibana? Is ELK even what I want? Every time I start looking…

It depends a LOT on how many machines and services on the machines you're dealing with. There's a remarkable amount of stuff at the small end which is good, cheap, and fast to deploy. I've been using Loggly for my personal machines (~8, mostly cloud VPSes). On the plus side, it's free at my scale, and the analysis and reporting tools are nice at least in theory. On the minus side, I can't get my logs past 7 days arch…

For small to medium log volume, I can only recommend Loggly.

The killer feature it has is for me is searching structured (JSON) logs. Just use the Logstash/Greylog library in the language of your choice and send the logs to Loggly, and you quickly have a logging system where you can zoom in on the logs comming from different subsystems of your codebase or produced by a specific user.

Re: Learning from a Year of Security Breaches

#50

Post by [deleted] »

kevinr wrote: post #47
acidbaseextract wrote: post #10

Where do I start on centralized logging? I'm primarily an application developer, deployment isn't my strong suit. My hair is on fire at my current startup. There's a ton to do, we're trying to launch several new major efforts in January. What's a good plug and play solution that I don't have to think about? Are there hosted installs of Elasticsearch/Logstash/Kibana? Is ELK even what I want? Every time I start looking…

It depends a LOT on how many machines and services on the machines you're dealing with. There's a remarkable amount of stuff at the small end which is good, cheap, and fast to deploy. I've been using Loggly for my personal machines (~8, mostly cloud VPSes). On the plus side, it's free at my scale, and the analysis and reporting tools are nice at least in theory. On the minus side, I can't get my logs past 7 days arch…

[deleted]
Post reply on HN