Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Matano (YC W23) – Open-Source Security Lake Platform (SIEM) for AWS

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 55 posts

Re: Launch HN: Matano (YC W23) – Open-Source Security Lake Platform (SIEM) for AWS

#11

Post by slt2021 »

0x4e53 wrote: post #7
debarshri wrote: post #5

I have been exploring this realm of SIEM, XDR, NDR etc. Sure, all proprietary SIEMs are expensive. But what is not clear is how you are going to price it. Security teams have dedicated budget. If you are coming cheaper than them, they you are destroying your TAM because I know customer would not mind paying those license fees. OSS GTM might work but might against your TAM.

At least from my time at SpaceX - this is untrue. SIEM costs were rapidly ballooning, and we were being charged by RAM. RAM?? Of all things!! After our SIEM costs for ELK ramped up to where Splunk was - we just bought Splunk instead. I imagine there are many security teams out there that would entertain a cheaper alternative that isn't priced by RAM.

the reason for that is near real-time detection of threats requires aggregation of terabytes of data according to rules (continuous GROUP BY on thousands columns on a sliding window) - and these aggregates by design have to be stored in RAM.

Otherwise these detections stop being near-realtime and become offline detection instead, just like any other sql server.

Re: Launch HN: Matano (YC W23) – Open-Source Security Lake Platform (SIEM) for AWS

#12

Post by protoduction »

Hi Shaeq and Samrose - congrats on the launch! Matano looks great.

Out of curiosity, at some point I believe you were working on a predecessor called AppTrail whic tackled (customer-facing) audit logs, it was something I was interested in at the time (and still am! I would've loved to use that).

Would you perhaps be willing to share your learnings from that product, and (I assume) why it evolved into Matano?

Re: Launch HN: Matano (YC W23) – Open-Source Security Lake Platform (SIEM) for AWS

#13

Post by waihtis »

I'm a vendor in the cyberspace so not a potential customer (feel free not to waste time answering) but am just intellectually curious who you're targeting this at. High-skill tech companies who are just building up a security program? I don't see most security teams building their own SIEM'ish solution just because they really don't have the chops or resource to do it. OTOH, it would be a big rip-out operation for F100 companies to change to this from Splunk et al.

Re: Launch HN: Matano (YC W23) – Open-Source Security Lake Platform (SIEM) for AWS

#14

Post by lmeyerov »

waihtis wrote: post #13

I'm a vendor in the cyberspace so not a potential customer (feel free not to waste time answering) but am just intellectually curious who you're targeting this at. High-skill tech companies who are just building up a security program? I don't see most security teams building their own SIEM'ish solution just because they really don't have the chops or resource to do it. OTOH, it would be a big rip-out operation for F1…

(not them, but in this space with major enterprises and gov agencies deploying Graphistry)

We are pretty active here with security cloud/on-prem data lakes teams as a way to augment their Splunk with something more affordable & responsive for bigger datasets. Imagine stuffing netflow or winlogs somewhere at TB/PB scale and not selling your first born child. A replacement/fresh story may happen at a young/midage tech co, and a bunch of startups pitching that. But for most co's, we see augmentation and still needing to support on-prem detection & response flows.

It's pretty commodity now to dump into say databricks, and we work with teams on our intelligence tier with GPU visual analytics, GPU AI, GPU graph correlation, etc. to make that usable. Most use us to make sense of regular alert data in Splunk/neo4j/etc. However, it's pretty exciting when we do something like looking at vpc flow logs from a cloud-native system like databricks and can thus look at more session context and run fusion AI jobs like generating correlation IDs for pivoting + visualizing.

Serverless is def interesting but I've only seen for light orchestration. Everyone big has on-prem footprint which is an extra bit of fun for the orchestration vs investigation side.

Re: Launch HN: Matano (YC W23) – Open-Source Security Lake Platform (SIEM) for AWS

#15

Post by sullivanmatt »

This issue exists to the right of your solution and is (for now) out of scope, but the biggest issue I have with security data lakes is the need to (easily) get both row-based data and visualizations. Back when I had access to a well-built and cared for Splunk environment, I would constantly run queries, build visualizations, go back to the results index, tweak the query, go back to viz, etc. This feedback loop is important and allows for fast iteration, especially if you are conducting a high-stakes investigation and need answers rapidly. I should be able to look at my available fields and tweak the viz accordingly in under a few seconds; preferably in one mouse click.

Now I live on an ELK stack and I experience nothing but full-time agony as I switch between Kibana and Kibana Lens constantly. It's clear they are two completely separate "products" built for different use-cases. The experience reminds you constantly that they were not purpose-built for how I use them, unlike Splunk.

Increasingly we are moving towards the reality of a security data lake, and all I can think is that I'm about to lose what little power I had left as I have to move to something like Mode, Sisense, or Tableau which again, were not purpose-built for these use-cases and even further separate the query/data discovery and visualization layers.

I hate how crufty and slow Splunk has gotten as an organization, and they use their accomplishments from 15 years ago to justify the exorbitant price they charge. I really hope the OSS/next-gen SaaS options can fill this need and security data lake becomes a reality. But for that to happen, more focus is needed on the user experience as well.

Regardless, very cool stuff and could definitely fill a need for organizations that are just starting to dip toes into security data lakes. I wish you success!

Re: Launch HN: Matano (YC W23) – Open-Source Security Lake Platform (SIEM) for AWS

#16

Post by shaeqahmed »

waihtis wrote: post #13

I'm a vendor in the cyberspace so not a potential customer (feel free not to waste time answering) but am just intellectually curious who you're targeting this at. High-skill tech companies who are just building up a security program? I don't see most security teams building their own SIEM'ish solution just because they really don't have the chops or resource to do it. OTOH, it would be a big rip-out operation for F1…

Many enterprises using Splunk are already being forced to purchase products like Cribl to route some of their data to a data lake because writing it all to Splunk is just way too expensive at that scale 1-100TB+/day (7 figures $).

But a data lake shouldn't just be a dump of data right? Matano OSS helps organizations build high value data lakes in S3 and reduce their dependency on SIEM by centralizing high throughput data in object storage using Matano to power investigations. To give you an example, one company is using Matano to collect, normalize, and store VPC Flow logs from hundreds of AWS accounts which was too expensive with traditional SIEM.

Matano is also completely serverless and automates the maintenance of all resources/tables using IaC so it's perfect for smaller security teams on the cloud dealing with a large amount of data and wanting to use a modern data stack to analyze it.

Re: Launch HN: Matano (YC W23) – Open-Source Security Lake Platform (SIEM) for AWS

#17

Post by waihtis »

lmeyerov wrote: post #14
waihtis wrote: post #13

I'm a vendor in the cyberspace so not a potential customer (feel free not to waste time answering) but am just intellectually curious who you're targeting this at. High-skill tech companies who are just building up a security program? I don't see most security teams building their own SIEM'ish solution just because they really don't have the chops or resource to do it. OTOH, it would be a big rip-out operation for F1…

(not them, but in this space with major enterprises and gov agencies deploying Graphistry) We are pretty active here with security cloud/on-prem data lakes teams as a way to augment their Splunk with something more affordable & responsive for bigger datasets. Imagine stuffing netflow or winlogs somewhere at TB/PB scale and not selling your first born child. A replacement/fresh story may happen at a young/midage tech…

Thanks, this is interesting. I work a bit closer to the "source" as in doing detection things on on-prem & cloud side, so not too well-versed on the data processing and management side.

Re: Launch HN: Matano (YC W23) – Open-Source Security Lake Platform (SIEM) for AWS

#18

Post by waihtis »

shaeqahmed wrote: post #16
waihtis wrote: post #13

I'm a vendor in the cyberspace so not a potential customer (feel free not to waste time answering) but am just intellectually curious who you're targeting this at. High-skill tech companies who are just building up a security program? I don't see most security teams building their own SIEM'ish solution just because they really don't have the chops or resource to do it. OTOH, it would be a big rip-out operation for F1…

Many enterprises using Splunk are already being forced to purchase products like Cribl to route some of their data to a data lake because writing it all to Splunk is just way too expensive at that scale 1-100TB+/day (7 figures $). But a data lake shouldn't just be a dump of data right? Matano OSS helps organizations build high value data lakes in S3 and reduce their dependency on SIEM by centralizing high throughput…

nice thanks, makes a lot of sense

Re: Launch HN: Matano (YC W23) – Open-Source Security Lake Platform (SIEM) for AWS

#19

Post by shaeqahmed »

sullivanmatt wrote: post #15

This issue exists to the right of your solution and is (for now) out of scope, but the biggest issue I have with security data lakes is the need to (easily) get both row-based data and visualizations. Back when I had access to a well-built and cared for Splunk environment, I would constantly run queries, build visualizations, go back to the results index, tweak the query, go back to viz, etc. This feedback loop is im…

I completely agree with you and the need for a fully integrated solution with great visualizations without hosting additional tools that aren't purpose built! Unfortunately there are very few SIEMs that get this right today..

Here's how we are thinking of it. We think it's important for a successful security program to first have high quality data and this is why we want help every organization build structured security data lakes to power their analysis using our open source project. The Matano security lake can sit alongside their SIEM and be incrementally adopted for a data sources that wouldn't be feasible to analyze otherwise.

Our larger goal as a company though is to build a complete platform that allows a security data lake to fully replace traditional SIEM -- including a UI and collaborative features that give you that great feedback loop for fast iteration in detection engineering and threat hunting as you mentioned. Stay tuned I think you will be excited by what we are building!

Re: Launch HN: Matano (YC W23) – Open-Source Security Lake Platform (SIEM) for AWS

#20

Post by alecbell »

This is awesome. Nice work open-sourcing it! I used Splunk at Expedia and it was super expensive and slow. While I wasn't using it for security purposes, it could take 15-30 min for us to detect error logs, and I can imagine that's not okay for security purposes. Good luck guys!
Post reply on HN