Forgive me for stating the obvious, but if a problem gets reported more than once, you don't close ALL reports as duplicates and ignore the problem. Whether the original security breach was the user's fault or not, Amazon Support dropped the ball here.
Hacker Accessed AWS for $50k+ – AWS Ignoring Me
31–40 of 55 posts
Re: Hacker Accessed AWS for $50k+ – AWS Ignoring Me
#32Earlier quoted context omitted.
AWS support completely sucks unless you are paying 10K+ per month for the enterprise support tier. My average ticket response with paid AWS Developer is probably 72+ hours, and that's just for the initial triage what's up query. Unless you are either a seasoned Linux developer with many years of Linux internals experience, or you have an enterprise level account, all of the lower class AWS support rungs are largely m…
Support being slow and support being bad are two different things. Billing team is separate anyway. You get a better "treatment" if you're paying (as in, they will look at your case closer), but you have access to the same service regardless.
Re: Hacker Accessed AWS for $50k+ – AWS Ignoring Me
#33You are aws user for 3 years but don’t have aws rep? In my case we’ve got a miner on our jenkin for a day. I just call my aws sales rep and he get me a free lunch and a few credit to pay the business support for 1 month, then open the ticket through that business support. At the end of the week aws gave us extra credits around 10% of our yearly usage. I don’t think they will waived all yours 26k. Thats your dev team…
As a busy founder, I'll just go with the providers that don't chain me to their dashboard/email instead of providing meaningful caps.
Re: Hacker Accessed AWS for $50k+ – AWS Ignoring Me
#34i had set up some billing alerts on my AWS and it was just all terrible and sh*tty, clicking around in a million places to get not what i actually wanted.
i thought about building my own service to just let me know a daily total of my expected bill at the end of the month
then i'd add stuff to tell me about fast-increasing charges, as quickly as was necessary depending on the steepness of the charge rise curve.
then i found Billgist, which i tried for a bit -- it worked great, looks great, etc., so i'm not building my own. no connection to them.
i used it at first just to see if it worked at all, then to see if it sucked (in which case I would prob try to build my own), and then ultimately to try to help me get comfortable with the idea that i probably wasn't going to wake up one beautiful Saturday morning to a $50k AWS bill.
that never worked -- that is, i never got comfortable with the idea that I would _not_ wake up to a $50k AWS bill one beautiful Saturday morning -- it just seems completely plausible, even likely.
so i shut down most of my aws stuff (i was always particularly worried about my Lambda stuff), moved some things to Digital Ocean, and i'm guessing i'll revisit AWS at some point when i reach some critical mass of:
* "i actually need AWS", and
* "i actually have something to implement that has the possibility of making money", and
* "i'm comfortable, thru my own alerts/billing limits/cutoffs/aws-expertise, that i probably won't wake up to that $50k hacker AWS bill".
one thing i learned is that AWS charges you for _everything_ -- including your single daily API call to figure out how much you're going to owe at the end of the month -- the fee for that call is 3 cents per call, or at least for the first call.tho you can log in thru the console and check this estimate for free.
one of the things you get charged for is 'Configs' -- pretty much any config setting you've customized in any way -- permissions, roles, tags (?), etc.
i understand the logic, but damn -- i'm trying to use AWS so i can get things done, not so i can worry about the costs of every. single. not-completely-optimized. miniscule. design decision. down to the penny. nickle and diming would be a luxury.
i can imagine my hypothetical company's AWS Cost Saving Specialist coming to me and saying, "I'm glad you've set up this incredibly fast and secure and resilient system, but....we need to save a few bucks, so....yeah, i'm gonna need you to come in tomorrow...."
i may have a former co-worker that works there - if you run out of options i'll try to ping someone in my chain, see if i can contact them.
Re: Hacker Accessed AWS for $50k+ – AWS Ignoring Me
#35Earlier quoted context omitted.
I mean, it’s hard to have sympathy. Your setup sounds like a complete mess especially if you didn’t notice it. You didn’t even have MFA and you’re using the console to create resources? It’s not on AWS to flag anything. They give you the tools to comprehensively monitor your account, but you choose not to use them. Cloudwatch is also “standard”. Datadog has a free tier, I’d suggest checking that out because you don’t…
Nonsense, AWS is an industry leader in use of the asterisk. With exception to perhaps S3 and AWS Lambda, the predominant majority of AWS services are cloaked with accounting legalese and hidden billing gotchas that can easily result in a several thousand dollar bill within days if you don't analyze every aspect of AWS service offerings including their EULAs and acceptable use policies. So yes, it was bad form for OP…
AWS can certainly do a better job in telling me how to optimise my AWS hardware and expenses
In OPs case, if you search in Google, you will find this AWS hacking happen a lot with many users including me and AWS support was extremely kind to me to give additional credits to offset that expense.
Re: Hacker Accessed AWS for $50k+ – AWS Ignoring Me
#36Did you set up billing alerts? If you had an alert at say, $1500/month, you would've noticed almost immediately.
Re: Hacker Accessed AWS for $50k+ – AWS Ignoring Me
#37Have you contacted the FBI and Europol? A police report is the first thing you need before any company starts taking you seriously about crime being committed on your billing accounts.
https://www.fbi.gov/investigate/cyber
https://www.europol.europa.eu/report-a-crime/report-cybercri...
Re: Hacker Accessed AWS for $50k+ – AWS Ignoring Me
#38Re: Hacker Accessed AWS for $50k+ – AWS Ignoring Me
#39Re: Hacker Accessed AWS for $50k+ – AWS Ignoring Me
#40Earlier quoted context omitted.
I don't think i'm leaving anything out. It was my account (which now has had password changes and MFA set up), but I don't understand how there weren't red flags on the Austrian IP address login and the sudden spike in usage. I realize (now) that CloudWatch exists, but not sure why this isn't standard. I was at fault for the double post of the support case, but that was a simple error on my part due to not thinking t…
AWS is like a weapons cache you've stumbled upon in the middle of the desert, lots of fun, useful and interesting stuff in it but you're going to get yourself hurt if you don't take proper precautions. This sounds like a cautionary tale. I have spending alarms on my personal account for this very reason, I'll know within 5-10 minutes if my monthly spend is going to break $50 because I've set up my alarms. Your other…