I love his hash table solution to level0! I would never have spent much time optimizing such a trivial challenge as level0, but after seeing the leaderboards with such ridiculous scores, curiosity got the best of me, and I had to figure out how people were achieving such speedups. I used a precomputed bloom filter to get 2641 points on level0, because I couldn't think of another way to avoid doing the expensive proce…
Stripe-CTF 3 Writeup
41–50 of 74 posts
Re: Stripe-CTF 3 Writeup
#42I think this particular bit says it all. To get blazingly fast results... > I can say with great confidence that we all cheated. But I disagree with the word used. Pre-computing the hash-table and embedding that into the final binary to eliminate relatively expensive calculations when it matters? That's not cheating, that's effing brilliant . Hard-core software engineering, that's what it is. There's also a saying th…
Re: Stripe-CTF 3 Writeup
#43Re: Stripe-CTF 3 Writeup
#44I read through this and was amazed by the OP's approach to the problems even though I could barely understand 10% of the reading. How do I learn to think and code like this? I honestly have no idea where to begin.
Re: Stripe-CTF 3 Writeup
#45I think this particular bit says it all. To get blazingly fast results... > I can say with great confidence that we all cheated. But I disagree with the word used. Pre-computing the hash-table and embedding that into the final binary to eliminate relatively expensive calculations when it matters? That's not cheating, that's effing brilliant . Hard-core software engineering, that's what it is. There's also a saying th…
Op here, just wanted to say thanks. Of course it is not cheating in the sense that I tricked the system to give me a better score than warranted but my solution only works for this ONE dictionary so.. maybe it is not cheating but using tighter assumptions than those which were provided by the original task description :-)
# Our test cases will always use the same dictionary file (with SHA1
# 6b898d7c48630be05b72b3ae07c5be6617f90d8e).Re: Stripe-CTF 3 Writeup
#46So we know for next time, how strong are other people's feelings on this?
Re: Stripe-CTF 3 Writeup
#47After many hours I failed to complete Level 4. How frustrating! But what a great experience. I think Stripe did a great job at building a programming challenge. By going through the challenge I learned something new in each of the following: Python, Ruby, Scala, Go, Node/Javascript, Git, Hashes, DDOS, Consensus Algorithms, raft, transactional logs, Inverted Indexes, BitCoin, IRC Chat, and more. Thanks Stripe you made…
Re: Stripe-CTF 3 Writeup
#48Very fair feedback on moving the deadline. We'd debated doing it or not, and ended up moving it in the spirit of distributed systems education. A lot of people in the chat appreciated it, but that's obviously a selected segment. So we know for next time, how strong are other people's feelings on this?
More people learning & having fun is better. IMO people only complain about the extended deadline because of its effect on the rankings.
The scoring system was an interesting addition over CTF2, but it also created these complaints and silliness like people submitting the same code repeatedly to get the best score possible.
It may have been better to only tell players what broad percentile group they're in. There would still be an incentive to write faster code, without the frustration of being edged out of position #20 because someone else's submission was (arbitrarily) scored a few points higher.
Re: Stripe-CTF 3 Writeup
#49(1) Python. Used a set() for the dictionary. Got ~200 points. I was planning to go back and implement this in Go to see the difference in performance but never got around to do it.
(2) Python. I wrote a single-threaded miner which could do around 300K hashes/s. My miner calculated 1M hashes (~3s), then fetched origin, then continued until it got a hit. I stopped after the first Gitcoin.
(3) Node. My solution was very simple and pretty much like the OP's. I kept a map between IP and count and always let through IPs with count 4, then I let them through with 0.3 probability to keep the backends working.
(4) I spend most of my time on this problem, largely trying to keep the memory within the constraints. I started with Node and created a simple inverted index of all words to their positions. Since we needed substring search I had to loop through the keys of the index which was too slow. My next attempt was to index all substrings but that used too much memory. My third, and best, Node attempt was using a prefix tree to index all suffixes, effectively building a suffix tree. This was quite fast, got ~2200 on local tests but was still over the memory limits.
I then gave up on Node and moved to Python. I used Flask for the server and built two solutions. A simple inverted index and a prefix tree (using datrie) with all suffixes. To keep the memory footprint low, I stored file/line locations as bit-shifted longs. Amazingly, the simple index and the prefix tree performed the same (!), so I submitted the inverted index solution and got a passing score.
Thanks for a great week Stripe. I learned tons! Looking forward to the next one.
Re: Stripe-CTF 3 Writeup
#50I held the top leaderboard spot of level1 for a day or so (my hashrate varied between 1-5GH/s) but was pushed out in the last few hours by pushrax and the Stripe server load. Used an OpenCL + Go solution. Was a close race there by all.
http://bochs.info/~aegis/rounds.tar.gz - tarball of all the Gitcoin round repos if anyone wants to play with the data.