Hardware hacking: Rowhammer, spectre, meltdown, cache eviction attacks and a bit about designing chips (see one of my submissions for that)
Ask HN: What technologies did you learn in 2018?
411–420 of 620 posts
Re: Ask HN: What technologies did you learn in 2018?
#412Picked up some Docker and some Azure. Haven't applied them since. I'm hard-pressed to remember the last time I learned something new and actually applied it in the future. I'm getting pretty good at running projects where I know nothing about the stack.
Re: Ask HN: What technologies did you learn in 2018?
#413ReactJS. ElasticSearch.
Not disappointed.
Re: Ask HN: What technologies did you learn in 2018?
#414I started my job last year in July as a search engineer in a major Indian e-commerce company straight out of college and was really glad to be able to work using one of the most beautiful tech stacks I've seen - Golang microservices talking to Apache Solr, Redis and a little bit of MySQL/Mongo connected to the external world through RabbitMQ. I learnt writing good abstractions, organising code, the value of commentin…
I suppose your from the IIT/NITs, getting into Flipkart is not easy otherwise.
Re: Ask HN: What technologies did you learn in 2018?
#415Learned Scala, Kafka, and spark (and spark streaming/structured streaming). I will say that after using spark, the big data community badly needs better distributed computing tools. It was just awful to work with. Also good luck debugging anything easily when you create stuff with the RDD apis. I did come to like Scala, and Kafka as well, though.
Are you using Scala, Kafka and Spark for Machine/Deep Learning? I've been wanting to learn Spark, I only have have a vague idea if it. What's the best way to start and can you explain how Kafka ties in with Scala and Spark?
For the first question, we were looking at it to perform distributed matrix factorization in the immediate use case, but were planning to use the platform for more advanced techniques later on. We also tried to implement a quick and dirty distributed ALS algorithm and that's where we used the RDD API came in and my subsequent hatred for said API. Kafka was our streaming platform through which all our customer data was sent.
For starting out, the tutorials on the spark documentation pages are pretty good actually. However for streaming sources it seemed hacky to use spark with our kafka use since we had to basically scour maven/google for a package that did anything useful with avro-serialized data. Even after finding relevant packages we had to copy and paste some avro-to-sparkstruct lib code into an internal library since it didn't play nice with our versions of spark in it's public form. We were using structured streaming so that we could leverage the dataframe API which is much easier for data scientists to use and grasp coming from pandas/R. It's a lot nicer for a data engineer as well (imo).
JSON-serialized data worked okay actually, so we ended up using kstreams to deserialize the avro data and output it to another topic that was JSON-serialized since spark handled that better (read: more easily). Then we used the dataframe APIs to implement some of the higher order operations such as building interaction matrices and factoring them out, then outputing the resulting factors to cloud storage for collaborative filtering to use as a recommendation system we had elsewhere.
I guess to me it seemed wholly immature for Kafka-sourced data and it made me think that if we dumped into Hadoop and then operated on microbatches it would have been much easier.
About Scala vs Java vs python... We saw that python still wasn't the first-class language contrary to what they say I'm the docs and that to utilize any newer, fancier algorithms that would come up it would be beneficial to use Scala since that is the language most developed against (as of our use of spark). There are plenty of libraries to connect with Kafka in any language so Kafka didn't factor into the language decision at all.
I will leave out some details about managing the resources on a cluster as well as implementing CI/CD. That could be a whole blog post on it's own.
Hope this helps. It may be the best tool at the moment for what it offers, but it was painful to use and I hope someone makes something better with the knowledge gained from the usage of spark.
Re: Ask HN: What technologies did you learn in 2018?
#416This was a Vue.js year for me.
I'm the past I've used angular and react, but when I was tasked with designing a project manager for artists I decided to try out Vue.
I was originally interested because it seemed easy to include in an existing PHP project without needing to pull in webpacker or extra tooling, while still giving me the best parts of React.
It was. I quickly fell in love with the syntax and was happy to see that the ecosystem was vibrant enough that I could find a good Vue drag and drop library (built on sortable.js) on github.
Re: Ask HN: What technologies did you learn in 2018?
#417Earlier quoted context omitted.
My priorities are unique to my situation, so a responsible answer to your question is "it depends on what you're trying to do". I am trying to create a scalable platform. Without having best practices to learn from, I had to discover what that required while trying to make the most of Rust. The discovery process involves many iterations of trying approaches out until a decent one emerges. On the other hand, once I la…
how are you building dynamic queries without a query builder? (for example creating a complex where condition based on given GET parameters)
You could answer this question for your Python work if you refactor one of your dynamic queries to SQL Alchemy's query binding facility. You're working with a query string so you manipulate the string using whatever tools are available in the language. Lots of possible approaches in Python. Same for Rust.
Re: Ask HN: What technologies did you learn in 2018?
#418Angular, TypeScript, Ionic.
After working with Scala for 7 years and Scala.js for the last 3 years, a side project led me to learn what life is like on the frontend outside of Scala.
TypeScript is an impressive language, really like union types, string literals, and to some degree structural typing. Development is fairly friction-free, and VS Code is a breath of fresh air compared to cumbersome IDE support in Scala.
As for Angular + Ionic, generally terrible, but I'm not a fan of heavy frameworks. You can create cross platform apps, yes, but performance and development speed leave something to be desired -- Scala is known to have a fairly slow compile/reload dev cycle, but Ionic + Webpack is on an entirely different level of slow.
Looking at Scala + Scala.js + Cordova + Sqlite as a possible stack for cross platform apps that use Quill[1], a type safe query DSL, on backend and frontend.
Re: Ask HN: What technologies did you learn in 2018?
#419Earlier quoted context omitted.
Have you found the certifications useful?
Yes and No. I don’t pay much credence on certification as proving competence. I take them as a slightly negative signal when I am interviewing someone. Certifications for me are partially forcing functions to make me get a broad overview of a subject matter. At the level I’m at now, my “whiteboard interviews” are not about leetCode they are about how would I architect systems, meaning I don’t need to know the details…
I'm also going for the Big Data one, looks like a lot of fun
Re: Ask HN: What technologies did you learn in 2018?
#420Earlier quoted context omitted.
> derivatives or multiply two matrices > college math Those should be high school on most places
Derivatives were available for the most advanced level students in high school where I went. Even then it was understood that they were receiving college credit early. Linear algebra not at all.
Which is odd, in my opinion. Linear algebra is easier than Calculus and would likely be more useful to the average person.