Live data from Hacker News

Ask HN: Help, I'm Drowning in JavaScript

news.ycombinator.com

31–40 of 56 posts

Re: Ask HN: Help, I'm Drowning in JavaScript

#31
post #8

Hey, it's okay. I was in your exact boat a few years ago. Javascript has exploded in popularity in just a few years and yes, the ecosystem is horribly complex. It's an running gag even among full time JS devs how crazy it's gotten. None of us can keep up. It's OK. Just tackle it one step at a time. Forget Typescript for now. Don't use third party libraries until you need them. Arrow functions are optional. Ternaries…

Thanks for the kind words. Slowly going into the water is my approach as well, but sometimes it just gets to me. My learning projects die on the hill, because of the frustration I have on the job with these techniques. Plus, overwhelming ecosystem. About my regex problem: This is a structual mess. JSON/XML with HTML code in the data fields. We process them and send them to multiple job boards. Our clients mainly use…

I spent a decade parsing text-with-angle-brackets with regexes, and it sucks. It’s always tempting to try an html parser but if the code is written by a human (or worse, a mixture of human and machine, especially if the machine involves MS Word) it just doesn’t work.

I’d suggest rather than attempting to do big regexes that capture a bunch of stuff in one call, break it down to a bunch of smaller, more targeted calls - one call to capture the text of the whole record, another with 3 variants to get the title, another with 2 variants to pick up a tag line, etc.

Re: Ask HN: Help, I'm Drowning in JavaScript

#32
post #9

You can't parse [X]HTML with regex. Because HTML can't be parsed by regex. Regex is not a tool that can be used to correctly parse HTML. As I have answered in HTML-and-regex questions here so many times before, the use of regex will not allow you to consume HTML. Regular expressions are a tool that is insufficiently sophisticated to understand the constructs employed by HTML. HTML is not a regular language and hence…

This was the answer I was hoping to see with regards to OP's current troubles. Thank you.

Re: Ask HN: Help, I'm Drowning in JavaScript

#33
Accept that your understanding of JavaScript is outdated. Appreciate you have at least some experience though.

If you care to learn modern JavaScript, which you should, check out Eloquent JavaScript [0]. It’s a free web-based ebook providing an introduction to modern JavaScript.

Also, stop with the negative narrative. You are overwhelmed, not helpless. Instead of telling yourself you don’t understand code and giving up, just Google said code. It’s easier than ever. In a few weeks, you won’t need to reach for Google. If you encounter a dependency you don’t understand, try to find some documentation.

Accept that there are black boxes that “just work” despite being deprecated or lacking documentation.

Your job is not to learn the history of JavaScript and how it evolved from your understanding. Your job is to get up to speed with modern JavaScript enough to make functional programs to solve the business’ needs.

0: https://eloquentjavascript.net/

Re: Ask HN: Help, I'm Drowning in JavaScript

#34
Stop using regular expressions. Honestly, it sounds like you were having trouble with that, and blamed JavaScript.

Regular expressions are the worst.

Also if someone scraped web pages and then stored the HTML, that seems like a bad idea. You would want to actually extract the data at that point.

Also regular expressions are the worst way to extract data from HTML.

I think the real problem is the business model and technical approach. It almost sounds like you are scraping a massive number of websites using regexes, which is guaranteed to break somewhere on an almost daily basis since developers change the HTML and regexes are very brittle.

You are probably taking advantage of a lot of other people's work, possibly quite a bit of it copyrighted and certainly most of it NOT curated with the idea of some other company profiting off of it.

So you deserve your pain.

But don't blame JavaScript or lack of familiarity with new versions when it's all of your other life choices that are the real problem.

Re: Ask HN: Help, I'm Drowning in JavaScript

#35
I'm sort of in the same boat, but I'll tell you what's been helping me keep my sanity:

The debugger is your friend. Figure out how to get it attached anywhere if you haven't already.

That means running node with --inspect-brk= that also means figuring out where spawns are.

This exercise will greatly aid you in overcoming any surprising control flow going on by allowing you to single step the program. JS is one of those languages I just have to cling to my debugger in.

Two: node has two "ClassLoaders"(CommonJS/ESM). They don't mention squat about the differences between them until the middle of the nodejs documentation.

Start learning dependencies one by one. There's really no other way around it.

Avoid using regex for XML/HTML. Try something more akin to SAX parsers. Seriously. There:s way too much pain in doing regex XML processing. You could literally rewrite the damn thing in java faster than getting a shoddy, flaky JS monstrosity reliable.

God help you if they're doing weird Async/Event nonsense.

Keep cracking at it, but be aware, I think JS has broken my brain, because I just don't even like programming anymore since I started dealing with the language and all the necessary tranpilation/toolchains.

Re: Ask HN: Help, I'm Drowning in JavaScript

#36
post #8

Hey, it's okay. I was in your exact boat a few years ago. Javascript has exploded in popularity in just a few years and yes, the ecosystem is horribly complex. It's an running gag even among full time JS devs how crazy it's gotten. None of us can keep up. It's OK. Just tackle it one step at a time. Forget Typescript for now. Don't use third party libraries until you need them. Arrow functions are optional. Ternaries…

Thanks for the kind words. Slowly going into the water is my approach as well, but sometimes it just gets to me. My learning projects die on the hill, because of the frustration I have on the job with these techniques. Plus, overwhelming ecosystem. About my regex problem: This is a structual mess. JSON/XML with HTML code in the data fields. We process them and send them to multiple job boards. Our clients mainly use…

Thank you for the context! What you're doing is actually much harder than regular web dev. It's a specialized kind of data processing, often called a "extract, transform, load" (ETL) workflow.

Most web devs don't need to do that, and that you're willing to tackle it at all just shows how willing to learn you are, despite the frustration.

If you hate this situation, it's totally understandable lol. That kind of work has all the tedium of dealing with someone else's arcane data format, and none of the joy of seeing your creativity come to life. Some people love that sort of work, and specialize in it, becoming backend people or DB engineers or data scientists or the such, but it's not usually what web devs are known for (who tend to focus instead on UIs and some level of design and interactive stateful apps). Nothing wrong if ETL just isn't your cup of tea. I'd go crazy if I had to do that often, too.

Anyhow, if I'm understanding you right, you have HTML embedded in either JSON and/or XML. Do you know what "escaping" is in the text embedding sense? Like if you have quotes inside quotes, or tag brackets inside tags, how to separate each layer of embedding? If your JSON and XML files are cleanly escaped, you should be able to (as a first step) just iterate through the files and get the HTML parts out (without regex).

Like if the HTML is just a data string inside JSON, you can transform the JSON into an array of HTML strings using array.map() or object.values.map().

In the XML, if the HTML is stored in CDATA fields, you can access it using an "XPath" selector... you know how CSS has selectors that let you say headings should be styled one way, paragraphs another? XML has its own selector language that lets you directly target a certain node inside the document, without using regex, by specifying the hierarchical path that takes you there (like a CDATA inside a description inside a job inside a company, or whatever). Although there is a learning curve to XPath, it is much more suited to the task than regex, because the regex can't easily account for the complexity within XML (especially when there's nested layers).

It would help if you can post some example snippets, but that might be better suited for Stack than HN (though feel free to link to it here).

Once you have the HTML out, then you can run it through a sanitizer -- that's an optional step, but would let you strip out unnecessary divs, old font tags, whatever, keeping old basic formatting (headers, paragraphs, links, bold, etc.) which should be much cleaner to hand off to your clients. That would be much easier to embed on someone else's site vs a scraped page with all the HTML mess from someone else's framework.

I know there is a lot of complexity in each of those steps, but there are great tools and documentation for each step of the way. That's just to get you started.

At the end of the day what you're doing isn't really a Javascript issue at all, it's just a different kind of work that Javascript happens to be able to handle if you really need it to (but so can Python or Java or specialized command line tools like jq). It's a different body of work, which is why your casual web dev skills aren't providing easy answers. It's OK! You can learn it once and make it work (and then decide never to do that again, like I did lol). Or switch tracks, totally up to you :)

But feel free to ask here or on Stack if you have followups!

Re: Ask HN: Help, I'm Drowning in JavaScript

#37

It looks like you’re lacking some fundamentals. And not if/else type of, but rather oop, software design and organization of code kind of things. What will help is to master a single decent beginners course that focuses on applying fundamentals to building practical projects. And it’s better to be python or ruby based course (to minimize friction and infra pain). Don’t jump on anything else until you finished and hav…

I can agree on the fundamentals, even though I took OOP and CS in college, but that was a long time ago and not my main field of study. I find it hard to apply to JS though, as you said, it is not easy to comprehend. Are you suggesting a different technology stack for the fundamentals? Others seem to go the same way, but they are suggesting it within the ecosystem of JS, just not with the overhead of the new technolo…

> Are you suggesting a different technology stack for the fundamentals?

It's not that important in general. But I assumed that if you're already overwhelmed by JS infra, you might get a smoother initial experience with less messy yet still interpreted dynamic language like ruby.

Re: Ask HN: Help, I'm Drowning in JavaScript

#38
I just read https://insanecoding.blogspot.com/2009/11/they-actually-want... and didnt believe it. Turns out its true :o

You are very lucky nobody at your company caught on yet. My suggestion is hire me as a subcontractor - I will do your job for half your wage, you will have plenty of spare time to do whatever.

Re: Ask HN: Help, I'm Drowning in JavaScript

#39
You should check out the free online ebook series You Don't Know JS [0].

Go straight to the last book in the series, called "ES6 & Beyond" [1] which specifically covers the most important features of JavaScript that have appeared since you last worked with it.

For hands on practice with newer JS features, the best resource is Execute Program's "Modern JavaScript" [2] course. Once you work through that, move on to their TypeScript Basics course [3]. Both are amazing. Execute Program is the best tool for learning a programming language that I've ever used.

[0]: https://github.com/getify/You-Dont-Know-JS/blob/1st-ed/READM...

[1]: https://github.com/getify/You-Dont-Know-JS/blob/1st-ed/es6%2...

[2]: https://www.executeprogram.com/courses/modern-javascript

[3]: https://www.executeprogram.com/courses/typescript-basics

Re: Ask HN: Help, I'm Drowning in JavaScript

#40
post #9

You can't parse [X]HTML with regex. Because HTML can't be parsed by regex. Regex is not a tool that can be used to correctly parse HTML. As I have answered in HTML-and-regex questions here so many times before, the use of regex will not allow you to consume HTML. Regular expressions are a tool that is insufficiently sophisticated to understand the constructs employed by HTML. HTML is not a regular language and hence…

WTF, dude? I can’t even parse that enough to decide if it should get an up or down vote and I haven’t even tried regex.

WDYM WTF? He's clearly telling you that you can't parse HTML with regex. Don't even try.
Post reply on HN