Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Tech jobs on the command line

github.com

Post reply on HN
Print view View original

51–60 of 61 posts

Re: Show HN: Tech jobs on the command line

#51

Post by hughdbrown »

I have not read all the code closely but it looks like you are scraping the HTML directly. True?

It would be easier to use the JSON feed for a couple of reasons: - the data is segregated into types of things that you otherwise have to back out from the HTML/CSS - you don't need logic for parsing multiple pages

Have a look at code I wrote on this: https://github.com/hughdbrown/who-is-hiring

The code to get a month of Who Is Hiring posts just hits `https://hn.algolia.com/api/v1/items/{article_id}` with a GET request for a particular article_id. Finding the monthly article_ids is the only manual maintenance.

The code is split into multiple commandline tools that extract JSON from HN, preprocess the JSON, and filter on what I was interested in (rust job postings). You should be able to replace downstream parts like preprocessing and filtering.

See if any of this is helpful.

Re: Show HN: Tech jobs on the command line

#52

Post by RyanShook »

kpandit wrote: post #35
RyanShook wrote: post #25

This is awesome. I've been looking for a way to scrape Who's Hiring for my site https://200kjobs.com , thanks for sharing.

Your site looks really nice and addresses a real problem. Are you targeting US only remote roles? If not then including any domicile or timezone restrictions in job snippets may be helpful. Also thanks for providing the feed.

That’s helpful feedback, thank you!

Re: Show HN: Tech jobs on the command line

#53

Post by nico »

hughdbrown wrote: post #51

I have not read all the code closely but it looks like you are scraping the HTML directly. True? It would be easier to use the JSON feed for a couple of reasons: - the data is segregated into types of things that you otherwise have to back out from the HTML/CSS - you don't need logic for parsing multiple pages Have a look at code I wrote on this: https://github.com/hughdbrown/who-is-hiring The code to get a month of…

Cool, thank you

Yes, scraping the html directly was relatively easy

Will try Algolia and the api when doing a cron job for it

Re: Show HN: Tech jobs on the command line

#54

Post by nico »

thomasfromcdnjs wrote: post #46
azophy_2 wrote: post #40

would be interesting to turn this into something like `npx commandjobs` or `ssh commandjobs.com` so you wouldn't even need to install new software

There is something similar https://github.com/alexpovel/ancv/

This looks really cool, thank you for sharing!

Re: Show HN: Tech jobs on the command line

#55

Post by cheesehead »

nico wrote: post #31
cheesehead wrote: post #29

Nice work! I did something similar with a personal project a few months ago using an open source llm. Also, not sure if you know, there is an api you can use. https://github.com/HackerNews/API

Awesome, did not know about the HN api, nice! What open source LLM did you use? Anything that would run on a CPU of a 5 year old MacBook Air (which I’m using as my home server) Do you have the code for your project? Or maybe some features/ideas that you could share? Thank you

I'm using https://github.com/ggerganov/llama.cpp and currently mistral 7b (on a m1 macbook pro). I'm sure with some prompt examples you can get pretty good results on a smaller model.

At the moment I don't have it open sourced due to it being part of a larger project that I'm working on that contains tailwindui licensed components.

A cool feature that I'm working on is creating a firefox plugin so you can save/index job postings from other sites and extract out meta information via an LLM. Very similar to this chrome plugin. https://chromewebstore.google.com/detail/huntr-job-search-tr...

Re: Show HN: Tech jobs on the command line

#56

Post by nico »

cheesehead wrote: post #55
nico wrote: post #31

Earlier quoted context omitted.

Awesome, did not know about the HN api, nice! What open source LLM did you use? Anything that would run on a CPU of a 5 year old MacBook Air (which I’m using as my home server) Do you have the code for your project? Or maybe some features/ideas that you could share? Thank you

I'm using https://github.com/ggerganov/llama.cpp and currently mistral 7b (on a m1 macbook pro). I'm sure with some prompt examples you can get pretty good results on a smaller model. At the moment I don't have it open sourced due to it being part of a larger project that I'm working on that contains tailwindui licensed components. A cool feature that I'm working on is creating a firefox plugin so you can save/index…

Love the extension idea, definitely useful for tracking opportunities

Thank you for the perspective on running local LLMs

Do you think that saas or local LLMs will be the most popular in the next 5 years?

Re: Show HN: Tech jobs on the command line

#57

Post by cheesehead »

nico wrote: post #56
cheesehead wrote: post #55

Earlier quoted context omitted.

I'm using https://github.com/ggerganov/llama.cpp and currently mistral 7b (on a m1 macbook pro). I'm sure with some prompt examples you can get pretty good results on a smaller model. At the moment I don't have it open sourced due to it being part of a larger project that I'm working on that contains tailwindui licensed components. A cool feature that I'm working on is creating a firefox plugin so you can save/index…

Love the extension idea, definitely useful for tracking opportunities Thank you for the perspective on running local LLMs Do you think that saas or local LLMs will be the most popular in the next 5 years?

Hoping it's local :)

Re: Show HN: Tech jobs on the command line

#59

Post by ydant »

nico wrote: post #21
ab_testing wrote: post #19

Excellent project. Is there a way to scrape and pull jobs from workday portals. I see lots of companies using workday and usually it is a pain to go through their job postings?

Thank you! It’s been fun building it and I’m looking forward to keep improving it I definitely want to add scrapers for more sources Haven't tried workday, is it popular for tech jobs (ie. with people that use the command line)? Do you have a sample url with listings you could share?

Here's some with tech jobs listed:

https://workday.wd5.myworkdayjobs.com/Workday https://shipt.wd1.myworkdayjobs.com/en-US/Shipt_External

It doesn't seem like there's a global listing across myworkdayjobs.com sites - each company using it has its own sub-domain. Which sort of makes sense - Workday is an internal HR/employee portal tool licensed to companies.

Re: Show HN: Tech jobs on the command line

#60

Post by nico »

ydant wrote: post #59
nico wrote: post #21

Earlier quoted context omitted.

Thank you! It’s been fun building it and I’m looking forward to keep improving it I definitely want to add scrapers for more sources Haven't tried workday, is it popular for tech jobs (ie. with people that use the command line)? Do you have a sample url with listings you could share?

Here's some with tech jobs listed: https://workday.wd5.myworkdayjobs.com/Workday https://shipt.wd1.myworkdayjobs.com/en-US/Shipt_External It doesn't seem like there's a global listing across myworkdayjobs.com sites - each company using it has its own sub-domain. Which sort of makes sense - Workday is an internal HR/employee portal tool licensed to companies.

Excellent, thank you so much for the links, it's super interesting

It should be pretty straightforward to create a scraper for one of those portals

If you feel like taking a crack at it, checkout the code of the Ask HN who's hiring scraper here: https://github.com/nicobrenner/commandjobs/blob/47b5c89402a3...

I added a comment under the Add Sources issue, to support Workday's career portals: https://github.com/nicobrenner/commandjobs/issues/23#issueco...

Post reply on HN