Live data from Hacker News

Ask HN: How do you search for $string if a webpage “supports” infinite scroll?

news.ycombinator.com

31–40 of 97 posts

Re: Ask HN: How do you search for $string if a webpage “supports” infinite scroll?

#31

I see lots of replies suggesting scrolling for a long time and then using CTRL+F. However, this won't work if the page is using virtualised scrolling (common with React et al. SPA for performance reasons, to avoid huge DOM trees as the page expands). The majority of content that is outside of the visible window will simply be unmounted from the DOM. I'm not sure what the best-practice for a webapp designer is here? P…

> I'm not sure what the best-practice for a webapp designer is here? Perhaps intercepting Ctrl+F

Oh god no, the solution is simple stop reimplementing the browser in the browser.

Re: Ask HN: How do you search for $string if a webpage “supports” infinite scroll?

#32
I open dev tools (or dev proxy) to see all the connections the site makes, grab the url to the "page" AJAX call. Put ' (single quote) into the URL which then returns a SQL error. Then I make a SQL injection to make the page return everything. And search that.

Re: Ask HN: How do you search for $string if a webpage “supports” infinite scroll?

#33

I see lots of replies suggesting scrolling for a long time and then using CTRL+F. However, this won't work if the page is using virtualised scrolling (common with React et al. SPA for performance reasons, to avoid huge DOM trees as the page expands). The majority of content that is outside of the visible window will simply be unmounted from the DOM. I'm not sure what the best-practice for a webapp designer is here? P…

> Perhaps intercepting Ctrl+F and displaying a custom search that will do the correct filtering on the back-end and update/retarget the view?

Stripe does this in their docs and I find it absolutely infuriating.

Re: Ask HN: How do you search for $string if a webpage “supports” infinite scroll?

#34

What I have done in the past: Scroll down a couple of days/pages/whatever and then use Ctrl+F. It‘s annoying as hell, but that usually works as most pages just add stuff at the bottom and don‘t unload the previous content on top

As mentioned above, there are some sites that page out the content that is no longer in the viewport after you've gone suitably far from it.

I still remember my consternation when I first noticed this, and it was because I tried exactly what you had suggested, but a previous match further up the page disappeared upon subsequent CTRL+F-ing.

Re: Ask HN: How do you search for $string if a webpage “supports” infinite scroll?

#35

I see lots of replies suggesting scrolling for a long time and then using CTRL+F. However, this won't work if the page is using virtualised scrolling (common with React et al. SPA for performance reasons, to avoid huge DOM trees as the page expands). The majority of content that is outside of the visible window will simply be unmounted from the DOM. I'm not sure what the best-practice for a webapp designer is here? P…

> I'm not sure what the best-practice for a webapp designer is here? Perhaps intercepting Ctrl+F Oh god no, the solution is simple stop reimplementing the browser in the browser.

Google Drive does this within Google Chrome, fwiw. I don't disagree with the spirit of your comment though.

Re: Ask HN: How do you search for $string if a webpage “supports” infinite scroll?

#36
post #22

Other problems: - Scroll, scroll, scroll, and then click a link. Now click back. You have lost your place. - How can anyone link to a section far down? - Impossible to read or use anything in the footer. I suppose if you install infinite scrolling, you remove the footer. But I have heard a story where they forgot.

Twitter seems to have the back thing down somehow. I don't know how it works (my best guess is making a and only loading a few tweets around the scroll position and making that work with bfcache) but so it is apparently possible to make that work fine. Linking to things in the middle of a page was solved in the 90s with the hash part of the URL. Aside from legal terms or privacy policies, I haven't yet seen a site wi…

> Linking to things in the middle of a page was solved in the 90s with the hash part of the URL.

And then promptly broken again in the mid-00s more or less till now, by way of sub-par client side routing, giving rise to SPAs and breaking back buttons, history management and deep linking all over the place. Things are getting slightly better since the history api came about, but you still see loads of sites with non-existent or broken deep linking, and crappy history management, such as reddit. This is of course because the browser behavior of fragment links, history etc. on static content is well defined, but for dynamic content in most cases have to be considered and implemented in the application logic, and it’s never a priority.

Hands up everyone who’s been in planning sessions for your project and right from the start you’ve considered deep linking and history management! Anyone..? No, ok.

Truth is, in my near 15 years of web dev experience, I’ve never once seen this be part of the requirements, because you kind of just expect it to work. Yet it’s incredibly easy to break, unintentionally even, and three weeks before release it’s usually incredibly difficult to fix because of all the corners we’ve painted ourselves into. So then it becomes a thing you fix after the fact, and spend tons of time tryin to figure out how to wrangle your router or framework or what have you to work it out, but turns out you’ll probably have to rewrite it all. But of course you won’t, because it’s silly, so it just becomes one of those things you paper over where possible and quietly ignore otherwise... sigh

Sorry for the rant.

Re: Ask HN: How do you search for $string if a webpage “supports” infinite scroll?

#37
More and more I feel like we should be simulating a human by macroing an actual typical web browser, including macroing the browser's dev tools.

(When scraping is fragile and you may have to periodically maintain scripts anyway, this is my conclusion after trying to automate a React website using recently-mentioned lib taiko, and finding it to be hit & miss, although it probably speaks more to my inexperience in general.)

Re: Ask HN: How do you search for $string if a webpage “supports” infinite scroll?

#38
post #28

I see lots of replies suggesting scrolling for a long time and then using CTRL+F. However, this won't work if the page is using virtualised scrolling (common with React et al. SPA for performance reasons, to avoid huge DOM trees as the page expands). The majority of content that is outside of the visible window will simply be unmounted from the DOM. I'm not sure what the best-practice for a webapp designer is here? P…

that's what discourse does. it's frustrating because it's significantly slower than the built-in browser search, probably in part due to excessive network accesses, but also probably due to poor caching and inefficient implementation.

The custom UI is also IMO terrible. It's completely different than the browser Ctrl-F UI. It does not simply highlight and move to the matches. Instead it shows a "preview", so you have to navigate to each match separately and back to the search UI. And it just navigates to the matching comment, but for very long comments I still have no idea where in the comment the match is. Thankfully you can press Ctrl-F twice to get the browser UI.

Re: Ask HN: How do you search for $string if a webpage “supports” infinite scroll?

#40
post #26

For reddit specifically, the answer seems to be to use "old.reddit.com" instead of "www.reddit.com". The is no general solution. Infinite scroll style web apps are implementing their own content view, in essence a web browser inside a web browser. It will never behave as users expect.

I personally prefer the interface of "i.reddit.com", but they semi-frequently let the cert expire for that one.
Post reply on HN