Viewing profile — mebkorea

Sure, here you go: https://lar.liverpool.gov.uk/planning/index.html?fa=search Heads up tho... it's behind AWS WAF with a JS challenge. Solving the challenge once works fine, but th…

Thanks! Yup,... the freemium + paid analysis route is the obvious next step from where I am. You're right that my commercial intent is muddying the message. The sell skills, not da…

All fixed. The bug was in how I was building the postcode-to-council mapping. The script was extracting prefixes from individual application addresses, and Ceredigion's database ha…

Cheers, and things get sticky isn't lost on me, especially after some of the feedback. Seems all the feedback is converging on the same direction namely open-source the data layer,…

Yeah, unfortunately you are not wrong about the national tradition of wanting to pull down clever-clogs... The open source angle is something I'm increasingly considering, especial…

Fair points and yeah you must be sick of unrate-limited mass scraping. I run with 1.5-3 second delays from a single residential IP and back off when portals push back, but from you…

Thanks. Will look into that right away!

All good points. I've been so busy with the data collection and just "irl stuff" that I haven't spoken to planners directly which is an oversight on my part — they're the obvious p…

Nice thinking. Hadn't thought of DevTools MCP that way. Curl_cffi I've used for TLS fingerprinting (Edinburgh was the first one) but the discovery side I've been doing manually... …

Cheers, appreciate the feedback. The architect/consultant precedent angle is interesting and a couple of other commenters have already nudged me in similar directions. Tbh... you'r…

[dead]

Updated and pushed live: planninglens.co.uk/terms. Acceptable Use clause now permits programmatic access that respects rate limits, while still protecting our derived analysis and …

Thanks for the feedback. On TOS: the same answer as I gave others... the data is statutorily public, I respect rate limits. That being said, I admit it's a grey area I haven't 100%…

Thanks and yeah, some of my boundary data (for the choropleth) comes from ONS open boundary files which I think are OGL but I'd need to check the chain of derivation. On the data.g…

Fair points and I appreciate the feedback. Database right is real but the threshold is "substantial part". I'm literally only showing aggregates and letting people search by postco…

Honest answer... I don't fully know, zero paying customers so it's still very much a hypothesis. The two use cases I think hold up: (1) people pre-buying a house with extension pot…

Planning data on gov.uk is really patchy and not useful for what I want. There's planning.data.gov.uk which has some boundary/policy data but no actual decisions. The decisions onl…

Cheers! Yeah, it's honestly mental how fragmented it is. Every council is its own little island. On the shutting-off worry: the data is statutorily public. Councils are legally req…

Around four months part-time. The bulk was the first 6 to 8 weeks building the three main scrapers (Idox, Northgate, Ocellaweb). After that, councils on those platforms are mostly …

Yeah, that's essentially what I'm doing. Claude handles most of the look at the portal, work out the search form, write the config loop. The actual bottleneck isn't code tbh, it's …

Thanks, honestly that means a lot! Yeah, the pre-submission review idea is interesting and I've thought about it. I have the data to surface "applications similar to yours in your …

Fair catch and pretty embarrassing... ngl. That's a generic template clause I didn't think hard enough about at the time and it's obviously contradictory given what the site does. …

No, I haven't tried Browserless. So far, it has all been from a single residential IP which is probably the bigger issue with Liverpool than the WAF challenge itself. Once I have a…

Thanks, hadn't come across them. I will have a poke around and reach out. Appreciate the pointer.