Why hasn't this been automated? Digit recognition technology is pretty good and it seems like the forms are pretty standardized.
Probably because 99% accuracy isn't good enough.
I doubt you would get 99% accuracy with humans, either.
UK scientists ask for help in transcribing 200 years of rainfall data
21–30 of 43 posts
Re: UK scientists ask for help in transcribing 200 years of rainfall data
#22It would be great if the data entry panel and the main document scrolled separately. Right now I have to scroll up and down to enter data, which is frustrating.
I usually scroll down the main document and use the Tab key to tab between each field to enter data - that way, I don't need to keep scrolling back up and down.
Re: UK scientists ask for help in transcribing 200 years of rainfall data
#23Why science must always be done for free, and all the other non scientific stuff, including things dificult to explain logically, are gladly and generously overpaid?
Universities take an economic profit from this volunteer work. They can just share some and hire a few students to do it. And if there is a scarcity of more volunteers, maybe they should think about if repeatedly firing, whimsically dismantling expert teams, and cut the funds to scientists at the slighest opportunity really pays at long term.
Re: UK scientists ask for help in transcribing 200 years of rainfall data
#24Re: UK scientists ask for help in transcribing 200 years of rainfall data
#25Why hasn't this been automated? Digit recognition technology is pretty good and it seems like the forms are pretty standardized.
Is there a technology that would help ocr-ing standardized forms? I’ve got a couple thousand pages of historical train schedules that i would like to digitize (tabulated, printed data, including symbols/icons), but I’m not sure how to automatically recognize the structured data.
(disclaimer MS employee but nothing to do with this product)
You could check out the Azure Forms Recognizer. It's designed to do exactly this and make even training a custom model pretty easy. It's also pretty cheap (free up to 500 pages) so you can always do a quick experiment and see if it does what you need it to
https://docs.microsoft.com/en-us/azure/cognitive-services/fo...
Re: UK scientists ask for help in transcribing 200 years of rainfall data
#26Having collected so many data points, I wonder how it was used at the time? Clearly it was probably summed to give yearly totals and possibly monthly averages, and so on. But were the same sheets revisited year after year then?
Re: UK scientists ask for help in transcribing 200 years of rainfall data
#27Why hasn't this been automated? Digit recognition technology is pretty good and it seems like the forms are pretty standardized.
Is there a technology that would help ocr-ing standardized forms? I’ve got a couple thousand pages of historical train schedules that i would like to digitize (tabulated, printed data, including symbols/icons), but I’m not sure how to automatically recognize the structured data.
I remember getting some reference software and methods from NIST maybe 20 years ago on handwriting recognition. Never put it into practice, but maybe they have some updated software available
Re: UK scientists ask for help in transcribing 200 years of rainfall data
#28that's a job. go pay someone for their work.
Re: UK scientists ask for help in transcribing 200 years of rainfall data
#29Why science must always be done for free, and all the other non scientific stuff, including things dificult to explain logically, are gladly and generously overpaid? Universities take an economic profit from this volunteer work. They can just share some and hire a few students to do it. And if there is a scarcity of more volunteers, maybe they should think about if repeatedly firing, whimsically dismantling expert te…
Zooniverse was a direct response to a PhD project that Chris Lintott was supervising. His student was tasked with classifying galaxies and in the end decided that the volume of data was just too great.
There is a good supply of volunteers, and the tasks are enjoyed by those who undertake them. Where volunteers identify notable things, they are also credited by name in the resulting papers.
> maybe they should think about if repeatedly firing, whimsically dismantling expert teams, and cut the funds to scientists at the slighest opportunity really pays at long term.
I don't see much evidence for this happening in the UK, where the project is based.
Re: UK scientists ask for help in transcribing 200 years of rainfall data
#30What they should do is make a crypto platform like curecoin but for this where you earn coin for transcription of these documents (and others, could be a cool platform).
Or, you know, cut out the middle man and just pay per transcribed page.