Roarrsome, congrats!
roar (:
Launch HN: Datasaur (YC W20) – data labeling interface for NLP
61–67 of 67 posts
Re: Launch HN: Datasaur (YC W20) – data labeling interface for NLP
#62Earlier quoted context omitted.
I think the semi-automated annotation tool is quite the same with prodigy
Both can be described as semi-automated annotation, but we use different approaches! Datasaur allows you to plug in any pre-existing model to pre-label or validate your labels. One such model we integrate with is spaCy, so we're certainly fans of the Prodigy/Explosion team.
Yes it is more similar to Figure Eight Text Annotation which allows you to use pre-existing models including Spacy or you can bring your own model
https://www.figure-eight.com/platform/machine-learning-assis...
Re: Launch HN: Datasaur (YC W20) – data labeling interface for NLP
#63Interesting product. Could have used this at previous companies. How is this different from FigureEight or Scale?
Yes looks directly comparable to Figure Eight. This product is clearly meant to be similar to the Text Annotation tool: https://www.figure-eight.com/platform/machine-learning-assis...
Both products offer the opportunity to use provided labelers or to bring your own.
Re: Launch HN: Datasaur (YC W20) – data labeling interface for NLP
#64Earlier quoted context omitted.
We've looked into it! So far we've chosen to integrate with spaCy. Can I ask what you like about AllenNLP?
Hi there, great product! I'm with a nonprofit edtech writing tool that uses both spaCy and AllenNLP, and we've found AllenNLP's models to be more accurate for tasks like co-reference resolution. AllenNLP's models are built on top of spaCy and tools like neural coref. It'd be great if we could harness things like AllenNLP's semantic role labeling.
We actually allow you to plug in any existing model (including your own). So we should be able to support AllenNLP as well. happy to chat further!
Re: Launch HN: Datasaur (YC W20) – data labeling interface for NLP
#65Looks interesting, signed up to try this out and see if it might deal with some of our labelling.
A few notes -
Some help docs would be good, or better links to them. You specify a few types of projects but don't really explain what they are - I tried searching for "constituency" type projects but I have no idea what they are still.
You're sending error messages to the frontend. "Cannot read property 'startsWith' of undefined" is not something that should be reaching an end user, and this is happening unreliably when I upload files.
If I upload a CSV file I cannot seem to do NER from "new project". NER specifically chosen supports TSV but not CSV. My TSV files just say "server error", though they load as just txt files.
What's a question set? What's the format you need from me (it just says "csv").
Autolabelling seems to do nothing. Do you have example text where that should work?
I can navigate the text but the hotkeys for labelling don't do anything until I've already clicked once.
Search & label all is interesting but doesn't seem to give me any labelling options. Also the regex search for "someword \w" just returns all two words next to each other which seems wrong to me.
Congrats on the launch!
Re: Launch HN: Datasaur (YC W20) – data labeling interface for NLP
#66Looks interesting, signed up to try this out and see if it might deal with some of our labelling. A few notes - Some help docs would be good, or better links to them. You specify a few types of projects but don't really explain what they are - I tried searching for "constituency" type projects but I have no idea what they are still. You're sending error messages to the frontend. "Cannot read property 'startsWith' of…
Hi IanCal -
Really appreciate you taking the time to test Datasaur out and provide this feedback! Responses below:
- We took the risk of launching before our tutorial is in place to get feedback, so I do apologize for lack of clarity for first time users.
- Good point, we'll try improve error messages for users.
- We've been taking import formats on a case-by-case basis, but we're looking to improve/expand our import/export format flexibility further.
- This is probably one of the highest priority issues for us to fix - making sure expected format and input is clear to the user.
- Happy to send you additional sample files and instructions on how to get autolabelling working. We can accept your own models/api endpoints as well!
- Hotkeys for labeling should work - this bug seems odd. What kind of project is this for?
- Search and label should work as soon as labels have been uploaded, sorry that it wasn't working for you.
Happy to discuss any of this in further detail. Thanks again for leaving such comprehensive thoughts!
Re: Launch HN: Datasaur (YC W20) – data labeling interface for NLP
#67Looks great, Ivan. Congrats!
If I understand correctly you would use Spacy and some pre-trained models to validate human labels. Now my question is that what is the point of collecting labels for ML training if we already have a valid model for the same task that can complement human labels?