Anyone have technical details about this story? What kind of bot used, languages, algorithms, etc?
IBM Watson
Here's a better article that was linked a few days ago: http://www.wsj.com/articles/if-your-teacher-sounds-like-a-ro...
A professor built a chatbot to be his teaching assistant
41–50 of 120 posts
Re: A professor built a chatbot to be his teaching assistant
#42This is the key: > The system is only allowed to answer questions if it calculates that it is 97 percent or more confident in its answer. Knowing when you don't know the answer is something that has been lacking from AI I've seen. Of course the problem is that not all AI has the option of deferring to a human. Anyone know what percent of questions the AI answered, vs redirected?
> Anyone know what percent of questions the AI answered, vs redirected?
That's actually a very good question. The article doesn't give many details other than: "There are many questions Jill can’t handle."
For comparison, here's are the highlights of a study from 2014 on the mobile assistants (Siri, Cortana, Google): http://searchengineland.com/google-now-beats-siri-cortana-di...
Google was able to offer "enhanced" answers (i.e. more than a web search) 58% of the time. Siri was at 29% and Cortana was at 20%. Siri improved some with iOS9, but not enough to beat google: http://www.recode.net/2015/9/20/11618696/how-intelligent-is-...
From the actual study: ... we took 3086 different queries and compared them across all three platforms. These were not random queries. In fact, they were picked because we felt they were likely to trigger a knowledge panel. ( https://www.stonetemple.com/great-knowledge-box-showdown/ )
Re: A professor built a chatbot to be his teaching assistant
#43This sounded cool until, "Now Goel is forming a business to bring the chatbot to the wider world of education.” // are all educators just trying to create companies these days?
Re: A professor built a chatbot to be his teaching assistant
#44Actual URL path: /this-professor-stunned-his-students-when-he-revealed-the\ -secret-identity-of-his-teaching-assistant/ HN can do better than clickbait sites like the WaPo.
I've never read much of the Washington Post, so I can't say whether there's actually been a decline since Bezos bought them. But there's been a remarkable quantity of terrible reporting from them lately, so at least we can say it's not very good under his ownership.
Re: A professor built a chatbot to be his teaching assistant
#45Earlier quoted context omitted.
I think you'd be surprised at how often students don't read webpages and fire off an email to the class TA or instructor. Replying with a link to the relevant part of the webpage is something I'd love to automate. (I teach a largish introductory university class)
You can automate replying with a link to the course webpage ToC. Also, make it absolutely clear that the reply is automatic. Hopefully they'll get the hint. Or heck, just make it a policy that if they haven't received an answer in a week, this means they need to look at the website. I don't know why some people strive to be so accommodating to laziness nowadays.
Because in many cases, the lazy are paying the bills.
Re: A professor built a chatbot to be his teaching assistant
#46This sounded cool until, "Now Goel is forming a business to bring the chatbot to the wider world of education.” // are all educators just trying to create companies these days?
In my honest opinion, innovating in the Educational sphere and showing value to one or more groups - educators, students, administrative tasks - should really be rewarded fiscally. An institution won't really do that, except in the long-term by way of job stability and benefits (all subject to modern fiscal pressures). So, while it might seem the case, I think this avenue is perfectly reasonable.
Now, educators who write texts in concert with publishers and then somehow coerse students into buying brand new copies every semester, that's what I'd call profiteering. Not so highly esteemed to me. But, as noted, the profession is a tradeoff a lot of times.
Re: A professor built a chatbot to be his teaching assistant
#47That's stunning. I studied AI in undergrad and I just didn't give it enough of my focus at the time. Little did I know that in my lifetime I'd see real world evidence of computers passing the Turing Test with flying colors. Of course, Turing was right. What's outside of the set of computable sets? Not much! I recently read an article headline that talked about creeps using AI to stalk porn stars. It's interesting to…
> What's outside of the set of computable sets?
Human intution can jump. Mechanical deduction must expend energy.
Let me give you an example: Dmitri Mendeleev goes to bed, has a dream, and then delivers the periodic table.
Can your machine do that?
Re: A professor built a chatbot to be his teaching assistant
#48This is definitely an impressive result, but I'm pretty sure it was made easier by the opposite pattern. I've had a couple of TAs who I strongly suspected of being chatbots, even when I was talking to them in person. I'm still not 100% convinced they were human.
Yep, and it's not like you want to make such an accusation when you're in that position.
In fact, that's one way to pass the Turing test: any judge that suggests a subject is bot, experiences a permanent loss of social status regardless of correctness!
Re: A professor built a chatbot to be his teaching assistant
#49Rather than inventing sophisticated chat bots, couldn't he have just had a well organised web page? If the question were where is assignment two / when is assignment due. The examples given seem like a "high tech" (possibly unreliable) solution to a low tech problem.
Having written documentation for end users, that will work for only a portion of them. Probably most, which is good. But there are a significant percentage of users who can't (or won't) read documentation. Even they do manage to read it, they don't understand it. For some reason, human interaction works better for those people. They ask a question, and someone copies the answer from the documentation, and pastes it i…
I often suffer from this - and it stems for me from apparent time pressure and or resentment.
The time pressure is the source of all sorts of sub-optimal issues - "I don't have time to write tests, documentation, do the right thing here ... Whatever. Including reading around the subject"
I do think modern corporate culture contributes to this - I remember being impressed by a PhD DSP guy who was in my scrum stand up who most days would report "I was reading yesterday and will be reading today". Every so often he would say "I finished reading and wrote ten lines that solved the problem"
As a contractor I felt I could not take a similar response, but I should ... or at least should be "I will be refactoring our Dbase access code / Our API / whatever cos it sucks rocks"
But I am weak.
Oh and resentment - "I don't want to find out the details of the pointless process this pathological bureaucracy has created and I really don't want to read a badly written set of documentation that probably does not have the real answers in it because no one likes to write the honest answers"
Hmm, glad it's Friday. I am worse than usual
Re: A professor built a chatbot to be his teaching assistant
#50This is definitely an impressive result, but I'm pretty sure it was made easier by the opposite pattern. I've had a couple of TAs who I strongly suspected of being chatbots, even when I was talking to them in person. I'm still not 100% convinced they were human.
Yeah, there exist humans who fail the Turing test. Another thing is foreign students and teachers who, even if otherwise competent, quickly get one used to language comparable with machine output. I think I may have been guilty of that at one time :)
You know this is very interesting. I've often wondered if other humans I'm interacting with were in fact sapient.