I took it before they rolled out the chat bot TA, unfortunately.
A professor built a chatbot to be his teaching assistant
51–60 of 120 posts
Re: A professor built a chatbot to be his teaching assistant
#52This is the key: > The system is only allowed to answer questions if it calculates that it is 97 percent or more confident in its answer. Knowing when you don't know the answer is something that has been lacking from AI I've seen. Of course the problem is that not all AI has the option of deferring to a human. Anyone know what percent of questions the AI answered, vs redirected?
> Anyone know what percent of questions the AI answered, vs redirected? That's actually a very good question. The article doesn't give many details other than: "There are many questions Jill can’t handle." For comparison, here's are the highlights of a study from 2014 on the mobile assistants (Siri, Cortana, Google): http://searchengineland.com/google-now-beats-siri-cortana-di... Google was able to offer "enhanced" a…
I emailed and asked. If I get a reply I'll post it here.
Re: A professor built a chatbot to be his teaching assistant
#53Rather than inventing sophisticated chat bots, couldn't he have just had a well organised web page? If the question were where is assignment two / when is assignment due. The examples given seem like a "high tech" (possibly unreliable) solution to a low tech problem.
Having written documentation for end users, that will work for only a portion of them. Probably most, which is good. But there are a significant percentage of users who can't (or won't) read documentation. Even they do manage to read it, they don't understand it. For some reason, human interaction works better for those people. They ask a question, and someone copies the answer from the documentation, and pastes it i…
Situation 1: "Hey can you call me?" Sure, what do you need to talk about? "The status of the Spark server." Okay.
Find a quiet place, place the call, debug connection problems so we can hear each other.
So what do you need? "Is the Spark server up?" Yes. "Okay, thanks, bye."
Situation 2: Email to coworker: "Hey the other day you said you said that the downloaded certificate is one factor, so with the password, that's already two factors, and the rotating token would be three. But we only need two, so that would mean we don't need the rotating token now. But the CTO is confused then -- how was the certificate securely transmitted in the first place, to the point that we can count it as a second factor?"
Two days later: Coworker approaches me: "So what was that email about?"
(Only exaggerating a little bit.)
Re: A professor built a chatbot to be his teaching assistant
#54This sounded cool until, "Now Goel is forming a business to bring the chatbot to the wider world of education.” // are all educators just trying to create companies these days?
Re: A professor built a chatbot to be his teaching assistant
#55That's stunning. I studied AI in undergrad and I just didn't give it enough of my focus at the time. Little did I know that in my lifetime I'd see real world evidence of computers passing the Turing Test with flying colors. Of course, Turing was right. What's outside of the set of computable sets? Not much! I recently read an article headline that talked about creeps using AI to stalk porn stars. It's interesting to…
> What's outside of the set of computable sets? Human intution can jump. Mechanical deduction must expend energy. Let me give you an example: Dmitri Mendeleev goes to bed, has a dream, and then delivers the periodic table . Can your machine do that?
A line of elements that almost never react with anything. A bunch of things that gain one electron, some that lose one. And so on.
Re: A professor built a chatbot to be his teaching assistant
#56For anyone interested, the class, which covers interactive intelligence and knowledge based AI, is freely available at https://www.udacity.com/course/knowledge-based-ai-cognitive-... . It's a great starting point if you're looking to automate the sort of human intelligence tasks that don't lend themselves well to the traditional searching/planning/proving/minimax/retrieval route that underpins most AI. I took it befo…
I'm sure I asked/answered more than a dozen questions during that semester within the Piazza platform and likely unknowingly contributed to Jill Watson's training.
Re: A professor built a chatbot to be his teaching assistant
#57This is the key: > The system is only allowed to answer questions if it calculates that it is 97 percent or more confident in its answer. Knowing when you don't know the answer is something that has been lacking from AI I've seen. Of course the problem is that not all AI has the option of deferring to a human. Anyone know what percent of questions the AI answered, vs redirected?
Definitely, and I'm not sure why that space isn't getting more attention. For RL there's KWIK (knows what it knows) http://www.research.rutgers.edu/~lihong/pub/Li08Knows.pdf [pdf]. A large class of problems (the robo TA being one of them) lend themselves very well to AI with error bounds, for lack of a better term.
Re: A professor built a chatbot to be his teaching assistant
#58This is the key: > The system is only allowed to answer questions if it calculates that it is 97 percent or more confident in its answer. Knowing when you don't know the answer is something that has been lacking from AI I've seen. Of course the problem is that not all AI has the option of deferring to a human. Anyone know what percent of questions the AI answered, vs redirected?
Re: A professor built a chatbot to be his teaching assistant
#59Now this is a forum where others can chip in as well, but the FAQ page could prevent you from having to post in the first place. And if you don't have to post publicly to get an answer from the chat bot (because it's a FAQ page and not a bot) you can do 20 searches before deciding it's not in there.
Re: A professor built a chatbot to be his teaching assistant
#60Earlier quoted context omitted.
> Anyone know what percent of questions the AI answered, vs redirected? That's actually a very good question. The article doesn't give many details other than: "There are many questions Jill can’t handle." For comparison, here's are the highlights of a study from 2014 on the mobile assistants (Siri, Cortana, Google): http://searchengineland.com/google-now-beats-siri-cortana-di... Google was able to offer "enhanced" a…
> That's actually a very good question. The article doesn't give many details other than: "There are many questions Jill can’t handle." I emailed and asked. If I get a reply I'll post it here.