Become a member.

Tired of the toxicity of social media? Why not join Ricochet for just $5/month? Think of it as an online bar. Come in, have a drink, hang out, say whatever you want. But don’t badger, bully, or berate the other drinkers. That's our only rule.

Confessions of a Failed AI Trainer

Old Bathos
@OldBathos
Join to bookmark15likesJoin to like10comments

On a lark, I signed up with a couple of sites to be an AI trainer. To qualify, there were a number of qualifying initial tasks to construct responses and evaluate and compare sample responses. AI-assisted humans train other humans whose input will train the AI. Farther along, the training tasks involved constructing responses that met a certain rigorous quality control standard. After a bunch of that sort of thing, and as patterns emerged, I dropped out.

The criteria for judging/creating content is very similar to that employed by the most demanding English teacher you ever had [NOTE for younger readers: teachers of previous generations of students demanded (a) correct use of grammar; (b) no spelling errors (there was no spell check in those days); (c) adherence to clear rules of structure and composition and (d) the students’ feelings, identity and/or delusions of creative genius were not tolerated much less encouraged.]

The AI trainer training was centered on a detailed grid of criteria. The order and relevance of content, whether additional useful facts were found and brought to bear, and whether actual new insights were included were all rated. And there can be absolutely no offensive or abrasive language of any kind. Done right, a commercially customized AI could presumably handle a wide variety of customer interactions quickly and pleasantly so that the company need not retain a phone bank in Mumbai or Manilla with very polite but heavily accented customer service reps of varying knowledgeability.

A well-trained AI is like a moderate Republican—instinctively recoiling at abrasive partisan language and too polite to rip semi-popular ideas it has good reason to know are deeply stupid. My impression is that no commercial-grade AI is being trained by studying the works of Howie Carr, Marc Steyn or Kurt Schlichter.

The big question is whether AI is likely to be just a reflection of the Overton Window, shifting as public sentiments change or will AI act to reinforce or cement the existing prevalent outlook and style or even allow for its manipulation by insiders. In any case, the Turing test will become easier – the machines will be instantly recognizable as polite, armed with the most soothing and seemingly reasonable statements about almost everything while speaking in perfect English whereas humans will increasingly be ignorant, angry, edgy, poorly educated and unable to stay on point.

Recommended by Members

Discussion (10)

  • Lunchbox Gerald@Jose
    Old Bathos (View Post):
    My impression is that no commercial-grade AI is being trained by studying the works of Howie Carr, Marc Steyn or Kurt Schlichter.

    You made me laugh!

    Old Bathos (View Post):
    In any case, the Turing test will become easier – the machines will be instantly recognizable as polite, armed with the most soothing and seemingly reasonable statements about almost everything while speaking in perfect English whereas humans will increasingly be ignorant, angry, edgy, poorly educated and unable to stay on point.

    What a thought. Humans devolving into Morlocks was already a danger before AI, but now...

  • Randy Weivoda@RandyWeivoda
    Old Bathos (View Post):
    My impression is that no commercial-grade AI is being trained by studying the works of Howie Carr, Marc Steyn or Kurt Schlichter.

    Would you want to call up customer service and deal with Kurt Schlichter? It might make for a good comedy sketch, but real life customers who are having problems with your product aren't going to enjoy an abrasive personality on the other end of the line.

  • Mark Camp@MarkCamp

    In a Turing test administered by a minimally educated tester, an AI software instance that was trained to meet the criteria you listed—i.e., which had 9th grade or higher thinking, reading, and writing skills—would be quickly identifiable as the fake human. A pool of 99 humans drawn at random from the population of Americans with a Bachelor’s degree or higher would likewise all soon reveal themselves as the real people.

    I wouldn’t be surprised to see a tester give them a question of moderate complexity and average emotional charge, and catch ‘em all in the first round.

  • Old Bathos@OldBathos
    ReplyingtoMark Camp

    Mark Camp (View Comment):

    In a Turing test administered by a minimally educated tester, an AI software instance that was trained to meet the criteria you listed—i.e., which had 9th grade or higher thinking, reading, and writing skills—would be quickly identifiable as the fake human. A pool of 99 humans drawn at random from the population of Americans with a Bachelor’s degree or higher would likewise all soon reveal themselves as the real people.

    I wouldn’t be surprised to see a tester give them a question of moderate complexity and average emotional charge, and catch ‘em all in the first round.

    Prove that Mitt Romney did not generate that response.

  • The Reticulator@TheReticulator
    ReplyingtoOld Bathos

    Mark Camp (View Comment):

    In a Turing test administered by a minimally educated tester, an AI software instance that was trained to meet the criteria you listed—i.e., which had 9th grade or higher thinking, reading, and writing skills—would be quickly identifiable as the fake human. A pool of 99 humans drawn at random from the population of Americans with a Bachelor’s degree or higher would likewise all soon reveal themselves as the real people.

    I wouldn’t be surprised to see a tester give them a question of moderate complexity and average emotional charge, and catch ‘em all in the first round.

    Old Bathos (View Comment):

    Prove that Mitt Romney did not generate that response.

    Cannot or will not?

+5 more comments
Join the conversation

You're seeing five of ten comments. Sign in, or join for just $5 per month, to read them all and add your own.