Learning with Social-Emotional Robots


Orpheum Children’s Science Museum in Champaign-Urbana has a great resource upstairs: a closet packed full of LEGO mindstorms parts. Dozens of motors and sensors, a classroom set of Mindstorms CPUs and laptops to program them, and bins & bins full of parts.

When I started volunteering as an assistant, I was really pleased with the diversity of activities Greg Fabry stuffed into the robotics camp sessions: gum drop toothpick engineering challenges, blindfold-your-friends and ‘programming’ their movement through an obstacle course, and a good old egg drop. There was certainly a lot of learning going on, from social+teamwork to mechanics intuition (building strong things out of weak materials is a wonderful foundation for engineering).

However, I always thought it was a shame the kids didn’t get more hands on time with the expensive robotics tools that were unique to that space. The class met twice a month for less than 2 hours per session, so the actual hands-on experience with robotics was very modest. So, the next year when I was asked to lead the class, I decided I would try my own off-the-cuff teaching with as much robot building and programming as possible.

The usual LEGO bot. Photo credit: nxtprograms.com

Over the course of 2 or 3 sessions with the classic curriculum of building a simple rover and programming it to navigate obstacles, I made a few observations of how a classroom of mixed skill-sets interacts with these robots.

The kids that have experience reading LEGO build instructions (which involves lots of counting, unless you’ve already mastered estimating the lengths at a glance) get ahead while newbies make mistakes that necessitate taking the robot apart (very discouraging for a newbie). So you’ve already got a split classroom where some kids maybe get the feeling that they’re not in the right place.

And it’s the same story with programming: with 2–4 kids per laptop, it’s hard to get around the fact that the kid who’s done it before is going to take the mouse from a newbie struggling to solve the introductory problems, unless you specifically tell them not too, in which case maybe they feel frustrated and slowed down. You can get past that OK by setting the expectation at the beginning of class that people who know how to do something should instruct their less experienced teammate by pointing at the screen, not taking away the mouse, but I still never really liked the dichotomy, and it just got worse as the programming challenges quickly progressed from pre-programmed route to line-following robot. The previous curriculum had each 2 hour session introducing a more difficult programming challenge, but that quickly gets too difficult for kids who have never looked at a program before.

With all of that in mind, I set out to come up with a project that all the kids could start with a fresh slate — a build with programming challenges that none of them had encountered yet. That’s where the robot face pictured above comes in.

https://www.youtube.com/watch?v=X8g4dJPTvKo

I decided being able to turn a motor on to make a mouth open and close + being able to twist some eyebrows up and down would allow for a lot of fun with really basic programs. I take a cue from Rachel Hellenga’s teaching style in thinking that it’s much better for everyone to use the basics in a lot of different ways instead of moving onto very specific uses of more-difficult-to-absorb knowledge.

The best part about building the robot face (A Lego CAD model to help you build your own is here RobotFaceV2.lxf, the software to view it is available from Lego here) is that it allows modular construction where groups can decide how to delegate the parts: advanced kids can pick the challenge of assembling the more complex linkages of the mouth and beginners can have fun picking out different parts to make eyeballs (they certainly don’t have to look like the example robot — so they get to be a little creative here.) If I remember right, some components were even challenging for the adult helpers, and having adults and kids work together on a problem that’s difficult for both of them is my favorite 😉

So our first lesson was simply getting things moving, turning motors on, changing the speed of the motor, learning that you can go 1 revolution at one speed, and the next revolution at another speed, and build patterns of these motor blocks to create patterns in the robot’s movement. Without much prompting there was much puppeting and kids coming up with things to say to accompany the robots miming (typical “I AM A ROBOT” stuff, but still lots of fun.) Also it’s lots of fun to make the robot’s eyebrows wiggle.

Before I had an idea of what the next activity would be, programmer-extraordinaire Caden had set up a couple of buttons to control the robot’s movements live and used his remote control face to tell a knock knock joke. After that it was pretty obvious what we could spend a few weeks on. The next session, each group was prompted to keep their laptops closed, no programming yet! One by one, tell your best knock knock joke to your group. Then decide amongst yourselves who’s got the funniest knock knock joke and in a few minutes we’ll ask each group to tell their joke to the class.

This was my favorite robotics class ever because it got started with everyone laughing at each others jokes. Kids get some ‘public speaking’ practice with each other and have to make a decision as a group, which is a good thing to get practice with, too.

Once each group settled on a joke, they were instructed to program their robots to tell the joke as puppets. They were encouraged to try different strategies, whatever it took to get their robots to talk to one another. This facilitated the different skill levels by allowing different solutions to the same problem. Some students got to try out simply lining up ‘move’ blocks with different speeds, some students got to try using timers and events, some students knew how to use conditionals to add those remote-control buttons, but everyone got to try what they knew how to do and share the result with their teammembers: because even the simplest program solved the problem of making the robot talk, there was no sense of being left behind or being slowed down. Students got to see how other teams were working and asked what their programs did (and getting one student to explain their program out loud to another is really great to see.)

When it came to testing the programs, reciting knock knock jokes along to your miming robot face was a lot more fun to fail than having your line-following robot fail to follow a line. For some reason, when trying to line up the pace of your speech with the pace of the robot’s mouth movements, it’s really funny for the robot to stop moving in the middle of a word, or to keep moving after you’re done talking — I guess because the robot flubbed a joke? Something about it had the kids laughing and I had never been in a programming class where debugging what went wrong was that funny.

Trying out lots of different ways to do the same thing also meant everyone was ready to have a crack at presenting their project at the end of the day — it was pretty much impossible to fall behind, you could just tell your joke using whatever program you had working. Getting a camera and a clapperboard (“Action! *CLAP*”) is always fun for kids, and we recorded a few takes of each groups knock knock joke.



The second iteration of the robot face involves a motor for each eyebrow and a piston mechanism that results in different eyebrow movements depending on which direction the motor is turned (the force of the motor will push or pull on the end of the eyebrow and change what direction it’s pointed.)

I finished designing this face pretty much the day before I wanted to use it, so unfortunately instead of creating build instructions I just built 4 copies. But that meant that the session after knock knock jokes could jump right into giving the robots emotions.

Before programming the robot to express emotions, we had to learn what different emotions looked like. So the activity went like this:

  • Sit everyone at a long table so each person has someone sitting directly across from them. It doesn’t matter if the kids are paired with adults or other kids.
  • Give everyone on one side of the table paper and markers.
  • Instruct everyone on the opposite side of the table to pick an emotion and express it on their face.
  • Instruct the drawing side to draw the expression with the fewest lines possible: just get the shape of the eyebrows and mouth first.
  • Ask the drawing side to guess what emotion the face-maker is trying to express. Just happy, sad, surprised, angry, that kind of thing.
  • Have everyone check their answers with the person across from them and then switch sides.

Don’t make a big deal out of it if someone can’t pick a face to make or can’t make a guess. The important thing for the kids is to spend some time focusing on an expression and practice identifying what it means. This is intuitive to some people, but lots of people have to learn what emotions facial expressions correspond to. At the end of the activity, students will have compiled a list of drawings and corresponding emotions.

The programming activity now becomes moving each eyebrow motor into position to match a drawing. This can also be a lesson in how 60 degrees compares to 180 degrees and so on, because the eyebrows only need to be moved a few degrees to change expression.

This expressive robot face now allows for a storytelling activity: think about what emotions you can get your robot to express, and write a story for the robot to tell (from his/her own perspective, i.e., come up with an identity and backstory for your robot) that would incorporate an emotion. Extra points if the robot’s emotion changes in the story and has its eyebrows move accordingly.

This allowed some of the students who were timid about building mechanisms or writing programs to express their creativity: the quietest kids had some really surprising stories for their robots to tell, and it was really exciting to see each kid decide what role they wanted to take in the project: changing the robot physically (some kids spent their time building props out of LEGO to go with their story — or adding red lights to the face to be laser cannons on their bad guy robot), writing code to control the face, or writing a script and drawing backdrops for the story.

It was a pretty long process to write and produce a story told by robots, so we didn’t get to the point of recording finished projects, but I have high hopes for this format of robotics education in the future.