Work from Home, Educate from Home - Paul Mumma, CEO of Cerego

In this podcast, Paul Mumma, CEO of Cerego, reveals how employees from is company quickly settled into working from home while the kids learned from home. It's been such a comfortable switch we wonder if they'll ever go back. You will also learn how the science of learning is meeting technology. These ideas will very well revolutionize the way we learn in the future as well as provide you with some great tips for today.

Paul Mumma, COO of Cerego, has been changing the world of remote education for years. When Silicon Valley was headed into a shelter in a place he quickly transitioned Cerego into a remote workforce. Being at the forefront of educational technology the company was able to quickly adapt and help its employees create new, efficient, and satisfying work environments.

Several important subjects were covered in this podcast including:

• Tips for remote workforces
• Working at Home With Children
• The unseen advantages of working from home.
• How the cognitive sciences are being combined with AI to create new and highly efficient ways of learning

Paul and Cerego are changing the face of remote learning.
If you are a parent, a CEO, or have transitioned to working from home there are lots to learn in this podcast.

Where to Find The Podcast

 

Subscribe on Itunes

 

Subscribe on Google Podcasts

 

Subscribe on Spotify

 

Transcript 

Paul Mumma:   Sometimes the reason someone is struggling is for something unrelated to the material. It's something personal going on in their life. It's an undiagnosed learning disability or some other issue that they need to work around.

And in those cases, there may be times when they would get a lot of benefit from personal attention if that teacher, mentor, coach, the manager knew that they were having trouble. And often that person doesn't know. 

Liz Weaver:   Hi, this is Liz Weaver and you are listening to the Learning Success Podcast, an information-packed podcast with the latest news, information, and tips to help you overcome a learning difficulty.

For anyone suffering from a reading difficulty, writing difficulty, a math difficulty, a focus problem, dyslexia, dyscalculia, dysgraphia, or ADHD, this is the place for you. The Learning Success Podcast is brought to you by learningsuccesssystem.com.

Phil Weaver:   Hello, and welcome to the Learning Success Podcast, where we learn to embrace your child's brilliance and unleash their full potential. Today we have Paul Mumma from Cerego. Cerego is an adaptive learning platform, which uses AI and cognitive science to improve remote learning outcomes.

Paul is going to talk to us about Cerego, how Cerego has transitioned to remote working. And we'll also talk about remote learning. Since Paul has access to a great deal of data and research on the subject, I think this is something that everyone can learn from so that we can all find the silver lining in this situation. Welcome, everybody.

Paul Mumma:   Thanks for having us, Phil.

Phil Weaver:   Thank you for coming along. First off, we'd like to talk about your company's transition to remote learning or to remote working, and then also about what you do with remote learning. I think these are really important subjects for a lot of people right now.

Can you tell us just to start, I know you're being a technology company, where were you before you started? What part of your staff was already remote working and what was in office?

Paul Mumma:   Absolutely. Cerego is based in San Francisco and the majority of our team is co-located. We work in an office right downtown in San Francisco.

We've generally had a flexible enough policy where on some days some folks are working from home, but primarily the San Francisco Bay Area team would be in an office together every day. About a third of the company though has been distributed from the beginning.

And they're in a number of states across the U.S. I would say we were partially familiar with some of the unique challenges or opportunities with working remote, but in mid-March, we accelerated that and suddenly had to learn how to get everyone on the same page, working remotely.

Paul Mumma:   And the kind of interesting thing which you alluded to is, as a company we've been helping our customers bring their learning and training online for years, and we'd been that agent of change, helping them make that adjustment.

And we laughed realizing that in some ways we had to help ourselves with that comparably big change, moving our whole company online. And that gave us, I think some new empathy for what it means to change the way you work or the way you learn and doing that stuff online for sure.

Phil Weaver:   Right. I bet that there was huge empathy. We'll jump into what you learn there. That's a great point. When did you actually start the transition? Did you see it coming or?

Paul Mumma:   I think the Bay Area, the reputation seems to be accurate, which was, we all seem to know something big was coming a little bit on the early side. I think it was by March 6th or 7th, we were getting ready to go remote.

We began doing some dry runs and practice events as a team, thinking about how we'd run our meetings all from our own homes. And it was by the middle of that second week of March, the 13th or so that we were fully remote.

Phil Weaver:   So, you were fully remote in under two weeks, 10 days or so?

Paul Mumma:   That's right.

Phil Weaver:   Was there any downtime?

Paul Mumma:   Yeah, there were a couple of days of adjustment where some folks began to come in a little bit less, and then we moved to the complete remote operation a few days later, but the business kept on going continuously, nothing soft. Yeah, [inaudible 00:04:02].

Phil Weaver:   Great. How much contact do you have with your customers? Is there a lot of customer service in that going day or is it mostly all the tech that does the work?

Paul Mumma:   There's a lot of customer success and support. Although to be honest, we've done a lot of that through video calls and phone calls for a while, even before this crisis.

Phil Weaver:   I see.

Paul Mumma:   Luckily staying in touch with our customers barely skipped a beat. In fact, we've tried out some new things in the last couple of weeks, recurring office hours and open webinars and new ways of getting folks engaged, which have worked really well. Luckily we didn't have a lot of in-person engagement that we needed to rely on beforehand.

Phil Weaver:   Okay, good. In the transition, what were the biggest roadblocks to getting transitioned?

Paul Mumma:   For us as a company, because the majority of our team is used to working in the same place, that bumping into one another, sharing ideas, quickly grabbing someone to spend a few minutes in a room talking about a problem or an idea, that was the biggest gap we noticed right away.

You can have planned meetings and you can join by video, and those can happen just about as well as they would in real life, but it's the spontaneous stuff, looking across the floor and seeing who else can help with a problem that is lacking. That was the first, and I would say the biggest challenge.

Paul Mumma:   We've been creative in trying to get beyond that. And so in the last four or five weeks, we've tried some things, including we have a video chat channel that's open all the time.

We call it the Water Cooler and it allows people if they have a few minutes free or they're just working on something quietly, but they want to say hi to some other folks, they can jump into that video channel and just see who's there.

And that's a crude way to approximate the good serendipity of walking in and seeing someone in the kitchen or in the office, but it works surprisingly well. And that's been one approach we've taken.

And we also use tools like Slack to chat format, keep up with each other. And there's a lot higher volume of, call it non-mission, critical stuff people are talking about right now just to keep conversations going and stay in front of one another.

Paul Mumma:   And that's helped everyone's feelings, I think pretty well connected. We ran a survey to the team at the end of last week, just checking in on this specific question of how does it feel to be all on our own versus how it felt when most of us were in the same office.

On balanced, the team has said, “It feels a lot less bad than we feared it would. And there are some trade-offs, but we're feeling pretty well connected and things are going pretty well.”

Phil Weaver:   How much usage do you get out of the Water Cooler? And is that like casual conversations or is it just socialization mainly or this-

Paul Mumma:   It's the whole mix. I'll say we get waves of usage. And one thing we've learned is everyone knows there's an always open Water Cooler channel and that's really helpful, but it's not a muscle we're familiar with, just jumping into a video chat just because.

Something we found can help is, sometimes people will take it upon themselves just to remind the group in Slack, which is our chat tool, “Hey, I'm going to be in the Water Cooler channel for a few minutes while I eat lunch.” Or, “Hey, I'm going to be in there as I start the day.”

And then you might see a good handful of people jumped in as well. And that is a little bit less spontaneous, but it still has the same effect. The fun thing is, to your question, is it social or other stuff, everything gets discussed in there.

Paul Mumma:   I've been in that channel and watched our engineers talk to a customer success person about a particular bug that we're hearing about from a customer and they're troubleshooting.

I've been in that room, the video chat room, and I've been showing a presentation I was working on for our board to some other folks just to get their feedback. Just kind of an informal audience.

We have one member of the team who is a fantastic guitarist, and he has done a couple of five-minute concerts where he'll jump in and just relax for a few minutes, but will tell folks he's playing the guitar in the chat and people will come in and listen. It's the whole range.

Phil Weaver:   That's fantastic. What tech are you using for that?

Paul Mumma:   We've been using Zoom.

Phil Weaver:   Okay. You just leave the meeting open all the time and-

Paul Mumma:   Yeah, exactly. And we have the URL for that meeting in a place where people can see it and then we just periodically remind them to jump in and folks are in there.

Phil Weaver:   That's a great idea. I'm going to use that one. Do people simulate like the hallway meetings, are they more apt to just fire up a quick video call or people not used to that yet?

I know the formal ones are always there. We're going to have a webinar, this and then, but I mean just more like, “Hey, I want to talk for two seconds.” Does that happen?

Paul Mumma:   This is an interesting question. I'll give you hopefully a pretty quick answer. And then I'll tell you something that reminds me of some of the challenges of learning online, that's related to it.

Your question, are people jumping into a quick video chat, or what's going on when there's something to discuss or resolve? Our first instinct was to do that. Someone has something they need to troubleshoot, suggest a quick call, video call, get into it with a few people, try to resolve it, maybe keep it quick and then move on.

What we found after a couple of weeks is, sometimes the whole rigmarole of setting up a video chat or a meeting might not be the best way to sort out a problem.

Paul Mumma:   And sometimes just having a simple conversation or a document someone has written up with the problem and letting people come in and add comments or questions in a written form on their own time is sometimes a more efficient way to solve a problem unless it's a burning issue that needs to be solved in five minutes.

And what that's been reminding us is, sometimes we have some instincts about how we think we're supposed to work. And those instincts are built on the world of being in-person and being in a room. Where the most efficient way is probably just to grab someone and chat for a minute by their desk.

Once you're remote and you're more relying on technology, there might be better instincts that we can cultivate.

Paul Mumma:   And one of them may be, it's useful to write down the issues, you're thinking about more frequently.

So, you can quickly share that doc with someone and let them weigh in when they have five minutes and in an asynchronous way, collaborate on problems during the day, or even throughout the day and the night. That looks a bit different than the way you'd work if you were thinking mostly about eight or nine-hour days in a physical office.

The kind of interesting thing for us as a company is we're thinking more and more, that's not so different from the way we encourage our customers to think about learning with technology. They got a lot of instincts that may be good instincts in a world where the only way to learn is in a classroom with people or a house with your kids.

Paul Mumma:   Once you're thinking about how technology can come into the mix, there may be different instincts about learning a bit more on your own or in different schedules and bite-sized pieces that actually work really well and better once you have the technology to make it happen. It's kind of a paradigm shift for work and for learning.

Phil Weaver:   I bet. I bet. My guess is that people are going to obviously think more about the question and maybe even answer a lot of their own questions just simply by writing it down. Is that-

Paul Mumma:   Yeah.

Phil Weaver:   Is that something-

Paul Mumma:   There's a bit of that, for sure. They call it the rubber duck test. If you're a software engineer and you're having trouble with some code and you can't figure out why sometimes they'll recommend you talk to your rubber duck.

It's not even a live person, but just the act of talking to someone. You work through the process, you might find an answer on your own. I think something is similar when people are writing down or documenting their issues. Sometimes they even solve their own problems.

Phil Weaver:   Yeah. I find that myself with mentors that I have. I typically will have a question, but then as soon as I think about asking the mentor, I'm like, “Well, I know what he's going to say,” and answer. It's the same because I know their voice.

Paul Mumma:   Right.

Phil Weaver:   Yeah, probably it makes things far more efficient, I would imagine. Before, you mentioned that we might hear some screaming children in the background.

Paul Mumma:
Yeah.

Phil Weaver:   Let's talk about that. About when parents with younger children are working at home, how is that working for them? Are they keeping the children doing what they need to be doing and them during the class [inaudible 00:12:19]?

Paul Mumma:   Sure. Yeah. I can speak to my individual experience since I have three little kids. I'll talk a little bit about how some other folks at Cerego with kids have worked. And that's probably my relevant field of experience. I have three kids, all of whom are under the age of four.

My kids are really small. The eldest almost four years old. He's in that early learning mode, talk great, watch stuff, think about stuff, learn stuff. The younger two are a bit younger. They just got to be watched.

So, I have a busy young kid's house. Some other folks in Cerego have kids that range in age from similar toddler age, through grade school and some in that middle school bucket. I think as a group of parents, we found some challenges in common and some things are a bit different depending on the age of our kids.

Paul Mumma:   The big, common challenge is, in a world where those kids are not going to school or not going somewhere for care, it's our job to give them good attention and formation, whatever that's going to be while we try to do our jobs.

That's never easy. I think what we found generally as a group is, it's helpful to be transparent with our colleagues, that we can't work in the same long unbroken blocks at the time that we otherwise would because of that. That actually goes back to my comment around the best way to work when you're remote in the first place.

And as it turns out, maybe it's not just because you have kids working a little bit less synchronously, but more asynchronous and the blocks that work for your schedule can be a pretty effective way for a team to operate.

Paul Mumma:   And certainly for us parents that might mean, like in my case, a couple hours in the morning, I may be watching my kids while my wife is doing work and then we'll flip in the afternoon. And that means I'm present in different ways for the team at different points of the day. The other thing that I-

Phil Weaver:q   As well as present for your kids, right?

Paul Mumma:   Exactly. Being present for the kids, that's been an interesting thing to compare notes on, depending on the age of our kids. The reality is the world is in a pretty unusual place right now, but the younger your kids are, the less aware they are of what's going on outside.

And the more they're aware of whatever's going on at home. I feel lucky that my kids are young enough to barely even know anything is wrong with the world. They're happy they're getting a lot of good time with mom and dad.

Paul Mumma:   And that gives me new energy to try to use the time effectively and help them learn new things or practice new things. Some of my other colleagues have kids that are a bit older, they're a bit more fearful or fretful about what's going on in the world around them.

And that gives my colleagues extra reason and energy to try to spend a good time to be of comfort and source of stability for those kids at this time. It's really, it depends on the age of the kid, but this is a moment for an opportunity for parents to do things a bit differently if they can.

Phil Weaver:   Yeah. Yes. Two important points there. You've mentioned that asynchronous a few times. I know there are old techniques like the Pomodoro Technique that increase the efficiency of working. This may be a big opportunity.

My wife and I actually created a product in ‘07, which was called Break Pal. And the idea was to, it was just that it was a timer, but then it presented you with desk exercises. They were based on yoga and Kung Fu and Tai Chi. Every 20 minutes it pops up, you do a three-minute exercise.

Phil Weaver:   Again, the idea was that you have this efficiency, you start your day off, you got plenty of coffee and you're going along, but your efficiency drops off all the day. And you mentally try to fight that all day long when really just working in the way you're saying, asynchronously is, probably a really fantastic solution for that.

Paul Mumma:   Absolutely. It's really interesting. I think one of the ways that that phenomenon is described is psychologists will talk about ego depletion, which as I understand, essentially means as the day progresses the amount of mental energy you might have to make decisions and get things done tends to deplete for all the normal reasons. You kind of get tired out, stretched out your day.

Phil Weaver:   Especially in willpower, right?

Paul Mumma:   Exactly, willpower especially. And so find ways to counteract that natural ego depletion is important for productivity and also for learning and other things.

And so it's funny actually, the reason this is less random then it sounds is a year or so ago, a lab of psychologists at the University of Toronto, where we're looking at the evidence for or against these theories of ear depletion.

And ironically, one of the best data sources they found, try to run some experiments was Cerego data because Cerego as a tool is all about learning in relatively short bursts, across all parts of the day. And this lab ended up looking at some of the data we collect around how well people are able to learn throughout the day to assess how much ego depletion affects things like how well you can learn.

Paul Mumma:   Not so surprisingly the result I think, is what we'd expect, people tend to have more energy either at the start of their day or towards the very end of their day when other concerns like their kids are off the bed and out of the way.

And that's when you can learn most productively. The secret is, in all things learning, short bursts that are focused is usually better than a multi-hour block. That feels the same for me from how I get work done when I'm at home and trying to take care of my kids as well. Short bursts have broken up, that's the recipe.

Phil Weaver:   Right. This is fitting with, so I actually teach. My passion is a very rare form of Kung Fu. It's a very difficult form of Kung Fu. People have this conception that, from the movies or whatever that Kung Fu martial artists train for hours and hours and hours on end. And really my success in it was, I would train in literally five-second increments.

Paul Mumma:   Awesome.

Phil Weaver:   Five seconds, constantly all day long to have these little thoughts. And that was where the concept of Break Pal that we did came up because we realized that learning was so fast by doing that and it didn't require discipline. It required habit.

Paul Mumma:   Right.

Phil Weaver:   Yeah, that absolutely coincides. It's really interesting. It's good. Really revolutionized learning, I see.

Paul Mumma:   Yeah.

Phil Weaver:   Going back to the parents and kids, do you have any advice for it? I see a lot of things going on, people mentioning on Facebook about their kids distracting them from work and all that. I can kind of visualize that some of these interactions may not be very positive.

Do you have any advice for parents on how to make sure that that is psychologically advantageous for their kids rather than making the kids feel like, “My dad's paying attention to work when he should … Why am I not more important?” That sort of thing.

Paul Mumma:   Yeah. As you can imagine, I have a lot of sympathy for any of these parents, myself included who feel or even worry that they're not being positively attentive enough to their kid. It's such a struggle.

And probably at the end of every day, every parent thinks that “Did I not do well enough on this,” each day. Maybe I'm too forgiving. My first piece of advice, which I try to remind myself of is, you have to forgive yourself when you don't think you've done a good enough job the last time you were engaging with your kids, you can try again.

Try again. That's kind of like everything in life, you want to learn, you want to improve, you want to build a habit, forgive yourself for not doing a great job, do it better the next time, the next time. Practically, I'm lucky my wife and I, luckily I have a wife and we're both at home and we're both working, but we're both trying to watch these kids.

Paul Mumma:   One of our coping mechanisms is, we try to let someone be all on while someone else can be all off. And that means even if it's only for a shorter window, maybe it's a one-hour block and not eight-hour block.

I might know I have a whole hour and I don't need to worry. If I hear kids who are fighting, rusty, I'm not going to run out and try to help. I know my wife's going to do it and vice versa. That trade, letting someone turn off for a little while, if you have the luxury of someone else who can be part of the mix, that's huge.

And then the other part is, I've tried to help remind the team members at Cerego that have kids, that doesn't expect us all to be working in the same way right now and reminding yourself that you're not meant to be fully on.

Paul Mumma:   And just as available to the team means if you're not going to be able to be fully on with the team, go be fully on with your kids and try to remind yourself that sure, you may be worrying about work in your back of your mind, but try to just focus on the kids when you can.

And then when you're back and you got a few minutes or a bit longer to do work, focus on the work. And again, it's a mindset shift because most of us are not used to a world where we're trying to go back and forth between productive work and being really good productive, educating parents.

Reminding ourselves to do that and don't do both. Just don't do both. If you're with the kids, focus on them. When you're doing work, even with the short burst, that's when you focus on work. It sounds easy, but it's not.

Phil Weaver:   No. That falls into Cialdini's work, where he talks about what we focus on is what our brain assumes is important.

Paul Mumma:   True. And it's all about habit-building like you said. As you do something, if you build a habit that shows your team, there may be 15-minute blocks, kind of randomly where it's hard to reach you and you don't respond quickly in Slack.

And that's because you're focusing on your kids at the right time, it becomes a habit. The team will reform its activities around that new habit and things will evolve and it's going to work out.

Phil Weaver:   I would imagine just having the right mental attitude for that, of accepting that as the way it is rather than, “Oh, this is a problem,” that it could really make the shift.

Paul Mumma:
Yes. Yeah, I think so.

Phil Weaver:   What other policies and practices have you done at Cerego to help parents?

Paul Mumma:   Well, one thing that we're going to be continuing to do is we haven't changed some of our policies around things like new parent leave. And we have luckily some employees who are imminently or in the next few weeks expecting to have a first child or have another child.

And our view is, just because we're theoretically all at home, to begin with, doesn't make it any less helpful for a new parent to go take time off and really focus on being a new parent. That's something that, again, it's not a change in our policy, but the world has changed a lot and we've decided it's important not to change that policy.

We're still encouraging that kind of leave. We've always been a bit flexible as to how people take that leave. You take a couple of weeks all at once, you shift your schedule for a longer number of weeks, that kind of flexibility is still part of the plan. We're encouraging the newer parents to do that for however long, this new world kind is the case around us.

Phil Weaver:   Sure, sure. What other advantages that we haven't talked about for the employees, do you think about this new work environment and as well as for the company, what's better?

Paul Mumma:   Yeah. I think in general where things are better, whether or not someone's a parent, whether or not they have kids, what can be better is the flexibility to use bits of time throughout the day in the most way.

And this is about whether you're working productively by not trying to work for eight hours in a row. It's also things like, if someone has a good habit of exercising or trying to eat healthily, the constraint of having to do all of that around your commute time or time being in an office is freed up.

Now, if you'd like to eat healthily and that might mean you'd rather spend 15 minutes preparing lunch in your kitchen rather than reheating something in the office, you can do that. And it's not really going to change your productivity for the day, but it's going to make you feel happier and healthier.

Paul Mumma:   If you want to go for a run, you can do it when it's bright and sunny and warm at 2:00 PM. And not at 6:45 when you get home and it's getting dark. That's healthy, that's helpful.

And the same certainly goes for kids. I've been lucky that with commuting distance and time, I'd always see my kids in the morning and at the end of the day, but now I see them in different parts of the day.

We may have different energy and when they are going to be doing different things, and that gives me a chance to do some things that otherwise, I couldn't do during the workweek. Reminding ourselves of whether it's related to kids or not, what we can do now that we couldn't do before with honestly, a minimal impact on the amount of work we do in the day, that can be really healthy.

Phil Weaver:   Yeah. I think health is a huge one. That's actually my lifestyle because I do work remotely. I do the interval I will garden throughout the day, I will cook extremely healthy meals. Personally, I've been pushing people to, “This is great. This is a benefit. You can change your lifestyle for…”

Paul Mumma:   You guys are the pros. You guys have seen the benefits for a while now. And the rest of us are just catching up in an accelerated fashion as the world forced us to. I think there's a lot of good reasons to work in this way to be [inaudible 00:25:27].

Phil Weaver:   Right. And what about the company?

Paul Mumma:   Yeah. For the company, our core work, which is building software, doesn't change. It doesn't change whether we're remote. Hold on one second.

Liz Weaver:   Do you have a smart child who is struggling in school? Are you feeling overwhelmed? Do you feel like the struggle is holding your child back from their true potential? Maybe the anxiety and worry over your child's future just beat you down every day.

You don't have to live that way. Learn how to stop a learning disability from becoming a lifetime disability. A child with a learning disability is stressful for the child and the parent. The disability may be eroding their confidence and shattering their self-esteem.

Other people may perceive your child as unintelligent and antisocial. If not addressed and fixed early, the child may develop permanent challenges later in life when looking for a good job or meeting a potential spouse.

Our current school system does not know how to properly help our children, but at Learning Success, we do. We've created a system you can easily do at home with your child.

Liz Weaver:   And with just 15 minutes per day after school with your child, you can save them from a life of struggle and heartbreak. Learn how to unleash your child's potential and embrace their true intelligence. As a special gift for being a loyal podcast listener, we're going to give you a free trial of the Learning Success System.

Try it out absolutely free for 15 days. If it is not the perfect fit to help your child succeed in school and in life, just cancel before the trial ends and pay nothing. You even get to keep the free bonuses. Go to www.learningsuccesssystem.com/podcast to get your free trial now.

Liz Weaver:   You'll be so happy you did once you see the great grades your child is capable of getting. Imagine being so proud of your child when they bring home a great report card and hand it over with a beaming smile.

Get your free trial now at www.learningsuccesssystem.com/podcast. You've got nothing to lose except the stress and anxiety that is holding you and your child down. I'll see you there.

Paul Mumma:   Yeah. For us, how's it going for the company, or what are some advantages? The core task of building software and delivering it is just as doable when you're all remote. That's great. That means what we try to do is build software that helps to learn, we can do that with no change.

n terms of connecting with customers, helping them think about the best way to use Cerego in their classroom or in their workplace or even on their own, we're pretty much able to do that without change as well. Good video calls, webinars, office hours, we can do all of that.

That's all great. If anything, what else has been good is this has pushed our team to be creative in thinking about the most efficient way to help the world know what we have to offer and decide if it's the right fit.

Paul Mumma:   We probably made more experiments and more improvements and producing some resources and creating webinars and setting up office hours to help a lot of people quickly understand Cerego.

We'd probably done more of that in the last five weeks than we would have because we realized we had no choice. And so that's been healthy. It's been a good kind of kick in the pants and get us to think differently and try new things and some have worked out.

Phil Weaver:   What do you see as disadvantages that would be both solvable disadvantages and things that may not be so solvable?

Paul Mumma:   Yeah, it's a good point. Where Cerego has tried to be unique as a company is really in taking proven principles of cognitive science or learning science and combine them with all the best technological advances that we can, to make a really flexible experience that will work for almost anything you're trying to learn.

And that combination of learning science and technical opportunity does require a fair amount of R&D and thinking and experimenting and testing and trying. And it's hard to approximate a great brainstorming session, sitting in a room, trying out ideas. It's hard to do that as effectively on the Zoom video.

You can have plenty of other meetings that feel like they're almost as effective, but really good brainstorming for whatever reason it's harder to do than we felt we've been able to do it in-person.

Phil Weaver:   Does that mean we'll be less innovative?

Paul Mumma:   Probably not overall, but it may feel like it takes more work and takes just a bit more time and energy to try to work through ideas and actually make stuff happen.

And that's so important to us as a company, is being ahead of the curve and providing new opportunities. It's important for us to get it right, but it doesn't feel easy.

Phil Weaver:   Any ideas on why brainstorming sessions or not?

Paul Mumma:   I would bring it back to instinct and habit. If we had started our life as a company that only had employees distributed around the world, we probably would have built muscles all about brainstorming in a remote way, that would feel good.

But when you have the luxury of being in the same room, there's probably something around even body language and how enthusiastic is the person to the idea you're trying to think throughout loud. And then if they're enthusiastic, you talk about it more. I think just it works differently in the room. I don't know why.

Phil Weaver:   I was exactly going for body language. That's awesome. Yeah. We read so much and communicate, so much more through it.

Paul Mumma:   Yeah.

Phil Weaver:   Interesting. Yeah. To give you a bit of background. I've been teaching my form of Kung Fu online for six years now. And so obviously we have to read the body well and it's definitely fair.

I think so much of that is ingrained, is working at such a subconscious level and our subconscious is not picking it up in 2D. It's used for 3D. It's not programmed for it. Maybe we'll adapt as a species or not. I don't know.

Paul Mumma:   Oh yeah. Good on you for having tried the experiment for a number of years already of teaching something inherently physical at a distance and using video. I'm sure it's more possible than we fear it would be, but it's not easy in so many ways.

Phil Weaver:   It's not. Yea. One of the things that we've learned, and every time I tell somebody I teach Kung Fu online, they give me this weird look and say, “I didn't think that was possible.”

Paul Mumma:   What's the punchline? Yeah, exactly.

Phil Weaver:   Yeah. What's the punchline? Right. And part of that came from, one of our advantages was, is that the early lessons were on VHS. I'd watched many of them and they were absolutely a joke. And so people got this idea of it.

Well, a large part of the problem was not video, it was the storage amount. My beginning class alone has over 200 hours of video and that's just to get into a kindergarten class. We couldn't do it on VHS. We've successfully taught Kung Fu online for just in a handful of 10 students.

It's all we've done. We wanted to see if I could bring somebody through. I think if it can be done with that, a physical science, what couldn't be done with [inaudible 00:33:14]?

Paul Mumma:   Absolutely. That's very cool and creative. And so much of our work at Cerego's about how many things can you help people learn effectively using primarily technology?

Cerego's off and just one piece of learning rather than the whole experience, but we find those quite a lot. People can learn and practice digitally and that's great, opens up all kinds of new opportunities.

Phil Weaver:   It sure does. Do you see this changing or do you see this as something here to stay once everyone has the opportunity to change? How much of it'll stay?

Paul Mumma:   I think it's going to change some behavior for sure. I think we are, as a company individually, we see that there's plenty of silver linings. That as much as we like being together, we get along, we find some efficiencies, I think we're all finding particular benefits to having more flexibility.

I will be really surprised if we move back to an identical way of working to what we did before this. I bet we'll have minimally more flexibility. We might even build a much more by default working remote culture. I think that could happen to us.

Paul Mumma:\   And I think when we talk to our customers who are schools, companies, and organizations, a lot of them are still in the adjustment phase. There's a lot to work through, but I think some of them are seeing that there are some silver linings or some advantages as well. Or, just that their business is changing or that the number of students they have maybe changing. And I think things will be a bit different even as the world gets a little bit more back to normal.

Phil Weaver:   Nice. Well, it's a forced adoption.

Paul Mumma:   Yeah.

Phil Weaver:   My wife works in the mortgage industry and the particular company she works for, the founder had put tens of millions of dollars into technology and it was not being used at all. Less than 20% of what they were doing was through that.

And now it's bumped up to 80% and I see that company as a whole, just fortunately for them, they had it in place. And the only thing that didn't or that wasn't working is that it was new and people didn't want to want the change. And then they had to change.

Paul Mumma:   They had no choice. Right.

Phil Weaver:   They had no choice. I see the same thing in Kung Fu in that I've been teaching a handful of students, and then I've got all these black belts that just wouldn't look at what I was doing. They're like, “No.”

And now all of a sudden they're all in my group and loving it. It was just the forced adoption that changed so much. I see that happening in the schools. I've seen school principals and teachers talking about really creative ideas. Do you see other industries that forced adoption was ready for?

Paul Mumma:   Well, the industry I can speak the best to is industries that rely on tools for learning. And I'll tell you something that I could imagine changing more because of this, even in the area of education, high school, middle school, not just higher ed, where we do have some customers.

No, I'll tell you something that's always been an advantage of something, a tool like Cerego. If you use Cerego, which is all about getting ongoing practice and reinforcement of what you're learning and has the benefit of letting our system get a really good estimate of what you know and how well you know it, that experience has always been a pretty good alternative to a traditional onetime test.

Paul Mumma:   That in many ways, we have data that suggests it can be a better measure of what you actually know rather than the randomness of whether you score well on a test or not. That's true even before people were forced to learn in the distance.

But, something we're seeing is, some of our school customers who are fearful about whether using a traditional test is the right approach, given that students are on their own, theoretically could spend more time trying to cheat or get their way around an exam.

Some of those schools are realizing, “Maybe this is a great time to use experiences like Cerego and the data that comes out of them as an alternative. And that's going to be our measure of whether someone has learned enough to progress or to do X, Y, or Z, well in a class.”

Paul Mumma:   It could be that what's going to go on here is, teachers, schools will think differently about what's the right way to gauge, who knows what? What's the best set of tools or the best time and place to use assessments or not use assessments to make those decisions.

And if in the end, what they turn to are the best ways to let students really demonstrate what they do know and not penalize them for having one bad luck day, when they take a test, that'd be a really nice chain and a good outcome. I could see some things like that changing in behavior on the education side for sure.

Phil Weaver:   At this point, can you give us an overview? Cerego's getting more and more interesting as you speak here. Can you give us an overview of how it works?

Paul Mumma:   Sure. Yeah. We are a software platform that has three components. You can author, create a learning experience in our system. We deliver that learning experience through our mobile apps or through your browser on your computer.

And we have data and analytics for teachers, for managers or for the students themselves to see the systems estimate of what they know and how well they've learned it, where they need to spend their time.

And the goal of the system as a whole is to give the learner or the student, the most efficient and effective learning possible. By using these principles of learning science and cognitive science, we want to give them the highest value thing to do for just a couple of minutes at a time.

Whenever they log-in and use Cerego, they'll either learn something new or they'll practice it or review something they've already learned and is designed to make sure they learn everything they need to know as efficiently as possible.

Paul Mumma:   That's what the system does. And the beautiful thing about the way it's built is, it's very flexible. And so you can be helping people learn anatomy or biology, music theory, chemistry, history, any subject can be learned using Cerego.

And so our customers span a really wide range. They're high schools and colleges, they're companies and organizations that have to train their staff. Everything you can think of, a wide range of customers using Cerego.

Phil Weaver:   How is it deciding when to practice something or to learn something new, or as you said to review that?

Paul Mumma:   Two of the most important principles of cognitive science helped make that decision. And the two principles are one, something called distributed learning, which is this idea that if you actually want to remember what you learn, you should learn or review over time.

Don't cram the night before, but learn for days and days or weeks and weeks in little bite-size pieces. And the second principle is something that you might hear. It described as the testing effect or retrieval practice.

Which the idea is that if you want to again, learn, and use information, you have to practice it and actively engage with it and not just listen to a lecture or just read a page in a book. So, an exercise, a practice, a quiz is an effective way to do that.

Paul Mumma:   When you combine those two principles, learning spaced out over time and actively practicing what you learn, you get a recipe, which is what Cerego delivers, which suggests first, you need to learn something, to begin with.

And then over time you want you to be exposed to that information again, but in different ways. This time you're going to practice it. You're not just going to read it or listen to it again. And depending on how you do when you practice it, the system will have a good sense.

Did you remember it? Did you understand it or did you not? And whether you know it or don't know it, the system decides how quickly to show it to you again in yet a different form, a different quiz, a different exercise.

Practically it means for each person, we all begin learning the same body of material in the same way, but our schedules and what we'll spend our time doing for the next couple of days or weeks will start to look different based on how well each one of us picks up that information.

Phil Weaver:   How about long-term learning, how long does that information stay? Does the AI know how long it's going to stay or?

Paul Mumma:   Yeah, this is a really fascinating thing. The answer is, our AI, our algorithms have a really good estimate of how long you'll know or remember what you've learned. And the even cooler part is the way of calculating how long you'll know that information.

It's not a brand new idea. This is the kind of research that's been done for over 100 years. And some people describe it as the forgetting curve, the rate that you forget what you've learned. We talk about a learning curve, which is flipping that around and talking about what you need to do to learn and remember for the long-term.

And the fascinating thing is when you look at what it takes to remember something for months and months, or years and years, rather than just for hours or days, we're just talking about engaging with that material a couple of times for a couple of minutes.

Paul Mumma:   If you spread the engagement out over a couple sessions, rather than cramming it all at once, you don't need to spend a ton of time to make sure you'll remember that new information for a really, really long time, hopefully forever. And that's the secret behind why Cerego works because of all the research that's shown, this is how the human brain works.

Phil Weaver:   I don't know if this is relative or not, but how about incorporating the kinesthetic senses and emotions on that? I noticed when you speak that you illustrate everything with your hands and you express emotion and we know that these things are tied to learning. Is there any element of that going on?

Paul Mumma:   I'm not an expert on this corner of the research. I believe some of those theories around the learning styles, there's probably some debate in the academic community, but-

Phil Weaver:   In learning styles there is, but incorporating them as a unit seems to [crosstalk 00:43:08].

Paul Mumma:   Yeah. We are always thinking about what else we can do to deliver that comprehensive learning experience. Right now, since we're all software-based, there's not a whole lot that engages physically with our learners, but it would be cool if we can.

And we've thought about things, including virtual reality and augmented reality and new ways you could deliver part of the experience to help people engage and do things in the physical world around them.

And that is really important, especially when you're learning things that deal with doing something physical in the real world. I think there's a lot of opportunities here. Again, that goes back to why R&D is so important and is important to get right even when we're all remote.

Phil Weaver:   Right. Well, I had read on your website that you were in a classicist and it talked about the ancient Greek and Latin. Are you familiar with the Memory Palace?

Paul Mumma:   I am. Yeah.

Phil Weaver:   Yeah. That's, that's the incorporation of visual-spatial learning and they were able to do these long orations using that practice.

Paul Mumma:   Absolutely.

Phil Weaver:   I think there are some interesting things there. I hope to see that in the future. What are some of the best practices gleaned from the data and for learning?

Paul Mumma:   Yeah. A lot of the data that we capture helps reinforce how true the findings of cognitive scientists have been. In other words, I talked a little bit earlier about the research that scientists at the University of Toronto were doing on ego depletion.

Looking at our data, what they find is absolutely learning in short bursts is the most effective, the most efficient way to learn. You learn the most and you retain it the longest when you learn in short, small bursts, rather than real long dragged out sessions.

That's definitely something that comes from the data. The other things we've learned are around what is it mean for people to learn quickly versus learn effectively? And we have some metrics that come out of the Cerego experience.

That shows that just because someone may take a little bit longer to learn in the first place, by no means suggests that they haven't learned it just as well or that they won't remember it just as long as someone who may learn something a little bit more quickly.

Paul Mumma:   That idea of your sort of agility in learning does not have to be correlated with the amount of mastery of the material at all. And the reality is that just speaks to how much prior knowledge one person has versus another or another sort of style as to how they learn.

Rounding out the picture of what it means to learn quick versus learn well, versus remember for the long-term is one of the cool byproducts of this sort of data we get.

Phil Weaver:   I wonder if people that may take longer to learn a subject may in a lot of cases learn it deeper. Do you think that they?

Paul Mumma:   No. I think the answer is that it can be. If you take two hypothetical people, some people may have basically no prior knowledge that is related to what they're learning.

And so in some ways, they have no choice, but to take longer to build that foundational level and work their way up with them trying to learn. Someone else may have a bit of prior knowledge, may have by temperament or instinct, a bit more of a diligent mindset.

Both of those people will effectively take longer to learn. And I suspect if we looked at the data, both would walk out, learning something well, and remembering it for the long-term. Both would outperform.

Paul Mumma:   In other words, someone who might have just a bit of prior knowledge, they felt a little bit overconfident, didn't have a whole lot of diligent energy, and just quickly jumped in and then quickly ran away from the material.

That person's going to do the least well, no matter how smart they are. I think that is to your point, it could be evidence that a little bit slower and steady, can often get the best result for whatever reason that is [inaudible 00:47:11].

Phil Weaver:   Yeah. We see that in Kung Fu that you have people that have like natural athletic ability, and then they'll assume they're getting something. We call them dancers because they're just dancing through the material.

They're not really understanding the visualizations that are going on. They're not feeling the muscles working and all that. Those people, at least in our art, will hit a point where it becomes very difficult for them to progress because of not to try diving in deep. And it's because the learning was actually too easy for them through this in the beginning.

Paul Mumma:   Right. And that's so much of what Cerego tries to do by personalizing the experiences, quickly work out just automatically what you know well and what you don't know.

As well as, give you the right mix of things that challenge you enough to keep you engaged, but also build new knowledge and not waste your time with things that you already know or don't need to spend as much time on and getting that mix right.

It's much easier when a good coach or a good teacher tells you how to focus or in our case, some artificial intelligence can help quickly guide you. All of that leads to a more effective experience.

Phil Weaver:   Yeah, it is that basing. We'll use like a 10% fail rate. We're looking for people to be 90% successful to give them just a little bit of challenge, but not too big. Is that kind of the idea?

Paul Mumma:   Yeah. It's funny. Our scientists try to calibrate our system right in that same zone. 85% to 90% of what you do in Cerego ideally should be stuff you're going to get right. And that's part of the motivation.

But also it's being in that flow of building knowledge by learning it just at the moment where you might otherwise have forgotten, but ideally just before, and if you calibrate that right, it's maximally rewarding and most effective.

Phil Weaver:   Okay. At that point, when you might have just about forgotten, is that just statistically it knows?

Paul Mumma:   Yeah. You run a basic calculation for what that forgetting curve suggests the time to forget is going to be, and you update that calculation for each person. Each time they're using the system, you get more and more feedback about how well for a given bit of material that they're learning, how much they might over-perform or underperform the basic rate of forgetting.

And that's running constantly in the background calculation that becomes tailored to each person. And it's a really good guide. It's not 100% perfect, but it's going to have a really good guess as to how long it'll take for you to forget something. And then the system schedules it enough in advance that ideally, you haven't forgotten much by the time you review.

Phil Weaver:   Okay. It's not a standard curve. It's personalized?

Paul Mumma:   It's individualized. Yeah, exactly.

Phil Weaver:   Nice. Interesting. You talked about the short bursts of learning. Is there a place for immersive learning as well? We find there is. We find the short bursts as very effective.

But then if you take somebody and in our case two days without, we've even found that not allowing any outside exposure to anything, they can't go home, they can't watch TV, they can't do anything but staying immersed in a long period. Is there a place for that as well and to use that?

Paul Mumma:   Yeah. I think there is, even speaking outside of my Cerego experience. Like you discovered my academic background was as a classicist. I spent a lot of time learning Greek Latin.

And just because of where a lot of those resources are, they tend to be older. They're in physical book form. A lot of my learning had to be with a physical book in a library and not online. It's forced me to be in a more immersive, slower-paced form for some of my learning.

And there were times, and that felt really, really effective as a way to let stuff I had been learning and other ways, kind of distill and all come together that focus and a long time spent on the task.

Phil Weaver:   Do you think, is that the time to do it when you've got that foundation built from the short bursts and you've got all that in there?

Paul Mumma:   I think it can be. I think cognitive science shows that the brain is just not well suited to spending really long stretches learning something new. It leads to fatigue. That's just biology.

I think once you build up that foundation, especially when by building a foundation, you have a bit more motivation, you know you've made some progress, you have something to kind of hang on as you try to work through something, exercise, that's probably a good time to spend more time, less structured time, engaging with the material.

Phil Weaver:   Fantastic. Okay. That's great knowledge right there. As a classicist, I think you had mentioned about the way knowledge was transmitted by the Greeks and Latin. Do you have anything to say about that? There are some mentions of it on your personal site.

Paul Mumma:   I'll tell you this much. If things like Dropbox had existed in AD50, we'd be in a much better place. It's so amazing when you think about all this great knowledge that had been collected thousands of years ago, and it was transmitted in physical form, person to person, from one library to another, but at the mercy of a fire or a flood or earthquake, it could be lost or destroyed.

And that was it. It was gone. We are so lucky today that there are other ways to replicate and distribute what we've learned, what we've written down or recorded. We are at a much lower risk of losing stuff that people spent the painstaking time to learn.

That seems obvious, but it's like, that's a fundamental change in how learning happens. We can now learn in our lifetime and help our kids or our grandkids or other societies get the best of that knowledge with a lot less risk than we used to.

Phil Weaver:   Right. Right. Yeah. It's interesting. Like I said, the form of Kung Fu that, we teach is very rare and I just had the experience of meeting somebody that his instructor broke off from it in 1970.

And it was really interesting to see this knowledge. We pass it down through from person to person like that, but that is the history of martial. And to see that, okay, this body of knowledge was preserved somewhere over here and we didn't know about exists. We didn't know this first existed-

Paul Mumma:   It can be a lot of randomness or luck as to what gets remembered and what gets transmitted versus what didn't.

Phil Weaver:   Of course. Yeah. I see that as well. And in arts or in all of the fighting arts, is that most of it has been lost because somebody died and didn't pass it down. Who's using Cerego?

Paul Mumma:   Wide range, as you had identified. Mostly an age group that is high school and above. The bulk of our customers tend to be college age. Every kind of college, community college, two-year, four-year, public, private, from Harvard Medical School to Arizona State University to Santa Rosa Junior College.

Big, big range, but also people that need to learn something for their work or job formally or informally. And so that's people being trained by the U.S. Army on basic medical techniques when they join the army or people who work in a call center and need to learn about cell phone plans, or about how a certain kind of equipment works.

All of these people need to build up the foundational knowledge efficiently and all these people are using Cerego. It's a really wide range. And some of it is quite creative. Icon Collective is a school that teaches people music theory using Cerego. And some of it is very lifesaving, the medical training that soldiers get. And a lot of it is just traditional academic stuff. It's a big range.

Phil Weaver:   Everything, right. In typical learning, in what places is a person really going to need personal attention? Where's the-

Paul Mumma:   Well, it's a fun question. Where does someone need personal attention? Because there's the personal attention that you can replicate with an app like ours. There's the personal attention that an app like ours can't replicate, but that maybe our app can help make happen.

I'll tell you a little bit about that. I think everyone deserves some personal attention to make the most efficient progress early on. If you're just trying to learn the basics, which can be pretty complicated.

You want to spend time on the bits that you don't really understand and not waste time on what you already understand, personalizing how much time you spend on each of those pieces. That's a great time to personalize and that you can do pretty well for now. It's about what you recommend and when.

Paul Mumma:   Sometimes the reason someone is struggling is for something unrelated to the material. It's something personal going on in their life. It's an undiagnosed learning disability or some other issue that they need to work around.

And in those cases, there may be times when they would get a lot of benefit from personal attention if that teacher, mentor, coach, the manager knew that they were having trouble. And often that person doesn't know.

That's an area where sometimes something like Cerego can indicate with the data that someone is struggling or not making as much progress as you might expect. It can be a flag to help a live human do what humans do really well, which has come in and kind of worked through what actually is going on and see if there's a different strategy to help that person. We've seen that happen and it's really rewarding when the tools or the technology help the human do the thing that only humans can do well.

Phil Weaver:   Right. And it sounds like those might be situations where the human had the flag that they may have missed it in the first place.

Paul Mumma:   Exactly.

Phil Weaver:   [inaudible 00:57:02]. Interesting. How are you helping the classrooms mitigate the shift to online learning?

Paul Mumma:   I would say we don't have the full answer yet, partly because everyone is trying to figure out what they need help with. We've been reaching out to a lot of our existing customers and taking their temperature on what's proceeding smoothly and where they're struggling.

And we're learning a lot from what they've told us. Generally, one thing we found is, people are really interested in making remote learning more effective. That means a lot of people are thinking about tools like Cerego.

We're trying to make it easy to try out our tool and imagine different ways to run a classroom when it's virtual. We're doing that by holding office hours a couple times a week and anyone, customer or not customer can join, jump in, hear from some of our team members or customers, what they use Cerego for, where it works, how they got started.

Paul Mumma:   A lightweight easy way to imagine what you could do differently without forcing you to try or not try a tool. That's one thing we're doing. The other things we're doing are really trying to give people a little extra time and attention to the customer success front.

To make sure they feel good as they get started building out new lessons with new tools and delivering things online, they didn't use to do online. And again, it kind of comes back to human touch to help people use tools well. That's what we're trying to focus our energy on now.

Phil Weaver:   Right. How big of an effect, what do you think about the effect of scaling education that this is going to have?

Paul Mumma:   We'll see. I think a stock that's always been fascinating to me is, when you look at, especially higher education, college-level or above online programs in the last couple of years, what you'll find is very often the institution that's offering an online course may be enrolling students in that online course who live within 50 miles or so of the institution.

Which would make me think shouldn't those people just be going to a class on campus or in-person? The answer is for lots of reasons, they're choosing to learn online. Maybe it's because of a work-life balance or some other considerations or costs or other preferences they have.

And so that might mean that what's going to happen now is not a big expansion of online learning, but just an improvement in the experience. There's going to be more attention paid to the right way to do this.

Paul Mumma:   It could be though that people will say, “You know I had never thought about learning online or learning not in the classroom. And it worked better than I thought.”

And that may change how many people enroll in some of these kinds of courses or how many schools offer online programs. We could see a really nice expansion. I don't know the answer yet, but I think it's going to be fascinating either way.

Phil Weaver:   I think so too. I think this whole situation is going to have some long-term positive effects. Just what I've seen personally of people that have refused to jump into technology for learning are now just no question, they have to. Yeah.

Paul Mumma:   Yeah, absolutely.

Phil Weaver:   And so it's going to have a huge effect. Where did the idea for Cerego originally come from?

Paul Mumma:   The idea for Cerego came before my time. The founders of the company had spent a lot of time looking at how people learn foreign languages especially, those who wanted to get educated overseas.

And what they observed was people had a dual challenge of learning a new language and preparing to do well in school somewhere else. And what they needed to do was build basic competency in the language and the new subject area, and often do it in a place where they didn't have a physical class to go to.

And so this was in the slightly earlier days of the internet and it felt like a perfect time to take this great research about how the brain works and this new medium of delivering experiences over the internet and build a way for people to learn more effectively.

Initially focused on languages and then over time broadly to everything because at the end of the brain learns the same way, whether it's learning a new language or any other subject area.

Phil Weaver:   Sure. Yeah. Interesting. And when was that? When did it get to start?

Paul Mumma:   This was in the early 2000s.

Phil Weaver:   Oh, really?

Paul Mumma:   Yeah.

Phil Weaver:   You've been working at this for quite some time then?

Paul Mumma:   That's right.

Phil Weaver:   That's then some amazing knowledge built up. That's fantastic. Is there anything else you want to tell us about any of these subjects, remote working, remote learning, what haven't we covered?

Paul Mumma:   I think we've covered far and wide a lot of the things that I've observed happening. And I think just my view on this is there's a lot of great tools out there. A lot of ways to try to teach skills or knowledge online.

And I'm sure that for folks that are engaging with you in your show, they might be interested and basically go out and explore. There's a lot of good stuff out there and now maybe people have the time to look around and try stuff out. So, now's a good time to do it.

Phil Weaver:   Yeah. And I see a lot of people are, I see just people mentioning all kinds of different things online. I see a real big positive change happening here in a terrible situation.

Thank you very much. This has been a fantastic interview and I'm sure everyone will enjoy it. Thank you for coming on.

Paul Mumma:   My pleasure. Thanks for having me.

Liz Weaver:    Thank you for listening to the learning success podcast. We hope you enjoyed it. We also hope you have learned something useful, something that you can take back and improve your life with today.

If you would like to say thank you, the best way for you to do that is to share this podcast with a friend, help us help others along this journey. And if you haven't already, please rate and comment on the podcast. Every rating helps us and helps this podcast get out to more people.

We appreciate it, and we appreciate you. Thank you again and make today a great day. No one should have to live with alerting difficulty.