Andy, thanks for taking the time to sit down with us. Could you tell us a bit about how you got into programming?

I think, like many people, it was through games. I actually wasn't allowed to have game consoles for a certain period of my youth, and in that period I went to the public library a lot. When I was nine, I found these books that were meant, I think, to teach programming, but in fact they just had huge listings of basic source code. There were entire books that were source listings, and they were source listings for games. So when I would play games I would type in this book of source, and often there would be a syntax error halfway through, and I wouldn't understand the error messages, and wouldn't know how to fix them. So I'd have to start over, and then I could play the game, and after a little bit of doing that I realized that I could change the games, and very rapidly things degraded from there. I learned C++ — I guess I was 10 — because that was what people made games with, and tried to write a MUD — that was my first big project. It went okay...

Did you finish it?

I got it all going happily as a single-player situation, but of course that does not accomplish the multi-user part of a MUD — so I suppose in a way, no. I got distracted by graphics, getting obsessed with GL and 3D rendering and all that.

And then you kept on making games?

I was doing game engine stuff for a while, going through all your normal OpenGL tutorials, all the spinning cubes and particle engines and all that stuff, and I started making actual games. And then, it was really a case of a 13-year-old yak shaving. I realized that I didn't have a graphics tool that was good for making the content of the games. So, my roommates and I started making a pixel editor. It was called Pixen. That was my introduction to Cocoa and it was open source. When we hit 1.0, I think I was 14, and we went to the OS X Panther (or maybe Jaguar?) launch party at the Apple store in the mall. They had those at the time! We tried to give away CDs of our pixel editor.

Is there an alternative reality where you would not have become a programmer? Did you have anything else that you found super interesting?

It was going to be engineering of some kind. I was fascinated by electrical engineering and even chemical engineering in little high school experiments, so it was going to be one of those things, and software engineering was what I could tinker with. At the time, I called it programming and not software engineering, and then some people told me, "No, no, you're supposed to call it computer science!" Of course, at the time, I didn't understand the interesting cultural distinction that is ongoing between computer science, software engineering, and programming, which I do actually think are useful distinct terms. But we sort of misuse them interchangeably.

Right. So now you're a software engineer, mostly?

I'm definitely not a computer scientist, although that's where my training is. I would say that I am sort of a software engineer, and I am a heavily computer science-influenced software engineer. And I spend about half of my time these days on more research stuff, reading papers and books on child development and working through ideas for how we might help that.

Leaving Apple and going to Khan Academy, helping people to learn and to educate, seems like quite a big jump. Why did it happen? What motivated you to do that?

Well, there are a few ways I can approach that question... I went to this college, Caltech, that fed into a lot of national science programs, NASA and DARPA and things like that, and so the discussions there were always about human scale impact. And we had the NASA administrator at the time come by and say to us, "OK. You're the best and brightest. You're going to one of the best research universities in the world, and you are privileged, and you have all of these possibilities open to you. Good for you. Now, you have to use them for good. You have to do something that matters."

When I got out in the broader software community, I was one of the only people who didn't go into research. I wanted to make things at the time when I graduated. I was disappointed by the impact and the ambition of most commercial software engineering and sales — so I thought long and hard about where I could make an impact. The first step for that was really deciding the metric of what that impact is and what matters. I've been heavily influenced by the work of the physicist David Deutsch, and, through him, have sort of derived the metric for myself that I want to expand the reach of human knowledge and ability. So I thought about how I might do that.

We would expand our reach if we lived longer, so there are great opportunities there in biotech and chemical engineering. And we would expand our reach if we had better tools for experimentation. And so fabrication tools and synthesis tools are appealing, virtual reality is appealing, artificial intelligence strongly is appealing along that axis. But eventually I decided that, given my background and my opportunities, making a contribution to education would be my best bet, because if I can create a generation of lifelong learners, then we will have more people trying to solve these problems.

So you want to build tools for coming generations to learn better, so that they can solve difficult problems going forward?

Yes, that's my hope. A big challenge in learning is about the meta learning. It's a way for people to learn without destroying their self confidence. It's a way for people to learn without destroying their attitude toward education. I would be satisfied with the same efficacy: if the tools I create yield the same math scores for graduates, if only their attitudes toward math were better at the end and they were more interested in continuing to learn later on in life.

So clearly you think that the schools and universities we have already are not good enough — could you elaborate on this?

No, it's not good enough. And it's not good enough along many axes. So the most fundamental axis along which it's not good enough is that most people don't have access to what I just described. And that's, I think, the most important thing to fix and the thing to fix first. And that's why the mission right now is a free education for everybody. And Khan Academy's mission is actually a free world-class education for everybody, so we're perhaps focusing on the free and for everybody part first, and then world-class will come. But that gets to the second point, which is that I'm not really satisfied with what counts as world-class right now.

I had so many classmates who had the best education money could buy and were taught by Nobel laureates. Nevertheless, they had awful attitudes toward learning, and considered themselves done as soon as they graduated. They just wanted to go off and find some steady job, where they can kind of turn their brains off, because they'd been so beaten down by the system and their curiosity and intellect had been so damaged.

That sounds like a big frustration...

I think so. I think you ask most people what they think of math and they say either, "Oh, math's boring," or "I'm not good at math. I'm not a math person." You get similar responses for sciences. If you ask people about literature, maybe they'll talk about some books whose plots they found interesting, but they used to read in school, and now that's kind of over. So I suppose the biggest change I would like to affect is to change our perception of education being over after college is done, which is not really a technical problem, and that's why I sort of hesitate with the label "Software Engineer." That's definitely where a lot of my expertise is, but my ambitions are bleeding elsewhere.

With this really big plan, focusing on lifelong learning, do you have a vision where you want to be in 10 years, what you would like to have achieved?

Right now, the standard means of learning are broadly prescriptive. What I mean by that is that you read a textbook, and you passively absorb that information, or you watch a video, you watch a lecture. You are meant to sit and absorb, and perhaps there's a conversation afterward if you are wealthy. After you graduate, you just sort of have to construct those things for yourself. I'm really interested in so-called constructivist learning environments. That's a term from Papert, who was a learning researcher a few decades ago. In a constructivist learning environment, the student creates the knowledge for herself, rather than by reading it from a text and passively absorbing it. In these models, it's a little bit more like an environment that has been constructed, such that the features of the environment will encourage or require you to have understood something in order to manipulate them.

The example from Papert's research is that he was trying to use Logo to teach children not about programming really, but about abstractions and procedural thinking, and so he had a classroom where there was the literal physical Logo turtle. There was a mechanical turtle at the time. It would draw on physical paper. And so surrounding the classroom were drawings that the turtle had made. On some of these drawings were source code, and on many of them were not. And the drawings would change over time, with the students, and the idea is that the student would maybe see a flower and the student would say, "Oh, I like that drawing. Can I have it?" And the instructor would say, "Well, you can have it. You just have to type this code into the thing and then the turtle will draw it." And the student will say, "OK, cool," and will have the drawing, but will look at another one, and that one doesn't have the source code.

These other drawings in the environment are designed so as to require that the student understood something in order to have produced them. So the environment is sculpted in that way. The hypothesis that I'm investigating, and likely will be investigating for many years, is that by allowing people to construct knowledge for themselves, we can hopefully — I will be happy with this — achieve the same learning outcomes, first off. But with much-improved metacognitive outcomes — that is, their self esteem is higher, their attitude toward learning is higher, their interest in learning more is greater.

Next to the more active learning that the constructivist approach encourages, another major difference between what you're doing at Khan Academy and more traditional lines of education is that you're free to learn what you want to learn and when you want to learn it. What do you think is the impact of this change?

Another child development researcher, Piaget, has a lot to say about this. He says that one of the very important things that must happen in our development of young minds is the training of autonomy. And autonomy is meant here in a broader sense than what we normally mean when we talk about autonomy. It's not just autonomy in the sense of "I'm free to play on the playground," but autonomy like, "I am making my own choices — broadly, in general — for all of the things that are going on in my life." This is contravened by somebody telling you how to spend your day, or by an appeal to authority. For example, a heteronomous answer to "Bobby cheated when we played this game" would be "Oh, well let me go talk to Bobby." The opposite of that, an autonomous answer, would be for the teacher to say "And what did you say to Bobby?" I think the training of autonomy is important for many reasons. There is some research that suggests that it provides greater outputs in the short term, but my belief is that it will lead to greater thinkers long term.

I'm skeptical of an entire day being scheduled by someone other than the student. I feel more positively about the idea, though, of an educator modifying the environment of the student, day to day, in a way that is sort of sculpted for the students' progress. For instance, to be a little more specific, I think we can do better than giving the student the entire catalogue of human knowledge, with every element of that catalogue being sort of emphasized to the same degree. If we picture all of human knowledge as books on a bookshelf, we should place the books on the lowest shelf that we think would be best for the toddler coming up to it.

Is there something you miss from your time working at Apple?

Yeah, I miss the people. That's a big thing. This is actually something I find really frustrating about the industry right now. There are so many brilliant people locked up in these enormous corporations, and occasionally they're being put to good use, but often they're just gears in a machine and their intellectual contributions could be enormous and so much more impactful. I don't just mean working on my projects. I mean it broadly, in general. It makes me sad both because I miss them, on a very sentimental level — I miss having that stimulating conversation with these very experienced people who, for example, were at NeXT. But I also wish that their impact could be greater.

Can you tell us a bit more about your time at Apple? What was it like to build a framework that's used by so many people? And how did you get there from building games?

One of the dots filling this gap was this framework I built when I was 16, called Sparkle. That taught me a little bit about the realities of framework development, namely that you change APIs on people, and then they get angry. It's one of the fundamental realities of framework development, but honestly, I really didn't understand what I was doing until I got to Apple, and that was kind of the point. I finished college, and I had these broader ambitions, but Apple seemed like a great place to train, and it was. It still is — I would still recommend it to anybody for that reason. I worked on a team of people who have been building frameworks for 10+ years, and I guess I was kind of thrown in the deep end. I was forced to really rapidly train and understand many unusual and unique concerns when designing frameworks at that scale. Especially frameworks with binary compatibility concerns, which Sparkle did not have. Luckily, everybody around me was incredibly generous with their time and attention and I got great mentorship. After making many, many mistakes, I understood what the hell I was doing, and then a couple years later I left and may never use those skills again...

What was the biggest surprise for you in building frameworks at this scale?

I think binary compatibility was my greatest surprise. I did not understand the difficulties and complexities which it induced. So, to illustrate, those are bugs of the form: on iOS 8, this button has a white background, and on iOS 7, it's transparent. iOS 8 hadn't been released yet, so it must be our fault. You start digging a little bit and you realize that maybe a method was called in iOS 7 which took care of this issue, but this method is not called anymore on iOS 8. Then very rapidly this deteriorates into disassembling third-party code to which you do not have the source, in order to try to figure out what's going on. About half the time, somebody was doing something heinous and this was the strange way in which that manifested, and half the time we had changed something in an unexpected way, and we would need to add some defenses for this reason.

That sounds like a lot of work...

Yeah, it was sort of detective work.

It sounds a little bit like what Peter Steinberger is doing in the other direction, disassembling UIKit to fix bugs...

Yeah, for sure. The releases were so secret that we couldn't ask, "Hey, Facebook guys, why are you doing this?" We'd have to figure it out. On this note, I absolutely admire the work that Peter and his team are doing. I think that's a great thing, and I would love to see more of that. I would love to see more development of frameworks by the community. Facebook's making great contributions there — GitHub as well.

I've never thought of the fact that Apple also has to disassemble other people's binaries to figure out bugs.

Oh yeah, and as a third-party developer now, I have to disassemble UIKit on a pretty regular basis, because I'm like, "UIKit, why the hell?" And the easiest way to get an answer is to just look at the disassembly.

There was a ton of new stuff last year. What are you really excited about? You're also excited about Swift?

Yeah, these are pragmatic excitements, I should say. When we talk about what stuff I'm really excited about, it is not in the consumer software engineering industry in general. But as far as helping me get my work done, the category of things I'm most excited about are really interactive prototyping tools. So if you're trying to make an interaction that did not exist before — or even one that sort of vaguely has existed before — there really are no tools for it. We're still broadly designing in Photoshop. When you design in Photoshop, you produce something that looks like you designed in Photoshop. I remember at Apple, I could tell the difference between stuff that was made according to specs that came in from Photoshop, and stuff that was made according to specs that came in via After Effects, and stuff that was made according to specs that came in via live prototypes — it was a huge difference. So there are some tools out there that are trying to do this. I think none of them have gotten it right yet, but I am watching this with great fascination, hoping to contribute as well.

You mean tools like Framer?

Yeah, Framer is a pretty good example. I'm very interested in things like Form, which was just bought by Google, and Origami, of course, which is interesting. I worked on a prototyping tool internal to Apple that got some things more right and other things less right. Each of these tools has pros and cons. We've actually been compiling a list of "you might use this tool for this purpose, you might use this tool for this purpose." We find that we have to bounce around. I feel that none of them actually excel at any of the purposes except perhaps making things that are very much like things that already exist.

Do you think storyboards are also moving in this direction?

Well, I don't think so, but I'll have to lunge out a bit to explain. There are discrete modes of interaction and there are continuous modes of interaction. Something like a mouse click is a discrete form of interaction and something like a gesture is a continuous form of interaction. Similarly, there are discrete states of your application and there are continuous states of your application. So something like scrolling around a scroll view is continuous, and navigating to a different level of a navigational stack is discrete.

Storyboards allow you to manipulate only the discrete and discrete section of that quadrangle. You can make it so that when you tap a thing, then there is a discrete transition to a completely different place. But I find that to be the least interesting kind of interaction, and frankly, I think that our user interfaces are dominated by that kind of interaction primarily because we don't have tools that allow us to think about other kinds.

But even starting in iOS 7, the sort of discrete modes are blurred a great deal. For example, being able to tap a table view cell and expand something beneath it — it's a very trivial example of something that storyboards would already have difficulty representing.

We would need more Bret Victors working on that problem.

We do, and that's part of why I wish fewer people were making photo-sharing apps.

You mentioned before that the consumer software engineering space is actually less exciting to you. What excites you outside this space?

I've been reading a lot of papers in different fields, and I'm still wrapping my head around this. I think the ones that have stood out to me recently have been about child development, or about educational or pedagogical models. One of the most exciting ones I read was a survey in which a classroom of first graders who are five were given no formal math instruction of any kind, and instead they were given a bunch of activities which change day by day that were just placed around their environment, that were games. After a year they were assessed, and even though the children hadn't been taught any formal arithmetic methods, they were all able to answer these questions and had great metacognitive characteristics as well. Things like that are incredibly exciting to me.

So what's the most important thing you learned since starting at Khan Academy?

Leading teams is hard...

Why?

It is a distinct skill set in which I had little practice. In the software industry, we throw individual contributing engineers into leadership roles on a pretty regular basis and expect them to excel when, of course, it is a distinct skill set, and we have to recognize that and appreciate that. So, a lot of learning has been required there.

You mentioned earlier that you had really good mentors at Apple. What made them good and what did you learn from them?

The primary thing that made them good was the generosity with their time. That was always a challenge at Apple. Apple had the problem that it culturally does not value mentorship. One is not particularly rewarded for that, and teaching and learning is not really built into the culture.

Anyway, I was fortunate enough to be able to pester a few people into spending a fair amount of time teaching me things, and in certain cases I made mistakes egregious enough to have people spend their time teaching me things without requiring pestering. And let's see, what made it effective? Probably their experience. The industry is an interesting place right now, because it's flooded by new grads, and the new grads are noisy and they're on Twitter and they're the ones that are speaking at conferences. But this industry has been around for a while. And there sure are a whole lot of software engineers with 15 or 20 years of professional experience in the field, and they're not as noisy. You don't read stuff by them all the time. They're mostly locked away in companies, where they can't talk, but they sure do know a lot.

At Khan Academy, you and your team adopted Swift very quickly. Can you tell us a bit about your experience?

We have something like 20,000 lines of Swift code, and there are four contributors to that particular project, all contributing in Swift. And the issue that I have or that I've spoken to is really an issue of practicality versus idealism. I'm sure the issues that I have with the language itself are temporary, and they don't really matter in the grand scheme of things, just in the same sense that any app that you can build in six months probably doesn't matter. So take my complaints with a grain of salt. As far as the long-term health of the ecosystem and of professional software development goes, I think it will be an excellent contribution, and these rocky first months don't particularly matter.

On the plus side, it has helped us to write clearer code. That's the primary thing that I enjoy when I read a Swift interface. When I read a block of Swift code, I have a much better idea of what's happening and what's not going to happen. I can see return values and the return values tell me more. I can use enums to model more accurately the states of the system, we can use value types in order to constrain and confine and limit dependencies, most critically. More broadly, we can move toward this pattern that I strangely endorse, which is really treating UIKit as a library and not as a framework.

In terms of problems, the tooling is not there, and that's not a dig on the engineers. I happen to think, actually, that the Swift engineering team is one of Apple's finest engineering teams, but it's not done yet. It's full of bugs, it crashes too often. It generates faulty code occasionally. Most critically for us, it's just really slow. Really, really slow. The lack of incremental compilations is the biggest pain point for me. We're building a UI application, and as much as I'd like to spend all of my time thinking about how to improve learning, at some point, I've got to polish interactions and animations, and doing that with a two-minute turnaround time is really damaging — not only to productivity but actually to just morale. It just makes everybody angry. Everybody's just angry, all the time. And worse, it makes us lazy. We don't want to change stuff — so I would not be surprised if the Swift-implemented UI areas of this application turn out less polished than the areas of the application implemented in Objective-C, just because we can't iterate on it without going insane.

But that's temporary; I'm not worried about it. They know about this and they'll fix it. It's just that expectations were mismanaged, not out of malice, but I think Apple just didn't know, and hadn't built anything this big.

What effect do you think Swift will have on Apple's framework APIs? Do you expect something here in the short term?

I don't actually have insider knowledge here, so this is just speculation, but I think it will be a long process. At least when I was there, the teams spent the majority of their time not maintaining and improving frameworks, but really supporting market features like new screen sizes or support for new hardware. That's what takes most of the time. So it will take a conscious decision to do anything non-trivial, and I don't see that forthcoming.

I think that if we fast-forward 10 years into the future, the APIs are all going to change. I have no doubt — either that will happen or it will be an incredible disappointment. In a year, I think you'll get better annotated headers...

Or we'll have somebody, like the Reactive Cocoa guys, wrapping it all up in a nice way...

Yeah, that's going to happen in the short term for sure. Facebook's doing it right now, but it's not Swift. It's a C++ thing, with their components project, which I think is valorous. That's a great idea. React is a lovely design. One of the complaints that I have about my time at Apple is that, because the culture doesn't value learning, nobody read. I mean this in the sense that scientists say when they talk about reading. Nobody reads the journals. Nobody reads the papers. Nobody knows what's going on in the broader community. People didn't know about React and all this stuff. That was very frustrating.

Anyway, I think those things are great contributions and I have no doubt that somebody will treat UIKit and other frameworks as libraries and wrap them up in something nicer. However, it won't get great adoption until Apple does it, and it'll forever be behind and feel like an impedance mismatch until Apple does it.

Since you've left Apple, you've also done more writing and speaking. Do you have any other plans there?

Mainly more of the same. I'd like to publish more open-source stuff. I've been a little bitten by that in the past. When you publish open-source stuff, funny enough, people expect you to maintain it. I think we haven't quite gotten clear the contracts that we expect socially around open-source software. There's a kind of open-source software that has a community around and is well maintained, and then there's a kind of open-source software that's just being shared for educational purposes or for interest, but not really being maintained. I probably only have time for the latter, because, frankly, software engineering is not the interesting part of the problems I'm trying to solve right now.