They chat about the non-determinism of social media algorithms, the need for balance between efficiency and human dignity in technology, and the role that trust plays in AI.
Human Agency in the Digital World is an “AI-era self-help book” about reclaiming our role as pilots—not passengers—in the technology revolution. It’s available now on Amazon and everywhere books are sold.
Connect with Marcus on LinkedIn and learn more about his work at his website.
Congrats to user Romain for winning a Populist badge on their answer to Django: show the count of related objects in admin list_display.
[Intro Music]
Ryan Donovan: Hello everyone, and welcome to the Stack Overflow Podcast, a place to talk all things software and technology. I am Ryan Donovan, your host, and today we are talking about how we can steer technology towards human dignity and not just for efficiency – how understanding technology helps us live in it better. My guest today is Marcus Fontoura, who is a technical fellow at Microsoft and the author of the book ‘Human Agency in a Digital World.’ Welcome to the show, Marcus.
Marcus Fontoura: Thanks for having me, Ryan. Pleasure to be here.
Ryan Donovan: Before we get into human agency stuff, tell us a little bit about how you got into software and technology.
Marcus Fontoura: When I was a kid, I loved math; so, I thought that I was going to study math and be a math professor; but then when I went to college, I had to take a basic computer science class, and I had done a little bit of programming before, but I really fell in love with programming and computers when I took my first CSO one-on-one class. And then, from there, I switched majors, did computer engineering, which was new at the time, and then this was back in the 90s. And then, I moved on to do a PhD in computer science and moved to the US for working in Big Tech since the early 2000s.
Ryan Donovan: Computer science grew out of the math department. So, in a way, you did go into math, right?
Marcus Fontoura: When I was a kid, I had a computer, but I did a little bit of programming, but I really loved just sitting in my room and solving algebra problems much more than programming. And then, somehow in college, [there was] that switch. Anyway, I do a lot of math today still, because we try to solve hard technical problems, and it’s all related, but I’m glad that I made the switch, given how much the field change over the years since I graduated up to now.
Ryan Donovan: Let’s talk a little bit about the book and the ideas that you talk about – how to enable human agency within a digital world. So, how can understanding how computers and technology work better, how can that enable people to have more agency in today’s world?
Marcus Fontoura: Yeah. The idea came to me because my daughters, they of course, are very curious about AI and technology, and they kept asking me, ‘dad, what should I do for college? Does it make sense to study this profession, or that profession?’ And then, I realized that at least they have me to guide them a little bit, but for most people, it’s hard to understand what’s going on. With so much hype about AI, about technology, and so much news, and the news tend to be very polarized; either it’s like, ‘AI is going to solve all different humanity problems, or AI is going to cause the extinction of humanity.’ And of course, I believe the truth lies a lot more in between. I felt that it’s hard for people to relate to a technology if they’re so distanced to it, right? So, if you completely don’t understand how things work, it’s really hard for you to feel that you have any sort of agency that you can influence, that you can have educated opinions about it. So, that’s the main intuition and motivation that led me to write the book.
Ryan Donovan: Yeah, I remember a former colleague tried to start learning programming, and I remember he ran into a sort of issue and understanding what arrays were, and things that I think listeners of this program would take as a pretty basic data structure. And I like, in the book, that you talk about things in a very sort of generalized, understandable way. How did you get to that approach? How did you [start to] be able to break down the sort of understanding of technology into a way that was like ‘Alice and Bob’ level?
Marcus Fontoura: That was what I was aiming for. I felt that I needed to convey it almost like a Malcolm Gladwell book, right? In the sense, he– of course he’s a much better writer than I’m, like a professional writer. For him, he is able to convey complicated and interesting concepts so that lay people can understand. I think the concept of the tipping point, for instance, is one very important one that he was able to clearly disseminate. So, when I was thinking about the book, that’s the frame of reference I had, right? That it doesn’t do me any good to try to explain how sorting works, and what efficiency is for computer scientists, because they all know about it. And my main intent was to help educate people that are interested in technology, that care about what’s going on in the world, but they are totally capable of understanding these basic concepts about computer science. So, I tired to put myself in their shoes and say, ‘how can I explain this if I were not a computer scientist, or to somebody that– to my mother, or a psychologist, or to a lawyer?’ That was what I was trying to achieve.
Ryan Donovan: I’ve had this problem too, and a lot of listeners have had that where talking to my father, or whatever, about computer issues. And it’s like, you don’t have the sort of fundamental understanding of what this even is. And I think stepping through the algorithms, where you walk through the indexing algorithms, ‘ here’s what happens at this number, [and] what happens at this point,’ is very helpful. Have you had to have those difficult conversations with non-technical people before, and how did they change your approach?
Marcus Fontoura: Yeah. One of the things that I think helped me is that I mentor a lot of people as part of my day job at Microsoft, and even before in other companies. And one of the things I love to say to especially new grads that I think what helps the most in people’s career, especially on early on, is if they have a clear understanding of what they are working on. So, as part of my mentoring that I do with a lot of recent grads, is just help them have a crisp idea of why the problem that they are working on is important. How does it fit in the overall context of their organization, or the company, or the whole industry? So, I was training myself to say, ‘how can I simplify this concept and present it in the most clear possible way?’ That’s one thing that I always try to do, and I think it really helps people, because I’ve seen some brilliant people that perhaps [are] doing something brilliant, but then you ask what they’re doing, and they’ll tell you in very complex way that you can’t even understand. And then, that tells me that they really don’t understand the context, right? When they cannot really explain to you it’s because they really didn’t internalize how the things truly work and what are the foundational principles behind those.
Ryan Donovan: That’s the expert’s dilemma, right? You get to a certain level of knowledge, and you forget the beginner stuff. You internalize the context, I think a little too well sometimes.
Marcus Fontoura: Yeah. And then, I think that’s one thing that we technologists have to do all the time, right? And I feel that, unfortunately for us, I don’t think we do a great job on those things, because if you think about the amount of systems that we work on in our daily lives, and that everybody’s forced to work, right? There’s no way around technology these days. And economists and lawyers, they have a much more pronounced impacting in government society law than we technologists do, because I feel that we don’t try to explain things in a clear way, and then we don’t try to position the work that we do in a way that we are working for society, and for advancing societies. So, I think that’s kind of also a call for action, for new technologists to really try to think why we are doing those things, and is this the real impact that we want to have in society? And how can we amplify our impact?
Ryan Donovan: Yeah, and I think the sort of thing you’re touching on is that a lot of technology people explain technology within the context of technology. Economists explain the economic stuff in terms of societal impact, and it’s getting that larger, broader human context.
Marcus Fontoura: Yeah. And that’s something I’ve had to learn how to do. And I don’t claim I have the formula, but that’s something that I attempted to do in the book.
Ryan Donovan: So, in terms of the societal impact of technology, obviously in the last 20 years it’s been pretty immense, and it has felt very much like this is the storm coming at us. How can regular people use this understanding of technology that the book has to gain a better foothold in the modern technological world?
Marcus Fontoura: I talk about in the book, I try to explain the algorithms behind social media, and I’m not claiming that all uses of social media are bad, but there are like the uses of social media for information dissemination. That’s the topic that I try to address in the book. And basically, I simply do a high-level technical analysis of how content propagation happens in networks. And then, I show that these algorithms are very fragile, right? They are non-deterministic, so more perturbations in your network will lead to content being disseminated above other contents that are probably more assertive, or more reputable. So, by explaining the foundations of these algorithms, which demystifies discussion of ‘[are] social networks good or bad?’ I really think that’s a pointless debate if you don’t know what you’re talking about, but if I can clearly explain to you that this is based on an algorithm that is not stable, and then you produce widely different results based on the small perturbations on the input, then you clearly know, ‘ oh, this is probably not the right algorithm for us to use for content dissemination,’ especially for a large part of the population uses this to consume news. So, I wanted to give the foundation to people to be able to think about the societal problems with deeper understanding, and more structure to the conversation, talking about algorithms, inputs, and outputs, and expected results.
Ryan Donovan: Yeah, the social media is interesting because there’s an argument to be made that greater information dissemination is disruptive to society as a whole, right? The printing press came along, and then you had a hundred years of wars of reformation in Europe. Is there a way to have that greater information control connection with some sort of guardrails on it to prevent the sort of disturbances we’ve seen?
Marcus Fontoura: Yeah, that’s one of the things that I address in the book because I talk from the evolution from books to the web, and then from web to social media, and you have millions of books being published every year in the us. But it’s still a high barrier of entry, right? Normally, people can find out what the good books are because the number is high, but it’s not ridiculous, right? So, we can have some control of which books are written by experts, and so on. But instead of having a million books, you have a trillion webpages, and then it becomes a lot more unstable, and how do you distill the assertive web pages from the non-assertive ones? And then, we addressed that by developing algorithms like page rank and that use the link structure of the web to be able to determine that The New York Times is probably a more assertive force than a random person’s blog. And then, page rank became the key algorithm that propelled Google to be a much more assertive search engine compared to its predecessors, because we started using more structure, and more link structure. And then, when we went from the web, that’s like a trillion web pages, or more than a trillion web pages, to social media that is democratizing even more publishing. And then, we did this leap that we went from a trillion pages to billions of posts being posted every day. And then, we don’t have page rank anymore for social media, right? It’s all based on likes and information cascades that, as I said before, is a completely fragile algorithm. So, I think what’s important for us to understand is when you’re doing these leaps ,we have to democratize access to information for sure. And then, we have to lower the entry of barriers, so that we can have more people accessing more content quicker. But we cannot lose our footing in the sense that there’s no point in doing that if you cannot control the information in the same way that you wouldn’t go to a doctor that always prescribes you the wrong medicine. So, you don’t want to use a platform that we always prescribe you the wrong news. So, that’s the technical discussion that I would like to have, or at least provide the technical arguments so that we can have an informed discussion here.
Ryan Donovan: It seems like with the doctor comparison, you and I have better metrics for that, right? We understand if this doctor prescribes medicine that is actively harmful or doesn’t work, nothing happens when people get sick, but with social media, it’s basically how fun is this news?
Marcus Fontoura: Yeah, we can spend the whole time talking about social media, but I was going to say it’s all based on this concept of weak ties, right? Social media networks, they encourage us to make connections that are not strong connections or acquaintances, but to broaden our network to reach far out people that we don’t directly know. So, this allows information to propagate very fast. And the way information propagates through these graphs is using these propagation models that is based on how popular they are, but not without any metric that associates relevance or assertiveness to the content creator. Like [how] we have page rank when we were talking about webpages, for instance.
Ryan Donovan: Yeah, and I think there’s a sort of fundamental conflict between what the company produces, it wants, and what we want, right? The company has to make money so they go for engagement views, ’cause they sell more ads, and I think understanding that conflict as well will help people navigate social media, or all of it. It’s on search engines, as well.
Marcus Fontoura: Yeah, exactly. So, I talk a lot about ads chewing the book, and then I love Yuval Noah’s ‘Harari’ – he has a definition of data-ism, [how] we’re moving from capitalism to data is the new equivalence of the gold rush of the money. These companies will target and build platforms in this to attract users to provide the most information that they can provide, and it’s all because of ads. However, ads is also a very fragile and unstable system. So, that’s why I think we need to really understand these systems a little bit better, because I feel most of the population just says, ‘ oh, this is a fact of life that you have ads in structure results,’ but in fact, it’s not a fact of life, right? If you think this is harmful, we can provide regulation. We can think more deeply about this. And this is coming from somebody that spent years of my life working on advertising systems
Ryan Donovan: Somebody has to pay for it at some point, right?
Marcus Fontoura: Yeah, somebody has to pay for it.
Ryan Donovan: The pitch talked about regaining agency instead of just pure efficiency, but a lot of what computer science does is enable efficiency, and you talk about it in the book too, especially in terms of like organizations and such. How do you think about and resolve that tension?
Marcus Fontoura: This is one key theme of the book. When I started writing the book, I thought I was going to write a whole book about efficiency, because that’s basically what computers do, right? Even with the early inceptions of computers during the World War II Manhattan project, and so on. Computers would use to replace what we call ‘human computers,’ that are basically women and men that are doing numerical competitions by hand. And computers could do that a lot faster. And then after that, we started using computers for all sorts of things, but basically, computers still cannot do much else other than computing functions very fast, right? So, that’s all that computers know how to do. I think the key point is that why we are doing that, for instance, why do we want to implement a content dissemination platform? And then, probably because we have good arguments that we say that this will really positively impact society, or we get access to more information quicker. And then, if you can verify that the sources are reputable, even better. We really lower the barrier of entry for people to publish content, and so on, right? So, once we understand the system that we want to build, and that it has the properties that will impact society positively, then at that point we can think about what are the efficient algorithms that this can scale, and this can reach a global population, and then we can really lower the cost so that we can provide services at a low cost for most people in the world. But to me, it should be a secondary thought, because if not, we are doing efficiency for the sake of efficiency, and then that’s not really beneficial, and then overly focusing just on efficiency – probably that’s not we should be doing.
Ryan Donovan: Right. The universal paperclips story, where the machine optimized to make paperclips eventually turns the whole world into paperclips.
Marcus Fontoura: Yeah, blindly following orders. That’s all that computers do. They blindly follow orders. So, if the orders are not going to create a positive impact for us, then probably we should revise these orders, and that’s like the balance that we should strive for. But what I argue in the book, this efficiency that computers create, of course, is super welcome, right? Because the most that are relevant for us to solve in society, like protein folding, distribution problems, healthcare problems, and all that really need efficient algorithms that scale to 8 billion people, that are not fragile, that have robust properties, and so on. So, architecting these algorithms is really important, but we have to do it to problems that are relevant to society.
Ryan Donovan: I think something I’ve been thinking about lately is the value of friction in some cases, because a lot of what computers have done is reduce friction in terms of transactions, interactions. But you talk about it in the case of publishing information, it’s much less friction to get some information out there. In terms of learning, I think AI has made it very frictionless to get information, but sometimes you need that friction.
Marcus Fontoura: For instance, for books, I think it really worries me that with self-publishing and AI, [there’s a] federation of books coming out, and it is really hard to watch the quality at this point. And then, if you have an influx of books, then a lot of the systems break, right? How are we going to select the best books? How are we gonna promote the best books? And all that. If we don’t have a page rank equivalent for books, I think really lowering the barrier of entry for writing books should just come along. If you solve the problem of like, how are we gonna distill garbage from good books?
Ryan Donovan: Yeah. And that’s a bigger problem, now. It’s so easy to produce the end product. You talk about the story of Gabrielle Garcia Marquez sending off half of his manuscript. Would Gabrielle Garcia Marquez have even been discovered in the flood of AI books today?
Marcus Fontoura: Yeah. It’s really hard to know, right? And then, I think I told that story in the book that he had basically no money, and he was writing for one year and a half, and when he was going to send a manuscript, he didn’t have money to ship the whole thing, and he split it in two piles, and by mistake, he sent the second part. And then, he was lucky that the editorial loved it so much that they sent him the money to send the first part. But one of the things he said is that he was typing on a typewriter, and then when he found a bug or a typo in the page, [he would] tear apart the page and write over. And then, it makes me wonder, that friction, did it really improve the quality of the book or not? If it was harder to write, would it make books better now? Because probably we can just gloss over now, because as I’m typing, I have like spell correction, grammar correction, and so on, right? So, the barrier of entry is much lower, but I’m not sure what is the impact on quality overall.
Ryan Donovan: And I wanna go back to the AI question because obviously that’s the topic du jour. You said the answer is between the doomers and the utopians. What is that real middle ground look like?
Marcus Fontoura: I think the real middle ground, can we apply AI of today to solve real problems of today? And my answer is that we can do a much better job on that. And I feel that instead of concentrating on that, I think a lot of people are spending their time and saying, how can I make AGI, right? How can I make an AI that is much smarter than humans? And all that. And that’s all valid, and I think we should be pursuing that track as well, but my point is that we already have AI to a point that is good enough to have a huge impact in society today, and to unleash a lot of things, right? From lowering healthcare costs, aiding in vaccine development, aiding basic science. I think that’s the point that is– we should realize that it’s more– I really believe that all of today’s problems can be solved today with the technology of today. If you invent AGI, it would be great, but we already have enough technology to have a huge impact on society. We are not really thinking of about the applications, and I urge people to shift to think about the possible applications. Not being afraid of AI. And I really don’t like this almost like an argument that we want to say, ‘oh, this technology is so powerful that it’ll either destroy us or propel us forward.’ But let’s just take it for what it is, right? That it’s a prediction platform, but it’s a very good and accurate prediction platform. So, let’s take advantage of it to fix distribution. Let’s really make a huge progress in self-driving cars. Let’s make a huge progress on telemedicine, on diagnosis, and all that. And I think that’s where the money is for me, and where we should be spending our time and energy.
Ryan Donovan: I like that focus. It’s a tool; it’s not the thing that will destroy us, hopefully. We have other tools that are poised to destroy us, as well.
Marcus Fontoura: Yeah. And then, I think really destroying us, I don’t even understand this argument because AI is like a prediction tool, right? So, when we talk about agents that are software programs that are using these predictions, the agents, they are deterministic, right? They are binary code that we know the input, we know the output. So, it was coded by someone. It could be generated by AI, but we can analyze the code. What we’re saying when you’re saying AI is predicting the world, is that we humans are going to use the predictions that AI generate to destroy the world. So, then the problem is not the AI, the problem is us, right? Because there is no AI code. All the code that we run on any computer in the world is deterministic and can be analyzed by humans.
Ryan Donovan: Yeah, I think that’s a fair point that ultimately, all this comes down to is mistrust of humans, because it’s the humans who will use the tools.
Marcus Fontoura: Yeah, of course. And then, think that’s brings us to the point that I think it’s more beneficial for society to try to understand, how does ChatGPT work, right? And how does Microsoft Co-Pilot work? Is this magic? It’s not magic. And any of us can understand what it’s doing under the covers, and then it’ll help to demystify being afraid of it. Because it’s basically a technology that uses a lot of statistics, uses a lot of complicated technology, but we can show how it works, even to a fifth grader, and I urge people to get curious, right? Because the more people are understanding the technology, the more people will be able to think about applications to leverage it to the good of society.
Ryan Donovan: So, what is the single thing that you think people misunderstand the most and that would help them the most to navigate the digital world?
Marcus Fontoura: I think people attribute too much to computers. One point that I bring over and over in the book is that computers just compute functions very fast, right? And then, they make very little mistakes. All the rest is us humans using it in the top. And then, I think people misunderstand that, and they misunderstand that technology by itself is not either good or bad, right? And it can be changed. So, I think people just assume that, ‘okay, social media exists, like advertising systems exist, web search–’ when I was born, none of that existed. And then, we created those things, and we can modify it, and we can make them better. Just don’t assume that things happen [and] are [just] there, right? They are there because we made it, and we can change so that they can become better and more useful.
Ryan Donovan: Yeah, there is always societal policy end to this. One thing I always think about is that when I was a kid, I grew up on GI Joe and Transformers cartoons, but those only exist because they changed the law so you could advertise towards children, because those shows are basically advertisements. And I have great fondness for them, but they exist as advertisements, right?
Marcus Fontoura: Yeah, exactly. I think one of the things that I do in the book is I bring over and over that these things are just fiction, right? We build the systems, we can change those systems. For the young generation, they probably think the cell phones always were they are, social media was there. It is not the case. And then, we can envision a world that doesn’t have those things, and perhaps have other things, right? So, a friend of mine used to say that, before Twitter, nobody knew people who had the urge to tweet, right? Like, this is something that was a discovery. The people like doing that, and then we can envision a world [where] there is Twitter, or Twitter is different. And to this point, I feel that AI is the thing that seems like a revolution. It seems that people we’re not working on AI, and then suddenly in 2023, we had ChatGPT, but it’s not really the case, right? This is an evolution. People were working on AI since the dawn of computing, and we made a lot of progress in the early 2000s in machine translation, we made a lot of progress in spell checking, and all that. And I think this is just a combination of us having a lot of computational power, and having a lot of data in the internet because of search systems, and the democratization of content publishing on the internet. Without that, none of these revolutions would be possible. So, I think trying to understand this historical context of these technologies is also important.
Ryan Donovan: It’s that time of the show where we shout out somebody who came on to Stack Overflow, dropped some knowledge, shared some curiosity, and earned themselves a badge. So, congrats to Populous Badge winner, @Romain, who dropped an answer that was so good, it outscored the accepted answer. And they dropped it on the question, ‘Django: show the count of related objects in admin list_display.’ If you’re curious about that, we’ll have the answer for you in the show notes. I am Ryan Donovan. I edit the blog, host the podcast here at Stack Overflow. If you have questions, concerns, comments, topics to cover, please email me at podcast@stackoverflow.com, and if you wanna reach out to me directly, you can find me on LinkedIn.
Marcus Fontoura: Hi, I am Marcus Fontoura, Technical Fellow at Microsoft. You can find me on LinkedIn and at fontoura.org.
Ryan Donovan: And how can they find the book?
Marcus Fontoura: Yeah, the book is available on Amazon and everywhere books are sold.
Ryan Donovan: All right. Thank you for listening, everyone, and we’ll talk to you next time.
