The skill of what's worth knowing with Sasha Mitts

Listen

Watch on YouTube · Apple Podcasts · Spotify

Summary

Sasha Mitts, independent researcher studying human-centered AI, on the skill of what's worth knowing — the tension between AI's drive for confident answers and research's need for uncertainty and skepticism; the discernment layer and bias awareness (be upfront about your priors before prompting AI, because the model will not come back unbiased); reproducibility and transparency in AI-assisted research (you cannot say 'the AI said it was true' — build tools that facilitate analysis rather than dumping data end-to-end); a concrete failure story from approximately 1.5 years ago where an AI model misapplied qualitative codes from a codebook, the remedy being multiple providers plus inter-rater reliability scoring (mid-80% agreement); genericism risk (dumping all data into a model invites generic or hallucinated responses — opinionated storytelling through a lens is where research produces its real value); AI strengths and weaknesses fit (the strawberry-R-counting example, the painter-asked-to-fix-a-car analogy, Claude Code and Codex as scaffolding that extends model capabilities beyond next-token prediction, the build-your-own-tools future); Current 101 segment — STOP: exhausting LLM-written content online (write short things yourself, the stereotyped syntax dilutes meaning); MORE: open discussion of AI research tooling (open-science norms for sharing tools and methods, the tools are not the precious part of how researchers do their work); soft skill: build and maintain real relationships with decision-makers (context does not come from talking to a language model — situational awareness, interpersonal awareness, and emotional intelligence are what determine the actual value of any tool).

Guest

Sasha Mitts — Independent Researcher · Supplier-side

From this episode — top claims

• Research must be reproducible and transparent. If a colleague asks why you believe something, you cannot say 'the AI said it was true' — you need to show the full trace of reasoning: the method, the analysis, the sampling. If you use AI not to do the analysis end-to-end but to build the tools that facilitate your own analysis, transparency is retained and you save even more time in aggregate because you have tooling tailored to your actual practice. — Ep. 74, Sasha Mitts
• Research produces its real value from opinionated storytelling — being able to communicate a narrative that is informed, persuasive, and has a point of view. If you ask AI to have the point of view for you, that is where the greatest risk of genericism lies: dumping data in and asking it to tell you what to think brings marginal value. Cultivating the organizational and product awareness to form that perspective is the most important thing. — Ep. 74, Sasha Mitts
• As long as there are new things to understand about the world — which there always will be — research requires curiosity, the desire to go from not knowing something to thinking that thing is worth knowing, and then putting in the work to get there. The fundamental curiosity of having an opinion about what is worth knowing is something that needs to be a skill, not something outsourced to AI. — Ep. 74, Sasha Mitts

Full transcript

0:00 So, I've always found myself drawn to understanding the way people make sense of the world. Before really knowing that AI was going to become this big thing, did AI research all those years ago, maybe like 15 years ago now. Was using early neural networks just to predict random decision-making behavior because we thought maybe there were some patterns underlying what people thought they were doing randomly. You know, we were asking them to pull marbles out of the bag and guess the color, which no good way to do that. But, it turned out you actually can make predictions of a random behavior. And that curiosity over time, we realized that there was not just an opportunity to design better things there, but to leverage all of the data from across the health system and work with various departments to do more predictive understanding of their patients.

0:00 Hello, fellow insight seekers. I'm your host, Molly, and welcome to the Curiosity Current. We're so glad to have you here.

0:00 And I'm your host, Stephanie. We're here to dive into the fast-moving waters of market research, where curiosity isn't just encouraged, it's essential.

0:00 Each episode will explore what's shaping the world of consumer behavior, from fresh trends and new tech to the stories behind the data.

0:00 From bold innovations to the human quirks that move markets, we'll explore how curiosity fuels smarter research and sharper insights.

0:00 So, whether you're deep into the data or just here for the fun of discovery, grab your life vest and join us as we ride the curiosity current.

0:00 Today on the Curiosity Current, we are joined by Sasha Waltz, an independent researcher who studies human-centered AI. Sasha spent over 4 years as a UX researcher at Fair, Meta's fundamental AI research lab, before setting out to run his own research practice. As AI becomes even more and more involved in a research workflows, it brings an interesting contradiction to the surface. These models are designed to produce confident answers, while great research often depends on uncertainty, curiosity, and exploring multiple possibilities.

0:00 Sasha's work challenges us to think differently about what AI is actually optimized for, and whether those strengths always align with what researchers need to understand about people.

0:00 So, today we're exploring the tensions between AI's drive for confident answers and researchers' needs for uncertainty and greater exploration. Sasha, welcome to the show.

0:00 Yeah, thanks for having me. I'm very excited.

0:00 Same. You know, it's interesting, your work really seems to sit between rapidly advancing AI systems and the inherent messy reality of human behavior. And I'm very curious to know what first drew you to that intersection.

0:00 So, I've always found myself drawn to understanding the way people make sense of the world. You know, way back when I decided to study philosophy and neuroscience together in college, mostly because I was trying to understand better how do our anatomy and our ways of reasoning sort of collectively explain various theories of what is really going on out in the world and how we how we make sense of what's going on out in the world, whether it's to do with ethic or theories of mind or theories of perception, these kinds of things.

0:00 Before really knowing that AI was going to become this big thing, did AI research all those years ago, maybe like 15 years ago now, um was using early neural networks just to predict random decision-making behavior because we thought maybe there were some patterns underlying what people thought they were doing randomly. You know, we were asking them to pull marbles out of the bag and guess the color, uh which there's no good way to do that. But, it turned out you actually can make predictions of a random behavior, and that curiosity grew over time. I was later at a hospital and working in a small design group in the hospital that was trying to bring human-centered practices to medicine, which, as anyone who's working in health care knows, doesn't necessarily happen automatically.

0:00 And we realized that there was not just an opportunity to design better things there, but to leverage all of the data from across the health system and work with various departments to do more predictive understanding of their patients. So, rather than react after patients get sick, can you anticipate and diagnose things early on? And obviously, AI in medicine is not a field that we pioneered. It's something that has had legs for quite a long time because of the wealth of data. There have been a few stuffing stumbles. So, the two that I mentioned and then as you kindly introduced, I was at Fair for a while and I think there saw an even richer interaction between understanding people and building completely new technology simultaneously.

0:00 So, it wasn't just how do we add a feature to an app, but how do we change the frontier of what technology can do. And you know, I joined pre-chat GPT when people still would look at me when I said AI and weren't entirely sure what I meant. And you know, now under the modern era where every company has a AI department or head of AI or AI innovation, whatever. But the question still remains, you know, of is hyped right now, but that doesn't mean it's a solution to everything. And there's so many layers to unpack to make sense of how these technologies don't just, you know, show up on a marketing slide, but actually change the qualitative experience for people day-to-day. And it's not the case that you can, at least yet, just ask a model solve all my problems and come back to a well-reasoned good solution that is a fit for you, your users, etc.

0:00 And that ambiguity I think is just fun. You know, like figuring out in a principled fashion how to make use of the tools, figuring out how and where they can fit into the world to not undercut human creativity and agency, but lift those things up. It's definitely a solved problem. Um I would like to say that I'm certain it's going to go well, but I think it actually will require like a lot of deliberate work from a lot of people to go well.

0:00 I'm glad that you brought up, I mean, a lot of those things cuz it's a very interesting space. I want to dig into one of the things you sort of alluded to, which was the idea around just sort of outsourcing the brain capacity and just kind of putting things out there to then create results, to just get it done. Because I don't think the models are there yet to do that. We know that large language models are uniquely trained to produce the best or most preferred response even. Like you see that on ChatGPT, which response do you prefer? It helps to craft the better response for you. On and it learns each time that a unique user interfaces with it. Research, on the other hand, can often mean that it's challenging hypotheses about our current understanding of things.

0:00 So, given those kind of discrepancies, where do you see as the biggest tension between those two different ways of of understanding the world?

0:00 Yeah, and and I want to say first, there I think actually is even a place in research for that more high probability on the rails point of view, as long as you're aware that that's what it is. Right? I think there's like a risk when you look at that and you see that it's convincing and persuasive and you are drawn to trust it, that's where things can can tend to go wrong. But, it is helpful to know, how might an average really intelligent, informed person reason about this thing? But, like sort of, you know, given them the median distribution. And you know, as models get smarter, we we do see that their ability to think in more divergent ways or think along the margins of things get better. Certainly, there's a whole sort of field of context engineering where you give them more of the unique information or the relevant information and they make more informed or maybe like closer to high-quality human decisions.

0:00 But, I think your point is still well taken, right? Like, doing good research requires thinking not just divergently, but skeptically, critically, rigorously, and to be able to have an opinion that you can defend when other people, who may disagree with you, challenge it. I think a lot of this comes down to understanding the models' capabilities well enough that you rely on them for what they're good at, which is writing software, which is looking across large sets of data and finding um but even in those cases, you have to be aware that the way it's so much they're going to find patterns are not necessarily the same way a human would think. Uh and we can talk about this more later, but I think there's a responsibility that researchers have and and any users of these technologies professionally have to equip themselves with enough of an understanding of their strengths and weaknesses that they are not pulled in by the persuasiveness.

0:00 Because yeah, it feels good to be told that you're right or that the problem's already solved, right? Like of course it would be great if I could tap on my keyboard for 30 seconds and then hand over a deck that was perfect and that really actually solved the business's needs. But yeah, we're not quite there and as as you said,

0:00 To that point, let's talk a little bit about what happens when a researcher hasn't maybe built that discernment layer and that ability to kind of be able to to judge whether content is you know, hitting the mark or not. How does AI's tendency to produce confident answers actually affect that researcher's ability to remain open to things like ambiguity, conflicting evidence, changing hypotheses. Like what are the risks?

0:00 I think this actually touches on an important point which, you know, any undergraduate or graduate program where you're learning research skills or you know, even even in younger ages, people will point out uh that there's potential for bias, that there's potential for reaffirming things that you already believe, having unquestioned assumptions. And in some ways, I think AI actually gives us an opportunity to come back to the rigor and methods even more strongly maybe than we have before where if you're the one doing the analysis and you're sharing it with peers, you may think, "Ah, as it's going, I'll sort of catch myself and keep track of things." And like this is obviously not an ideal practice, but I've certainly seen it happen.

0:00 But, if you're going to be handing off large stretches of work that are unmonitored, it's actually even more important to upfront be self-aware and self-critical and think like where am I coming from? What do I anticipate the data are going to tell me? How am I that belief inform the prompt that I write, the way I present the context, the way I draw conclusions from what's presented, and sort of re-prompt the model to continue. So, I would say even if you don't yet have the discernment or think you may not have the discernment to interrogate a model's responses and its reasoning, be really clear upfront with all of the baggage ideas that you're bringing to the research. Obviously, we're not perfectly self-aware.

0:00 This is an area where collaboration with another researcher can help, where you can say, this is what I I'm sort of assuming in. Do you think what is sort of what's top of mind for you? You know, like maybe write these things down even before you share them to avoid influencing each other. And then recognize that the AI is not going to come back in an unbiased fashion. Um and part of actually what I advocate for and then what I suggest other researchers do is don't expect that dumping in all of your data and asking for an analysis is going to give a perfect answer. And I say this for two reasons. One is it might give a good answer, but maybe not, and it's hard to tell all the time. But, the other is that the process is not easy to interpret or interrogate.

0:00 And the reason that's a problem is research should be reproducible, and it should be transparent. If you know, someone on my team comes and says, why do you think this? I can't just say, well, the AI said it was true. Well, this was the method I used, this was the method of analysis, this was, you know, the sampling, et cetera, all of these things. And I can show them the whole the whole trace of reasoning. And if you use AI not just to do the analysis end to end, but to build the tools that facilitate your own analysis, a lot of that transparency can be retained, and you can still save a ton of time. And in aggregate, probably even more time than you would just dumping things into the model because you have tooling that is well suited to what your actual research practice is, rather than hoping that each time you run the model, it's just going to magically pull all the right pieces together and give you an output that is correct.

0:00 I love that point about the reproducibility. It's a good one. Yeah.

0:00 Well, let's talk about a concrete example. We know that predicting trends or not, sometimes it just straight up makes stuff up. It can just hallucinate in a wildly convincing way. Um so, I'd love to to walk through a moment with you where there was a sort of obvious, but maybe perhaps slightly buried AI-generated insight, or even maybe not a synthetic created research insight, but even just an analysis of a set of data that truly wasn't the real story.

0:00 And I think this is actually instructive that even as we move along this path, there's room for the tools that we're building and the tools that we're using to not be perfect. And maybe a good example of this is This is maybe a year and a half ago. So, this is earlier model in AI time, this is like 3,000 years ago, right? Even when I was sort of trying to take an early version of my own advice and build more replicable processes and build things that are sort of small, discrete units that can be stacked together. One of the things that I often want to do is have an AI work with me to create a first a codebook to understand what patterns exist in a set of qualitative data. And then a separate set of models to apply those codes.

0:00 And in each of those steps, you can imagine how a person might either do that on their own or oversee an AI doing that work, where you maybe read lots and lots of examples of responses, you write up some ideas of how the coding might go. You might do it the other way around where you propose codes ahead of time and then see whether or not your data maps to it, right? There's many methods around this. There was an instance where I have a codebook that I felt reasonably good about that I worked with a model to produce. And then the next step is go through a bunch of data and apply the codes. And one benefit of using AI, if you know you're going to have access to it, is you can collect more and more complex qualitative data than you might otherwise, you know, whether it's an survey or an AI-moderated interview or in diary study or all these other things.

0:00 There's an temptation to lean on that, but as you lean on it more, especially early on, you're not necessarily sure if it's not going to do a good job with the coding. There's there's potentially a risk there. And so there was a a model I was using that was not doing a great job applying the codes that we'd identified, but that was not obvious at first, right? Because like the codes were applied, it wasn't making up new codes, but it was saying this code applies to this question, this code applies to this question. And there's a step, of course, you have to take of like, well, let's, you know, let's see if this is right. But that is time-intensive and there's pressure to write up insights. So I'm starting to write what the conclusions I might draw from the distribution of codes that have been assigned and the inferences I'm making from those.

0:00 I'm like, I am not 100% sure this is consistent with the samples I read earlier on. I'm not 100% sure this is consistent with the quantitative data coming out of the study. Is it some interesting thing where the quantitative and qualitative are in tension or maybe there's just something going off. And it turned out that that model was just not strong enough, was not up to the task. It was applying codes erroneously and that took, you know, obviously some human effort to to correct. One concrete learning that I would say like a takeaway I'd recommend in for anyone taking on a similar process is use models from multiple providers and measure the inter-rater reliability between them. There's a number of statistics you can compute to understand where the where divergence exists.

0:00 Certainly many models now are sufficiently strong that I wouldn't expect them to do a bad job, but they will still disagree with each other. Even when I use now, you know, like medium-size frontier I I guess a current-generation medium-size models to do these tasks, the inter-rater scoring only really ever gets to like the mid-80% which is still higher than you might expect from humans over a really large sample and certainly enough that if you interrogate the disagreements, you can adjudicate it with a stronger model or by yourself. There's like lots of ways to handle this, but it this was certainly an instructive example of are things as good as they seem early on in building the tools not so much and certainly with earlier models not so much.

0:00 That answer hearkened back to me Cronbach's alpha. Is that do you I just haven't thought about that statistical test in a long time, but it's for labor rate of reliability. Yeah.

0:00 It's my earlier point on methods. Like as you're doing this, you are forced to return to the ways you decide between methods very carefully because you're encoding them in a system. You know, like you want to build it and then have it be reliable and repeatable. And if you're doing it one-off by yourself, I'm like, "Okay, I'm just going to like change the drop down or change what I'm doing in Excel or in John, but whatever, you know, whatever analysis software you're using." But when you're deciding, "I'm going to run this kind of study 10 times in the next 6 months and I want to ensure it's really tightly tailored, you're going back and you're looking at the equations and you're comparing the tradeoffs.

0:00 Maybe you're talking to an AI about why one or another might be well-suited the data and when you might want to flip it and it's a little bit time consuming, but it's also fun. It's something that yeah, maybe AI is weirdly inviting us to be more in the weeds than than even before.

0:00 It's such an interesting way to think about it. Yeah. Well, let's get a little bit practical too with some tips from you. So, what tools or practices can researchers use to keep an AI generated explanation from becoming that default hypothesis too early?

0:00 I mentioned a couple of these things, but I I want to touch on them more. So, one is certainly being upfront about what you think is true or is not true going in. Another is if you decompose the steps enough, you're not actually ever asking the AI to reason across the data and the hum the more high-level and the end end your expectations of it are, the more there is a risk of genericism because at that point, maybe it's consuming, you know, depends on your process, but maybe it's consuming thousands of responses, tens of thousands of responses, you know, in a large survey with multiple open-ended questions or whatever, and even though the model can fit all that data into its context, asking it to reason carefully about it with no oversight invites generic responses, in the case of some models maybe higher proclivity for hallucinated responses.

0:00 It's not fully avoidable, right? And like there are going to be cases where the data are rigorously analyzed, and the narrative becomes the area where it feels a bit generic or weak. And that is really on the researcher to say like, okay, well, the AI has assisted me with maybe cleaning the data, maybe coding the data, whatever it is, applying statistical tests that I've selected. It's now up to me to go through and think, what what is the story here? And I think an open question of like, where does research produce the most of its value? And I think a lot of it is from opinionated storytelling. Right, not you know, stories in the invented sense, but being able to communicate a narrative that is informed and persuasive and that has a point of view.

0:00 Through a lens.

0:00 Yeah, exactly, the lens the lens. And I if you're asking AI to have the point of view for you, I mean, that is where the guy I think the greatest risk of genericism is. You're saying, well, here's the data. I'm going to dump all of it in. It's going to tell us what to think. I mean, at that point, you're really bringing pretty marginal value to the equation. I I can't tell everyone how to what what point of view to have, but I would encourage that maintaining that perspective and cultivating the organizational awareness and product awareness to form that perspective is maybe the most important thing.

0:00 Yeah, great advice. And that gets to something that I think is the core of what I'm thinking about, specifically the research impact or the business impact. Research at a fundamental level is helping to discover things that we don't know yet, or to have an understanding of human behavior that maybe hasn't been captured in any specific ways so far. Um I think about who would have thought that we would buy out toilet paper and cans of beans in the middle of a pandemic. Like there's all these different types of things that is just is really hard to understand. So, how should research researchers be using these models in a way that helps to achieve that goal when they're designed to predict something that is perhaps the most likely outcome when that's not actually a mimic of human behavior.

0:00 I think this is an interesting question. There's maybe a few different threads to it. So, it's it's true that there is some probabilistic sequencing of words, but a lot of what steers that is the context you give and the kind of question you ask and the kinds of counter examples you also maybe append to your prompt. If you ask the model, "What's going to happen in the next pandemic?" you're not going to get anything useful, you know. I think if you have an opinion and you have data and you have pre-existing hypotheses that you want to test or you have questions that you think are unanswered and you want to understand maybe what data to try to collect to better address those questions, I think those are some of the areas that AI can have value.

0:00 I've been harping on this point a little bit, but AI is a tool that needs to be slotted into the kinds of things that it is good at doing. For example, like there's this, you know, recurring example, AI is not good at counting. It's not good at telling you how many times the letter R shows up in strawberry. And it's not because the models are bad or good, it's because the kinds of things they're good at are not similar to the kinds of things human brains are good at. So, if you try to lean on them and say like, "Well, do my human labor end to end." I'm going to not pay any attention to where your strengths are and where your weaknesses are. That's kind of like a recipe for failure. And it's the same with a person, you know, if I go out and I'm like, "Who's like a famous painter?" And I say like, "Ah, it would be really wonderful if you build me a car." And maybe I get lucky and they happen to be a good mechanic.

0:00 But like, it's probably not a good bet. And this is even more extreme with AI because we see them do some things maybe better than 99.9% of humans could, right? Like, there's lots of AI now AI's now writing mathematical proofs and understanding the sciences better than you know, almost anyone alive will. But that doesn't mean that they can, you know, walk your dog for you. I mean, maybe not. Robots are coming, but not quite yet. Um, and the models that will under underlie them are a bit different. So, I think that's one thing, right? Is if you're trying to do something that they're not good at end to end, help them do the parts that, you know, assign them to the parts they're good at. The parts that they're not good at, you can do on your own.

0:00 There's gray area where they could do it, but they need scaffolding to do it. And that's where we've actually seen a lot of progress. Like, Claude Code and Codex and a lot of these tools are valuable because they give the appendages, whatever, you know, like mixed analogy you want to use. They they give the the tools and the facility to models to extend their capabilities beyond what they can just do by predicting the next token. And more and more, I think it's going to be the obvious step to build your own versions of those things, not necessarily replacements, but additional ways to augment the capabilities so that when you have a request, it's not just, "Can you predict the next word?" But, "Can you use the tool I designed for you to go do this thing that on your own would be hard if you were just trying to spit out language?"

0:00 Yeah, I think that's a fascinating way of thinking about the use case for it and that we in the research space are responsible for thinking of it as a tool that has a use case and it's a change in the workflow. It's not a change in an outsourcing the mental uh the mental work that goes into any type of research work.

0:00 Yeah, I think that's exactly right. As long as there are new things to understand about the world, which there always will be, it requires this the lens we were talking about, it requires curiosity, it requires the desire to go from not knowing something to thinking that thing is worth knowing, and then putting in the work to get there. And amazing if AI helps us accelerate answer more questions, answer questions differently, more deeply in some cases, there's a ton of value there, but the fundamental curiosity of having an opinion about what is worth knowing is something that needs to be a skill.

0:00 Yeah, the skill of what's worth knowing. That's the name of this episode. If it's If it's not the name of this episode, it should be.

0:00 Uh well, Sasha, we have a segment that we run on every episode that is called the current 101. Uh we ask all of our guests the same question. In your experience, what is one trend or practice in UX research that you would like to see stop, and what is one thing that you would like to see more of?

0:00 First response is just an appeal to everyone even outside of UX research. The amount of LLM written content online is exhausting. It's not It right in the individual case it makes sense. There's something I want to write, I want to make sure it sounds professional, I want to make sure that search engines index it, whatever, you know, whatever your reason is. But the aggregate effect is this very tragic, depressing state of content online. It's not just, you know, like recreational social media, I see it on professional blogs. I think if you want to write something concise, simple, whatever, just write a short thing. Most of these LLM posts are not like that thought-provoking, you know, they really only make one or two points.

0:00 Write down your two points in two sentences. Do whatever you have that you put it on Twitter, put it on LinkedIn, like be done with it. Or if you want to write something interesting, write a long thing. If you need help writing a long thing, that's fine, like, you know, collaborate, but do the thinking or work with it on the thinking and write it yourself or or something. And I'm not just saying this because like like I'm bitter and it's boring, whatever. But, to the earlier point of this genericized routine content, this is an area where it kind of crops up in its most obvious form because someone may say like I watched this, you know, like keynote that some CEO gave. I want to like say a, you know, intriguing thing to my audience about it.

0:00 Write me a post. What was was the motivation that you actually heard something interesting and you want to share it? Okay, if so, do that. If not, you don't need to comment on it. You can just, you know, be silent if you have nothing that you actually if no thoughts have come to mind.

0:00 Right. Yeah. It reminds me of the I'm going to get this wrong, but it's the that typical it's not this, it's that, right? That that you can always spot a mile away. Yeah.

0:00 The syntax and rhetoric of AI is partially nauseating because it's so stereotyped to each model, but at first it actually sounds really good. I mean, this is how it gets it gets done. You know, whenever Claude 6, GPT 6, you know, and like in whatever 6, 12, 18 months, the first little bit of how those models talk might actually sound really novel, it might sound really compelling and well-written and articulated and interesting, but then everyone catches on to that and everyone's using it to write, and so all of a sudden it's it was always the case that the meaning was diluted, but now it's obviously the case that the meaning is diluted. Yeah.

0:00 And what's the second part of that, too? The see more of.

0:00 You know, this is somewhat selfish, but I would love to see more discussion of the tooling and approaches people are taking to use AI to do good research. I've seen a flavor of this, which is people saying here are my prompts or here is the agent that I like configured in, you know, whatever interface, and I think that's fine. Uh leaving aside that a lot of people want to put these behind the paywall, I think that's, you know, I'm somewhat neutral on that. I think there's there's room for both more more open source and more productized versions of this. But I don't see as much discussion of ways to build end-to-end or even in some substantive subpart the tools that people think are well suited to do research.

0:00 And I think that's a little unfortunate just because there's relatively well understood, well defined methods that we all make use of every day. And it shouldn't be the case that every person needs to struggle in the dark through how do I get this to turn into a tool that is useful and repeatable, etc. It's not that it's so hard, right? If you describe what you want and ask Cloud to build you a little tool, it will usually do a good job. But that's not how science has worked historically, at least in, you know, recent history of open science is in academic papers where you communicate the thing that you built, how it works, how someone else can use it. This happens somewhat more in academia, but a bit less in industry for, I think, I guess somewhat obvious competitive reasons.

0:00 But I'd like to think that the tools that we use to do analysis are not the precious part of, you know, how researchers in industry do their work. It's that they are applying to highly relevant, you know, valuable questions. Um so I'd love to see people having more of a conversation. And I may just be missing those conversations, so if so, please let me know where they are.

0:00 Well, so this last question that we have for you boils down to what you do in in your work, which is for researchers navigating a very clearly AI-driven future. It's becoming an expectation that people are versed in this, and it's the nuance of how they do it. It's not just do you have it or not. What's one soft skill or perhaps a uniquely human skill that professionals should consi- continuously work on mastering?

0:00 The first thing that comes to mind, and I I I really do think this is the most important point is to build and maintain real relationships with the people who are you are making decisions alongside. I mentioned earlier, right? That like how much the perspective and context and conviction and curiosity, all of these things are what determines the actual value of any tool, you know, whether it's something as simple as your email or something as powerful as AI. You have maybe your own opinions about where to steer it, but getting leverage and having a more informed perspective and seeing your work go as far as it might go and making sure that you're actually doing the work worth doing relies completely on understanding the social and intellectual and economic ecosystem that you're living in and it doesn't just happen by talking to a language model all day.

0:00 It doesn't just happen by reading the docs that were posted on your internal workplace. It comes from having the conversations with people, understanding the maybe unspoken priorities or the rapidly changing priorities or what was previously tried. It's something that more and more of this context might get recorded, it might get uploaded, but reasoning over it requires situational awareness and interpersonal awareness and emotional intelligence, um which are not things that we've really seen AI be good at yet.

0:00 Yeah, the things that are not something that are likely to be outsourced effectively.

0:00 Yeah, and you know, at the end of the day, even if AI can get good at say reading a room visually, maybe it will have more camera inputs. That doesn't necessarily mean that setting perspective should be abdicated, right? Like the we we do all of these things, you know, all of the businesses that exist have a point of view. Maybe you disagree with it, maybe you like it, maybe you don't, but each business has a thing it is trying to make and sell and those are human opinions about why they even want to be in that business in the first place, why a product should be what tweaked one way or another. There's not a right answer. We're all just trying to figure out you know, what what is available to us and how can we um whether it's make money off of it or make someone's life better, you know, whatever your motivating factor is.

0:00 But even if you can say to an AI like help me do this thing or do this thing for me, it does not mean that every one of those decisions needs to be outsourced.

0:00 Well, Sasha, this was a fascinating conversation. I feel like we have a lot of these different AI-related conversations on this podcast, but this was a really refreshing conversation and an interesting perspective.

0:00 Yeah, thanks for having me. It's a really fun to be asked the questions and to think through with you guys.

0:00 Well, Sasha, it's been really wonderful to have you on the show today. One thing I I'm really going to take away from this conversation is the idea that good research isn't just about finding the answers or finding the fastest road to the answers. It's knowing when a little bit of uncertainty can be valuable.

0:00 Absolutely. I also loved that reminder that while AI excels at recognizing patterns, research often depends on questioning those patterns and remaining open to what hasn't yet been considered.

0:00 Right, and your perspective on stewardship and how to use these tools really stood out to me, too. As AI becomes more capable, the challenge really isn't deciding on what these systems can do, but what they should do in service of better research and better outcomes. I think your example of you wouldn't ask an artist to fix your car is really interesting because you're right, it's very tempting to outsource everything to an AI tool, and instead of an artist that may just tell you, "I can't do that." The AI tool never tell you that it can't do that. It will try its best. So, it's up to us um to really discern what it can and can't be used for.

0:00 Sasha, thank you so much for sharing such a thoughtful perspective on AI, and certainty, and what it means to practice responsible research in a rapidly changing world.

0:00 And to everyone listening, thank you so much for being part of the Curiosity Current. We'll see you next time.

0:00 The Curiosity Current is brought to you by AYTM. To find out how AYTM helps brands connect with consumers and bring insights to life, visit aytm.com. And to make sure you never miss an episode, subscribe to the Curiosity Current on Apple, Spotify, YouTube, or wherever you get your podcasts. Thanks for joining us, and we'll see you next time.

Produced by aytm

Curiosity Current is made by aytm, the consumer-insights company. Guests speak for themselves; the synthesis is ours. About aytm