Saket Kumar — From Data Pullers to Insight Architects: Gen AI in Enterprise Analytics

Listen

Watch on YouTube · Apple Podcasts · Spotify

Summary

Roughly 80% of enterprise data work is reporting and visualization, which is why Saket Kumar sees conversational analytics as generative AI's highest-leverage entry point: hundreds of dashboards collapsing into one interface. The conversation maps a three-phase adoption path, training, copilots, then products, and argues the analyst's role isn't disappearing: it's shifting from data puller to insight architect.

Guest

Saket Kumar — VP of Consumer Analytics, Citi · Brand-side

From this episode — top claims

• Four techniques together control LLM hallucination risk in enterprise products: rule-based hybrid checkpoints, traceability/explainability of each step, human-in-the-loop review, and grounding through fine-tuning or retrieval-augmented generation. — Ep. 4 · 18:05, Saket Kumar
• AI governance is not optional; zero-trust architecture with role-based access control is essential because a single model holding broad context creates real information-leakage risk. — Ep. 4 · 27:00, Saket Kumar
• There must be a balance between a generalized AI that handles broad tasks and a fine-tuned AI grounded in the organization's specific context; tilting too far toward bespoke tuning yields a hard-coded system, while pure generality misses business-specific KPIs. — Ep. 4 · 40:25, Saket Kumar

aytm's take on this conversation

The shift from data puller to insight architect is real, but it stalls wherever the warehouse stays fragmented. Build an LLM-ingestible layer encoding KPIs, metadata, and strategy first, then let descriptive automation free analysts for the prescriptive work models still can't do.

aytm's perspective, voiced by Stephanie.

Full transcript

0:04 Stephanie Today, we're delighted to welcome Saket Kumar, VP of consumer analytics at Citi and a trailblazer in the world of data science and analytics. Cicada has an impressive background that spans industries like aerospace, energy, and finance with a passion for harnessing advanced analytics and AI to drive innovation and empower teams.

0:26 Matt From building digital frameworks at Arcadis to leading transformative projects at Sidious Saket has a deep understanding of how technology like generative AI is shaping data analytics and customer experience. Today, we'll dive into his expertise on how large language models, uh, like GPT are revolutionizing analytics, bridging enterprise knowledge systems, and driving actionable insights. Saket, welcome. Yeah. Welcome.

0:50 Saket Sure. Thank you for your kind words, though. Yeah.

0:52 Matt Yeah. Of course. We're really happy to have you. So just jumping right in, you know, Saket, your career journey has been pretty extraordinary and and unique, you know, spanning a lot of different industries. You've been in aerospace. You've been in energy. Um, you've been in finance, all while staying really at the forefront of analytics and innovation in analytics. What really sparked your interest in data science and analytics? How how is your perspective on its on its potential evolved over the years?

1:25 Saket Oh, sure. Um, so my journey in data science and GenAI and artificial intelligence, it has been intertwined by with my curiosity of how data shapes the world and the power of AI to extract the meaningful insights. Uh, it all started back after my engineering in aerospace, and my starting career was, uh, into data engineering. And in 2013, there was a whole shift of paradigm to cloud computing where the traditional database warehousing was transitioned to this whole cloud architecture to scale, like, real time analytics. So I was part of a big aerospace client where I worked with them to transition their traditional data warehouse to a cloud databases through which they can, uh, monitor in real time how jet engines are performing in real time. Um, so, yeah, I mean, that's how the career started. Like, for two years, I worked at as, like, data engineer. Then I moved to automotive industry where it was more on the data analytics and the statistical analysis of, uh, finding the failures of the models, you know, like, where, um, the equipments of automotive industries and parts, like, they are, like, uh, you know, uh, they are, like, uh, creating other survival models to make sure, um, that the vendors are actually supplying parts so we can assist their quality with that. So that actually helped the industry to also, you know, um, to gauge the performance of their parts and also increase their revenue. Uh, I think, like, these two were my setting, uh, step to start my career in, and then it went into something really beautiful where I got the opportunity to work in the energy and the environmental sector, uh, in Washington DC, uh, where I was leveraging data science and AI for water commission, uh, for transportation planning, uh, for traffic planning, and also, like, environmental remediations. Uh, these projects were, uh, very, like, impactful when it comes to improving the quality of life for the people, like what we talk about. So kind of like AI for good. So that's where I saw that shift going from there. I worked for almost four years in that industry and then moved to finance and marketing. So now I'm leveraging more of the Gen AI capabilities to transform how the complete data analytics can revive through, you know, the power of Gen AI and large language models, what we talk about. So it has been pretty exciting journey to be here.

4:09 Matt That's very cool. I. So, you know, in full transparency, I don't consider myself a data scientist at all. So, like, um, a lot of the questions I will have for you, you're gonna feel like, you know, you're you're explaining what you do for a living to your parents or something. Um, what give me an example of, like, how you have used, um, like, a large language model such as GPD, uh, to really change the way an organization that you've sat in has approached data analytics? Like, what does that intersection look like?

4:45 Saket Yeah. So that intersection is, um, is actually based on the power of large language models. So just to set that up, right, uh, there was a a very transformational or a very disruptive shift in 2021 when this GBD got released in the market, and then Charge GBD came, like, late later on top of it. Um, the power of these models, like, is about it can understand your questions like humans. Uh, it can orchestrate and create tasks like what to do based on those questions, and then it can write content on top of it. So it really has these three key structure to it, which is so much powerful, like, compared to before with the, like, traditional deep learning models used to because it used to just, uh, perform one task at one time. Right? Um, these, like, what we call it, like, a large foundational models. So it's just one model which does a lot of, uh, things at once. Uh, but but just like a paradigm shift if we talk about it. And as an organization, to you to leverage something so powerful, it it can become, like, very, uh, you know, like, intensive when it comes to and, uh, a lot of organizations can fall into this kind of pits of investing in something which may which might not drive your revenue because there is something cost that comes with it. So the best approach to start with it was to focus on using AI as a thought leader rather than as a task doer. Right? Mhmm.

6:21 Matt Um,

6:21 Saket so thought leader and also as a thought partner. So now you can actually engage with these models and talk to them, and they can be your copilot. So doing data analytics or any kind of summarization of reports. So we took the these kind of models. And one of the basic things that what we were trying to work on was, like, conversational reporting, which is, like, I will say, like, 80% of the data task in any organization is all about, like, reporting and data visualizations, you know, and the these are, like, regular reports that come into. So that's the really, uh, you know, like, a good area to focus and, uh, leverage in AI, which can solve most of the problem. Um, so one of these things where large language models are really good is to understand your question and translate that into a data question. Right? So you have, like, a lot of, like, big data tables, like, lying behind all these companies, you know, like, data warehouses, like, data lakes. And what GenAI can actually do is to translate your question into a data question or maybe you can and it can generate, like, a SQL, uh, statements, like a code for the for that question. Retrieve that information and create, uh, textual insights, uh, create, like, data visualizations, like, dynamic. So now you're, uh, reporting to look at each dashboard, like, hundreds of dashboards that just boils down to one conversational UI. And you can ask whatever question you you can have, and then it can create a data story as well to find what are the, you know, like, outliers that that you're seeing, why is trend going up, why trend is going down. So find out these key drivers that can drive that. So that actually brings down the time to insights, like, really, really fast, which is where the leadership, uh, you know, like, gets a lot of value out of that to drive and understand the numbers in a in, like, a very short amount of time. Amazing.

8:27 Stephanie That's great. Yeah. And and we definitely see that in our own business too. So it's, you know, it's it's definitely something that we can relate to and have found to be, you know, a big time saver. And I love that idea of of uh, a thought partner more than an executional, you know Right. Mhmm. Arm for you. So this next question, it may be a little bit, uh, redundant with with some of what Matt was asking about. But I'm just really curious, like, if we zoom out a little bit. Mhmm. How and and think about, like, current state and how adoption looks. How are you seeing generative AI or LLMs like GPT shifting the way that the businesses are actually right now approaching data analytics? So not only in your role, but, like, when you think about the, you know, industries more broadly. And I think I'm especially curious if you think there are specific capabilities of LLMs that have become, like, the lowest hanging fruit for organizations that are, like, the most immediate to adopt and see value in?

9:31 Saket Oh, sure. Um, so, uh, like, starting with the, you know, uh, like, how these adoption can be increased is to first, I would say, there will be requirement of a organizational change. Uh, it's kind of like a change management process because it's a new kind of technology. Uh, most of the employees end up, like, not having AI background. They can get stuck to how to use that. Right? So I think, like, in my organization itself, like, we started with a lot of, like, training modules, like, bringing in AI innovation partners from the industry. And, uh, you know, like, engaging the workforce to learn how to use AI, uh, not to, like, become, like, developers because that's a different, like, you know, like, area to go into. But you don't need really a development background to learn how to use these tools. Most of these tools are very, uh, you know, like, they have the capability to do these tasks. It's just that you should know how these models understand and respond. Uh, how do you want to provide a set of instruction, like, in a clean and concise way, uh, giving a lot of context. Uh, so that I think, like, that knowledge, the workforce needs to be trained on, like, how to interact with AI, uh, especially with these kind of models. And most of the organizations should also like, to increase the adaptability of GenAI, they should start working on these co pilot projects first, where AI is working side by side adversely with the workforce, and it is improving their, uh, you know, like, the efficiency of the operations rather than, you know, like, doing like, replacing their task itself. Uh, so that can help, uh, work the employee and the workforce to have more, uh, like, reliability and, you know, trust in AI if they end up, like, working with AI for a long time. And once they are comfortable, then I think then organization should shift the gears towards, like, product engineering, more on the product design. And because by that Copilot journey, most of the organizations can learn the pits and falls, like, where it can go wrong, where it where it is performing well, and where it's not. Uh, so then you take all that learning, and then you start, like, uh, creating these products specifically, uh, to perform task where AI is not just your Copilot, but it is actually giving you and, like like, doing tasks for you. So it's not just a thought partner. No. It's a action taker as well. So I would say that three step, uh, like, uh, like, chronologically, it makes a lot of sense. Uh, a lot of our organizations, if they just start with, like, product design, they have lesser idea what will work for them and what and what will not, and they will always end up spending a lot of money, uh, in this proof of concepts and which will not even serve the need. So I would definitely suggest, like like, following, like, these three paths, like, in their subsequent order.

12:53 Stephanie Makes sense.

12:54 Matt Yeah. I'm curious. You know, in the research world that, like, Stephanie and I sit in, uh, AI, generative AI, LLMs is, like, all people can talk about. Right? It's like it's it's, uh, definitely the hottest new thing whether, um, the organization hates the idea and is just, you know, completely adverse to you know, they they think robots are coming for our jobs, or they've really embraced AI in in really meaningful ways. It's really like a spectrum in anywhere in between. I'm curious if you see the same thing from your your perspective in in data science. Like, are are there different personas when it comes to AI? Like, are you seeing organizations that, uh, are really reluctant to lean into it versus some organizations that are really embracing it overall when it comes to data science. Uh, do you think that that that role is making the most out of the AI tools that are at their disposal today?

14:05 Saket Yeah. Yeah. So I think, uh, a bit of reluctancy is good, uh, for some of the organization. And I can definitely say that the this reluctancy comes from the fact that their data warehouse or the information that they want to train these models are not organized enough. Right? So a really good data governance and a, like, really robust data management policies, if they are not in shape, uh, in any big organizations. And mostly, I'm talking about the big corps. Uh, that's where you have, like, a huge amount of datasets, like, which are not, like, really regulated most of the time. And that can impact your product, like, Gen AI product, uh, product engineering a lot. Uh, and, uh, especially when you will end up, like, getting all these, like, the like, falling into thick, falls and pits of this, uh, where the information that your Gen AI will be trained on is not good enough. So it will not be serving the task that you are, like, working on. So I think, like, that reluctancy is good to have just, uh, to start with, uh, metadata management properly and and managing your tables. Because if you see the journey, like, how data analytics is evolving, we started with, like, Excel, like, fifth fifteen, twenty years ago. That was, like, siloed, like, Excel spreadsheet all over. Then business intelligence came in, and, uh, the beauty of business intelligence was was not just a live dashboard, but also it actually pushed the organizational leaders to create a really, like, robust data governance to manage tables, to manage their relationships, uh, and then came AI. Right? And with AI, your responsibility of managing data is even more important than it was ever before. Uh, so that's where I think the more AI, uh, you know, like like, being evolved, like, all the time, the better data governance that you will need. So now the, uh, the organizations need to, uh, manage their structured datasets and also manage their metadata, like, the information about your data, like, really efficiently and also the unstructured datasets that they have, like, in terms of the reports. So it really needs, like, really good management to even be able to get into this kind of, like, shift in the industry.

16:37 Matt Makes a lot of sense. Sure.

16:39 Stephanie Yeah. It does. I mean, it's such a good point about there there's a lot of work beforehand that needs to be done before you can really leverage these tools, uh, to to their fullest extent. Um, another challenge that we often hear discussed, uh, with generative AI is balancing the accessibility of information, which, you know, is we're just we have a wealth of things at our fingertips that can be summarized so easily and quickly and and analyses that I couldn't run on my own, but that, you know, with the right script, I can. And so, you know, this beautiful democratization of data and insights, but we also need to balance that with, uh, accuracy and avoiding oversimplification or misrepresentation, especially among people who are not, you know, experts in that domain. And we know that that's what generative AI is going to allow people to do is to is for other people within the org to be able to put on that that analytics hat and that, you know, insights hat. And so, I guess, what strategies have you seen work well? And I'm thinking too, like, you work in finance, which is such a high stake industry. How do you think about tackling that balance of accessibility with, like, accuracy and proper representation?

18:00 Saket Yeah. Yeah. So there are, like, three to four ways to handle, uh, these kind of scenarios, especially, uh, generative AI or, like, large language models are prone to hallucinations because they are trained on, like, so much vast amount of data, and it's just one model. So chances of, like, uh, creating its own context without just focusing on your context is very much possible. Uh, so in the organizations, I think two approaches are very viable and easy to, uh, you know, implement as well. One is like a hybrid model, which is your product is not just, uh, based on large language models, but it has some rule based systems itself as, like, checkpoints, which can at every step, it you have to push in some of these kind of controls of making sure that the all of these steps, like step by step, what LLM is doing, is getting traced and also being evaluated through these rules. And if they violate these rules, then you should not, like, respond or ask for more context or more questions. So that hybrid model is something which product design team should always, uh, you know, like, have this in place to build this, like, checks in terms of, uh, automating this control of the Gen AI or, like, large language model uh, responses. The the others, uh, like, really good approach is explainability mechanism. Right? So whenever Gen AI gives a response, the product design team should also think about that. It should also, like, explain itself, like, how did this answer was generated by large language model. So we call it, like, traceability. So you can create these traces. So if I ask a question like, why is my accounts are performing low in last thirty days, it will create that task, find that answer, but then also in traces, it will show me, like, I did step one this because so it sky it's kind of like having a reasoning, uh, because large language models also work chain of thoughts. Right? Just like one thought after each thought. So it really has that capability to explain itself. Uh, so these two features in any product will really help. Uh, the other two approaches, uh, are I would say it's, like, very case to bay like, case to case base. Uh, one is human in the loop, like, HITL approaches where, uh, it it depends upon how the organization, like, runs that operation, but that can also help. And through AI based models, you can also fine tune your models. Uh, there are different ways like fine tuning, or you can also use something called RAG, which is retrieval augmented generated, you know, uh, techniques which can, uh, ground your AI knowledge to just focus on what it is supposed to do and not, like, hallucinate outside the rails. Um, but definitely, these four techniques can help you to make sure that accuracy is met, and it is not producing any response which is wrong.

21:22 Stephanie That's great. Yeah. Thanks.

21:25 Matt I'm curious. I I just wanna ask you flat out. Well, I'll I'll take a step back. I think, again, looking at, like, the research consumer insight space, I would say and, Stephanie, you can you can agree or disagree with me on this point. I'd be curious. But when we think about how we're using AI tools, oftentimes, our conversations sound more like an evolution of the role of the market researcher. So it's the market researcher now has these new capabilities. Their expectations of them are going to change. Um, their daily work profile is going to change. Sure. But they're still going to be there at the desk conducting market research. Of course, there are fears about displacement and all that.

22:17 Saket But I

22:17 Matt think, at least from from my view in consumer insights, largely, we've been looking at AI tools as an augmentation of our capabilities. I'm curious if you see it the same way in data science. Is it is it a superpower? Is it an augmentation? Or is there a risk for displacement?

22:42 Saket Yeah. Yeah. Uh, I think, uh, it's more of, like, redefining the roles rather than replacing the roles. Uh, of course, so, like, once, uh, an organization puts Gen AI products in place, uh, a lot of tasks, like, repetitive tasks can become automated, which for which if you needed, like, 10 employees, you can do the same task in, like, three employees. So in terms of replacement, I won't say it will really replace a lot, but it will create a new kind of, uh, or a new way how the data analyst or data scientist work in dust in in the industry right now. Because, uh, traditionally, data scientists and data analysts spent a lot of time just pulling the information at the first place. And that used to be, like, 60% of the tasks, uh, that where they have to just get the information. And then 20% of the time that they have, they will invest in creating insights, like building these models and everything. But now I think that getting the information is faster. So the data scientist and data analyst now have to transition their roles from data pullers to actually strategic advisers because now they have enough time to work and, like, re like, redefine the insights. Right? And they can focus more on, like, like, re like, designing AI driven, like, workflows and refining, like, model output. So it's kind of, like, more efficiency. But, of course, uh, that's kind of, like, paradigm shifts that we are seeing in. And the workforce, uh, also has to make sure that they are learning these tools, like, really well, uh, because now the roles will be shifting to becoming, like, more of, like, insight architect, I would say, rather than data analyst. Uh, so that's where I think it's going. So, I mean, in short, it will alleviate the data teams completely and, like, free and it will, like, free them from repetitive tasks and enable them to have these kind of high value strategic work.

24:55 Stephanie Right.

24:57 Matt Yeah. That's I that that's what we see a lot in research as well. Like, it it's okay. It's no longer good enough to just report the news and and provide what was found. It's like, you know, the expectations now are you should be able to spend more time on crafting the story and, you know, being able to sell in recommendations in a really kind of impactful or more consultative way, which, you know, for for a lot of, like, research analysts, I know that's just a total, like you were saying, a total mindset change. Um, so it's been interesting to see.

25:30 Stephanie But but largely positive. Right? Like, that's such a positive, like, uh, I think framing of of generated Bayview and the and the power that it brings. Because I I don't think I've ever met a a researcher who was like, no. I just dataset to get it into the right shape for the you know what I mean? Like, that's the the parts that we do because we have to. And so I think the more that we can think of this as a tool and a copilot and thought partner and and really do that more strategic work, the more exciting that it gets. Um, so Kate, to kind of, like, switch switch gears a little bit, um, you know, as companies are starting to embed these models into their analytics process processes, data privacy and governance are becoming ever more critical issues. And they were critical before, but we are certainly finding that we're you know, the expectations from clients, the kinds of certifications that they're expecting, uh, the kinds of concerns that they show up with as relates to data privacy are, you know, uh, order of magnitude higher in in the in this current, uh, world order. And I'm I'm curious what best practices or frameworks have you found or do you believe are essential to ensuring that we're being ethical and responsible in using these technologies?

26:49 Saket Yeah. Sure. Um, I think, like, uh, AI governance is not an option. Like, it's a must think that should be followed, uh, especially when one model is having, uh, you know, like, all these contexts, like, generating context generating contents, you know, uh, there are chances that it can leak a lot of information. Right? So one of the way is always having, like, zero trust AI architecture when creating these products, like leveraging large language models, uh, which actually means having access control management in place. And especially in an organization, uh, we'll call it like RBAC. It's like role based access control, where every role or the user has a persona, and they have access to all information what they are allowed to. You know? Uh, and that's, like, really critical to, you know, like, mask your datasets based on who is using it and not provide any information. And, also, uh, while using these, uh, large language models, you can also have while fine tuning and training, you, uh, you you can also think about, like, removing all the, like, personal sensitive information, like BII. Uh, so that should be, like, always removed while fine tuning these or, you know, it should not be considered as a part of what LLM is, like, learning on. Uh, the other way is also to have this explainability feature, what I talked about before, having traces that it should also explain itself while creating a response. So even if there is any, you know, like, a wrong or a misinformation coming out of it, someone can go and, like, control and audit that information, like, why this particular information was generated, and it should not be consumed because the way they, um, this product has thought about it is not right way. Uh, also, uh, that also brings in a really important, uh, role of, like, cybersecurity audits, where all of your responses, uh, coming from this Gen AI product, uh, should also, uh, be regulated by control and and audit team. Uh, they they have to analyze all these, like, logs. Right? What was, uh, you know, like, asked for and what it a response. And going from here, organizations should also think of creating these kind of quality metrics of analyzing these responses, like, in the historical responses and assign, uh, you you know, like, a score that these kind of questions are answered well. Some kind of questions are not answered well or the information is not right. So maybe it needs to be it needs a little bit of fine tuning that. Right? So that kind of cybersecurity audits should really be a part of your Gen AI product and how it operates. Um, uh, the the other ways is to just build a data infrastructure if your information is really sensitive, especially in finance if you see. Uh, that data infrastructure, you should not go on cloud completely. Of course, you should have more of the on premise solution if there there is any risk of, like, data being, like, leaked or, you know, being threatened by the external factors.

30:24 Stephanie That makes a lot of good sense. I I didn't hear this come up yet, and so I would be very curious to hear you talk about it if you don't mind. Uh, Uh, do you foresee the regulatory landscape evolving and and how, you know, as generative AI becomes more integrated and, you know, we're all thinking about these things around data privacy and governance?

30:44 Saket Yeah. Uh, I think, like, regulatory, uh, landscape is also evolving. Like, uh, like like, you know, it needs to be evolved, like, uh, to control how AI is understanding and creating content at all the time. So there's definitely a a need for, like, policing kind of, uh, you know, like, how these are working and interacting with your data. Uh, as of now, like, we don't have any, uh, like, GDPR. Like like

31:17 Stephanie Yeah.

31:18 Saket Has, like, GDPR, but we don't have, like, something equivalent to that, which is, like, that impactful. But, definitely, uh, government, uh, will push going ahead to have more regulatory, uh, you know, like, standards to, uh, you know, to actually, uh, monitor, like, how AI is consuming your information and how it is, uh, you know, uh, generating or creating insights, like, for all of the businesses. One of the critical way that organizations should think also about is to how to use, like, differential privacy. Uh, so you can still, uh, you know, like, understand your consumer without getting their sensitive data into your databases. Right? So Yeah. These kind of, I think, architectures, like, differential privacy are, like, really required, Uh, and somehow, like, the government should take a push and, uh, you know, like, make sure that organizations are leveraging these kind of architectures in their product, uh, to make sure that data is, you know, like, safe and secure, especially of the consumers.

32:26 Stephanie Yeah. Absolutely.

32:29 Matt Great call. Sounds like it's really, um, you know, you mentioned a lot of responsibility on the out of your IT and cybersecurity teams and then, obviously, your your legal and regulatory teams. It it sounds like, you know, one of the the good indicators of of success in rolling out a successful utilization of of LLMs is having this really good cross functional alignment. Right? Yeah. I'm curious. What have you see what are the common barriers that you have seen, um, to preventing an organization from from doing this well from a data science side? Um, what have been kind of the the common trip ups?

33:13 Saket Oh, sure. Um, I would say the biggest barriers that I have seen is, like, the lack of data unification and especially in the big corporations. Uh, because of the vast amount of data, it's not, like, seamlessly tied. So for a large language model to understand your datasets, which is very fragmented, uh, that's that's always, uh, you know, like a pinpoint for any organization, uh, where they have to, like, have a very, like, really seamless integration of datasets and curated information. So if large language model goes and it tries to learn, it has all kind of you know, it's not like learning from one table, but it's also, like, correlating a lot of information across your silo datasets. So I think that's one of the barrier. The the organization should, like, definitely invest in unifying their data layer, which is for GenAI. So so that's also, like, really interesting when we talk about it. So before, we used to have a business intelligence layer for reporting. Now I think with Gen AI, we'll need an additional more intelligent databases and layers, which is more connected, uh, you know, and it's it is, like, more ingestible by large language models. So I think Gen AI data layer will be a a key thing, like, going into the future where all all will try to, you know, like, get into that workspace. Uh, the other thing is computational cost. Like, it's it's very expensive. Like, each and every, uh, question that you ask, it comes with, uh, you know, like a like, some sense of cost per tokens. So tokens are your each word in your, you know, like, the whatever context or response you are getting. So to process this information, there's a cost. So if in an organization, there are, like, 1,000 employees, 500 of them, like, firing 20 queries every day, that has a cost to it. And Mhmm. Especially with the large language models, uh, you will need to invest in lots of, like, digital infrastructure, uh, like your servers. You you need a lot of memory to store your datasets. Uh, you know? So that's one of the key things as well. Apart from this technology, uh, barriers, I would say there's also an organizational barrier in terms of the employees or workforce, like, having this resistance of change to adapt to this, uh, having not much trust into these AI tools, which is why Copilot is really necessary. Uh, this kind of projects to kick off engaging your workforce to work with AI as a side by side, you know, uh, and increase your efficiency, uh, see that impact in your work for one year and then leveraging those learnings into a product. Uh, and also, like, training, uh, you know, uh, like, training your workforce, that's also, like, really important. Most of the workforces that we see in the in organizations are not AI. You know? Like, uh, like, they they don't have, like, background in AI. So, uh, it's it's also, like, really important for them to be trained, uh, with bringing in partners from AI innovation leads and the companies.

36:36 Matt Great advice.

36:38 Stephanie Yeah. Absolutely. Um, I have a an area that I'm particularly interested in that I'd love to pick your brain about. So as, you know, LLMs are continuing to evolve, I I'm really curious about their application to what I think of as foresight, uh, which is, you know, predictive, but then also prescriptive analytics. Like, hey. I you know, this is me impersonating, uh, AI. Hey. I know what you need to build next. Right? And then, you know, strategy. Like, actual strategy and innovation work, and I'm really, really fascinated by that potential. And so I'm curious if you think that, like, is that on the horizon? Is it is it here? Is it is it the kind like, do you think generative AI is going to be massively impactful to organizations in those

37:29 Saket ways? Oh, sure. Absolutely. Um, like, large language models are evolving from descriptive analytics to predictive and perspective as well. Right? So that's where I would see for descriptive, I would say we are already in the place. Uh, like, Gen AI can, like, definitely do descriptive analytics for you without any, uh, you know, like, much prompt or fine tuning your models. Uh, Gen AI has some limitations when it comes to predictive models, uh, and prescriptive models. The reason being not the inefficiency of the Gen AI or large limited models, but the inability of them to understand your business or your, you know, like, the key objectives, uh, by themselves. Right? So prescriptive models, let's say, if I am just trying to say that, okay. Predict if my acquisitions of customers is gonna fall in, like, next thirty days, and what should I do about it? So then Gen AI or large language models to even suggest you, like, what action should you take if it sees your trend going down for acquisitions of customers, it needs to know your business strategies as well. So that needs for the organization to also not just give them structured data that what's going on, but also, uh, you know, like, understand, uh, like, make the large language models understand how the business strategy also works. So that's where it's not just a structured data, but you will also need a lot of metadata, unstructured data set training to these models. Uh, so from where it can learn, and it can also, like, create numbers, like KPIs, and also suggest actions based on each KPI. So that's why we need a more intelligent data learning, uh, of, like, different sources, like, from marketing, from digital, from tech, from product.

39:29 Stephanie Right.

39:29 Saket I think, like, at one place. So that's where I think it will go. Um, and once you have that information, of course, uh, large language models have the ability to understand those scenarios and plan those, like, scenarios likewise as what you require. And it can give you recommendations for sure. But the process of making or, like, making these, like, models learn your strategies is kind of, like, the more you know, like, the bigger barrier, I would say, than, uh, large language models, like, giving our recommendations.

40:04 Stephanie So it's not just something I can put into my AI data layer, like our our, you know, company vision and mission and what our KPIs are? I can't just provide that and then expect it to have all the context it needs at this point? Like, we're not quite there yet?

40:18 Saket We are quite not there. And, also, it's because every KPI in every organization is not generalized. But because when we talk about AI, we talk about a generalized AI. Uh, like, what is going on in Citi, it's it might not be same what is going on. Like, Goldman Sachs, they have their own different business strategy. So that's where AI needs to be your like, it needs to be fine tuned to your needs and your information rather than having this generalized information, which is kind of like we have to balance that. Uh, a generalized information learning is also really good because then it can do a lot of tasks, which are like general tasks. But if you we cannot, like, completely remove that because if we remove that, then we'll end up with very, you know, like, a hard coded problem. Right? So there needs to be a balance between a generalized AI, and, also, it is aware of your context. So that's where, like, rag, uh, retrieval augmented generated, like, these kind of techniques are really good to have, which, uh, make sure that all these responses are grounded to your information. And then connect the and it can then orchestrate these informations, like, really well.

41:35 Stephanie Very cool.

41:36 Saket Yeah.

41:37 Matt It's a great point that it it needs to be individualized at least to some extent to have any level of validity, which makes my next question my last question, really, one of my last questions seem kind of ridiculous. But I'm I'm curious if you had to imagine, you know, how good can it get for for for a data professional like yourself? So imagine a landscape where these tools have been integrated seamlessly into your analytics workflows. What would that look like in an ideal state for the end user?

42:14 Saket Yeah. Yeah. No. Sure. Um, I think the, um, the future of analytics is more conversational. Uh, we are getting into this phase of not spending more time in, like, looking for information by ourselves, but it's all in, like, one UI. So it's like and it's like more insight driven rather than, like, the numbers driven. Uh, I would say it will also enable teams in future where data management can be completely revived, especially like writing codes, uh, creating, like, retrieving information from different tables. Uh, these co pilots will completely revolutionize these, um, data migration, data creation, like, tasks as well rather than, like, just giving insights as well. Right? Uh, which is a lot of revenue savings for big organizations because they also spend a lot just to, you know, like, manage the information. Uh, I would say, uh, in terms of how, you know, like, good it can get is, like, a voice enabled analytics. So you can imagine something like Siri or or Alexa where you can just just talk to your data in that way and just ask the questions. And it can just explain to to your things, like, while you are just going in a subway or, you know, like, uh, driving somewhere. And you can still talk to your data and, like, just be informed all the time. So that's, like, a very real time voice enabled, like, insight generator. Right? So that's where I think it's it will go, uh, in future. Uh, there are a few companies which have already worked on that. Uh, one of them is Pyramid Analytics, uh, where they have, like, used this kind of voice enabled feature in their reporting purpose, which was also, uh, you know, like, a new way of, like, looking at, like, your insights. Uh, so so I would say it will go somewhere there.

44:10 Matt Seriously getting close to the Star Trek future. Yeah. All of us nerds want to become a reality. Right?

44:18 Saket For sure.

44:19 Stephanie Right. And we don't have to be nerds to do to use it at that point. Right? Just the the natural language querying. Yeah. It's built for people like me. So the question is, like, I'm trying to so time this topic to the audience of our podcast, which will largely be consumer insights professionals and market researchers. Just thinking about how can we be better partners with our data science and our, like, business intelligence, uh, type of of partners in the business to sort of, uh, use these tools in a way that's coherent and cohesive and and allows us to be successful together?

46:31 Saket No. Sure. Yeah. Yeah. Of of course. So data scientist and data analyst. Right? So their role in GenAI product engineering is is is, like, really crucial. Right? So but the creation of this product and how they are made cannot be made, like, completely successful until the users groups like marketing researchers, or, you know, uh, the digital sales teams. You know, like, these kind of teams, they partner with the data scientists, like, really well. Especially, what we need, uh, in these kind of scenarios is more under understanding of each other's work. Uh, like, marketing researchers should be open and explain their how their operations work. So because that's also a key because your analytical life cycle should match your operations life cycle. So that's what data science and product teams will need to understand from the business users, um, so that they can create these products, uh, based on, you know, like, a really good product user experience. Because user experience is the is the key metric for your adoption. If user experience is bad, then, of course, the the adoptions will go, uh, like, a lot. So that kind of meetings, like, having scrums, like, weekly scrums to understand each other's work, uh, that's really important in this scenario. Uh, and also, like, sharing of the information across, that's also, like, a good way to have that.

48:07 Stephanie Well, that's a great answer. Thank you.

48:11 Matt Yeah. Staying in the loop and staying aligned and making sure everyone has access to the same information. That's a common theme, I think, we hear when we're talking to cross functional business partners for sure. Sure. Um, I I think that's all we have. I mean, I I have our last question. I don't know if we would keep this in here or not, and I know we're at time. Um, but I I think we already hit it. You know? Basically, it's just asking what's kind of your one piece of advice that you would you would leave the audience with. But I think we we've worked that into really all of the other questions that we've we've

48:44 Saket been through.

48:45 Matt But, um, and I know we we put you through the ringer, so we appreciate, uh, we appreciate you sticking with it and, uh, giving us such a a wonderful detailed download on kind of what you see happening in the industry.

48:59 Stephanie Yeah. It's been a pleasure to speak with you today. We appreciate your time.

49:03 Saket Oh, sure. Yeah. Thank you for the the invite. I I really like the con like, conversations. And, uh, no. Yeah. I mean, I'm hopeful, like, how Jenny and I is, like, impacting the market and, uh, like, in couple of years. And I can definitely say that the world will be, like, really different, like, how the business organizers are are are, like, working and generating revenues in terms of getting insights. That's gonna, like, shift a lot. So

49:29 Stephanie For sure. Yeah. Come back and talk to us then. Okay?

49:33 Saket That's right. Sure. Definitely. Thank you.

49:36 Stephanie Awesome. Thanks so much.

49:37 Matt Thank you. Take care.

Produced by aytm

Curiosity Current is made by aytm, the consumer-insights company. Guests speak for themselves; the synthesis is ours. About aytm