Transcript#

This transcript was generated automatically and may contain errors.

Hey there, welcome to the POSIT Data Science Hangout. I'm Libby Herron, and this is a recording of our weekly community call that happens every Thursday at 12 p.m. U.S. Eastern Time. If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience.

Can't wait to see you there. Okay, everybody, let's go ahead and get started. I'm so excited to introduce our featured leader today. She is a Data Science Hangout crew member. I met her through the Data Science Hangout years ago now, which is wild to think about. It's Liz Eisenhauer, Senior Statistical Associate at Westat. Liz, it's so great to have you here. Could you give us a little introduction? Tell us a little bit about what you do and something you like to do for fun.

Absolutely. It's really, really amazing to be here. This is such a great community. I'm grateful to have been able to join whenever I can over the past few years. And it's nothing without its organizers. So you guys are really incredible. I will start and say that any of my views are my own and they're not the views of my organizations or affiliations or any of that. My kind of identification is as a statistician. So I'd like to start by saying what that means to me because that kind of has defined my entire career. If you've ever done a t-test or any, you know, thought about statistics at all, statistical thinking, then I would consider you a statistician. So I just want to say that up front when I say statistician, I'm talking about you.

This kind of my background is that when I was in high school, I took a statistics class at my local community college down in South Jersey. So down near the shore in Seattle City, I waitressed in Seattle City for many years when I was younger. So I took this community college statistics course and it was, you know, great. I was like, this is really cool. And I think I'm decently good at it. So I should keep going with this. But I kept kind of going with statistics. And every time I would finish a goal, I would be like, am I a statistician yet? Like, like I finished the course and I was like, okay, I think I can go do statistics for, you know, my bachelor's degree. And I went and I got my bachelor's degree in math with a statistics specialization, but I was like, I still don't really know what it means to be a statistician. So I went and did the PhD and I got a PhD in statistics. And then I was thinking about going and teaching statistics, like in a kind of academic space. And I was like, well, I don't know if I feel like I can tell students that I'm a statistician, like have I really been a statistician yet?

So I went to this company that I'm currently located in and it's called Westat. So this is, I don't know if this is true, but it seems to me like it's one of the biggest companies that's completely founded by statisticians. So these were, this was a group of statisticians that lived out in Colorado and they founded this company to do work for the federal government mostly and large surveys. And they called it Westat because they were in the West and they were statisticians. And so they called it Westat. And then of course, they realized that if they wanted to do work for the federal government, they would need to move to the East coast to be closer to DC. So they eventually moved to the East coast, which made the name kind of didn't really make sense anymore. But this pretty large company founded by statisticians. And so now I get to be a statistician and I feel like I've, you know, finally made it. But at this point I feel like I was a statistician back when I did my first t-test.

So that's my kind of conclusion about how I got to where I am today. And I guess the other questions were where, I mean, I live in New Jersey now. I've started in New Jersey. I went Penn State. I went Boston for a short time. My husband's from Boston area. I wish I was still there so I could join the hangout, you know, gatherings that you all have there. But now I'm in North Jersey. My husband got a job at Merck. So we're hanging out in Rahway. And so what I do for fun, I think I spend a lot of time thinking about statistics. I don't know how to do other things. I do listen to audiobooks and I guess I listened to a pretty good audiobook recently called The Correspondent, which was, it's a fiction book, which I sometimes, I try to listen to fiction every once in a while. I do listen to a lot of non-fiction, but The Correspondent has been pretty interesting. It's a series of letters. The Correspondent by Virginia Evans, maybe? I think so. Yeah, that sounds right. I'm going to put a link in the chat.

Okay. For anybody curious, I also put a poll in the Slido. If you were looking at the Slido in your browser, there's a little slider at the top between Q&A and polls.

And so far we have 11 votes that my question was, have you ever felt statistics imposter syndrome, which is unnecessary feelings of inadequacy. I don't really like the phrase imposter syndrome. I like unnecessary feelings of inadequacy a little bit better. But so far, not one person has said no. Everybody has said yes. I think this might be a universal statsy feeling.

But I also identify with taking a stats course and being like, oh, I like this. I don't want to do anything else. I really love this thing. But then I could not find anyone who needed me to do statistics. I got out of school and absolutely nobody needed me to do stats. And I realized that that was going to be a really hard thing to do. And when I met you, I was like, what is Westat? There's a whole organization of consultants who are doing statistics. This is amazing. I have since learned that there are lots of people doing statistics in lots of little places, including program evaluation, which was a whole new world to me. I know that you have some program evaluation on your resume as well. So I was hoping that you could talk a little bit about the types of work that you do, the types of problems that you get to solve, the types of data that you get to work with, and also the types of tools that you like to use. So like, did you learn everything in Excel and on paper? Were you an SPSS? Are you an R person? Are you a Python person? All that stuff.

Sure. Yeah. Yeah. I think R has been kind of my main language since I was in the bachelor's degree program. R is just a great, yeah, I use it for everything, except when I have to use something else, then I use something else.

Statistical work at Westat

But the thing that I'm kind of excited about in my work, and that has come up a lot in program evaluation context, like I said, Westat started as a survey organization. So they're very interested in really rigorous probability surveys. And they also have work that looks at if you have a program, can we determine whether this program works for whom and provide additional information to improve the program or policy? Policy is not similar questions. And so that work has kind of branched my statistical expertise into these two areas. And one area that's common across both of them that I've found to be really interesting recently is sample size justification.

So when you start a study, I'm really interested in how people who are statsy or statisticians can join at the beginning of a project or beginning of a research endeavor and provide statistical justification for why this is worth it. I think a lot of science rests on statistical shoulders in terms of us being able to say, we should do this, and this is why we should do this. We should spend this money, we should collect this data, we should put together this infrastructure, because this is my statistical argument.

And whether that's in a large survey context, where you often want a representative estimate for the US population a lot of times, or a large population, and you're interested in a really rigorous mean and confidence interval for a particular value, then you might look at the precision of that estimate and argue for why you need the certain number of people that you need, and you need it to be designed in a particular way so that you can get this precision in your estimate. Versus in something like a program evaluation, where you want to estimate the effect of something, your argument is we need to randomize, we need to put together this particular study design, and the goal is to have a particular statistical power for the effect that we're trying to estimate. So there are different kinds of studies, different kinds of ways of making the statistical argument at the beginning of a study, and I'm really interested in what that all means.

I know there's a lot going on right now with funding mechanisms, and unless you've been living under a rock, you've probably heard about some grant things that have been happening. And a lot of this happens at the stage of designing studies, and so the statistical arguments behind these studies are just really fascinating to me. There seem to be two kind of camps of statisticians. There's the one that's like doing the calculation, that's like, you know, do whatever you can afford. If you can afford it, have a giant study. And then there's the camp of statisticians who are saying, we're reviewing these, and you need to give me a solid argument. So if you are telling me the effect size is 0.2, give me evidence that you actually think the effect size is 0.2. Show me previous data, show me pilot studies, show me something that convinces me that this is what you're actually going to see in the study. So there's two camps of statisticians on this, and I think it's a really open area that I'm working in this area right now. A lot of my work ends up falling in these series of questions. So that's kind of the big picture.

Communicating statistical findings

We're going to continue talking about that, because we have some questions about this in the Slido. And I love Rowan's question, which has bubbled to the top here. It has an asterisk, so I will ask it for Rowan. And I know that you and I have talked about this, so let's dive in. How do you grapple with conveying statistical findings to audiences who are not well-versed in statistics? I have found statistics to be particularly difficult to properly explain at a layman level without losing nuance.

Yeah, nuance is a big part of it, right? The statistical nuance that statisticians always end the conversation with like a caveat, right? But we can't really say this with certainty. I was saying in the chat when you were like, am I a statistician yet? I said, it depends. If you can answer everything with it depends, you might have made it to the statistics brain level. Because I feel like when I was learning, I was really frustrated by it depends. I was mad about it. I'm like, give me an answer, right? Like this is so frustrating that everything is it depends. You're going to make me use critical thinking skills for every single question? Come on. But then at some point, people were asking me questions. And I was like, oh, it does depend. It depends on so many things. I can't give you an answer either. The teacher had become the master, yeah. Or the student had become the teacher.

But you and I have talked about boundary objects, which is one thing that could be helpful. So take it away, Liz. Yeah, yeah, yeah. I found it to be... So I work a lot with other technical people, people who are very technically minded, but not necessarily statisticians or they don't necessarily have a background in statistics. And so that perspective and that kind of interaction is often more interesting to me than talking to a lay person or a person that really has no skin... I don't know. They're not like stakeholders in whatever terminology you want to use. They're not actually one of the people that's making the decisions. Obviously, this is a whole other area of interest, people who are thinking about all the different types of people that could be involved in decision making.

But in my experience, a lot of times when I'm talking to technical people, the question is really, can I find something that they understand? And can I then latch onto that and connect it to something that I'm trying to say? And that connection often happens more easily for me when I have something separate from what I'm saying, like a visual of some sort. And so I found flowcharts. I make flowcharts for all my projects. Anytime there's a question and someone is having a technical conversation and they're trying to figure out, well, this affects this and that affects that. How does this all work together? I make a flowchart and I send it to everyone and I say, this is probably wrong. Tell me where it's wrong. Let's fix it. Let's make this something that everyone can agree on. And then once you have the one term for this kind of thing is boundary objects, Susan Lee Starr is a person who came up with this terminology for objects that have

meaning to different groups, but they mean something different to the two different groups. So you can use them to collaborate or coordinate around something because they have meaning for both groups. But the statisticians, I see it as, how do we adjust for these different things? How do we think about the causal relationships? Whereas maybe I put numbers on it, like this is the total number of people in this particular node on my flowchart.

Someone who comes from a different technical background might say, okay, I understand that there are a certain number of people in this node and that's what that means. So they understand it a little bit differently, but we can have a conversation looking at the same visual. So that's something that I have found to be really useful. In general, I mean, just connecting with what people know, meeting people where they are is probably pretty good practice. Meet people where they are, figure out what their understanding is right now. This is important in all kinds of teaching. And also with toddlers, you look at them and you're like, what are you doing? Your adult brain says, this kid's making a mess. This kid's destroying something. But then if you ask yourself what they're seeing, they're experiencing something, right? That's all their own. They're like sensory experience or visual experience or whatever it is. It's totally different from whatever your adult perspective is of what that toddler is doing.

And I feel like that is so similar to being a stats person or being a technical person and talking to someone that you are assuming is non-technical. They're a fully fledged human being, an adult. They're so smart and capable. They're just seeing something in a different way from you. Figure out what that is so that you can build bridges of connection between what they are thinking and what you need them to know about this. Okay. Teacher, teacher soapbox off the teacher soapbox Libby. Okay. Let's go back to Slido. Let's find some more questions.

And I feel like that is so similar to being a stats person or being a technical person and talking to someone that you are assuming is non-technical. Figure out what that is so that you can build bridges of connection between what they are thinking and what you need them to know about this.

We had another one from Jared Allen. Jared has an asterisk. I'm going to ask for them. What is the most counterintuitive result that a stats analysis has thrown at you? Can you think of anything? Share in the chat too, if anybody has any that, that, that they're thinking of counterintuitive result. I mean, a lot of something that's interesting because I work in a policy space. Sometimes things that you think are going to change, don't change, right? Like particular types of measures. I did a study in my PhD where we were looking at students attitudes towards probability and the SATS, the survey of attitudes towards statistics was the instrument that I use to, to try to look at probability, not statistics.

And it barely changes. Like the average, you know, students take a probability class and it goes a little bit up. Their attitudes barely changed on average. And I found since then that there are some scenarios where you just don't see the sensitivity to change that you would expect because either the measurement mechanism doesn't change or because the intervention itself is just in such a complex environment. Another case that I've seen is in job trainings. Like I've, I've gone to APAM, which is the association for public policy analysis and management conference. They had a session on job trainings and they showed that in almost all the cases, none of them had any significant differences. It was just a scenario where it's really hard to show change.

I love this. I will say in the chat, we are saying Simpson's paradox, right? Like that's, that's the funnest one, right? Where you've got, if you could imagine like a scatterplot where you have this big unified group of like positive association, and then you turn all the groups, different colors, and you realize that it's actually just negative associations that are stacked up together that look like one big positive association or vice versa. But another one that I really like is in marketing as a discipline. When you ask people what they want, they're often just wrong according to their actual behavior, right? They'll be like, what do I want? I want A or I want C or I want B, whatever. And they, people don't actually know what they want ahead of time. And then when you study their behavior, what they actually choose when they're in a store or retail environment or online, whatever is incorrect. That's, that's not right. Or study UX design is another thing. Like someone might say, oh, I would want a button that does this. And then in reality, when you test their user behavior, they don't click that button. They didn't want that button. They want something else, but they don't know.

A typical day in the life

I really love it. Okay. Fantastic question, Jared. Let's head to Slido and ask Natasha's question. Another start. Nobody wants to ask out loud today. I'll read to be everyone asking the question. So I get this.

What does a typical day or week look like for you? What areas do you split your time between programming meetings, for example? Yeah, that's a great question. There is a lot of both of those things, programming and meetings. And I work fully remote, as a lot of statisticians do. So I am looking at what the tasks of the day are. What are my deadlines that I have to make? If I have meetings where I'm advising, sometimes I have some of these projects where I'm in an advisory role, kind of coming in like on a power analysis and coming in with kind of suggestions or questions, and I need to make sure that I'm prepared to come into the meeting and have those conversations.

And then other projects where there's an analysis that has to get done, and I'm walking off time to do that. So yeah, it really depends on the week. Sometimes it depends on the month or time of year for me. PositConf is coming. What will they be doing? Probably something to do with PositConf, right? Because it's just that time of year. The conferences is looming. Okay. Fantastic question, Natasha. Thank you.

Causal vs. predictive modeling

We have one from Sadie, and Sadie has an asterisk. Rachel, would you like to read Sadie's question? Sure. I was just jumping in so you could hear some other voices. I would love it. I love this. Mine or Sadie's question was, how do you explain the difference between causal and predictive modeling? When do you decide to use one type of model versus another?

This is an excellent, excellent question. NIH just put out a request for information that I helped ASA submit a response to that we submitted yesterday, and ASA has it posted on their website. Part of the request for information asked this exact question, what makes research causal research? This is a very open question right now. I don't know the answer to that particular question because it's changing.

It used to be statisticians would say randomization. If you randomize, you could make sure that you're accounting for all of the unobserved confounding, and you can say that you have a causal result. But now we have quasi-experimental designs, we have natural experiments, and that can make something causal too. You can justify it, right? If you have the assumptions that are appropriate and that you're able to make an argument for it being causal, then it can be a causal conclusion. That's a really challenging question.

The other question related to what models to use is also a really challenging question because so many people just use the models that they're familiar with, and that's becoming more of a problem as there are more and more and more models. Now there's AI choosing models, and actually in the same RFI response that we put together for ASA, there's a lot of discussion about the extra degrees of freedom that are related to the analyst choosing the model. And how the conclusion changes when you have a different analyst, an AI is choosing the model, all of the different variability that comes into that, and quantifying that is a huge question.

Choosing the model is a huge question too. I don't have answers, I feel like, for either of these questions. I'm sorry. Becomes like a complex problem or complex systems problem, right? Any problem in which a human being is involved is like automatically now a complex system. It feels like there's infinite possibility. Yeah, Morgana says, oh no, so many more new variables. It is a web of mystery. All right, thank you Sadie so much for your question. Thank you Rachel for asking for Sadie.

Should you get a master's or PhD in statistics?

Zach has a question, also has an asterisk, and it says, would you recommend in today's day and age someone getting a master's or a PhD in statistics? For someone getting a master's or a PhD? Would you recommend? Would you recommend getting it? Yes, in today's day and age, in today's climate, and I feel like the implicit thing is they're like in the age of AI too. What would you recommend personally?

In terms of whether or not, I think I'm going to interpret this as whether or not to go for a master's or PhD in statistics, right? I think that's the question. Okay. I think so. Zach can confirm in the chat for us. Okay, so statistics as a discipline, I think is a pretty strong direction to go in. It's been around for a long time, and statistical thinking underlies a lot of science, and being the statistician in the room has its benefits. A lot of people that I talk to that did not go to a statistics route say, I wish I knew more statistics. So just based on that, I think it has a lot of value, and I think it will continue to have value in terms of conversations, just having discussions among people. AI can't be responsible for making decisions. At the end of the day, a human being has to be responsible for saying, I support this decision, and statisticians, I think, could be potentially the people who can be in that role.

At the end of the day, a human being has to be responsible for saying, I support this decision, and statisticians, I think, could be potentially the people who can be in that role.

So I think it's a good direction to go, but maybe someone else will step into that position that's not a statistician, and maybe statisticians will become obsolete, and then I'll be wrong about all of it. Hey, the option to be wrong is part of being a statistician. Oh, yeah. You've always quantifying uncertainty and quantifying your own probability of wrongness, right? You're like, this might be wrong. I'm just giving you what might happen. I feel like that was a great chance for you to be able to use the answer, it depends. It depends on everything.

Chat highlights and audience Q&A

Fantastic. There are also so many good conversations that are happening in the chat if you're listening to this later. Gosh, I wish you were here in person because the chat is so much fun. Noe was talking about how Simpsons Paradox is like the MAUP problem in geography. I have not had time to look up MAUP yet. Noe says it's Modifiable Aerial Unit Problem. When you have your data as a point, and you aggregate to hexagons, or you aggregate into stats area or states, and they have different results. It's still the same type of problem, but geographically. This is fantastic, fascinating. I'm going to go Google this after we get done. I wanted to share that because everybody who's listening and maybe not in the chat is not reading this fantastic conversation.

Okay. Let's hop back over to the Slido here. If you do not know, I know that we're in our last half of the Hangout here, but if you have hopped in at the half, maybe you just got off a meeting, you can ask questions for Liz, who is a statistician at Westat in the Slido. We can paste some links in there to actually get there if you need it. Thank you, Rachel. Okay.

Okay. We have a anonymous question. It says, since we deal with probabilities all the time, do you use phrases like chance of, or odds in your slides, et cetera? Yes, definitely. Language matters, doesn't it? Definitely. A lot of the work that we do at Westat is for government clients. The result is a report oftentimes, but that is for government clients and the result is a report, oftentimes, but sometimes it's a one page report, sometimes it's a presentation, sometimes it can change, but it is generally for a general audience and needs to be understandable. So I am still figuring out how to write for a general audience. It's really hard because in your statistics PhD, they drill into you.

You need to be very careful how you say things and you need to follow this particular structure when you're providing a result. And then that gets kind of turned around when you have to write it for a general audience and you have to use things like the chance of, you have to use numbers instead of proportions. If you want to say 10%, or you wouldn't say 0.10, you would say maybe 10% if you're talking to a technical audience that's not statisticians. If you're talking to a general audience, you would say 10 in 100, 10 people out of 100 people.

So you have to translate to what the, and there are a lot of people who are good at forcing me to do that and telling me what you've said does not make sense. And I listen to those people. And that's, I think the best way to make it better. Right. Some of those words are very contentious. Like Renato in the chat said the word significant is another example of that. That's really tough because significant is just a normal English word. We use it all the time. That's significant to me, a significant other, a significant object. And in statistics, it means something very specific to the person who's saying it. And yet it is very contentious. Do we even want to talk about significance? Do we want to fight about P-values? Do we want to use them at all? All of those things are tough. Thank you so much for your question, anonymous asker. It was great.

All right. We have more questions popping up in Slido. We have one from Kelsey. It has an asterisk again. Nobody wants to ask any of their questions live today. This has never happened. Every single question has an asterisk. Don't be afraid y'all. I'm muting it. It's not too scary.

Kelsey says, when you said you follow debates between statisticians about, for example, sample size calculations, where and how are you following these? Is it on socials like X or LinkedIn or in journals? I feel like I miss out on conversations like this after leaving Twitter.

Yeah. It's a lot of times difficult just finding people to talk to about it is the best way that I've figured out. I will say there is a community, the isolated statisticians that have a listserv and there are a lot of statisticians on there that sometimes get into arguments. So that's one place that I find some fun arguments. There's often a discussion, a recurring discussion on there about statistical significance, whether it should be renamed to statistically discernible or something. Can we rephrase it in a way that will change the way that people think about what it means? That comes up on the isostat listserv a lot. So that's one place.

But yeah, talking to people is just finding people that'll listen to me and answer my questions.

Meeting people, building community. You can meet a lot of stats people here at the hangout too. You know, you could just put it up in the chat. Hello, I'm a stats person. I want to talk to other stats people. Let's connect. You can ask the same question in the discord or, you know, wherever you are, blue sky, like people will find you, but also just the blue sky, our stats group, our stats hashtag has been really nice. I really enjoyed blue sky.

It's not 2018 Twitter. I think that's a, that was a flash in the pan that will never happen again. We are all mourning the loss of that type of sharing community. However, there are people that I like to follow on there, like Frank Carroll. Oh, Donald just mentioned Frank Carroll in the chat as well, which is hilarious. So yes, and Donald is posting some links. So anybody in the chat, go take a look at those. If you're listening later, I'm going to put them in the YouTube description. No fear. But I also there's JSM. Do you want to talk about JSM that you, I think you attend almost every year, right? Yeah. JSM is an incredible place and it gets better every year. I think the more you go to these conferences, the more you meet people. And then every year you're like, Oh, I have to connect with this person again from last year and this other person. And it's joint statistical meeting for anybody who's not, doesn't know what JSM means. I think it's always in the U S right. And it's in a different city every year, just like positive conference. If you want to go to JSM and you're interested, Donald's usually there.

Liz is usually there. You could meet Donald. He's very nice. Liz will say hi to you, you know, reach out to people and say, Hey, I'm going to JSM for the first time next year. Can we meet up and have coffee or something? And I'm sure you will find nice people. All right. Thank you, Kelsey for asking.

Avoiding over-engineering models

Let's see, what do we have here and what time is it? Okay. We have about 15 minutes left. We have an anonymous question that says, how do you fight over engineering and picking most involved models, et cetera, when you need to speed up delivery? For example, a good, good enough forecast work or forecasts would work. I don't do a lot of, I do some forecasting, I guess, but yeah, there might be other people who would have, who would be able to contribute interesting ideas on this. So feel free to jump in if you, if you have ideas, but in my experience, if there's the argument of interpretability and you can always lean on that kind of, if I give you a really complicated model and you don't understand how it works, then we can have a conversation about it. And I really want to have a conversation and have this be an interactive discussion, not just here's a number, go use it. So that's one of the arguments that I use a lot. And yeah, you try, you try not to go too complicated.

I feel like model parsimony is something that is taught by a lot of statistics teachers and a lot of people kind of like as they're coming up, learning about statistics might have that drilled into them, that model parsimony is important. Model parsimony just means like striving for the most simple model that you can, that still explains things well.

And this is a very interesting question because I think it's being asked in the context of business where often you have all of these like time constraints, let's go, let's go, let's go. This is where I would encourage anybody to look up operations research, which is my background, it really excites me, mathematical optimization and heuristics, right? Because a heuristic is kind of like a rule of thumb or maybe a business rule that might get you to maybe not the optimal solution, but a good enough solution in place of a model in place of a calculation or in conjunction with one. Because often with statistical modeling or with mathematical optimization, there might be many optimal solutions and it might take a lot of calculations to get to them and it's not worth the computational power that it's going to take or the time it's going to take to get to that optimal when there might be like 15 different slightly suboptimal but perfectly good solutions that you could go with using a heuristic or just a more simple model. All right, thank you so much for your question, that was fantastic.

Bayesian vs. frequentist approaches

I wanted to ask because I did see a question here in the Slido that was about hiring, but Liz, I don't know if you do any hiring, do you do any hiring at Westat?

Personally, no, I don't. Okay, anonymous asker, don't be mad at me for not asking your question, I'm so sorry, I don't think that Liz does a lot of hiring. But we could put this to the chat as well, if you are a person in the chat who does hiring or who is involved in hiring adjacently or maybe contributes thoughts to it. The question was, what are technical qualities that you look for in statisticians when you hire? I've seen hiring standards for statisticians be very varied and depends on teams, specific requirements, etc. So if anybody has any experience in this, please feel free to share it in the chat.

Okay, we have Jared asking a burning question here in Slido, which is, are you Bayesian or Frequentist? Do you prefer Bayesian or Frequentist? I like both. Why use wise words? It depends. Yeah, I use Bayesian analysis in a lot of my projects and it can be for interpretability, it could definitely be useful at the end of the model for giving a statement that is in terms that can be understood a little bit better, like there is a 70% chance that the population parameter, the population mean, the population effect size, whatever it is, is greater than zero. Full stop. That's a really nice statement. It's way easier for people to understand than me saying, given the null hypothesis is true. So I do like it for interpretability. Often when I see it used, most often it's because of the usefulness of Bayesian methods for really complicated structures, like hierarchical structures where you have schools and students within schools and years within students or something like that. You have a really complicated structure, Bayesian methods can be really useful for that. It just creates this really convenient environment for you to put all of that into one model. And so that seems to be in a lot of cases where I see people actually using it.

I don't know if it's because the interpretability argument, everyone's just so used to p-values and so in the end they want to see the p-value. If you give them the Bayesian argument, they also want to see the p-value. So you can do both, but then that's a lot more statistical explaining when you're presenting the results. So a lot more work too. Yeah. And what if your conclusions are slightly contradictory between the frequentist and the Bayesian approach?

That's a really nice statement. It's way easier for people to understand than me saying, given the null hypothesis is true.

Yeah. There's a lot of discussion and there's some work that's coming out of Mathematica that's like Bayesi framework for education contexts where the goal is interpreting using Bayesian methods and you have to decide in the beginning what combination of the p-value being in this range and also the Bayesian probability being in this range will mean it works. You have to predefine what it works versus it didn't work, what that looks like. So it has some decisions involved that you need to reason through. Right. Deciding ahead of time on something is also how you prevent p-hacking, right? You're not just going to be like, well, let's just massage these until this all feels right. That was one of the things that also bothered me. I feel like when I was learning was like, oh, well, how do I know unless I run all these different models? I was like, no, you have to pick the model ahead of time. You have to figure out ahead of time what you're going to use before you use it on the real thing. And I'm like, but that doesn't seem like the most correct or right thing. And there's so many different definitions of correct and right when it comes to statistics. Okay. Fantastic question. Thank you, Jared.

Also, as you were talking about levels and classes, I was like, oh, Markov chains as well, which feels to me like it's a secret third category. Okay. We won't talk about that.

But guess what? We have a question that doesn't have an asterisk next to it. Donald asked a question in Slido. Donald, would you like to ask that question live? I would love to have reach out to you. Yay. I'll put it in the chat as well. So everybody can can see it too. Great. So I'm also a statistician. So I do a lot of work with my academic or and medical colleagues who know like a touch of statistics, but probably not as much as either of us do. And I was just curious, do you use like simulations or graphical examples in a simpler context to like what you're working on to help lead them to water, like for that type one type two error or that type M type S error? And have you found that extremely helpful?

Yeah, I am not a huge simulation person. I know some people who are really intuitive and can come up with a really good simulation that then can make an argument. I'm I don't know that I gained that skill as fully as I probably could. Simulations can be extremely useful for power analyses too, like sample size justifications can be very complex and not just a closed form, you know, A plus B equals C. So simulations can be very useful if you have feasible assumptions that you can put into them and then produce, OK, this is the power that we roughly expect to get. So there's definitely opportunities for it to be to be useful. I haven't tried to explain a simulation to very many people recently.

All right. I will hop in with my operations research brain and say, if you like simulation, go explore operations research. There's so much simulation in also like supply chain management. Imagine you have this like very complex supply chain and you want to model what's going to happen if I change one thing in this supply chain. Let's say our supplier in another country stops making this part. What happens or what happens if we run out of stock in this supply center or distribution center? How does that change all the downstream effects with price and availability of our product or getting it into our factories to be able to make our laptops,

whatever? Fascinating. So if you really like simulation, check out supply chain as a field.

Yeah, causal inference can be can use a lot of simulation to what you're describing, what if this happens? That also can be related to causal inference. Monte Carlo simulations. Go Google the heck out of all of these simulations. They're so interesting.

Thank you, Donald, so much for asking. And you know what? I saw another question from Beverly that does not have an asterisk in it. Beverly, does that mean that you would be willing to ask your question live? Let's see if we can find Beverly to unmute.

I did copy Beverly's question over from the chat as well. So that's OK, Beverly, if you can't unmute, I could ask it too. All right. I don't see Beverly unmuting, but that's OK. Rachel is going to ask it. Yeah. So Beverly's question was, how do you handle bias in the following examples? So one, some subgroups are not included in surveys. And two, the training data for model building has higher percentage from a particular subgroup.

And I'm going to copy it into the chat too. Thank you, Rachel. Yeah, I love that. Two different things. Some subgroups are not included in the survey. And then the other one, even in balance of subgroups. In the training data. I will let you pick one. I know two-part questions are hard. Yeah. No, this is a great. I love this question. I'm really interested in transportability and generalizability. I went to a conference or workshop recently that IMSI did that was on exactly this question. And there are causal inference people who think about internal validity versus external validity. So internal validity for causal inference, thinking about how can we make sure that we identify the causal effect that we're interested in for the particular system that we actually are looking at. And then external validity, meaning you have either a bigger group that has some features that are not in the group that you actually have in your data, or you have a totally different group for transportability. You have a totally different group, a different population. Like you're looking at New York and you care about a different state. Then you're trying to see, well, this worked in New York, but does it transport to a different state?

So there are a lot of lessons I think that can be learned from survey statistics, I think, in this area. That's my take on it because survey statisticians for years have been thinking about what does it look like for our population to be represented by the sample that we have. And they have very rigorous ways of quantifying the uncertainty through weighting. So creating probability weights that adjust for the non-response behavior. So we have something like a decision tree, and the decision tree will be an indicator of, the indicator is whether or not people responded. And you can use this decision tree to put people into cells, and then you can use those cells to say, well, the people in this cell are somewhat similar. The ones who responded can represent the ones who did not respond. And so we can adjust the weights based on that. Non-response weighting, there's other ways to do it, but that's one way. And then you can continue weighting to look at, if you're then taking a subsample, you're trying to up-weight a certain population, down-weight a different population. We do oversampling sometimes because we are trying to balance a particular subgroup that we want estimates for the subgroup versus the overall, making sure we have representation overall. And so we might oversample this one, and then we have to down-weight them for the overall estimate. So there are, it is a huge, yeah, there's so much you can do and so much you can think about. I think survey statistics is a place to go.

I agree. It's really, really hard. And this is reflective of, I feel like the hardest subject when everybody's learning, which is missingness as a problem in general. Why does it matter? What are the different patterns of missingness? Missingness not at random, missing completely at random, and how patterns of missingness can tell a story about your data. Ask questions about that. And surveys are the number one example all the time. Like, okay, you're missing salary data from all of these respondents. What do all those respondents have in common? Maybe their salary is super, super high and they don't want to respond. Maybe they're all in a class, a socioeconomic group together. Maybe it's very, very low income. What does it mean if you leave those people out of your study? And what do you do about it? Gosh, I don't know. It depends, doesn't it, on every single thing because there's no answer that's straightforward. Beverly, thank you so much for your great question. I loved it. Ooh, Tom. Tom in the chat says, impute. Oh, no.

Yeah, there's so many ways. I mean, there's imputation, there's waiting, there's regression, like using regression for adjustments, there's- Upsampling and downsampling. Yeah. All kinds of things. Smote. Uh-oh, go Google smote everybody and get real scared.

Career advice for statisticians

Okay. We have time for one lightning round question, which is, do you have any career advice for us? Because we have two minutes left. What off the top of your head would you, what advice would you give people coming up in stats or getting into it for the first time? Advice that someone else, I heard from someone else, so it's not my original advice, but try to really understand something that's not statistics and see if you can connect the two. If you can, if you're interested in health research, try to figure out how to speak that language. And then you try to understand how the two languages connect to each other, because translating between different perspectives seems to me to be a really valuable skill to move statisticians, to move data people into more of a leadership role. We can lead if we can have a conversation with other types of people. So that would be my advice.

seems to me to be a really valuable skill to move statisticians, to move data people into more of a leadership role. We can lead if we can have a conversation with other types of people.

Amazing. Thank you. This was so much fun. Stats people, I'm so glad we got to have a fun chat. If you made some stats friends in the chat and you want to remember who they are and what their LinkedIn profiles were, you can save the chat. There's three dots at the top. Click those, click save chat, unless you're joining in a browser, it might be harder to do that. But please also join the discord so you can stay connected. Liz is on the discord, but I don't know if she frequents the discord. I tagged her in the data science hangout channel. You can make your way there. Liz, I hope you had a good time. This was really fun. Thank you so much. Yay. Thank you so much for joining us.

All right, everybody. Feel free to reach out to me if you, you know, anyone at all. Yes. She is Elizabeth Willow Eisenhower on LinkedIn. If you are looking, if you type in those three words, I think you will find her. And if you want to hang out with us on Tuesday at the data science lab, you will be hanging out with me and Isabella and Daniel Chen. Daniel Chen is here today. He might still be here in the chat. He's going to be talking about building the fundamentals of building a shiny app in 2026. Are you new to shiny app building? Has it been a while since you built one? Come hang out with us. Dan's a fantastic teacher. And we're going to be talking about, there he is waving. We're going to be talking about shiny on Tuesday at the data science lab. If you would like to register to join us and reminder, if we're not on your calendar, if today on Thursdays, this time slot on zoom is not on your calendar, please renew it for another, you know, course of however many weeks in your Google calendar, we're going to attempt to send out an email that might have a calendar link in it that will hopefully work. Look out for that for us. Yeah. Rowan says until forever. Yeah. So forever, forever. It's so hard. Google calendar. I don't know about outlook, but Google calendar, I think it has like a max number of instances that you can put on there. It's like 60 something.

We also have that in zoom events, right? So I've gotten to the end of my events. I'm having to create new ones by hand. Every once in a while, I create one that has like a wrong time or date and have to go fix it. So bear with us while we, while we go through, you know, it's just been a year. I also wanted to just add real quickly, cause I see a few people jumping in right now. So it might be like a daylight savings time. Make sure that everyone has it on their calendar from 12 to 1 PM Eastern time. I'm so sorry for anyone who's just joining right now. Thank you for jumping on, but we're, we're just wrapping up right now. But also if you do make your own calendar invite, you all have your own unique join URL. So like the link that you click in the email is your own. So just be careful about sharing. Don't share that with someone.

Yes. If you give your link to somebody and they click your link to the hangout, they will kick you out of the hangout. If you are a professor, don't give your students your link, give them the link to sign up themselves. We've had that happen. Somebody was like, Oh no, I'm getting kicked out over and over again. And like her students were kicking her out. Basically the JASB. I just saw you join. I'm so sorry. I know the time zone woes have made it hard. We are the hour before 12 PM to 1 PM US Eastern time. All right, everybody. Fantastic seeing you today. Cannot wait to see you on Tuesday and next Thursday. Bye everybody. Thanks Liz again. Bye.