Transcript#
This transcript was generated automatically and may contain errors.
Hey there, welcome to the Paws at Data Science Hangout. I'm Libby Herron, and this is a recording of our weekly community call that happens every Thursday at 12 p.m. U.S. Eastern Time.
If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience. We can't wait to see you there. I am so happy to introduce you to our featured leader today, Kimberly Olney. Kimberly is a computational scientist at TGen. It's so nice to be joined by you today. I would love it if you could introduce yourself, tell us a little bit about what you do, and also something that you like to do for fun.
Yeah, good morning, everyone. Firstly, I'm so excited to be here. Thank you for joining me today and the Posit team. I can see several of us are wearing our Posit conference shirts. I will be joining virtually this year, unfortunately, but I look forward to participating in person again in future years. It's just a wonderful community. So thank you again for having me today.
My work is broadly in the field of omics, which is a broad term that includes genomics, which most people are familiar with, like DNA, what kind of variants you have and how that could be linked to your increased risk of certain diseases, as well as transcriptomics, so what genes are kind of expressed and turned on, or metabolomics or proteomics, so anything that ends in omics. And then something fun that I like to do, I'm in Arizona, and I love to garden, which sounds counterintuitive, being in the low desert of the heat definitely has its challenges, but I have over 30 fruit trees in my yard.
Oh my goodness, that is some niche gardening. I lived in Arizona for nine months, and I realized that, wow, it sure depends on where you live in Arizona, you know, what your elevation is and stuff, what you can grow. It is really exciting to have somebody on talking about omics because I, every time we talk about this, I'm like, why are there so many omics? There's so many different types. We learn about a different one each time.
So I know that you study Alzheimer's disease and Lewy body disease, and I was asking you all kinds of questions about Shiny apps. I would love to just dig into some of that as a little bit of background for people so that they know what questions to ask. So if you could tell us a little bit about the way that you use Shiny in your research and how you express your research to all of us so that we can understand it and other people can benefit from it. I think that would kick us off really well.
Using Shiny apps to make research accessible
Yeah, certainly. I'll start with Shiny apps. For any research publication that me or my colleagues make, we always like to have a Shiny app that accompanies that research publication because historically you would have, you know, maybe a max of six figures in that publication and then supplemental data, but there's only so much that you can put into a research paper because oftentimes there's like even word limits on it. You can't be like, this is all the things that we found. You just have to do the high level highlights, but you might have missed something that was really critical to other aspects of the field, but you didn't know to highlight that, but maybe other researchers would be really interested in that part of it.
So with a Shiny app, they can go and parse through the data much more easier than just looking at, you know, an Excel table and being like, all right, well, here's my gene of interest now, but how does this look compared to this other gene of interest in my different groups? And then they're basically having to redo the analysis based off of their unique viewpoint of it. But with a Shiny app, they can already look at it and kind of parse through it without having to start over fresh each time. And that's something that I really enjoy about Shiny apps is that it just makes it more accessible to not only to researchers, but also to the broader community as a whole. When other people see papers being highlighted in the news and they're like, oh, well, that sounds really interesting. But then they go and try to read this really dense paper that they're like, oh, I don't know what these topics are. And then they just kind of give up. But with a Shiny app, it feels like they have more control over it because then they can kind of play around with it themselves without needing to know how to code or know any of the big terms that are used.
And that's something that I really enjoy about Shiny apps is that it just makes it more accessible to not only to researchers, but also to the broader community as a whole.
The scale of omics data
So yeah, or without needing to process a massive amount of data, because if we could talk a little bit about the types of data that you process for all of us who probably work with totally different data types, I believe that your data is quite enormous. Can you talk about the size?
Yeah. So as our lab as a whole, and this encompasses multiple projects. So this isn't just one project. We have over a hundred terabytes of data that we're currently sitting on for either research that has already been completed or is in the works of being completed. So with omics data, which typically includes sequencing data, you will have your A's, T's, G's, and C's, your nucleotide sequences, the blueprint of life. But with that, you also have quality scores that are associated with it. And it's not just linear where the human genome has like 3 billion base pairs. Well, you're not just looking at each one. You have to have a certain amount of coverage for it to be really confident and calling that because there's errors in sequencing when you get this. So if you have like one error, then you're like, okay, but the rest of the hundred read depth that I have is all saying the same thing. Then I'm really confident that this really is an A at this site. So it starts to add up really quickly.
I mean, I cannot even imagine. I have never dealt with that much data and I don't know anybody else who does. If you deal with that much data in the chat, chime in and let me know because I would love to know if you are also doing omics stuff, if you're working with genes and if this is just a high throughput genomic problem that's really common or if anybody else has that much data, a hundred terabytes is a wild amount.
I don't know if you already expressed exactly the research question that I think that we can share Shiny apps on, but if you wanted to talk a little bit about it and if we wanted to share some links, I would love to do that if that's okay. So people can see the Shiny apps that we're talking about.
Lewy body disease and Alzheimer's research
Yeah, certainly. So one of the most recent projects that we published on last year was comparing Lewy body disease to Alzheimer's disease. So this is postmortem human tissues. So these are individuals that donated their brains for science and Lewy body disease. Most people are familiar with Alzheimer's disease. It's the poster child of dementia, but dementia is really an umbrella term that can refer to vascular dementia, frontal temporal dementia, or dementia with Lewy bodies. And dementia with Lewy bodies is actually pretty common, but is often misdiagnosed as being Alzheimer's disease. And that has huge consequences for the types of treatments that they should be on and their long-term care. So being able to actually at the molecular level, parse out what is both shared and unique between dementia with Lewy bodies and those that have just pure Alzheimer's disease is important for moving the field forward and developing more targeted therapies to treat both of these diseases better. And these diseases, how they're actually defined is based off of their underlining pathology, which is like an accumulation of misfolded proteins in the brain.
I don't know if anybody else has a history of Alzheimer's in their family, but I absolutely do. And I feel like this touches so, so many people in every community. So I'm really excited to be talking to somebody who works on this and is working on making things better. Now, I know I have two links, two Shiny apps. I put one in the chat that is the LBD CWOW. And I'm hoping that that is the one that I am hoping to talk about. And we may give it the hug of death, but I want everybody to sort of like have it so that they can look at it is the one with two plots side by side. And I was wondering if you would be willing to talk a little bit about what that is showing us.
Yeah. So this one, this project is currently under review for publication and what it's looking at is the actual cell types in the human brain. So everyone's familiar with neurons, right? That's what allows us to walk and talk and do all those wonderful things. You know, we have almost 86 billion cells in our brain, but only about half of those are neurons. The other half is other cell types, microglia and astrocytes and all good dendrocytes. And they're kind of broadly clumped into being called glial cells. They help support the neurons.
And when we think about neurodegenerative diseases like Alzheimer's disease and dementia with Lewy bodies, we know broadly that the neurons are failing, right? Because of this accumulation of misfolded proteins, but what's kind of causing that to happen. So the microglia, which are the resident innate immune cells of the brain, they're the ones that typically are going to react to these misfolded proteins and hopefully recruit them to be processed and removed. Or if that doesn't happen, then you get kind of inflammation in the brain and too much of it. And then it becomes like this vicious feedback loop. So what you're looking at in that plot of side-by-side is it's a 2D plot of three-dimensional data of where you have your different cell types. So they're annotated based off of the genes that are expressed. So neurons express certain genes versus like the microglia express certain genes. So we can annotate like these are the microglia and these are the neurons. And then within that, we can compare between our different disease groups relative to individuals that didn't have any pathology.
Okay. Wonderful. I promise I will stop talking. It's now up to everybody else to ask questions, but I knew that that was a heavy bit of explanation that I wanted to get out there for everybody. Very scientific. That I wanted everyone to hear. Well, I saw Hubert asked a question in the Slido that just popped up and I'd love to know the same. Hubert, if you were available to ask that live, that would be fantastic.
Data challenges and managing terabyte-scale data
Can you hear me? There he is. Hey, Hubert. Okay. I'm here. I'm here. Yes. So I'm curious about the data challenges. So you've got these huge, huge amount of data sitting around. So in your work, what's a current data challenge that you have, or maybe a prior data challenge you had to solve to keep moving forward? Because many of us work with data science and we know that data science many of us work with data sets that aren't nearly this large and this size has got to bring hardware, high-performance computing, whatever challenges there may have been in there.
Yeah. Thank you for the question. So all my work that I do is on a high-performance computing cluster. So thankfully with GPUs being more readily available, that's really helpful. One of the biggest challenges, especially working with human data is missing information. Thankfully in this particular data set for the Shiny apps, all of these samples were annotated by the same neuropathologist. So they're all annotated based off the pathology, but any clinical notes such as even reported sex can sometimes be wrong because once you have over 500 individuals and then maybe 32 columns of information, it's just bound that there's going to be typos or mistakes that are within that data set. So I would say one of the biggest challenges is missing information.
And then as far as managing the 100 terabytes, I, from really early on of working with omics data, I was like, this is going to get confusing really quickly and I'm going to need some kind of way to neatly organize all of this. And this is where my, I call it my type A plus personality is really helpful is that I have an Excel sheet. It's all color coordinated by projects. It lists the absolute paths to those projects and any Git repositories that are associated with them. And then the associating metadata with them so that I can be like, oh, what about this project? Where is that data? I have like a master read me.
That is amazing. Type A plus cracks me up. I think I might be type A minus or type B plus plus. Thank you Hubert for that question. Because I was wondering the same too, when you have that much data, how do you possibly manage it all? Because even with much less data, those same problems are unmanageable. Sometimes I think like naming things and storing things is so, so hard.
Okay. Let's hop over to our next question, which is from Noor. And I bet Noor could ask hers live. We'll just have to make sure Noor gets unmuted. Let's see. There you are. Hey, Noor. Okay. Hi, fellow bioinformatician. Very exciting. I was on microbiome though, not less brain, less neuro stuff. But this is sort of like a broad question, but AI is very big now because everyone loves AI, especially people who want to do AI and bioinformatics. And they're like, AI can replace bioinformaticians, obviously. And I'm like, but what are your thoughts about the push to incorporate AI as workflows into omics, as well as the assumption that it can revolutionize bioinformatics? Like I can understand from like a data munging, data cleaning standpoint,
like why it can be incredibly useful because like we have huge datasets. But downstream aspects, I'm still very much like, it's not smart. Well, thank you so much for the question.
I really appreciate that question so much. Cause I would say at first I was really hesitant of using AI. I, you know, there's all the talk of like, oh, like the more you use it, like the, you know, your skill, you're going to become de-skilled. Right. And then I read this book recently that I would recommend. It's called open to work. So it talks about AI and it essentially has like three different buckets of AI that you can like, you can totally push that work into AI without supervision. And then bucket two is AI with supervision. And then bucket three is things that AI can't do and what AI can't do. And I would agree with this is AI isn't creative.
It's using what's already existing to rearrange it, but that's already there. So if you have a novel question, which is what research often is, it's research that we don't know the answer to this yet. So that's why we're going to investigate it. AI can't come up with those novel questions, but as far as using AI for coding, I have found it to be really helpful of being like, okay, here's, you know, I give it my prompt of what I need it to do. And it can generate a script so quickly that normally would have taken me like a half a day. But I do have that skill set of knowing how to read the script, which I do have a fear that some individuals that didn't have that prior background will just be like, hey, this is what I needed to do. And then I'll generate it and then they won't review it. So I think there does need to be a broader conversation as a whole about putting those things in bucket two, where you work with AI, but you continue as that supervision of it. And then recognizing that your creativity and your curiosity and your knowledge is, can't be replaced.
And then recognizing that your creativity and your curiosity and your knowledge is, can't be replaced.
Learning to code
Oh, absolutely. Well, thank you so much, Norah, for asking that question. And that leads me to a follow-up question, which was, where did you learn to code? Was it in undergrad? Was it in your PhD? So I only coded a small bit in my undergrad, and it was actually for an oceanography class that I was taking as an elective.
But I was like, I was blown away. I was like, this is awesome. There was just something about it being like, well, I just quickly generated this code and now it made this beautiful plot that it explains exactly what we were wanting to look at, which was like the Mariana Trench and the length of it. I was like, this is awesome. Because prior to that, as a biology undergrad, it's a lot of wet lab work. And if you don't do those steps perfectly, you're not going to get anything left in your reagents at the end, which I felt like Murphy's law always applied to me in the wet lab. So I think that I really resonated towards coding, where it was okay to make a mistake because you could just quickly change it and then rerun it. And then when it worked, it was like this beautiful, oh, I can immediately see that it worked.
I love it. I feel like when I started coding, I was terrified of making mistakes. I didn't feel that way. I didn't feel like, oh, I can just redo this. I was like, everything must be perfect. And I was terrified of when things wouldn't run because then I would feel so stuck and alone until I could go find a professor and ask them a question.
TGen and the tech stack
Well, I also wanted to ask about TGen itself, because we know, I think you were with City of Hope, right? And we have met Matt Maloney, who's from City of Hope. And so I was wondering how you two were connected and then also what tools you use. So if you learned how to code in oceanography class, what are you coding in now? Are you still using that same tool? And what do people use at TGen for their tech stack?
Yeah. So TGen, which stands for the Translational Genomics Research Institute, is a nonprofit organization. They specialize in investigating rare diseases, primarily a lot of cancers, and they're part of City of Hope. So that's how I know Matt. I didn't work at City of Hope. It's just TGen and City of Hope work together now. But prior to TGen, I was actually at Mayo Clinic.
Oh, nice. Awesome. So I feel like, Matt, when we were talking with him, they used a lot of Python. Are you all using Python, R, a mix of the two? Obviously, you're using ShinyApps. Are you also using Quarto? So I definitely use a mix of both Python and R. I find that Python is really great for a lot of the data processing steps that I need to do before I get my nice table that I then want to plot in R or make ShinyApps with.
I haven't done too much with Quarto. Unfortunately, I would love to. Prior to learning about Quarto, I used to make a lot of R markdowns for things. But I do want to get more into making Quartos and publishing them on dashboards. And I think that would be really useful to have. And then as far as learning coding, that was primarily done in graduate school. And then it's just expanded since then. And I love going to the POSIT conferences to learn about different tools and packages, like dbduck, being able to quickly parse through really large data sets. I'm like, thank you.
Amazing. Yeah. I think that the jump from RMD files to Quarto files, I was really hesitant at first. And I was kind of feeling like, oh, I don't want to make this change. Same thing with RStudio to Positron. But as with so many things, I needed to teach in Quarto. So I just started using it because I needed to teach it and learning it. And now I'm like, I can't go back. It's so magical. But I asked because I know that you write papers. So I, you know, some people do a lot of their publishing in Quarto. Or, you know, do a ton of LaTeX. Who knows? I know that that's still probably in everybody's tool bag. It's still in mine, for sure. Even though I tell myself I need to switch to Typed.
A typical day and workflow
All right. We have more questions in Slido. So let's talk about those. Natasha asked a question. It has an asterisk. So I will ask it. And it says, what does a typical day or week look like for you? What areas do you split your time between, like programming, meetings, et cetera? Thank you, Natasha. Yeah. Yeah. Thank you so much for the question.
I would say I like to always have one focus for the day rather than kind of like splitting up my day. So one day might be dedicated just to coding and research. So really, like, if I need to do something for that project, then I will just be on the computer that entire time. And then other days will be more dedicated towards meetings, collaborations, or going to seminars, or reading papers. So I really like to kind of have one focus for the day. And that's what I do for that day and block out that time accordingly so that I'm not switching between things. Because I find if I do a bunch of coding and then leave it for three hours and then come back to it, I'm like, oh, wait, hang on. Where was I? So I like to kind of just stick with it for as long as I kind of need to.
So it might be three days in a row, or it might just be one day, depending on whatever task it is, if I'm setting up something brand new that it might require, like, more dedicated days in a row of doing that same task. But I would say generally 50% of my time is coding and working with the data sets. And the other 50% of the time is meetings, seminars, reading research papers, and connecting with others.
Okay, cool. I was wondering, is writing a big chunk of that? Do you have like a writing day? Or is writing something that happens more like at the end of something and all of your other writing is more documentation? I've been very curious about documentation lately because I feel like the game has changed a little bit with the way people are using AI. Also, a lot of people are writing documentation with AI in mind, right? Like, not just human readable, but machine readable. So I figured I'd ask. Yeah, I really appreciate that question. I used to have more of like, this is my writing day, which was always on Fridays. I'm like, oh, this seems like a good way to end the work week. But I actually find it best to just dedicate time every single day to writing.
So whether that be like, if that was a day that was dedicated to coding, then at the end of the day, writing out the steps that I did and making sure that it's documented so that my future self will be like, oh, this is where you were at, and this is why you were doing it. And then same if I found anything interesting in the results that I was parsing through, I want to make sure that I write that down, even if it's not beautiful and well written. I have like a notes Word doc where I'm like, I'm just going to copy and paste my things in there. So then when I do have time, that's that day is dedicated to writing. It's not a blank page. I already have pieces that I can start putting the puzzle together.
I feel like this is just a really great opportunity to ask a type A plus plus person questions as a type B plus person who like aspires to be that, but is definitely not. But that's so funny because I kind of leave myself half done things or partially started things as well, because I struggle with task initiation often and blank page syndrome for me is horrible. I don't like it at all. Blank task syndrome is even worse for me. So if I can like start a little bit of a task at the end of one day, and then I come back to my computer and there's like a video that's partially edited or something that's like almost ready to upload, but close, I can get myself so much more quickly into that thing. And I think I'm like that with writing for sure.
Shiny apps and peer review
Okay. Thanks everybody for putting up with my, my intrusive question there. I was just so curious. Um, we had a question from Victoria. It has risen to the top and I love it. Uh, it's got an asterisk, so I will read it for Victoria. It says, are there any particular challenges with peer review and Shiny apps? Do you get any review on your Shiny apps? I haven't gotten any feedback on the Shiny apps particularly from the peer reviewers. But what I love about having like the shinyapps.io account is that I can actually see if anyone has like clicked on it and how much they've used it. So when we do send a paper out for peer review, um, since most people don't have that link, I can be like, all right, well, I'm pretty sure that a peer reviewer clicked on this link. It won't tell me, you know, where they were or anything like that. So it still stays anonymous, but I'm like, oh, at least they're like kind of going through it. But as far as peer reviews for publications, I've never had a peer reviewer actually specifically mentioned the Shiny app.
Oh man. So does that mean that your Shiny apps are done before you send things into review and like you include links and stuff to them? Oh, that's very cool. I was thinking of them as like an after, after the fact thing. No, I really like them to be in the final publication as well. So once the paper is accepted for publication, I want to make sure that there's a Shiny app right there in the publication so that when individuals are reading the paper and they're like, oh, well, what about this or that? Hopefully they can just go to the Shiny app and kind of start playing around with themselves and see if the data will be of use to their own research.
Yeah, that's amazing. It's Shiny as, as a tool for, like you said, sort of extending what you can publish, you can include if you can't include everything in there. I think I've done this by adding a YouTube video to a cover letter one time where I was like, here, watch this YouTube video that I made, which actually was great. And I got that job. Oh, that's awesome. I love that idea.
I mean, highly recommend. I feel like I come across better this way just to be able to explain something than I do making somebody read a whole paragraph. And I feel like the Shiny app is sort of that as well. Like here, go play with this, be hands-on with it and be able to click around.
I see that you have things deployed on shinyapps.io. I have to ask, are you ready for the switch to Posit Connect cloud? Because there is a migration coming. I actually cannot wait. I'm so excited because once I saw how the layout is where you can add like a little snapshot of like what the homepage is of the Shiny app and then have them all listed, it sings to me. I'm more than ready. It's very exciting. I know that some people are nervous about it. Don't be nervous. It's very exciting. It's only nervous because we're all just kind of used to shinyapps.io, but I promise Connect cloud is featureful and amazing and it does all of this same stuff. And I love the vanity links. I love all of it.
Reticulate and bridging Python and R
Okay. Let's see. What do we have next on our Slido here? I had one from Saptarishi, which also has an asterisk. I will read it for Saptarishi. It says, have you used reticulate to work between Python and R? Unfortunately, I've only had to use it once. And it didn't work well in our HPC environment the way that we have it set up with containerization. So, no, unfortunately. But I love the idea of it and I wish I could work with it more easily. But I think that's something that I'll be working with our HPC team to get that more easily implemented because I think it would be really helpful to just be able to more easily bridge R and Python together.
Passing objects back and forth, I know that that's definitely one thing I've used it for but you're right, it's challenging in some of those containerized environments.
Are you using an IDE that lets you do both like Positron or are you like working in VS
Code for Python and RStudio for R? What does your IDE tool stack look like?
Yeah, so with the Posit Workbench where I'm personally using Positron, which coming more from the RStudio at first I was like oh this this seems like too much but then once I started playing around with it I was like oh no this is the way and I would say like honestly my favorite feature of it, which I know RStudio now has but where it just automatically plots the distribution of your data and lets you know if there's anything that's missing.
I love it, it's so helpful. Oh, you mean the Data Explorer? Yes. Yes, yeah that has been one of my favorite things too. I was just talking with somebody who was like I cannot let go of RStudio and I'm like first of all you don't have to let go of RStudio, it's not going anywhere, but if you're curious about playing in Positron the one thing that might get you is the Data Explorer because it got me and then also we had Nick Crane from the Arrow team join us on the Data Science Lab a while ago and we found out that the Positron Data Explorer just natively displays parquet files as well and I was like that's enough for me to be like this is everyone to go use this and you don't even have to load your data in it's just in your folder and you click it without loading it in and it just loads it shows you what's missing and it shows you histograms of all of your your columns and all their data types. I'm not going to share my screen because we're at the Data Science Hangout, not the lab, but everybody just know that I'm exuberant about that feature.
Data privacy and Shiny apps for omics data
All right Saptarishi, thank you for asking that wonderful question. We have another question from Noor. Noor, would you like to ask your question live? Wait, Noor has two questions. I will let you pick one. I can only choose one, that's so mean. Well, you got to pick one or the other I guess is what I mean. Fair, fair. I'll ask the more relevant one because the first one is probably more of a point of contention argument one, but when you're making Shiny apps, what are some of the limitations you've noticed with working with omics data, especially with things like PII, PHI and so forth? I know that's a big issue working with genomics, especially when yes, you have public databases, but often you have your own data and that has a lot more limitations, especially with privacy and so forth.
Yeah, another wonderful question. Since much of the data that I work with is human data, it's really important that they're de-identified or the type of data that we're working with, it can be made publicly available and if it can't be made publicly available, then I absolutely would not make a Shiny app for it. Then it's something that would have to be where you'd have to have special permissions in order to work with it. It's only data that has been already approved prior consent from both the individuals prior to them deceasing that they were okay with their data being used in this way, and then IRB approval to make sure that we're checking all those boxes and that everything that we're doing is in line with ethical regulations to make sure that we're being respectful of the data. I think that's so important in scientific research is that trust aspect of that we're not going to be misusing the data in any kind of way or making it more accessible than how they wanted it or intended it to be used.
I think that's so important in scientific research is that trust aspect of that we're not going to be misusing the data in any kind of way or making it more accessible than how they wanted it or intended it to be used.
There are limitations for some of the data sets to where we don't have permissions to do that, so I would not make a Shiny app for it and then other permissions where we do and they want it to be readily accessible to the community, in which case then I would accompany a Shiny app. In addition to almost all of our projects, the raw data as well as the process data also are available. There's resources like SRA, which stands for the Short Read Archive, that's hosted by the NIH where you can deposit your raw sequencing data so that people don't have to trust you of being like, oh yeah, they analyze this perfectly the same way that I would have analyzed it. Maybe they want to look at it differently and then you can deposit your data on there to make it even the raw data available.
Thank you so much, Norah, for asking your questions. I just put a poll in the Slido. If anybody would like to head to Slido and go to the top and change the little slider to polls, I sure hope that's still how the UI works on your end because it used to work that way and you can go answer questions. But also let me know if I'm missing a category. The question I asked is, if you code in any language, where or when did you learn to code? I put on my own, which is like private courses, learning online, reading from books. I also put in school, but then also when I got on the job and that was like from co-workers or on my own so that I might be missing a category, but those are the ones that I was thinking about.
And Norah has also staged her other question she was talking about that could be contagious to everybody in the chat, so go check out her question in the chat and answer. And we'll come back to our poll and see our results in just a minute. All right, thank you, Norah. Let's head over to Ken. Ken had a question. Let's see. Ken Roosevelt, if you are available to ask that live, that'd be great. Sorry, Ken. I went to press unmute again and I muted you. It happens. I do this all the time. There you go. Hey, how's it going?
Predicting rare events and variable sparsity
I don't know if my camera's on. I tried turning it on. Oh, it's covered. Hi. Okay, so is your data limited? I have a multi-part question, so I'll give it to you in pieces if that's all right. Is your data limited to patients with Alzheimer's or do you have some large data sets where the set of patients is much broader and where you're trying to predict the likelihood of Alzheimer's developing like a rare event?
Yeah, really great question. So the data set includes controls that don't have any pathology. They died of other causes, as well as the individuals that have just pure Lewy body disease, as well as individuals that have both Lewy body disease and Alzheimer's disease. And then, of course, every single one of those individuals, they don't fit in this perfect little bin. They have other kind of... It's very just heterogeneous, I would say. But it is a large data set. So there's a little over 600 individuals in that one particular data set. And then that's just one project. We also have other projects looking at sepsis. But same thing that's on a spectrum of potentially how quickly they had sepsis, how long they had it for. And for those that don't know what sepsis is, your body is like severe reaction to an infection and it can cause long-term organ damage and long-term cognitive decline as well.
Okay. Just for context, I work at a hospital and I'm working when I have time on a model that is going to predict falls. So I have a very broad set of covariates that I'm mining out, but a relatively rare event. So that leads to my next question. So again, multi-part. If so, what are your go-to methods for predicting rare events? And then in my reply there on quarto, it says sparsity. So predicting rare events, sparsity, and this one's a big one for me, are variables that are partial substitutes for other variables within your data. For example, you can have labs represent like a score in a lab can present a diagnosis. So the two things are saying the same thing, but different ways. Or a procedure can be a response, a treatment for that diagnosis in that lab. So all these things can be variables within your data set. And it's like, okay, how do I, but not all of those will be populated. So you don't want to just eliminate one. So what do you do? I'm just interested in your approaches.
Yeah. So, oh, there's an echo. Oh, we have an echo. Okay. So firstly, you're right about the sparsity. So you can do things like, I'm sure you're well aware of, of imputing. But if you have more than half of your samples are missing that variable, then I don't know if you would want to impute. So one of my favorite packages is actually called variance partition. So to look at how much of the different covariates can be, how much variation can be explained for each of those covariates in the model. And then also determining the most parsimonious model. This is probably getting a little bit too on the statistical side, but it will be great for those that love statistics, but making sure that you're picking a parsimonious model. So there's different approaches for that. So I use Bayesian information criteria for that.
And essentially what that's doing is if you have, you know, a hundred different variables, age, sex, postmortem interval, how long they had the disease for, other symptoms that they might have had, other diseases, medications, how much of those different variables contribute to the overall variation that you're seeing in the data. And that's, I think essentially the type of question that you were asking Kenneth. But the other part that you were asking about this like prediction, can we predict early on, you know, how long they were going to have the disease for or the severity of it. And that's work that I'm hoping to do in the very near future now that I have multiple omics layers for the same set of individuals. So I have genomics, transcriptomics, and proteomics. And with that, it's a little bit different than just the clinical data, because I actually do have each one of those layers for every single one of those individuals. So there's not missing information in that sense. But the question is, is based off of these omic layers, do the combination of them, can we use that as like a system level approach for predicting their neuropathological outcome that they had, so that we can use that as early biomarkers and detectors, which would ultimately help individuals, because one of the biggest things with Alzheimer's disease and Lewy body disease treatment is being able to identify it early on.
It's often that it's when they have symptoms, and that's when you're like, oh, well, you have Alzheimer's disease, and that's unfortunate, and we'll do the best. But if you can detect it very early on, then there actually is more treatment options available to those individuals.
Thank you so much, Ken, for your question. And I love the connection of predicting falls also with Alzheimer's, because my grandmother who had Alzheimer's fell several times and made things much, much worse. And I think that these are problems that probably go together quite often.
Polling results and dimension reduction
All right, let's head back over to Slido. We have 34 votes in for our Slido poll, which was, if you code in any language, where did you learn to code? 35% of us said in school, 38% of us said on my own, and 26% said when I got to the job. So almost equal on that in school versus on my own. And as we get more votes coming in, I think that we'll see those equalize a little bit more. Actually, they just did. They're both at 37%. If you are somebody who learned on your own, do you wish that you went to school? If you're somebody who learned in school, do you wish that you did it on your own? That's just a personal question. Let me know in the chat, because I talk to people all the time who are like, oh, I wish I'd done it differently, right? The grass is always greener.
Okay, we have 12 more minutes here. We have time for a couple more questions. But we also have such great conversation happening in the chat that I kind of want to bring it in a little bit, because it is about dimension reduction. Connor was asking about dimension reduction. I think this was in the context of Ken asking about missingness or dealing with missing data. Do you have any comment, Kimberly, on dimension reduction, or do you use that in your work at all?
Yeah, so we definitely use it in our work. It's extremely useful for single-cell data. So that Shiny app that was shared with those two plots side by side of actually clustering the neurons together versus like the microglia together. So dementia dimensionality reduction is really important for omics data, and it was difficult to follow the whole chat thread, but I loved that someone was talking about like, well, it's not randomly distributed, and how do you take these things into consideration with missingness? And I mentioned earlier like the omics data that we don't have missing data, but in omics data for an individual, there is sparsity where you could have that gene just didn't get the proper amount of coverage for it, but it doesn't necessarily mean that they don't have that gene or they're not expressing that gene. It's just the sparsity of the data. But they're absolutely right in that at least for an omics perspective, it's not random, especially for genes that are expressed. They tend to be co-expressed together, so you can have a set of genes that are co-expressed together, and often what has historically been viewed at for understanding these research questions for like, oh, is this gene expressed in this disease versus in the control samples? We're just looking at an individual gene perspective, but when you understand that the genes are co-expressed together, that does help for that system level analysis. Yes, WGCNA!
Raha and Kimberly should get together after this. I have to introduce you two. I keep laughing thinking, oh, these two would have so much to talk about.
Okay, if anybody wants to add a link to WGCNA analysis stuff in there for the rest of us plebs to learn about it, that would be amazing. Yeah, and they have a new one, HD-WGCNA, which is for that high dimensional weighted gene co-expression network analysis.
I love that everybody threw in their little two cents on their favorite types of dimension reduction as well. There are some, I feel like, fields where PCA is really, really valuable and used a lot and like maybe not so much in this instance. Is there a certain type of method of dimension reduction that is your go-to? For single cell data, it's often UMAPs, but the UMAPs are also priorly built off of the PCA. We're on that knee for the PCA of how many components are actually contributing to the overall variation, but then you also have additional dimensionality reduction, which is those UMAPs, which is uniform manifold approximation projection. Oh man, I was about to say, let's define UMAPs. You didn't. You remembered the full acronym. I'm so proud. Yeah, so please don't ask me to spell out the math without a cheat sheet. I'm having flashbacks to interpreting scree plots incorrectly, like those elbow plots for PCA, feeling so much anxiety around them, but I do absolutely love PCA.
If anybody, I'll put a link in the chat. We can't talk about PCA for eight minutes or else I would do it. Okay, thank you so much for fantastic discussion in the chat. We should do more of this. If you want to talk more about dimension reduction and PCA and UMAP, let's do it at the Discord. You can do it in Data Science Hangout channel as well.
Tips for making Shiny apps
Okay, let's go over to Natasha's question, which was, do you have any tips for making Shiny apps? For example, how to make them clearer or to use good ways of making them scalable? Maybe you have some tips in your pocket that you've learned over the years. Yeah, wonderful question, because one of the challenges that I had with the Shiny app that was shared, which that Shiny app, big shout out to ShinyCell. So those were the developers that actually made that package with the beautiful tabs and stuff, and then I was able to use ShinyCell to make my Shiny app with my data, but use their already kind of pre-built nice app for it with the beautiful tabs and being able to parse the data.
But one of the challenges that I had with it is because since we have almost four million individual cell points, is that the data set was quite large and even with the shinyapps.io I was having difficulties of it like uploading because then it would crash. So one of the things that I did to make it more feasible was not all the genes need to be in part of that matrix up within that our object, because some of them are just so lowly expressed that they're not contributing to the overall variation, they're not contributing to our differences in our disease group. So I was like one way to reduce this would just to be to remove those from the overall objects. They weren't removed from the total analysis or the results from the publication or anything like that, but it just helps to reduce the overall size of that object. So then that way I could make the Shiny app, but that is one of the challenges with working such large data is being able to be like, okay I can't throw in the whole pie, but let me figure out the biggest pieces that I can that can still give you that full picture.
I can't throw in the whole pie, but let me figure out the biggest pieces that I can that can still give you that full picture.
And then as far as like, and I think this is true even of coding, I love to find an example and run through that example set to make sure that it works to be like, okay, I know how to make a Shiny app, even if it was something totally unrelated, maybe like sales or something, but something somewhere someone posted of like, here's how to do it. And then once I've done a template run through, then that's where I feel more confident to make my own tweaks for it of how I want to put one together.
Fantastic. That is my suggestion as well. Make the smallest possible thing that works. Daniel Chen, who teaches data science at UBC joined us on the data science lab this week to talk about how he gets that working example going. And he used cloud code to sort of just like spin up a working example. And then he walked through all the basic components of it, which was really, really helpful. Things like that, I think are the best possible advice.
Career advice
And speaking of advice, we have five minutes left, four minutes left. I would love to ask you a question we ask everybody here, which is, do you have a piece of career advice for us? Maybe something you wish you knew before you got into the field that you're in or just something you might, you think might help others? Yeah, I would say be authentic to yourself, which I think every single person is curious and creative and confident and courageous. And for whatever reason as society, maybe sometimes we're told like, oh, like, don't ask that silly question. I've been told throughout my career, there's no such thing as a silly question. A silly question is one that you don't ask. So I would just say, be yourself and ask, ask questions, you know, be open to listening to others and explore, have fun with it. That is the best advice, be yourself and have fun.
Follow the fun, what feels fun. I think that I got a lot of advice when I was in undergrad and grad school that was like, follow the difficulty, follow the challenge, follow like, what feels hard. And that didn't get me anywhere. When I followed what felt right, I began to get somewhere and I, you know, didn't do that on purpose. I did it on accident and I felt I was battling myself the entire time. Yeah, when I graduated with my undergrad, my grandmother was like, oh, I really hope you find a job that you love and enjoy. And it sounds crazy, but I think that's the first time someone said that to me. I was always under the impression that you have to work for money. That's what you do. You work for money. So when she was like, oh, I hope you find something that, you know, you really enjoy. I was like, oh, well, that does make more sense. And I value my wise grandmother because she lived a wonderful life and was always happy, you know, drinking her wine. And so I was like, you know what, I'm going to listen to my wise grandmother. And I'm so glad I did.
A silly question is one that you don't ask.
Oh, I love that. I think some of us in the chat probably have wise grandmothers that we can share experiences with it as well. I know Rachel definitely does.
Well, this was a wonderful time. I hope everybody had a good one. I know that there were lots of scientific terms. We talked about all kinds of interesting things. If you don't know what any of that stuff is, I encourage you to go Google it, go check out Kimberly's app, and I will share it again in the chat really quickly. If you would like to save your chat, you can do that at the three dots, the top right corner, so that you don't lose our resources that we shared, but also come hang out in the Discord server, and we often share a link to that chat as well or a file.
Kimberly, thank you so much for hanging out with us today, and I hope that you had a really good time. I had a wonderful time. Thank you all so much. Yay, thank you, everybody. We will see you again next week. I hope you have a wonderful day and weekend. Bye, everybody. See you on the Discord.