Using Python with RStudio Team

Transcript#

This transcript was generated automatically and may contain errors.

Awesome. So let's start from the beginning. It's a very good place to start. Okay. Perfect. So let's pretend I just introduced you. Everything is working out perfectly. Welcome to everyone to the meetup. And now, David, it looks great on Zoom. There you go.

Okay. So from the top, right, I think the goal here is to provide an overview of the different components of RStudio Team and how they can support the work you're doing in Python. So I'll give a quick sort of overview of those three components, and then we'll jump into sort of trying to solve some problems that you might confront as a data scientist with them.

And so just to outline what they are, and again, I'll stress that they do work independently. So you should feel free to consider the ones that work specifically for your collection of problems, right? RStudio Workbench is going to give you IDs, right? So places to develop and build that code, right? So the RStudio IDE, which you can use to edit Python code, there's some functionality that's been built in there to support doing that, as well as more Python-native editors like Jupyter or VS Code, right? So that's going to give you all of those options. Connect is going to be where you share things with people who you wouldn't expect to be working in a development environment. So applications you built with Shiny or Python frameworks like Dash or Streamlit.

APIs, so if you're trying to facilitate machine-to-machine communication, or reports or documents that you want to share, perhaps on some kind of regular schedule. And then supporting those two services, and where we'll start kind of walking through how these things can be used as package manager, which is going to give you the ability to bring those open-source repositories inside your network, which solves an important networking kind of constraint that some teams have.

It gives you the ability to serve your own internal packages. So if you've already got a development ecosystem of those things, then you can serve those from package manager. And it's going to make it possible for you to impose some reproducibility governance.

So, it's going to give you a lot of freedom in terms of experimentation and just, like, being able to deploy things without worrying about taking other things down.

And so, this is going to take a second to deploy. While that happens, let me pause. Rachel, any questions I should answer, or should I keep going?

Q&A: Conda, Django, and the examples repository

Yeah, let me check over to the Slido and to YouTube here. Thank you all again for switching over with us and bearing with us through any technical difficulties here. As a reminder, you can also ask questions anonymously through Slido.

One question I just had, David, if people want to follow along with some of these examples that you're sharing, where's the best place to get those? Or can you share that GitHub link?

That's an excellent leading question. That's the next link I have here. So, all of the examples, there are a bunch of, as I mentioned, different types of things you can deploy to connect that are Python-based. And so, we have this repository, GitHub.com slash Solange Python examples. And that's just going to give you all of these different types of content. So, Streamlit apps, Flask apps, Reticulated apps. So, that's apps where you say you have a Shiny front end, but a lot of the work is being done by Python, FastAPI, Flask. We've really got sort of everything there. So, feel free to fork this repo. The examples, we try as hard as we can to keep them deployable. So, if you need, say, something to test on your Connect server and fork the repo and deploy it to your Connect server just as a way of understanding the different capabilities that Connect has for supporting Python content.

Awesome. There's a few questions that came through Slido. And one was, I configure my Python environment using Conda. Can I use Conda with RStudio Connect?

The answer is sort of. So, I think Connect's focus is on where Connect does the best is with packages that are on PyPI. So, you can use Conda to manage your environments. And if the packages that you're using are available on PyPI, then it should be reasonably straightforward to restore. There are ways of getting Connect to restore packages that might not be available from PyPI that's a little bit more involved. It's going to involve some collaboration and coordination with your administrator to make that work. So, yeah, I think sort of our workflow is pretty PyPI centered at the moment.

Okay. Thank you. And one other question that came through was, will Django be supported?

Um, I am not the right person to ask about whether Django will be supported. What I would say is that, you know, I think Django tries to do a much bigger set of things than is like the modal or the typical thing that a data scientist is trying to build an app to do. Part of what Connect makes so easy is some of the concerns around user management or things like that, where Django is trying to be a truly sort of full stack web framework. So, you know, people ask occasionally. And I think it's something that we can sort of monitor the interest in. But right now, I'd say it's not something we're thinking about in the immediate collection of priorities.

Jupyter on RStudio Workbench

So, yeah, like I mentioned, there are a bunch of different Python examples that we have available in the repository. So, one of the things I'll just do is jump back to Workbench and open up Jupyter, right, which is going to give you, again, if you're, I'm going to use the classic notebook interface for this example. JupyterLab is also a supported option.

And so, in this case, I've got an example. It's pretty bare bones. It just runs a couple of visualizations and is going to give you some plots that you could share. And so, I think the lesson here is, or the takeaway, right, is that if you're somebody who likes to use Jupyter to do your data science work, right, the experience that you're going to have, right, it should not be substantially different on Workbench. And so, it's going to continue to be that familiar home for you to do some of that development work. There are really only two extensions right now that we provide in, say, the Jupyter classic interface. One of them is just going to take you back to the Workbench homepage. And then the other one is going to support the publishing workflow, that mirrors pretty closely what it looks like to use the push button publishing workflow from the RStudio IDE.

And so, that's just going to mean that, you know, you've done some interactive development in the notebook. You're ready to share those results with other people, and we're going to give you an easy way to push that notebook to a remote server.

So, you can see, right, I've already configured my, I've configured sort of the target server where this is going to be deployed. I'm going to call this presentation notebook.

And then for all kinds of documents that you publish to Connect, and this is, excuse me, both on the R and Python side, you're going to have the option of either publishing the document with source code, and that's going to unlock a lot of the power to have Connect serve, take custody of the document, execute it on a schedule. But also, you know, if you have, say, either dependency conflicts that are too complicated to resolve on the Connect server, or you just, you know, you don't actually want to make the source code available, you can just render the document locally and then push it to the Connect server.

The reason I'm going to call this presentation notebook is that one of the things we enable you to do as you're publishing is specify that you don't actually want people to be able to see the code in the notebook, right? So, this hide all input code cells, and there's a corresponding flag in the command line interface. We'll just make it be the case that, you know, when this notebook is deployed, what we'll see is just the plots, but we won't actually see the code that generated them. And so, for things that you intend to be for, you know, sharing, but not necessarily, you know, a walkthrough of how you got the results, being able to hide those cells is really useful, and you can also hide specific cells if you tag them.

And so, I'll hit publish, right? In this case, I'm getting the new location, right, new title, presentation notebook, and then I'm going to publish this. And so, again, the process I described as we went through deploying a Streamlit application is going to be the same, right, which is we're going to make sure we have an accounting of the files we need. In this case, it's the Python notebook files, the requirements that we have for re-executing this environment, right? So, in addition to, say, Jupyter, right, there's also some NumPy manipulation taking place in this file, and so we need to make sure that that library is installed. In addition to the Jupyter libraries, we're going to push that over to the Connect server. Connect rebuilds this notebook in an isolated environment, and then you can see we're going through the process of rendering the notebook, and once that's done, we'll have something that's ready to present and share with other people.

So, that covers sort of the process of getting things from the development environment to Connect where they're going to be shared, and so the last couple of things I want to do is just show you some of the possibilities once you've deployed a document to Connect, right? So, in this case, you can see, right, the code required to generate these plots not being shown, right? So, you've got a notebook that's presentation ready.

One of the first things you might want to do, right, is make this a little bit easier to find, right? So, when you deploy a piece of content to Connect, you're going to get, you know, it deployed at this unique URL, which is not necessarily the most memorable thing in the world. So, one of the things we enable you to do is set a custom content URL, and that's just going to make it, you know, really simple, give you a memorable, a place with a more memorable name to share that notebook. And the nice thing about that is once, you know, once you've generated that URL, if you open up a new window and you share the notebook with someone there, right, then they're just seeing that content, right? They don't even see any of the sort of Connect UI, right? And so, for, you know, stakeholders who aren't, say, on your data science team, right, this is going to be probably a presentation that they much prefer, right? You know, you're insulating them from, like, the deployment details, and you just get them to focus on the results.

The other thing you're going to have control over is who's allowed to see this notebook, and since Connect sort of integrates with, it can be integrated with whatever enterprise authentication system you're using, right, I could say add the rest of the members of the solutions engineering team as viewers of this content. I could add specific people. I can give them collaborator permissions, which is going to mean that they have the ability to come in and make further modifications to that document. I can make the document more permissive and say, okay, you know, anybody who reaches this server can view this document, but they have to log in first, so you get a really wide range of ways that you can share this content with people in your enterprise.

The other nice thing that you get on the document side is going to be the ability to set things to run on a schedule, right? So, say that rather than a notebook that's just kind of read once, you have something that you want to run repeatedly. This is going to, you know, because you are fetching external data, that might be a nice way to sort of have, say, some monthly reporting or, you know, on whatever schedule is going to work for you, and you can set that to sort of run on a schedule and then be emailed to people you're collaborating with or other viewers.

Q&A: Workbench editors and enterprise setup

Yeah, quick question. I love that now we're making this very interactive here. Somebody asked, they said, I'm a bit actually a lot confused. They expected to see the usual RStudio, like I imagine RStudio on their desktop, but we're curious, what environment is this work being done in?

So, and yeah, if I was not clear enough about that, I'm sorry, right? So, Workbench, right, is gives you the option of starting lots of different editors. So, the Streamlit applications I was working with, right, I'm working with in VS Code, which is, you know, a pretty popular sort of editor for doing lots of kinds of code editing and management. Jupyter is going to support doing a lot of that sort of notebook style development. You can edit Jupyter notebooks in VS Code as well. That's actually my preferred workflow. And then, again, the RStudio IDE is also sort of available to you in Workbench. And so, you get the full sort of range of different things, right? It's a, right, when we talk about it being a home for R and Python, right, it's because you have the ability to run both R as well as like different Python editors. They're, you know, they have access to the same files if you do configuration for access to databases. All that information is going to be sort of consolidated in one place.

Awesome. Thank you. I think also for anybody who's never even seen RStudio Workbench 2, it might be helpful to just cover what's the difference between Workbench and maybe the open source RStudio IDE.

So, you know, Workbench has a few different flavors, right? Or sorry, RStudio has a few different flavors, right? There's the desktop IDE, right, which is the open source one that lots of people use. I used it for a long time before I worked here, right? And that's just going to run on whatever, you know, that runs on Mac OS. That's going to run on Windows. That'll run on Linux desktop if you're that kind of person. And then, there's the open source RStudio server, which gives you the same kind of browser-based capability that I was talking about.

There are a few differences in terms of features and functionality that the professional version of the IDE supports. Some of them are around, like, you know, auditing, just like things that are going to be single sign-on configuration, things that are going to be much more important to enterprise users. And that's where Workbench comes in. So, there are lots of open source ways of running, like, the RStudio IDE. There are, like, Docker images from the Rocker project that make that really easy. I think once people find that they need to run those things in an enterprise context, then Workbench is where they tend to go. And so, Workbench supports, you know, running those multiple editors, the things I was mentioning earlier about running in these different computing contexts. So, if you need to work in Kubernetes or Slurm or some other external resource manager, then that's when you'd be looking for some of the functionalities that Workbench provides.

I mean, that is the, like, I think the core of the workflow, right? We've seen, right, there's, you know, you get the ability to manage packages from package manager. You, you know, Workbench gives you a collection of different editing environments that you can use to get your work done in a way that feels comfortable for you. And it's going to support your team and their diverse preferences really well. And then Connect is going to, you know, be able to run and make it easy for you to share things, whether they're built with Shiny or Dash or Streamlit or what have you. So, I'm happy to just take questions for the remaining time we have here because I think that covers sort of the basic flow.

Further Q&A

I see somebody asked on Slido, can I publish an application with, I'm going to, I'm not sure if I pronounced this right, IPy widgets to allow collaborators to change settings and rerun the notebook?

So, not currently. So, right now, Jupyter, the Jupyter support in Connect is limited to what you think of as the static document model, right? So, there's no running Python process behind the document. I know that that's something we're thinking about in terms of trying to make sure we're supporting the full range of Python sort of interactive use cases. But right now, if you deploy a notebook, then you can do sort of the client-side interactivity I illustrated earlier, but sort of interacting with a running Python process isn't supported today.

Thank you. And I see someone else asked on YouTube, is VS Code just on the latest version of RStudio Workbench? Because I think they have Workbench, but they don't have that yet.

So, yeah, VS Code support would have gone out in beta for like Juliet Rose, which was like 1.4.17. We've since switched to calendar versioning, and you should install the latest anyway. So, if you're running a version that's older than 1.4, it's unlikely that you have VS Code. And I would say that the more recent versions are where that support is going to be the best. So, if you're interested in upgrading, I would just sort of go for the most recent version.

One other question that's on Slido is, is there a substantial difference in configuration when setting up Workbench to allow editing on all the platforms, R or Python, or is that a fairly standard setup?

There is a little bit more configuration to do. So, you know, you go from just installing R, right, to having to install Python, and then the Python packages that support doing these different styles of editing. For Jupyter, for VS Code, you're installing another kind of binary. So, there are a few more steps. It's not, I think, terribly involved. There is some additional work required to get an installation that's fully featured up and running. And then, you know, if you decide you want to, you know, there may be additional complexity depending on like how many instances you want to run of Workbench and if you're trying to cluster them or things like that.

So, somebody asked on YouTube, just a clarification about RStudio team and if that was a name for the triple threat combo of Workbench, Package Manager, and Connect. And yes, that is absolutely correct. So, RStudio team is RStudio Workbench, Package Manager, and RStudio Connect.

Someone asked anonymously, what switches are you using on your mechanical keyboard? They are, excuse me, they are Kelly Golds. Yeah, I have a ErgoDocs split keyboard and Kelly Golds, which is, I'm glad you can tell.

I thought someone was going to ask if they had to use virtual environments, and the answer is yes. You always have to use a virtual environment. Always.

I thought someone was going to ask if they had to use virtual environments, and the answer is yes. You always have to use a virtual environment. Always.