Posit Assistant is cost-efficient

LLM tokens are expensive, especially if you use coding agents daily. We’ve heard from users at organizations with tight AI budgets, from academics paying out of pocket, and from many others caught off guard by how fast costs add up. In last week’s post in our series on LLM costs, we talked about why you (unfortunately) do need to worry about token caching. Now, we have some good news!

Posit Assistant is designed to make your token budget go further. To see how it compares with other coding agents, we evaluated cost and performance on bluffbench2, a benchmark that measures how well coding agents notice subtle data quality issues while carrying out exploratory data analysis.

On a set of data science tasks, Posit Assistant was 18% cheaper than Claude Code and 25% cheaper than Codex. Posit Assistant and Claude Code performed the tasks similarly well, while transitioning from Codex to Posit Assistant increased scores on the set of tasks by 44%. Posit Assistant is a cost-efficient alternative to leading coding agents.

How we measured cost efficiency#

We evaluated Posit Assistant, Claude Code, and Codex on Posit’s benchmark bluffbench2, which measures how well coding agents notice subtle data quality issues during exploratory data analysis. Posit Assistant never scores worse on this eval than the other harnesses, and is strictly cheaper:

We also evaluated the agents on Python software engineering tasks from DeepSWE 1.1. DeepSWE is currently one of the industry-standard agentic coding benchmarks. On these tasks, Posit Assistant scored about the same as Claude Code and Codex for about the same price (Posit Assistant was 4% and 12% cheaper, respectively).¹

Complete evaluation logs for both evaluations are available here.

How Posit Assistant saves tokens#

Under the hood, coding agents are powered by large sets of prompts–system prompts and tool descriptions. When you send a message like “Hello!”, that message is a very small portion of what’s actually provided to the underlying model. Posit Assistant, Claude Code, and Codex each provide thousands of tokens worth of guidance on how to interact with the user and their environment.

A coding agent’s token efficiency is, partly, a matter of providing this guidance as succinctly as possible. The set of guidance that Posit Assistant provides along with the user’s message is the leanest of this set of coding agents:

This is with Claude Code v2.1.309, Codex v0.161.0, and Posit Assistant v1.9.1. These findings can be reproduced with the code here.

Token efficiency is also a matter of how ergonomic the harness is for the model to work inside of. For example, for Posit Assistant to make a ggplot, it can write something like:

{
  "name": "executeCode",
  "input": {
    "code": "ggplot(penguins, aes(bill_len, body_mass)) + geom_point()",
    "description": "Plot body mass against bill length"
  }
}

In response, Posit Assistant receives the plot image.

In addition to writing the code to create the ggplot, Claude Code and Codex must also write boilerplate code to initialize the R session and save the object as an image file, and then call another tool to read the image file:

{
  "name": "Bash",
  "input": {
    "command": "Rscript -e 'library(ggplot2); p <- ggplot(penguins, aes(bill_len, body_mass)) + geom_point(); ggsave(\"/tmp/penguins.png\", p, width = 6, height = 4)'",
    "description": "Plot body mass against bill length"
  }
}

{
  "name": "Read",
  "input": {
    "file_path": "/tmp/penguins.png"
  }
}

Writing this boilerplate code again and again—and sometimes making mistakes in writing the boilerplate and having to retry—adds up over the course of a conversation. Configuring the agents with an MCP tool like mcp-console, which gives them access to a persistent R session, helps to reduce the amount of boilerplate, but introducing it also increases the number of tokens in the initial payload.

Another ergonomics consideration is how well the harness prevents the agent from “accidentally” reading too much text. For example, a coding agent might read a text file with a file name that makes it seem relevant to the task at hand. It’s possible that that text file is tens of thousands of lines long; the agent probably doesn’t need to see all of them in order to complete its task, and might not have anticipated that there would be so many results. The minimum viable coding agent would provide the agent with the full contents of the file. In this situation, however, Posit Assistant would see a smaller subset of the file and a note that it was truncated, requesting that the agent search inside of the file with a more specific query.

A variety of other features also affect Posit Assistant’s cost-efficiency. For one, when carrying out potentially parallel actions—for example, reading multiple files—Posit Assistant is prompted to carry out those actions in parallel rather than one-by-one. This results in fewer rounds of back-and-forth with the model. Posit Assistant also has near-perfect cache efficiency, meaning that just about every token that could be a cache hit is indeed a cache hit. Additionally, Posit Assistant keeps the cache warm when it suspects that you’ll revisit the conversation soon, which saves you money compared to other coding agents when you take a coffee break between messages.

Cost-effective AI with Posit Assistant#

In the last couple of years, coding and other agents have also become important in many data analysis and scientific workflows. But the cost of using these tools can put them out of reach for many people. Part of our mission at Posit is “[t]o enhance the production and consumption of knowledge by everyone, regardless of economic means.” Making AI more affordable, then, is part of that mission.

Making our flagship coding agent, Posit Assistant, more cost-efficient is one part of this effort. We’ll continue to focus on keeping costs down for our users.

We’ve also focused on making lower-cost models available to our users. Posit AI Pass is a monthly subscription service which provides $20 in credits for use across a number of AI models for $20/month. Alongside the most recent Claude models, we’ve made a number of open-weight models available with strong privacy guarantees and substantially lower per-token prices than their proprietary counterparts. Models like GLM 5.3 Flash and DeepSeek V4.1 Flash 1017 are remarkably capable at their price point and help those monthly credits stretch much further.

Together, we hope these efforts help more people access AI for science and data analysis.

Evaluation details#

bluffbench2#

Posit’s bluffbench2 benchmark measures how well interactive coding agents notice subtle data quality issues during exploratory data analysis. Agents make a number of data visualizations before creating one with a subtle visual artifact. For example, in one sample, an agent creates this plot:

In the above plot, a subset of points are suspiciously straight, suggesting that they could be imputed. To receive full credit on the task, the agent must note an artifact like this one in its summary of the plot. The agent can also receive partial credit by noting the artifact in a follow-up turn, in response to the user asking “What do you see?”. If the agent never mentions the issue, the agent receives no credit on the task. The score on the evaluation as a whole combines scores across many of these tasks.

We ran the evaluation against three setups:

• Posit Assistant has a persistent R (or Python) session built into the harness. The agent runs code in a “normal” R session and sees results, plots, documentation, etc., similarly to how a human would.
• Coding agents like Claude Code and Codex do not have a persistent R session built into their harnesses. Instead, they write small scripts using Rscript to carry out data analysis. Often, to see plots that they create, agents first save the plots to image files on the computer before reading them. In this way, the agents tend to spend more tokens to do the same data analysis tasks due to writing code to work around these limitations in their harnesses. At the same time, it’s reasonable to suspect that these “plain” coding agents might save somewhat on costs since their prompts don’t describe this additional tool for data analysis that Posit Assistant has.
• Coding agents like Claude Code and Codex can also be hooked up to a persistent R session via MCP, using (Posit’s own!) Tomasz Kalinowski’s mcp-console. Adding this additional tool gives those harnesses a more ergonomic interface to R and Python, but also costs slightly more at startup, since additional information about how to use mcp-console is included in the prompt.

DeepSWE 1.1#

As of fall 2026, DeepSWE 1.1 is one of the gold-standard software engineering benchmarks.² In each task, a coding agent is placed in a software repository and asked to carry out a long-horizon coding task, often requiring thousands of lines of code changes. Given the coding capability of modern LLMs, software engineering evaluations must be very difficult in order to prevent the models from acing every task.

Running DeepSWE 1.1 is quite expensive. Running the full evaluation across multiple models and multiple resamples would cost thousands of dollars. To cut down on costs, we ran only the Python tasks, ran only one resample, and only ran the evaluation against two models at the medium thinking level. As a result, we would only be able to statistically detect very large changes in performance.

We ran the evaluation against Claude Code with Sonnet 5.5 (Medium), Codex with GPT 6.1 Sol (Medium), and Posit Assistant with both models. Switching from Claude Code or Codex to Posit Assistant didn’t seem to affect the evaluation score, and the costs to complete the tasks decreased slightly (4% and 12%, respectively), though not substantially enough for us to assert a genuine change. From these scores, we suspect that Posit Assistant is not meaningfully worse for software engineering tasks than Claude Code or Codex. Anecdotally, we use Posit Assistant ourselves when developing Posit Assistant, and do not experience a shift in coding proficiency.