Blog on Posit Open Source

Introducing Toolbars: Supercharge your Cards and Inputs

Liz Nelson — Tue, 26 May 2026 00:00:00 +0000

You’re building a dashboard. It’s looking pretty good. But every time you talk to your users, someone comes up with another must-have filter or button. Maybe you’re even adding an AI chat feature for the first time and realizing that you’ve got less space than it takes to write “this text box is smaller than I thought” to fit all those buttons and selects.

Good news! Toolbars are here to save you.

Their small size and low-profile aesthetic is perfect for keeping your favorite filters and buttons compact and legible. They’re modular and they work in some of your favorite existing components. Seeing is believing, so let’s jump in!

We’re going to provide some examples of classic spots where you might want a toolbar.

Short on time?

Jump straight to the functionality you’re most interested in:

Toolbars in Card Headers & Footers
Toolbars in Input Labels
Complex Text Inputs with Toolbars

Getting toolbars

Toolbars ship in bslib 0.11.0 (R) and py-shiny 1.6.0 (Python).

R
Python

install.packages("bslib")

pip install --upgrade shiny

Our Example

We’re going to build a version of this app piece by piece:

Setup Code

Let’s get you set up with a basic app with some cards. We’ll use this canvas as a starting point for our toolbar components.

R
Python

app_1.R
library(shiny)
library(bslib)

ui <- page_fluid(
  h2("Example App"),
  layout_columns(
    card(
      card_header(
        "Toolbar in header & label"
      ),
      card_body(
        p("Placeholder text.")
      )
    ),
    card(
      card_header(
        "Toolbar in header, footer, label"
      ),
      card_body(
        p("More placeholder text.")
      ),
      card_footer(
        p("Footer content here.")
      )
    ),
    col_widths = c(6, 6)
  ),
  card(
    card_header(
      "Toolbar in Text Input Submit Area: Message Composer"
    ),
    card_body(
      layout_columns(
        p("Placeholder for an input"),
        div(
          p("Placeholder for outputs")
        ),
        col_widths = c(6, 6)
      )
    )
  )
)

server <- function(input, output, session) {
  # Placeholder
}

shinyApp(ui, server)

app_1.py
from faicons import icon_svg

from shiny import App, Inputs, Outputs, Session, reactive, render, ui

app_ui = ui.page_fluid(
    ui.h2("Example App"),
    ui.layout_columns(
        ui.card(
            ui.card_header(
                "Toolbar in header & label",
            ),
            ui.card_body(ui.p("Placeholder text.")),
        ),
        ui.card(
            ui.card_header(
                "Toolbar in header, footer, label",
            ),
            ui.card_body(ui.p("More placeholder text.")),
            ui.card_footer(ui.p("Footer content here.")),
        ),
        col_widths=[6, 6],
    ),
    ui.card(
        ui.card_header(
            "Toolbar in Text Input Submit Area: Message Composer",
        ),
        ui.card_body(
            ui.layout_columns(
                ui.p("Placeholder for an input"),
                ui.div(ui.p("Placeholder for outputs")),
                col_widths=[6, 6],
            ),
        ),
    ),
)


def server(input: Inputs, output: Outputs, session: Session) -> None:
    pass


app = App(app_ui, server)

A toolbar on its own is pretty simple. All you have to do is:

R
Python

toolbar(align = "right")

ui.toolbar(align="right")

That’s just an empty container. The real fun is when we start adding our toolbar inputs and connecting them to card contents.

Toolbars in Card Headers & Footers

Toolbars are great for card headers and footers because they provide a clear way to associate a set of controls with a given subset of the app. For example, you can see below that the header toolbar provides controls that clearly correspond to the contents of the card.

R
Python

card_header(
  "Sales Data",
  toolbar(
    align = "right",
    toolbar_input_select(
      id = "region_filter",
      label = "Region",
      choices = c("All", "North", "South", "East", "West"),
      icon = icon("filter")
    ),
    toolbar_divider(),
    toolbar_input_button(
      id = "save_btn",
      label = "Save",
      icon = icon("floppy-disk")
    )
  )
)

Complete app code

AI Newsletter: Gemma 4 in Posit Assistant

Sara Altman — Fri, 22 May 2026 00:00:00 +0000

External news

Through 2023 and 2024, the dominant narrative on AI token pricing was that the cost to use AI was dramatically decreasing and would continue to do so . However, in the last year, that trend is reversing course.

On Tuesday, Google released Gemini 3.5 Flash , and priced it at three times the previous Flash model, Gemini 3 Flash.
GPT-5.5 , released last month, is twice as expensive as GPT-5.4.
Opus 4.7 , also released last month, has the same price per token as Opus 4.6, but uses an updated tokenizer that results in ~45% more tokens for the same text .

Combined with changes to subscription pricing ( Copilot , Codex , Claude ), this suggests that AI prices are on the rise. Over the last few years, the major model providers have subsidized tokens to acquire users, and it seems like that period is ending.

Costs to run the Artificial Analysis benchmark suite ( https://artificialanalysis.ai/ ) are higher for recent models compared to their direct predecessors.

You can read more about this trend, and also how it shaped the thinking behind pricing Posit AI, in this blog post: Posit AI is priced for the long run .

Posit news

Gemma 4 in Posit Assistant

Gemma 4 is now available in Posit Assistant via the Posit AI provider. Gemma 4 is a relatively small open-weights model released by Google Gemini, and one example of recent improvements in local models small enough to run on a laptop.

Gemma 4 selected in the Posit Assistant model picker.

The primary motivation for including a model like Gemma 4 in Posit AI is cost. We wanted to include a budget model that is still capable and reliable enough to power Posit Assistant across a wide range of tasks.

We recommend using the more capable models with longer-running tasks (e.g., implementing a feature that touches code across many files in a package), but Gemma is still capable of assisting with basic data analysis. You can read more about model choice in the announcement blog post .

Tidy design principles

Hadley Wickham restarted his Tidy design principles substack this week, starting with a thoughtful reflection on the tension between LLMs’ helpful and harmful effects.

The following is an excerpt of that post, but we encourage you to read the entire thing.

In future posts, I’ll get more technical, but I wanted to begin by acknowledging your likely deeply conflicted feelings about AI.

Programming accessibility. There are tons of people who could benefit from a programming language like R, but can’t justify the investment in learning it. AI has dramatically lowered that barrier and you can now get many of the benefits of reproducible programming with R much faster than you could before. Similarly, effective usage of git is now within reach to a much broader audience.

Translation. While machine translations are still far from perfect, their quality has improved radically in the last few years. This means that much more of the programming ecosystem is now available to the majority of the world who are not fluent English speakers.

Voice input. Voice input is a super exciting technology because it means that you no longer need to be a fluent touch typist in order to quickly get your thoughts into a computer. (Not to mention making a lot more technology available if you can’t read or write.) That’s a meaningful expansion of who gets to participate in technology.

Wide and shallow expertise. I love Tukey’s quote that statisticians get to play in everyone’s backyard. And it’s now easier than ever thanks to AI. AI will not make you an expert but can give you shallow expertise in basically anything you’re curious about. I think that’s pretty cool.

Finally, I have found AI to be a tremendous accelerator in my own work. It’s allowed me to fix 100s of issue in core infrastructure packages like roxygen2 and testthat. This is not AI slop; this is carefully vetted code that I can now write ~2-5x faster than I could before.

But you can’t use AI without also considering the harms, of which there are many.

Environmental impact. At the individual level, I believe that if you want to reduce your environmental footprint, there are higher-leverage changes that you can make. But at the societal level, the picture is more concerning: the rush to create new data centers is increasing need for electricity and water, and leading companies to rollback their climate commitments.

Copyright theft. LLMs are trained on vast quantities of copyrighted material, taken at an unprecedented and industrial scale, without permission or compensation.

Concentration of wealth. The AI craze is pushing more and more money into the hands of fewer and fewer people. I find the concentration of wealth and power into the hands of a very small number of people to be genuinely disturbing and I think is something that we should all be concerned about.

Intellectual laziness. AI supports a kind of shallow engagement where you never have to strain your brain on any task. The path of least resistance is to disengage and just let the model handle it. You no longer have to experience any mental discomfort, and thus you never really learn.

Equity and access. I’ve built my career around open source software, and one of the things I love about it is that it’s available to everyone, everywhere in the world, regardless of their means. That’s not possible with AI. The best tools cost real money, usually charged in US dollars, and that makes them out of reach for a lot of people in a lot of places.

In Defense of YAML

Rich Iannone — Thu, 21 May 2026 00:00:00 +0000

Every programmer has opinions about configuration files. These opinions tend to be strongly held and inversely proportional to the stakes involved. In the last few years, the consensus view has shifted: YAML is bad, TOML is good, and enthusiastic users of YAML just might be plainly uninformed. This post takes a different view. We intend to present an argument for YAML which is grounded in history, its specification, and the state of tooling in 2026.

The case against YAML was, for a long time, a reasonable one. The format attracted its critics for real reasons, through years of surprising behavior that burned even careful users. But the specification evolved, and the tooling is finally catching up. To understand why the current consensus is outdated, we need to trace the lineage of configuration formats themselves, because this sort of argument has played out before.

A brief history of configuration formats

The INI file emerged in the early 1980s alongside MS-DOS and the first versions of Windows.¹ It was the simplest thing that could possibly work: key-value pairs, grouped into sections denoted by square brackets, with semicolons for comments. They are flat, readable, and human-editable. For the needs of that era (like configuring device drivers, specifying font paths, or setting application preferences) it was entirely adequate. Its only real limitation was structural: you could not nest deeper than one level, and there was no formal specification, which meant every parser implemented its own dialect. But for two decades, this was fine.

[boot]
shell=COMMAND.COM
device=HIMEM.SYS

[display]
resolution=640x480
colors=256

Then came XML. In the late 1990s, the enterprise software world adopted angle brackets broadly. XML could represent arbitrary hierarchy. It had schemas, namespaces, transformations. It was self-describing. For a while it seemed as though the debate was settled. But XML configuration files grew large in practice. Anyone who maintained a Java web.xml or an Ant build file in 2003 knows what it was like to edit dozens of nested elements just to change a database connection string. The verbosity made the files difficult to maintain by hand, which is precisely what configuration files demand.

  
    myServlet
    com.example.MyServlet
    
      database.url
      jdbc:postgresql://localhost/mydb

JSON appeared as the lightweight reaction. Douglas Crockford, who claims to have discovered rather than invented the format,² offered the simplicity of the JavaScript object literal: curly braces, square brackets, quoted strings, and a tiny set of types. JSON displaced XML in web APIs through the late 2000s and early 2010s. But as people began using it for configuration (rather than machine-to-machine data exchange), its limitations became apparent. JSON has no comments. It has no multiline strings. Trailing commas are illegal. These are reasonable constraints for a serialization format, but they make JSON miserable for files that humans must author and maintain. The removal of comments from JSON’s spec was, according to Crockford himself, motivated by people abusing them for parsing directives.³ It was the right call for data interchange, but it left a gap.

YAML (2001) and TOML (2013) each arose to fill that gap, and both positioned themselves explicitly against what came before. YAML offered the full expressive power of a serialization language (including arbitrary nesting, multiple documents, references, and custom types) with a syntax built on indentation rather than brackets. TOML, created by Tom Preston-Werner a dozen years later, was a reaction to YAML’s complexity: it aimed to be a “standardized INI” with explicit typing, obvious semantics, and a formal specification.⁴ The pattern repeats in each generation: the previous format’s excess becomes the new format’s founding grievance. What is interesting about the current moment is that YAML’s problems were not inherent to the format’s design. They were artifacts of a particular specification version and the parsers frozen on it.

The case against YAML (as it was)

The criticisms of YAML are not fabricated. They reflect real experiences that real programmers had over many years.

The most infamous problem is the Norway incident, which has become shorthand for YAML’s implicit typing behavior. In YAML 1.1, the bare scalar NO was interpreted as the boolean value false. This meant that a list of country codes would silently transform Norway into a falsehood:

# What you wrote:
countries:
  - dk
  - fi
  - is
  - no
  - se

# What YAML 1.1 parsed:
["dk", "fi", "is", false, "se"]

The same applied to yes, on, off, y, n, and various capitalizations thereof. Ruud van Asseldonk’s widely-circulated “YAML Document from Hell”⁵ catalogued these and other problems: port mappings like 22:22 parsed as sexagesimal (base-60) integers, version numbers like 10.23 parsed as floats rather than strings, date-like values parsed as timestamps,⁶ and tags beginning with ! could trigger arbitrary code execution in some parsers.

This is not only a country-code problem. In data science and machine learning code, n and y are natural variable names:

variables:
  x: features
  n: sample_size
  y: target

Under YAML 1.1’s implicit boolean rules, a parser can resolve those keys as booleans instead of strings:

{"variables": {"x": "features", False: "sample_size", True: "target"}}

These were not edge cases encountered only by the reckless. They emerged from the YAML 1.1 specification’s design philosophy of aggressive implicit typing, where the parser attempted to be “helpful” by guessing the intended type of unquoted values. The intention was readability (you could write true without quotes and get a boolean), but the result was unpredictable behavior in practice. Configuration files are precisely the domain where surprising behavior is least tolerable. They are edited infrequently, often by people who did not write the original file, and a silent misparse can propagate through a system undetected for months.

The complexity of the full specification compounded the problem. The YAML 1.2.2 spec runs to ten chapters with sections numbered four levels deep.⁷ There are dozens of ways to express multiline strings. Anchors and aliases create a reference system that, while powerful, adds conceptual weight far beyond what most configuration tasks require. And the security implications of tag-based object deserialization (the yaml.load() vulnerability in Python) became a well-known attack vector.⁸ All of these criticisms were valid, and they were valid specifically of YAML 1.1 and the tooling ecosystem built around it.

What TOML gets right

TOML deserves some credit. For flat or shallow configuration structures, it is clean, readable, and unambiguous. The syntax is familiar to anyone who has seen an INI file, but with the addition of explicit types, a formal specification, and support for nested tables via dot-separated keys.

Consider a pyproject.toml or a Cargo.toml. These files are typically one or two levels deep, with well-defined sections and predictable content. TOML handles them well. Strings are always quoted, so there is no ambiguity about whether no is a boolean or the word “no”. Integers are integers, floats are floats, and dates are first-class types. Comments work exactly as you would expect. For this class of problem, TOML works well, and its adoption by the Python packaging ecosystem (PEP 518)⁹ and the Rust community (Cargo) makes sense.

TOML also benefits from simplicity of implementation. The specification is short enough that a competent programmer can write a compliant parser in a weekend. This means that the ecosystem of parsers is large, well-tested, and consistent. There is no equivalent of the YAML 1.1/1.2 version split. TOML 1.0 is TOML 1.0, everywhere. These are real advantages.

Where TOML strains

The trouble begins when configuration needs to express depth. TOML’s handling of nested structures relies on either dot-separated section headers ([servers.alpha]) or arrays of tables ([[products]]), both of which become difficult to read as the nesting increases. This is not a theoretical concern: it is the reason that Martin Vejnár, the author of the PyTOML parser, eventually abandoned his own project. When asked whether his library should become a dependency for pip, he declined and explained: “TOML is a bad file format. It looks good at first glance, and for really really trivial things it is probably good. But once I started using it and the configuration schema became more complex, I found the syntax ugly and hard to read”.¹⁰

Consider a moderately complex configuration. In YAML, the indentation communicates the hierarchy at a glance:

services:
  web:
    image: nginx:latest
    environment:
      DB_HOST: postgres
      DB_PORT: 5432
    resources:
      limits:
        memory: 512M
        cpu: "0.5"

The equivalent in TOML requires repeating the full path in each section header:

[services.web]
image = "nginx:latest"

[services.web.environment]
DB_HOST = "postgres"
DB_PORT = 5432

[services.web.resources.limits]
memory = "512M"
cpu = "0.5"

The reader must mentally reconstruct the tree from a flat sequence of qualified names. The StrictYAML documentation measured this concretely: equivalent TOML files use approximately 50% more characters to represent the same data, largely because of the repeated path prefixes.¹¹

There is also the matter of meaningful indentation itself. Python demonstrated decades ago that indentation as structure is not a weakness but a strength: it eliminates the class of bugs where visual structure disagrees with syntactic structure. YAML inherits this property. TOML does not require indentation (though many authors add it voluntarily, as a non-parsed visual aid), which means that the relationship between a key and its containing table exists only in the section header, not in the physical layout of the file. For deeply nested configurations, this makes TOML files harder to scan and harder to edit confidently.

What YAML 1.2 changed

The YAML 1.2 specification was published in 2009, with a clarifying revision (1.2.2) completed in October 2021.¹² Its changes address the complaints described above directly.

The implicit typing that created the Norway problem is gone. In the YAML 1.2 Core Schema, only true and false (and their capitalizations True, False, TRUE, FALSE) are recognized as booleans. The words yes, no, on, off, y, and n are plain strings. Sexagesimal number literals (the 22:22 problem) are removed entirely. Timestamp is no longer a core type, so an unquoted 2026-05-05 is a string under the core schema rather than an automatically detected date. JSON is now a strict, proper subset of YAML 1.2, which means any valid JSON document parses identically as YAML. The tag resolution rules are tightened and clarified. The specification itself, while still substantial, is written more clearly and maintained openly on GitHub.

In short, the YAML that people complain about is YAML 1.1. The specification that actually governs the language today is a different, safer, more predictable document. The problem is that most people’s experience of YAML is mediated not by the specification but by their parser, and for most Python users, that parser has been PyYAML, which implements YAML 1.1 and has not changed its core semantics since 2006.

The Python YAML parser landscape

PyYAML , written by Kirill Simonov in 2006, is the de facto standard YAML library in Python. It wraps LibYAML (a C library) for performance and provides a pure-Python fallback. It is downloaded millions of times per week, it is a dependency of countless packages, and it implements YAML 1.1. This last fact is the root of most YAML complaints in the Python ecosystem. When someone says “YAML parsed my country code as a boolean”, they are describing PyYAML’s behavior, not YAML’s specification. PyYAML’s GitHub repository shows over 200 open issues and 100 open pull requests.¹³ The project is maintained but moves slowly, and a major version bump to YAML 1.2 semantics has not materialized.

The ruamel.yaml library, maintained by Anthon van der Neut, offers YAML 1.2 support with round-trip preservation of comments, flow style, and key order.¹⁴ It is widely used and is significantly more capable than PyYAML for tasks requiring comment preservation or format-aware editing. However, it is primarily a pure-Python implementation in its default round-trip mode, which makes it considerably slower than PyYAML’s C-backed fast path. Its packaging history has also been complicated: namespace package issues and a dependency chain that has occasionally confused deployment pipelines.

StrictYAML takes a different approach entirely, implementing a deliberate subset of YAML with all implicit typing removed, no tags, no anchors, and no flow style.¹⁵ Philosophically it is closer to TOML than to full YAML: a safe, simple format that happens to use YAML’s indentation syntax. It is Python-only, has no implementations in other languages, and does not aim for spec compliance.

What has been missing from this landscape is a library that is fast, fully 1.2-compliant, and simple enough to use as a drop-in replacement for PyYAML’s basic interface.

Introducing py-yaml12

The py-yaml12 library is a YAML 1.2 parser and formatter for Python, implemented in Rust for speed and correctness. It is built on the saphyr crate¹⁶ (a Rust YAML library) and exposes a minimal, focused API: parse_yaml() and read_yaml() for loading, format_yaml() and write_yaml() for serialization.

Simple

The design philosophy is straightforward. For the vast majority of use cases, you work with plain Python builtins end to end: dict, list, int, float, str, and None. There is no special document class, no custom node types in the common path. Because YAML 1.2 is a superset of JSON, all valid JSON parses identically. The library achieves 100% compliance with the yaml-test-suite,¹⁷ the community-maintained corpus of edge cases and conformance tests.

from yaml12 import parse_yaml, format_yaml
from rich.pretty import pprint

config = """
server:
  host: 0.0.0.0
  port: 8080
  debug: false

database:
  url: postgres://localhost/mydb
  pool_size: 5

regions:
  - us-east-1
  - eu-west-1
  - no         # Norway, not false
"""

data = parse_yaml(config)
pprint(data)

{
│   'server': {'host': '0.0.0.0', 'port': 8080, 'debug': False},
│   'database': {'url': 'postgres://localhost/mydb', 'pool_size': 5},
│   'regions': ['us-east-1', 'eu-west-1', 'no']
}

Notice the no in the regions list. Under PyYAML (YAML 1.1), this would silently become False. Under py-yaml12 (YAML 1.2), it is the string "no", as the specification requires. This single behavioral difference encapsulates the entire argument: the format is not broken, the old tooling was.

The file API is similarly direct:

from yaml12 import write_yaml, read_yaml

path = "config.yaml"
write_yaml(data, path)

The round-trip is lossless. Writing a Python dictionary to disk and reading it back produces an identical object:

round_tripped = read_yaml(path)
assert round_tripped == data

print(f"Round-trip matches: {round_tripped == data}")

Round-trip matches: True

For advanced YAML features like tagged values, py-yaml12 provides the Yaml wrapper type. It is opt-in and unnecessary for typical configuration work.

Safe

The defaults in py-yaml12 are not just about ergonomics and simplicity; they also improve safety. PyYAML shows the risk of the opposite approach: treating tags as instructions can execute arbitrary Python code simply by reading a YAML file.¹⁸

For example, someone can produce a YAML file that aliases PyYAML’s Python object-apply tag namespace:

dangerous_yaml = """\
%TAG !py! tag:yaml.org,2002:python/object/apply:
--- !py!builtins.eval
- "(__import__('os').environ.__setitem__('YAML_PAYLOAD_RAN', '1'), {'debug': False, 'retries': 3})[1]"
"""

with open("dangerous.yaml", "w") as f:
    f.write(dangerous_yaml)

Then a user expecting only to load a config file runs that code during parsing:

import yaml

with open("dangerous.yaml") as f:
    data = yaml.load(f, Loader=yaml.Loader)

print(data)

{'debug': False, 'retries': 3}

The yaml.load() call looks like ordinary data loading: it returns an ordinary dictionary. But producing that dictionary executed Python code first. Unless you inspect the YAML itself, nothing in the result tells you that happened.

import os

os.environ["YAML_PAYLOAD_RAN"]

'1'

In contrast, py-yaml12 keeps an unhandled tag as data unless you explicitly opt in:

from yaml12 import read_yaml

read_yaml("dangerous.yaml")

Yaml(value=["(__import__('os').environ.__setitem__('YAML_PAYLOAD_RAN', '1'), {'debug': False, 'retries': 3})[1]"], tag='tag:yaml.org,2002:python/object/apply:builtins.eval')

Fast

Performance is a practical concern for any library that might be called in startup paths or CI pipelines. The py-yaml12 benchmarks¹⁹ compare read and write performance against PyYAML (both its default pure-Python path and the fast CSafeLoader/CSafeDumper backed by LibYAML) and ruamel.yaml, across file sizes ranging from kilobytes to megabytes. Because the core parsing and formatting logic is implemented in compiled Rust rather than interpreted Python, py-yaml12 is competitive with PyYAML’s C extension while maintaining full 1.2 compliance. As of this writing, few other Python libraries offer both.

Conclusion

The YAML-versus-TOML debate, as typically conducted, is an argument against a format that no longer exists in its problematic form. The complaints are real, but they are historical. They describe YAML 1.1 as mediated by PyYAML, not YAML 1.2 as specified and now properly implemented. TOML remains a good choice for shallow, flat configurations, and pyproject.toml is well-suited to its role. But the claim that YAML is inherently unsafe or unpredictable does not hold against a compliant 1.2 parser.

This is, in the end, a familiar pattern in computing. Every generation of configuration format is a correction of the previous generation’s excesses: INI was too flat, so XML added hierarchy; XML was too verbose, so JSON stripped it bare; JSON was too austere for humans, so YAML and TOML each offered different compromises. The interesting question is never “which format is best in the abstract” but “which format, with which tooling, serves this particular task well”. For complex, nested, human-authored configuration, YAML 1.2 with a modern parser is a strong answer. Perhaps in another decade, something new will arise to correct YAML’s remaining rough edges, and the cycle will continue. That is how formats improve.

In the meantime, you can pip install py-yaml12 and see what a modern, spec-compliant YAML experience looks like in Python.

And if you work in R, the r-yaml12 package brings the same benefits: full YAML 1.2 compliance, Rust-backed performance, and safe defaults. Everything good about the Python package is in the R version as well.

Wikipedia, “ INI file ”. ↩︎
Douglas Crockford, “ The JSON Saga ”, presentation at Yahoo, 2011. ↩︎
Douglas Crockford, in the same presentation (see [^2]), explains that comments were removed because “people were using comments to hold parsing directives”. ↩︎
Tom Preston-Werner, “ TOML: Tom’s Obvious Minimal Language ”. ↩︎
Ruud van Asseldonk, “ The yaml document from hell ”, January 2023. ↩︎
Examples include Stack Overflow, “ Skip converting entities while loading a yaml string (using PyYAML) ”; PyYAML issue #382 ; and ruamel.yaml ticket #509 . ↩︎
“ YAML Ain’t Markup Language (YAML) Version 1.2, Revision 1.2.2 ”, October 2021. ↩︎
See CVE-2017-18342 , CVE-2020-1747 , CVE-2020-14343 , PyYAML issue #420 , and the PyYAML wiki page on yaml.load(input) deprecation . ↩︎
Brett Cannon et al., “ PEP 518 – Specifying Minimum Build System Requirements for Python Projects ”, 2016. ↩︎
Martin Vejnár (avakar), comment on avakar/pytoml#15 , May 2016. ↩︎
StrictYAML documentation, “ What is wrong with TOML? ”. ↩︎
“ YAML Ain’t Markup Language (YAML) Version 1.2, Revision 1.2.2 ”, October 2021. ↩︎
PyYAML GitHub repository , accessed April 2026. ↩︎
ruamel.yaml on PyPI . ↩︎
StrictYAML documentation . ↩︎
saphyr crate on crates.io. ↩︎
yaml-test-suite on GitHub. ↩︎
See the PyYAML object-construction examples in issue #420, cited above. ↩︎
py-yaml12, “ Benchmarks ”. ↩︎

Welcome to the New Home for Posit Open Source

Jeroen Janssens — Tue, 19 May 2026 00:00:00 +0000

For over 15 years, Posit has been building wonderful open source software and content. Over the years, that wealth of knowledge expanded across hundreds of package sites, dozens of blogs, and multiple YouTube channels. While this rapid, creative growth was wonderful, it also created a practical problem: it became difficult for you to keep up with it all.

Today, we are changing that. I am very excited to announce the Posit Open Source website, opensource.posit.co . I am incredibly proud of the team that built it.

This new central hub connects you to our open source software, blog posts, videos, events, and cheatsheets. We built the site on three core principles: Curate, Connect, and Contribute.

Curate

It starts with bringing our content together. Charlotte Wickham took over 900 posts that were originally scattered across a dozen different blogs and moved them all into one single open source blog. We also gathered over 1,600 videos that were scattered across multiple YouTube channels. Now, you can find all of this content curated and easily accessible in one place.

Connect

Our software remains front and center. Every package now gets its own profile featuring a short description, links to the repository, a list of contributors, and related resources.

We also made our video content more useful. With the help of Wes McKinney, we built a video transcription pipeline using Whisper and Claude . So far we have transcribed 1,600 videos, which is roughly 800 hours of footage. You can now click any text in the transcript to jump right to that exact moment in the video.

Connecting this ecosystem also requires a clear voice. Isabella Velasquez writes the copy to ensure our story resonates with the community.

Furthermore, the content on the site is heavily interconnected. Events link to talks, talks link to people, and people link to software. These connections, combined with a site-wide search, help you navigate the ecosystem easily. Greg Swinehart led the design to ensure the entire experience is accessible and on-brand.

Contribute

The new website is open source because we want the community to contribute. If you have any feedback or suggestions, please open an issue on our GitHub repository .

Everyone who has contributed to Posit Open Source deserves a profile on the website. If you want to add or update your profile, we encourage you to submit a Pull Request.

Head over to opensource.posit.co to explore the new site. I hope you like it.

Ten Great Things We Added to Great Docs

Rich Iannone — Wed, 13 May 2026 00:00:00 +0000

Great Docs started with a premise: you should be able to point a tool at your Python package and get a documentation site that looks good without any design work. Run great-docs init, run great-docs build, open the result in a browser, and you are done. That three-command workflow has not changed since v0.1, and the simplicity of the entry point is important. But behind that simple surface, ten releases have added a world of capability for when you are ready to go further.

With the release of v0.10.0, we have reached a nice, round number and it’s worth looking back and surveying all the things that were done! So what follows is a survey of ten features (one per release) that represent the range of what Great Docs has become. Taken together, they tell the story of a documentation generator that starts simple and scales with your ambitions.

1. Auto-Discovery

The original release (v0.1) came out with the feature that defines the project’s philosophy. When you run great-docs init, Great Docs inspects your package and discovers its public API automatically. It finds classes, functions, dataclasses, protocols, enumerations, exceptions, type aliases, and more, using a combination of runtime introspection and static analysis via griffe . It detects your docstring style (NumPy, Google, or Sphinx) and writes a great-docs.yml configuration file that captures the full structure of your API.

reference:
  sections:
    - title: Core
      contents: [MyApp, Config, build, preview]
    - title: Utilities
      contents: [parse, validate, format_output]
    - title: Exceptions
      contents: [ConfigError, BuildError]

The practical consequence is that you don’t have to enumerate your exports by hand. If you add a new class or function to your package, the next build picks it up. If you remove something, it disappears from the site. The configuration file exists so you can reorder sections, add display names, or exclude internal symbols, but the default is complete and correct without any having to do any manual intervention.

This was the design decision that everything else built on: if Great Docs can figure something out automatically, it should, and configuration should be for expressing preferences rather than providing information that we could have inferred.

2. SEO and Proofreading

The v0.2 release added two capabilities that address different aspects of the same problem: making sure people can find your documentation and making sure what they find reads well.

On the discoverability side, Great Docs now generates sitemap.xml, robots.txt, canonical URLs, and JSON-LD structured data automatically. Per-page Open Graph meta tags are injected so that when someone shares a link to your documentation, the preview card shows a title, description, and image rather than a generic URL. The great-docs seo audit command checks your site for common problems (e.g., missing descriptions, broken canonical links, orphaned pages, etc.).

great-docs proofread

On the quality side, the great-docs proofread command runs local grammar and spelling checks powered by Harper , a fast grammar checker that runs entirely on your machine. It skips code blocks and YAML frontmatter, ships with a technical dictionary so it does not flag standard programming terms, and supports project-specific custom dictionaries for your package’s terminology. The output is available as plain text or JSON (which is well-suited for CI pipelines).

The combination means that documentation sites built with Great Docs tend to surface well in search results and tend to read cleanly once readers arrive. Neither of these things sound all that exciting, but both matter more than a lot of visual features.

3. Documentation Linting

Release v0.3 introduced great-docs lint, a static analysis tool for your documentation rather than your code. It inspects your package’s public API and checks for problems that would otherwise surface only when a user encounters a confusing or broken page.

great-docs lint

The linter catches missing docstrings (a function exists in your API but has no documentation), broken cross-references (you reference MyClass.process but the method is actually called MyClass.run), style mismatches (you declared NumPy-style docstrings but a function uses Google style), and unknown directives in your docstrings. It produces machine-readable JSON output, making it straightforward to add a lint step to your CI pipeline that fails the build when documentation quality degrades.

This sort of feature is more useful than you’d expect. You might run it, discover three functions that never got docstrings and two cross-references that broke during a refactor, fix them, and then feel quite happy. Using it again and again will probably prevent the same sort of problems from recurring. The goal is that no user should ever land on an API reference page and find it empty or confusing because of some oversight.

4. Internationalization

Great Docs v0.4 made it possible to present your entire documentation site in any of 23 languages with a single configuration option. The actual content of your pages remains in whatever language you write it in (usually English), but every piece of UI text (navbar labels, button tooltips, relative timestamps, search placeholders, accessibility attributes, pagination controls) is automatically translated.

site:
  language: fr

That single line transforms every “Next page” into “Page suivante”, every “Search” into “Rechercher”, and every “2 days ago” into “il y a 2 jours”. Translations include proper plural forms for languages that need them (like Polish, which has three plural forms depending on the number) and localized date expressions.

The reasoning behind this feature is straightforward: if your package has international users (and most packages do, whether they realize it or not), the friction of navigating a site where every label is in a language you do not speak fluently is real. While translating content is a large undertaking, translating UI chrome is something a tool can do for you.

The v0.5 release added a full keyboard shortcut layer that ships with every documentation site. Press / or s to focus search. Press [ and ] to navigate to the previous or next page. Press d to toggle dark mode. Press c to copy the current page as Markdown. Press h or ? to see a help overlay listing all available shortcuts.

All shortcuts are disabled when a text input has focus (so typing in search does not trigger navigation) and respect prefers-reduced-motion for users who have requested reduced animation. The system is enabled by default and can be disabled entirely via keyboard_nav: false in the configuration for projects that don’t want it.

Keyboard navigation is one of those features that, once you have used it for a day, makes every site without it feel slightly slower. The ability to browse through a long user guide using [ and ] without moving your hands to the trackpad is a small daily improvement that compounds over time. It also makes the sites more accessible to users who navigate primarily or exclusively by keyboard.

6. Page Tags and Status Badges

Release v0.6 introduced page-level metadata that surfaces in both the page body and the sidebar, giving documentation sites a lightweight content management layer.

Page tags let you categorize pages by topic using YAML frontmatter:

tags: [Configuration, Theming, Advanced]

Great Docs renders these as pill-shaped links above the page title and auto-generates a tags index page listing all tags across the site with links to their associated pages. If you maintain a large user guide with dozens of pages, tags give readers an alternative navigation path: instead of scrolling through the sidebar, they can click a tag to find all related content. Every User Guide page on the Great Docs site uses page tags, and the Page Tags guide explains how to set them up.

Page status badges mark pages with lifecycle states:

status: beta

The supported statuses are new, beta, deprecated, and experimental, each rendered as a color-coded badge below the page title. In the sidebar, these appear as compact icons so that readers can see at a glance which parts of the API are stable and which are still evolving. Status badges are automatically translated for non-English sites (the same i18n system from v0.4 applies) and include built-in Lucide icons and color schemes.

Together, tags and status badges give documentation authors a way to communicate structure and maturity without writing prose about it. A reader landing on a page immediately knows whether it describes a stable feature or an experiment, and can navigate to related topics through shared tags. The Page Status Badges guide covers the full set of options.

7. Scale-to-Fit

Great Docs v0.7 introduced a solution to a problem that plagues documentation for data-oriented packages: wide tables and HTML widgets that overflow the content column. When a table produced by Great Tables or a Pandas DataFrame is slightly wider than the page, the reader has to scroll horizontally to see a sliver of remaining content, which is a frustrating experience. But when content is dramatically wider than the container, shrinking it would make text unreadably small. So, in essence, different situations demand different responses.

Scale-to-Fit handles both cases. It automatically shrinks targeted elements so they fit within the container width, and falls back to horizontal scrolling when the element would need to shrink beyond a configurable minimum threshold. The default threshold is 60% of natural size, meaning content will shrink up to 40% to avoid scrollbars, but anything more extreme gets a scrollbar instead.

scale_to_fit: true
scale_to_fit_min_scale: 0.6

The feature can be enabled globally (all rendered HTML output gets the treatment), per-page via frontmatter, or manually on individual elements using a :::{.scale-to-fit} div wrapper. The global setting is what most data-oriented packages want: turn it on once and every GT table, every DataFrame display, and every custom widget in your documentation adapts gracefully to the reader’s viewport without any per-page configuration. The Scale-to-Fit guide demonstrates the behavior at different widths.

8. Versioned Documentation

Release v0.8 addressed a fundamental challenge of maintaining documentation for evolving libraries: users on older releases need docs that match the version they have installed. Great Docs makes multi-version documentation a build-time concern rather than a deployment concern. You declare your versions in configuration, and the build system produces a coherent, version-aware site.

versions:
  - "0.3"
  - "0.2"
  - "0.1"

That configuration produces three independent copies of your site: the latest version at the root URL, and previous versions under /v/0.2/ and /v/0.1/. A version selector dropdown appears in the navbar, letting readers switch between versions. Each version can reference an api_snapshot to regenerate API pages from that release (rather than the current code), and pages can use versions frontmatter to declare which releases they apply to. Pages that did not exist in older releases are automatically excluded from older builds.

The system also supports prerelease versions (marked in the dropdown but not served as “latest”), end-of-life versions (visually distinguished), and floating URL aliases like /v/stable/ that always resolve to the current release. Maintainers declare versions once and the build handles the rest. The Great Docs documentation site itself is a good example of this feature in practice: its version selector currently contains ten versions, one for each release from v0.1 through v0.10.0. The Versioned Docs guide covers the full configuration in detail.

9. Color Swatches

Great Docs v0.9 added a shortcode for documenting color palettes interactively. If your package has a theming system, a set of brand colors, or a palette of status indicators, you have probably faced the problem of showing colors in documentation. Screenshots go stale, hand-coded HTML swatches are tedious, and plain text hex codes are meaningless without a visual reference. The color-swatch shortcode turns a YAML file into an interactive palette.

Each swatch displays its hex code, shows RGB and HSL breakdowns on hover, evaluates WCAG and APCA contrast ratios against white and black backgrounds, and lets readers copy the color value with a single click. Two display modes are available: circles (compact, scannable) and rectangles (detailed, with contrast information front and center). Great Docs also ships with built-in presets for its own gradient themes, so you can reference preset="sky" or preset="lilac" without creating a YAML file at all.

For packages that deal with color (charting libraries, design systems, theming frameworks), this feature eliminates a category of maintenance burden. The YAML file is the source of truth; the rendered output is always current, always interactive, and always accessible. The Color Swatches page in the User Guide covers the full range of options.

10. Table Previews and Table Explorer

The most recent release, v0.10.0, added two complementary tools for embedding data directly in documentation pages. If your package works with tabular data (and many Python packages do), showing what a dataset looks like is one of the most common documentation tasks, and also one of the hardest to do well with static publishing.

tbl_preview() generates a self-contained, JavaScript-free HTML table from almost any data source:

from great_docs import tbl_preview

tbl_preview("data/example.parquet")

It accepts Polars DataFrames, Pandas DataFrames, PyArrow Tables, CSV files, Parquet files, and plain Python dictionaries. Each preview includes a colored type badge identifying the source format, compact dtype labels beneath every column header, row-number gutters, automatic head/tail splitting for large tables (showing the first and last rows with an ellipsis in between), and missing-value highlighting. The output works identically in light and dark modes and requires no JavaScript at all.

tbl_explorer() is the interactive counterpart. It embeds all data as inline JSON and progressively enhances a static fallback table with sorting, token-based filtering, pagination, column toggling, copy-to-clipboard, and CSV download. A Quarto shortcode variant lets you embed an explorer directly in a .qmd page by pointing at a data file, without writing any Python.

The distinction between the two is deliberate. tbl_preview() is for showing what data looks like: schema, types, a representative sample. tbl_explorer() is for letting readers interact with data: sort columns to find extremes, filter rows by values, hide irrelevant columns, and export subsets. Together they cover the full spectrum of data documentation needs, from a quick “here is the structure of this DataFrame” to a fully explorable reference dataset. The User Guide has dedicated pages for Table Previews and Table Explorer ; the latter is worth visiting just to try the interactive tables yourself and see how sorting, filtering, and column toggling feel in practice.

What Ten Releases Add Up To

Looking back across ten releases, what is striking is not any single feature but the range of concerns the tool now addresses. Documentation generation is the starting point, but the problems that come after generation (discoverability, quality, accessibility, internationalization, versioning, interactivity) are where most of the ongoing work has gone. A documentation site is not just a rendering of your docstrings; it is a product that needs search engine optimization, proofreading, linting, keyboard accessibility, responsive design, and support for readers across languages and connection speeds.

The guiding principle remains the same as it was in v0.1: you should be able to get a good site with minimal effort! But “minimal effort” scales so, for a new package, that means three commands and zero configuration. For a mature package with international users, multiple supported versions, and extensive user guides, it means a great-docs.yml file that declares your preferences while the tool handles the mechanics of producing a site that meets the standard you have set.

Great Docs is available on PyPI so if you maintain a Python package and want to try it:

pip install great-docs
cd your-python-project
great-docs init
great-docs build
great-docs preview

The documentation site covers everything discussed here and much more. If there is a feature you wish your documentation site had (something that would save you time, help your readers, or make maintenance less painful), we would genuinely like to hear about it. The best ideas for the next ten releases will come from people who use the tool daily and notice what is missing. Open an issue on GitHub with your idea, however rough, and we will take it seriously!

A memory model for production async R: mirai 2.7.0 and mori 0.2.0

Charlie Gao — Tue, 12 May 2026 00:00:00 +0000

Async R has always come with tradeoffs. Parallel batch compute meant duplicated data and an unbounded queue that could push your session out of memory, while Shiny gave you an event loop but background-worker submissions had no way to tell you when to slow down.

Shared memory across processes is standard in other programming languages. In R it’s been either platform-specific (fork on Unix-alikes, unsafe from GUIs like Positron or with multi-threaded code) or file-format-specific (memory-mapped Parquet).

Two CRAN releases address those constraints directly:

mirai 2.7.0 — daemons(memory = ...) puts a hard byte-level cap on the dispatcher queue. try_mirai() returns NULL immediately when that cap is hit, instead of blocking your session. The dispatcher itself runs as an in-process thread now; and through nanonext 1.9.0 underneath, peak memory during serialized sends has halved.
mori 0.2.0 — graduates from experimental status, with a stable wire format underneath. Sub-lists and extracted elements now expose stable path-form names (/mori_abc[2,3]) that any process can attach to via map_shared() directly, without going through the parent.

Building out the foundations

mirai already shipped a lot of what production async needs: microsecond round-trip latency, FIFO scheduling, task cancellation, OpenTelemetry tracing, reproducible parallel RNG, custom serialization for torch / Arrow / polars, an event-driven promises method for Shiny ExtendedTask, and deployment across SSH / Slurm / Posit Workbench.

What these releases add is more foundational. R hasn’t really had a built-in memory model for asynchronous work — no way to size a task queue in bytes, and no portable way to share data across processes without paying for a full copy. mirai 2.7.0 introduces the first: a bounded queue built into the dispatcher, with try_mirai() exposing non-blocking submission under this constraint. mori 0.2.0 brings the second to a stable footing: a self-describing shared-memory format, addressable down to individual list elements by name, that any process on the machine can attach to.

Together they extend what R can credibly be used for — streaming pipelines, multi-stage data flows, and event-loop applications that previously had to reach outside the language.

mirai 2.7.0 — bounded queues and `try_mirai()`

`daemons(memory = ...)`

The dispatcher is the single point in your session where every mirai() call lands before being routed to a daemon. Each call carries its serialized arguments — the queued payload — and that payload sits in memory until a daemon is ready for it. By default the queue is unbounded, which is fine until it isn’t.

daemons() now takes a memory argument that caps the approximate total payload of queued tasks at the dispatcher, in MB:

library(mirai)
daemons(4, memory = 100)  # 100 MB queue capacity

When the queued bytes are below the cap, mirai() calls submit normally. When the cap is reached, mirai() blocks the calling R thread until daemons drain the queue enough for the new payload to fit. Existing code keeps working — the only visible difference is that your session can no longer be pushed out of memory by a runaway producer.

status() surfaces current and peak usage:

status()$memory
#     used     peak capacity 
#     12.4     87.3    100.0 

peak is the high-watermark queued payload over the lifetime of the dispatcher, which gives you an empirical way to size the cap. The recommended workflow is to run a representative workload with memory = NULL first (where capacity reports NA), observe peak, then set memory at or above that watermark.

If profiling isn’t practical, the local machine has to fit your session, the dispatcher queue, and n daemon processes (each a full R session holding an in-flight payload while executing). Half of currently available RAM (ps::ps_system_memory()[["avail"]] / 2e6, in MB) is a reasonable starting memory cap, revised down for large n or payloads. Remote daemons don’t draw on the local budget, so when most workers are remote, the cap can be more generous.

For n itself, one less than your number of CPU cores is the right starting point for CPU-bound work (one core left for your session), while I/O-bound work (API calls, database queries, file reads) can comfortably exceed core count since idle daemons cost little.

The capacity is measured in bytes, not task count. A queue capped at 100 tasks can hold either 100 small tasks or 100 huge ones, and only the second runs out of memory. A queue capped at 100 MB never runs your session out of memory regardless of the mix.

`try_mirai()`

A blocking submit is fine in a batch script, but not an event loop: a Shiny session that calls mirai() from inside an ExtendedTask can’t afford to block while the queue drains, because that same session is also driving the UI for all users.

try_mirai() returns immediately when the queue is full, instead of blocking. The semantics are simple:

If the queue is below capacity, behaves identically to mirai() and returns a mirai.
If the queue is at capacity, returns NULL immediately without blocking, and leaves the caller to decide what to do next.

m <- try_mirai({ slow_thing() }) %||% stop("queue full")

The three response strategies are: drop the task (best when the work is idempotent and frequent), retry later (queue the request behind a later::later() call), or propagate backpressure upstream — as in the example above, by raising a condition. Which is right is application-specific — try_mirai() hands you the flexibility.

If memory isn’t set, or the daemon configuration has no dispatcher, try_mirai() always returns a mirai and is identical to mirai() — so the function is safe to use unconditionally; it only does anything different in the bounded-queue case.

The two halves — daemons(memory = ...) and try_mirai() — are the right combination for any event-loop integration. Set the cap to reflect what your session can actually afford to hold, submit through try_mirai(), and the application gracefully adapts to load instead of locking up or crashing.

Under the hood

A few transport-layer changes ship alongside 2.7.0 — invisible most of the time, important when payloads are large or throughput is high:

Dispatcher as a thread. The dispatcher is no longer a separate R process; it runs as an in-process thread, courtesy of a C-level reimplementation in nanonext. mirai round-trip latency is now in the tens of microseconds.
Halved peak memory on sends. nanonext 1.9.0 , the transport underneath mirai, now writes serialized sends directly instead of going through an intermediate buffer. A 500 MB argument going to a daemon used to need ~1 GB of momentary headroom in your session; now it needs ~500 MB. The dispatcher’s actual resident memory tracks the daemons(memory = ...) cap more accurately as a result.
>2 GB payloads on macOS and Windows. Transfers above 2 GB previously failed silently near the INT_MAX-byte boundary on those platforms. They now go through cleanly. Linux was already fine.

mori 0.2.0 — out of experimental status

mori is a younger package — its first CRAN release was just last month — and its job is the other half of memory pressure: the data itself, rather than the queue. R processes don’t share memory, and data crosses between them through message-passing. When eight workers each need the same 200 MB data frame, that’s 1.6 GB of serialize / transfer / deserialize, plus eight separate copies of the data resident in RAM.

mori::share() writes the object once into OS-level shared memory and returns an ALTREP (alternative representation) wrapper. Multiple processes on the same machine map the same physical pages — zero-copy, lazy, and managed by R’s garbage collector. Pass the wrapper to a daemon and only the shared memory identifier (hundreds of bytes) crosses the wire; the daemon attaches and reads the pages directly.

We implemented this in mori 0.1.0, but 0.2.0 takes mori out of experimental status, landing a stable wire format. A key consequence of this is that every sub-list and extracted element of a shared region now has a stable identifier of its own. shared_name() emits path-form names (/mori_abc_1[2] for the second element of region /mori_abc_1, /mori_abc_1[2,3] for nested addressing), and map_shared() accepts the path form directly.

library(mori)
library(mirai)

lst <- share(
  list(weights = rnorm(1e6), targets = rnorm(1e6), inputs = rnorm(1e6))
)

# Sub-elements have their own path-form names

shared_name(lst)
#> [1] "/mori_abc_1"
shared_name(lst[[1]])
#> [1] "/mori_abc_1[1]"

Pass the shared object directly to a daemon — the ALTREP wrapper serializes as just the name, and the daemon attaches on deserialize:

mirai(length(weights), weights = lst[[1]])[]
#> [1] 1000000

A consumer in a separate session that only has the name as a string — from a config file, message queue, or HTTP response — attaches with map_shared():

# In a separate R session

weights <- mori::map_shared("/mori_abc_1[1]")
length(weights)
#> [1] 1000000

A path-form name is a small thing on the wire – just a string – but it enables a lot:

Attach by name. Path-form identifiers are plain strings, so they fit anywhere a string can go: a queue payload, a config entry, an HTTP response, a database row. A consumer reads the name, calls map_shared(), and gets exactly that slice — the parent region never has to enter their session.
Decoupled pipelines. Stage A shares a 100-column frame and tells stage B the names of the three columns to process. Stage B passes the name of its output to stage C, and so on. No stage holds more in memory than it actually uses, and no stage needs to know how its inputs were produced — only the names.
Code symmetry. map_shared(shared_name(x)) returns an equivalent of x whether x is a root region or a single leaf element, so consumer code doesn’t need a special branch for sub-objects.
Lifetime through one chain. Holding a reference to a leaf keeps the whole parent region alive automatically — the garbage collector walks the chain of dependencies — so a consumer never has to manually pin the root to stay safe.

Composing with `daemons(memory = ...)`

The two memory tools compose. mori::share() shrinks each task’s queued payload from megabytes to hundreds of bytes, which means the dispatcher’s memory cap rarely fills up at all when workers are reading shared data. The pieces are complementary:

mori::share() removes the per-worker copy of large data.
daemons(memory = ...) caps the dispatcher backlog when something is being sent.
try_mirai() makes hitting that cap a non-event in an event loop.

Putting R on the concurrency map

The design of daemons(memory = ...), try_mirai(), and mori::share() draws on how async runtimes in other languages handle the same problems. Here’s the comparison:

Byte-budgeted backpressure. Async frameworks elsewhere — Python’s asyncio.Queue, Ray’s max_pending_tasks, Tokio’s bounded mpsc channels — measure queue capacity by task count. daemons(memory = ...) caps by bytes instead, which tracks what actually drives memory pressure.

Non-blocking submission on a full queue. Tokio’s try_send is the closest direct analogue: it returns immediately when the channel is full instead of blocking. try_mirai() provides the same semantics for R submission, with the additional advantage that the cap itself is byte-aware.

Integrated cross-process zero-copy. Ray’s Plasma object store offers cross-process zero-copy, but it’s a heavyweight component — you run a Ray cluster to use it, not a function. mori::share() is just a function call, producing an ALTREP wrapper that consumer processes attach to via lazy reads, down to individual columns by name.

Combined with mirai’s microsecond round-trip latency and event-driven C-level transport, these three pieces give R a concurrency story that stands on its own: byte-aware backpressure, non-blocking submission, and integrated zero-copy sharing.

Try it

Install from CRAN:

install.packages(c("mirai", "mori"))

Pointers from here:

mirai reference vignette — the memory-management section walks through daemons(memory = ...), status()$memory, try_mirai(), and shared-memory composition.
mirai promises vignette — Shiny ExtendedTask and event-driven async.
mori package site — share(), shared_name() and map_shared().
nanonext package site — the transport layer and concurrency primitives.
The r-lib agent skill in the posit-dev-skills plugin includes an LLM-optimised mirai guide for AI coding assistants.

Issues, feedback, and questions are very welcome on GitHub: r-lib/mirai , r-lib/nanonext , shikokuchuo/mori .

Acknowledgments

A big thanks to everyone who contributed to mirai 2.7.0, mori 0.2.0, and nanonext 1.9.0 through their issues, pull requests, and discussions: @HenrikBengtsson , @kentqin-cve , @king-of-poppk , @manforkr , @mcol , @michaelmayer2 , @pmac0451 , @t-kalinowski , and @wlandau .

Positron May Release Highlights

Julia Silge — Mon, 11 May 2026 00:00:00 +0000

Note

Positron is Posit’s new, next-generation IDE for data science. Positron is designed to be an extensible, polyglot tool for exploring data and reproducible authoring in Python, R, and more.

Welcome back to another edition of our monthly Positron updates! Each month we share highlights from our latest release and useful resources.

Product updates

Announcing the Packages pane

Our Packages pane is available in this release as a preview feature. The new Packages pane brings streamlined package management directly into the IDE so you can see at a glance what’s installed, what’s attached, and what’s out of date.

Find it in the Primary Sidebar by clicking the package icon. Positron automatically detects and integrates with your current interpreter; today we have support for pip, venv, uv, and conda for Python, plus base R, pak, and renv for R. You can search, sort, install, update, remove, filter by category (including “Outdated Packages”), and even see which packages are attached in your current session. To hide the pane, disable positron.packages.enable . Let us know how it fits into your workflow.

Inline output for Quarto

Inline output for .qmd documents was one of Positron’s most-requested features ever, with many users telling us it was the one thing keeping them tied to RStudio or Jupyter notebooks. We are happy to share that Quarto inline output is available this release as a preview feature.

Quarto kernels start automatically when you open a document with inline output, so you don’t wait for a kernel on first execution. Each inline output cell shows a status line with execution status and elapsed time, and outputs are now collapsible when they get long. We’ve also added a tool to switch the interpreter used for inline output, support for Show Notebook Console for Quarto documents, and fixed a handful of papercuts around execution indicators, autocompletion, and the kernel selector. Enable inline output via the positron.quarto.inlineOutput.enabled setting.

Extensions available from Posit Public Package Manager

Positron’s extension gallery now uses Posit Public Package Manager (P3M) by default, replacing Open VSX as the source for browsing and installing extensions. The catalog is the same set of extensions you already use, served from Posit infrastructure for reliable distribution. If you prefer to stay on Open VSX, the new positron.extensions.gallerySource setting lets you switch back at any time. For organizations on Posit Workbench, Posit Package Manager also added support for mirroring and serving VS Code extensions from your own instance, bringing the same governance, security, and air-gapped distribution to extensions that you already have for R and Python packages.

Positron Notebook Editor is now in beta

The Positron Notebook Editor for .ipynb files is officially moving from alpha to beta.

This release brings a wave of polish and reliability we hope makes it ready for your daily workflow:

Cleaner git diffs by default: New notebook.save.outputs and notebook.save.executionCounts settings let you exclude outputs and execution counts from saved notebook files so reviewers see only the relevant code changes.
Stronger R support: R notebooks can now be debugged with breakpoints and the “Debug Cell” action, and R dataframe outputs in notebook cells now render in the inline Data Explorer for the same interactive exploration you get in the Console.
Help on F1: Pressing F1 on a function in a notebook cell now opens the Help pane , just like in .py and .R files. Works for both Python and R Jupyter notebooks.
UX polish: Long cell outputs default to scrolling instead of overwhelming the page, the cell action bar stays visible as you scroll through tall cells, markdown cells render footnotes, and your scroll position is preserved when switching between notebook tabs.

AI chat and code assistance

This release introduces two new AI options: Posit Assistant and Posit AI.

You may have tried some of our previous experiments for AI code assistance available in Positron, Positron Assistant and/or Databot. We are now migrating to a more data-science-specific, holistic, unified approach to AI chat, built on the same Posit Assistant interface that we recently brought to RStudio . We acknowledge that the number of different tools we’ve made available and how their names have evolved have been pretty confusing, but this space is changing fast and our initial experiments have helped us understand the best ways to integrate AI into your data science workflow. With Posit Assistant, instead of switching between tools, you get code generation, next edit suggestions, chat, and agentic tools all in one place. Learn how to get started .
Posit AI is a new, optional model provider service available to use with Posit Assistant. Posit AI is a subscription priced at $20/month for individual users; both Positron and Posit Assistant remain free to use. Learn how to sign up for an account .

This release also provides additional improvements to the broader Assistant experience. Each Assistant response now shows which model generated it, so you always know whether you’re getting output from Claude, GPT-5, or another provider. We also improved the documentation and error handling for Microsoft Foundry and prompt caching for AWS Bedrock.

What’s coming next

Last release , we announced Positron Server for academic use in JupyterHub, making it free to host Positron for teaching. We are now expanding support for Positron Server on Open OnDemand. If you are interested in using Positron for teaching, follow the instructions to get a teaching license and book time with our team to discuss your use case.
We’re prototyping first-class SQL editing and execution in Positron, including support for visualizations with ggsql . We’ll share more as this takes shape, and in the meantime, let us know your thoughts and current pain points.
Join our next webinar on May 12 to learn how Positron, Workbench, Connect, and Package Manager work together in the reproducible analysis lifecycle.

Tip

Download Positron to try out the new features and improvements in this release!

AI Newsletter: A Data Cleaning Mode for Posit Assistant

Sara Altman — Fri, 08 May 2026 00:00:00 +0000

Posit news

The latest release of Posit Assistant includes a Data Cleaning Mode. When Posit Assistant enters the mode, it spends time identifying and fixing data quality issues and preparing your data for analysis. When certain decisions (e.g., how to recode a variable) require user decisions, it surfaces those decisions in a specialized interface. When you and Posit Assistant are done making decisions about the cleaning process, all the cleaning code is written to a script.

Posit Assistant in Data Cleaning Mode, surfacing data quality decisions to the user.

Broadly, we’ve seen that many coding agents seem to have a superficial regard for data quality, focusing only on errors thrown by analysis code. This is one of several features we’re iterating on to tailor the agent experience more closely to the real work of data science.

The initial release of the tabpfn package recently made it to CRAN . TabPFN is a pre-trained neural network for tabular data. In a typical predictive modeling workflow, you train a model on existing data (the training data), and then apply the resulting model to new, unseen data. TabPFN, in contrast, is a neural network pre-trained on a vast array of synthetic datasets. When using TabPFN, the model learns from the analyst’s training data in-context, in a similar way that an LLM can ’learn’ over the course of a conversation, allowing it to predict on new data without an explicit training step. The tabpfn package provides R tidymodels bindings to the pre-trained model.

Terms

TabPFN and LLMs rely on in-context learning to provide more helpful responses.

Many recent model releases have announced increased context windows of 1 million tokens.¹ The context window is the maximum length of the ‘conversation history’. So, if you’ve had 10 messages back and forth with an LLM, and each message was composed of 100 tokens, the context length would be 1,000, well under the context window for most modern LLMs.

LLMs learn in a few different stages. The most recognizable stage is probably pre-training, in which the model gains general knowledge and capabilities by learning from massive collections of text. Another important stage of learning is in-context learning, which happens over the course of a conversation. For example, if you paste a document into an LLM chat and then start asking questions about it, the document has been learned in-context.

TabPFN follows this same learning setup as modern LLMs. The TabPFN team pre-trains the neural network and then allows users to download the resulting weights. Then, the model learns from the analysts’ training data in-context.

Learn more

OpenAI released GPT 5.5 , an incremental improvement over GPT 5.4. Google Gemini released Deep Research Max , an incremental improvement over its Deep Research tool.
Yihui Xie wrote up a thoughtful reflection on his experience with AI-assisted coding .
LLM use for open-source contributions can be controversial. Zig has instituted a ban on LLM contributions . You can read their rationale here .

A token is, roughly, a word. ↩︎

Bringing OpenTelemetry to R in production

Charlie Gao — Thu, 07 May 2026 00:00:00 +0000

We’re bringing OpenTelemetry to R. As a Posit-wide initiative across our open source packages, we’ve instrumented some of the most widely-used R packages for production workloads – Shiny , plumber2 , mirai , httr2 , ellmer , knitr , testthat and DBI . You can add observability to your R applications with no code changes: set a few environment variables and you get traces, logs, and metrics flowing to the backend of your choice.

This is part of our commitment to R in production. As R applications scale – more users, more processes, more machines – you need tools to understand what’s happening across your entire system. That’s what OpenTelemetry is built for, and it’s now available for R.

What is OpenTelemetry?

OpenTelemetry (OTel) is a vendor-neutral, open source observability framework backed by the Cloud Native Computing Foundation . It has broad industry support across languages and platforms, and is already the standard in the Python, Java, JavaScript, and Go ecosystems. Now it’s available for R.

OpenTelemetry defines a standard for collecting telemetry data:

Traces follow an operation as it moves through your system, showing exactly which functions ran, in what order, and how long each took.
Metrics capture numerical measurements over time – things like request counts, response latencies, or memory usage.
Logs record detailed events as they happen, providing context when you need to investigate a specific moment.

The instrumented packages described in this post all use the otel package under the hood, and focus on traces, which provide the most immediate value for understanding production behavior. You can also use otel directly to add your own metrics and logs to your application code.

Why observability matters for R in production

When you’re developing interactively in RStudio or Positron, debugging is straightforward – you can step through code, inspect objects, and add print statements. But when your R code runs in production – a Shiny app serving hundreds of users, a plumber2 API handling thousands of requests, a batch pipeline running across a cluster – the picture changes.

The core concept in OpenTelemetry is a trace: the full path of a request through your system. Each trace is made up of spans – individual units of work with a name and a duration. Spans nest inside each other, so you can follow how each operation triggered the next:

This structure gives you four things that are hard to get any other way:

Performance: Which part of a request is slow? Span durations pinpoint where time is spent – and nesting reveals unnecessary overhead.
Errors: In development and testing, you know where errors are – you wrote the test, you control the inputs. In production, errors surface far from their root cause, across process boundaries and async operations, triggered by conditions you never anticipated. Traces show you the full chain of real operations that led to each failure, in the context where it actually happened.
Centralized view: When your application extends across multiple R processes or machines – a Shiny app with mirai workers, or a plumber2 API behind a load balancer – traces are aggregated into a single view across all of them.
Real-time monitoring: OTel is designed to be left on in production, not just enabled during testing or staging. With low overhead, it runs continuously – so you see what’s happening as it happens, and dashboards and alerts catch problems before users report them.

Instrumented packages

We’ve worked across teams to add OpenTelemetry instrumentation to the R packages where it matters most:

Package	Version	What it traces
Shiny	≥ 1.12.0	Session lifecycle, reactive updates, reactive expressions, background tasks
plumber2	≥ 0.2.0	API request handling, routing, endpoint execution
mirai	≥ 2.5.0	Task dispatch, daemon execution, results
httr2	≥ 1.2.2	HTTP requests and responses
ellmer	≥ 0.4.1	LLM API calls, tool execution, token usage
knitr	≥ 1.51	Document rendering, chunk evaluation
testthat	≥ 3.3.2	Test execution
DBI	≥ 1.3.0	Database queries and connections

Together, these packages cover the most common production R workloads: web applications, APIs, parallel computing, HTTP clients, AI/LLM tools, report rendering pipelines, CI test runs, and database access. Because the instrumentation is built into the packages themselves, you benefit from it automatically – no wrapper functions, nor modifications to existing code. If you already have a Shiny app or plumber2 API, your app will generate traces as soon as you enable OpenTelemetry. Your application code stays exactly as it is.

Seeing it in action

To make this concrete, let’s look at a Shiny chat app built with shinychat and ellmer that fetches weather forecasts. It uses mirai for async execution and httr2 for weather API requests.

A user asks about the weather in Atlanta and Newcastle:

With OpenTelemetry enabled, every step is captured automatically. Here are the traces from those two queries:

The trace reveals the full chain of operations from user input through to the HTTP request, across process boundaries, with no manual logging. The nesting shows how each step triggered the next, and the durations show where time was spent.

The second query failed. Without tracing, you’d see an error in your logs and start investigating. Here, the failure is immediately visible – the red span pinpoints where it occurred and the surrounding context shows why. In a production system with many concurrent users, that’s the difference between minutes and seconds of debugging.

Getting started

Getting started requires the otelsdk package and a few environment variables. No changes to your application code.

Here’s how the pieces fit together: the instrumented R packages generate telemetry data as they run. The otelsdk package collects and exports this data over HTTP to a backend – a service that stores your traces and provides a web dashboard where you can search, filter, and visualize them (like the trace screenshots above).

Step 1: Install otelsdk

install.packages("otelsdk")

Step 2: Choose a backend

OpenTelemetry is vendor-neutral, so you can send your data to any compatible backend:

Cloud services: Logfire , Grafana Cloud , Langfuse
Self-hosted: Jaeger , Zipkin , Prometheus

During development, you can also run an OpenTelemetry Collector on your own machine and inspect traces directly, without sending data anywhere else.

Each backend will give you an endpoint URL and an authentication token. The otelsdk collecting telemetry data guide has examples for some common backends.

Step 3: Set environment variables

Add these to your .Renviron file (use usethis::edit_r_environ() to open it), replacing the endpoint and token with the values from your chosen backend. This example uses Logfire , which offers a free tier to get started:

OTEL_TRACES_EXPORTER="http"
OTEL_EXPORTER_OTLP_ENDPOINT="https://logfire-eu.pydantic.dev"
OTEL_EXPORTER_OTLP_HEADERS="Authorization="

If you’re deploying content to Posit Connect, refer to how to set environment variables on Posit Connect .

Step 4: Run your app

That’s it. Restart R, then run your Shiny app, plumber2 API, or any code that uses the instrumented packages. Traces will flow to your backend automatically. Open your backend’s web dashboard to see them – you’ll see a view like the trace screenshots shown above, with each span representing an operation in your application.

You can verify that tracing is active at any time:

otel::is_tracing_enabled()
#> [1] TRUE

OpenTelemetry is designed to be safe in production. If anything goes wrong in the telemetry code itself, it will never crash your application – errors are silently suppressed so your app keeps running.

Zero-code instrumentation

Beyond the packages that ship with built-in instrumentation, otel supports zero-code instrumentation for any R package. Set the OTEL_R_INSTRUMENT_PKGS environment variable to a comma-separated list of package names, and otel will automatically create spans for their exported functions:

OTEL_R_INSTRUMENT_PKGS=dplyr,tidyr

You can also fine-tune which functions are instrumented using include and exclude filters:

OTEL_R_INSTRUMENT_PKGS_DPLYR_INCLUDE=mutate,filter,select

This is useful for adding visibility to any package in your stack, even those without built-in OTel support. See the otel documentation for full details.

Configuration options

The otelsdk package is configured entirely through environment variables, following OpenTelemetry conventions:

Variable	Purpose
`OTEL_TRACES_EXPORTER`	Exporter type for traces (e.g. `"http"`)
`OTEL_LOGS_EXPORTER`	Exporter type for logs
`OTEL_METRICS_EXPORTER`	Exporter type for metrics
`OTEL_EXPORTER_OTLP_ENDPOINT`	URL of the OTLP-compatible backend
`OTEL_EXPORTER_OTLP_HEADERS`	Authentication headers for the backend
`OTEL_R_INSTRUMENT_PKGS`	Packages for zero-code instrumentation
`OTEL_R_EMIT_SCOPES`	Restrict telemetry to specific packages
`OTEL_R_SUPPRESS_SCOPES`	Exclude specific packages from telemetry

See the otelsdk environment variables reference for the complete list.

Looking ahead

With instrumentation built into the packages that R users already rely on, observability becomes something you can turn on, not something you have to build.

We’re also integrating an OpenTelemetry collector into Posit Connect , giving you an end-to-end observability solution you can simply turn on or off.

OTel support across the ecosystem continues to expand. If you’d like to learn more:

otel package documentation – the instrumentation API
otelsdk package documentation – the SDK for collecting and exporting telemetry
Shiny 1.12 OTel blog post – deep dive into Shiny’s OpenTelemetry support
OpenTelemetry project – the upstream standard

We’re excited about what this opens up for the R community. Whether you’re running a Shiny dashboard for a small team, a plumber2 API serving thousands of requests, or a data pipeline distributed across a cluster – you now have the tools to see what’s happening, in real time, with no code changes required.

Quarto 2: Parsing and Source Maps

Carlos Scheidegger — Thu, 07 May 2026 00:00:00 +0000

This is the first of a series of posts about the design and features in Quarto 2.

UX Requirements for a text-centric authoring system

Although Quarto 2 is now a standalone, new version of the Quarto system, it started as an attempt to solve long-standing parsing problems in Quarto 1. We soon realized there were three fundamental, separate syntax concerns: syntax errors, awareness of source locations during document processing, and syntax stability. We eventually concluded that none of these features could be solved incrementally in Quarto 1, which led to where we are today.

Requirement 1: Syntax errors

Markdown is a very convenient language for lightly formatted text, and its minimalism keeps the source exceedingly readable on its own. Unfortunately, Markdown (in)famously has no syntax errors; every sequence of characters is a valid Markdown document. This is explicitly enshrined in the CommonMark spec :

Any sequence of Unicode characters is a valid Commonmark document.

We believe this to be a fundamentally misguided principle. Instead, we believe that error messages are communication scaffolds, and that accepting error messages as a useful tool better reflects the reality of Markdown authoring in 2026. In short, Quarto has expectations about input documents, and users make typing mistakes.

In the course of teaching Quarto, we repeatedly witness learners make the same classes of Markdown syntax errors when authoring Quarto documents. Let’s take the following, typical example. Quarto makes extensive use of fenced divs, structural elements in Pandoc Markdown documents that can denote a variety of constructs, such as figures, multiple-column layouts, and callouts.

::: {.callout-warning appearance="minimal"}

If you make syntax errors in Quarto 1, the system is unable to tell you about them.

:::

Fenced divs can have classes and attributes, but the attribute syntax in Pandoc Markdown is somewhat brittle: {key="value"} produces an attribute, but {key = "value"} doesn’t. But Markdown has no syntax errors. As a result, at best users see the attributes in the text and need to fix their source. At worst, this mistake falls through the cracks all the way to the published document. Quarto 1 attempts to detect and patch over these rough edges, but this isn’t robust enough. If a user accidentally adds spaces between the key and value of an attribute, they get a mangled paragraph with ::: in it instead of a div.

If we accept this reality, then the best we can do is provide guidance, as clearly as we can, about the sources of errors.

This requirement provided the initial motivation for us to design a formal grammar of the Quarto Markdown (“qmd”) dialect using the Tree-sitter system . Because we have a formal grammar, documents might fail to parse as Markdown, and must be fixed before output is produced. But this trade-off allows us to provide contextual feedback in editors and in the command-line tooling.

Our early experience with Quarto 2 gives us reason for optimism: we find that early reporting of syntax errors is not overly cumbersome and helps catch real problems. This includes, notably, several syntax errors which had slipped through our review into the Quarto website . It also gives us more than just the ability to reject invalid input. Parse failures have additional information that we can use to produce precise, actionable error messages. We’ll have more to say about that in the future: stay tuned!

In the meantime, here’s a preview of what syntax errors can buy you. Consider this simple Quarto file:

---
format: html
---

I _accidentally forgot to end this emphasis.

A new paragraph.

In Quarto 2, you will get an error like this:

syntax-error-1.qmd: Error: [Q-2-5] Unclosed Underscore Emphasis
   ╭─[ syntax-error-1.qmd:5:45 ]
   │
 5 │ I _accidentally forgot to end this emphasis.
   │  ─┬                                         ┬
   │   ╰──────────────────────────────────────────── This is the opening '_' mark.
   │                                             │
   │                                             ╰── I reached the end of the block before finding a closing '_' for the emphasis.

Requirement 2: Accurate, fine-grained source maps

Most error states in Quarto can be associated with a particular region of a source document. Syntax errors can always be traced to the first character that fails to correspond to the grammar of the language being parsed (and often to more useful diagnostics). YAML metadata problems, such as using a number where a string is expected, are not syntax errors, but are errors nevertheless, and also can be associated with the portion of the document where the user typed a number (intentionally or not).

Quarto 1 has good support for YAML error messages (as well as auto-completion). What it lacks is support for error messages like those of the YAML system beyond YAML metadata. For example, there are only a fixed number of callout types in Quarto. If someone writes ::: callout-beware, it’s likely that this is a mistake. Even if we don’t want to issue a syntax error, it would be great to offer a warning with accurate source information in the command-line application; even better, we should have diagnostics available for modern text editors and IDEs, so the warning shows up instantly as the user is authoring the document.

In order to do this reliably, Quarto needs access to source information for the entirety of the document: metadata, headings, divs, spans, attributes, and so on. In addition, this information needs to be preserved through the entire processing pipeline, from parsing to crossref generation to the application of format-specific templates. That granularity of source information simply isn’t compatible with a system like Pandoc, whose entire point is to provide independence between input and output formats. This isn’t to say that Pandoc is wrong here; it’s a brilliant design and system that will remain useful and necessary in the Markdown ecosystem (and Quarto 2 will continue to bundle Pandoc for a number of tasks). But Quarto’s constraints are different, and require a different solution.

We note that Quarto will continue to interoperate with Pandoc. The final notable feature of Quarto 2’s source maps is that Quarto’s JSON representation of its AST is fully compatible with Pandoc, and yet includes source mapping information for every node in the AST. We designed it such that Pandoc accepts the document, by picking field names in the JSON schema that are not used by Pandoc, and maintaining the Pandoc fields precisely as they are.

For example, in Quarto 2 this means that error messages include source locations deep in document templates. Concretely, if a user doesn’t define an expected template variable in Quarto 2, we’re able to emit diagnostics. Consider what happens if a user specifies a custom template with an $author-greeting$ variable but the Quarto 2 document doesn’t define that:

<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>$title$title>
  head>
  <body>
    <header>by $author-greeting$header>
    <main>$body$main>
  body>
html>

The diagnostic you’ll get looks like this:

Warning: [Q-10-2] Undefined variable: author-greeting
   ╭─[ post.html:8:16 ]
   │
 8 │     by $author-greeting$
   │                ────────┬───────
   │                        ╰───────── Undefined variable: author-greeting
───╯

Requirement 3: Syntax stability over time

In the half decade since its inception, Quarto 1 has accrued a large number of workarounds to support its qmd dialect, many inherited from the RMarkdown and knitr ecosystems.

Consider this basic example of an executable code cell in Quarto/RMarkdown syntax:

```{r}
cat("hello world")
```

This syntax is widely supported in the Quarto/RMarkdown ecosystem, but the syntax hasn’t actually been supported in Pandoc since Pandoc 3 , released in January 2023.

Quarto also supports shortcodes inspired by Hugo: {{< video https://youtube.com/... >}}. Early versions of Quarto processed shortcodes after Pandoc’s parser was done. But this is a fundamentally impossible task, because transforming Markdown into an internal representation (the “AST”) is a many-to-one mapping. Both *hello* and _hello_ translate to (Emph [Str "hello"]); if we find an instance of that syntax inside something that looks like a shortcode invocation, there’s no way to uniquely pull these nodes back to a single string. As a result, Quarto 1 now ships with a complex “pre-parser” that understands enough qmd syntax to transform shortcodes (and other constructs like the code blocks above) to a syntax that Pandoc can safely parse.

This process is necessary in the Quarto 1 architecture, but is also unacceptably slow. In our benchmarks of rendering quarto-dev/quarto-web, we find that parsing Markdown takes about 1/3 of the total rendering time. In addition, it’s very brittle: we have to constantly maintain it to respond to small changes in Pandoc’s Markdown parsing.

Finally, Quarto is now five years old. The traditional heuristic tells us we can expect it to be a useful system for at least five more. We are a bit more ambitious. We want to deliver a system with a useful lifespan of at least 10 to 20 years, and that’s what we’ve started planning for.

In our view, this means that syntax changes become less enticing over time; as a result, in Quarto 2, we’ve replaced the front end of our pipeline with a dedicated parser and AST processor, to ensure we fully control our largest source of slow but substantial syntax drift.

A tree-sitter parser for Markdown

Our first experiment started from a pair of tree-sitter grammars from the community , and extended it to add support for Quarto’s unique syntax constructs. Unfortunately, the split between block and inline parsers meant we couldn’t get the quality of the resulting parses to match our requirements: Markdown’s inline and block processing are fundamentally intertwined, especially when handling indentation-sensitive constructs like block quotes and bulleted lists. As a result, we needed to write a single parser from scratch.

The main technique we employ is tree-sitter’s coupled parser+lexer infrastructure. Originally designed to allow incremental parsing, tree-sitter only lexes as much of the document as necessary to make a parsing decision. As a result, it can allow the lexer to use information from the parser to make lexing decisions. This makes tree-sitter parsers technically context-dependent, because tree-sitter lexers are actually in principle Turing-complete, and written in C (for the academics in the audience, we think this is an understudied object of theoretical interest!) After a few false starts, we realized we could lean heavily into this to cleanly solve some thorny issues in Markdown parsing.

The prototypical example is a string like ^hello^beautiful^world^. Should this be parsed as Superscript [Str "hello", Superscript [Str "beautiful"], Str "world"] or [Superscript [Str "hello"], Str "beautiful", Superscript [Str "world"]]? Markdown is full of such ambiguities. In Quarto 2’s new parser, we generally opt for a “shortest bracket” interpretation: if a character can close an active bracket, we greedily choose to do so. This is possible in tree-sitter because at the moment of lexing the second ^, we know the set of possible productions; if we know that CLOSE_SUPERSCRIPT is an acceptable production (and the tree-sitter API offers this information), then we produce it. Otherwise, we produce OPEN_SUPERSCRIPT.

The general consequence is that Quarto 2’s Markdown parser will open brackets when closing them would be a syntax error, and close them otherwise. This technique also works well in other cases.

The source location infrastructure allows information about the textual source of the document to travel forward in Quarto 2's processing pipeline. One of the exciting new features of Quarto 2 is its ability to “pull source information backward”. Quarto 2 includes infrastructure to take a change in an AST node, and reason through the necessary changes in the Markdown source, without having to rewrite the entire document. Our upcoming follow-up post on bidirectionality will continue this discussion.

posit::glimpse() Newsletter – May 2026

Isabella Velásquez — Wed, 06 May 2026 00:00:00 +0000

Welcome to our newsletter, posit::glimpse()!

If you’re currently reading this on our blog, consider subscribing to Product Updates - Open Source on our subscription page to receive this newsletter directly in your inbox.

posit::glimpse() is our roundup of the most important open-source news for Posit’s community! We’ve moved to monthly editions, and we still have so much to share.

Registration for posit::conf(2026) is now open!

Check out the keynote speakers , workshops , and agenda for our upcoming conference, happening in Houston and online. It’s sure to be incredible! Register here .

Key product updates and new releases

ggsql alpha release

The alpha release of ggsql brings the grammar of graphics to SQL, enabling powerful data visualization directly within SQL queries using declarative clauses. Built by the ggplot2 team with 18 years of experience, ggsql executes visualization computations as optimized SQL queries on backends, works without R or Python runtimes, and is designed for integration with Quarto, Jupyter, Positron, and AI agents.

Learn more in the ggsql alpha release blog post.
We love seeing folks in the community be early adopters! Harry Snart shares how he explored ggsql with Oracle Database , and Ansgar Wolsing tried it out for the #30DayChartChallenge .

RAG with raghilda

LLMs are great at reasoning and generating text, but their knowledge is frozen at training time. Retrieval-augmented generation (RAG) solves this by giving the model access to relevant information at query time, without needing to retrain it.

The new raghilda package simplifies building RAG systems in Python. It supports multiple storage backends (DuckDB, ChromaDB, OpenAI Vector Stores) with a consistent API, enabling teams to start locally and scale to hosted solutions without code rewrites.

Read more in the RAG with raghilda blog post.

Quarto 1.9 and Quarto 2

The Quarto team has announced plans for Quarto 2, a complete rewrite in Rust with a major focus on collaborative editing! Key features include a collaborative editor for web and command-line, a visual editor that works seamlessly alongside source editing without disrupting code, and enhanced error detection across projects. The public release is expected in at least 6 months.

Learn more in the What’s next: Quarto 2 announcement .

Meanwhile, be sure to take a look at all the exciting updates from v1.9, including:

All v1.9 updates can be found in the roundup blog post .

What’s new in Positron

Our new data science IDE, Positron , has been significantly updated with numerous improvements, including Positron Server for Academic Use via JupyterHub, AI enhancements , telemetry updates and R improvements (like Addins!).

Read more in the April 2026 Newsletter .
For continuous updates on Positron, follow the new Positron LinkedIn page and subscribe to the Positron newsletter .

Shiny for Python 1.6

Shiny for Python 1.6 is now available. The release introduces toolbar components designed for tight spaces such as card headers, input labels, and AI chat interfaces.

The second major addition is built-in OpenTelemetry support, enabling zero-configuration observability by automatically tracing session lifecycles, reactive updates, and individual reactive expressions. Set SHINY_OTEL_COLLECT=reactivity and send traces to any OTLP-compatible backend like Pydantic Logfire, Jaeger, or Grafana Cloud.

Read more in the Shiny for Python 1.6 brings toolbars and OpenTelemetry blog post.

Pointblank 0.24.0

Pointblank is a package for data validation. With the release of Pointblank 0.24.0, there is now OpenTelemetry integration for bridging data validation and production observability. After running checks on a table, you can push pass/fail metrics, per-validation-step trace spans, and structured threshold-breach logs to your OTel-compatible backend (Grafana, Datadog, New Relic, etc.). For those that run pipelines in Airflow, Prefect, or Dagster, validation spans slot into existing distributed traces automatically.

Read the full OTel integration guide for setup instructions and examples.

Tidymodels

The tidymodels team is on a roll! There are new updates available for dial, parsnip, yardstick, tune, and tidymodels, as well as two new cheatsheets.

Read the New tidymodels Releases for April 2026 blog post and tidymodels Cheatsheet blog posts.

The group has also been developing a set of skill files for machine learning with tidymodels as well as developer focused skills. You can find the current versions at skills.tidymodels.org . Give them a try; the group would love some feedback.

mori

mori is a new R package for shared memory across processes. Parallel R no longer has to mean duplicating your dataset in every worker’s RAM. mori places it in OS-level shared memory once, and every worker maps the same physical pages via R’s ALTREP framework. Works with any parallel backend that uses R serialization, including mirai, parallel, and callr.

Learn more in the mori 0.1.0 blog post.
Tyler Morgan-Wall has already implemented a fork to the targets package to incorporate mori! Read the discussion here . It’s fantastic to see the community adopt and improve the ecosystem with open source.

tabpfn 0.1.0

The new tabpfn v0.1.0 package provides an R interface to TabPFN, a pretrained deep learning model for tabular data that delivers strong predictive performance without requiring model training. The package integrates with tidymodels syntax and future updates will add parsnip model types and additional interpretability tools.

Learn more in the tabpfn 0.1.0 blog post.

nanoparquet 0.5.1

nanoparquet is a small, self-sufficient R package for reading and writing Parquet files. Version 0.5.1 introduces list columns, bit64::integer64 and blob::blob support, writing Parquet to the standard output.

Read more in the nanoparquet 0.5.1 blog post.

torch Ecosystem Updates

The team has expanded torch ecosystem support to include cudatoolkit packages, torchvision datasets, and advanced model architectures and transformations for computer vision.

Read more in the torch Ecosystem Updates blog post.

roxygen2 8.0.0

roxygen2 uses specially formatted comments in your R code to generate .Rd files. This version offers new support for S7, a raft of improvements to R6 documentation, a more natural way to configure roxygen2 in your DESCRIPTION, the changes to rendered .Rd files you’re most likely to see, and some other minor improvements, and a bunch of new vignettes.

Learn more in the roxygen2 8.0.0 blog post.

Great Docs

Last month, we introduced Great Docs for beautiful documentation for Python packages. Author Rich Iannone shares more details in the Great Docs introductory blog post .

Learning and community

Posit website relaunch

The Posit website has a new look! Check out the refreshed pages, in particular the wonderful demo gallery with examples of workflows using Posit tools.

Showcases from the community

There are so many community examples to share, here is just a small sample():

Dianyi Yang , DPhil candidate in Politics at the University of Oxford (DPIR), shares practical guide to structuring reproducible academic research projects using Git, renv, Quarto, and GitHub, from data cleaning to manuscript preparation.

Read the blog post here !

Sophia Zilber shared a public-facing dashboard for the Cure Mito Foundation Leigh syndrome patient registry using Shiny for R.

See the dashboard here .

Tom Geens and his team used Quarto to generate HTML and LaTeX reports on occupational accidents in Belgium.

See the report here .

Gen Re uses Posit Connect to automate their underwriting workflow, ingesting broker submissions every minute and routing them through AI services that extract key information and generate an early risk assessment. What used to take 30 minutes per submission now takes just 5, saving the team roughly 600 hours of cumulative processing time each day.

Check out their story .

What’s next?

You can join us every Tuesday at the Data Science Lab and every Thursday at the Data Science Hangout !

On May 12, Nicola Rennie will live code a TidyTuesday visualization from end-to-end and share the secrets of her craft!
On May 21, our DSH will be a Data Career Panel with Gabriela de Queiroz, Dan Boisvert, and Makarand Malu. Bring your career questions about the field of data, hiring, asking for promotions, and more!

I’m a real person, and I would love to hear any feedback on the newsletter! Find me on LinkedIn and Bluesky , or email me at isabella [dot] velasquez [at] posit.co.

A Plotnine Skill for AI Coding Agents

Hassan Kibirige — Tue, 05 May 2026 00:00:00 +0000

We now have a plotnine skill for AI Agents. If you write plotnine and use AI coding assistants day-to-day, this is for you. It targets the Agent Skills standard, so it works across any Agent that supports skills (Claude Code, Codex, and others) and helps an Agent produce consistent, runnable plotnine code.

Install

Install once:

uvx skillsmd add has2k1/plotnine-skill

npx skills add has2k1/plotnine-skill

With either option, you will be prompted to choose the specific agents you wish to configure.

For versions of plotnine >= 0.15.4, the skill is built-in and you can install it using library-skills .

# Install into .agents
uvx library-skills install -s plotnine

# Install for Claude (.claude)
uvx library-skills install --claude -s plotnine

Try it

Once installed, simply ask your Agent to use plotnine and create a plot. For example:

Use plotnine to create a scatter plot of bill_length_mm vs bill_depth_mm from the penguins dataset, colored by species, with a colorblind-safe palette and a smoothed trend line per species.

Compare the results for the above prompt with skill and without skill.

With Skill
No Skill

from plotnine import ggplot, aes, geom_point, geom_smooth, scale_color_brewer, labs
from plotnine.data import penguins

(
    ggplot(
        penguins.dropna(),
        aes(x="bill_length_mm", y="bill_depth_mm", color="species"),
    )
    + geom_point(size=2, alpha=0.7)
    + geom_smooth(method="lm", se=True, alpha=0.2)
    + scale_color_brewer(type="qual", palette="Set2")
    + labs(
        x="Bill Length (mm)",
        y="Bill Depth (mm)",
        title="Penguin Bill Dimensions by Species",
        color="Species",
    )
)

from plotnine import aes, geom_point, geom_smooth, ggplot, scale_color_brewer
from plotnine.data import penguins

p = (
    ggplot(penguins, aes(x="bill_length_mm", y="bill_depth_mm", color="species"))
    + geom_point(alpha=0.7)
    + geom_smooth(method="lm", se=True)
    + scale_color_brewer(type="qual", palette="Dark2")
)
                       
p.save("scatter_penguins.png", width=8, height=6, dpi=100)

Note that even without a skill, the Agent (Claude Code Opus 4.6) generates readable and idiomatic code. The differences then map directly to some of the aspects that the skill enforces. In this case they are:

Accessibility — axis labels with units (where possible) and a real title
Data Completeness — penguins.dropna() means that the smoother isn’t quietly skipping rows.

As Agents are prone to doing, without a skill the Agent was presumptuous and generated code to save the plot to a file. The skill makes that experience tighter with fewer surprises, fewer fixes after the fact.

What is next?

We want this skill to rock, hard, and this is only the beginning. The vibes may seem good but, with intent, we want to build a sense of where the skill reliably helps and where it does not, so that performance and improvements are grounded in more formal evidence rather than vibes.

Your feedback can help. If you try it, tell us how it performs; the prompts it handled poorly, those were it was surprisingly exceptional, and even those where it was good but it can be exceptional. The show is still going on, and we will be glad to see what you show us at the repo .

roxygen2 8.0.0

Hadley Wickham — Fri, 01 May 2026 00:00:00 +0000

I’m pleased to announce that roxygen2 8.0.0 is now on CRAN. roxygen2 turns specially formatted comments in your R code into the .Rd files that power R’s help system. You can install it with:

install.packages("roxygen2")

This release has been a long time in the making and includes well over a hundred bug fixes and improvements. This post covers the highlights: new support for S7, a raft of improvements to R6 documentation, a more natural way to configure roxygen2 in your DESCRIPTION, the changes to rendered .Rd files you’re most likely to see, and some other minor improvements, and a bunch of new vignettes. You can read the full list of changes in the release notes .

library(roxygen2)

Documenting S7

The headline feature is initial support for S7 . S7 is a new object-oriented programming system built to be a successor to S3 and S4. It’s been designed and implemented collaboratively by the R Consortium Object-Oriented Programming Working Group, which includes representatives from R-Core, Bioconductor, tidyverse/Posit, and the wider R community. S7 is still a work in progress, but it is useful today, and adding roxygen2 support makes it easier to use in packages.

roxygen2 now supports documenting S7 generics, classes, and methods:

Generics are documented just like regular functions.
Classes are documented like regular functions (because S7 constructors are functions), but you can also use @prop to document properties that aren’t constructor parameters. If you document several related classes on the same page, @prop ClassName@prop_name description lets you group properties by class.
Methods registered with method(generic, class) <- fn are picked up automatically, and roxygen2 generates the right usage and aliases for you.

Read vignette("rd-S7") for a full rundown, and please let us know if you discover any issues or have suggestions for improvement!

A cleaner home for configuration

roxygen2 has historically been configured through a Roxygen: field in DESCRIPTION, with a separate RoxygenNote field recording the version. This release introduces a more natural home using the Config/roxygen2/ namespace that packages like testthat and lifecycle have adopted:

# before
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.3.2

# after
Config/roxygen2/markdown: TRUE
Config/roxygen2/version: 8.0.0

The old fields still work, and roxygen2 will quietly migrate RoxygenNote the next time you run devtools::document(). Over time, devtools and usethis will switch to the new form by default.

Rendering tweaks

When you re-document your package for the first time with roxygen2 8.0.0, you might notice some small differences in the rendered output. The most commonly encountered changes are:

All generated cross-reference links now go through the same code path and share a single style, e.g. \code{\link[=compose]{compose()}}.
Links to external packages now use the topic alias rather than the .Rd filename. This brings roxygen2 into alignment with current CRAN best practices.
Tags that typically expect a single line of input now warn if you spread them across multiple lines. This catches a common class of silent mistakes, e.g. a @name or @rdname with no content that quietly included the next line of the block. This will create warnings for some existing legitimate uses, but I think the payoff (eliminating problems that are otherwise very hard to spot) is worth it.
People with both "aut" and "cre" roles now appear in both the Authors and Maintainer sections of package documentation.

Other important changes

roxygen2 now requires R 4.1 and no longer depends on purrr, stringr, or stringi. Those last two are the big win because it means that stringi is now gone from the complete devtools dependency graph, making it easier to install for folks on constrained Linux images.

There’s a new helper, needs_roxygenize(), that tells you whether your .Rd files are out of date by comparing modification times against their sources. It’s much cheaper than running roxygenize() just to find out nothing has changed, which makes it a good fit for pre-commit hooks or CI checks.

Finally, parameter inheritance has gained a small but very useful feature: @inheritParams now supports filtering, just like @inheritDotParams. If you only want to inherit a subset of arguments from another function, you can now list them explicitly:

#' @inheritParams other_fn x y

Or exclude the ones you don’t want:

#' @inheritParams other_fn -z

Speaking of @inheritDotParams, it now also works more like @inheritParams: it inherits documented parameters rather than formal arguments. This may introduce new false positives (replacing the old approach’s false negatives), which you can prevent by explicitly listing the argument names to inherit.

New vignettes

roxygen2’s own documentation has had a significant tidy-up. The old vignette("rd-other") has been broken into focused vignettes — vignette("rd-datasets"), vignette("rd-packages"), vignette("rd-S3"), vignette("rd-S4"), and vignette("rd-R6") — with the new vignette("rd-S7") joining them. The main vignette("rd") has been renamed to vignette("rd-functions"), and the “getting started” content has moved to vignette("roxygen2").

vignette("rd-S3") has also been rewritten with clearer guidance for documenting S3 generics, classes, and methods, including how to use the new doclisting package to automatically list methods for a generic — a long-standing pain point for generics with methods in multiple packages. You can also use doclisting with S4 and S7 generics.

Acknowledgements

A big thank you to everyone who has contributed issues, pull requests, and discussion since the last release! @achubaty , @adithya604 , @akersting , @alandipert , @alannearme , @alecw , @alexgenin , @AlexisDerumigny , @AliSajid , @alisonmosky , @aljabadi , @allenzhuaz , @andrew-schulman , @andrewmarx , @aphalo , @apreshill , @arilamstein , @arnaudgallou , @ashbythorpe , @ateucher , @b-niu , @bahadzie , @balthasars , @BartJanvanRossum , @bastistician , @batpigandme , @beginb , @BenEngbers , @BenWiseman , @bgctw , @BGWKlein , @bhagwataditya , @billdenney , @Bisaloo , @bluewomble , @bobjansen , @boshek , @brendanf , @brodieG , @BroVic , @brpetrucci , @brry , @bryanhanson , @bwiernik , @cbielow , @cboettig , @cderv , @CGMossa , @chlebowa , @chrarnold , @ChristopherEeles , @chrk623 , @chuxinyuan , @cjyetman , @coatless , @ColinFay , @courtiol , @cthombor , @d-morrison , @d-sci , @daattali , @DanChaltiel , @DanielHermosilla , @danielvartan , @DarioS , @davidrubinger , @DavisVaughan , @daynefiler , @dfrankow , @dgkf , @dieghernan , @dipterix , @dmurdoch , @dpprdan , @drag05 , @dragosmg , @dsweber2 , @dvg-p4 , @dwachsmuth , @eddelbuettel , @eitsupi , @ejosymart , @elcortegano , @ElsLommelen , @espinielli , @FelixErnst , @florisvdh , @flrd , @gaborcsardi , @GABurns , @galachad , @gavinsimpson , @genomaths , @ggrothendieck , @ghost , @goldingn , @gowerc , @gregorgorjanc , @gustavdelius , @gwd666 , @hadley , @harrelfe , @hdarjus , @HenningLorenzen-ext-bayer , @HenrikBengtsson , @hongooi73 , @hughjonesd , @iferres , @IndrajeetPatil , @J-Moravec , @jakubnowicki , @jameslamb , @jan-abel-inwt , @JanaJarecki , @JanMarvin , @JantekM , @jcubic , @JDenn0514 , @jdprimus , @jeffcraggy , @jennybc , @jensmassberg , @jeroen , @jeroenjanssens , @JesseAlderliesten , @jgellar , @jgutman , @JiaxiangBU , @Jiefei-Wang , @jimhester , @jmbarbone , @jmpanfil , @johanneswerner , @johnbaums , @JohnCoene , @JonathanUrbach , @jonkeane , @jonocarroll , @jonthegeek , @JosiahParry , @jranke , @JulieBlasquiz , @jwijffels , @kamapu , @karchjd , @karoliskoncevicius , @kathi-munk , @kellijohnson-NOAA , @kevinushey , @kingaa , @klmr , @Klorator , @kongdd , @kortschak , @kostrzewa , @kpagacz , @krivit , @krlmlr , @kurt-o-sys , @kylebutts , @lgatto , @lindeloev , @LiNk-NY , @lionel- , @llrs , @llrs-roche , @lorenzwalthert , @LouisLeNezet , @LukasWallrich , @lukasz-bednarz-reddeersystems , @m-muecke , @maelle , @malcolmbarrett , @mamueller , @math-mcshane , @maxheld83 , @MaximilianPi , @mbojan , @mccarthy-m-g , @mccroweyclinton-EPA , @mcol , @mcsage , @MichaelChirico , @michaelquinn32 , @mikemahoney218 , @mikkmart , @mikmart , @MilesMcBain , @mine-cetinkaya-rundel , @MislavSag , @mjsteinbaugh , @mkoohafkan , @MLopez-Ibanez , @mnazarov , @mnneely , @monkeywithacupcake , @moodymudskipper , @mpadge , @mrchypark , @ms609 , @msaltieri , @msberends , @mschilli87 , @multimeric , @muschellij2 , @musvaage , @naikymen , @nathaneastwood , @nbenn , @nealrichardson , @Nelson-Gon , @neshvig10 , @netique , @ngreifer , @nick-robo , @NikNakk , @nlneas1 , @nouvrita , @nr0cinu , @nteetor , @ntguardian , @oharar , @okhoma , @olivroy , @orgadish , @p-carter , @p00ya , @pakjiddat , @pat-s , @PauloJhonny , @PavelBal , @pbreheny , @phargarten2 , @pnacht , @pvanlaake , @ralmond , @ramiromagno , @RaphaelS1 , @retostauffer , @RiboRings , @richelbilderbeek , @rjake , @RMHogervorst , @robchallen , @Robinlovelace , @romainfrancois , @rorynolan , @rossellhayes , @rsbivand , @russHyde , @rvernica , @s-fleck , @saipenikalapati , @Salatbesteck , @salim-b , @sbgraves237 , @sboehringer , @schloerke , @schradj , @sckott , @sebffischer , @setgree , @ShixiangWang , @simonpcouch , @simonsays1980 , @simpar1471 , @slager , @smilberg , @stefanfritsch , @stefanoborini , @stellathecat , @stemangiola , @stla , @strazto , @strboul , @sven-stodtmann , @swnydick , @t-kalinowski , @TanguyBarthelemy , @tappek , @tau31 , @tdhock , @ThierryO , @tjebo , @TomKellyGenetics , @tommarshall2 , @trusch139 , @turgeonmaxime , @tzakharko , @uhkeller , @unDocUMeantIt , @vertesy , @VPetukhov , @wch , @wibeasley , @wilcoxa , @wurli , @wviechtb , @yogat3ch , @Yunuuuu , @yutannihilation , @zachary-foster , @zeehio , @zettlchen , and @zkamvar .

nanoparquet 0.5.1

Gábor Csárdi — Thu, 30 Apr 2026 00:00:00 +0000

We’re very chuffed to announce the release of nanoparquet 0.5.1 (and 0.5.0). nanoparquet is a small, self-sufficient R package for reading and writing Parquet files.

You can install it from CRAN with:

install.packages("nanoparquet")

This blog post will go over some of the improvements in nanoparquet 0.5.0 and 0.5.1.

You can see a full list of changes in the release notes here and here .

library(nanoparquet)

List columns

Parquet has a LIST type for columns whose values are variable-length sequences of scalars. nanoparquet 0.5.0 adds support for reading and writing such columns.

Note: for now nanoparquet supports one level of nesting: each element of a list column must be an atomic vector of a single type, not a list of lists. All elements in a column must have the same scalar type.

To write a list column, put a regular R list into your data frame. Each element must be an atomic vector (integer, double, or character), NULL for a missing list, or an empty vector for an empty list. NA values inside an element vector encode missing elements.

df <- data.frame(id = 1:4)
df$scores <- list(c(80L, 95L, 70L), c(100L), NULL, integer(0))
write_parquet(df, tmp <- tempfile(fileext = ".parquet"))

read_parquet_schema(tmp)

# A data frame: 5 × 14
  file_name  r_col name  r_type type  type_length repetition_type converted_type
                                        
1 /var/fold…    NA sche…               NA                       
2 /var/fold…     1 id    integ… INT32          NA REQUIRED        INT_32        
3 /var/fold…     2 scor… list(…            NA OPTIONAL        LIST          
4 /var/fold…     2 list                NA REPEATED                  
5 /var/fold…     2 elem…    INT32          NA OPTIONAL        INT_32        
# ℹ 6 more variables: logical_type >, num_children , scale ,
#   precision , field_id , children

read_parquet() reads LIST columns back as R list columns:

as.data.frame(read_parquet(tmp))

  id     scores
1  1 80, 95, 70
2  2        100
3  3       NULL
4  4

infer_parquet_schema() shows how nanoparquet maps each column to a Parquet type. For list columns, the r_type shows e.g. list(integer):

infer_parquet_schema(df)[2:7]

# A data frame: 4 × 6
  r_col name    r_type        type  type_length repetition_type
                                 
1     1 id      integer       INT32          NA REQUIRED       
2     2 scores  list(integer)            NA OPTIONAL       
3     2 list                         NA REPEATED       
4     2 element           INT32          NA OPTIONAL

A LIST column occupies three rows in the schema: the outer list node, a repeated group node, and the leaf element node.

When you need to specify the element type explicitly, you can use parquet_schema():

schema <- parquet_schema(
  id     = "INT32",
  scores = list("LIST", element = "INT32")
)
write_parquet(df, tmp2 <- tempfile(fileext = ".parquet"), schema = schema)

read_parquet_schema(tmp2)

# A data frame: 5 × 14
  file_name  r_col name  r_type type  type_length repetition_type converted_type
                                        
1 /var/fold…    NA sche…               NA                       
2 /var/fold…     1 id    integ… INT32          NA REQUIRED                  
3 /var/fold…     2 scor… list(…            NA OPTIONAL        LIST          
4 /var/fold…     2 list                NA REPEATED                  
5 /var/fold…     2 elem…    INT32          NA OPTIONAL        INT_32        
# ℹ 6 more variables: logical_type >, num_children , scale ,
#   precision , field_id , children

New types

`bit64::integer64`

Parquet’s INT64 type holds 64-bit integers. R’s native integer is only 32 bits, so nanoparquet has mapped INT64 to double by default. nanoparquet 0.5.1 adds support for bit64::integer64, which gives you true 64-bit integer arithmetic in R.

write_parquet() now writes bit64::integer64 columns as INT64:

library(bit64)
df2 <- data.frame(id = as.integer64(c(1e18, 2e18, 3e18)))
write_parquet(df2, tmp3 <- tempfile(fileext = ".parquet"))
read_parquet_schema(tmp3)

# A data frame: 2 × 14
  file_name  r_col name  r_type type  type_length repetition_type converted_type
                                        
1 /var/fold…    NA sche…               NA                       
2 /var/fold…     1 id    double INT64          NA REQUIRED        INT_64        
# ℹ 6 more variables: logical_type >, num_children , scale ,
#   precision , field_id , children

To read INT64 columns back as bit64::integer64 instead of the default double, use the read_int64_type option. The bit64 package must be installed; if it isn’t, nanoparquet throws a clear error.

read_parquet(tmp3, options = parquet_options(read_int64_type = "integer64"))

# A data frame: 3 × 1
       id
  
1    0e18
2    2e18
3    3e18

`blob::blob`

read_parquet() previously returned raw BYTE_ARRAY and FIXED_LEN_BYTE_ARRAY columns (i.e. those without a string, UUID, or decimal annotation) as plain lists of raw vectors. They are now returned as blob::blob objects, which print more neatly and come with the full set of blob helpers. write_parquet() now also accepts blob::blob columns, so round-tripping binary data is straightforward:

library(blob)
df3 <- data.frame(id = 1:3)
df3$payload <- blob::blob(
  charToRaw("hello"),
  charToRaw("world"),
  charToRaw("!")
)
write_parquet(df3, tmp4 <- tempfile(fileext = ".parquet"))
as.data.frame(read_parquet(tmp4))

  id   payload
1  1 blob[5 B]
2  2 blob[5 B]
3  3 blob[1 B]

nanoparquet as a filter

In Unix, a filter is a program that reads from standard input and writes to standard output, making it a composable building block in shell pipelines. write_parquet() now supports writing to standard output via file = ":stdout:":

write_parquet(mtcars, ":stdout:")

The most common use case is from the command line:

Rscript --quiet -e 'nanoparquet::write_parquet(mtcars, ":stdout:")' > mtcars.parquet

You can build this into a data pipeline. For example, to convert a CSV to Parquet, and then process Parquet with another tool in one shot, without an intermediate .parquet file on the disk, you can do:

cat data.csv |
  Rscript --quiet -e '
    df <- read.csv(file("stdin"))
    nanoparquet::write_parquet(df, ":stdout:")
  ' | another-parquet-tool

Since nanoparquet 0.4.0, read_parquet() can also read from an R connection, so you can pipe Parquet data in as well:

url <- "https://raw.githubusercontent.com/r-lib/nanoparquet/main/inst/extdata/userdata1.parquet"
con <- pipe(paste("curl --silent", url))
df <- read_parquet(con)

Acknowledgements

We thank all contributors to nanoparquet so far, for opening issues, submitting pull requests, and providing feedback: @Aariq , @alvarocombo , @apalacio9502 , @atsyplenkov , @cboettig , @ChandlerLutz , @cmrnp , @D3SL , @damonbayer , @DavideMessinaARS , @eitsupi , @gksmyth , @hadley , @jack-davison , @jeroenjanssens , @lbm364dl , @lschneiderbauer , @mrcaseb , @pmarks , @PMassicotte , @r2evans , @RealTYPICAL , @tanho63 , @thisisnic , @torfason , @TurnaevEvgeny , @Upipa , @vankesteren , @vincentarelbundock , @wlandau , @YipengUva , and @yutannihilation .

torch Ecosystem Updates

Daniel Falbel — Thu, 30 Apr 2026 00:00:00 +0000

We’ve just published a new round of CRAN releases across the torch ecosystem. Here’s a tour of what’s new in each package.

torch v0.17.0

The most exciting experimental new feature is support for the cudatoolkit packages. With this, you no longer need a global CUDA toolkit installation in order to use torch on the GPU.

You can now do:

install.packages(
  "cuda12.8", 
  repos = c("https://mlverse.r-universe.dev", "https://cloud.r-project.org")
)
install.packages("torch")

The {cuda12.8} package bundles all the CUDA runtime libraries and torch can find it and use it by default. See more details in the installation docs .

We also highlight the update to LibTorch v2.8.0 led by Troy Hernandez ( #1419 ).

Additionally, this release includes many small bug fixes and small additions to the API. See the full release notes in the changelog .

torchvision v0.9.0

torchvision provides datasets, model architectures, and image transformations for computer vision. This is a big release with new models, datasets, and many improvements — largely driven by community contributors.

New models:

model_maskrcnn_resnet50_fpn() and model_maskrcnn_resnet50_fpn_v2() for instance segmentation.
model_convnext_*_detection() for object detection (tiny/small/base).
model_convnext_*_fcn() and model_convnext_*_upernet() for semantic segmentation (tiny/small/base).

New datasets and features:

vggface2_dataset() for loading the VGGFace2 dataset.
New coco_segmentation_dataset(), split from coco_detection_dataset(), reducing memory usage by ~50%.
Collection dataset catalog with search_collection(), get_collection_catalog(), and list_collection_datasets() for discovering and exploring datasets.
New visualization utilities draw_segmentation_masks() and vision_make_grid().

See the full release notes in the changelog .

A huge thank you to the community contributors who made this release possible: @cregouby , @ANAMASGARD , @Chandraveersingh1717 , @DerrickUnleashed , and @srishtiii28 .

Other releases

Most of the other packages don’t have significant changes, and the releases add minimal improvements to docs, CI infrastructure and CRAN related updates.

luz v0.5.2 — Higher-level API for torch with a Keras-like interface for training neural networks.
hfhub v0.1.2 — Download and cache files from Hugging Face Hub repositories, making it easy to use pretrained models and datasets from R.
tok v0.2.2 — Fast tokenizers for R, powered by the Hugging Face Tokenizers library written in Rust. Supports BPE, WordPiece, and other tokenization algorithms.
torchdatasets v0.3.2 — Extra ready-to-use datasets for torch, complementing the built-in datasets in torchvision.
safetensors v0.2.1 — Read and write the Safetensors file format, a safe and fast format for storing and loading tensors.
tfevents v0.0.5 — Write event files compatible with TensorBoard from R for experiment tracking and visualization.
wav v0.2.0 — Read and write WAV files in R.

New maintainer

We’re excited to welcome Tomasz Kalinowski as the new maintainer of torch and the broader mlverse ecosystem.

tidymodels Cheatsheets

Edgar Ruiz — Wed, 29 Apr 2026 00:00:00 +0000

After almost 8 years, tidymodels finally has its first cheatsheets, and not just one, but two! The first one , covering data preprocessing with recipes, was released a couple of months ago. Today, we are delighted to announce a second cheatsheet , this time focusing on modeling with parsnip.

Both cheatsheets have a dedicated HTML version on the Posit Open Source site, so you can browse and search them without opening a PDF. In this post we’ll walk through what each one covers, starting with the newest.

Create Models with parsnip

The "Create Models with parsnip" cheatsheet — click to enlarge

The cheatsheet is organized into three main parts: an introduction to parsnip’s basics, a catalog of all models available through the package, and a hands-on operations reference for fitting and inspecting models. The basics section introduces how parsnip provides a single, unified interface for defining and fitting models, regardless of the underlying package powering them.

Model catalog

The largest section of the cheatsheet catalogs all models available through parsnip, grouped by use case:

Classification only: models for binary and multiclass prediction. It also includes probability-based classification using Bayes’ theorem and models for ordinal responses.
Regression only: models for predicting continuous numeric outcomes, from standard linear regression to generalized linear models for count data.
General use: a versatile mix of model types that work for both classification and regression, including decision trees, nearest neighbors, neural networks, and spline-based approaches.
Discriminant analysis: models that estimate the distribution of predictors separately for each class and use Bayes’ theorem to assign probabilities, available in linear, quadratic, flexible, and regularized variants.
Ensemble methods: models that combine many individual learners into a stronger prediction, including random forests, gradient boosting, bagged trees, and Bayesian additive regression trees.
Support Vector Machines: models that find an optimal boundary between classes, or fit a robust regression, using linear, polynomial, or radial kernel functions.
Feature rules: models that extract simple, human-readable rules from tree ensembles and use them as the basis for prediction.
Survival models: models for time-to-event data, covering both proportional hazards and fully parametric approaches.

One design choice in particular makes this section much easier to navigate: pills. Each model’s compatible engines and supported modes are shown as small, visually distinct tags, so you can see at a glance which mode a given engine supports, without having to read through the description text. Each mode is encoded in the pill with a number: Classification (1), Regression (2), Censored Regression (3), and Quantile Regression (4). A legend mapping each number to its mode is available at the top of page one.

Engine pills show the name and supported modes of each engine at a glance

And true to the R cheatsheet tradition, individual models or groups of related models are paired with small illustrations, thoughtfully designed for visual impact to aid recall. Each one attempts to accurately represent the function or functions it accompanies, making them a genuine navigation aid rather than decoration, especially when you have a vague memory of “that tree-based ensemble that used Bayesian analysis” and need to scan quickly.

Operations

The last section covers the practical workflow of fitting and using a model. Each function is paired with a quick runnable example, and the examples build on each other starting from the two lines of code right below the section title, making it easy to follow the full workflow from model specification to results.

Explore the parsnip cheatsheet

Preprocessing Data with recipes

The "Preprocessing Data with recipes" cheatsheet — click to enlarge

After a quick Basics section covering the core workflow, the vast majority of the cheatsheet is dedicated to step_*() functions, the building blocks of any recipe, before finishing with role and type management.

Step catalog

The steps are organized into groups based on what they do, each listed with its arguments and a short description:

Filters: steps for removing variables that are sparse, zero-variance, linearly dependent, highly correlated, or missing too many values
In-place Transformations: basis functions (splines, polynomials), discretization, and normalization steps
Imputation: steps for filling in missing values, ranging from simple statistical substitution to model-based approaches
Encodings: type converters (e.g. factor to string, numeric to factor), value converters, and other factor-handling steps
Dummy Variables: one-hot and binary encoding, text pattern matching, and conversion helpers
Multivariate Transformations: signal extraction (PCA, ICA, PLS, and friends) and centroid-based distance measures
Date & Time: steps for converting date and datetime columns into usable numeric or factor features
Row operations: sampling, shuffling, slicing, and removing rows with missing values
Other: interaction terms, renaming, rolling window statistics, geographic distances, and ratios

As with the parsnip cheatsheet, each group of steps is paired with small, thoughtfully designed illustrations to help you visually locate a step family when scanning.

Role & type

The last section focuses on the selection and management of variable roles and types within the recipe. The selection side covers ways to target variables by their role (outcome, predictor, or any custom role) as well as by their type (numeric, factor, logical, and so on), including a handy set of convenience selectors for the most common combinations. The management side shows how to add, update, and remove roles, showing you how to gain fine-grained control over how each variable participates in the recipe.

Easily find the right selector function with this at-a-glance guide

Explore the recipes cheatsheet

Need them on the go? Print them!

A lot of care went into ensuring both cheatsheets hold up when printed, particularly in black and white. We know that many folks print cheatsheets to keep at their desk for quick reference, and we wanted to make sure they remain fully usable in that medium. That meant making sure font sizes and weights stay legible on paper, that the illustrations remain perceptible without color, and that contrast levels are strong enough that no text ends up too pale to read or too heavy to parse. Accessibility in print mattered to us just as much as clarity on screen.

New tidymodels cheatsheets are fully readable when printed

Customize Positron with these community-curated resources

Isabella Velásquez — Tue, 28 Apr 2026 00:00:00 +0000

We’ve been having a blast at the Data Science Lab. It’s a new, experimental, and super informal weekly community call (which you can register for here: pos.it/dslab !). In the spirit of being bold and trying things out, Isabel Zimmerman and Davis Vaughan from the Positron dev team joined us for the very first Lab “pilot” back in November, sharing some of their favorite Positron settings and extensions.

The Lab with Isabel and Davis was so much fun that we officially started the series in December, and then, in January, we jumped at the chance to have Andrew Heiss from Georgia State University join us to share his favorite tips and tricks for his Positron data science workflow (and beyond). He even wrote a helpful roundup on his blog! Andrew’s workflow might give you data superpowers.

Following Andrew’s Data Science Lab, Emil Hvitfeldt felt inspired to write up his list of extensions and settings on his blog ! We love community-event-driven development 😉

Speaking of community: Libby posted on Bluesky about wanting Positron’s main title bar to function like RStudio’s, showing the project and git branch name, and Emil shared a setting he’d learned from Davis that does exactly that! Want to customize your own Positron title bar? Check out the post about the setting here and try it for yourself. It’s in the comments!

You know how in #RStudio, it puts your branch name at the top? Like "project_name - branch - RStudio" (shown here)
I want that in #Positron 🤔 and I haven't found a popular extension for it. (I think there is one that works in VSCode, but not in Positron!)

Any ideas, #rstats #databs crew?

[image or embed]
— Libby Heeren (@libbyheeren.bsky.social) November 6, 2025 at 10:26 PM

We highly recommend watching the Lab videos if you love hearing IDE thoughts and other off-the-cuff ideas, but if you are just itching to find the settings and extensions mentioned, there’s a list of resources below. Do you have your own Positron setting and extension list? Let us know all about it 💙

Keyboard shortcuts

Settings

Native Tabs

Extensions

Extension	Description
Pastum	Like datapasta, but for Positron
positron-plus-1-e	Garrick Aden-Buie’s data science extension bundle package
indent-rainbow	Change the color of the indents in front of your text
Rainbow CSV extension	Highlight columns in distinct colors
Enter Folder	Make your folder look like the root folder, making your Positron file explorer work more similarly to RStudio’s File Pane
Catppuccin themes	A cozy set of pastel themes used by many in the DS Lab community
Better Comments	Categorize your to-dos
File Properties Viewer	Display a menu for file system properties
Peacock	Change your workspace colors, making the differences between projects more striking and noticeable
Project Manager	Switch between projects
Spell Right	Spell check your documents

New tidymodels Releases for April 2026

Max Kuhn — Mon, 27 Apr 2026 00:00:00 +0000

We’ve released a sequence of tidymodels packages over the last few weeks: dials (1.4.3), parsnip (1.5.0), tune (2.1.0), yardstick (1.4.0), and tidymodels (1.5.0). You can install them via:

# tidymodels installs all of the new versions
require(pak)
pak::pak("tidymodels")

Here are links to the NEWS files for each package:

Let’s first talk about the two biggest updates enabled by this group of releases, then we’ll cover some of the other changes for each package.

Ordered Outcomes

parsnip has a new model type, ordinal_reg(), analogous to multinom_reg(), for fitting various generalized linear models with ordered class levels.

The ordered package by Cory Brunson is now on CRAN. This contains the specific engine code for these models, including:

ordinal_reg(): three engines: "polr", "ordinalNet", and "vglm".
gen_additive_mod(): "vgam"
decision_tree(): "rpartScore"
rand_forest(): "ordinalForest"

These models can be fitted, tuned, and evaluated with tidymodels. For the evaluation, we’ve added a specific performance metric for ordered categories: the ranked probability score (RPS). The function ranked_prob_score() is in the new yardstick release and requires an ordered factor for the outcome.

Quantile Regression

We previously reported that parsnip supports quantile regression models. With the latest set of releases, new boosting and neural network engines are available, and these models can now be tuned and evaluated using a relevant metric. yardstick now includes the weighted interval score ( Bracher et al (2021) ) to evaluate the quality of the quantile predictions.

Here’s a simple one-dimensional example using the Ames data; we’ll predict the sale price as a function of latitude. To start, let’s make a training/test split, generate some resamples, and plot the training data.

library(tidymodels)
# We'll also need the qrnn package for the neural network engine

set.seed(1215)
ames_split <-
  ames |>
  select(Latitude, Sale_Price) |>
  initial_split(strata = Sale_Price)
ames_train <- training(ames_split)
ames_test <- testing(ames_split)
ames_rs <- vfold_cv(ames_train, strata = Sale_Price)

ames_train |>
  ggplot(aes(Latitude, Sale_Price)) +
  geom_point(alpha = 1 / 5) +
  geom_smooth(se = FALSE) +
  labs(x = "Latitude", y = "Sale Price (USD)")

Note that we almost always model these data with a log transformation on the outcome due to its inherent skewness. That helps us avoid making negative predictions, be more robust to overly influential points (i.e., locations with very large sale prices), and stabilize the variance. However, we don’t necessarily have to do that with quantile regression. The objective functions used to estimate parameters do not impose requirements on the normality of the data or heterogeneity of residuals. For this analysis, let’s stick with the original units of the outcome (USD).

There are a few engines for quantile regression, and we’ll use a neural network model. To get started, the quantiles to be predicted need to be specified. We make a model specification with a few additions:

# Pre-defined quantiles of interest
qnt_lvls <- c(0.05, 0.25, 0.5, 0.75, 0.95)

nnet_spec <-
  mlp(hidden_units = tune(), penalty = tune(), epochs = 10) |>
  # Set the quantile levels with the mode:
  set_mode("quantile regression", quantile_levels = qnt_lvls) |>
  # A new engine for quantile regression with neural networks via the
  # qrnn package. We'll add an engine argument to specify the
  # optimization method for training the model:
  set_engine("qrnn", method = "adam")

# Scale the single predictor to help the model initialize its
# parameters.
nnet_rec <- recipe(Sale_Price ~ ., data = ames_train) |>
  step_normalize(all_predictors())

nnet_wflow <- workflow(nnet_rec, nnet_spec)

From there, we can use any of our tuning functions to optimize the number of hidden units and the amount of weight decay. By default, the weighted interval score is used for this particular mode.

We’ll consider 25 tuning parameter candidates to optimize model performance.

set.seed(971)
nnet_res <-
  nnet_wflow |>
  tune_grid(
    resamples = ames_rs,
    grid = 25,
    control = control_grid(save_workflow = TRUE)
  )

We can get the performance metric and visualize which tuning parameter combinations have the smallest weighted interval score:

nnet_mtr <- collect_metrics(nnet_res)

nnet_mtr |>
  ggplot(aes(penalty, hidden_units, size = mean)) +
  geom_point() +
  scale_x_log10() +
  coord_fixed(ratio = 1) +
  labs(x = "Penalty", y = "# Hidden Units", size = "WIS")

select_best(nnet_res, metric = "weighted_interval_score")

# A tibble: 1 × 3
  hidden_units     penalty .config         
                            
1           10 0.000000215 pre0_mod24_post0

The model appears to prefer a smaller penalty and more hidden units.

It’s hard to conceptualize how well the model functions with just these numbers. To show that the metric does select good models, let’s fit the best, median, and worst models and see how they look on the test set.

set.seed(8281)
best_model <- fit_best(nnet_res)

set.seed(8281)
worst_model <-
  nnet_mtr |>
  slice_max(mean, n = 1) |>
  select(hidden_units, penalty) |>
  finalize_workflow(nnet_wflow, parameters = _) |>
  fit(ames_train)

set.seed(8281)
mid_model <-
  nnet_mtr |>
  # Since we have an odd number of grid points:
  filter(mean == median(mean)) |>
  select(hidden_units, penalty) |>
  finalize_workflow(nnet_wflow, parameters = _) |>
  fit(ames_train)

Now let’s plot the results. We’ll color the predicted quantiles: black indicates the predicted median sale price, orange lines indicate the inner quartiles, and smoky periwinkle lines indicate the 0.05 and 0.95 quantiles (which could serve as 90% prediction intervals).

bind_rows(
  best_model |> augment(ames_test) |> mutate(Model = "Best Results"),
  mid_model |> augment(ames_test) |> mutate(Model = "Meh Results"),
  worst_model |> augment(ames_test) |> mutate(Model = "Worst Results")
) |>
  mutate(
    .pred_quantile = map(.pred_quantile, ~ as_tibble(.x))
  ) |>
  unnest(.pred_quantile) |>
  arrange(Latitude) |>
  ggplot(aes(Latitude)) +
  geom_point(aes(y = Sale_Price), alpha = 1 / 30, cex = 3 / 4) +
  geom_path(
    aes(
      y = .pred_quantile,
      group = .quantile_levels,
      col = factor(.quantile_levels)
    ),
    show.legend = FALSE,
    linewidth = 1
  ) +
  scale_color_manual(
    values = c("#8785B2FF", "#D95F30FF", "black", "#D95F30FF", "#8785B2FF")
  ) +
  facet_wrap(~Model)

These plots show that configurations with very large score values have poor fits (linear in this case). The “meh” model is nonlinear but not responsive enough to the datas’ ups and downs. The best model, with more hidden units and a low penalty, appears to be flexible enough to model the data well.

We’ll have more metrics in yardstick that can use quantile predictions in the future. For example, we can extend the ones that we have, such as rmse() or rsq(), to use a predicted value from the center of the predictive distribution, such as the 0.5 quantile.

Now we’ll describe various other improvements in the recently released versions.

dials

The latest dials release contains several new parameters for new-ish models in parsnip: For the ordinal_reg() models, dials now contains ordinal_link() and odds_link(). For the tab_pfn(), dials contains num_estimators(), softmax_temperature(), balance_probabilities(), average_before_softmax(), and training_set_limit().

The other user-facing changes were related to input checking and related error messages. The most prominent example is that parameters() and the grid_*() functions now give more information in the error message when non-parameter objects are passed in: which inputs aren’t a parameter object and what they are instead.

@corybrunson , @daltonkw , @hfrick , @jeroenjanssens , @topepo , and @vmikk contributed to the package since the last release.

yardstick

Beyond the two new metrics ranked_prob_score() and weighted_interval_score() described above, this release adds a further 8 metrics.

Three new regression metrics:

mse() — mean squared error (the squared counterpart to the existing rmse()).
rmse_relative() — root mean squared error normalized by the observed value range.
gini_coef() — normalized Gini coefficient.

Three new classification metrics:

fall_out() — false positive rate (1 − specificity).
miss_rate() — false negative rate (1 − sensitivity).
markedness() — predictive power of a classifier, computed as PPV + NPV − 1.

Two new probability-based classification metrics:

roc_dist() — Euclidean distance from the perfect-classifier corner of ROC space.
sedi() — Symmetric Extremal Dependence Index.

In addition to these new metrics, we have also updated the documention of all metrics. Now each metric shows the formula used to calculate it, as well as the valid values it can produce.

We also have pages that list all metrics of the same type. These can be found with ?class-metrics , ?numeric-metrics or linked within each metric documentation.

We are thankful to the developers who contributed to this version: @abichat , @astamm , @corybrunson , @DarioS , @EmilHvitfeldt , @FvD , @hfrick , @JavOrraca , @jeroenjanssens , @jkylearmstrong-temple , @mle2718 , @nathant181 , @SimonDedman , @topepo , and @tripartio

parsnip

Version 1.5.0 of parsnip had a variety of changes. Besides the additions for the two new model types shown above:

We enabled case weight usage for the "nnet" engines of mlp() and bag_mlp() as well as for the "dbarts" engine of bart().

Many of the other changes are most likely to be noticed by developers:

The interface for declaring tunable parameters has been simplified and is the same for main arguments as well as engine parameters. Also, these values can now be set inside extension packages.
We now export the generics for predict_quantile(), predict_class(), predict_classprob(), and predict_hazard().
format_predictions() is a new unified function for formatting prediction outputs, consolidating the logic from the individual format_*() functions. The individual functions format_num(), format_class(), format_classprobs(), format_time(), format_survival(), format_linear_pred(), and format_hazard() are now deprecated.

Thanks to those who contributed to parsnip since the last release: @CeresBarros , @corybrunson , @EmilHvitfeldt , @hfrick , @iamYannC , @jack-davison , @jameslamb , @martinju , and @topepo .

tune

The core functionality of tune is to do all the model fitting (including pre- and postprocessing) and performance evaluation across various resamples and tuning parameter combinations. For grid search, we could take the full parameter grid, splice one parameter combination into the workflow at a time, and run with it. That can be pretty inefficient though. So what actually happens in tune are a few optimizations in how we do all that fitting and evaluating: For preprocessing, we do it once for a resample (per preprocessing parameter combination) and then evaluate all model candidates on it. This lets us avoid unnecessarily repeating the same preprocessing multiple times. For model fitting, we make use of what Max calls “the submodel trick” : For certain models, like a boosted tree, you can use a submodel to make predictions without having to refit the model. A boosted tree ensemble fitted with 20 trees can be used to make predictions for any number of trees up to the 20 used for fitting. That allows us to evaluate different tuning parameter candidates for, here, the number of trees, without having to refit the model. When we added postprocessing, we temporarily disabled this (to ensure we got the integration right) - now we’ve brought it back. We make use of this speedup for both the main model as well as the calibration model.

Another big update is that the Gaussian process model package was changed from GPfit to GauPro because the former is no longer actively maintained. There are some differences:

Fit diagnostics are computed and reported. If the fit quality is poor, an “uncertainty sample” that is furthest away from the existing data is used as the new candidate.
The GP no longer uses binary indicators for qualitative predictors. Instead, a “categorical kernel” is used for those parameter columns. Fewer starting values are required with this change.
For numeric predictors, the Matern 3/2 kernel is always used.

Some other changes of note:

When calculating resampling estimates, we can now use a weighted mean based on the number of rows in the assessment set thanks to Tyler Burch. You can opt-in to this using the new add_resample_weights() function. See ?calculate_resample_weights
The warning threshold when check the size of a workflow is now a parameter to the control functions and has a new default of 100MB.

Some bug fixes:

Models with submodel parameters would train all calibration models on predictions from a single submodel value instead of the correct value for each submodel. We sorted this out.
We fixed a bug for cases where we tune a grid without a model parameter but with a postprocessing parameter.
Another bug was fixed for augment() when using last_fit() objects

Thanks to the following contributors: @edgararuiz , @EmilHvitfeldt , @hfrick , @jeroenjanssens , @jjcurtin , @mikewolfe , @mthulin , @ncalliencsu , @rvalieris , @StevenWallaert , @tjburch , and @topepo

finetune

This release was mostly focused on internal changes to support the new version of tune.

tidymodels

A basic release that updates the version numbers to require the latest releases of the core packages.

processx 3.9.0

Gábor Csárdi — Mon, 27 Apr 2026 00:00:00 +0000

We’re happy to announce the release of processx 3.9.0. processx is an R package to run and manage system processes.

You can install it from CRAN with:

install.packages("processx")

This blog post discusses the major new features in processx 3.9.0. You can see a full list of changes in the release notes .

Pipelines

New new pipeline class lets you connect two or more processes with kernel-level pipes, exactly like a Unix shell pipeline (cmd1 | cmd2 | cmd3): data flows directly between child processes without passing through R.

pl <- pipeline$new(
  list(c("sort"), c("uniq", "-c"), c("sort", "-rn")),
  stdin = "|",
  stdout = "|"
)
pl$write_input("banana\napple\nbanana\norange\napple\nbanana\n")
pl$close_input()
#> NULL
pl$read_all_output_lines()
#> [1] "   3 banana" "   2 apple"  "   1 orange"
pl$wait()
pl$get_exit_statuses()
#> [[1]]
#> [1] 0
#> 
#> [[2]]
#> [1] 0
#> 
#> [[3]]
#> [1] 0

The pipeline$new() constructor takes a list of character vectors — one per command — along with the usual stdin, stdout, and stderr arguments. These apply to the ends of the pipeline: stdin connects to the first process, stdout reads from the last, and stderr controls all processes.

The key benefit over calling run() in sequence is efficiency: intermediate data never materialises in R. A pipeline processing gigabytes of log lines uses the same small kernel buffers as a shell pipeline would.

Because each step in the pipeline is a regular process object under the hood, you can access individual processes via $get_processes() — useful for reading per-process stderr or checking exit codes when a stage fails.

pipeline works on Unix and Windows and is currently experimental: the API may still change.

Pseudo-terminal support

`processx::run(pty = TRUE)`

Many command-line tools behave differently when their output is not connected to a terminal: they disable colour, turn off progress bars, or buffer output more aggressively. The pty = TRUE option runs a process inside a pseudo-terminal so it sees a real terminal — colour and interactive behaviour included.

run() now supports pty = TRUE directly:

out <- run("ls", c("--color", path.expand("~/works/processx")), pty = TRUE)
cat(out$stdout)
#> DESCRIPTION    NAMESPACE      README.md      inst           tests
#> LICENSE        NEWS.md        _pkgdown.yml   man            tools
#> LICENSE.md     R              air.toml       processx.Rproj vignettes
#> Makefile       README.Rmd     codecov.yml    src

When pty = TRUE, stderr is merged into stdout (the result’s $stderr is always NULL), because a PTY has a single stream. You can also supply a file path as stdin; its contents are fed to the process via the PTY master, followed by an EOF signal.

Windows support

processx 3.9.0 adds support for pseudo-terminals (PTYs) on Windows, starting from Windows 10 version 1809. The Windows implementation uses the ConPTY API (CreatePseudoConsole), loaded dynamically so processx continues to load on older Windows and emits a clear error if pty = TRUE is requested on an unsupported version.

Other improvements

New process cleanup article

A new article, Process cleanup , documents all five mechanisms processx provides for ensuring subprocesses don’t outlive their intended scope:

Explicit cleanup with on.exit() — always deterministic.
Automatic cleanup on garbage collection (cleanup = TRUE, the default).
Process-tree cleanup (cleanup_tree = TRUE).
Linux parent-death signal (linux_pdeathsig) — Linux only, handles R crashes.
Supervisor process (supervise = TRUE) — all platforms, handles R crashes.

Death signal support on Linux

On Linux, you can now tell the kernel to deliver a signal to the child process automatically if the parent R process exits — even if R crashes. Set linux_pdeathsig = TRUE to send SIGTERM, or pass an integer signal number directly:

p <- process$new("sleep", "100", linux_pdeathsig = TRUE)

This is useful when you want child processes to clean up after an R crash, without the overhead of running a supervisor. The argument is silently ignored on macOS and Windows.

Record the time when a process exits

process$get_end_time() returns the time when the process exited as a POSIXct, or NULL if it is still running. This makes it straightforward to measure wall-clock duration without having to record timestamps yourself:

p <- process$new("sleep", "1")
p$wait()
p$get_end_time() - p$get_start_time()
#> Time difference of 1.010295 secs

Append stdout/stderr to files

process$new() and run() now support ">>" as a prefix for stdout and stderr file paths to append output instead of truncating the file:

log <- tempfile()
run("echo", args = "first line", stdout = log)
#> $status
#> [1] 0
#> 
#> $stdout
#> NULL
#> 
#> $stderr
#> [1] ""
#> 
#> $timeout
#> [1] FALSE
run("echo", args = "second line", stdout = paste0(">>", log))
#> $status
#> [1] 0
#> 
#> $stdout
#> NULL
#> 
#> $stderr
#> [1] ""
#> 
#> $timeout
#> [1] FALSE
readLines(log)
#> [1] "first line"  "second line"

This is handy when you run the same process repeatedly and want to accumulate output in a single log file.

Binary standard output and error

run() and process$new() now support encoding = "binary" to capture raw bytes. In binary mode, run() returns stdout and stderr as raw vectors, and process$read_output() / process$read_error() return raw vectors rather than character strings. All bytes are preserved exactly, including null bytes and non-UTF-8 sequences.

result <- run("cat", args = "/bin/ls", encoding = "binary")
typeof(result$stdout)
#> [1] "raw"
length(result$stdout)
#> [1] 154624

Two new methods, process$read_output_bytes() and process$read_error_bytes(), and the conn_read_bytes() function, provide direct access to raw bytes from processx connections.

Acknowledgements

Thanks to everyone who contributed to processx 3.9.0 through code, issues, testing, and feedback:

@advieser , @cderv , @chwpearse , @HenrikBengtsson , @king-of-poppk , @r2evans , @sckott , @sda030 , @stupidpupil , and @Yunuuuu .

2026-04-24 AI Newsletter

Sara Altman — Fri, 24 Apr 2026 00:00:00 +0000

The AI newsletter has moved! From this edition forward, we’ll be publishing here, on the Open Source website. You can read past editions on the main Posit blog .

External news

AI and cybersecurity

Anthropic recently announced Project Glasswing , an effort to bolster critical software against the threat of AI systems that are increasingly capable of finding and exploiting security vulnerabilities. Through the project, a number of organizations that maintain critical software will be allotted credits to Claude Mythos, Anthropic’s yet-to-be-publicly-released model that reportedly found high-severity security issues in “every major operating system and web browser” . The goal is to give these organizations a head start in finding and patching these vulnerabilities before bad actors have access to the same technology.

A step change for local coding agents

One of the most common questions we receive when releasing LLM-enabled software is “Can I use this with a local model?” Up until this point, we’ve largely discouraged using our products with LLMs small enough that you could run them on a laptop. Local models are not there (yet) showed that, at the time, models small enough to run on a laptop failed a simple refactoring task every time, while the frontier, cloud-hosted models were largely successful.

However, the last couple weeks have brought two important releases for the local model space— Gemma 4 and Qwen 3.6 . Both of these new models successfully completed the refactoring task in 9 of 10 attempts .

Posit news

Posit recently announced ggsql , an implementation of the grammar of graphics for SQL. One of the motivations for ggsql is safer AI agent tooling for visualization. With ggsql, rather than giving a visualization agent a full R or Python runtime (where the agent could execute arbitrary code, write files, or make network calls), you can provide it with a SQL runtime instead. SQL limits the agent to database operations, and you can restrict even those by connecting read-only, reducing the risk that the agent will do something undesirable.

Terms

A dense model is one where every parameter is used to process every token. For example, if a model has 14 billion parameters, in a dense model, all 14 billion are involved in generating each token.

Dense models are in contrast to sparse models, a subset of which use an architecture called Mixture of Experts (MoE). In MoE models, only a fraction of the model’s parameters are involved at any given time. The model contains many parallel sub-networks called “experts,” and a small router decides which ones to activate for each token. This means a model only needs to pay the computational cost of the small active subset per token.

This is the architecture of the most recent local models, and partially explains the jump in ability we saw. Gemma 4 26B-A4B, for example, has 26 billion total parameters but only activates 4 billion per token (the “A” is for “active parameters”). The models we tested from the previous generation were dense, requiring you to choose between a model small enough to run on a laptop or one that was good enough to be useful. The MoE architecture helps break that tradeoff.

Learn more

Anthropic released Opus 4.7 , an incremental improvement over Opus 4.6. OpenAI released GPT-5.4-Cyber , a fine-tune of GPT 5.4 for cyberdefense.
A new paper shows that vision language models excel at many image benchmarks even when the actual images aren’t provided, because the answers are implicit in the questions.
Posit’s Jeremy Allen has been keeping a log of the ways AI is transforming work .

Blog on Posit Open Source

Introducing Toolbars: Supercharge your Cards and Inputs

Our Example

Setup Code

Toolbars in Card Headers & Footers

AI Newsletter: Gemma 4 in Posit Assistant

External news

Posit news

Gemma 4 in Posit Assistant

Tidy design principles

In Defense of YAML

A brief history of configuration formats

The case against YAML (as it was)

What TOML gets right

Where TOML strains

What YAML 1.2 changed

The Python YAML parser landscape

Introducing py-yaml12

Simple

Safe

Fast

Conclusion

Welcome to the New Home for Posit Open Source

Curate

Connect

Contribute

Ten Great Things We Added to Great Docs

1. Auto-Discovery

2. SEO and Proofreading

3. Documentation Linting

4. Internationalization

5. Keyboard Navigation

6. Page Tags and Status Badges

7. Scale-to-Fit

8. Versioned Documentation

9. Color Swatches

10. Table Previews and Table Explorer

What Ten Releases Add Up To

A memory model for production async R: mirai 2.7.0 and mori 0.2.0

Building out the foundations

mirai 2.7.0 — bounded queues and try_mirai()

daemons(memory = ...)

try_mirai()

Under the hood

mori 0.2.0 — out of experimental status

Composing with daemons(memory = ...)

Putting R on the concurrency map

Try it

Acknowledgments

Positron May Release Highlights

Product updates

Announcing the Packages pane

Inline output for Quarto

Extensions available from Posit Public Package Manager

Positron Notebook Editor is now in beta

AI chat and code assistance

What’s coming next

AI Newsletter: A Data Cleaning Mode for Posit Assistant

Posit news

Terms

Learn more

Bringing OpenTelemetry to R in production

What is OpenTelemetry?

Why observability matters for R in production

Instrumented packages

Seeing it in action

Getting started

Step 1: Install otelsdk

Step 2: Choose a backend

Step 3: Set environment variables

Step 4: Run your app

Zero-code instrumentation

Configuration options

Looking ahead

Quarto 2: Parsing and Source Maps

UX Requirements for a text-centric authoring system

Requirement 1: Syntax errors

Requirement 2: Accurate, fine-grained source maps

Requirement 3: Syntax stability over time

A tree-sitter parser for Markdown

mirai 2.7.0 — bounded queues and `try_mirai()`

`daemons(memory = ...)`

`try_mirai()`

Composing with `daemons(memory = ...)`

`bit64::integer64`

`blob::blob`