metadata-files is slow downing the single file render if there is a R file with yaml header format

[byzheng]

I have:

• searched the issue tracker for similar issues
• installed the latest version of Quarto CLI
• formatted my issue following the Bug Reports guide

Bug description

quarto render a single file is slow with extra calling Rscript.exe under these conditions

• _quarto.yml file has metadata-files attribute
• a R file with yaml header

it will take very long time to render a single file if there are many R scripts with yaml header in the project (40 s vs 3 s).

Steps to reproduce

• Check out the repository with a minimum reproduceable example: https://github.com/byzheng/quarto-metadata-files-test
• Call quarto render .\index.qmd --log-level debug to render index.qmd file
• Notice call Rscript.exe three times in the console output

[execProcess] C:\Users\<username>\AppData\Local\Programs\R\R-4.5\bin\Rscript.exe C:\Users\<username>\AppData\Local\Programs\Quarto\share\rmd\rmd.R
[execProcess] Success: true, code: 0
[execProcess] C:\Users\<username>\AppData\Local\Programs\R\R-4.5\bin\Rscript.exe C:\Users\<username>\AppData\Local\Programs\Quarto\share\rmd\rmd.R
[execProcess] Success: true, code: 0
[execProcess] C:\Users\<username>\AppData\Local\Programs\R\R-4.5\bin\Rscript.exe C:\Users\<username>\AppData\Local\Programs\Quarto\share\rmd\rmd.R
[execProcess] Success: true, code: 0

• rename utils.R to qmd extension or merge _website.yml into _quarto.yml. The same call to render index.qmd will not see the extra calling of Rscript.exe.

Actual behavior

See details from this minimum example: https://github.com/byzheng/quarto-metadata-files-test

Expected behavior

No extra calling of Rscript.exe rmd.R if

• _quarto.yml file has metadata-files attribute
• a R file with yaml header

Your environment

• VS code
• Windows 11
• Quarto version: 1.8.27 and 1.9.35

Quarto check output

Quarto 1.8.27
[>] Checking environment information...
      Quarto cache location: C:\Users\<username>\AppData\Local\quarto
[>] Checking versions of quarto binary dependencies...
      Pandoc version 3.6.3: OK
      Dart Sass version 1.87.0: OK
      Deno version 2.3.1: OK
      Typst version 0.13.0: OK
[>] Checking versions of quarto dependencies......OK
[>] Checking Quarto installation......OK
      Version: 1.8.27
      Path: C:\Users\<username>\AppData\Local\Programs\Quarto\bin
      CodePage: 1252

[>] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[>] Checking LaTeX....................OK
      Tex:  (not detected)

[>] Checking Chrome Headless....................OK
      Using: Chrome found on system
      Path: C:\Program Files\Google\Chrome\Application\chrome.exe
      Source: Windows Registry

[>] Checking basic markdown render....OK

[>] Checking Python 3 installation....OK
      Version: 3.13.12
      Path: C:/Users/<username>/AppData/Local/Microsoft/WindowsApps/PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0/python.exe        
      Jupyter: (None)

      Jupyter is not available in this Python installation.
      Install with py -m pip install jupyter

[>] Checking R installation...........OK
      Version: 4.5.2
      Path: C:/Users/<username>/AppData/Local/Programs/R/R-4.5
      LibPaths:
        - C:/Users/<username>/AppData/Local/Programs/R/R-4.5/library
      knitr: 1.51
      rmarkdown: 2.30

[>] Checking Knitr engine render......OK

[mcanouil]

Your Quarto documents have no code and no engine set, so Quarto has no need to call Rscript or Jupyter.
Your R script on the other hand, is not a Quarto document thus requires Quarto Knitr internals to do the translation.
So in your setup that's totally expected.

metadata-files is not a shortcut to simply copy/paste content.
There are guardrails to ensure proper merger of metadata.

graph TD
    A[Base Metadata] --> B{Check for metadata-file/metadata-files}
    B -->|Found| C[Read YAML Files]
    B -->|Not Found| D[Use Base Metadata Only]
    C --> E[Validate Against Schema]
    E --> F[Merge with Base Metadata]
    F --> G[Return Combined Metadata]
    D --> G

Loading

With/without metadata-files:

2.40s user 0.88s system 80% cpu 4.068 total
1.37s user 0.42s system 106% cpu 1.686 total

without R script and with/without metadata-files:

2.44s user 0.85s system 83% cpu 3.953 total
1.51s user 0.62s system 81% cpu 2.605 total

Even if I ran only once, hard to state that the R script is costing really something in your example, but it would be expected to have more time spent to parse R script via R with more files than parsing Quarto documents via TypeScript.

I would bet you have something in your R environment that is causing this "40 s vs 3 s" difference.
You might have renv activated with implicit discovery, meaning before Quarto/knitr is doing anything, they have to wait for renv to finish packages discovery for example.

Now regarding the three calls, it's different phase of the processing of the R files which are required. The three "identical" calls are in fact three different calls doing different things.

• converting R script to Markdown (knitr spin)
• execute the R code to retrieve the output
• extract dependencies from the R script

You can see the "main" dispatch logic in:

• https://github.com/quarto-dev/quarto-cli/blob/main/src/resources/rmd/rmd.R#L5

[mcanouil]

To clarify, I only reported latest macOS timing, but I'm seeing similar results on other setups.

edit: Codespaces (https://github.com/mcanouil/quarto-codespaces with 1.9): 2s without R script, 3.5 s with R script

To summarise:

• the example alone is not enough to reproduce the delay imputed to R script.
• the additional time from metadata-files processing seems to be fixed (and unrelated to R) and expected.
• the three calls to R, even if calling different sub-functions might create performance issues in some setup using renv, .Rprofile, etc.

[mcanouil]

@byzheng Could you try setting the environment variable QUARTO_KNITR_RSCRIPT_ARGS="--vanilla" before rendering?

[byzheng]

@mcanouil Thanks for your testing.

The original one is only a minimum example to reproduce my problem. I add more R files to this repo to test my real case.

• Add 20 R scripts under script folder
• Add three quarto config files
  • _quarto1.yml to render index.qmd with metadata-files and rscript files
  • _quarto2.yml to render index.qmd without metadata-files and with rscript files
  • _quarto3.yml to render index.qmd with metadata-files but without rscript files

After calling test.R in the root folder

• _quarto1.yml takes 58 s
• _quarto2.yml takes 2.7 s
• _quarto3.yml takes 2.6 s

The runtime is under window 11 laptop with a single test.

PS:

After calling Sys.setenv(QUARTO_KNITR_RSCRIPT_ARGS="--vanilla") , the runtime of _quarto1.yml still takes about 58s

[mcanouil]

How are you calling Quarto? Via the R package?
Be sure to check the --vanilla is accounted for in your debug log:

[execProcess] C:\Users\<username>\AppData\Local\Programs\R\R-4.5\bin\Rscript.exe --vanilla ...

[byzheng]

Yes I call it from R and can confirm --vanilla in the Rscript calls.

[execProcess] C:\Users\username\AppData\Local\Programs\R\R-4.5\bin\Rscript.exe --vanilla C:\Users\username\AppData\Local\Programs\Quarto\share\rmd\rmd.R

[mcanouil]

You only tried 1.8, correct?

As I said earlier, 1.9 is about to be released, so my tests were using 1.9, not 1.8.
You can see the full changelog: https://quarto.org/docs/download/

[byzheng]

I have tried 1.9 and revert back to 1.8.

1.8.27 and 1.9.35 have the same behaviours.

[byzheng]

@mcanouil

In my repo, I also created an github action to test on windows, macos, and linux with latest R and quarto-1.8.27 for the three scenarios above. Can find the github action log.

os	scenario	platform	r_version	quarto_version	rscript_calls	elapsed	user	system
Linux	metadata_file + rscript	Linux | 6.14.0-1017-azure | x86_64	4.5.0	1.8.27	0	31.124	0.23	0.03
Linux	inline_metadata + rscript	Linux | 6.14.0-1017-azure | x86_64	4.5.0	1.8.27	0	1.498	0.0720000000000001	0.017
Linux	metadata_file - rscript	Linux | 6.14.0-1017-azure | x86_64	4.5.0	1.8.27	0	1.52800000000001	0.072	0.017
Windows	metadata_file + rscript	Windows | Server x64 | x86-64	4.5.0	1.8.27	0	49.57	0.35	0.06
Windows	inline_metadata + rscript	Windows | Server x64 | x86-64	4.5.0	1.8.27	0	1.99	0.09	0.03
Windows	metadata_file - rscript	Windows | Server x64 | x86-64	4.5.0	1.8.27	0	2.02	0.1	0.02
macOS	metadata_file + rscript	Darwin | 24.6.0 | arm64	4.5.0	1.8.27	0	25.331	0.091	0.021
macOS	inline_metadata + rscript	Darwin | 24.6.0 | arm64	4.5.0	1.8.27	0	1.049	0.031	0.01
macOS	metadata_file - rscript	Darwin | 24.6.0 | arm64	4.5.0	1.8.27	0	0.991999999999997	0.03	0.00900000000000001

[mcanouil]

Thanks, the team will take a look.

Important

Accessibility: For better accessibility, please ensure link text is descriptive and meaningful rather than using generic terms such as "here". This helps all users understand the link's destination. See Writing meaningful link text (WCAG).