Barret Schloerke: Lessons Learned Testing 2500+ Shiny Apps Every Day
About the Talk:
The Shiny team tests 2500+ different combinations of Shiny Applications, R versions, and operating systems to verify no feature regressions occur within the bleeding edge of the Shiny-verse. Over the past few years, we have learned a few lessons to keep our tests robust and honest. There is a non-zero chance that a test failure is not your fault. When working with CI systems, external influences such as installation failures or slower computing power will prevent tests from executing properly. A picture is worth a thousand tests but visual testing is known to have a non-zero false-positive rate. To address this, we have utilized threshold based image comparisons for flaky tests. We have found that it is best to test the minimal amount of content where possible. Changes in App package dependencies can produce unintended changes in expected outputs. These hard learned lessons have reduced unreliable test results allowing the Shiny team to quickly respond to feature regressions.
Speaker's bio:
Dr. Barret Schloerke is a Software Engineer at Posit. He currently develops and maintains many R packages in the Shiny ecosystem at RStudio including shiny, reactlog, plumber, learnr, leaflet, and shinyloadtest. Dr. Schloerke received his PhD in Statistics from Purdue University under the direction of Dr. Ryan Hafen and Dr. William Cleveland, specializing in Large Data Visualization
Barret Schloerke, Shiny Team
leaflet
learnr
plumber
reactlog
rstudio
Shiny
shinyloadtest
