How can I render plotly plots without using plotlyOutput and renderPlotly?

Fixing your solution

The height is not really 200px, but instead the plots overlap. How can I fix that?

Pass the height-style to the divs holding the plotly plots, that way the plots will follow the height of the container they are placed in.

library(shiny)
library(plotly)

ui <- fluidPage(
  htmltools::findDependencies(plotly_empty()),
  selectizeInput(
    inputId = "plotsToShow",
    label = "Choose what plots to diplay. Note that the order is respected.",
    choices = letters,
    selected = letters[5:7],
    multiple = TRUE
  ),
  DT::DTOutput("dt")
)

server <- function(input, output, session) {
  output$dt <- DT::renderDT({
    plotsToShow <- input$plotsToShow
    df <- data.frame(
      x = paste0("plot", seq_along(plotsToShow)),
      y = paste0("plot", plotsToShow),
      z = sapply(seq_along(plotsToShow), \(i) {
        as.character(
          plot_ly(iris, x = ~Sepal.Width, y = ~Sepal.Length, color = ~Species, type = "scatter", mode = "markers") |>
            plotly::layout(title = list(text = paste("plotly position", i, ", name", plotsToShow[i]))) |>
            div(style = list(height = "200px")) # <-- define plot heights here
        )
      })
    )
    DT::datatable(df, escape = FALSE,
                  options = list(
                    columnDefs = list(list(targets = 3, width = "90%")),
                    drawCallback = V8::JS('function() { Shiny.bindAll(this.api().table().node()); }')
                  ))
  })
}

shinyApp(ui, server, options = list(launch.browser = TRUE))

Nothing is drawn when I comment the lines containing Shiny.bindAll and Shiny.unbindAll. Why are these two lines responsible for drawing? Is there a more direct way to draw the plots?

Nothing is drawn if you omit Shiny.bindAll. I will borrow this explanation because it's great:

"In most web applications the HTML markup and all CSS/JS assets are loaded more or less at the same time, and then an onLoad/onDOMReady event is fired, which is the signal for all the JS-driven components to go look for instances of themselves on the page and run initialization code, attach event handlers, etc. In the case of Shiny, such logic instantiates fancy selectize.js controls for selectInput, fancy ion rangesliders for sliderInput, and for all inputs/outputs whether or simple or fancy, tells Shiny of the existence of these inputs/outputs" like your plotly elements.

"When the HTML markup is modified so that inputs/outputs are either added or removed, this kind of logic needs to run again, or else removed outputs will still be considered on the page and new inputs/outputs won't be initialized and hooked up to Shiny. We call this process "binding", and when you modify the HTML using our built-in functions like uiOutput/renderUI, insertUI/removeUI, etc., we call Shiny.bindAll()/unbindAll() automatically. But if you modify the HTML at runtime using your own JavaScript, you have to explicitly invoke these functions yourself." In the data.table case, you have to rebind everytime the table is drawn = preDrawCallback/drawCallback.

Now that you understand this, you can

Bind plotly plots inside data.table manually

You were asking about different solutions. In this one I create the empty divs in which the plots will go and then simply draw them on drawCallback using plotly.JS.

library(plotly)
library(htmltools)
library(shiny)

ui <- fluidPage(
  htmltools::findDependencies(plotly_empty()),
  selectizeInput(
    inputId = "plotsToShow",
    label = "Choose what plots to display. Note that the order is respected.",
    choices = letters,
    selected = letters[5:7],
    multiple = TRUE
  ),
  DT::DTOutput("dt")
)

server <- function(input, output, session) {
  output$dt <- DT::renderDT({
    plotsToShow <- input$plotsToShow
    df <- data.frame(
      a = seq_along(plotsToShow),
      # build empty divs for plotly plots - adress by id later, give them a height
      z = sprintf('<div id="plot_%s" style="height: 200px;"></div>', seq_along(plotsToShow))
    )
    # build as many plots as plotsToShow is long
    plot_json <- lapply(seq_along(plotsToShow), \(i) {
      plotly_build(plot_ly(iris, x = ~Sepal.Width, y = ~Sepal.Length, color = ~Species, type = "scatter", mode = "markers") |>
                     plotly::layout(title = list(text = paste("plotly position", i, ", name", plotsToShow[i]))))$x[c("data", "layout")]
    })
    
    DT::datatable(df, escape = FALSE, options = list(
      columnDefs = list(list(targets = 2, width = '50%')),
      # in each drawbackCall (when data.table rerenders, use plotly js to draw plots from "plot_json" inside the divs
      drawCallback = DT::JS(sprintf(
        "function(el, x) {
          %s.forEach((p, i) => {
            let d = document.getElementById('plot_' + (i+1));
            d && Plotly.newPlot(d, p.data, p.layout, {displayModeBar: false});
          });
        }", jsonlite::toJSON(plot_json, auto_unbox = TRUE))
      )
    ))
  })
}

shinyApp(ui, server, options = list(launch.browser = TRUE))

Use <iframes>

Finally, you can also save each widget as a .html file and then source these in your shiny app. The app below whould just demonstrate the steps, I would not suggest doing it exactly like this because each time datatable is rerendered the widgets are saved again and again. Typically, you would save them once in the www folder and then source whenever they are needed. In your case, you would pregenerate 26 plots, on per letter and then reference them in src of the iframes.

library(shiny)
library(plotly)
library(DT)
library(htmlwidgets)

ui <- fluidPage(
  selectizeInput(
    "plotsToShow", "Choose plots to display (order respected):",
    choices = letters, selected = letters[5:7], multiple = TRUE
  ),
  DTOutput("dt")
)

server <- function(input, output, session) {
  temp_dir <- tempdir()
  addResourcePath("plots", temp_dir)
  
  output$dt <- renderDT({
    plotsToShow <- input$plotsToShow
    
    df <- data.frame(
      x = paste0("plot", seq_along(plotsToShow)),
      y = paste0("plot", plotsToShow),
      z = sapply(seq_along(plotsToShow),\(i) {
        p <- plot_ly(iris, x = ~Sepal.Width, y = ~Sepal.Length, color = ~Species, type = "scatter", mode = "markers") |>
          plotly::layout(title = list(text = paste("plotly position", i, ", name", plotsToShow[i]))) |>
          div(style = list(height = "200px"))
        
        file <- file.path(temp_dir, sprintf("plot_%d.html", i))
        saveWidget(p, file, selfcontained = TRUE)
        sprintf('<iframe src="/plots/plot_%d.html" style="width:100%%;border:none;"></iframe>', i)
      }
      )
    )
    
    datatable(df, escape = FALSE, options = list(columnDefs = list(list(targets = 3, width = "90%"))))
  })
}

shinyApp(ui, server, options = list(launch.browser = TRUE))