For a real-time interactive Mandelbrot viewer I was making in R & Rcpp+OpenMP & Shiny I am on the lookout for a performant way to display 1920x1080 matrices as raster images in the hope of being able to achieve ca. 5-10 fps (calculating the Mandelbrot images themselves now achieves ca. 20-30 fps at moderate zooms, and certainly scrolling around should go fast). Using image() with option useRaster=TRUE, plot.raster or even grid.raster() still doesn't quite cut it, so I am on the lookout for a more performant option, ideally using OpenGL acceleration.
I noticed that there are qt wrapper packages qtutils and qtpaint
http://finzi.psych.upenn.edu/R/library/qtutils/html/sceneDevice.html
where you can set argument opengl=TRUE and
http://finzi.psych.upenn.edu/R/library/qtpaint/html/qplotView.html
again with argument opengl=TRUE and http://finzi.psych.upenn.edu/R/library/qtpaint/html/painting.html.
And I also noticed that one should be able to call SDL and GL/OpenGL functions using the rdyncall package (install from https://cran.r-project.org/src/contrib/Archive/rdyncall/ and SDL from https://www.libsdl.org/download-1.2.php)`, demos available at http://hg.dyncall.org/pub/dyncall/bindings/file/87fd9f34eaa0/R/rdyncall/demo/00Index, e.g. http://hg.dyncall.org/pub/dyncall/bindings/file/87fd9f34eaa0/R/rdyncall/demo/randomfield.R).
Am I correct that with these packages one should be able to display a 2D image raster using opengl acceleration? If so, has anyone any thoughts how to do this (I'm asking because I'm not an expert in either qt or SDL/OpenGL)?
Some timings of non-OpenGL options which are too slow for my application:
# some example data & desired colour mapping of [0-1] ranged data matrix
library(RColorBrewer)
ncol=1080
cols=colorRampPalette(RColorBrewer::brewer.pal(11, "RdYlBu"))(ncol)
colfun=colorRamp(RColorBrewer::brewer.pal(11, "RdYlBu"))
col = rgb(colfun(seq(0,1, length.out = ncol)), max = 255)
mat=matrix(seq(1:1080)/1080,nrow=1920,ncol=1080,byrow=TRUE)
mat2rast = function(mat, col) {
idx = findInterval(mat, seq(0, 1, length.out = length(col)))
colors = col[idx]
rastmat = t(matrix(colors, ncol = ncol(mat), nrow = nrow(mat), byrow = TRUE))
class(rastmat) = "raster"
return(rastmat)
}
system.time(mat2rast(mat, col)) # 0.24s
# plot.raster method - one of the best?
par(mar=c(0, 0, 0, 0))
system.time(plot(mat2rast(mat, col), asp=NA)) # 0.26s
# grid graphics - tie with plot.raster?
library(grid)
system.time(grid.raster(mat2rast(mat, col),interpolate=FALSE)) # 0.28s
# base R image()
par(mar=c(0, 0, 0, 0))
system.time(image(mat,axes=FALSE,useRaster=TRUE,col=cols)) # 0.74s # note Y is flipped to compared to 2 options above - but not so important as I can fill matrix the way I want
# magick - browser viewer, so no good....
# library(magick)
# image_read(mat2rast(mat, col))
# imager - doesn't plot in base R graphics device, so this one won't work together with Shiny
# If you wouldn't have to press ESC to return control to R this
# might have some potential though...
library(imager)
display(as.cimg(mat2rast(mat, col)))
# ggplot2 - just for the record...
df=expand.grid(y=1:1080,x=1:1920)
df$z=seq(1,1080)/1080
library(ggplot2)
system.time({q <- qplot(data=df,x=x,y=y,fill=z,geom="raster") +
scale_x_continuous(expand = c(0,0)) +
scale_y_continuous(expand = c(0,0)) +
scale_fill_gradientn(colours = cols) +
theme_void() + theme(legend.position="none"); print(q)}) # 11s
question from:
https://stackoverflow.com/questions/48119360/performant-2d-opengl-graphics-in-r-for-fast-display-of-raster-image-using-qtpain 
