javascript - Convert touch events to mouse events within canvas in iframe on a webview

Question

I'm porting a huge game collection of web based games to run on Windows touch devices (i.e. Kiosk mode devices) within an Electron app. Unfortunately, the games sometimes use 'mouseup' instead of 'click' (for instance), so they don't fire on a touch screen, at all. Also the games are always using a canvas to display the content of the game and terribly, it's nested via iframe.

The games are often transcompiled/minified and vary a lot between each other, so I'm not interested and re-writing stuff in the games code at all (too much effort, I'm serious, there are a lot of games). Since I host the games within my own Chromium WebView, I can inject my own JavaScript, so I hope there is a solution.

My initial idea was to catch the 'touchstart', 'touchmove' and 'touchend' events to simulate at least 'mousedown' and 'mouseup'. This would be enough, because there is no dragging of elements required in any of the games.

This is my attempt and it works on normal HTML pages, however the games nest the canvas within an inner iframe, so the webview disallows me to receive those events.

This cannot work for content in iframes:

(function() {
    var x, y;
    document.addEventListener('touchstart', function(e) {
        console.log("touchstart", e);
        var element = document.elementFromPoint(x = e.touches[0].clientX, y = e.touches[0].clientY);
        console.log("element", element);
        var rect = element.getBoundingClientRect();
        console.log("rect", rect);
        element.dispatchEvent(new MouseEvent('MouseDown', {
            clientX: x - rect.left,
            clientY: y - rect.top,
            bubbles: true
        }));
    });
    document.addEventListener('touchmove', function(e) {
        console.log("touchmove", e);
        x = e.touches[0].clientX;
        y = e.touches[0].clientY;
    });
    document.addEventListener('touchend', function(e) {
        console.log("touchend", x, y);
        var element = document.elementFromPoint(x, y);
        console.log("element", element);
        var rect = element.getBoundingClientRect();
        console.log("rect", rect);
        element.dispatchEvent(new MouseEvent('MouseUp', {
            clientX: x - rect.left,
            clientY: y - rect.top,
            bubbles: true
        }));
    });
})();

This is how the webview is implemented in Electron (vue.js):

<webview
  ref="games"
  :src="url"
  :class="{ 'fullscreen': fullscreen, 'hidden': false }"
  id="games"
  disablewebsecurity="true"
  onload="onwebviewload"
  allowfullscreen="true"
/>

I enabled devTools to inject the JavaScript above, for now. It would be good to be able to use disablewebsecurity="false", however it won't even work with disablewebsecurity="true".

What can I do to make it work?

question from:https://stackoverflow.com/questions/65862643/convert-touch-events-to-mouse-events-within-canvas-in-iframe-on-a-webview

Answer 1

Waitting for answers