| Issue 72754: | Mouse Lock Javascript API | |
| 138 people starred this issue and may be notified of changes. | Back to list |
Sign in to add a comment
|
Mouse Lock Prototype Request I'm under the impression that to make a specification change a prototype is recommended from an implementor. So if someone is interested in writing a prototype of this then it would be appreciated. Purpose: To allow browsers to lock the mouse to the region inside of a DOM element. The other feature is the ability to get the change of the mouse position in a mousemove event. The reason for this is to further the movement of applications into the browser. In the realm of games locking the mouse opens up the ability for a first person shooter (FPS), such as the Quake WebGL demo, to use mouse look. It also allows real-time strategy games to scroll the map by moving the cursor to the edge of the game screen. In normal applications it allows google maps to allow someone to drag the map without stopping just because the mouse hit the edge of the screen. Implementation: Two events are required and could be registered to any DOM element: mouselock(event) mouseunlock(event) mouselock is invoked after the mouse cursor is successfully locked to an element. The event in the function could be called MouseLockEvent. This fits the current naming scheme of MouseEvent, KeyboardEvent, and WheelEvent. Three methods would be added to elements: element.lockMouse() element.unlockMouse() element.setMousePosition(x, y) lockMouse would lock the mouse to the DOM region. If called on another element it would unlock on the current element and then lock on the new one. unlockMouse would unlock the mouse from the DOM region if currently locked. It would be implementation dependent if a restriction is placed on the lockMouse function. In the specification there should be a suggestion to implementors that on the first call a balloon (such as the yellow bar in Firefox or Chrome) would ask: "This website has requested to lock your mouse [Allow] [Deny] [What is this?]" This would be per domain ideally to prevent exploitation. (It's still up for discussion if more should be done for users to prevent locking the mouse continuously so that users don't have to close the browser if that occurs). Assuming the mouse is locked then calling element.setMousePosition(x, y) which would move the mouse pointer relative to the locked region. Release Key It has also been brought up that there should be uniform support for a release key separate from the webpage releasing the key. A recommendation should be made for making the escape key the release key. It would simply call element.unlockMouse() on the current mouse locked DOM element. Fullscreen An application in fullscreen shouldn't have to ask for permission to lock the mouse. However if the webpage isn't given permission to lock the mouse then switching out of fullscreen would unlock the mouse. Mouse Move Delta The other feature required is the ability to get the change in the mouse position in the mousemove event. This requires modifying the MouseEvent to add a deltaX and deltaY variable representing the changes in the mouse position. However there is a side affect to this which might be a bad design decision. The WheelEvent inherits the MouseEvent and has its own deltaX and deltaY. Either different names should be made for deltaX and deltaY or deltaX and deltaY should be removed from WheelEvent. They're both the same data type so there's no conflicts in moving deltaX and deltaY to the MouseEvent structure. They'd just be used differently. In mousemove deltaX and deltaY would represent the change in the mouse position and in a wheel event they'd represent the mouse wheel scroll deltas as defined in the DOM Level 3 specification. Use case from the mailing list: It applies to non-game uses, too. For example, a common annoyance with Google Maps is when you're dragging the map and your mouse cursor hits the side of the screen, the map stops moving; you have to release the button and reposition the cursor. Mouse grabbing would trivially fix this. -- Glenn Maynard To illustrate this you have a mousedown event on the map region that calls element.lockMouse() then on mouseup calls element.unlockMouse() give or take. Buglist Entry at W3C where I log changes to the ideas. I'd recommend posting there. If not I'll probably merge the ideas I see elsewhere into it so they're in the same place: http://www.w3.org/Bugs/Public/show_bug.cgi?id=9557#c12 Mailing list entries can be found here which seems more active: http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/
Comment
1
by
jshin@chromium.org,
Feb 12, 2011
Labels: -Type-Bug -Area-Undefined Type-Feature Area-WebKit OS-All
,
Feb 15, 2011
(No comment was entered for this change.)
Labels: Mstone-X
,
May 18, 2011
(No comment was entered for this change.)
Labels: ImportantForGames
,
May 18, 2011
Issue 82963 has been merged into this issue.
,
May 18, 2011
eseidel has looked at this I think. There are security implications.
Owner: esei...@chromium.org
,
May 18, 2011
IMHO the main implication is that the user would have to allow a page to grab the mouse or move it (otherwise a rogue page can get you stuck), and have some way to let it go (a shortcut?). Are there other implications?
,
May 18, 2011
I think that it will be secure if: - A dialog box appears asking the user if he want to enable the "captured" mouse mode - The mouse does not affect to the system when the "captured" mouse mode is on (like vmware does), it's captured into the canvas - Once the user press a shortcut the "captured" mouse mode is disabled.
,
May 18, 2011
I know nothing about this bug and have no plans to touch it beyond this comment.
,
May 19, 2011
Thus we need a champion.
Owner: ---
,
May 26, 2011
(No comment was entered for this change.)
Labels: -ImportantForGames Hotlist-ImportantForGames
,
May 31, 2011
I just wanted to add another game use case for this, which is pretty important to my project and quite a few others. Besides just FPS and top-down action camera controls (which are essentially impossible to implement without mouse locking), regular action-oriented puzzle games and top-down adventure games need this for a smooth user experience. If you take a popular game type like the Zuma clones, the player is expected to click very rapidly while moving around all over the game canvas. It's _very_ easy to accidentally click outside of the game canvas, either losing game focus or even activating some browser control. I'm totally okay with having some kind of opt-in query, similar to how the GeoLocation API works, especially if the browser can remember that the user allowed a page previously. I do want to note that use cases like the Google Maps drag seems to be a slightly different issue, and from my experience working with desktop toolkits like GTK/Qt, I'm not sure that an application-controlled mouse locking API is best suited to that. It really sucks when an application errors leaves the mouse locked, even if there is a hotkey to break it. For drag locking, it might be a lot better to just allow the drag/scroll API to specify that the mouse should be locked so long as the drag is active, thus leaving it up to the browser to automatically break the lock when the drag button is released. The other advantage there is that that API doesn't need privileges (the locking is directly under the control of the user interaction, not th eapplication's whims), so any site could use it without needing opt-in permissions or risking confusing a user with a locked mouse and a potentially unknown/hidden hotkey to restore their desktop functionality.
,
Jun 7, 2011
(No comment was entered for this change.)
Status: Assigned
Owner: sch...@chromium.org
,
Jun 8, 2011
Local discussion, where myself and Joel (aka Mr ForPlay) are former game and graphics developers, led to this: 1. Full screen apps effectively own the pointer anyway, but they need mouse move deltas to handle the mouse moving off screen. That is, in cases where the position is not meaningful because it would no longer be on screen. 2. For apps with panning interactions (map dragging, dragging out a selection box, ...), a drag event should capture the mouse during the drag. This is standard UI behavior and everyone sort of expects it. Chrome already does it, but Firefox does not. That's bad Firefox behavior in my opinion. 3. The use cases for non-dragging mouse capture "in window" seem insufficient to overcome the usability problems with apps that infinitely capture the mouse. The problems are discussed above and a permissions model doesn't solve the usability issues. Users will just click yes to allow the capture (how many users will even understand what it means?) and then be horribly confused and/or offended when the mouse ceases to behave as they expect. So my plan is to add mouse move delta information and see where that gets us.
,
Jun 8, 2011
Just thought you might not have considered this: it won't work with multi monitor setups.
,
Jun 8, 2011
About the point 3
Maybe the user doesn't understand what the yes button does. But once the user clicks the button if the mouse cursor disappears and a persistent overlay message appears into a Chrome's window corner ("Mouse cursor locked. Press XXX+YYY keys to show the mouse cursor again") the user is not horribly confused and/or offended because an explicit and presistent message is present.
,
Jun 8, 2011
In response to point #1, keep in mind that fullscreen apps only "own" the pointer on single-display systems, and only when they have no overlays or popups that the user might interact with. So far as point #3, no locked cursors means no action/fast-paced games in the browser in pure HTML5, period. I've reviewed non-browser games that have a similar behavior (no cursor locking) and it invariably ends up being the single worst aspect of the game's UI, often resulting in lost focus or an inability to pan around the game world (where you really don't want to have to hold a key down to do so). I'm with the previous commenter that a persistent overlay is acceptable if deemed necessary.
,
Jun 8, 2011
+1
,
Jun 8, 2011
In fullscreen mode currently, moving the mouse to the top of the screen causes the menu bar to appear.
,
Jun 9, 2011
Yeah that's a problem. I solved it in another bug post two days prior to when I made this one. http://code.google.com/p/chromium/issues/detail?id=72515 I'm glad this was assigned to someone. :)
,
Jun 17, 2011
Just a suggestion: This spec should have another recommendation, that in case a user persistently is tapping the "unlock mouse" key the UA should notify the user with a "Do you want to prevent this page from locking the mouse" like when a page spits too much alert() messages in chrome or firefox.
,
Jun 22, 2011
(No comment was entered for this change.)
Status: Started
,
Jun 22, 2011
draft specification: https://docs.google.com/document/d/1uV4uDVIe9-8XdVndW8nNGWBfqn9ieeop-5TRfScOG_o/edit?hl=en_US&authkey=CM-dw7QG
,
Jun 25, 2011
Hi Vincent, Thanks for working in this issue. It's great to see some progress. I quickly read the spec and it looks good so far. However I'm wondering wether always hiding the mouse is a good idea. I can see usecases where keeping the cursor visible is desirable. Mostly when you want to lock on mouse down and unlock on mouse up. For some dial/slider control. Another important feature for this to work as expected would be to explicitly state that a successive lock/unlock should not require user interaction anymore. In other words, don't ask me multiple times to allow a site to lock my mouse to an element. That's all I can think of for now. Thanks again, Jonas
,
Jun 25, 2011
To add a bit to Jonas' comment (which I fully agree with), instead of hiding the mouse cursor by default, just fix issue # 87489 and then let the app developers decide whether they want to hide the cursor or not.
,
Jun 30, 2011
Jonas, Sean, Hiding the cursor is necessary on OSes that only support reading mouse position. To implement sending the deltas the cursor must be warped to e.g. the center of the screen. The point of mouse lock is to use all mouse input in a way different than how the OS interprets it as a cursor position on screen. Mouse Capture is a different use case that allows mouse events to continue to be delivered to a target so long as the mouse button is held. See e.g. https://developer.mozilla.org/en/DOM/element.setCapture That API is not supported in Chrome, but it is implicitly supported at the window level. E.g. this snippit: <!DOCTYPE html> <body onload="window.onmousemove = function (e) { display.innerHTML = e.screenX; }"> Drag mouse from here to outside of the window, see the X position continue to update: <form name="display"> http://software.hixie.ch/utilities/js/live-dom-viewer/?%3C!DOCTYPE%20html%3E%0A%3Cbody%20onload%3D%22window.onmousemove%20%3D%20function%20(e)%20{%20display.innerHTML%20%3D%20e.screenX%3B%20}%22%3E%0ADrag%20mouse%20from%20here%20to%20outside%20of%20the%20window%2C%20see%20the%20X%20position%20continue%20to%20update%3A%0A%3Cform%20name%3D%22display%22%3E
,
Jun 30, 2011
Right, good point, I was thinking more of my use case than the FPS one and didn't think about relative mouse movement requirements. In that case, would it be advantageous to support an API to set a virtual mouse position, or just have developers use an absolutely positioned image/div or a canvas/WebGL sprite for their custom cursor? In traditional game engines we'd just draw it ourselves entirely but then we'd also be in control of all the UI controls entirely and not be using a host platform' native controls (HTML elements in the case of the Web platform). With HTML5 games, that's not necessarily the best case as HTML is rich enough and flexible enough to be worth using for UI controls. When interacting with HTML elements and a locked mouse there's then the whole issue of how to get events fired for the normal mousemove/over/out/click events without requiring a huge pile of developer hacks. In our particular use case, we actually do want regular positioned mouse events, as well as interaction with HTML overlays. Our primary purpose of mouse locking is just to ensure that the user doesn't accidentally click or move outside the "viewport" while aiming/firing shots or moving.
,
Jun 30, 2011
(No comment was entered for this change.)
Labels: -Mstone-X Mstone-16
,
Jul 7, 2011
We can satisfy your use case by allowing synthetic mouse events even when mouse lock is enabled. You can then hold the mouse lock and implement a cursor with HTML/DOM. You can restrict it's movement as necessary. You can still use HTML/DOM/Script for your UI by generating mouse events, which will react just as if they were non-synthetic and without mouse lock. E.g. this code would function identically under mouse lock or not: https://developer.mozilla.org/en/DOM/document.<https://developer.mozilla.org/en/DOM/document.createEvent> createEvent <https://developer.mozilla.org/en/DOM/document.createEvent> A quick example I tested with my work in progress. On a 'real' click it generates a synthetic click which it delivers to my own custom cursor's position: document.addEventListener("click", function (e) { if (e._isSynthetic) return; // send a synthetic click ee = document.createEvent("MouseEvents"); ee._isSynthetic = true; x = myCursor.x; y = myCursor.y; ee.initMouseEvent("click", true, true, null, 1, x + e.screenX - e.clientX, y + e.screenY - e.clientY, x, y); var target = document.elementFromPoint(x, y) if (target) target.dispatchEvent(ee) });
,
Jul 29, 2011
(No comment was entered for this change.)
Blocking: 41781
,
Aug 2, 2011
(No comment was entered for this change.)
Labels: -Pri-2 Pri-1
,
Aug 2, 2011
Does this mean next canary release is going to have this enabled!?
,
Aug 2, 2011
Sorry, not quite yet.
,
Aug 20, 2011
I've written a post suggesting a similar API: mouseFreeze http://blog.vjeux.com/2011/javascript/idea-mousefreeze-a-solution-for-browser-fps-games.html
,
Aug 20, 2011
vjeuxx, The mouse lock specification already covers those use-cases and more. If you look at comment 22 you'll see the current draft which is being discussed.
,
Sep 1, 2011
(No comment was entered for this change.)
Summary: Mouse Lock Javascript API
,
Sep 22, 2011
WebKit bug where I'm posting work in progress: https://bugs.webkit.org/show_bug.cgi?id=68468
,
Sep 22, 2011
(No comment was entered for this change.)
Blockedon: 41781
,
Sep 23, 2011
Design doc for UI specific notes: https://sites.google.com/a/chromium.org/dev/developers/design-documents/mouse-lock Initial implementation may not yet support Content Settings, that has been filed as a separate issue: http://code.google.com/p/chromium/issues/detail?id=97768
,
Oct 11, 2011
(No comment was entered for this change.)
Blockedon: 99869
,
Oct 17, 2011
(No comment was entered for this change.)
Blocking: -41781
,
Oct 17, 2011
(No comment was entered for this change.)
Labels: Feature-MouseLock
,
Oct 20, 2011
WebKit flags landed: http://trac.webkit.org/changeset/98016
Labels: -Mstone-16 Mstone-17
,
Nov 11, 2011
We still on track fo M17 with this?
,
Nov 13, 2011
M17 is about a 50/50
,
Nov 17, 2011
(No comment was entered for this change.)
Labels: WebKit-ID-70530 WebKit-ID-72150 WebKit-ID-72158 WebKit-ID-72286 WebKit-ID-72303 WebKit-ID-72315 WebKit-ID-72427 WebKit-ID-72431 WebKit-ID-72662 WebKit-ID-72659
,
Nov 17, 2011
(No comment was entered for this change.)
Blockedon: 97768 103982
,
Nov 17, 2011
(No comment was entered for this change.)
Labels: -WebKit-ID-70530 -WebKit-ID-72150 -WebKit-ID-72158 -WebKit-ID-72286 -WebKit-ID-72303 -WebKit-ID-72315 -WebKit-ID-72427 -WebKit-ID-72431 -WebKit-ID-72662 -WebKit-ID-72659 WebKit-ID-70530-RESOLVED WebKit-ID-72150-RESOLVED WebKit-ID-72158-RESOLVED WebKit-ID-72286-RESOLVED WebKit-ID-72303-RESOLVED WebKit-ID-72315-RESOLVED WebKit-Rev-100328 WebKit-ID-72427-RESOLVED WebKit-Rev-100366 WebKit-ID-72431-RESOLVED WebKit-Rev-100373 WebKit-ID-72662-NEW WebKit-ID-72659-NEW
,
Nov 17, 2011
The following revision refers to this bug:
http://src.chromium.org/viewvc/chrome?view=rev&revision=110632
------------------------------------------------------------------------
r110632 | scheib@chromium.org | Thu Nov 17 17:50:22 PST 2011
Changed paths:
M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/about_flags.cc?r1=110632&r2=110631&pathrev=110632
M http://src.chromium.org/viewvc/chrome/trunk/src/content/renderer/render_thread_impl.cc?r1=110632&r2=110631&pathrev=110632
M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/app/generated_resources.grd?r1=110632&r2=110631&pathrev=110632
M http://src.chromium.org/viewvc/chrome/trunk/src/content/browser/renderer_host/render_process_host_impl.cc?r1=110632&r2=110631&pathrev=110632
M http://src.chromium.org/viewvc/chrome/trunk/src/content/public/common/content_switches.h?r1=110632&r2=110631&pathrev=110632
M http://src.chromium.org/viewvc/chrome/trunk/src/content/public/common/content_switches.cc?r1=110632&r2=110631&pathrev=110632
Pointer lock added to about:flags and passed to WebKit runtime flags
BUG=72754
Review URL: http://codereview.chromium.org/8536003
------------------------------------------------------------------------
,
Nov 17, 2011
(No comment was entered for this change.)
Labels: -WebKit-ID-72662-NEW WebKit-ID-72662-RESOLVED WebKit-Rev-100715
,
Nov 23, 2011
(No comment was entered for this change.)
Labels: WebKit-ID-73031
,
Nov 23, 2011
(No comment was entered for this change.)
Blockedon: 105318
,
Nov 28, 2011
http://trac.webkit.org/changeset/100328http://trac.webkit.org/changeset/100366http://trac.webkit.org/changeset/100373https://bugs.webkit.org/show_bug.cgi?id=72659 http://trac.webkit.org/changeset/100715https://bugs.webkit.org/show_bug.cgi?id=73031 http://trac.webkit.org/changeset/101111
Labels: -WebKit-ID-72659-NEW -WebKit-ID-73031 WebKit-ID-72659-RESOLVED WebKit-ID-73031-RESOLVED WebKit-Rev-101111
,
Dec 9, 2011
(No comment was entered for this change.)
Labels: -Mstone-17 Mstone-18
,
Dec 16, 2011
Is this currently working in one of the dev channels?
,
Dec 16, 2011
Not quite yet, but coming!
,
Dec 16, 2011
Anxiously awaiting to test it!
,
Dec 18, 2011
Mouse Lock API and settings UI available in 17.0.963.12 dev after enabling pointer lock in about:flags, but the interface ignores my calls, it seems. Are there some examples available somewhere that I could test this on? Perhaps my code is buggy.
,
Dec 19, 2011
So the flag doesn't add JavaScript support, yet. Right now the implementation is only in NaCL. The JS APIs will be landing shortly, and once they do we'll publish some examples. For now you'll just have to sit tight, sorry :(
,
Dec 19, 2011
It'll be coming soon. I've a prototype running with both NaCl and JavaScript bindings on my workstation, and will start landing in early January both Chrome and WebKit side patches. I'll need to build up API tests as I go on the WebKit side, Chrome will stay manual functional tests till I get it landed behind a flag. I'll make it clear when it is available in a Canary build for people to easily start experimenting with. Thank you for your interest! I should also point out that a patch for FireFox has been developed and may have test builds available: author's blog: http://vocamus.net/dave/?cat=28 and the bug: https://bugzilla.mozilla.org/show_bug.cgi?id=633602
,
Jan 6, 2012
(No comment was entered for this change.)
Blockedon: 109471
,
Jan 9, 2012
(No comment was entered for this change.)
Labels: -WebKit-ID-70530-RESOLVED -WebKit-ID-72150-RESOLVED -WebKit-ID-72158-RESOLVED -WebKit-ID-72286-RESOLVED -WebKit-ID-72303-RESOLVED -WebKit-ID-72315-RESOLVED -WebKit-ID-72427-RESOLVED -WebKit-ID-72431-RESOLVED -WebKit-ID-72662-RESOLVED -WebKit-ID-72164-RESOLVED -WebKit-ID-72659-RESOLVED -WebKit-ID-73031-RESOLVED WebKit-ID-75762
,
Jan 9, 2012
https://bugs.webkit.org/show_bug.cgi?id=75762
Labels: -WebKit-ID-75762 WebKit-ID-75762-NEW
,
Jan 9, 2012
(No comment was entered for this change.)
Labels: GamesRoadmapped
,
Jan 11, 2012
Basic functionality will be available behind an about:flags in the canary after these patches are reviewed and landed: https://bugs.webkit.org/show_bug.cgi?id=75762 http://code.google.com/p/chromium/issues/detail?id=109957
Blockedon: 109957
,
Jan 27, 2012
https://bugs.webkit.org/show_bug.cgi?id=75762
Labels: -WebKit-ID-75762-NEW WebKit-ID-75762-RESOLVED
,
Jan 27, 2012
Chromium http://src.chromium.org/viewvc/chrome?view=rev&revision=119406 and WebKit http://trac.webkit.org/changeset/106134 just landed, (webkit needs time to pass builders and then be rolled in DEPs). Developers will be able to experiment with mouse lock in the next Chrome Canary.
,
Jan 27, 2012
Do you know how much till next canary release?
,
Jan 27, 2012
Canaries (Win & Mac) roll daily: http://dev.chromium.org/getting-involved/dev-channel For Linux see continuous builds: http://commondatastorage.googleapis.com/chromium-browser-continuous/index.html (must wait for WebKit to roll and build - maybe late today).
,
Jan 29, 2012
Tried out the new Canary build this morning. Looks great, but I ran into two issues: 1) On the Mac build, trying to open preferences crashes the browser every time. Probably not directly related, but does make #2 much more difficult. I've logged a separate issue for this: Issue 111856 2) If you have visited a mouselock-enabled site and given it fullscreen privileges prior to this build it will no not show you the permissions prompt again with the new Canary build, which effectively means that you can never allow mouselock on that site. I had to clear my browser cache/history before it would prompt me properly. (hence why #1 makes life difficult) If this is more appropriate as a separate issue I'd be happy to log it as such. (Both of these tested on OSX)
,
Jan 29, 2012
To track it here, on OSX (not repro on Windows, untested on Linux): If a fullscreen tab has locked the mouse cursor, closing the tab with command-w will not release the mouse - only closing the browser will make the mouse visible again. Also hit Tojiro's previously-fullscreen enabled site bug. I just thought I was crazy. The wording on the dialog is a bit odd - maybe 'has hidden your mouse cursor' instead of 'disabled'? Awesome work!
,
Jan 29, 2012
ben.vanik: Can you file a new bug for the OS X issue please?
,
Jan 29, 2012
Filed as Issue 111860
,
Feb 7, 2012
Spec has been updated to use the API paradigm from Fullscreen for entering / exiting the lock state: http://dvcs.w3.org/hg/pointerlock/raw-file/default/index.html
,
Feb 8, 2012
(No comment was entered for this change.)
Labels: -Feature-MouseLock Feature-Input-MouseLock
,
Mar 21, 2012
Updating to a future milestone for getting mouse lock (pointer lock) out from behind a flag.
Labels: -Pri-1 -Mstone-18 Pri-2 Mstone-20
,
May 1, 2012
I am going to land http://code.google.com/p/chromium/issues/detail?id=72754 72754 - Mouse Lock Javascript API first. Moving to M21.
Labels: -Mstone-20 Mstone-21
|
||||||||
| ► Sign in to add a comment | |||||||||