HoshinoKoji/jsPsych
1
0
Fork 0
mirror of https://github.com/jspsych/jsPsych.git synced 2025-05-10 11:10:54 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

allow recording of mouseup and mousedown

Browse Source
This commit is contained in:
Josh de Leeuw 2021-10-14 15:25:48 -04:00
parent dac7be71f7
commit 0e0d884a1e
4 changed files with 202 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.changeset/ten-owls-talk.md
View File

@ -2,4 +2,4 @@
"@jspsych/extension-mouse-tracking": major
---
Created an extension that enables mouse tracking. The extension records the coordinates and time of mouse movements, as well as optionally recording the coordinates of objects on the screen to enable mapping of mouse movements onto screen objects.
Created an extension that enables mouse tracking. The extension records the coordinates and time of mousemove, mousedown, and mouseup events, as well as optionally recording the coordinates of objects on the screen to enable mapping of mouse events onto screen objects.

7
docs/extensions/mouse-tracking.md
View File

@ -1,7 +1,7 @@
# mouse-tracking
This extension supports mouse tracking.
Specifically, it can record the `x` `y` coordinates and time of [mousemove events](https://developer.mozilla.org/en-US/docs/Web/API/Element/mousemove_event).
Specifically, it can record the `x` `y` coordinates and time of [mousemove events](https://developer.mozilla.org/en-US/docs/Web/API/Element/mousemove_event), [mousedown events](https://developer.mozilla.org/en-US/docs/Web/API/Element/mousedown_event), and [mouseup events](https://developer.mozilla.org/en-US/docs/Web/API/Element/mouseup_event).
It also allows recording of the [bounding rectangle](https://developer.mozilla.org/en-US/docs/Web/API/Element/getBoundingClientRect) of elements on the screen to support the calculation of mouse events relative to different elements.
## Parameters
@ -20,7 +20,7 @@ initJsPsych({
Parameter | Type | Default Value | Description
----------|------|---------------|------------
minimum_sample_time | number | 0 | The minimum time between mouse samples. If mouse events occur more rapidly than this limit, they will not be recorded. Use this if you want to keep the data files smaller and don't need high resolution tracking data. The default value of 0 means that all events will be recorded.
minimum_sample_time | number | 0 | The minimum time between samples for `mousemove` events. If `mousemove` events occur more rapidly than this limit, they will not be recorded. Use this if you want to keep the data files smaller and don't need high resolution tracking data. The default value of 0 means that all events will be recorded.
### Trial Parameters
@ -38,12 +38,13 @@ var trial = {
Parameter | Type | Default Value | Description
----------|------|---------------|------------
targets | array | [] | A list of elements on the page that you would like to record the coordinates of for comparison with the WebGazer data. Each entry in the array should be a valid [CSS selector string](https://www.w3schools.com/cssref/css_selectors.asp) that identifies the element. The selector string should be valid for exactly one element on the page. If the selector is valid for more than one element then only the first matching element will be recorded.
events | array | ['mousemove'] | A list of events to track. Can include 'mousemove', 'mousedown', and 'mouseup'.
## Data Generated
Name | Type | Value
-----|------|------
mouse_tracking_data | array | An array of objects containing mouse movement data for the trial. Each object has an `x`, a `y`, and a `t` property. The `x` and `y` properties specify the mouse coordinates in pixels relative to the top left corner of the viewport and `t` specifies the time in milliseconds since the start of the trial.
mouse_tracking_data | array | An array of objects containing mouse movement data for the trial. Each object has an `x`, a `y`, a `t`, and an `event` property. The `x` and `y` properties specify the mouse coordinates in pixels relative to the top left corner of the viewport and `t` specifies the time in milliseconds since the start of the trial. The `event` will be either 'mousemove', 'mousedown', or 'mouseup' depending on which event was generated.
mouse_tracking_targets | object | An object contain the pixel coordinates of elements on the screen specified by the `.targets` parameter. Each key in this object will be a `selector` property, containing the CSS selector string used to find the element. The object corresponding to each key will contain `x` and `y` properties specifying the top-left corner of the object, `width` and `height` values, plus `top`, `bottom`, `left`, and `right` parameters which specify the [bounding rectangle](https://developer.mozilla.org/en-US/docs/Web/API/Element/getBoundingClientRect) of the element.
## Examples

147
packages/extension-mouse-tracking/src/index.spec.ts
View File

@ -1,5 +1,5 @@
import htmlKeyboardResponse from "@jspsych/plugin-html-keyboard-response";
import { clickTarget, mouseMove, pressKey, startTimeline } from "@jspsych/test-utils";
import { mouseDown, mouseMove, mouseUp, pressKey, startTimeline } from "@jspsych/test-utils";
import { initJsPsych } from "jspsych";
import MouseTrackingExtension from ".";
@ -38,14 +38,159 @@ describe("Mouse Tracking Extension", () => {
expect(getData().values()[0].mouse_tracking_data[0]).toMatchObject({
x: targetRect.x + 50,
y: targetRect.y + 50,
event: "mousemove",
});
expect(getData().values()[0].mouse_tracking_data[1]).toMatchObject({
x: targetRect.x + 55,
y: targetRect.y + 50,
event: "mousemove",
});
expect(getData().values()[0].mouse_tracking_data[2]).toMatchObject({
x: targetRect.x + 60,
y: targetRect.y + 50,
event: "mousemove",
});
});
test("adds mouse down data to trial", async () => {
const jsPsych = initJsPsych({
extensions: [{ type: MouseTrackingExtension }],
});
const timeline = [
{
type: htmlKeyboardResponse,
stimulus: "<div id='target' style='width:500px; height: 500px;'></div>",
extensions: [
{
type: MouseTrackingExtension,
params: { events: ["mousedown"] },
},
],
},
];
const { displayElement, getHTML, getData, expectFinished } = await startTimeline(
timeline,
jsPsych
);
const targetRect = displayElement.querySelector("#target").getBoundingClientRect();
mouseDown(50, 50, displayElement.querySelector("#target"));
mouseDown(55, 50, displayElement.querySelector("#target"));
mouseDown(60, 50, displayElement.querySelector("#target"));
pressKey("a");
await expectFinished();
expect(getData().values()[0].mouse_tracking_data[0]).toMatchObject({
x: targetRect.x + 50,
y: targetRect.y + 50,
event: "mousedown",
});
expect(getData().values()[0].mouse_tracking_data[1]).toMatchObject({
x: targetRect.x + 55,
y: targetRect.y + 50,
event: "mousedown",
});
expect(getData().values()[0].mouse_tracking_data[2]).toMatchObject({
x: targetRect.x + 60,
y: targetRect.y + 50,
event: "mousedown",
});
});
test("adds mouse up data to trial", async () => {
const jsPsych = initJsPsych({
extensions: [{ type: MouseTrackingExtension }],
});
const timeline = [
{
type: htmlKeyboardResponse,
stimulus: "<div id='target' style='width:500px; height: 500px;'></div>",
extensions: [
{
type: MouseTrackingExtension,
params: { events: ["mouseup"] },
},
],
},
];
const { displayElement, getHTML, getData, expectFinished } = await startTimeline(
timeline,
jsPsych
);
const targetRect = displayElement.querySelector("#target").getBoundingClientRect();
mouseUp(50, 50, displayElement.querySelector("#target"));
mouseUp(55, 50, displayElement.querySelector("#target"));
mouseUp(60, 50, displayElement.querySelector("#target"));
pressKey("a");
await expectFinished();
expect(getData().values()[0].mouse_tracking_data[0]).toMatchObject({
x: targetRect.x + 50,
y: targetRect.y + 50,
event: "mouseup",
});
expect(getData().values()[0].mouse_tracking_data[1]).toMatchObject({
x: targetRect.x + 55,
y: targetRect.y + 50,
event: "mouseup",
});
expect(getData().values()[0].mouse_tracking_data[2]).toMatchObject({
x: targetRect.x + 60,
y: targetRect.y + 50,
event: "mouseup",
});
});
test("ignores mousemove when not in events", async () => {
const jsPsych = initJsPsych({
extensions: [{ type: MouseTrackingExtension }],
});
const timeline = [
{
type: htmlKeyboardResponse,
stimulus: "<div id='target' style='width:500px; height: 500px;'></div>",
extensions: [
{
type: MouseTrackingExtension,
params: { events: ["mousedown"] },
},
],
},
];
const { displayElement, getHTML, getData, expectFinished } = await startTimeline(
timeline,
jsPsych
);
const targetRect = displayElement.querySelector("#target").getBoundingClientRect();
mouseMove(50, 50, displayElement.querySelector("#target"));
mouseMove(55, 50, displayElement.querySelector("#target"));
mouseDown(60, 50, displayElement.querySelector("#target"));
pressKey("a");
await expectFinished();
expect(getData().values()[0].mouse_tracking_data.length).toBe(1);
expect(getData().values()[0].mouse_tracking_data[0]).toMatchObject({
x: targetRect.x + 60,
y: targetRect.y + 50,
event: "mousedown",
});
});

57
packages/extension-mouse-tracking/src/index.ts
View File

@ -15,7 +15,12 @@ interface OnStartParameters {
* An array of string selectors. The selectors should identify one unique element on the page.
* The DOMRect of the element will be stored in the data.
*/
targets: Array<string>;
targets?: Array<string>;
/**
* An array of mouse events to track. Can include `"mousemove"`, `"mousedown"`, and `"mouseup"`.
* @default ['mousemove']
*/
events?: Array<string>;
}
class MouseTrackingExtension implements JsPsychExtension {
@ -32,6 +37,7 @@ class MouseTrackingExtension implements JsPsychExtension {
private currentTrialStartTime: number;
private minimumSampleTime: number;
private lastSampleTime: number;
private eventsToTrack: Array<string>;
initialize = ({ minimum_sample_time = 0 }: InitializeParameters): Promise<void> => {
this.domObserver = new MutationObserver(this.mutationObserverCallback);
@ -43,10 +49,13 @@ class MouseTrackingExtension implements JsPsychExtension {
};
on_start = (params: OnStartParameters): void => {
params = params || {};
this.currentTrialData = [];
this.currentTrialTargets = new Map();
this.currentTrialSelectors = typeof params !== "undefined" ? params.targets : [];
this.currentTrialSelectors = params.targets || [];
this.lastSampleTime = null;
this.eventsToTrack = params.events || ["mousemove"];
this.domObserver.observe(this.jsPsych.getDisplayElement(), { childList: true });
};
@ -56,13 +65,29 @@ class MouseTrackingExtension implements JsPsychExtension {
this.currentTrialStartTime = performance.now();
// start data collection
window.addEventListener("mousemove", this.mouseEventHandler);
if (this.eventsToTrack.includes("mousemove")) {
window.addEventListener("mousemove", this.mouseMoveEventHandler);
}
if (this.eventsToTrack.includes("mousedown")) {
window.addEventListener("mousedown", this.mouseDownEventHandler);
}
if (this.eventsToTrack.includes("mouseup")) {
window.addEventListener("mouseup", this.mouseUpEventHandler);
}
};
on_finish = () => {
this.domObserver.disconnect();
window.removeEventListener("mousemove", this.mouseEventHandler);
if (this.eventsToTrack.includes("mousemove")) {
window.removeEventListener("mousemove", this.mouseMoveEventHandler);
}
if (this.eventsToTrack.includes("mousedown")) {
window.removeEventListener("mousedown", this.mouseDownEventHandler);
}
if (this.eventsToTrack.includes("mouseup")) {
window.removeEventListener("mouseup", this.mouseUpEventHandler);
}
return {
mouse_tracking_data: this.currentTrialData,
@ -70,7 +95,7 @@ class MouseTrackingExtension implements JsPsychExtension {
};
};
private mouseEventHandler = (e) => {
private mouseMoveEventHandler = (e) => {
const x = e.clientX;
const y = e.clientY;
@ -82,10 +107,30 @@ class MouseTrackingExtension implements JsPsychExtension {
event_time - this.lastSampleTime >= this.minimumSampleTime
) {
this.lastSampleTime = event_time;
this.currentTrialData.push({ x, y, t });
this.currentTrialData.push({ x, y, t, event: "mousemove" });
}
};
private mouseUpEventHandler = (e) => {
const x = e.clientX;
const y = e.clientY;
const event_time = performance.now();
const t = Math.round(event_time - this.currentTrialStartTime);
this.currentTrialData.push({ x, y, t, event: "mouseup" });
};
private mouseDownEventHandler = (e) => {
const x = e.clientX;
const y = e.clientY;
const event_time = performance.now();
const t = Math.round(event_time - this.currentTrialStartTime);
this.currentTrialData.push({ x, y, t, event: "mousedown" });
};
private mutationObserverCallback = (mutationsList, observer) => {
for (const selector of this.currentTrialSelectors) {
if (!this.currentTrialTargets[selector]) {