Player Performance in Portal 2: Impact of communication type, behavior displays, and experience level
A project proposal was conducted as a requirement for a culminating project in ISE 212 (Human Factors Experiments). A copy of the report is available upon request.
Background and Goals
The primary goal of this proposed study is to determine how different in-game communication methods, experience levels, and behavior types affect an individual’s performance while playing team-based online games.
Hypothesis
This proposal hypothesized there is a difference in player performance between beginner, intermediate, and expert players when experiencing positive or negative behaviors displayed through voice chat, text chat, or non-verbal ping communication systems in team-based video games.
Process
A Literature Review was conducted to outline common video game genres, describe the different communication methods, and cover findings from similar studies on communication methods’ effects on player enjoyment, distraction, and performance.
​
The Methods and Procedure section explains the proposed experimental design of this study including independent and dependent variables, apparatus used in the study, procedures, and the analysis of data.
Experimental Design
A 3x3x2 between-subjects design was proposed conducted to reduce order and learning effects. This design included:
-
Independent Variables:
-
communication methods (voice, text, and CM ping)
-
experience level (inexperienced, intermediate, and experienced)
-
language behavior type (positive and negative)
-
-
Dependent Variables:
-
completion time
-
the number of unique communication interactions
-
the number of positive and negative behavior displays
-
​
To create a balanced study, participants will be randomly paired with another participant of the same experience level and randomly assigned to one of three communication method conditions.
​
163 participants would be placed into nine pairs and be assigned to one of 9 groups, for a total of 27 groups (45 participants) per communication method and experience level condition:
​
​
Procedure
Participants would complete three randomly ordered custom build test chambers with different puzzle components created by the research team in the video game Portal 2:
-
Each test chamber is designed with unique puzzle mechanics to reduce learning effects
-
Order of the test chambers will be randomly assigned to prevent order effects
-
To prevent the use of communication outside of the assigned method, all other methods of communication will be removed from the session and participants will be seated in separate rooms
​
Each Test Chamber was designed to elicit communication and coordination between participants:
-
Test Chamber 1 requires one participant to protect their partner as they create bridges to reach the end of the map
-
Test Chamber 2 required participants to synchronize their movements and enter portals simultaneously
-
Test Chamber 3 requires participants to coordinate portal placements while quickly moving on the ground and in the air
Data and Analysis
Data collected would include:
-
Participants’ completion time of milestones in each chamber
-
Total completion time for each trial
-
Number of unique communication interactions
-
Number of positive and negative behavior displays
​
Analysis of data will be conducted after the conclusion of the study:
-
Three one-way ANOVAs will be used to separately analyze the main effects
-
A 3x3x2 factorial ANOVA will also be conducted to analyze the combined effects
-
If statistically significant results are obtained, a Tukey post-hoc analysis will be run to identify which group of conditions are significantly different from other condition
My learnings
What I learned from this project:
-
How to create a DOE and a research proposal that thoroughly outlines all steps of the research process
-
How to use G*Power to calculate the number of participants required at the chosen statistical power
-
The importance of narrowing down the scope of research to make a feasible study in which the results can aid in future research or be directly applied to the development of a product increase usability