1. Introduction |
A group of
20 students are assigned a homework problem once a week for fifteen weeks. Students
can group and regroup as they wish in order to complete their homeworks. Which
parameters will become crucial in their interactions, and what dynamics will
emerge?
In
this paper we propose a computational analysis of this scenario grounded on the
use of Brahms - an agent-oriented, activity-based language developed to “model
knowledge in situated action and learning in human activities” (Sierhuis and
Clancey [1997]). In what follows, we
define a model of students’ motivations and behaviors; we then simulate
students’ activities as they form and reform different groups week after week
in the course of completing their assignments according to those
motivations. The setup of the model,
together with the underlying assumptions and the solution algorithm on which is
based, are discussed first. Then, we discuss the methodology that we adopted
for the scenario, and present an overview of its actual implementation and
initial results. While the results are preliminary, the model’s flexibility
allows us to discuss various computational experiments and social scientific
applications that would extend the scenario and its implications.
2. Model setup |
How to
model the interaction of students who are free to group and regroup week after
week in order to complete homework assignments? We approached this problem
through an agent-based computational model, where students are agents defined
by the ability and effort spent in completing team homework. Each team is made of n students. N
can vary across different runs of the scenario but is fixed throughout the
fifteen iterations (weeks) of a single run.
Along with the agents and their teams, a ‘Professor’ acts as a
meta-agent that grades the homework and facilitates the matching between agents
and teams, hence regulating the formation and dissolution of teams through the
rounds.
3. Model Structure |
Each student i is defined by
a combination of two parameters: an ability a and an effort e. Both parameters are distributed randomly for
each student:
Each student i is a member of
a team s at a time t:
The grade g for a team’s home
work is assigned by the ‘Professor’ on the basis of a function g(.) over
the function f(.) of all team members’ performances. For example, g(.) could be a simple
average of f(.), which would be on its turn a sum of the arguments of a
students’ performances, given by combinations of ability and effort. The interaction of these two parameters
within the f(.) function presents some interesting modeling alternatives
that will be explored further in the paper.
At the end
of the week, each student receives the same grade as that of the team she was
member of during that week. Students receive utility over time from the grades
they get and from the free time they enjoy, which is inversely proportional to
their effort levels. Hence, students try to maximize:
and where alpha and beta
are
weights.
After grades are
made public, some students will be dropped by their teams and some others will
voluntarily leave their team. A process of new matching is facilitated by the
‘Professor’ and implemented through an algorithm discussed further below.
4. Model assumptions |
The values
of each agent’s ability and willingness to work (effort) are fixed. These
parameters are private information, even though the performance f that
results from them is known to all members inside the team. The only public
information available outside the team is the grade of each agent (and team) at
the end of each round. However, there is (initially) no learning: students who
happened to be in the same teams do not make assumptions on whether the effort
and ability levels of their colleagues have remained the same or varied over
time.
We restrict
the analysis to cases where the number of components of each group is fixed. We
consider the following explicit forms for g and f:
This
grade is calculated at the end of each round and a cumulative grade is
maintained for each agent that is updated through each round.
5. The algorithm |
Our
computational approach implements the following steps for the game:
6. Hypotheses |
7. Methodology
and preliminary results |
The above
model structure and algorithm have been implemented in Brahms, an
agent-oriented, activity-based language developed to “model knowledge in
situated action and learning in human activities” (Sierhuis and Clancey
[1997]). Brahms can be used to simulate the behavior of people and artifacts
when collaborating together.
Brahms
links knowledge-based models of cognition with discrete simulation and the
behavior-based subsumption architecture. In Brahms, agents’ behaviors are
organized into activities, inherited from the groups they belong to. Activities
are located in time and space – therefore issues like resource availability and
human participation are considered.
More
specifically, the architecture includes the following constructs (cf. Sierhuis
and Clancey [1997]):
Groups
of groups containing
Agents who are located and have
Beliefs that lead them to engage
in
Activities
that are specified by
Workframes that
consist of
Preconditions
of beliefs that lead to
Actions,
consisting of
Communication
Actions
Movement
actions
Primitive
Actions
Other
composite activities
Consequences
of new beliefs and facts
Thoughtframes that
consist of
Preconditions
and
Consequences
Brahms is used at NASA to analyze existing work practices and design models of new work systems. In our scenario, we design the Students and the Professor as agents with goals, attributes and beliefs. For instance, a Professor might grade a homework and then communicate the grade to the team’s members with the following (highly stylized) construct:
workframe
wf_completePayoffTeamA {
repeat: true;
variables:
foreach(Student)
std;
when(knownval(std ismemberof Team))
do {
CompletePayoffTeam();
conclude((Team.latest_team_score =
current.tempscoreTeam / n), bc:100, fc:100);
AnnounceGrades(std);
}
}
A view of
one of the simulations we run is presented in the following Figure:
where students are sending in
their homeworks (through communication lines) and the Professor (on the top
line) is returning the grades. Analysis of the numerical outputs (see next
Figure) currently shows a quick adjustment to an equilibrium where groups become
homogeneous in terms of the performance of their members. No cyclical or
chaotic dynamics emerge. These results
are ongoing as weight changes and other computational experiments (discussed
below) are also being performed.
8. Computational experiments |
We
identified four dimensions along which the model can be varied in order to
compare different computation experiments. Some of these variations are
implicit in the parameters’ structure we have proposed. Some require extensions
to the model.
A.
Who is the decision maker: the agent or the team?
B.
How is the exit option determined? Is it based on the observed performance or
on the level of effort?
C.
What is each agent’s memory process? Do agents remember the revealed abilities
and effort levels of agents with whom they have worked in the past, to what
extent is their knowledge imperfect and for how long do they retain this
information?
D.
To what extent can agents alter their initial parameters of ability and effort?
Can they become for example harder workers over time?
We
comment below on each of the above 4 points.
In
the current model the decision process about who will exit the team and in what
way is very team-centric: we assume a simple metric, where those whose performance
is below the team average will be asked to leave the team. This approach can be significantly adjusted
in two directions:
1. Leaving
the decision to the team, but introducing more complex decision procedures. For
example, in each team a voting process at the end of the round would determine
by majority ruling the individual(s) to be forced out. The team would then be
endowed with a performance measure of its own, which it will try to
maximize. If we were also willing to
adjust the assumption of fixed team size, the team then would reorganize
according to its ideal composition and size. Uncertainty would be determined by
the fact that the team’s expected payoff would be a weighted combination of the
signals it receives from its members, but the information about actual ability
and willingness to work remains private.
2.
In a different approach, the decision makers could be the students
themselves. This would be the most fluid and complex form of the model along this
dimension. Each agent would maintain an array of beliefs about the expected
performance of every other agent in the model.
He would then try to maximize her expected payoff from all possible
combinations of a fixed team size within the group. She will then go about
attempting to form these groups.
B. Exit Rule
The
exit rule has several interesting implications for the model. It is possible to introduce learning in this
model by making public the information about the effort levels. By setting the exit rule such that those
with a level of effort below a certain team effort threshold are evicted,
interesting dynamics emerge should emerge (this is further discussed in the
final case). An interesting case here is that of the “weakest link” exit rules,
where teams vote off the people that they believe are contributing the
least. However, instead of measuring
contribution on the basis of each agent’s own performance only (), it
would be interesting to consider interactions such as:, that
include the abilities of all other agents in the team. This idea of
complementarity might cause people to be voted off the team simply because
their levels of effort and ability do not successfully enhance those of others
on the team.
The
case where agents have the ability to remember the features of agents that they
have interacted with in the last n rounds complements the scenario in
which agents become the primary decision makers. In this case, we might let the agents learn each other’s effort
levels with certainty but receive noisy signals about abilities. Depending on the weights given to the
parameters, incentives might emerge for the hardest working members to rapidly
select into stable teams with an above average but consistent payoff. If memory lengths could have varying
degrees, those with a longer memory may be able to do better in identifying and
forming successful teams across the time periods.
The
final case follows from the previous discussion. Given memory and agent based decision making, agents might be
allowed to alter their behavior and become harder working as they become more
aware of the benefits of being identified as such over time. On the other hand,
based on the weights given to the parameters, we may find overall average
grades declining across teams as harder working but less smart people start to
dominate the space. This represents an
interesting application of the principal/agent problem and the effort to form
incentive compatible contracts. In this
case, if the principal observes only face effort with certainty and
intelligence with wider margins of error, then he may be forced to use some
inadequate metric such as face time to determine who should be kept and who
should be removed from a team. This may shift the group toward hard working but
not necessarily smarter members, and hence a lower than potential average
payoff for the whole team.
9. Conclusions |
10. References |
Sierhuis, M. and W. Clancey (1997). “Knowledge,
Practice, Activities and People”, Presented at the AAAI Spring Symposium
Artificial Intelligence in Knowledge Management Stanford University, March
24-26, 1997.