Collective Intelligence Applications

Dawn G. Gregg*
dawn.gregg@cudenver.edu

A collective intelligence application is one that harnesses the knowledge and work of its

users to provide the data for the application and to improve its usefulness. The most hyped

examples of collective intelligence applications have been labeled as “Web 2.0” applications.

Web 2.0 is an amorphous term used to define a computing paradigm that uses the Web as the

application platform and facilitates collaboration and information sharing between users [7].

Classic examples of Web 2.0 applications include: wikis, blogs (or weblogs), social network

services, and social bookmarking [2]. Web 2.0 sites are database-driven and are considered to be

“infoware,” in that, they are data intensive and the more data they contain the more valuable they

become [6].

Collective intelligence is a fundamentally different way of viewing how applications can

support human interaction and decision making. Most pre-Web 2.0 applications have focused in

improving the productivity or decision making of the individual user. The emphasis has been on

providing the tools and data necessary to fulfill a specific job function. Under the collective

intelligence paradigm, the focus is on harnessing the intelligence of groups of people to enable

greater productivity and better decisions than are possible by individuals working in isolation.

This suggests that software developers need to have different ways of thinking about how

their how software might be used and what features would enable better visualization and use of

information among groups of people. The new breed of collective intelligence applications

needs to center around user defined data that can be reused to support decision making, team

*
Correspondence should be addressed to: Dawn Gregg, University of Colorado Denver and Health Sciences Center,
The Business School, Campus Box 165, P.O. Box 173364, Denver, CO, 80217-3364

1
building, or to improve understanding of the world around us. The users of these systems should

play a central role in defining what data is important and how the data is used. The essential

features of collective intelligence applications are similar to the design patterns for Web 2.0

applications except that collective intelligence applications can be custom applications designed

for small highly specialized domains instead of the larger Web audience served by most Web 2.0

applications (see sidebar) [3, 7].

The processes involved in designing and implementing specialized collective intelligence

applications are discussed below in the context of DDTrac, a web-based application that allows

for the easy collection and summary of special education data.

The DDTrac Application

To illustrate how features of collective intelligence systems can be combined and used in

special purpose applications, a system that combines distributed data capture, commenting,

tagging, a wiki and a blog was created. DDTrac is an information system designed to collect and

summarize information that is used to improve decisions related to education and therapy options

for special needs children. The heart of DDTrac is a web-based application that allows for the

easy collection and summary of special education data. Figure 1 shows the DDTrac system

architecture follows a basic hierarchical structure with functionality grouped into four major

areas: data entry, creating goals and objectives, data analysis, and administration. The program is

designed to allow data input from a Mac, a PC or a handheld device with a connection to the

Internet.

2
 Figure 1: DDTrac Architecture

The special education domain was selected for this study for three reasons.

• First, special education students often have many people involved in their education and

therapy team. Students can receive services in school, at home, in clinical outpatient

environments and from consultants from other regions of the country. These

practitioners rarely have time to communicate details related to student progress which

can lead to an inefficient duplication of effort, gaps in treatment, or team members

working towards conflicting goals.

• Second, special education literature emphasizes the importance of using data collection

and analysis procedures to monitor academic, social and behavior progress of students

with intensive special education needs [e.g. 1, 4, 5]. However, currently there are no

applications that support the data collection and analysis needs of this sector.

• Third, there is the opportunity to harness the collective intelligence of these practitioners

3
 to identify patterns of behavior and best practices, for both individual students as well as

groups of students with similar disabilities.

This suggests a need for software applications that can simplify the data collection and analysis

activities of special education practitioners and harness their collective intelligence to improve

their ability to make decisions about the children they serve.

The DDTrac system serves two primary purposes. First it serves as a communication

medium for therapists and teachers so that they know what to do when they sit down to work

with the special needs child. Second it collects data and provides data analysis tools to enhance

the ability to assess the adequacy of student progress and determine whether and when

instructional adjustments are necessary. The DDTrac system, shown in Figure 1, consists of four

functional areas:

• The data entry section allows four different types of data entry: instructional
objective and target data, observed social data, behavior data and narrative comments.

• The goals and objectives section allows the creation and maintenance of long-term
goals tailored to the needs of the individual student and shorter term objectives that

define the activities involved in the day to day treatment and education of the child.

• The data analysis section includes reporting and charting. These features make it easy
for special education teachers and therapists to examine student progress and modify

student's objectives and targets to maximize a student's learning outcomes.

• The administration section contains additional functionality that was included to meet
other identified needs (e.g. to manage access to data).

Distributed Data Entry. Special education instruction frequently occurs in locations

throughout a school (e.g. the special education classroom, specialized therapy rooms, the regular

4
education classroom, in the gym or on the playground) and can also occur at home and in the

community. Similar to traditional Web 2.0 applications, DDTrac uses the Web as the application

platform which allows data entry forms customized to the individual student to be accessed from

anywhere there is an internet connection. DDTrac users can choose to take data in two different

ways: either online using a wired PC or any wireless device OR offline, using a downloadable

web form which can be uploaded later when a network is available. Figure 2 shows a series of

data collection screens as would be displayed on a standard PC.

Figure 2: DDTrac Instructional Data Entry1

Another way applications can support distributed data entry is by supporting a wide

variety of data input devices. Using the web as a delivery platform is one way to support

multiple platforms. However, to support handheld devices in addition to traditional PCs,

DDTrac also uses style sheets targeted at different devices (e.g. mobile devices) and has data

1
All screenshots shown in this paper are for a fictional student “Daniel Doo”

5
entry screens designed for devices as small as 320 pixels wide. In addition, the quantitative data

entry is all accomplished using standard HTML form elements and were optimized for use with a

stylus, the common data entry tool on many handheld devices.

If the user does not have access to the Internet OR if the user has a poor/slow Internet

connection they can also use offline data entry. In offline data entry mode all of the data entry

forms for the selected students are downloaded into a single long page that has links to allow the

user to quickly navigate to specific locations within the page. The data entered in the offline data

entry page is automatically saved to cookies every 2 minutes and remains saved until the data is

successfully uploaded. If the user accidentally closes the page they just redownload the offline

data entry page and the page will refill with the data stored in the cookie.

Student-Centric Blog. Qualitative data is an important part of the information exchanged

in many collective intelligence environments. For example, special education teachers use “back

and forth books” to communicate daily notes to parents and parents write back to teachers about

events at home that might impact learning in the special education classroom. In Applied

Behavior Analysis2 (ABA) programs daily communication needs are met through handwritten

daily notes to parents and other practitioners.

Qualitative data in special education programs is frequently the only way to capture the

complexity and the transactional interaction between the setting and the student’s performance or

behavior [8]. DDTrac allows the capture of observation notes related to instructional activities,

during social interactions and following behavior episodes. These observation notes are stored in

2
Applied Behavioral Analysis is an approach to teaching behaviors and cognitive skills to children with autism and
other developmental disabilities that uses careful monitoring and positive reinforcement or prompting to teach each
step of a skill. Data collection typically consists of a designation as to whether a response is correct, incorrect,
correct but prompted or if no response was given. Qualitative notes are also taken to communicate major difficulties
or successes [5].

6
a student-centric blog that includes the date and time they were recorded, the name of the

practitioner making the comment, and the name of the student the comment is being made about.

The descriptive notes represent the practitioner's best efforts to record what is occurring in the

context of the therapy session (e.g. describe student mood and overall performance on tasks).

The qualitative observations can also include the practitioner’s interpretation of how or why

certain behaviors unfolded as they did [8].

DDTrac automatically displays the five most recent days of observations about a

particular student as soon as the practitioner selects a student to work with. These qualitative

notes are an important mechanism for communicating recent changes in the child and in the

child’s educational programs between distributed team members.

Commenting. Similar to traditional blogs, the commenting feature allows DDTrac users

to comment on observations made by others. The comments are attached to a particular student-

centric blog post and allow users to share experiences and provide suggestions related to issues

raised about a student. It allows for a dialog between users so that the approaches that work best

with a given student can be identified and adopted by the entire education team. The ability to

comment and share insights is critical to DDTrac's support of the collective intelligence of

special education groups.

Tagging. In paper-based special education data collection environments much of the

qualitative data found in the daily comments are lost within days of capturing it. The volume of

data generated in the special education programs of developmentally disabled children can be

overwhelming. Performing any meaningful analysis of this qualitative data (which can be

accumulated over periods of years) is virtually impossible.

Tagging is one mechanism that can be used in collective intelligence applications to

7
improve the usefulness of both qualitative and quantitative data. For example, the DDTrac

system supports different types of tagging for different types of data. Every time a practitioner

works on a particular objective with a student they can “tag” the entire work activity to describe

the child’s overall mood during the activity. These predefined tags are designed to allow

practitioners to quickly convey the student’s mood to others and allow later assessment of the

impact of mood on the student’s performance.

Frequently special education students with intensive needs also have associated

emotional and behavioral disorders [4]. Behavior data documents a student's patterns of

behavior and is used to determine if efforts to minimize problem behaviors are effective.

Behavior data collection in DDTrac includes a variety of quantitative data related to the behavior

including the date, time, duration and number of behaviors counted during a behavior episode. It

also allows the practitioner to define a set of tags to allow problem behaviors to be analyzed in a

variety of ways. The practitioner defines tags for the types of behaviors being tracked for the

student, the trigger that preceded the behavior, and the location where the behavior occurred.

These tags can then be used to better understand patterns of behavior for a single student or for

groups of students.

For example, behavior tags can be used to generate a stacked bar chart (see Figure 3)

showing how individual behaviors contribute to the number or duration of behaviors observed

for a student. This allows users to see how individual behaviors are changing over time and

determine if replacement behaviors are increasing and less desirable behaviors are decreasing.

8
 Figure 3: Stacked Bar Behavior Chart

The final type of tagging available in DDTrac is the semantic tagging of the narrative

comments taken as a part of the daily instructional, behavior or socialization observation notes

taken by practitioners. These semantic tags closely resemble the tags common on many Web 2.0

sites (e.g. Flickr, Delicious, Blogger etc.). They are freely chosen keywords which allow for

overlapping associations and that can be used for later retrieval and analysis of specific

comments. For example, a student may exhibit a finger flicking behavior infrequently. The

practitioner might note this in the daily notes along with other observations. Then, if the

behavior becomes a problem, the practitioner could retrieve all of the comments tagged

“flicking” to look for any patterns.

Goals & Objectives Wiki. The education programs of developmentally disabled children

are defined in an Individual Education Program (IEP), which establishes long-term goals and

short-term objectives tailored to the needs of the individual student [9]. The IEP also includes

descriptions of the student’s current level of performance, strengths, and individual needs. In

most schools this document includes input from several different people including the special

education teacher, the regular education teacher, the therapy specialists, the student’s parents and

external advocates. The IEP is an important document because it defines the direction for

treatment to be taken for the upcoming year.

9
 DDTrac includes a goals and objectives wiki to meet this need. The wiki utilizes a

template that contains interconnected areas for goal creation, current level of performance

discussions, strengths, and greatest needs. Using a wiki structure for creating IEPs allows

practitioners with appropriate access permissions to add, edit, and delete goals as the new IEP

evolves. It is also possible for current IEPs to reference past IEPs for the same student, as well

as district standards or tests that the student should meet during the year.

The ease with which wiki pages can be created and updated is essential for the success of

the collaborative IEP generation tool. In addition, the ability to review changes before they are

added to the document and to roll back changes that don’t meet the approval of the teacher or

parents is essential in an environment where the IEP represents a legal contract between the

school and the parents. The goals & objectives wiki allows IEP goals and objectives to be edited

until the IEP is accepted then the approved goals and objectives are added to the data collection

portion of DDTrac so data collection can begin.

Support Wiki and Blog. Two other collective intelligence features were designed into the

DDTrac application. A wiki is being used for all software documentation and a blog is used to

communicate with users. Both are essential for feeding the collective intelligence of users back

into the software. The documentation wiki allows developers to quickly add documentation

related to new features. In addition, it allows users to edit the documentation themselves to

clarify instructions that are not clear or to add documentation that the developers did not think to

create. The blog allows the developers to post announcements about new features that have been

added (essential in an environment where updates go live every two to three weeks) and solicits

feedback from users on which new features they feel will be most beneficial.

10
Collective Intelligence in Practice

DDTrac was deployed in an eighteen month field trial with one student with autism. The

student participated in speech therapy, occupational therapy, ABA therapy, and socialization

therapy known as Relationship Development Intervention in a home environment as well as

receiving special education services at school. The practitioners working with the student rarely

met in person and instead used the web-based DDTrac application for data collection and

communication. Over the course of the eighteen month trial data was captured for 481 separate

work sessions and included more than 50,000 individual pieces of data. The wiki and the blog

were active for the final 6 months of the project. During the 6 months the wiki had 163 pages

added and the blog had 33 posts documenting 12 software upgrades.

All participants in the field study felt that DDTrac significantly enhanced their ability to

take data and evaluate the performance of the student they were working with. They reported the

following benefits to using DDTrac:

1. The distributed data collection features made data collection easier and faster.

2. The student centric blog enabled them to quickly understand the student’s recent behavior

trends and better prepare for their own work sessions.

3. The goal & objective wiki enabled them to easily develop new goals and objectives as

well as better understand the goals and objectives the student was currently working on.

4. The ability to analyze the data in a wide variety of ways enhanced their ability to assess

student progress and made it easier to comply with mandatory reporting requirements.

5. The support blog and the wiki enabled them to learn more about DDTrac so that they

could use it more effectively.

The biggest benefit reported by the student’s parents was the ability to analyze long-term

11
education and behavior patterns. The parents commented that they had implemented numerous

interventions with their child and the data provided by DDTrac enabled them to have an

unbiased measure of whether or not a particular intervention had an impact either on the

educational outcomes or on the behaviors of their child.

Conclusion

This paper illustrates the benefits collective intelligence applications can provide in

specialized domains. The DDTrac collective intelligence application allows special education

data to be captured and shared more efficiently than the pencil and paper methods currently

being used (Figure 4). In addition, its reporting and charting features allow this data to be

analyzed comprehensively and quickly. This allows practitioners to spend more time working

with their students. It will also help to provide more efficient treatment and education plans, and

improve the outcomes of millions of children and adults with cognitive impairments.

Figure 4: Replacing paper data collection with distributed data collection and data analysis

Use of a specialized collective intelligence application, like DDTrac, can potentially benefit

organizations in a wide variety of domains (e.g. health care, outsourcing environments). The

ability to apply the collective intelligence of individuals working on similar problems is an area

that has just begun to be addressed by software developers; however, these systems will change

the way information is shared and used and has the potential to dramatically improve decision

making.

12
References

1. Deno, S.L. "Developments in Curriculum-Based Measurement," The Journal of Special

Education 37, 3 (March 2003) 184-192.

2. Gibson, S. "Wikis are alive and Kicking in the Enterprise" eWeek.com, (20 Nov. 2006)
<http://www.eweek.com/article2/0,1895,2061135,00.asp> (Accessed on September 10,
2007).

3. Gregg, D. "Developing a Collective Intelligence Application for Special Education,"

Working paper, (September 2007)
<http://carbon.cudenver.edu/~dgregg/research/greggCI.pdf> (Accessed on September 10,
2007).

4. Gunter, P. L., Callicott, K., Denny, R. K., and Gerber, B. L. Finding a place for Data
Collection in Classrooms for Students with Emotional Behavioral Disorders, Preventing
School Failure, 47, 1 (Fall 2003), 4-8.

5. Lovaas, O. I. Behavioral treatment and normal educational and intellectual functioning in

young autistic children, Journal of Consulting and Clinical Psychology, 55, 1 (February
1987), 3-9.

6. McFedries, P. "The Web, Take Two," IEEE Spectrum, (June 2006), 68.

7. O'Reilly, T. “What Is Web 2.0: Design Patterns and Business Models for the Next
Generation of Software,” O'Reilly Media, Inc., (30 Sept. 2005),
<http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html>
(Accessed on July 9, 2007).

8. Schwartz, I. S. and Olswang, L. B. Evaluating child behavior change in natural settings:

Exploring alternative strategies for data collection, Topics in Early Childhood Special
Education, 16; 1 (Spring 1996), 82-101.

9. Wright, Peter and Wright, Pam “IEP Goals & Objectives: A Tactics and Strategy Session”
(2007) <http://www.wrightslaw.com/advoc/articles/Tactics_Strategy_IEPs.html> (Accessed
on August 31, 2007).

13
Collective Intelligence Sidebar

Collective Intelligence Application Requirements (adapted from 3, 7)

1. Task specific representations: Domain specific collective intelligence applications

should support views of the task that are tailored to the particular domain.
2. Data is the key: Collective intelligence applications are data centric and should be
designed to collect and share data among users.
3. Users Add Value: Users of collective intelligence applications know the most about the
value of the information it contains. The application should provide mechanisms for
them to add to modify or otherwise enhance the data to improve its usefulness.
4. Facilitate Data Aggregation: The ability to aggregate data adds value. Collective
intelligence applications should be designed such that data aggregation occurs naturally
through regular use.
5. Facilitate Data Access: The data in collective intelligence applications can have use
beyond the boundaries of the application. Collective intelligence applications should
offer web services interfaces and other mechanisms to facilitate the re-use of data.
6. Facilitate Access for All Devices: The PC is no longer the only access device for
internet applications. Collective intelligence applications need to be designed to integrate
services across handheld devices, PCs, and internet servers.
7. The Perpetual Beta: Collective intelligence applications are ongoing services provided
to its users thus new features should be added on a regular basis based on the changing
needs of the user community.

14