Archive: 2012

Trying to extract demographic data from twitter

participants

June Po and Renan Escalante joined us for the afternoon of the second day. They were trying to get demographic data from twitter feeds, specifically the r-shief #f29 dataset, initially by looking for keywords that could be linked to race/ethnicity. For example, they began by looking  for “Black,” but found that this approach mostly returned false positives (eg Blackberry). Their next step was to try and look for co-occurence of ‘race’ and ‘Black,’ but that didn’t work either. The third approach they tried was to look for hashtags with terms related to race, but that didn’t produce many results either. In the future they’re thinking about other approaches: for example looking at twitter usernames, looking at RT networks, and so on.

#OccupyData Mural

This is a visual representation of people’s answer to the OccupyResearch survey question:

If you participate in the Occupy movement, what TOP THREE concerns motivate you TO PARTICIPATE?

The larger the word, the more people from the survey responded with it. This is a processing app that will ideally be both animated and interactive. Next steps are to make a smarter text layout so that more of the responses are legible.

A bunch of visual explorations from today can be found here.

This is a collaboration between Nadia Afghani and Gilad Lotan.

Screen-shot-2012-03-24-at-5.18.13-PM

An older version:

6865834654_5b5c1165a8_b

Photos from #occupydata hackathon 2, day 2

Photos from #occupydata hackathon 2, day 1

Occupy General Survey Participation in the U.S., by State (by quantile)

ogsusparticipationbystate

Based upon self-reported ZIP codes of residents, states with the highest survey participation included:
California, New York, Massachusetts, Pennsylvania, Florida, Texas and Washington.

Map by Don Blair and Chris Schweidler, using GeoCommons.

Link to this data: http://geocommons.com/maps/152629

 

Faceted Browsing of ORGS Data

This team cleaned up the ORGS data set for use in the visualization, collapsing into a nested data structure and converting to JSON and removing any data not used for visualization.

Current version: http://orgs-facet.tirl.org

About
This is two different interfaces for faceted browsing of the results of the Occupy Research General Demographics and Participation Survey done at the Cambridge location of the Occupy Data 2 Hackathon. Thanks to Charlie DeTar for cleaning up the dataset.

The “simple” version:orgs-facet

 

The Exhibit version:

 Code for both is on GitHub.

Putting ORGS Data to Work – facilitating interoperability

Another team at the Boston Hackathon is using answers to the open ended, Question 42, “What are you trying to achieve with your participation in the Occupy Movement,” to mine the answers for commonly used language in order to source categories to organize information on Tech Ops.  The goal is to take make the categorization more intuitive, using common language from the 5k+ respondents to the survey.

OWS Tech Ops has been working to create the news aggregator at Newswire.Occupy.Net and the wiki at Occupy.net (http://wiki.occupy.net).

Occupiers are adding content to the wiki and a set of folks have been organizing it, adding categorization and structure.  Using the language of the survey respondents, this team will harmonize existing categories (created by the wiki team) with the language sourced from the much larger set of 5k+ respondents to ORGS.

wiki

Wiki http://wiki.occupy.net

The news aggregator currently pulls RSS feeds from occupation sites:

    http://newswire.occupy.net/

http://newswire.occupy.net/

The team will try to use some of the same categorization for this site to make it more navigable.

Visions of Occupy

This project seeks to pair two pillars of the Occupy movement: the beliefs we have which inspire us to occupy and the occupations themselves.
Using data collected this winter by the Occupy Research General Survey (administered by OccupyResearch), we take the answer to question 42—”In just a few words, what are you trying to achieve with your participation in the Occupy movement”—and pair it with a Flickr photo tagged with the camp name that the same respondent mentions. This means that while the photo displayed and quote may be completely unrelated (both in source and in specific content), viewers are presented with locational context and imagery. 

Exploring the civic anatomy of OWS

According to the  Occupy Research General Survey (ORGS), OWS sympathizers and participants are among the most civically engaged individuals of the U.S. population, they possess an active voting record, and tend to be involved in a wide range of organizations and civic actions.  The ORGS allows us to explore some of the characteristics of the diverse “civic cultures” of online sympathizers who have brought broad support to OWS in the U.S.

Investigating the civic anatomy of OWS is of critical importance for those who seek to understand the movement’s potential as a force to promote social change through different forms of social and political action, from community-based work and peaceful protest to participatory deliberation and electoral politics. Comparing the different patterns of organizational affiliation (Graph 1) between self-defined participants and non-participants in the movement helps us to think about different civic trajectories, and the diverse pool of mobilizing practices from which Occupiers draw their strategies and political views.

Graph 1. Dimensions of Organizational Affiliation, 

Organizational-Affiliation_Non-Participants3

Organizational-Affiliation-_Participants2

OWS Participants vs. Non-participants

The experience in non-profit organizations is clearly central for participants and non-participants. However, the space of the political in Occupy seems significantly shaped by the trajectory of sympathizers in social justice organizations, political parties, labor unions, voluntary associations, affinity and cultural groups  (see Table 1). In some cases, affiliation to these institutions is higher among Occupiers than in the overall American population. As aptly argued by professor Wendy Brown, OWS should be understood fundamentally as a civic movement galvanized by the economic crisis, and the reaction to neoliberal policies that have threatened the foundations of democratic governance and public life in America.

Table 1. Organizational affiliation: “I belong and actively participate in…

Organizations

Participants

in OWS

   Non-participants

   in OWS

   Sig.

n

%

n

%

Non-profits

(3193)

44.7%

(397)

43.3%

Social justice groups

(3137)

32.9%

 (387)

14.5%

   ***
Another voluntary associations

 (2994)

30.8%

(390)

24.4%

   *
Political Party

(3231)

24.1%

(398)

18.6%

   ***
NGO

(3056)

21.7%

(388)

16.0%

Professional associations

 (3095)

19.9%

 (388)

21.6%

Cultural groups

 (3035)

19.8%

 (385)

14.8%

   *
Affinity Groups

(3018)

16.6%

 (383)

7.3%

   ***
Churches

(3121)

15.2%

(388)

20.6%

   ***
Labor unions

(3149)

8.5%

(391)

2.6%

   ***
Sport groups or teams

 (3006)

8.1%

 (387)

11.4%

   **
Worker center

(2960)

2.2%

(384)

1.2%

   **

*** p <0.001, ** p < 0.01, * p < 0.05

Source: ORGS (U.S. residents, n=3,715)

In the electoral front, the voting record of Occupy participants in the ORGS is particularly high when compared to the average voter turnout of the country (64% in the 2008 presidential election). According to ORGS data, 87.5% of Occupiers voted in the 2008 presidential elections, and 80.6% plan to vote next November. These results echo findings of ongoing surveys of visitors to the OWS website.

The obvious conclusion of this evidence is that the popular outrage expressed by the movement is not the result of extremist factions of society but of civic discontent across broad sections of the American public. This is not to say that Occupy is merely an assemblage of political groups and identities. The diversity of  civic cultures and trajectories of occupiers should move activists and researchers to critically interrogate the historical roots of the movement, and its emergent identities and structures in connection with existent forms of civic governance in the American society. Answering all these questions may seem an elusive task in the case of a self-defined ‘leaderless’ movement; however, understanding the political and the ‘civics’ in Occupy is critical to assess the future of the movement, and its potential role as a transformative force of American democracy.

Merging lists of datasets for #occupydata hackathon

Picture-1

The #occupydata visualization hackathon 2 is well into the second day. The group in DC created a spreadsheet for possible projects, with links to various datasets; there is also the initial list of datasets from http://bit.ly/occupyhackathon. I just finished working with someone who’s going by occupydata388 in the IRC to get this initial list into the DC spreadsheet, then put the metadata into the column format they’re using.