Kayak.com for Social Network Data: The Influence Mapping Community Tackles Its Interoperability and Usability Problems

Knight Proposal

We’ve submitted a project to the Knight News Challenge that seeks to make projects around social network mapping tools better connected, more trustworthy, more communicative to users, and easier to use for everybody.

Read more about it here and give some !

In one sentence, describe your idea as simply as possible: The community of journalists and researchers who use social network mapping tools has a plan to make these projects better connected, more trustworthy, more communicative to users, and easier to use for everybody (and it starts with building a Kayak.com-style aggregator for social network data so that journalists and researchers can more easily track the activities of people and companies of interest).

“Influence mapping” refers to the use of social network mapping and analysis tools to understand patterns of power and influence. The methodology has a long history in journalism and research, and has seen rapid recent uptake in those communities as the underlying tools have become better, cheaper, and deployable online. Some of these projects are have grown into large, public resources for mapping important dimensions of political and economic power, such as OpenCorporates, Littlesis, and Poderopedia. But there are many others, small and large, one-off and long term, on subjects as diverse as international trade agreements, oil contracting, and parliamentary mapping. We are discovering more every day.

This growth of interest has produced parallel and sometimes duplicative innovation, but no mature, easy, widely-adopted tools or standards. Projects don’t connect. There are no metadata or API standards, little sharing of visualization or analytical tools, no consensus around how to document the provenance of data, and weak integration with narrative tools. There is, in short, enormous potential, but also a great deal of inefficiency.

The Influence Mapping network was founded a year ago to create a framework for addressing these problems collectively. The network includes most of the major projects in this area and provides a coordination layer for information sharing, best practices, and collaboration on technical development. Much of this work takes place via our email list and workshops. A forthcoming toolkit (by iilab) will clarify the options, steps, and decisions points associated with successful use of these tools.

  • “Who’s Got Dirt?” will be an online tool to search for information across different mapping projects – a Kayak.com for social network data. The initial implementation will provide a single interface for accessing data on people and organizations across three of the major projects in this area: OpenCorporates, Littlesis, and Poderopedia (and later others). For users, the tool will provide a vastly simplified one-stop-shop for information, while continuing to direct users to the participating sites. For the mapping community, it represents a soft approach to promoting data interoperability and integration, which is in turn a precondition of reducing the fragmentation of the field, enabling richer forms of research and storytelling, and allowing for more rapid creation of new datasets. Subsequent versions will be made available as drop-in interfaces for individual services, in an effort to standardize and simplify the user experience with these sites.
  • “Oligrapher 2” is a follow-on to the visual network exploration tool developed by Littlesis, designed to be easily deployable by any projects using the ‘Dirt’ standard. Current work is focused on enabling much better integration of network graphs with text, with the goal of getting definitively beyond non-communicative spaghetti-blob visualizations and making network graphs support storytelling. Follow-on work will introduce an expanded range of drop-in visualizations, including timeline features, and potentially a graphical authoring interface. Littlesis and Poderopedia are the primary developers of this new toolset.
  • “Unter” will develop the equivalent of citation standards and tools for social network mapping efforts, as a means improving the trustworthiness of data collected in these projects. The tools and practices will likely be based on the principles of ‘two clicks to a source document from any data point’ and ‘local storage of the source,’ drawing on early implementations by OpenCorporates and OpenOil. This capture and annotation practice will be abstracted into a set of tools for linking data to source documents, author notes, and/or captured webpages.

Together, these projects will go a long way toward making these projects better connected, more trustworthy, more communicative to users, and easier to use for everybody.

Briefly describe the need that you’re trying to address. Journalists and researchers are both creators and users of datasets that map corporate, political and social relations. But much of the software tool development surrounding these projects has remained idiosyncratic, raising costs and diminishing their utility for users. This project will tackle four of of the big challenges to better data-driven journalism and research in this area: interconnectivity, ease of use, trust, and storytelling. What progress have you made so far? The “Influence Mapping” network emerged a year ago to provide a coordination layer for information, best practices, and collaboration on development, via an email list and workshops. It currently counts 172 members and most of the major projects in this area. The community is an accomplishment and prerequisite for the proposed work. We have also done preliminary development on Who’s Got Dirt and Oligrapher 2, and will shortly release a tool kit to simplify adoption and use of these systems. What would be a successful outcome for your project? To lower the cost of making effective use of social network mapping tools by journalists, researchers, and the public by making the tools:

  • Easier to deploy.
  • More intuitive to use.
  • Easier to trust.
  • More supportive of narrative journalism and research.
  • Capable of leveraging the fragmented labor of data collection within the community.

This will be visible in the successful development and uptake of the tools within the Influence Mapping community.