Building Data Sets Using Social Media
Explore the concept of social media data sets and APIs, and understand how to gather data for research purposes. Learn about the structure of data sets and the caveats of using social media APIs for academic research. Gain insights into working with Web APIs and navigating the access to platforms like Facebook.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Do More with Digital Scholarship: Building Data Sets Using Social Media November 8th, 2017 Matthew Davis and John Fink
What is a social media data What is a data set? set? A data set is a collection of similar information, sharing a structure, that covers a fixed period of time Incoming student transcripts for the years 2010-2016 Medical records for a research trial Records of bequests kept in a parish register or courthouse A social media data set is records of entries on social media sites like Twitter or Facebook over a period of time. The structure comes from the internal structures the particular social media site uses to organize its data, generally exposed to the world through an API
What is an API? Stands for Application Programming Interface The set of rules that software programs follow in order to communicate with each other There s a lot of API s out there, and if you re going to work with computer programs and data you will be using more than one and probably already are without realizing it For our purposes, we re going to work primarily with Web API s
Whats a Web API? A Web API basically a set of commands that you use, via regular http methods to interact with a web client. https://github.com/abhishekbanthia/Public-APIs#music-discovery
Some Caveats Because you re using a structure that the social media company has provided, you are limited to what information they re willing to give you access to. Social media companies make their money off of data. These API s are designed for business purposes, not for academic use, and are structured accordingly. Some API s may not be publically accessible or may be experimental. If you try to grab too much information from these the company can and will shut you down. Not all the data you retrieve will be valid! Social media companies have no incentive to filter fake accounts out from their results. The responsibility to verify the utility of the data you retrieve rests on you as the researcher. Be ready to do a lot of sifting.
Facebook (taken from https://towardsdatascience.com/how-to-use-facebook- graph-api-and-extract-data-using-python-1839e19d6999) To access the Facebook API outside of Facebook s web interface, you have to have a developer s token. This is because Facebook assumes you re building an app to work with their service. Go to developers.facebook.com and create an account there. You may find it will take you directly to the apps page. If this is the case, skip the next bulletpoint. Go to My apps drop down in the top right corner and select add a new app . Choose a display name and a category and then Create App ID . Once you ve created the account, go to developers.facebook.com/tools/explorer. You will see Graph API Explorer below My Apps in the top right corner. From Graph API Explorer drop down, select your app. Then, select Tools and Support. Click on Access Token Tool or navigate to developers.facebook.com/tools/accesstoken. Select Debug corresponding to User Token. Go to Extend Token Access. This will ensure that your token does not expire every two hours. Note that you may need to grant permissions on your application in order to get an access token. Once you have an active token, navigate back to developers.facebook.com/tools/explorer
Twitter Twitter is an entirely different animal than Facebook in terms of what information is available and how it s organized, but the way you gain access to collect it is much the same. Navigate to https://apps.twitter.com/ and select Create New App. Note that Twitter requires you to have your phone registered with them to create developer tokens. Fill out and submit the form. Click on Keys and Access Tokens on the resulting page. These are what you need to access Twitter s information via tool.
Twarc Twitter has its own internal tool, called Twurl (https://github.com/twitter/twurl) and written in Ruby, but the Twarc Python tool (https://github.com/DocNow/twarc) is more powerful and has been around longer. Follow the instructions at the linked page (basically, twarc configure after you ve installed it) and put your consumer and access keys where indicated. The github page provides information on how to perform searches and export information to .json files, .html, and more.
Extracting your data Once you ve generated your data in Facebook, you can cut and paste it into a plain text file. Save this file with the extension .json If you are using twarc, you just need to add >FILENAME.json to your command and it will save the results in a .json file
Thank you! Matthew Davis davism17@mcmaster.ca John Fink jfink@mcmaster.ca
