Community Data Science Workshops (Spring 2014)/Saturday May 3rd lecture

Lecture Outline

1. API
   • definition of an API in general
   • definition of web API for data
2. Review material from last session
   • variables, different types
   • printing
   • if statements
   • lists
   • dictionaries
   • for loops
   • modules
   • example python program
3. New programming concepts:
   • urllib2 and urlopen
   • interpolate variables into a string using % and %()s
   • JSON file (JavaScript Object Notation)
     • what is json
     • import json; json.loads()
     • like Python (except no single quotes)
     • simple lists, dictionaries
     • can reflect more complicated data structures
   • Example file at http://mako.cc/cdsw.json
   • open a file and write to it
4. placekitten.com
   • API that takes specially crafted URLs and gives appropriately sized picture of kittens
   • example of placekitten in browser
     • kittens of different sizes
     • kittens in greyscale or color
   • show how to use place
   • write a small program to grab arbitrary square from placekitten by asking for the size on standard in
5. Wikipedia API
   • explain MediaWiki, exists on other wikis
   • navigate to api page and show the documentation, point out examples
   • looking at the images within a page http://en.wikipedia.org/w/api.php?action=query&titles=Seattle&prop=images&imlimit=20&format=jsonfm
   • looking at within two pages http://en.wikipedia.org/w/api.php?action=query&titles=Seattle%7CBellevue,_Washington&prop=images&imlimit=50&format=jsonfm
   • edit count http://en.wikipedia.org/w/api.php?action=query&list=users&ususers=Benjamin_Mako_Hill%7CJtmorgan%7CSj%7CMindspillage&usprop=editcount&format=jsonfm
   • give me the content of the main page http://en.wikipedia.org/w/api.php?format=json&action=query&titles=Main%20Page&prop=revisions&rvprop=content
6. Other APIs
   • every API is different, so read the documentation!
   • rate limiting
   • authenticaiton
   • text encoding issues