Community Data Science Workshops (Spring 2014)/Saturday May 3rd lecture: Difference between revisions

Content added Content deleted

Inline

Revision as of 13:21, 3 May 2014

Lecture Outline

API
- definition of API: just an interface for programs
- definition of web API
  - way to ask for data (almost always a URL)
  - way to get data back (almost always in a format called JSON)
- to use APIs to build a dataset we will need:
  - all our tools from last session: variables, etc
  - the ability to open urls on the web
  - the ability to create custom URLS
  - the ability to save to files
  - the ability to understand (i.e., parse) JSON data that APIs usually give us
Review material from last session
- variables, different types
- printing
- if statements
- lists
- dictionaries
- for loops
- modules
- example python program
New programming concepts:
- urllib2 and urlopen
- interpolate variables into a string using % and %()s
placekitten.com
- API that takes specially crafted URLs and gives appropriately sized picture of kittens
- example of placekitten in browser
  - kittens of different sizes
  - kittens in greyscale or color
- show how to use place
- write a small program to grab arbitrary square from placekitten by asking for the size on standard in
JSON file (JavaScript Object Notation)
- what is json
- import json; json.loads()
- like Python (except no single quotes)
- simple lists, dictionaries
- can reflect more complicated data structures
- Example file at http://mako.cc/cdsw.json
- open a file and write to it
Wikipedia API
- explain MediaWiki, exists on other wikis
- navigate to api page and show the documentation, point out examples
- looking at the images within a page http://en.wikipedia.org/w/api.php?action=query&titles=Seattle&prop=images&imlimit=20&format=jsonfm
- looking at within two pages http://en.wikipedia.org/w/api.php?action=query&titles=Seattle%7CBellevue,_Washington&prop=images&imlimit=50&format=jsonfm
- edit count http://en.wikipedia.org/w/api.php?action=query&list=users&ususers=Benjamin_Mako_Hill%7CJtmorgan%7CSj%7CMindspillage&usprop=editcount&format=jsonfm
- give me the content of the main page http://en.wikipedia.org/w/api.php?format=json&action=query&titles=Main%20Page&prop=revisions&rvprop=content
Other APIs
- every API is different, so read the documentation!
- rate limiting
- authenticaiton
- text encoding issues

@@ Line 2: / Line 2: @@
 # API
-#* definition of an API in general
+#* definition of API: just an interface for programs
-#* definition of web API for data
+#* definition of web API
+#** way to ask for data (almost always a URL)
+#** way to get data back (almost always in a format called JSON)
+#* to use APIs to build a dataset we will need:
+#** all our tools from last session: variables, etc
+#** the ability to open urls on the web
+#** the ability to create custom URLS
+#** the ability to save to files
+#** the ability to understand (i.e., parse) JSON data that APIs usually give us
 # Review material from last session
 #* variables, different types
@@ Line 16: / Line 24: @@
 #* urllib2 and urlopen
 #* interpolate variables into a string using % and %()s
-#* JSON file (JavaScript Object Notation)
-#** what is json
-#** import json; json.loads()
-#** like Python (except no single quotes)
-#** simple lists, dictionaries
-#** can reflect more complicated data structures
-#* Example file at http://mako.cc/cdsw.json
-#* open a file and write to it
 # [http://placekitten.com/ placekitten.com]
 #* API that takes specially crafted URLs and gives appropriately sized picture of kittens
@@ Line 31: / Line 31: @@
 #* show how to use place
 #* write a small program to grab arbitrary square from placekitten by asking for the size on standard in
+# JSON file (JavaScript Object Notation)
+#* what is json
+#* import json; json.loads()
+#* like Python (except no single quotes)
+#* simple lists, dictionaries
+#* can reflect more complicated data structures
+#* Example file at http://mako.cc/cdsw.json
+#* open a file and write to it
 # Wikipedia API
 #* explain MediaWiki, exists on other wikis