Importing a data snapshot: Difference between revisions

m
imported>Onceuponatimeforever
 
(13 intermediate revisions by 8 users not shown)
Line 1:
{{Hacking OpenHatch}}
When you get your own instance of the OpenHatch code running, you'll discover you're missing the ''data'' that are on the main OpenHatch site. SoTo assist developers that are working on features that ''data'' would be helpful, we take periodic snapshots of the data on the main openhatch.org site.
 
== Where you can find the snapshots ==
 
You may download a snapshot of the OpenHatch data from:
* http://inside.openhatch.org/snapshots/
 
'''Note''': We go through some effort to remove private information before we publish user data in these snapshots. The code for that is [https://github.com/openhatch/oh-mainline/blob/master/mysite/customs/management/commands/snapshot_public_data.py here].
 
== Privacy implications ==
Line 12 ⟶ 15:
== How to use a snapshot ==
 
'''Note''': You must run
To load a snapshot into your database, you can run this command from your local install.
 
<tt>python manage.py syncdb --noinput</tt>
'''Note''': You must run "syncdb" and "migrate" before this will work. Read README.mkd to learn more about those commands. It should take less than one minute.
 
and
Uncompress it, and load it:
 
<tt>python manage.py migrate</tt>
gunzip snapshot.json.gz
 
./bin/mysite loaddata ''snapshot.json''
'''Note''':before Youthe must run "syncdb" and "migrate" beforefollowing thissteps will work. Read <tt>README.mkd</tt> to learn more about those commands. It should take less than one minute.
 
 
1. Copy the downloaded snapshot file into the 'oh-mainline' directory. The snapshot file is named as ''date.json.gz'' where 'date' is in the form YYYY-MM-DD.
 
2. To load the snapshot file into your database, input the following (no need to uncompress the snapshot file):
 
python manage.py loaddata ''2012-08-12.json.gz''
 
'''Note:''' This may take a long time (10-15 minutes) without any output. This is normal.
 
You'll see output that looks something like this:
 
Installed 2528894858 object(s) from 1 fixture(s)
Installing json fixture 'snapshot' from absolute path.
 
Installed 25288 object(s) from 1 fixture(s)
3. Then, run the following command to update the database file ''mysite.db'' with the new data snapshot:
 
python manage.py syncdb
 
You can test that it worked by loading up your local projects page and ensuring it is not empty. Access http://127.0.0.1:8000/projects/ (and compare it to http://openhatch.org/projects/ if you like!) to check.
 
== In case of memory problems ==
In case your operating system has problems with loading the giant set of all the production data, you can get the db file directly from http://inside.openhatch.org/snapshots/. The important critical step is to remember to change the name of the existing development db file to a different name (in case you need it later) and then you change the newly downloaded db file name to ''site.db''.
 
python manage.py syncdb --noinput --migrate
You can test that it worked by loading up your local people page and the live one. Do you have the about the same number of people? Click on these things:
python manage.py migrate
* http://openhatch.org/people/
python manage.py loaddata NAME_OF_SNAPSHOT.gz
* http://127.0.0.1:8000/people/
 
== More about this ==
 
* We go through some effort to remove private information before we publish user data in these snapshots. The code for that is in [httphttps://gitoriousgithub.orgcom/openhatch/oh-mainline/blobsblob/master/mysite/customs/management/commands/snapshot_public_data.py snapshot_public_data.pyhere].
 
* We wantare tonow be savingcreating these snapshots periodically, hopefully once a day. We don't do that yet.
 
* We don't snapshot every single table. If you find there's something that we don't publish that we should, do [http://openhatch.org/bugs/ file a bug]!