Can I download data from Zillow?

How to Use Zillow Neighborhood Maps and HTML5 Geolocation

What you are going to create

A few years ago, Zillow released Neighborhood Boundary Files for the United States through the Creative Commons Sharealike 3.0 license. It's a great resource. If you are interested in maps for other countries visit OpenStreetMaps.

However, if you haven't used geographic shapefiles before, how to incorporate Zillow's map data into your own application can be a bit confusing.

I developed a free, open source demo application, MapApp, to demonstrate how Zillow's neighborhood boundaries are used and integrated with Google Maps, geolocation, and geocoding.

MapApp was created with the Yii Framework and can be run on any MySQL PHP capable server. MapApp also uses Google Maps API, Estebanav's HTML5 geolocation helper, eGeocoder and egMap (the last two are Yii extensions).

Set up the server

To get started, you can find the MapApp code on GitHub. Follow the installation steps tested for Ubuntu 14.04 at Digital Ocean, but they should work with any version of LAMP.

You can either clone the repository or download a copy. Configure the Apache site (as described in the installation steps) and restart Apache.

The process takes about 45 to 60 minutes. There's a lot to configure: configure your server, your DNS, extract the code, set up your Apache server and MySQL database, install the configuration file, migrate the active records, install the mapping libraries , load and import the Zillow data.

If you want to save time, I offer a pre-configured image from MapApp for Digital Ocean. However, you will learn more by going through all of the steps yourself.

Prepare the Zillow Neighborhood data

Now that you've created your MySQL database for MapApp, it's time to get the Zillow data.

Install the Geospatial Data Abstraction Libraries and extract:

sudo apt-get install gdal-bin sudo apt-get install unzip

Create a directory to temporarily store the Zillow data and copy the download scripts.

mkdir ~ / zillow cp /var/www/mapapp/docs/wget-zillow.txt ~ / zillow / wget-zillow

Customize the batch file to download the files for the states you want (e.g. California, Oregon, Washington, or all). Then run the download script. This will download all the desired ZIP files from Zillow:

bash wget-zillow

Then prepare the MySQL import scripts:

cp /var/www/mapapp/docs/import-zillow.txt ~ / zillow / import-zillow copies /docs/import-zillow.txt to ~ / zillow / import-zillow

Customize the list of states in the script whose shape files you want to import into MySQL. You will also need to customize the database name, credentials, and neighborhood table name in your local file and run the script. This will use the ogr2ogr tool to import the shape files (.shp) into MySQL:

bash import-zillow

Configure MapApp

You have to adapt the /docs/config-mapapp.ini file with your own settings, in particular with the MySQL access settings:

mkdir / var / www / secure cd / var / www / secure #reminder: The names are reversed from github to the server below. cp /var/www/mapapp/docs/mapapp-config.ini / var / www / secure / config- mapapp.ini sudo nano config-mapapp.ini

Then, migrate the active records database to initialize MapApp. Database migrations are part of the Yii Framework and are used for the programmatic creation of tables and schemas:

cd / var / www / mapapp ./app/protected/yiic migrate

If prompted, provide a username, email address, and password for your administrator account. This is how you log into the MapApp homepage.

Finally, you need to run a script to reverse the geographic coordinates in the MySQL table of the Zillow neighborhoods. Visit http://yourdomain.com/neighborhoods/reverse. Depending on the number of Zillow status files imported, this may take a few minutes. I've found that ogr is importing the Zillow latitude and longitude data in an opposite coordinate order than Google Maps requires.

Use MapApp

Visit your home page at http://mapapp.yourdomain.com. Log in with the username and password that you created during the database migration.

Browse and view neighborhood maps

Browse your imported neighborhoods and click on any view. The closest neighborhood link makes it easy to see more than one. You can also search by name, city, state, or county

I'm using the Yii extension, e.g.Map, to display Google Maps with the Zillow neighborhood polygon data. However, any PHP library for Google Maps or JavaScript will work just as well.

The function in the Neighborhoods model requests the Zillow polygon data from the database as well as the center point of the neighborhood (called the centroid). We use the center of gravity to position the display area of ​​the map.

public function preparMap ($ id) $ pg = Yii :: app () -> db-> createCommand () -> select ('AsText (SHAPE) as region, ASTEXT (Centroid (SHAPE)) as center') -> from (Yii :: app () -> getDb () -> tablePrefix.'neighborhoods') -> where ('OGR_FID =: ogr_fid', array (': ogr_fid' => $ id)) -> queryRow (); Yii :: import ('ext.gmap. *'); $ gMap = new EGMap (); $ gMap-> setJsName ('map_region'); $ gMap-> width = '500'; $ gMap-> height = '500'; $ gMap-> zoom = 13; $ center = new stdClass; list ($ center-> lat, $ center-> lon) = $ this-> string_to_lat_lon ($ pg ['center']); $ gMap-> setCenter ($ center-> lat, $ center-> lon); $ coords = $ this-> string_to_coords ($ pg ['region']); $ polygon = new EGMapPolygon ($ coords); $ gMap-> addPolygon ($ Polygon); return $ gMap;

The controller view "Neighborhoods" displays the page with the map:

public function actionView ($ id) $ gMap = Neighborhoods :: model () -> preparMap ($ id); $ this-> render ('view', array ('model' => $ this-> loadModel ($ id), 'gMap' => $ gMap,));

Use HTML5 geolocation

Click Geolocation in the navigation bar to determine your neighborhood based on your WiFi address. This does not work over cellular networks.

You may need to give your browser geolocation permission for this feature to work (look for a popup under the address bar).

Then you can click on the Find location automatically option:

Sometimes you need to update to get the correct answer after giving permission - or from certain WiFi locations. We use estebanav's geolocation script to support HTML5 geolocation with the greatest possible browser support.

Once your location is found, we'll show your location on a map with your Zillow neighborhood as well as geocoding information searched independently.

We use Yii's eGeocoding extension to get additional data about your location. The main purpose of this is to show additional data sources that you can use beyond the Zillow marginal data.

public function actionIndex () $ model = new Geolocation (); if (isset ($ _ POST ['Geolocation'])) $ info = Yii :: app () -> Geocoder-> reverse ($ _ POST ['Geolocation'] ['lat'], $ _ POST ['Geolocation ']) [' lon ']); $ gps_for_sql = "Point (". $ _ POST ['Geolocation'] ['lat']. "". $ _ POST ['Geolocation'] ['lon']. ")"; $ neighborhood = Neighborhoods :: model () -> lookupFromLatLon ($ gps_for_sql); $ gMap = Neighborhoods :: model () -> preparMap ($ neighborhood ['OGR_FID']); $ marker = new EGMapMarkerWithLabel ($ _ POST ['Geolocation'] ['lat'], $ _ POST ['Geolocation'] ['lon'], array ('title' => 'You are here!')); $ gMap-> addMarker ($ marker); $ gMap-> width = '400'; $ gMap-> height = '400'; $ this-> render ('view', array ('data' => $ neighborhood, 'info' => $ info, 'gMap' => $ gMap)); else $ this-> render ('index', array ('model' => $ model));

Move on

If you'd like to see more, including drawing your own region maps, check out my Geogram side project. You can create email-based communities near neighborhoods, user-created regions, places, and in the Google Places directory.

Geogram has a number of advanced mapping and email functions. If you're interested in a tutorial on Geogram's use of the Mailgun email API, see How Geogram Created a Free Group Email Service Using Yii for PHP Using MySQL. I may write a tutorial on how to draw regions for Google Maps in the future. Drop a comment below if you'd like to read this. You may also be interested in some of my other Yii-based tutorials. I'll be writing an introduction to the Yii Framework for Tuts + shortly.

Please feel free to post any corrections, questions, or comments. You can also reach me on Twitter @reifman or contact me directly via email.