I'm trying to create a geo viz using this data which maps out all of the zip codes in NYC: https://data.cityofnewyork.us/Health/Modified-Zip-Code-Tabulation-Areas-MODZCTA-Map/5fzm-kpwv. I've used geopandas to read the data as a geodataframe, used shapely.wkt - loads on the multipoloygon, & set the appropriate column as the geometry.
I then uploaded this tree census data (https://data.cityofnewyork.us/Environment/2015-Street-Tree-Census-Tree-Data/uvpi-gqnh) into a df and merged the two on the zipcode column (I renamed cols myself as datasets used different terms).
My goal is to create an interactive geo viz using Bokeh, which I understand uses GEOJson. I've used the following code to achieve: JNB screenshot of code
However, it is taking an exceedingly long time to run the code:
nyc_trees = json.loads(nyc_trees.to_json())
I've tried ujson as well, but it hasn't helped with the timing. Is there a way to speed this up or another work around?
Related
I am extremely new to python in general, but come from a decent programming background in Fortran and Matlab. I'm trying to figure out what seems to be a simple task:
I am importing a Shapefile using GeoPandas read_file command. It comes in with no problem, and includes the geometry column that contains the polygon represented by all the vertices as you would expect. The data is a single row.
What I need is a Shapely's Polygon to pass into py3dep so I can gather the spatial rasters I need, and I cannot figure out how to translate the geometry entry into a polygon.
The solution has to be simple, but I've been searching for hours, trying a thousand different things, all to no avail. If anyone could help with a simple demonstration I'd be eternally grateful.
Here's a snippet of what I'm trying to do, I've verified the code works when I pass it the tuple, and fails if I try to pass "shape" to "geom" as shape is not a shapelys polygon. (I understand this is a lame attempt and is just shown as "what I'm trying to do")
UPDATED CODE BELOW TO WORKING based on solution provided by #PaulH
import py3dep
import geopandas as gpd
shape = gpd.read_file("inputshape.shp")
print(shape.type)
geom = shape.loc[0, 'geometry']
dem = py3dep.get_map("DEM", geom, resolution=10, geo_crs="epsg:4326", crs="epsg:3857", output_dir="raster")
I am trying to export a large feature collection from GEE. I realize that the Python API allows for this more easily than the Java does, but given a time constraint on my research, I'd like to see if I can extract the feature collection in pieces and then append the separate CSV files once exported.
I tried to use a filtering function to perform the task, one that I've seen used before with image collections. Here is a mini example of what I am trying to do
Given a feature collection of 10 spatial points called "points" I tried to create a new feature collection that includes only the first five points:
var points_chunk1 = points.filter(ee.Filter.rangeContains('system:index', 0, 5));
When I execute this function, I receive the following error: "An internal server error has occurred"
I am not sure why this code is not executing as expected. If you know more than I do about this issue, please advise on alternative approaches to splitting my sample, or on where the error in my code lurks.
Many thanks!
system:index is actually ID given by GEE for the feature and it's not supposed to be used like index in an array. I think JS should be enough to export a large featurecollection but there is a way to do what you want to do without relying on system:index as that might not be consistent.
First, it would be a good idea to know the number of features you are dealing with. This is because generally when you use size().getInfo() for large feature collections, the UI can freeze and sometimes the tab becomes unresponsive. Here I have defined chunks and collectionSize. It should be defined in client side as we want to do Export within the loop which is not possible in server size loops. Within the loop, you can simply creating a subset of feature starting from different points by converting the features to list and changing the subset back to feature collection.
var chunk = 1000;
var collectionSize = 10000
for (var i = 0; i<collectionSize;i=i+chunk){
var subset = ee.FeatureCollection(fc.toList(chunk, i));
Export.table.toAsset(subset, "description", "/asset/id")
}
I want to create dynamically updated chart of network usage / time. Number of series can change due because I don't know how many network adapters will be in the system at the moment.
I wrote a script which returns data in json format:
[{"name":"eth0_upload","data":[[1417981348000,5585.0]]},
{"name":"eth0_download","data":[[1417981348000,4258.0]]},
{"name":"lo_upload","data":[[1417981348000,960.0]]},
{"name":"lo_download","data":[[1417981348000,960.0]]}]
how can I add this points to my chart? Can I update series like series.addPoints(jsondata, ...) or is there any way to call serie by id/name (like series[eth0_upload].addPoints(jsondata[0], ...)?
You can get serie by id like here
I am writing a demo web application that tracks multiple devices through my companies platform. I have the app working, but need a csv file that will simulate devices moving on a map as if they were a tracker attached to a car. The simulator works by reading 1 row of data every second (1 lat/lng point). Here is an example of the first few lines of a file that would work if the points weren't scattered across the US (the SclId is the device name).
SclId Latitude Longitude
HAT-0 44.968046 -94.420307
HAT-1 44.33328 -89.132008
HAT-2 33.755787 -116.359998
HAT-3 33.844843 -116.54911
HAT-4 44.92057 -93.44786
HAT-5 44.240309 -91.493619
HAT-0 44.968041 -94.419696
HAT-1 44.333304 -89.132027
HAT-2 33.755783 -116.360066
HAT-3 33.844847 -116.549069
HAT-4 44.920474 -93.447851
HAT-5 44.240304 -91.493768
If I had something that could create simulation data with mouse clicks it would save me a lot of time creating another program requiring me to drive around with a device and record the data to a CSV. Any help/recommendations would be greatly appreciated. If you don't understand the question please ask for clarification!
There is a website that I use to draw waypoint and download it as any format e.g., GPX, KML, JSON, CSV, etc.
https://www.alltrails.com/explore/map/map-e727fa5--12
I am trying to extract the latitudes and longitudes for the places listed on the right side of this page. I want to create a table like the following:
Place Latitude Longitude
Agarda 23.12604 87.19869
Ahanda 23.13099 87.18501
.....
.....
West-Sanabandh 23.24876 86.99941
Is it possible to do this in R without calling up the individual hyperlinks for "Agarda:, "Ahanda"... etc. one at a time?
The data appears on different pages. You can't get that data without requesting each page.
If R supports threads then you can call them up in parallel rather than one at a time.
It's possible to use RCurl to scrape each page in some type of loop or sapply. If you combine it with some regex and/or readHTMLTable (to identify the hyperlinks) then it's a relatively straightforward function.
Within RCurl, it's possible to create a multicurl which will do this in parallel, although given the number of queries involved, it might be just as easy to serialise it and put a small system sleep between queries.