combining data (.csv/json) with topojson - json

I'm trying to combine topojson (produced from a shape file ) and data so I could display the data for relevant selection on the map, but no luck yet.
Shape file features/properties:
id, code, name
Data files (I've got both .csv and josh)
file 1 columns:
year1, year2, year3,....identifier, %change.
('identifier' column in data files is equals to 'code' in the shape file)
I have data in 5 json files.
I was hoping, by combining these two files, to get a topojson file with properties,
id, code, name, year1, year2, year3,...%change.
Idea is, I could use just one topojson file for displaying map as well as relevant
data..
This is what I have tried so far,
Generating topojson:
1. ogr2ogr -f GeoJSON geojsonoutput.json shpefile.shp
2. topojson -o final.json -e *.json --id-property=identifier -p -- geojsonoutput.json
final.json :
{
"type":"Topology",
"objects":{"geojsonoutput":{"type":"GeometryCollection","geometries": [{"type":"Polygon","properties":{"id":"1","name":"some name"},"arcs":
, "file1" : [{id, code, name, year1, year2, year3,...%change}],
"file2" : [{id, code, name, year1, year2, year3,...%change}],
}
I could access map information by using the following,
topojson.feature(data, data.objects.geojsonoutput).features
however, not sure how I could access the data..for example in ("file1" or "file2") keys.
Actually..Am I going in the right direction? is what I have done so far correct? is there any better way achieve what I'm trying to do?
Any guidance would be great. I'm still kind of new to D3 but enjoying working with it so far.
Cheers

Thanks to this example http://bl.ocks.org/mbostock/5562380! managed to get what I'm after..here is solution..
topojson -e data.csv --id-property id_in_shapefile,id_in_datafile -p -o final.json -- shapefile.shp
it added the properties correctly..
Cheers

Related

Is it possible to extract from a map JSON file a list of a city's neighborhoods in a tree-structure format?

Forgive my ignorance, I am not experienced with JSON files. I've been trying to get a tree structure list of all the neighborhoods and locations in the city of Cape Town and this seems to be my last resort.
Unfortunately, I can't even open the file that can be found on this website - http://odp.capetown.gov.za/datasets/official-suburbs?geometry=18.107%2C-34.187%2C19.034%2C-33.988
Could someone tell me if it's possible to extract such as list.
I'd be forever thankful if someone could help me. Thank you in advance
[I am making my comments an answer since I see no other suggestions and no information provided]
I am on a unix/linux shell but the following tools can also be found for windows. My solution for getting a quick list would be:
curl https://opendata.arcgis.com/datasets/8ebcd15badfe40a4ab759682aacf8439_75.geojson |\
jq '.features | .[] | .properties.OFC_SBRB_NAME'
Which gives you:
"HYDE PARK"
"SPRINGFIELD"
"NIEUW MAASTRECHT-2"
"CHARLESVILLE"
"WILDWOOD"
"MALIBU VILLAGE"
"TUSCANY GLEN"
"VICTORIA MXENGE"
"KHAYELITSHA"
"CASTLE ROCK"
"MANSFIELD INDUSTRIA"
...
Explanation:
curl https://... - curl downloads the JSON file from the API you are using
jq: can process JSON on terminal and extract information. I do this in three steps:
.features: GeoJSON format seems to have a standard schema. All the retuned entries are in features array
.[] returns all elements in the array docs here
.properties.OFC_SBRB_NAME: Each element of the array has a field called "properties" which from my understanding carries/includes metadata of this entry. One of those properties in OFC_SBRB_NAME which looks like a name and is the only string in each element. Thus I extract this.
Hope it helps. If you add more detail as to which platform you are using or language, etc I can update the answer, however, the methodology should remain the same I think

Does anyone have a script to convert a Chrome Bookmarks file with [sub]*folders into a CVS file?

I want to be able to do Vimdiffs and Vimfolds on Bookmarks files that have been converted to CVS files ie with one description and one uri per line. However, because the Bookmarks file has multiple levels for the folders, the CSV file will also need fields for the different levels of folder names on each line.
I am new to jq but it seems like it should be able to do this sort of conversion?
Thanks,
Phil.
Have you tried to use any free tools like: https://json-csv.com/
or json2csv: https://www.npmjs.com/package/json2csv
If neither of those works, perhaps this approach.
When I need to reconstruct data I write a set of loops that identify each property I want for each line in my CSV. Let's say my JSON has Name, Email, Phone but for some reason all are at different object levels in my JSON.
First right a loop that resolves Name, then a loop for Email, and one for Phone. At the end of the first loop call the second, and from the second call the third.
Then you can use jq -n which allows to create JSON with no input.
So your CSV output would be like jq -n '{NewName: .["'$Name'"]}'
once you have a clean JSON with all data points at the same level CSV conversion is smooth.
Hope this helps

Renaming CSV column header and merge results with Powershell

So I'm just starting out with this whole Powershell thing and so far so good - until now. I just can't figure out how to do this!
I'm looking at manipulating CSV files which are output from one system (which I can't change at output), renaming some column headers and merging a couple of the results into one column so that it matches the input requirements to upload into another system (again, I can't change those parameters).
So, as an example.
The first file is created:
File1.csv
"A","B","C""1","2","3"
I want a powershell script that will output:
File2.csv
"X","Y""1","23"
So I can import it into another system.
I hope that all makes sense, and thanks in advance for any assistance.
I'm going to assume that your actual/desired formats of your files look like this:
"A","B","C"
"1","2","3"
"X","Y"
"1","23"
rather than having everything in one line. If that's correct you can import File1.csv with Import-Csv, rename and merge columns with calculated properties:
... | Select-Object #{n='X';e={$_.A}}, #{n='Y';e={$_.B + $_.C}} | ...
and write the result to File2.csv with Export-Csv.

Can't simplify topojson for d3 mapping

I'm trying to map some statistical data of Italy and I need the infrastructure (railway and motorway) on top of it.
The problem is that I'm not able to simplify the infrastructure json file.
I'm using the openstreetmap shape of Italy by geofabbriK: http://download.geofabrik.de/europe/italy.html#
I've converted the roads.shp to json selecting only motorway and and primary roads using this command:
ogr2ogr -f GeoJSON -where "type IN ('motorway', 'motorway_link', 'primary', 'primary_link')" -t_srs EPSG:4326 roads.json roads.shp
I get a 55Mb json file. You can download it here: http://www.danielepennati.com/prove/mapping/roads_mw_pr.zip
Than I tryed to simplify and convert it in topojson.
Whit no -s command the new json file is about 13Mb
If I use -s or --simplify-proportion with any value form 1 to 0 I always get a max semplification of 95% and a filesize of 11Mb
How can I get a more simplified topojson?
Thanks
daniele

Bounding csv data to topojson path - cloropleth

I am trying to bound some data to a topojson map.
So far I've been able to draw the map of the US, with each path labeled by id: state name.
What I'd like to do now is to bind data I have on an external csv to the map, to show each state of a hue based on the value in the csv (aka cloropleth).
I've been looking for guides but I don't have the knowledge yet to understand the one on Mike Bostock's site, so I tried to bind the color with a function like so:
.attr("fill", "rgb(0, 0, " + Math.round(d.value * 255 / d3.max(d, function (d) {return d.value; })) + ")")
but it doesn't work.
The page is here: http://www.dropbox.com/s/w2pe4omn895vt83/usa_malattie.html
Commenting out the last part, with the csv load, gives the map with the tooltip showing, that also disappear when loading the csv data.
Any help on how to resolve this would be highly appreciated, thanks in advance!
EDIT: the csv looks like this:
id,value
"Kentucky",207.4
"Mississippi",200.5
"West Virginia",196.6
"Louisiana",196.4
the topojson like this:
{"type":"Topology","transform":{"scale":
0.011125945304891298,0.005248969801868182],
"translate":[-178.21759836236586,18.92178634508703]},
"objects":{"usa":{"type":"GeometryCollection",
"geometries":[{"type":"MultiPolygon","id":"Hawaii",
"arcs":[[[0]],[[1]],[[2]],[[3]],[[4]],[[5]],[[6]]],
"properties":{"STATE_NAME":"Hawaii"}},
link for the csv: http://dl.dropboxusercontent.com/u/37967455/usa_malattie/death_parse.csv
link for the topojson: http://dl.dropboxusercontent.com/u/37967455/usa_malattie/usatopo.json
You may try the following command:
topojson -o final.json -e death_parse.csv --id-property=id -p STATE_NAME,color=+value -- usatopo.json