Hi I am new on Python and pandas. So please excuse my mistakes.
I want to know if it's possible to use a dataframe and sort it with pandas.
I'm still in class, but I'd like to advance in my projects.
I want / I'd like to extract part of the JSON file.
I want all the system_name =Métro de Paris to be in a JSON file.
Following are the current code I have (JSON and script).
Thanks in advance for your help!
import pandas as pd
# parse according to "system_name":"Métro de Paris"
data = pd.read_json(r'paris_lines_systems_and_modes.json')
[{"id":406,"name":"Ancien M 2nord","url_name":"406-ancien-m-2nord","color":"#000","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":405,"name":"Ancien M 2sud","url_name":"405-ancien-m-2sud","color":"#000","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":399,"name":"Ancien M 14","url_name":"399-ancien-m-14","color":"#000","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":460,"name":"Ancienne ligne A Nord-Sud","url_name":"460-ancienne-ligne-a-nord-sud","color":"#000","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":459,"name":"Ancienne ligne B Nord-Sud","url_name":"459-ancienne-ligne-b-nord-sud","color":"#000","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":411,"name":"Austerlitz-Auteuil","url_name":"411-austerlitz-auteuil","color":"#000","system_id":303,"system_name":"Navettes fluviales","transport_mode_id":0,"transport_mode_name":"default"},{"id":471,"name":"CDG Express","url_name":"471-cdgexpress","color":"#f0871f","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":7,"transport_mode_name":"people_mover"},{"id":465,"name":"CDGVal 1","url_name":"465-cdgval-1","color":"#0a3071","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":7,"transport_mode_name":"people_mover"},{"id":466,"name":"CDGVal 2","url_name":"466-cdgval-2","color":"#0a3071","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":7,"transport_mode_name":"people_mover"},{"id":410,"name":"Charenton-Auteuil","url_name":"410-charenton-auteuil","color":"#000","system_id":303,"system_name":"Navettes fluviales","transport_mode_id":0,"transport_mode_name":"default"},{"id":407,"name":"Funiculaire de Monmartre","url_name":"407-funiculaire","color":"#601e92","system_id":302,"system_name":"Funiculaire","transport_mode_id":0,"transport_mode_name":"default"},{"id":381,"name":"H","url_name":"381-h","color":"#7b4339","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":382,"name":"J","url_name":"382-j","color":"#cdcd00","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":383,"name":"K","url_name":"383-k","color":"#c7b300","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":384,"name":"L","url_name":"384-l","color":"#7584bc","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":472,"name":"Le Bus Direct 1","url_name":"472-le-bus-direct-1","color":"#b20066","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":8,"transport_mode_name":"bus"},{"id":473,"name":"Le Bus Direct 2","url_name":"473-le-bus-direct-2","color":"#7bbb58","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":8,"transport_mode_name":"bus"},{"id":474,"name":"Le Bus Direct 3","url_name":"474-le-bus-direct-3","color":"#0098d8","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":8,"transport_mode_name":"bus"},{"id":475,"name":"Le Bus Direct 4","url_name":"475-le-bus-direct-4","color":"#ff9b00","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":8,"transport_mode_name":"bus"},{"id":412,"name":" Le Louvre-Ablon","url_name":"412-le-louvre-ablon","color":"#000","system_id":303,"system_name":"Navettes fluviales","transport_mode_id":0,"transport_mode_name":"default"},{"id":643,"name":"Ligne Horizon","url_name":"643-ligne-horizon","color":"#0092e2","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":352,"name":"M 1","url_name":"352-ligne-1","color":"#ffcd00","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":353,"name":"M 2","url_name":"353-ligne-2","color":"#006db8","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":354,"name":"M 3","url_name":"354-ligne-3","color":"#9b993b","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":355,"name":"M 3bis","url_name":"355-ligne-3bis","color":"#88d3df","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":356,"name":"M 4","url_name":"356-ligne-4","color":"#ba4a9a","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":357,"name":"M 5","url_name":"357-ligne-5","color":"#f78e4b","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":358,"name":"M 6","url_name":"358-ligne-6","color":"#77c596","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":303,"name":"M 7","url_name":"303-ligne-7","color":"#f49eb2","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":359,"name":"M 7bis","url_name":"359-ligne-7bis","color":"#77c596","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":360,"name":"M 8","url_name":"360-ligne-8","color":"#c5a3cc","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":361,"name":"M 9","url_name":"361-ligne-9","color":"#cdc92b","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":362,"name":"M 10","url_name":"362-ligne-10","color":"#dfb03a","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":363,"name":"M 11","url_name":"363-ligne-11","color":"#8c6539","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":364,"name":"M 12","url_name":"364-ligne-12","color":"#008c59","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":365,"name":"M 13","url_name":"365-ligne-13","color":"#87d2df","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":366,"name":"M 14","url_name":"366-ligne-14","color":"#662c91","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":395,"name":"M 15","url_name":"395-m-15","color":"#a60f31","system_id":300,"system_name":"Grand Paris Express","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":396,"name":"M 16","url_name":"396-m-16","color":"#ec7cae","system_id":300,"system_name":"Grand Paris Express","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":397,"name":"M 17","url_name":"397-m-17","color":"#ec7cae","system_id":300,"system_name":"Grand Paris Express","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":398,"name":"M 18","url_name":"398-m-18","color":"#95bf32","system_id":300,"system_name":"Grand Paris Express","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":385,"name":"N","url_name":"385-n","color":"#00a092","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":476,"name":"Navette","url_name":"476-navette","color":"#000","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":394,"name":"Orlyval","url_name":"394-orlyval","color":"#7d7e7e","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":7,"transport_mode_name":"people_mover"},{"id":386,"name":"P","url_name":"386-p","color":"#f0b600","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":387,"name":"R","url_name":"387-r","color":"#e4b4d1","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":376,"name":"RER A","url_name":"376-rer-a","color":"#ec1b2a","system_id":297,"system_name":"RER","transport_mode_id":3,"transport_mode_name":"commuter_rail"},{"id":377,"name":"RER B","url_name":"377-rer-b","color":"#4c90cc","system_id":297,"system_name":"RER","transport_mode_id":3,"transport_mode_name":"commuter_rail"},{"id":378,"name":"RER C","url_name":"378-rer-c","color":"#ffdc1d","system_id":297,"system_name":"RER","transport_mode_id":3,"transport_mode_name":"commuter_rail"},{"id":379,"name":"RER D","url_name":"379-rer-d","color":"#00ab66","system_id":297,"system_name":"RER","transport_mode_id":3,"transport_mode_name":"commuter_rail"},{"id":380,"name":"RER E","url_name":"380-rer-e","color":"#c67db4","system_id":297,"system_name":"RER","transport_mode_id":3,"transport_mode_name":"commuter_rail"},{"id":414,"name":"STCRP","url_name":"414-stcrp","color":"#000","system_id":303,"system_name":"Navettes fluviales","transport_mode_id":0,"transport_mode_name":"default"},{"id":367,"name":"T 1","url_name":"367-ligne-1","color":"#14609e","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":368,"name":"T 2","url_name":"368-ligne-2","color":"#ba498d","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":369,"name":"T 3a","url_name":"369-ligne-3a","color":"#ef7d2d","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":370,"name":"T 3b","url_name":"370-ligne-3b","color":"#009b41","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":371,"name":"T 4","url_name":"371-ligne-4","color":"#d9922b","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":372,"name":"T 5","url_name":"372-ligne-5","color":"#914e8e","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":373,"name":"T 6","url_name":"373-ligne-7","color":"#e95c36","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":375,"name":"T 7","url_name":"375-t-7","color":"#885f30","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":374,"name":"T 8","url_name":"374-t-8","color":"#8c8235","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":389,"name":"T 9","url_name":"389-t-9","color":"#009cb4","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":390,"name":"T 10","url_name":"390-t-10","color":"#8c8235","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":391,"name":"T 11","url_name":"391-t-11","color":"#df5f3d","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":506,"name":"T 11 express","url_name":"506-t-11-express","color":"#e2562e","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":392,"name":"T 12","url_name":"392-t-12","color":"#a02a3a","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":393,"name":"T 13","url_name":"393-t-13","color":"#885f30","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":400,"name":"T Zen 1","url_name":"400-t-zen-1","color":"#c6cd01","system_id":301,"system_name":"TZen","transport_mode_id":6,"transport_mode_name":"brt"},{"id":401,"name":"T Zen 2","url_name":"401-t-zen-2","color":"#c6cd01","system_id":301,"system_name":"TZen","transport_mode_id":6,"transport_mode_name":"brt"},{"id":402,"name":"T Zen 3","url_name":"402-t-zen-3","color":"#c6cd01","system_id":301,"system_name":"TZen","transport_mode_id":6,"transport_mode_name":"brt"},{"id":403,"name":"T Zen 4","url_name":"403-t-zen-4","color":"#c6cd01","system_id":301,"system_name":"TZen","transport_mode_id":6,"transport_mode_name":"brt"},{"id":404,"name":"T Zen 5","url_name":"404-t-zen-5","color":"#c6cd01","system_id":301,"system_name":"TZen","transport_mode_id":6,"transport_mode_name":"brt"},{"id":409,"name":"Tuileries-Suresnes","url_name":"409-tuileries-suresnes","color":"#000","system_id":303,"system_name":"Navettes fluviales","transport_mode_id":0,"transport_mode_name":"default"},{"id":388,"name":"U","url_name":"388-u","color":"#d60058","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":408,"name":"Voguéo","url_name":"408-vogueo","color":"#99d420","system_id":303,"system_name":"Navettes fluviales","transport_mode_id":0,"transport_mode_name":"default"}]
So I managed to find how do extract part of the data you want to use / analyse.
You should simply call the column you are trying to extract (in my case system_name) and give its value (in my case "Métro de Paris"). Here The dataframe, data, will now have the data with value "Métro de Paris" form the column "system_name".
data = data[data.system_name == "Métro de Paris"]
I'm using the code described in the GAS documentation to retrieve my Twitter timeline.
The lines
var response = twitterService.fetch('https://api.twitter.com/1.1/statuses/home_timeline.json?count=1');
response=response.getContentText();
yield the following result:
[{"created_at":"Sun Jan 14 15:30:00 +0000 2018","id":952563458616172544,"id_str":"952563458616172544","text":"#UltimOra, \u201cAi piedi le Clarks\u201d; #LiberieUguali trova il primo punto di convergenza fra i vari leader https:\/\/omissis","truncated":false,"entities":{"hashtags":[{"text":"UltimOra","indices":[0,9]},{"text":"LiberieUguali","indices":[33,47]}],"symbols":[],"user_mentions":[],"urls":[{"url":"https:\/\/omissis","expanded_url":"http:\/\/www.lercio.it\/ai-piedi-le-clarks-liberi-e-uguali-trova-il-primo-punto-di-convergenza-fra-i-vari-leader\/","display_url":"lercio.it\/ai-piedi-le-cl\u2026","indices":[102,125]}]},"source":"\u003ca href=\"https:\/\/about.twitter.com\/products\/tweetdeck\" rel=\"nofollow\"\u003eTweetDeck\u003c\/a\u003e","in_reply_to_status_id":null,"in_reply_to_status_id_str":null,"in_reply_to_user_id":null,"in_reply_to_user_id_str":null,"in_reply_to_screen_name":null,"user":{"id":910827588,"id_str":"910827588","name":"lercio-notizie","screen_name":"lercionotizie","location":"","description":"Dal 13 aprile 2017 in fumetteria e libreria con \"Lercio, lo sporco che fa notizia. Il libro\" (Shockdom)","url":"http:\/\/omissis","entities":{"url":{"urls":[{"url":"http:\/\/omissis","expanded_url":"http:\/\/www.lercio.it","display_url":"lercio.it","indices":[0,22]}]},"description":{"urls":[]}},"protected":false,"followers_count":491614,"friends_count":25,"listed_count":712,"created_at":"Sun Oct 28 18:19:33 +0000 2012","favourites_count":385,"utc_offset":3600,"time_zone":"Amsterdam","geo_enabled":false,"verified":false,"statuses_count":7335,"lang":"it","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"FFFFFF","profile_background_image_url":"http:\/\/pbs.twimg.com\/profile_background_images\/696203701\/001bc779080491195bc16054af629016.png","profile_background_image_url_https":"https:\/\/pbs.twimg.com\/profile_background_images\/696203701\/001bc779080491195bc16054af629016.png","profile_background_tile":false,"profile_image_url":"http:\/\/pbs.twimg.com\/profile_images\/2932719853\/4bcd23b39293870bf1832aec72b6fe3d_normal.png","profile_image_url_https":"https:\/\/pbs.twimg.com\/profile_images\/2932719853\/4bcd23b39293870bf1832aec72b6fe3d_normal.png","profile_banner_url":"https:\/\/pbs.twimg.com\/profile_banners\/910827588\/1467828171","profile_link_color":"9B1212","profile_sidebar_border_color":"000000","profile_sidebar_fill_color":"DDEEF6","profile_text_color":"333333","profile_use_background_image":true,"has_extended_profile":false,"default_profile":false,"default_profile_image":false,"following":true,"follow_request_sent":false,"notifications":false,"translator_type":"none"},"geo":null,"coordinates":null,"place":null,"contributors":null,"is_quote_status":false,"retweet_count":73,"favorite_count":186,"favorited":false,"retweeted":false,"possibly_sensitive":false,"possibly_sensitive_appealable":false,"lang":"it"}]
But when I add the code line
response=JSON.parse(response);
the JSON file is totally altered, it doesn't even look a JSON file and its content becomes:
{possibly_sensitive_appealable=false, in_reply_to_status_id_str=null, in_reply_to_status_id=null, created_at=Sun Jan 14 15:30:00 +0000 2018, in_reply_to_user_id_str=null, source=TweetDeck, retweet_count=73, retweeted=false, geo=null, in_reply_to_screen_name=null, is_quote_status=false, id_str=952563458616172544, in_reply_to_user_id=null, favorite_count=186, id=9.5256345861617254E17, text=#UltimOra, “Ai piedi le Clarks”; #LiberieUguali trova il primo punto di convergenza fra i vari leader https://omissis, place=null, lang=it, favorited=false, possibly_sensitive=false, coordinates=null, truncated=false, entities={urls=[Ljava.lang.Object;#7d41de2d, hashtags=[Ljava.lang.Object;#5e05507d, user_mentions=[Ljava.lang.Object;#7b5572f2, symbols=[Ljava.lang.Object;#fb373c9}, contributors=null, user={utc_offset=3600, friends_count=25, profile_image_url_https=https://pbs.twimg.com/profile_images/2932719853/4bcd23b39293870bf1832aec72b6fe3d_normal.png, listed_count=712, profile_background_image_url=http://pbs.twimg.com/profile_background_images/696203701/001bc779080491195bc16054af629016.png, default_profile_image=false, favourites_count=385, description=Dal 13 aprile 2017 in fumetteria e libreria con "Lercio, lo sporco che fa notizia. Il libro" (Shockdom), created_at=Sun Oct 28 18:19:33 +0000 2012, is_translator=false, profile_background_image_url_https=https://pbs.twimg.com/profile_background_images/696203701/001bc779080491195bc16054af629016.png, protected=false, screen_name=lercionotizie, id_str=910827588, profile_link_color=9B1212, is_translation_enabled=false, translator_type=none, id=910827588, geo_enabled=false, profile_background_color=FFFFFF, lang=it, has_extended_profile=false, profile_sidebar_border_color=000000, profile_text_color=333333, verified=false, profile_image_url=http://pbs.twimg.com/profile_images/2932719853/4bcd23b39293870bf1832aec72b6fe3d_normal.png, time_zone=Amsterdam, url=http://omissis, contributors_enabled=false, profile_background_tile=false, profile_banner_url=https://pbs.twimg.com/profile_banners/910827588/1467828171, entities={description={urls=[Ljava.lang.Object;#7c7074a4}, url={urls=[Ljava.lang.Object;#80245ef}}, statuses_count=7335, follow_request_sent=false, followers_count=491614, profile_use_background_image=true, default_profile=false, following=true, name=lercio-notizie, location=, profile_sidebar_fill_color=DDEEF6, notifications=false}}
What's wrong?
You are probably using Logger.log for checking results of code execution.
As noted in documentation:
The method Logger.log expects a string value, but if you pass in an object or array it will do its best to convert that into a string.
So response in this case response=response.getContentText() will give you String with expected behaviour when used in Logger.log, but in case response=JSON.parse(response) will give Object, that will have output in Logger.log in format [key1=val1, key2=val2, ..].
You can get used to it, or you can use convertion of your object to string before logging it:
Logger.log(JSON.stringify(response))